Skip to main content

SECTION B: Studies with Errors in Design, Analysis or Reporting

  • Pin It

  • Pin It

I: Meta-Analyses and Systematic Reviews

Each paper is linked to either its original abstract on PubMed or the full-text article if available; click on the title to be taken to the abstract/ article.

A) Wax, J. R., Lucas, F. L., Lamont, M., Pinette, M. G., Cartin, A., & Blackstone, J. (2010). Maternal and newborn outcomes in planned home birth vs planned hospital births: a metaanalysis. Am J Obstet Gynecol, 203(3):243.e1-8, doi: 10.1016/j.ajog.2010.05.028.

This article presents a meta-analysis of the safety of planned home versus planned hospital birth. The authors conclude that planned home births are associated with similar maternal outcomes, but with a threefold increase in neonatal mortality. The methodology and statistical analysis employed in this systematic review were flawed. This meta-analysis contains calculation and numerical errors, selective and mistaken inclusion/ exclusion of studies when analyzing specific outcomes, as well as logical flaws in terms of definitions. Many of the odds ratios (ORs) and confidence intervals (CIs) were calculated incorrectly. In some cases, this was the result of errors apparently made in the extraction of data from the original studies. In addition, the software tool used to calculate the statistics had embedded errors that can dramatically underestimate confidence intervals (CIs), and resulted in at least 1 false statistically significant result. Wax et. al defined perinatal death as loss of a newborn of at least 20 weeks or 500 g, or death of a liveborn infant within 28 days of birth. Neonatal deaths were defined as deaths of liveborn infants within 28 days of delivery. This means that neonatal deaths should be reported as a subset of perinatal deaths. However, the paper reports that for planned home births, the neonatal death rates are far higher than the corresponding perinatal death rates. In addition, perinatal death statistics are derived from more than 500,000 births, whereas the neonatal death statistics are drawn from fewer than 50,000 births. Hence the conclusions on comparative neonatal death rates offered by the authors cannot be defended. Most notably, the de Jonge study, which contributed more than 95% of the births used in the analysis, did not define perinatal death according to the same definitions. It is unclear why Wax and colleagues excluded this study from the calculations for neonatal mortality but included the study for perinatal mortality. According to Michal et al. “If that study were removed from the calculations for the 2 outcomes for which it was erroneously included, the total number of births included in the meta-analysis would have been reduced from nearly 550,000 to just 65,000. This dramatic reduction in the size of the dataset would have significantly reduced the impact of any findings of the meta-analysis. On the other hand, if Wax and colleagues had defined perinatal death and neonatal death according to definitions used by de Jonge and associates, the conclusions for these outcomes would have been quite different.”

A more detailed critique of this article, authored by a team of experts in the field (including the principal investigators of studies included in the meta-analysis), is cited in Section C.I.A.

II: Cohort & Population-Based Observational Studies—North America

A) Chang, J. J. & Macones, G. A. (2011). Birth outcomes of planned home births in Missouri: A population-based study. Am J Perinatol, 28(7):529-536.

A retrospective cohort study to compare outcomes between planned home births attended by non-CNMs, physicians, and CNMs to outcomes of births in hospitals and birth centers attended by physicians and CNMs. Data was collected from linked Missouri live birth and fetal death files, for the years 1989 through 2005. The study sample included singleton pregnancies, delivered between 36-44 weeks gestation. Pregnancies with major fetal anomalies and breech presentation were excluded. Authors found that planned home birth by non-CNMs, physicians and CNMs was protective against selective obstetric procedures and complications such as fever, moderate to heavy meconium, and dysfunctional labour, but that planned home births attended by non-CNMs were associated with prolonged labour, and fivefold increased odds of newborn seizure. Planned home births attended by all three groups (physicians, CNMs and non-CNMs) held a higher risk of intrapartum death. There are several weaknesses in the design and interpretation of data in this study. The subset of non-CNM attended home births was too small for meaningful analysis of rare perinatal outcomes, and the authors used an unconventional definition of ‘low-risk’, which includes all births from gestational ages of 36-44 weeks. Further, there are multiple issues of data validity using birth record data related to identification of planned home births and type of attendant. Authors suggest the non-CNM group may include certified professional midwives but there were none in practice in Missouri at the beginning of the study period; and the CPM credential was not accepted for licensure in Missouri until 2008. Even today there are not enough Missouri based CPMs to attend the number of births indicated as attended by ‘other midwives’. Prior to legislation families who delivered outside the hospital filled out their own birth certificate record. Several of those births may be misclassified unplanned accidental home births, or attended by someone without credentials. Most importantly, given the sample size and wide confidence intervals, misclassification of even a few records could skew results.

B) Evers, A. C. C, Browers, H. A. A., Hukkelhoven, C. W. P. M., Nikkels, P. G. J., Boon, J., van Egmond-Linden, A., Hillegersberg, J., Snuif, Y. S., Sterken-Hooisma, S., Bruinse, H. W., & Kwee, A. (2010). Perinatal mortality and severe morbidity in low and high risk term pregnancies in the Netherlands: prospective cohort study. BMJ 341:c5639,  doi:10.1136/bmj.c5639.  (full text)

This was not a study of home birth safety but rather focused on primary and secondary care referrals. This cohort study compared the incidences of perinatal mortality and severe perinatal morbidity between low-risk term pregnancies in primary care with a midwife and high-risk secondary care with an obstetrician. The study found that infants of low risk women who started labour under primary care of a midwife had a significantly higher risk of perinatal death than infants of high risk women whose labour started in secondary care under the care of an obstetrician. While NICU admission rates did not differ between groups, infants who were referred to a physician by a midwife during labour had a 3.66 times higher risk of related perinatal death. Infants of nulliparous women had a significantly higher risk of NICU admission than infants of multiparous women. The most common reason for admission was asphyxia. Because data were extracted from a large birth registry database, adjustment for confounders, including appropriate referrals from primary to secondary care before and during the onset of labour, was not possible. These findings do not correspond with any previous studies of the Dutch maternity care system. The results may mostly be a reflection of the interprofessional relationships that are specific to the Utrecht region.

C) Malloy, M. H. (2010). Infant outcomes of certified nurse midwife attended home births: United States 2000 to 2004. J Perinatol, 30(9):622-27, doi: 10.1038/jp.2010.12.

A retrospective cohort study using linked US birth and death certificate files from the National Center for Health Statistics from 2000-2004, to compare the safety of CNM deliveries at home to CNM deliveries in hospital (data also examined delivery outcomes of ‘other’ midwives’ in hospital and home). Malloy concludes that neonatal mortality rates of certified nurse midwives or ‘other’ midwives are higher in out of hospital settings (home/ birthing center) compared to deliveries at the hospital. Method of selection did not distinguish planned from unplanned home birth nor if hospital birth CNMs were actually in attendance at home births or solely appeared on birth certificates as the certifier of the birth having occurred. Analysis does not distinguish between “other midwife” attendant and no attendant.

D) Wax, J. R., Pinette, M. G., Cartin, A., & Blackstone, J. (2010). Maternal and newborn morbidity by birth facility among selected United States 2006 low-risk births. Am J Obstet Gynecol 202(2):152.e1-5.  (full text)

A retrospective population-based cohort study to evaluate perinatal mortality by place of birth (hospital, birth center, home) using 2006 U.S. birth certificate data from 19 states available through the CDC. Of 745,690 total births included, 733,143 occurred in hospital, 4661 in freestanding birth centers, and 7427 at home. Excluded from the study were: preterm (<37 weeks), smokers, women with Type I, II or gestational diabetes, either chronic or pregnancy induced hypertension and a prior caesarean section. The authors concluded that home births are associated with less frequent adverse perinatal outcomes (chorioamnionitis, fetal intolerance of labour, meconium staining, assisted ventilation, NICU admissions and birthweights of <2500g), but more frequent abnormal labours and 5-minute Apgar scores of <7 and birth weight >2500g. The study does not differentiate between planned and unplanned home births, and does not provide data about home to hospital transfers.

E) Pang, J., W. Heffelfinger, J. D., Huang, G. J., Benedetti, T. J., & Weiss, N. S. (2002). Outcomes of planned home births in Washington State: 1989-1996. Obstet Gynecol, 100(2):253-59.

Method of selection did not distinguish between planned home births, out-of-hospital births that had no attendant, or births with unknown or unnamed attendants. Premature births occurring before 37 weeks were incorrectly included in the initial analysis. A higher incidence of congenital heart disease in the home birth population could partially explain the higher neonatal mortality and would reflect a difference in populations.

III: Cohort and Population-Based Studies

A) van der Kooy, J., Poeran, J., de Graff, J. P., Birnie, E., Denktass, S., Steegers, E. A. P., & Bonsel, G. J. (2011). Planned home compared with planned hospital births in the Netherlands: intrapartum and early neonatal death in low-risk pregnancies. Obstet Gynecol 118(5):1037-46.  

(The following is a review of the author’s secondary analysis; see review of main study inSection A, IV, B.)

In addition to reporting the usual statistics (RRs and adjusted ORs) to compare perinatal outcomes across birth settings, the authors performed additional analyses, e.g. they divided the crude mortality rates of the home and hospital groups by the prevalence of the ‘Big 4’ (congenital anomalies, IUGR, preterm birth, Apgar < 7; these 4 conditions accounted for 85% of the neonatal mortalities in the sample) to ‘obtain case mix adjustment’. The rationale for this adjustment was to remove clinical determinants of neonatal mortality, and focus on ‘setting’ dependent mortality. Using this approach, the authors reported up to 20% excess mortality in the home setting, leading the authors to conclude that women with certain risk factors (e.g. pregnancy duration more than 41 weeks and having an infant that is small for gestational age) can reduce their risk of intrapartum and early neonatal death by planning a hospital birth. It should be noted that the index does not allow for assessment of statistical significance (and thus more emphasis should be placed on the adjusted ORs reported in tables 2 and 3). As the authors themselves note in post-publication correspondence, “In both RCT and observational designs, post-hoc exclusion of patients or replacement of treatment allocation by the treatment actually received is not allowed under the intention-to-treat principle”; hence, at minimum the analysis and reporting of outcomes should have been limited to their “perfect guideline approach”.