Obstetric residency programs can be ranked according to the complication rates among patients delivered by the programs' graduates, according to a report in JAMA.

Moreover, the difference in maternal complication rates between graduates of the highest-ranked residency programs and graduates of the lowest-ranked programs was deemed relatively large. “A woman choosing an obstetrician who trained at a program in the top tier would face a 10.3% risk of a major complication, compared with 13.6% if she chose an obstetrician trained at a program from the bottom tier,” said Dr. David A. Asch of the Leonard Davis Institute of Health Economics at the University of Pennsylvania, Philadelphia, and his associates.

“These findings provide the first empirical support for widely held intuitions about the clinical implications of variation in medical education,” they added.

The investigators evaluated risk-adjusted rates of maternal complications in 4,906,169 births in New York and Florida between 1992 and 2007 “as measures to judge the quality of care” delivered by 4,124 graduates of 107 U.S. residency programs who attended these births. These residency programs were distributed among 22 states and represented 43% of the 249 currently accredited obstetric residency programs in the United States.

Six categories of complications were assessed individually and in three composites, for a total of nine measures. For vaginal births, lacerations, hemorrhage, and all other maternal complications such as infections and thrombotic events were assessed; for cesarean deliveries, hemorrhage, infection, and all other complications such as operative and thrombotic events were assessed.

The rate of each outcome for each residency program was estimated after the data were adjusted to account for numerous patient, hospital, and physician characteristics.

The complication rates of physicians trained in residency programs in the top quintile were substantially lower (absolute difference, 3.3%) than those of physicians trained in residency programs in the bottom quintile. “In general, the bottom-quintile programs had complication rates approximately one-third higher than those of the top-quintile programs,” Dr. Asch and his colleagues wrote (JAMA 2009;302:1277–83).

The rankings remained consistent when the data were broken down by the nine individual complication measures, suggesting that “these rates may reflect good measures of overall quality” in residency programs, they added.

A separate analysis was performed to examine “whether the estimated program rankings result from differences in a residency program's ability to attract talented residents,” as opposed to its ability to improve residents' skills.

Medical licensure test scores were available for a subset of 74% of the obstetricians in this study. Analysis showed that the caliber of medical students feeding into the residency programs had little effect on outcomes, suggesting that “skills developed during residency training are more important for producing good maternal outcomes than skills developed during medical school, and residency programs differ in skill development,” the investigators said.

“Recently, there has been significant interest within the graduate medical education community to assess the quality of residency training programs based on the academic (board pass rates, research productivity, etc.) and patient care outcomes of program graduates,” said Dr. Diane M. Hartmann, chair of the Council on Resident Education in Obstetrics and Gynecology.

“The academic indicators are relatively easy to compile. Clinical outcomes of program graduates are very difficult to obtain and virtually unavailable to training programs at this time. This study does raise an interesting question about the importance of facilitating a training program's ability to obtain this information and thus assess its own educational quality,” she said in an interview.

“Clinical outcomes of this type could potentially be used to highlight and replicate the components of excellent training environments. Ob.gyn. residency program directors across the country are committed to producing high-quality physicians. I believe that accurate information about the clinical performance of graduates would be a welcome addition to the information they use daily to alter and improve the obstetrical and gynecological training experience,” said Dr. Hartmann, also senior associate dean for graduate medical education and professor of obstetrics and gynecology at the University of Rochester (N.Y.).

This study was limited in that it examined deliveries in only two states, which likely do not represent all residency programs. It also assessed only maternal complications and did not include birth outcomes.

Additionally, this sample included obstetricians who completed residency at many different times. “A hospital's residency program in the 1960s might differ from its program in the 1990s because of different faculty, the evolution of new clinical techniques …, or trends in attracting different trainees,” Dr. Asch and his associates noted.

“Separate from these methodological limitations, stakeholders might object to the interpretation and use of the results,” they wrote.

This study was supported by a grant from the Stemmler Fund of the National Board of Medical Examiners. No financial conflicts of interest were reported.

Obstetric Residencies Ranked by Grads' Skills