Building evidence-based medicine skills in gynecology

Systematic reviews: What, why, and how?
Systematic reviews aim to overcome the deficiencies of single studies in a comprehensive and unbiased manner. They critically evaluate, summarize, and, when possible, combine all available studies addressing a given topic. By comparing a range of relevant interventions across populations and settings, systematic reviews may be more generalizable than single studies. Meta-analysis, or quantitatively combining study results, increases sample size and usually provides more precise estimates of effect sizes than the single studies. Critical appraisal of the combined studies can highlight methodologic and other concerns about the body of evidence to assess the overall confidence in the included studies.

A systematic review, like a well-conducted RCT, has a protocol that lays out the scope of the review and defines a priori criteria and analytic plans—all with the goal of minimizing bias. It starts with a well-formulated research question, explicitly defining the PICO elements—population, interventions, comparators, outcomes—in addition to the setting and study designs of interest.⁸ Based on these eligibility criteria, several sources of evidence (such as electronic databases and reference lists) are searched to find all potentially eligible studies.

Typically, several thousand citations are found that must be matched against the eligibility criteria. Potentially eligible studies are then rescreened in full text to further scrutinize their eligibility. The goal is to be highly sensitive to avoid missing relevant studies—even at the time cost of screening many articles. The individual study designs (including the study eligibility criteria, interventions, outcomes, and analytic methods) and the results for all outcomes of interest are extracted from each study.

For most systematic reviews, researchers also will assess the quality, or risk of bias, of each study for each outcome.^9,10 Study data are summarized across all included studies, with study results meta-analyzed and reasons for heterogeneity across studies explored. Several consensus statements detail the proper methodology to conduct and report a systematic review.^11,12 Ultimately, the review’s conclusions are based on analyses of all available evidence. By contrast, narrative reviews typically  start with a conclusion and then select evidence to support that conclusion, and are therefore more likely to be biased.¹³

As noted, systematic reviews often include meta-analysis, which may allow an exploration of some reasons for study heterogeneity. The meta-analysis is usually presented graphically in a forest plot, which displays point estimates for each study with their associated 95% confidence intervals and a description of each study.¹⁴ In a forest plot, one can see the estimate and precision of each study, assess the heterogeneity of results across studies, and compare individual studies to each other and to the overall summary estimate.

Systematic reviews should be read as critically as primary studies. Some important questions you should consider are:

• Did the review address the populations, interventions, comparators, outcomes, and settings relevant to your practice?
• Have studies been included in a nonbiased manner, and is the described body of evidence likely to be complete?
• Did the study authors evaluate and summarize the underlying risks of bias of the studies?
• Did the researchers avoid combining studies that are too different from each other to allow a coherent interpretation of the summary results?
• Did the researchers attempt to explain how and why studies differed from one another?

Of note, systematic reviews and meta-analyses are subject to the same biases as all retrospective studies. Also, the systematic reviewers’ own biases—due to factors such as funding source, researchers’ agendas, or specialties—may subtly affect systematic reviews just as biases may affect an individual study. Furthermore, the confidence you have in a systematic review’s conclusions may be limited by the quality and generalizability of the underlying studies.

Assessing harms
You make the ultimate management decisions for your patient (though, of course, with her participation). The likely benefit of a specific treatment—determined in an experimental trial and refined further in a systematic review and meta-analysis—must be balanced with the risk of harms. RCTs usually do not provide the highest quality evidence of harms due to their limited sample sizes and short follow-up duration. Rather, large observational studies, case series, and case reports commonly provide these important details. Increasingly, patient registries are being created to prospectively follow patients and gather uniform safety data. By providing a true denominator, more accurate estimates of adverse event incidence are possible. However, the disadvantages of all of these modalities are 1) there usually are no comparators (that is, “How does the adverse event incidence for surgery A compare to that for surgery B?”) and 2) data usually are gleaned from medical records and not directly from patients.