The Impact of Checklists on Inpatient Safety Outcomes: A Systematic Review of Randomized Controlled Trials

Analysis

Owing to the heterogeneity of the data and the small number of studies that satisfied the inclusion criteria, the data were analyzed using guidelines for the narrative synthesis of a systematic review.²⁸ Descriptive statistical findings from each included study were reported. The DerSimonian and Laird method for random-effects models was used to calculate a pooled estimate of 30-day all-cause mortality from the raw data available from a subset of studies (number of events, study population).²⁹ Stata SE version 13.1 (Stata Corp, LP, College Station, TX) was used to perform the statistical analyses.

The literature search identified 11,225 unique citations from which 83 abstracts were eligible for full-text review. We identified 9 full-text articles for inclusion in the review (Figure 1 [Supplementary File 3]). The main reasons for citation exclusion during the full-text review were that the study design was not an RCT (39%) or there was no checklist intervention (34%). Inter-rater agreement for full-text inclusion was fair (K=0.660, 95% confidence interval[CI],0.414-0.828).

Study Characteristics

Characteristics of the included studies are summarized in Tables 1 and 2. Six of the studies were conducted in at least 1teaching hospital.^30-35 The studies varied in target populations for both the checklist user and patients. The outcomes reported varied; 3 studies examined 30-day mortality,^21,30,36 4 studies examined hospital length of stay,^21,30,33,36 and 2 studies reported user compliance with the checklist.^21,31 Five of the studies reported patient outcomes,^{21,30,33,35,36} and 5 studies reported provider-level outcomes related to patient safety (eg, compliance with checklist items such as communication of medications, isolation precautions, etc.).^31-34,37

Description of Checklists

Supplementary File 4 (Table 3) provides a detailed breakdown of the checklists’ purpose and components. Six of the checklists were designed to directly reduce patient safety events,^{21,30,33,35-37} whereas 3 of the checklists were designed to indirectly reduce patient safety events by increasing compliance with processes of care.^31,32,34 Six checklists were constructed and pilot tested by the research team conducting the RCT^30-35 and the 3 remaining studies used modified versions of previously validated checklists.^21,36,37 The number of items included in the checklist ranged from 2 to 54.

Impact of the Checklist

Table 4 summarizes the adverse events, medical errors, resource utilization and/or compliance reported for each checklist. Chaudhary et al. reported significant decreases in Grade III (requiring intervention)³⁸ and IV (life-threatening)³⁸ postoperative complications (23% v. 33%, P = 0.04) and 30-day mortality (5.7% vs 10.0%, P = 0.04) for patients assigned to the Modified WHO Surgical Safety Checklist compared to controls.²¹ Conversely, Haugen et al. reported a nonsignificant reduction in 30-day mortality between the WHO Surgical Safety Checklist group and controls (1.0% vs 1.6%, P = 0.151).³⁶ Bassor et al. reported no significant difference in 30-day hospital readmission for decompensated heart failure for the heart failure discharge checklist group when compared to controls (6% vs. 4%, P = NS); however, an exploratory analysis that excluded patients who died during the follow-up period found a significant difference in 30-day readmission rates (2% vs. 20%, P = 0.02).³⁰ Gentili et al. reported a higher proportion of patients with pain control in the checklist group compared to the controls (67.6% vs. 54.8%), as well as fewer incidents of analgesic therapy–related uncontrolled adverse events (25.9% vs. 49.9%); however, the statistical significance of these differences were not reported.³⁵ The Writing Group for CHECKLIST-ICU reported no significant difference for in-hospital mortality between the checklist and control groups (adjusted odds ratio [AOR] 1.02, 95% CI, 0.82-1.26, P = 0.88), nor for the secondary clinical outcomes examined (Table 4).³³ However, there was a significant difference between the checklist group and control group for 3 of the 7 outcomes related to processes of patient care, including a reduction in the use of both urinary catheters (adjusted rate ratio [ARR] 0.86, 95% CI, 0.80-0.93, P < 0.001) and central venous catheters (ARR 0.90, 95% CI 0.83-0.98, P = 0.02). Masson et al. reported that when using the FASTHUG-MAIDENS checklist, more drug-related problems were identified by pharmacy residents (in relation to the number identified by the ICU pharmacist) both per patient encounter (P = 0.008) and overall (P < 0.001).³⁷ Ong et al. reported higher rates of compliance with isolation precautions for infectious diseases in the checklist group (71% vs. 38%, P < 0.01); however, compliance with the checklist was low (40%) and qualitative analyses found participants were dissatisfied with the checklist.³¹ Salzwedel et al. reported the number of items handed over by anesthesia residents postoperatively to be higher in the checklist group than the control group (48.7% vs. 32.4%, P < 0.001).³² In a more recent study, Salzwedel et al. reported that proportion of items deemed by the attending anesthesiologist as “must be handed over” were more often actually handed over by the anesthesia residents assigned to the checklist group when compared to controls (87.1% vs. 75.0%, P = 0.005).³⁴