The Impact of Checklists on Inpatient Safety Outcomes: A Systematic Review of Randomized Controlled Trials
BACKGROUND: Systematic reviews of non-randomized controlled trials (RCTs) suggest that using a checklist results in fewer medical errors and adverse events, but these evaluations are at risk of bias.
OBJECTIVE: To conduct a systematic review of RCTs of checklists to determine their effectiveness in improving patient safety outcomes in hospitalized patients.
METHODS: Ovid EMBASE, Ovid MEDLINE, PubMed, and the Cochrane Central Register of Controlled Trials were searched from inception until December 8, 2016. The search was restricted to RCTs. Included studies reported patient safety outcomes of a checklist intervention. Data extracted included the study characteristics, setting, population, intervention, outcomes measures, and sample size.
MEASUREMENTS AND MAIN RESULTS: 11,225 citations were identified, of which 9 (16,987 patients) satisfied the inclusion criteria. Citations reported evaluations of checklists designed to improve surgical safety, prescription of medications, heart failure management, pain control, infection control precautions, and physician handover. Studies reported significant reductions in postoperative complications and medication-related problems and improved compliance with evidence-based prescribing of medications, infection control precautions, and patient handover procedures. 30-day mortality was reported in 3 studies and was significantly lower among patients allocated to the checklist group (odds ratio 0.60, 95% confidence interval, 0.41-0.89, P = 0.01, I2 = 0.0%, P = 0.573). Methodological quality of the studies was moderate.
CONCLUSION: A small number of citations report RCT evaluations of the impact of checklists on patient safety. There is an urgent need for high-quality evaluations of the effectiveness of patient safety checklists in inpatient healthcare settings to substantiate their perceived benefits.
© 2017 Society of Hospital Medicine
In response to widely publicized reports highlighting the challenges of suboptimal quality of healthcare, improving patient safety has been a leading healthcare initiative for more than 10 years.1-4 Numerous strategies to improve patient safety have been proposed,5-9 but improvements have been limited, which raises questions about whether the right approaches are being employed.10,11
Checklists have served as a foundation for the standardization and safety of aviation and nuclear power12,13 and are advocated as simple and effective instruments for ensuring safe care.7,14,15 Systematic reviews of observational studies suggest that checklists can reduce medical errors and adverse events,15-19 but these reviews are at risk of bias due to the limitations of observational methods. Furthermore, discordant results of recent high-profile evaluations of the World Health Organization (WHO) Surgical Safety Checklist highlight the need for checklist evaluations using rigorous study designs.20-22 Therefore, we sought to conduct a systematic review of RCTs (randomized controlled trials) to determine whether checklists, as a type of decision-support tool, are effective at improving patient safety outcomes in hospitalized patients.
METHODS
The study protocol was registered with the PROSPERO Register of Systematic Reviews (registration number: CRD42016037441) and developed according to the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) statement.23
Search Strategy
On December 8, 2016, we systematically searched Ovid MEDLINE, Ovid EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The search was performed using no language or publication date restrictions and included 2 groups of terms (key words with similar characteristics): ‘checklists’ and ‘patient outcomes assessment’. We restricted our search to patient outcomes because these are more patient-oriented than the proximal processes of care that may not translate into outcomes. The search was restricted to RCTs using the Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials from the Cochrane Collaborative.24 The MEDLINE search strategy is depicted in Appendix I (Supplementary File 1). Reference lists of included articles were manually searched for additional publications. The search strategy was designed with the help of an information scientist (DL). EndNote X7 (Thomas Reuters, Philadelphia, PA, USA) was the reference software used for the management of citations.
Eligibility Criteria
We selected all studies reporting patient safety outcomes of a checklist intervention, using the following inclusion criteria: 1) acute care hospital inpatient population, 2) checklist intervention, 3) contain a control group (ie, no checklist), 4) report one or more patient safety outcome, as defined by the authors (eg, medical errors, adverse events, mortality), and 5) RCT design. We restricted our focus to inpatient populations given the heterogeneity of illness and patient care between acute and community settings. We defined a checklist as a tool that details the essential steps of a task, requiring the target provider to indicate whether an item was completed or not.1,7 Tools that included only 1 item (eg, electronic prompts) or did not require acknowledgement of the items (eg, guidelines) were excluded. We defined patient safety outcomes as the authors’ definition of patient safety (eg, medical error, adverse event, provider compliance with safety regulations).
Study Selection
Two reviewers (JMB, GW) independently, and in duplicate, reviewed the titles and abstracts of the retrieved citations against the eligibility criteria. The same 2 reviewers subsequently reviewed the full text of relevant articles for inclusion. Eligibility disagreements were resolved by consensus. A Kappa statistic was calculated for reviewer agreement of full-text screening.25 Reviewers were not blinded to author or journal names.26
Data Extraction
The structured data extraction form was calibrated using the first 2 articles. The 2 reviewers (JMB, GW) independently, and in duplicate, extracted data from included studies on the study characteristics, setting, study population, sample size, intervention used, outcomes examined, analytic method, and study quality. The data extraction form is depicted in Appendix II (Supplementary File 2). Coding discrepancies were resolved by consensus.
Quality Assessment
The 2 reviewers (JMB, GW) extracted data on study quality independently and in duplicate using 2 approaches. First, reviewers assessed study quality using a component method derived from the Cochrane Collaboration criteria.24 For each included study, the reviewers documented if the authors had adequately described inclusion/exclusion criteria, randomization, allocation concealment, blinding of participants/outcome assessors, attrition, cross over, baseline characteristics, and power calculation. Second, the reviewers calculated and reported the Jadad score for each included study, a validated assessment scale that assigns points (1 to 5) based on randomization, blinding, and attrition.27
