A Randomized Cohort Controlled Trial to Compare Intern Sign-Out Training Interventions

Data Collection Process

Eight trained senior residents, recruited by the last author (S.V.D.), volunteered to observe 10 evening sign-outs in each firm 1 month prior to the intervention and another 10 nights 4 months after training. Observations were standardized with a sign-out checklist developed from the literature review and the Joint Commission’s 2006 National Patient Safety Goal 2E that followed the Situation, Background, Assessment, and Recommendation communication structure with opportunities for questioning and information verification.^14,15 Observers indicated “1” for each of the 17 sign-out elements in the checklist they observed, as shown in the supporting Table. Observers did not have supervisory relationships with the interns. Occasionally, the pairs of observers were different depending on their availability.

Outcomes

We measured improvements in sign-out quality by the mean percentage differences for each of the 3 dimensions of sign-out, as well as a multidimensional measure of sign-out comprising the 3 dimensions for each firm in 2 ways: (1) pre- and postintervention, and (2) vis-à-vis the control group postintervention.

Statistical Analysis

We factor analyzed the 17 sign-out elements using principal components analysis with varimax rotation to confirm their groupings within the 3 dimensions of sign-out using Statistical Package for the Social Sciences (SPSS) version 24 (IBM, North Castle, NY). We calculated the mean percentage differences and used Student t tests to evaluate statistical differences at P < 0.05.

RESULTS

Five hundred and sixty-three patient sign-outs were observed prior to the training interventions (κ = 0.646), and 620 patient sign-outs were observed after the interventions (κ = 0.648). Kappa values derived from SPSS were within acceptable interrater agreement ranges. Factor analysis of the 17 sign-out elements yielded 3 factors that we named patient information, task accountability, and responsibility, as shown in the supporting Table.

The supporting Figure reports 2 sets of results. The line graphs show the pre- and postintervention differences for each firm while the bar charts show the postintervention differences between each firm vis-à-vis the control group on sign-out dimensions. The line graphs indicate the greatest improvements in patient information, task accountability, and responsibility for the I-PASS, policy mandate, and PDSA groups, respectively. Mandate and PDSA groups reported low relative scores on sign-out dimensions that were not the foci of their training while the didactics group scored around 0 pre- and postintervention. I-PASS had the highest improvement on the multidimensional measure of sign-out quality but was not significantly different from the PDSA group at P < 0.05 (see supporting Figure for the calculations). The bar charts indicate that all groups vis-à-vis the control had higher improvements in task accountability, responsibility, and the multidimensional measure of sign-out quality. I-PASS vis-à-vis the control had the highest improvement but was not statistically different from the PDSA at P < 0.05. No sentinel events were reported during the entire study period.

DISCUSSION

The results indicated that after only 1 hour of training, skill-based, compliance-based, and learner-initiated sign-out training improved sign-out quality beyond knowledge-based didactics even though the number of sign-out elements taught in the latter 2 was lower than in the didactics group. Different training emphases influenced different dimensions of sign-out quality so that training interns to focus on task accountability or responsibility led to improvements in those dimensions only. The lower scores in other dimensions suggest potential risks in sign-out quality from focusing attention on 1 dimension at the expense of other dimensions. I-PASS, which covered the most sign-out elements and utilized 5 facilitators, led to the best overall improvement in sign-out quality, which is consistent with previous studies.^3,12 We demonstrated that only 1 hour of training on the I-PASS mnemonics using video, role-playing, and feedback led to significant improvements. This approach is portable and easily applied to any program. Potential improvements in I-PASS training could be obtained by emphasizing task accountability and responsibility because the mandate and PDSA groups obtained higher scores than the I-PASS group in these dimensions.

Limitations

We measured sign-out quality in the evening at this site because it was at greatest risk for errors. Future studies should consider daytime sign-outs, interunit handoffs, and other hospital settings, such as community or rural hospitals and nonacute patient settings, to ascertain generalizability. Data were collected from observations, so Hawthorne effects may introduce bias. However, we believe that using a standardized checklist, a control group, and assessing relative changes minimized this risk. Although we observed almost 1200 patient sign-outs over 80 shift changes, we were not able to observe every intern in every firm. Finally, no sentinel events were reported during the study period, and we did not include other measures of clinical outcomes, which represent an opportunity for future researchers to test which specific sign-out elements or dimensions are related to clinical outcomes or are relevant to specific patient types.