Improving Resident Feedback on Diagnostic Reasoning after Handovers: The LOOP Project
Appropriate calibration of clinical reasoning is critical to becoming a competent physician. Lack of follow-up after transitions of care can present a barrier to calibration. This study aimed to implement structured feedback about clinical reasoning for residents performing overnight admissions, measure the frequency of diagnostic changes, and determine how feedback impacts learners’ self-efficacy. Trainees shared feedback via a structured form within their electronic health record’s secure messaging system. Forms were analyzed for diagnostic changes. Surveys evaluated comfort with sharing feedback, self-efficacy in identifying and mitigating cognitive biases’ negative effects, and perceived educational value of night admissions—all of which improved after implementation. Analysis of 544 forms revealed a 43.7% diagnostic change rate spanning the transition from night-shift to day-shift providers; of the changes made, 29% (12.7% of cases overall) were major changes. This study suggests that structured feedback on clinical reasoning for overnight admissions is a promising approach to improve residents’ diagnostic calibration, particularly given how often diagnostic changes occur.
© 2019 Society of Hospital Medicine
CONCLUSION
This study suggests that an intervention to operationalize standardized, structured feedback about diagnostic decision-making around transitions of care is a promising approach to improve residents’ understanding of changes in, and evolution of, the diagnostic process, as well as improve the perceived educational value of overnight admissions. In our results, over 40% of the patients admitted by residents had some change in their diagnoses after a transition of care during their early hospitalization. This finding highlights the importance of ensuring that trainees have the opportunity to know the outcomes of their decisions. Indeed, residents should be encouraged to follow-up on their own patients without prompting; however, studies show that this practice is uncommon and interventions beyond admonition are necessary.4
The diagnostic change rate observed in this study confirms that diagnosis is an iterative process and that the concept of a working diagnosis is key—a diagnosis made at admission will very likely be modified by time, the natural history of the disease, and new clinical information. When diagnoses are viewed as working diagnoses, trainees may be empowered to better understand the diagnostic process. As learners and teachers adopt this perspective, training programs are more likely to be successful in helping learners calibrate toward expertise.
Previous studies have questioned whether resident physicians view overnight admissions as valuable.6 After our intervention, we found an increase in both the amount of feedback received and the proportion of participants who agreed that night and day admissions were equally educational, suggesting that targeted diagnostic reasoning feedback can bolster educational value of nighttime admissions.
This study presents a number of limitations. First, the survey response rate was low, which could potentially lead to biased results. We excluded those respondents who did not respond to both the pre- and postsurveys from the analysis. Second, we did not measure actual change in diagnostic performance. While learners did report learning and saw feedback as valuable, self-identified learning points may not always translate to improved patient care. Additionally, residents chose the patients for whom feedback was provided, and the diagnostic change rate described may be overestimated. We did not track the total number of admissions for which feedback could have been delivered during the study. We did not include a control group, and the intervention may not be responsible for changing learners’ perceptions. However, the included programs were not implementing other new protocols focused on diagnostic reasoning during the study period. In addition, we addressed diagnostic changes early in a hospital course; a comprehensive program should address more feedback loops (eg, discharging team to admitting team).
This work is a pilot study; for future interventions focused on improving calibration to be sustainable, they should be congruent with existing clinical workflows and avoid adding to the stress and/or cognitive load of an already-busy clinical experience. The most optimal strategies for delivering feedback about clinical reasoning remain unclear.
In summary, a program to deliver structured feedback among resident physicians about diagnostic reasoning across care transitions for selected hospitalized patients is viewed positively by trainees, is feasible, and leads to changes in resident perception and self-efficacy. Future studies and interventions should aim to provide feedback more systematically, rather than just for selected patients, and objectively track diagnostic changes over time in hospitalized patients. While truly objective diagnostic information is challenging to obtain, comparing admission and other inpatient diagnoses to discharge diagnoses or diagnoses from primary care follow-up visits may be helpful. In addition, studies should aim to track trainees’ clinical decision-making over time and determine the effectiveness of feedback at improving diagnostic performance through calibration.
