Portable Ultrasound Device Usage and Learning Outcomes Among Internal Medicine Trainees: A Parallel-Group Randomized Trial
BACKGROUND: Little is known about how to effectively train residents with point-of-care ultrasonography (POCUS) despite increasing usage.
OBJECTIVE: This study aimed to assess whether handheld ultrasound devices (HUDs), alongside a year-long lecture series, improved trainee image interpretation skills with POCUS.
METHODS: Internal medicine intern physicians (N = 149) at a single academic institution from 2016 to 2018 participated in the study. The 2017 interns (n = 47) were randomized 1:1 to receive personal HUDs (n = 24) for patient care vs no-HUDs (n = 23). All 2017 interns received a repeated lecture series regarding cardiac, thoracic, and abdominal POCUS. Interns were assessed on their ability to interpret POCUS images of normal/abnormal findings. The primary outcome was the difference in end-of-the-year assessment scores between interns randomized to receive HUDs vs not. Secondary outcomes included trainee scores after repeating lectures and confidence with POCUS. Intern scores were also compared with historical (2016, N = 50) and contemporaneous (2018, N = 52) controls who received no lectures.
RESULTS: Interns randomized to HUDs did not have significantly higher image interpretation scores (median HUD score: 0.84 vs no-HUD score: 0.84; P = .86). However, HUD interns felt more confident in their abilities. The 2017 cohort had higher scores (median 0.84), compared with the 2016 historical control (median 0.71; P = .001) and 2018 contemporaneous control (median 0.48; P < .001). Assessment scores improved after first-time exposure to the lecture series, while repeated lectures did not improve scores.
CONCLUSIONS: Despite feeling more confident, personalized HUDs did not improve interns’ POCUS-related knowledge or interpretive ability. Repeated lecture exposure without further opportunities for deliberate practice may not be beneficial for mastering POCUS.
© 2020 Society of Hospital Medicine
Outcomes
The primary outcome was the difference in assessment scores at the end of the study period between interns randomized to receive a HUD and those who were not. Secondary outcomes included differences in HUD usage rates, lecture attendance, and assessment scores. To assess whether repeated lecture exposure resulted in greater amounts of learning, this study evaluated for assessment score improvements after each lecture block. Finally, trainee attitudes toward POCUS and their confidence in their interpretative ability were measured at the beginning and end of the study period.
Curriculum Implementation
The lectures were administered as once-weekly didactics of 1-hour duration to interns rotating on the inpatient wards rotation. This rotation is 4 weeks long, and each intern will experience the rotation two to four times per year. Each lecture contained two parts: (1) 20-30 minutes of didactics via Microsoft PowerPointTM and (2) 30-40 minutes of supervised practice using HUDs on standardized patients. Four lectures were given each month: (1) introduction to POCUS and ultrasound physics, (2) thoracic/lung ultrasound, (3) echocardiography, and (4) abdominal POCUS. The lectures consisted of contrasting cases of normal/abnormal videos and clinical vignettes. These four lectures were repeated each month as new interns rotated on service. Some interns experienced the same content multiple times, which was intentional in order to assess their rates of learning over time. Lecture contents were based on previously published guidelines and expert consensus for teaching POCUS in internal medicine.13, 24-26 Content from the Accreditation Council for Graduate Medical Education (ACGME) and the American College of Emergency Physicians (ACEP) was also incorporated because these organizations had published relevant guidelines for teaching POCUS.13,26 Further development of the lectures occurred through review of previously described POCUS-relevant curricula.27-32
Handheld Ultrasound Devices
This study used the Philips LumifyTM, a United States Food and Drug Administration–approved device. Interns randomized to HUDs received their own device at the start of the rotation. It was at their discretion to use the device outside of the course. All devices were approved for patient use and were encrypted in compliance with our information security office. For privacy reasons, any saved patient images were not reviewed by the researchers. Interns were encouraged to share their findings with supervising physicians during rounds, but actual oversight was not measured. Interns not randomized to HUDs could access a single community device that was shared among all residents and fellows in the hospital. Interns reported the average number of POCUS examinations performed each week via a survey sent during the last week of the rotation.
Assessment Design and Implementation
Assessments evaluating trainee knowledge were administered before, during, and after the study period (Figure). For the 2017 cohort, assessments were also administered at the start and end of the ward month to track knowledge acquisition. Assessment contents were selected from POCUS guidelines for internal medicine and adaptation of the ACGME and ACEP guidelines.13,24,26 Additional content was obtained from major society POCUS tutorials and deidentified images collected by the study authors.13,24,33 In keeping with previously described methodology, the images were shown for approximately 12 seconds, followed by five additional seconds to allow the learner to answer the question.32 Final assessment contents were determined by the authors using the Delphi method.34 A sample assessment can be found in the Appendix Material.
