The Design and Evaluation of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program
BACKGROUND: Literature supports the use of point-of-care ultrasound performed by the treating hospitalist in the diagnosis of common diseases. There is no consensus on the training paradigm or the evaluation of skill retention for hospitalists.
OBJECTIVE: To evaluate the effectiveness of a comprehensive bedside ultrasound training program with postcourse competency assessments for hospitalists.
DESIGN: A retrospective report of a training program with 53 hospitalists. The program consisted of online modules, a 3-day in-person course, portfolios, 1-day refresher training, monthly scanning, and assessments. Hospitalists were rated by using similar pre- and postcourse competency assessments and self-rating parameters during the 3-day and refresher courses.
SETTING: A large tertiary-care center.
RESULTS: Skills increased after the 3-day course from a median preassessment score of 15% correct (interquartile range [IQR] 10%-25%) to a median postassessment score of 90% (IQR 80%-95%; P < .0001). At the time of the refresher course, the median precourse skills score had decreased to 65% correct (IQR 35%-90%), which improved to 100% postcourse (IQR 85%-100%; P < .0001). Skills scores decreased significantly less between the post 3-day course assessment and pre 1-day refresher course for hospitalists who completed portfolios (mean decrease 13.6% correct; P < .0001) and/or monthly scanning sessions (mean decrease 7.3% correct; P < .0001) compared with hospitalists who did not complete these items.
CONCLUSIONS: A comprehensive longitudinal ultrasound training program including competency assessments improved ultrasound acquisition skills with hospitalists. Skill retention remained high in those who completed portfolios and/or monthly scanning sessions along with a 1-day in-person refresher course.
© 2018 Society of Hospital Medicine
DISCUSSION
This is the first description of a successful longitudinal training program with assessments in POCUS for hospital medicine providers that shows an increase in skill retention with the use of a follow-up course and bedside scanning.
The CHAMP Ultrasound Program was developed to provide hospital medicine clinicians with a specialty focused in-house training pathway in POCUS and to assist in sustained skills acquisition by providing opportunities for regular feedback and practice. Practice with regular expert feedback is a critical aspect to develop and maintain skills in POCUS.32,33 Arntfield34 described the utility of remote supervision with feedback for ultrasound training in critical care, which demonstrated varying learning curves in the submission of portfolio images.35,36 The CHAMP Ultrasound training program provided expert oversight, longitudinal supervision, and feedback for course participants. The educational method of mastery learning was employed by setting minimum standards and allowing learners to practice until they met that standard.37-39
This unique program is made possible by the availability of expert-level faculty. Assessment scores improved with an initial 3-day course; however, they also decayed over time, most prominently with hospitalists that did not continue with POCUS scanning after their initial course. Ironically, those who performed more ultrasounds in the year prior to beginning the 3-day course had lower confidence ratings, likely explained by their awareness of their limitations and opportunities for improvement. The incorporation of refresher training to supplement the core 3-day course and portfolio development are key additions that differentiate this training program. These additions and the demonstration of successful training make this a durable pathway for other hospitalist programs. There are many workshops and short courses for medical students, residents, and practicing providers in POCUS.40-43 However, without an opportunity for longitudinal supervision and feedback, there is a noted decrease in the skills for participants. The refresher training with its 2 components (1-day in-person course and monthly scanning sessions) provides evidence of the value of mentored training.
In the initial program development, refresher training was encouraged but optional. We intentionally tracked those that completed refresher training compared with those that did not. Based on the results showing significant skills retention among those attending some form of refresher training, the program is planned to change to make this a requirement. We recommend refresher training within 12 months of the initial introductory course. There were several hospitalists that were unable to accommodate taking a full-day refresher course and, therefore, monthly scanning sessions were provided as an alternative.
The main limitation of the study is that it was completed in a single hospital system with available training mentors in POCUS. This gave us the ability to perform longitudinal training but may make this less reproducible in other hospital systems. Another limitation is that our course participants did not complete the pre- and postknowledge assessments for the refresher training components of the program, though they did for the initial 3-day course. Our pre- and postassessments have not been externally shown to produce valid data, though they are based on the already validated CHEST ultrasound data.44
Finally, our CHAMP Ultrasound Program required a significant time commitment by both faculty and learners. A relatively small percentage of hospitalists have completed the final assessments. The reasons are multifactorial, including program rigor, desire by certain hospitalists to know the basics but not pursue more expertise, and the challenges of developing a skillset that takes dedicated practice over time. We have aimed to address these barriers by providing additional hands-on scanning opportunities, giving timely feedback with portfolios, and obtaining more ultrasound machines. We expect more hospitalists to complete the final assessments in the coming year as evidenced by portfolio submissions to the shared online portal and many choosing to attend either the monthly scanning sessions and/or the 1-day course. We recognize that other institutions may need to adapt our program to suit their local environment.
CONCLUSION
A comprehensive longitudinal ultrasound training program including competency assessments significantly improved ultrasound acquisition skills with hospitalists. Those attending monthly scanning sessions and participating in the portfolio completion as well as a refresher course significantly retained and augmented their skills.
Acknowledgments
The authors would like to acknowledge Kelly Logue, Jason Robertson, MD, Jerome Siy, MD, Shauna Baer, and Jack Dressen for their support in the development and implementation of the POCUS program in hospital medicine.
Disclosure
The authors do not have any relevant financial disclosures to report.
