The Design and Evaluation of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program
BACKGROUND: Literature supports the use of point-of-care ultrasound performed by the treating hospitalist in the diagnosis of common diseases. There is no consensus on the training paradigm or the evaluation of skill retention for hospitalists.
OBJECTIVE: To evaluate the effectiveness of a comprehensive bedside ultrasound training program with postcourse competency assessments for hospitalists.
DESIGN: A retrospective report of a training program with 53 hospitalists. The program consisted of online modules, a 3-day in-person course, portfolios, 1-day refresher training, monthly scanning, and assessments. Hospitalists were rated by using similar pre- and postcourse competency assessments and self-rating parameters during the 3-day and refresher courses.
SETTING: A large tertiary-care center.
RESULTS: Skills increased after the 3-day course from a median preassessment score of 15% correct (interquartile range [IQR] 10%-25%) to a median postassessment score of 90% (IQR 80%-95%; P < .0001). At the time of the refresher course, the median precourse skills score had decreased to 65% correct (IQR 35%-90%), which improved to 100% postcourse (IQR 85%-100%; P < .0001). Skills scores decreased significantly less between the post 3-day course assessment and pre 1-day refresher course for hospitalists who completed portfolios (mean decrease 13.6% correct; P < .0001) and/or monthly scanning sessions (mean decrease 7.3% correct; P < .0001) compared with hospitalists who did not complete these items.
CONCLUSIONS: A comprehensive longitudinal ultrasound training program including competency assessments improved ultrasound acquisition skills with hospitalists. Skill retention remained high in those who completed portfolios and/or monthly scanning sessions along with a 1-day in-person refresher course.
© 2018 Society of Hospital Medicine
Didactics were provided in a room with a 2-screen set up. Providers used 1 screen to present primary content and the other for simultaneously scanning a human model.
Image interpretation sessions were interactive smaller group learning forums in which participants reviewed high-yield images related to the care of hospital medicine patients and received feedback. Approximately 45 videos with normal and abnormal findings were reviewed during each session.
The hands-on scanning component was accomplished with human models and a faculty-to-participant ratio between 1:2 and
Portfolios
Refresher Training: 1-Day In-Person Course with Assessments and Monthly Scanning Sessions (Optional)
Only hospitalists who completed the 3-day course were eligible to take the 1-day in-person refresher course (supplementary Appendix 5). The first half of the course incorporated scanning with live human models, while the second half of the course had scanning with hospitalized patients focusing on pathology (pleural effusion, hydronephrosis, reduced left ventricular function, etc.). The course was offered at 3, 6, and 12 months after the initial 3-day course.
Monthly scanning sessions occurred for 2 hours every third Friday and were also available prior to the 1-day refresher. The first 90 minutes had a hands-on scanning component with hospitalized patients with faculty supervision (1:2 ratio). The last 30 minutes had an image interpretation component.
Assessments
Knowledge and skills assessment were adapted from the CHEST model (supplementary Appendix 6).24 Before and after the 3-day and 1-day in-person courses, the same hands-on skills assessment with a checklist was provided (supplementary Appendix 7). Before and after the 3-day course, a written knowledge assessment with case-based image interpretation was provided (supplementary Appendix 6).
Measurement
Participant demographic and clinical information was collected at the initial 3-day course for all participants, including age, gender, specialty, years of experience, and number and type of ultrasound procedures personally conducted or supervised in the past year. For skills assessment, a 20-item dichotomous checklist was developed and scored as done correctly or not done/done incorrectly. This same assessment was provided both before and after each of the 3-day and 1-day courses. A 20-question image-based knowledge assessment was also developed and administered both before and after the 3-day course only. The same 20-item checklist was used for the final skills examination. However, a new more detailed 50-question examination was written for the final examination after the portfolio of images was complete. Self-reported measures were confidence in the use of ultrasound, volume management, quality of physical exam, and ability to narrow the differential diagnosis. Confidence in ultrasound use, confidence in volume management, and quality of physical exam were assessed by using a questionnaire both before and after the 3-day course and 1-day course. Participants rated confidence and quality on a 5-point scale, 1 being least confident and 5 being most confident.
