Pilot Study for an Orthopedic Surgical Training Laboratory for Basic Motor Skills
The most effective way to teach and assess a resident’s knowledge of musculoskeletal medicine, including orthopedic-specific surgical skills, remains unclear.
We designed a surgical skills training session to educate junior-level orthopedic residents in 4 core areas: comfort with basic power equipment, casting/splinting, suturing, and surgical instrument identification. As part of the study reported here, 11 orthopedic residents (postgraduate year 1-3) completed a skills session and were evaluated with written examinations and an ankle fracture model before and after the session. Four other junior residents were unable to attend the session because of clinical responsibilities.
For the group of 11 residents who completed the written examination, mean (SD) presession percentile was 87.3 (10.4), mean (SD) postsession percentile was 92 (8.4), median was 96, and mode was 96. There was a significant pre–post difference among all test takers, regardless of training level (P < .05). In the ankle fracture model, for the entire group, mean (SD) overall presession percentile was 68.6 (13.9), and mean (SD) overall postsession percentile was 95.2 (5.2). There was a significant pre–post difference among all test takers, regardless of training level (P = .03). An intensive laboratory has the potential to improve junior-level residents’ basic surgical skills and knowledge.
The didactic PowerPoint presentation drew largely from content in chapters of the book AO Principles of Fracture Management.22 Content included condensed, to-the-point high-yield summaries of AO tenets, basic bone healing and biology, and orthopedic implant-design rationale focused on these elementary principles:
◾ Basic screw design, including cortical, cancellous, and locking screw designs.
◾ Evolution of plate osteosynthesis to currently used locking compression plate.
◾ Locking plate principles.
◾ Lag technique.
◾ Plate use: compression mode, neutralization, bridging, buttress, anti-glide.
The suturing portion was performed with thawed ham hocks (Figure 4). This model replicates live tissue layers and allows a layered closure technique as a training tool. Both 0 and 2-0 absorbable suture were available for a layered, deep fascial closure; also available was 2-0 nonabsorbable nylon for the skin. Staple guns were available, as were basic surgical instruments, including quality needle drivers, Adson forceps, and suture scissors. The knots demonstrated included simple, horizontal mattress, vertical mattress, and tension-relieving. One- and 2-hand tying and instrument tying were reinforced.
The final session consisted of surgical instrument identification. A certified orthopedic scrub technician participated. On site were multiple trays, including a basic bone set, a hand-and-foot set, small and large fragment sets, and a hip set. A detailed review of each set was led by the surgical technician. This review was followed by a question-and-answer session with the junior residents. After the session, we ended with the written examination and the ankle fracture model.
Statistical Methods
We report presession and postsession means, modes, and medians as measures of score-central tendencies. Our small sample size makes the assumption of Gaussian distribution tenuous and more susceptible to outliers. Therefore, in addition to reporting means, we include medians and modes to more accurately account for outliers. Moreover, the κ statistic is a robust measure of interrater agreement for 2 or more groups. We report κ statistics to determine the interrater reliability of 4 independent observers.
Results
Written Examination
Eleven residents (PGY-1 to PGY-3) completed the examination (Table 1). For the entire group, mean (SD) presession percentile was 87.3 (10.4), median was 88, and mode was 96; mean (SD) was 80 (12.6) for PGY-1, 89.6 (6.7) for PGY-2, and 96 (5.7) for PGY-3. For the entire group, mean (SD) postsession percentile was 92 (8.4), median was 96, and mode was 96; mean (SD) was 85 (10.5) for PGY-1, 96 (4) for PGY-2, and 96 (0) for PGY-3 (Table 2).
There was a significant presession–postsession difference in scores among all test takers, regardless of training level (P = .019). The PGY-1 level did not reach statistical significance in improvement from presession to postsession (P = .080); the PGY-2 level also did not reach statistical significance in improvement (P = .099); the PGY-3 level did not have enough participants to calculate a P value based on a paired Student t test.
Ankle Fracture Model
Actual percentile scores are listed in Table 3. For the entire group, mean (SD) overall presession percentile was 68.6 (13.9), median was 67, and mode was 67; mean (SD) was 58.8 (9.8) for PGY-1, 76.1 (13.6) for PGY-2, and 69.5 (9.8) for PGY-3. For the entire group, mean (SD) postsession percentile was 95.2 (5.2), median was 97, and mode was 97; mean (SD) was 91.8 (6.3) for PGY-1, 97.1 (3.5) for PGY-2, and 97.3 (2.4) for PGY-3.
There was a large and significant presession–postsession difference in scores among all test takers, regardless of training level (P = .03). Each group reached statistical significance in improvement from presession to postsession: PGY-1 (P = .04), PGY-2 (P = .01), and PGY-3 (P = .03).
For κ calculations, we adjusted all scores to ordinal data and thus used a standard grading system:
Score Grade
90–100 A
80–89 B
70–79 C
60–69 D
0–59 F
For the presession fracture model, the κ among the 4 independent observational scorers was 0.1148 (Table 4), which is poor based on κ scoring criteria and which we attribute to the particularly harsh grading by 1 observational scorer (faculty 1) relative to the other scorers’. Examination of the κ scores of faculty 1 and faculty 2 indicated only 9.09% agreement. Conversely, the κ among resident scorers agreed 54.55% of the time. Removing faculty 1 as an outlier raised the κ score dramatically, to 0.3125 (fair interobserver agreement).
For the postsession fracture model, the κ among the 4 independent observational scorers improved only marginally, to 0.1156 (still poor), again attributed to a difference in severity of grading: faculty 1 (harsh) versus faculty 2 (relatively kind). Examination of the κ scores of faculty 1 and faculty 2 revealed 72.73% agreement; residents agreed 81.82% of the time.
