The importance of technical skills assessment during an airway foreign body removal course

Purpose: Surgical simulation has proven useful in training for high-acuity, low frequency events such as airway foreign body removal (AWFBR). Studies have supported the role of simulation to improve trainee confidence and technical proficiency, but rigorous methodology is often lacking. Through a pilot study, we sought to evaluate the feasibility and utility of two-view video capture for rigorous assessment of trainee skill in an educational course setting.

Methods: Participants were asked to perform 1) self-assessment surveys using 5- point Likert scale questions and 2) rigid bronchoscopy with AWFBR on intubation trainers, at the beginning and end of a multi-institution airway course. Video was collected from endoscopic and wide-angle body cameras. The videos were blinded and evaluated in random order by two expert reviewers using a 5-point pediatric airway endoscopy specific objective structured assessment of technical skills (OSATS) instrument.

Results: Fourteen trainees submitted pre- and post-course surveys, and eight of these also had complete video data. Faculty feedback indicated the importance of pre- and post-course AWFBR recordings for real-time trainee feedback and post-course curricular refinement. Survey data showed an increase in confidence for AWFBR from 2.0 to 3.3 (p = 0.05). Average OSATS scores increased from 1.84 to 2.58 but this did not reach significance (p = 0.51). Paired improvements in confidence exceeded improvements in OSATS scores (1.29 vs. 0.18, p = 0.058).

Conclusions: Rigorously evaluated two-view video capture was feasible in an educational course setting. The course resulted in improvements in confidence to a greater degree than OSATS scores. This supports the importance of assessing course impact and refining curricula using all available data including objectively assessing technical skills.