Standardized approach to training for cataract surgery skill evaluation

Purpose: To assess interrater reliability in grading cataract surgery performance of ophthalmology residents and attending physicians before and after rater skill training.

Setting: Metropolitan Chicago, Illinois, USA.

Design: Prospective interventional test design.

Methods: Video versions of a previously validated scoring key for rating cataract surgery were created for participant training. All participants received 2 frame-of-reference training sessions lasting 2 hours each. Participants graded 4 videorecorded cataract surgeries before training (pretest) and 4 more after training (posttest). Intraclass correlation coefficients (ICCs), which measured the degree of rater agreement, were calculated before and after training. Participants completed a subjective confidence questionnaire at the beginning and at the end of the study.

Results: The study participants included 9 postgraduate year (PGY)-3 residents, 8 PGY-4 residents, and 5 ophthalmology attending physicians from 4 medical centers in metropolitan Chicago. The rater training sessions had the greatest increase in ICCs among PGY-3 residents. The ICC improvement between the PGY-4 residents and attending physicians was uneven. After training, all residents felt more confident about their ability to rate and to perform cataract surgery.

Conclusions: Learning to effectively grade performance of cataract surgery by reviewing video clips has the potential to be an important part of ophthalmology residency education. The benefit of such a protocol was greatest in the early stages of residency education and might provide a guideline for quality assessment that accelerates surgical skill development.

Financial disclosure: None of the authors has a financial or proprietary interest in any material or method mentioned.