Peak and ceiling effects in final-product analysis of mastoidectomy performance

J Laryngol Otol. 2015 Nov;129(11):1091-6. doi: 10.1017/S0022215115002364. Epub 2015 Sep 22.

Abstract

Background: Virtual reality surgical simulation of mastoidectomy is a promising training tool for novices. Final-product analysis for assessing novice mastoidectomy performance could be limited by a peak or ceiling effect. These may be countered by simulator-integrated tutoring.

Methods: Twenty-two participants completed a single session of self-directed practice of the mastoidectomy procedure in a virtual reality simulator. Participants were randomised for additional simulator-integrated tutoring. Performances were assessed at 10-minute intervals using final-product analysis.

Results: In all, 45.5 per cent of participants peaked before the 60-minute time limit. None of the participants achieved the maximum score, suggesting a ceiling effect. The tutored group performed better than the non-tutored group but tutoring did not eliminate the peak or ceiling effects.

Conclusion: Timing and adequate instruction is important when using final-product analysis to assess novice mastoidectomy performance. Improved real-time feedback and tutoring could address the limitations of final product based assessment.

Keywords: Clinical Competence; Computer Simulation; Education; Medical; Temporal Bone.

MeSH terms

  • Computer Simulation*
  • Educational Measurement
  • Feedback
  • Humans
  • Learning Curve
  • Mastoid / surgery*
  • Osteotomy / education*
  • Simulation Training* / methods
  • User-Computer Interface