Interrater Reliability in Analysis of Laryngoscopic Features for Unilateral Vocal Fold Paresis

Objective: The diagnosis of paresis in patients with vocal fold motion impairment remains a challenge. In particular, laryngoscopy examination may result in significant disagreement in diagnosis among providers. We hypothesize that systematically evaluating for a standard set of clinical parameters will increase the diagnostic concordance among providers.

Study design: Prospective case series conducted at a Tertiary referral Laryngology office.

Methods: Two laryngologists (rater 1) and two trainees (rater 2) rated laryngoscopy findings in 19 patients suspected of paresis. The diagnosis was confirmed with laryngeal electromyogram. A standard set of 27 ratings was used for each examination that included movement, laryngeal configuration, and stroboscopy signs. A kappa coefficient was calculated for agreement in laryngoscopy findings and effectiveness in predicting the laterality of paresis.

Results: A substantial agreement (kappa coefficient > 0.61) existed between the raters for vocal fold length, vocal fold thickness, bowing, and reduction in movement. A moderate agreement (kappa coefficient > 0.41) existed between raters for piriform opening and reduced kinesis. The senior author was accurately able to diagnose the side of paresis in 89.5% of cases for a kappa coefficient of 0.78, whereas the trainees correctly predicted the side of paresis in 63.1% for a kappa coefficient of 0.35. The raters agreed on the diagnosis in 73.7% of cases for a kappa coefficient of 0.50.

Conclusions: Using a standard set of laryngoscopy findings may improve the provider's ability to identify the laterality of vocal fold paresis and increase interrater reliability compared with other series.