Are Standardized Letters of Recommendation in Residency Applications Correlated With Objective Data?

Alice L Tang; Javier J M Howard; Eshita Singh; Meredith E Tabangin; James C Wang; Charles M Myer; Mekibib Altaye; Sarah L Rohde

doi:10.1177/2382120519893976

Are Standardized Letters of Recommendation in Residency Applications Correlated With Objective Data?

J Med Educ Curric Dev. 2019 Dec 9:6:2382120519893976. doi: 10.1177/2382120519893976. eCollection 2019 Jan-Dec.

Authors

Alice L Tang¹, Javier J M Howard¹, Eshita Singh¹, Meredith E Tabangin², James C Wang¹, Charles M Myer³, Mekibib Altaye², Sarah L Rohde⁴

Affiliations

¹ Department of Otolaryngology-Head and Neck Surgery, College of Medicine, University of Cincinnati, Cincinnati, OH, USA.
² Division of Biostatistics and Epidemiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.
³ Department of Otolaryngology-Head and Neck Surgery, Cincinnati Children's Hospital Medical Center, Cincinnati OH, USA.
⁴ Department of Otolaryngology-Head and Neck Surgery, Vanderbilt University Medical Center, Nashville, TN, USA.

Abstract

Objective: To evaluate whether the standardized letter of recommendation (SLOR), commonly used for medical students applying to otolaryngology residency, correlates with objective data in the application.

Background: Standardized letters of recommendation using Likert-type scales for different attributes are commonly used by evaluators because of their high interrater reliability and efficiency in preparation and interpretation. Given that these are subjectively scored, it is unknown how well these correlate with objective data.

Methods: Applications to the University of Cincinnati otolaryngology residency were evaluated in the academic cycle of 2017-2018. Standardized letters of recommendation were scored to determine whether certain attributes were correlated with objective data (United States Medical Licensing Examination [USMLE] scores and number of presentations/publications) provided in their application. Spearman correlations were used to evaluate the strength of the relationship between the subjective score in certain attributes with objective data.

Results: There were 217 applications to the University of Cincinnati that contained SLORs. Of these applications, 474 standardized letters were scored in categories of medical knowledge, research, and commitment to academic medicine. Total publications and presentations were weakly correlated with commitment to academic medicine (0.35, P < .0001, n = 369) and with research (0.44, P < .0001, n = 355). Medical knowledge was weakly correlated with Step 1 scores (0.20, P < .0001) and Step 2 scores (0.18, P = .0002).

Conclusions: Subjective research and commitment to academic medicine rating scores were weakly correlated with greater academic productivity. Similarly, medical knowledge scores were weakly correlated with Step 1/2 scores. Further research may be needed to assess how to interpret SLOR scores in addition to the information available in an otolaryngology application.

Level of evidence: 4.

Keywords: LOR; Standardized letters of recommendations; otolaryngology residency.