Agreement among sources of adult influenza vaccination in the age of immunization information systems

Introduction: Many vaccination studies rely on self-reported vaccination status, with its inherent biases. Accuracy of influenza vaccination self-report has been evaluated periodically, typically using the medical record as the gold standard. The burgeoning of electronic medical records (EMRs) and immunization information systems (IISs) and the rise of adult vaccine administration in community pharmacies suggest the need for a reevaluation of self-reported vaccination status.

Methods: Vaccination data from self-report, the state IIS, the health system EMR and other sources were compared for participants in outpatient and inpatient influenza vaccine effectiveness studies for four seasons (2016-2017 to 2019-2020). Agreement among the sources was calculated along with sensitivity and specificity. Tests for trend assessed changes in completeness of the Pennsylvania - Statewide IIS (PA-SIIS) data over time.

Results: With self-report as the gold standard, agreement with the local EMR, PA-SIIS, and all sources was 62%, 77% and 85%, respectively. Sensitivity of the EMR was 42% (95% CI = 41, 43) and specificity was 91% (90, 92). With PA-SIIS-as the gold standard, agreement with the local EMR and all sources was 77% and 78%, respectively. Sensitivity of all sources combined was 96% (95, 97) and specificity was (63% (62, 64). Capture of influenza vaccinations in the IIS has not consistently improved over time, with a significant increase among children (P = 0.001), no change among working-age adults and a decrease among older adults (P = 0.004). However, PA-SIIS provided the largest percentage of verified vaccines (69.3%) compared with EMR (43.3%) and other sources (12.4%).

Conclusion: Both self-report and PA-SIIS are good estimates of actual vaccine uptake. When high accuracy data are required, such as for vaccine effectiveness studies, triangulation using multiple sources should be conducted.