Large-scale automated assessment of radiologist adherence to the Physician Quality Reporting System for stroke

Purpose: Physician Quality Reporting System (PQRS) measure 10 assesses the percentage of radiology reports for possible stroke that document the presence or absence of hemorrhage, mass, and acute infarction. Although it is an important report quality metric, determining adherence to this measure is often laborious, limiting its practical use. The aim of this study was to assess adherence to PQRS measure 10 using an automated approach to facilitate continuous measurement. A secondary goal was to identify explanatory variables that may affect adherence.

Methods: To determine measure adherence, a computerized algorithm was built, validated, and executed on 4,045 reports from CT and MRI examinations performed between January 2008 and October 2010 in patients with suspected stroke. Radiologist adherence was measured, accounting for differences in imaging modality, the presence of abnormalities, and trainee participation in report creation.

Results: Of 4,045 reports, 58.1% met the PQRS requirement, documenting all 3 components. Although the presence of infarct increased the chance of PQRS adherence (P < .001), the existence of hemorrhage had the opposite effect (P < .001). Reports that had trainee participation were more likely to be in accordance with PQRS standards (62% vs 47%, P < .001). After controlling for pertinent variables, more than 2-fold variation in individual PQRS adherence (27%-68%) remained (P < .001).

Conclusions: A considerable portion of eligible radiology reports do not include all components proposed by PQRS measure 10. An important contributor to performance gaps resides in individual physician variability. By automating measurement and monitoring of radiologist PQRS performance, informatics tools may enable targeted interventions to improve report quality.