Background: Hospital report cards and financial incentives linked to performance require clinical data that are reliable, appropriate, timely, and cost-effective to process. Pay-for-performance plans are transitioning to automated electronic health record (EHR) data as an efficient method to generate data needed for these programs.
Objective: To determine how well data from automated processing of structured fields in the electronic health record (AP-EHR) reflect data from manual chart review and the impact of these data on performance rewards.
Research design: Cross-sectional analysis of performance measures used in a cluster randomized trial assessing the impact of financial incentives on guideline-recommended care for hypertension.
Subjects: A total of 2840 patients with hypertension assigned to participating physicians at 12 Veterans Affairs hospital-based outpatient clinics. Fifty-two physicians and 33 primary care personnel received incentive payments.
Measures: Overall, positive and negative agreement indices and Cohen's kappa were calculated for assessments of guideline-recommended antihypertensive medication use, blood pressure (BP) control, and appropriate response to uncontrolled BP. Pearson's correlation coefficient was used to assess how similar participants' calculated earnings were between the data sources.
Results: By manual chart review data, 72.3% of patients were considered to have received guideline-recommended antihypertensive medications compared with 65.0% by AP-EHR review (κ=0.51). Manual review indicated 69.5% of patients had controlled BP compared with 66.8% by AP-EHR review (κ=0.87). Compared with 52.2% of patients per the manual review, 39.8% received an appropriate response by AP-EHR review (κ=0.28). Participants' incentive payments calculated using the 2 methods were highly correlated (r≥0.98). Using the AP-EHR data to calculate earnings, participants' payment changes ranged from a decrease of $91.00 (-30.3%) to an increase of $18.20 (+7.4%) for medication use (interquartile range, -14.4% to 0%) and a decrease of $100.10 (-31.4%) to an increase of $36.40 (+15.4%) for BP control or appropriate response to uncontrolled BP (interquartile range, -11.9% to -6.1%).
Conclusions: Pay-for-performance plans that use only EHR data should carefully consider the measures and the structure of the EHR before data collection and financial incentive disbursement. For this study, we feel that a 10% difference in the total amount of incentive earnings disbursed based on AP-EHR data compared with manual review is acceptable given the time and resources required to abstract data from medical records.