Completeness, accuracy, and computability of National Quality Forum-specified eMeasures

Objective: To analyze the completeness, computability, and accuracy of specifications for five National Quality Forum-specified (NQF) eMeasures spanning ambulatory, post-discharge, and emergency care within a comprehensive, integrated electronic health record (EHR) environment.

Materials and methods: To evaluate completeness, we assessed eMeasure logic, data elements, and value sets. To evaluate computability, we assessed the translation of eMeasure algorithms to programmable logic constructs and the availability of EHR data elements to implement specified data criteria, using a de-identified clinical data set from Kaiser Permanente Northwest. To assess accuracy, we compared eMeasure results with those obtained independently by existing audited chart abstraction methods used for external and internal reporting.

Results: One measure specification was incomplete; missing applicable LOINC codes rendered it non-computable. For three of four computable measures, data availability issues occurred; the literal specification guidance for a data element differed from the physical implementation of the data element in the EHR. In two cases, cross-referencing specified data elements to EHR equivalents allowed variably accurate measure computation. Substantial data availability issues occurred for one of the four computable measures, producing highly inaccurate results.

Discussion: Existing clinical workflows, documentation, and coding in the EHR were significant barriers to implementing eMeasures as specified. Implementation requires redesigning business or clinical practices and, for one measure, systemic EHR modifications, including clinical text search capabilities.

Conclusions: Five NQF eMeasures fell short of being machine-consumable specifications. Both clinical domain and technological expertise are required to implement manually intensive steps from data mapping to text mining to EHR-specific eMeasure implementation.