Evaluating the accuracy of data extracted from electronic health records into MedicineInsight, a national Australian general practice database

Benjamin Daniels; Alys Havard; Rimma Myton; Cynthia Lee; Kendal Chidwick

doi:10.23889/ijpds.v7i1.1713

Evaluating the accuracy of data extracted from electronic health records into MedicineInsight, a national Australian general practice database

Int J Popul Data Sci. 2022 Jun 29;7(1):1713. doi: 10.23889/ijpds.v7i1.1713. eCollection 2022.

Authors

Benjamin Daniels^{1

2}, Alys Havard^{1

2

3}, Rimma Myton¹, Cynthia Lee¹, Kendal Chidwick¹

Affiliations

¹ NPS MedicineWise, Level 7 / 418a Elizabeth St, Strawberry Hills, NSW, 2012, Sydney, Australia.
² Medicines Policy Research Unit, Centre for Big Data Research in Health, UNSW Sydney, Australia.
³ National Drug and Alcohol Research Centre, UNSW Sydney, Australia.

Abstract

Introduction: MedicineInsight is a database containing de-identified electronic health records (EHRs) from over 700 Australian general practices. Previous research validated algorithms used to derive medical condition flags in MedicineInsight, but the accuracy of data fields following EHR extractions from clinical practices and data warehouse transformation processes have not been formally validated.

Objectives: To examine the accuracy of the extraction and transformation of EHR fields for selected demographics, observations, diagnoses, prescriptions, and tests into MedicineInsight.

Methods: We benchmarked MedicineInsight values against those recorded in original EHRs. Forty-six general practices contributing data to MedicineInsight met our eligibility criteria, eight were randomly selected, and four agreed to participate. We randomly selected 200 patients >18 years of age within each participating practice from MedicineInsight. Trained staff reviewed the original EHRs for the selected patients and recorded data from the relevant fields. We calculated the percentage of agreement (POA) between MedicineInsight and EHR data for all fields; Cohen's Kappa for categorical and intra-class correlation (ICC) for continuous measures; and sensitivity, specificity, and positive and negative predictive values (PPV/NPV) for diagnoses.

Results: A total of 796 patients were included in our analysis. All demographic characteristics, observations, diagnoses, prescriptions and random pathology test results had excellent (>90%) POA, Kappa, and ICC. POA for most recent pathology/imaging test was moderate (81%, [95% CI: 78% to 84%]). Sensitivity, specificity, PPV, and NPV were excellent (>90%) for all but one of the examined diagnoses which had a poor PPV.

Conclusions: Overall, our study shows good agreement between the majority of MedicineInsight data and those from original EHRs, suggesting MedicineInsight data extraction and warehousing procedures accurately conserve the data in these key fields. Discrepancies between test data may have arisen due to how data from pathology, radiology and other imaging providers are stored in EHRs and MedicineInsight and this requires further investigation.

Keywords: electronic health records; primary care; validation.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Australia
Electronic Health Records*
Family Practice
General Practice*
Humans
Records

Grants and funding

This study was funded by the Australian Government Department of Health. The funding body had no role in the design of the study, data collection, analysis or interpretation, nor in writing the manuscript. BD and AH are supported by the National Health and Medical Research Council (NHMRC) Centre of Research Excellence in Medicines Intelligence (ID: 1196900).