Measuring hospital performance in congenital heart surgery: administrative versus clinical registry data

Background: In congenital heart surgery, hospital performance has historically been assessed using widely available administrative data sets. Recent studies have demonstrated inaccuracies in case ascertainment (coding and inclusion of eligible cases) in administrative versus clinical registry data; however, it is unclear whether this impacts assessment of performance on a hospital level.

Methods: Merged data from The Society of Thoracic Surgeons (STS) database (clinical registry) and the Pediatric Health Information Systems (PHIS) database (administrative data set) for 46,056 children undergoing cardiac operations (2006-2010) were used to evaluate in-hospital mortality for 33 hospitals based on their administrative versus registry data. Standard methods to identify/classify cases were used: Risk Adjustment in Congenital Heart Surgery, version 1 (RACHS-1) in the administrative data and STS-European Association for Cardiothoracic Surgery (STAT) methodology in the registry.

Results: Median hospital surgical volume based on the registry data was 269 cases per year; mortality was 2.9%. Hospital volumes and mortality rates based on the administrative data were on average 10.7% and 4.7% lower, respectively, although this varied widely across hospitals. Hospital rankings for mortality based on the administrative versus registry data differed by 5 or more rank positions for 24% of hospitals, with a change in mortality tertile classification (high, middle, or low mortality) for 18% and a change in statistical outlier classification for 12%. Higher volume/complexity hospitals were most impacted. Agency for Healthcare Quality and Research (AHRQ) methods in the administrative data yielded similar results.

Conclusions: Inaccuracies in case ascertainment in administrative versus clinical registry data can lead to important differences in assessment of hospital mortality rates for congenital heart surgery.