Missing in action: a case study of the application of methods for dealing with missing data to trauma system benchmarking

Background: Trauma registry data are usually incomplete. Various methods for dealing with missing data have been used, some of which lead to biased results. One method that reduces bias, multiple imputation (MI), has not been widely adopted. There is no standardization of the approach to missing data across trauma registries.

Objectives: This study examined the effect of using selected methods for handling missing data on a recognized trauma outcome measure.

Methods: Data from the Victorian State Trauma Registry (VSTR) were used for the period July 2003 to June 2008. Three methods for handling missing data were investigated: complete case analysis, single imputation, and MI. The latter was applied using five distinct models, each with a different combination of variables (Trauma and Injury Severity score [TRISS] variables; prehospital Glasgow Coma Scale [GCS], respiratory rate, and systolic blood pressure; arrival by ambulance; transfer to a second hospital; and whether the GCS was "legitimate" according to the TRISS definition). For each method, TRISS analysis (comparing actual and expected deaths) was performed; the W-score and Z-statistic were derived. A Z-statistic greater than 1.96 in absolute value was considered statistically significant.

Results: Of 10,180 cases, 2,398 (24%) were missing at least one of the component variables necessary for TRISS analysis. With the use of complete case analysis, the W-score was 0.54 unexpected survivors for every 100 cases, with a Z-statistic of -1.96. Using two approaches to single imputation, the W-scores were -1.41, with Z-statistics of -5.19 and -5.30. Applying four of the five combinations of variables used for MI, there was a statistically significant number of unexpected survivors (W = -0.60, Z = -2.23; W = -0.52, Z = -1.97; W = -0.53, Z = -1.97; W = -0.63, Z = -2.24). However, using MI confined to TRISS variables only, there was a statistically significant number of unexpected deaths (W = +0.52, Z = +1.98).

Conclusions: Missing data methods can influence the assessment of trauma care performance and need to be reported in all analyses. It is important that validated standardized approaches to dealing with missing data are universally adopted and reported.