Improving risk classification of critical illness with biomarkers: a simulation study

Purpose: Optimal triage of patients at risk for critical illness requires accurate risk prediction, yet few data on the performance criteria required of a potential biomarker to be clinically useful exists.

Materials and methods: We studied an adult cohort of nonarrest, nontrauma emergency medical services encounters transported to a hospital from 2002 to 2006. We simulated hypothetical biomarkers increasingly associated with critical illness during hospitalization and determined the biomarker strength and sample size necessary to improve risk classification beyond a best clinical model.

Results: Of 57,647 encounters, 3121 (5.4%) were hospitalized with critical illness and 54,526 (94.6%) without critical illness. The addition of a moderate-strength biomarker (odds ratio, 3.0, for critical illness) to a clinical model improved discrimination (c statistic, 0.85 vs 0.8; P<.01) and reclassification (net reclassification improvement, 0.15; 95% confidence interval, 0.13-0.18) and increased the proportion of cases in the highest-risk category by +8.6% (95% confidence interval, 7.5%-10.8%). Introducing correlation between the biomarker and physiological variables in the clinical risk score did not modify the results. Statistically significant changes in net reclassification required a sample size of at least 1000 subjects.

Conclusions: Clinical models for triage of critical illness could be significantly improved by incorporating biomarkers, yet substantial sample sizes and biomarker strength may be required.