Comparison of Differences in Cohort (Forward) and Case Control (Backward) Methodologic Approaches for Validation of the Hypotension Prediction Index

Background: The Hypotension Prediction Index (the index) software is a machine learning algorithm that detects physiologic changes that may lead to hypotension. The original validation used a case control (backward) analysis that has been suggested to be biased. This study therefore conducted a cohort (forward) analysis and compared this to the original validation technique.

Methods: A retrospective analysis of data from previously reported studies was conducted. All data were analyzed identically with two different methodologies, and receiver operating characteristic curves were constructed. Both backward and forward analyses were performed to examine differences in area under the receiver operating characteristic curves for the Hypotension Prediction Index and other hemodynamic variables to predict a mean arterial pressure (MAP) less than 65 mmHg for at least 1 min 5, 10, and 15 min in advance.

Results: The analysis included 2,022 patients, yielding 4,152,124 measurements taken at 20-s intervals. The area under the curve for the index predicting hypotension analyzed by backward and forward methodologies respectively was 0.957 (95% CI, 0.947 to 0.964) versus 0.923 (95% CI, 0.912 to 0.933) 5 min in advance, 0.933 (95% CI, 0.924 to 0.942) versus 0.923 (95% CI, 0.911 to 0.933) 10 min in advance, and 0.929 (95% CI, 0.918 to 0.938) versus 0.926 (95% CI, 0.914 to 0.937) 15 min in advance. No variable other than MAP had an area under the curve greater than 0.7. The areas under the curve using forward analysis for MAP predicting hypotension 5, 10, and 15 min in advance were 0.932 (95% CI, 0.920 to 0.940), 0.929 (95% CI, 0.918 to 0.938), and 0.932 (95% CI, 0.921 to 0.940), respectively. The R2 for the variation in the index due to MAP was 0.77.

Conclusions: Using an updated methodology, the study found that the utility of the Hypotension Prediction Index to predict future hypotensive events is high, with an area under the receiver operating characteristics curve similar to that of the original validation method.