Assessment of Racial Bias within the Risk Analysis Index of Frailty

Objective: Our objective was to assess potential racial bias within the Risk Analysis Index (RAI).

Background: Patient risk measures are rarely tested for racial bias. Measures of frailty, like the RAI, need to be evaluated for poor predictive performance among Black patients.

Methods: Retrospective cohort study using April 2010-March 2019 Veterans Affairs Surgical Quality Improvement Program and 2010-2019 National Surgical Quality Improvement Program data. The performance of the RAI and several potential variants were compared between Black and White cases using various metrics to predict mortality (180-day for Veterans Affairs Surgical Quality Improvement Program, 30-day for National Surgical Quality Improvement Program).

Results: Using the current, clinical threshold, the RAI performed as good or better among Black cases across various performance metrics versus White. When a higher threshold was used, Black cases had higher true positive rates but lower true negative rates, yielding 2.0% higher balanced accuracy. No RAI variant noticeably eliminated bias, improved parity across both true positives and true negatives, or improved overall model performance.

Conclusions: The RAI tends to predict mortality among Black patients better than it predicts mortality among White patients. As existing bias-reducing techniques were not effective, further research into bias-reducing techniques is needed, especially for clinical risk predictions. We recommend using the RAI for both statistical analysis of surgical cohorts and quality improvement programs, such as the Surgical Pause.