Visualising and quantifying the usefulness of new predictors stratified by outcome class: The U-smile method

PLoS One. 2024 May 20;19(5):e0303276. doi: 10.1371/journal.pone.0303276. eCollection 2024.

Abstract

Binary classification methods encompass various algorithms to categorize data points into two distinct classes. Binary prediction, in contrast, estimates the likelihood of a binary event occurring. We introduce a novel graphical and quantitative approach, the U-smile method, for assessing prediction improvement stratified by binary outcome class. The U-smile method utilizes a smile-like plot and novel coefficients to measure the relative and absolute change in prediction compared with the reference method. The likelihood-ratio test was used to assess the significance of the change in prediction. Logistic regression models using the Heart Disease dataset and generated random variables were employed to validate the U-smile method. The receiver operating characteristic (ROC) curve was used to compare the results of the U-smile method. The likelihood-ratio test demonstrated that the proposed coefficients consistently generated smile-shaped U-smile plots for the most informative predictors. The U-smile plot proved more effective than the ROC curve in comparing the effects of adding new predictors to the reference method. It effectively highlighted differences in model performance for both non-events and events. Visual analysis of the U-smile plots provided an immediate impression of the usefulness of different predictors at a glance. The U-smile method can guide the selection of the most valuable predictors. It can also be helpful in applications beyond prediction.

MeSH terms

  • Algorithms
  • Heart Diseases
  • Humans
  • Likelihood Functions
  • Logistic Models
  • ROC Curve*

Grants and funding

(1) An internal research grant from Poznan University of Medical Sciences funded this study. This funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. (2) Open access publication charges, proofreading, and editing costs were supported by the Ministry of Education and Science, Warsaw, Poland (grant number: NdS/544750/2021/2022). This grant is associated with the "Development of the University Centre for Sports and Medical Studies in Poznan, Poland" project within the "Science for Society" Programme (Principal Investigator: Prof. Przemysław Guzik). The grant specifically supports the unrestricted dissemination of scientific achievements by researchers at Poznan University of Medical Sciences. The Ministry of Education and Science had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.