An evaluation of the accuracy and self-reported confidence of clinicians in using the ASA-PS Classification System

Objectives: The American Society of Anesthesiologists Physical Status (ASA-PS) is a grading system routinely adopted worldwide by physicians to classify patients' overall health status. Concerns have been raised surrounding the subjectiveness of this system, potentially leading to poor inter-rater agreement/reliability. We hypothesized that physicians are overconfident when assigning ASA-PS scores and that presenting them with the ASA-PS definitions/examples would improve accuracy. We therefore evaluated participants' accuracy and self-reported confidence on the ASA-PS Classification System (1) while assigning ASA-PS according to their baseline knowledge/judgment; and (2) after a single exposure to the ASA-PS definitions/examples.

Design: Prospective before-and-after web-based study.

Participants: 272 anesthesiologists and 114 non-anesthesiologists.

Interventions: Participants voluntarily answered a web-based questionnaire consisting of 10 hypothetical cases. They were asked to assign an ASA-PS score and rate their perceived self-confidence level (20-100%) on the accuracy of their assigned score for each case both (1) before and (2) after reviewing the ASA-PS definitions/examples. The correct ASA-PS for each hypothetical case was determined by consensus among investigators.

Measurements: Participants' accuracy, self-reported confidence, and calibration of confidence on the application of ASA-PS Classification System. Agreement between measures was tested using kappa coefficient.

Results: Anesthesiologists had better accuracy than non-anesthesiologists both on initial [6(5-7) vs. 4(3-5) out of 10; p < 0.001] as well as subsequent [7(6-8) vs. 6(4-7); p < 0.001] ASA-PS score assignments. Participants' self-reported confidence was greater than their accuracy for assigned ASA-PS scores (p < 0.001). ASA-PS agreement between anesthesiologists and non-anesthesiologists was poor (κ < 0.20). Participants' accuracy for hypothetical cases of ASA-PS I, II, and III involving adult patients was overall greater than for ASA-PS IV, V, and III (the latter involving a neonate) for both anesthesiologists and non-anesthesiologists (p < 0.001).

Conclusions: Physicians tend to disagree and be overconfident when assigning ASA-PS scores. A brief consultation of the ASA-PS definitions/examples improves the accuracy for both anesthesiologists and non-anesthesiologists.