Beyond the Usual Significance: Fragility Indices of Randomized Controlled Trials in Top General Orthopaedic Journals

Mohammad Poursalehian; Mahdi Sahebi; Mahboobeh Tajvidi; Amirhosein Sabaghian; Amir-Mohammad Asgari; Sean A Tabaie; Mohit Bhandari; Amir Human Hoveidaei

doi:10.5435/JAAOS-D-24-00691

Beyond the Usual Significance: Fragility Indices of Randomized Controlled Trials in Top General Orthopaedic Journals

J Am Acad Orthop Surg. 2024 Dec 19. doi: 10.5435/JAAOS-D-24-00691. Online ahead of print.

Authors

Mohammad Poursalehian¹, Mahdi Sahebi, Mahboobeh Tajvidi, Amirhosein Sabaghian, Amir-Mohammad Asgari, Sean A Tabaie, Mohit Bhandari, Amir Human Hoveidaei

Affiliation

¹ From the Joint Reconstruction Research Center, Tehran University of Medical Sciences, Tehran, Iran (Poursalehian, Sahebi, Tajvidi, and Sabaghian), the Students' Scientific Research Center (SSRC), Tehran University of Medical Sciences, Tehran, Iran (Sahebi), the Student Research Committee, Faculty of Medicine, Abadan University of Medical Sciences, Abadan, Iran (Tajvidi), the School of Medicine, Iran University of Medical sciences, Tehran, Iran (Sabaghian), the Orthopedic Surgery Research Center (OSRC), Sina University Hospital, Tehran University of Medical Sciences, Tehran, Iran (Asgari), the Department of Orthopaedic Surgery, Nationwide Children's Hospital, Columbus, OH (Tabaie), the Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada (Bhandari), the Division of Orthopedic Surgery, Department of Surgery, McMaster University, Hamilton, Ontario, Canada (Bhandari), and the International Center for Limb Lengthening, Rubin Institute for Advanced Orthopedics, Sinai Hospital of Baltimore, Baltimore, MD (Hoveidaei).

PMID: 39706153
DOI: 10.5435/JAAOS-D-24-00691

Abstract

Introduction: Understanding the reliability of outcomes in randomized controlled trials (RCTs) is crucial, as standard metrics, such as P-value do not fully capture result fragility. This led to the adoption of specific indices: the fragility index (FI), which measures the strength of trial through significant results by calculating the minimum number of patient status changes from nonevent to event required to make the results statistically insignificant; reverse fragility index (RFI), used for insignificant results; and continuous fragility index (CFI), which acts similar to FI for significant continuous outcomes. The objective is to assess the robustness of orthopaedics RCTs using these indices across leading orthopaedic journals.

Methods: A systematic review of RCTs published between 2019 and 2023 in the top five general orthopaedic journals, identified through Scientific Journal Rankings, was done. Data extraction focused on FI, RFI, and CFI and related measures for 160 RCTs. The indices were calculated using established methodologies, with sample size adjustments.

Results: 22 RCTs had statistically significant dichotomous primary outcomes and 17 studies had notable dichotomous secondary outcomes. Twenty-nine had negligible (insignificant) dichotomous primary outcomes, and 92 reported notable continuous outcomes. Only one RCT reported a FI in the article. The median FI was 5 with a median sample size of 142 for dichotomous outcomes. The median RFI was 3 with a median sample size of 100 for negligible outcomes, and the median CFI was 13 with a median sample size of 86.5 for continuous outcomes, showing dichotomous outcomes to be more fragile than continuous ones.

Discussion: Continuous outcomes are less fragile than dichotomous outcomes, with negligible dichotomous outcomes being particularly more fragile. This fragility stems from small sample sizes and limited outcome events. Using these indices, especially when considering patient loss to follow-up, can improve the reliability of findings.

Level of evidence: I.