The Majority of Sports Medicine and Arthroscopy-Related Randomized Controlled Trials Reporting Nonsignificant Results Are Statistically Fragile

Purpose: To evaluate the robustness of sports medicine and arthroscopy related randomized controlled trials (RCTs) reporting nonsignificant results by calculating the reverse fragility index (RFI) and reverse fragility quotient (RFQ).

Methods: All sports medicine and arthroscopic-related RCTs from January 1, 2010, through August 3, 2021, were identified. Randomized-controlled trials comparing dichotomous variables with a reported P value ≥ .05 were included. Study characteristics, such as publication year and sample size, as well as loss to follow-up and number of outcome events were recorded. The RFI at a threshold of P < .05 and respective RFQ were calculated for each study. Coefficients of determination were calculated to determine the relationships between RFI and the number of outcome events, sample size, and number of patients lost to follow-up. The number of RCTs in which the loss to follow-up was greater than the RFI was determined.

Results: Fifty-four studies and 4,638 patients were included in this analysis. The mean sample size and loss to follow-up were 85.9 patients and 12.5 patients, respectively. The mean RFI was 3.7, signifying that a change of 3.7 events in one arm was needed to flip the results of the study from non-significant to significant (P < .05). Of the 54 studies investigated, 33 (61%) had a loss to follow-up greater than their calculated RFI. The mean RFQ was 0.05. A significant correlation between RFI with sample size (R² = 0.10, P = .02) and the total number of observed events (R² = 0.13, P < .01) was found. No significant correlation existed between RFI and loss to follow-up in the lesser arm (R² = 0.01, P = .41).

Conclusions: The RFI and RFQ are statistical tools that allow the fragility of studies reporting nonsignificant results to be appraised. Using this methodology, we found that the majority of sports medicine and arthroscopy-related RCTs reporting nonsignificant results are fragile.

Clinical relevance: RFI and RFQ serve as tools that can be used to assess the validity of RCT results and provide additional context for appropriate conclusions.