Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy

Alexander D Sherry; Pavlos Msaouel; Zachary R McCaw; Joseph Abi Jaoude; Eric J Hsu; Ramez Kouzy; Roshal Patel; Yumeng Yang; Timothy A Lin; Cullen M Taniguchi; Claus Rödel; Emmanouil Fokas; Chad Tang; Clifton David Fuller; Bruce Minsky; Tomer Meirson; Ryan Sun; Ethan B Ludmir

doi:10.1016/j.ejca.2023.113357

Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy

Eur J Cancer. 2023 Nov:194:113357. doi: 10.1016/j.ejca.2023.113357. Epub 2023 Sep 22.

Authors

Alexander D Sherry¹, Pavlos Msaouel², Zachary R McCaw³, Joseph Abi Jaoude⁴, Eric J Hsu⁵, Ramez Kouzy¹, Roshal Patel⁶, Yumeng Yang⁷, Timothy A Lin⁸, Cullen M Taniguchi⁹, Claus Rödel¹⁰, Emmanouil Fokas¹⁰, Chad Tang¹¹, Clifton David Fuller¹, Bruce Minsky¹², Tomer Meirson¹³, Ryan Sun¹⁴, Ethan B Ludmir¹⁵

Affiliations

¹ Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
² Department of Genitourinary Medical Oncology, Division of Cancer Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Translational Molecular Pathology, Division of Pathology/Lab Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
³ Insitro, South San Francisco, CA, USA; Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
⁴ Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Radiation Oncology, Stanford University, Stanford, CA, USA.
⁵ Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, USA.
⁶ Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Radiation Oncology, Memorial Sloan-Kettering Cancer Center, New York, NY, USA.
⁷ Department of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, USA.
⁸ Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA.
⁹ Department of Gastrointestinal Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Experimental Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
¹⁰ Department of Radiotherapy and Oncology, University of Frankfurt, Frankfurt, Germany; Frankfurt Cancer Institute, Frankfurt, Germany; German Cancer Research Center (DKFZ), Heidelberg, German Cancer Consortium (DKTK), Partner Site Frankfurt am Main, Frankfurt, Germany.
¹¹ Department of Translational Molecular Pathology, Division of Pathology/Lab Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Genitourinary Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
¹² Department of Gastrointestinal Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
¹³ Davidoff Cancer Center, Rabin Medical Center, Petach Tikva, Israel.
¹⁴ Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
¹⁵ Department of Gastrointestinal Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA. Electronic address: [email protected].

Abstract

Background: The 'Table 1 Fallacy' refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials.

Methods: From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy.

Results: A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18-0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19-0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03-0.13; multiplicity-corrected P < 0.0001).

Conclusions: This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.

Keywords: Covariate imbalance; Oncology; Phase III; Randomised controlled trials; Significance testing for baseline characteristics; Table 1 Fallacy; Testing for baseline differences.

Publication types

Research Support, Non-U.S. Gov't
Research Support, N.I.H., Extramural

MeSH terms

Cross-Sectional Studies
Humans
Neoplasms* / epidemiology
Neoplasms* / therapy
Prevalence
Randomized Controlled Trials as Topic

Abstract

Publication types

MeSH terms

Grants and funding