The Role of Double-Zero-Event Studies in Evidence Synthesis: Evaluating Robustness Using the Fragility Index

Rationale: Zero-event counts are common in clinical studies, particularly when assessing rare adverse events. These occurrences can result from low event rates, short follow-up periods, and small sample sizes. When both intervention and control groups report zero events in a clinical trial, the study is referred to as a double-zero-event study, which presents methodological challenges for evidence synthesis. There has been ongoing debate about whether these studies should be excluded from evidence synthesis, as traditional two-stage meta-analysis methods may not estimate an effect size for them. Recent research suggests that these studies may still contain valuable clinical and statistical information.

Aims and objectives: This study examines the role of double-zero-event studies from the perspective of the fragility index (FI), a popular metric for assessing the robustness of clinical results. We aim to determine how including or excluding double-zero-event studies affects FI derivations in meta-analyses.

Methods: We conducted an illustrative case study to demonstrate how double-zero-event studies can impact FI derivations. Additionally, we performed a large-scale analysis of 12,184 Cochrane meta-analyses involving zero-event studies to assess the prevalence and effect of double-zero-event studies on FI calculations.

Results: Our analysis revealed that FI derivations in 6608 (54.2%) of these meta-analyses involved double-zero-event studies. Excluding double-zero-event studies could lead to artificially inflated FI values, potentially misrepresenting the results as more robust than they are.

Conclusions: We advocate for retaining double-zero-event studies in meta-analyses and emphasise the importance of carefully considering their role in FI assessments. Including these studies ensures a more accurate evaluation of the robustness of clinical results in evidence synthesis.