Validity of studies suggesting preoperative chemotherapy for resectable thoracic esophageal cancer: A critical appraisal of randomized trials

Background: In 2015, Kidane published a Cochrane review and meta-analysis to summarise the impact of preoperative chemotherapy versus surgery alone on survival for resectable thoracic esophageal cancer. The authors concluded that preoperative chemotherapy improved overall survival (OS).

Aim: The aim of this article was to assess the validity of the three most powerful studies included in the Cochrane review and the meta-analysis supporting the advantage of preoperative chemotherapy and to investigate the impact of an exclusion of these three studies on the result of the meta-analysis.

Methods: OS was selected as the endpoint of interest. Among the ten included papers which analysed this endpoint, we identified the three publications with the highest weights influencing the final result. The validity of these papers was analysed using the CONSORT checklist for randomized controlled trials. We performed a new meta-analysis without the three studies to assess their impact on the general result of the original meta-analysis.

Results: The three analysed studies revealed several inconsistencies. Inappropriate answers were found in up to one third of the items of the CONSORT checklist. Missing information about sample-size calculation and power, unclear or inadequate randomisation, and missing blinded set-up were the most common findings. When the three criticized studies were excluded in the meta-analysis, preoperative chemotherapy showed no benefit in OS.

Conclusion: The three most powerful publications in the Cochrane review show substantial deficits. After the exclusion of these studies from the meta-analysis, preoperative chemotherapy does not seem to result in an advantage in survival. We suggest a more critical appraisal regarding the validity of single studies.