Early studies reported extreme findings with large variability: a meta-epidemiologic study in the field of endocrinology

Objectives: To evaluate the presence of extreme findings and fluctuation in effect size in endocrinology.

Study design and settings: We systematically identified all meta-analyses published in 2014 in the field of endocrinology. Within each meta-analysis, the effect size of the primary binary outcome was compared across studies according to their order of publication. We pooled studies using the DerSimonian and Laird random-effects method. Heterogeneity was evaluated using the I(2) and tau(2).

Results: Twelve percent of the included 100 meta-analyses reported the largest effect size in the very first published study. The largest effect size occurred in the first 2 earliest studies in 31% of meta-analyses. When the effect size was the largest in the first published study, it was three times larger than the final pooled effect (ratio of rates, 3.26; 95% confidence interval: 1.80, 5.90). The largest heterogeneity measured by I(2) was observed in 18% of the included meta-analyses when combining the first 2 studies or 17% when combing the first 3 studies.

Conclusions: In endocrinology, early studies reported extreme findings with large variability. This behavior of the evidence needs to be taken into account when used to formulate clinical policies.