Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review

Background: There is an increasing number of meta-analyses including data from non-randomized studies for therapeutic evaluation. We aimed to systematically assess the methods used in meta-analyses including non-randomized studies evaluating therapeutic interventions.

Methods: For this methodological review, we searched MEDLINE via PubMed, from January 1, 2013 to December 31, 2013 for meta-analyses including at least one non-randomized study evaluating therapeutic interventions. Etiological assessments and meta-analyses with no comparison group were excluded. Two reviewers independently assessed the general characteristics and key methodological components of the systematic review process and meta-analysis methods.

Results: One hundred eighty eight meta-analyses were selected: 119 included both randomized controlled trials (RCTs) and non-randomized studies of interventions (NRSI) and 69 only NRSI. Half of the meta-analyses (n = 92, 49%) evaluated non-pharmacological interventions. "Grey literature" was searched for 72 meta-analyses (38%). An assessment of methodological quality or risk of bias was reported in 135 meta-analyses (72%) but this assessment considered the risk of confounding bias in only 33 meta-analyses (18%). In 130 meta-analyses (69%), the design of each NRSI was not clearly specified. In 131 (70%), whether crude or adjusted estimates of treatment effect for NRSI were combined was unclear or not reported. Heterogeneity across studies was assessed in 182 meta-analyses (97%) and further explored in 157 (84%). Reporting bias was assessed in 127 (68%).

Conclusions: Some key methodological components of the systematic review process-search for grey literature, description of the type of NRSI included, assessment of risk of confounding bias and reporting of whether crude or adjusted estimates were combined-are not adequately carried out or reported in meta-analyses including NRSI.