Evaluating agreement between individual nutrition randomised controlled trials and cohort studies - a meta-epidemiological study

Background: In nutrition research, randomised controlled trials (RCTs) and cohort studies provide complementary evidence. This meta-epidemiological study aims to evaluate the agreement of effect estimates from individual nutrition RCTs and cohort studies investigating a highly similar research question and to investigate determinants of disagreement.

Methods: MEDLINE, Epistemonikos, and the Cochrane Database of Systematic Reviews were searched from January 2010 to September 2021. We matched individual RCTs to cohort studies based on population, intervention/exposure, comparator, and outcome (PI/ECO) characteristics. Two reviewers independently extracted study characteristics and effect estimates and rated the risk of bias using RoB2 and ROBINS-E. Agreement of matched RCTs/cohort studies was analysed by pooling ratio of risk ratios (RRR) and difference of (standardised) mean differences (DSMD).

Results: We included 64 RCT/cohort study pairs with 4,136,837 participants. Regarding PI/ECO similarity, 20.3% pairs were "more or less identical", 71.9% "similar but not identical" and 7.8% "broadly similar". Most RCTs were classified as "low risk of bias" (26.6%) or with "some concerns" (65.6%); cohort studies were mostly rated with "some concerns" (46.6%) or "high risk of bias" (47.9%), driven by inadequate control of important confounding factors. Effect estimates across RCTs and cohort studies were in high agreement (RRR 1.00 (95% CI 0.91-1.10, n = 54); and DSMD - 0.26 (95% CI - 0.87-0.35, n = 7)). In meta-regression analyses exploring determinants of disagreements, risk-of-bias judgements tend to have had more influence on the effect estimate than "PI/ECO similarity" degree.

Conclusions: Effect estimates of nutrition RCTs and cohort studies were generally similar. Careful consideration and evaluation of PI/ECO characteristics and risk of bias is crucial for a trustworthy utilisation of evidence from RCTs and cohort studies.