A systematic survey of 200 systematic reviews with network meta-analysis (published 2020-2021) reveals that few reviews report structured evidence summaries

Per Olav Løvsletten; Xiaoqin Wang; Tyler Pitre; Marte Ødegaard; Areti Angeliki Veroniki; Carole Lunny; Andrea C Tricco; Thomas Agoritsas; Per Olav Vandvik

doi:10.1016/j.jclinepi.2024.111445

A systematic survey of 200 systematic reviews with network meta-analysis (published 2020-2021) reveals that few reviews report structured evidence summaries

J Clin Epidemiol. 2024 Sep:173:111445. doi: 10.1016/j.jclinepi.2024.111445. Epub 2024 Jun 26.

Authors

Per Olav Løvsletten¹, Xiaoqin Wang², Tyler Pitre³, Marte Ødegaard⁴, Areti Angeliki Veroniki⁵, Carole Lunny⁶, Andrea C Tricco⁷, Thomas Agoritsas⁸, Per Olav Vandvik⁹

Affiliations

¹ Department of Medicine, Lovisenberg Diaconal Hospital, Oslo, Norway; Faculty of Medicine, Institute of Health and Society, University of Oslo, Oslo, Norway. Electronic address: [email protected].
² Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada; University of Ottawa Heart Institute Research Corporation, Ottawa, Ontario, Canada.
³ Department of Respirology, University of Toronto, Toronto, Ontario, Canada.
⁴ Library of Medicine and Science, University of Oslo Library, University of Oslo, Oslo, Norway.
⁵ Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Ontario, Toronto, Canada; Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, Ontario, Canada.
⁶ Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Ontario, Toronto, Canada; Cochrane Hypertension Group and the Therapeutics Initiative, University of British Columbia, Vancouver, Canada.
⁷ Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Ontario, Toronto, Canada; Queen's Collaboration for Health Care Quality: A JBI Centre of Excellence, Queen's School of Nursing. Queen's University, Kingston, Ontario, Canada; Epidemiology Division and Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada.
⁸ Department of Internal Medicine, University Hospitals of Geneva, Geneva, Switzerland; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; MAGIC Evidence Ecosystem Foundation, Oslo, Norway.
⁹ Department of Medicine, Lovisenberg Diaconal Hospital, Oslo, Norway; Faculty of Medicine, Institute of Health and Society, University of Oslo, Oslo, Norway; MAGIC Evidence Ecosystem Foundation, Oslo, Norway.

PMID: 38942177
DOI: 10.1016/j.jclinepi.2024.111445

Abstract

Objectives: To map whether and how systematic reviews (SRs) with network meta-analysis (NMA) use presentation formats to report (a) structured evidence summaries - here defined as reporting of effects estimates in absolute effects with certainty ratings and with a method to rate interventions across one or more outcome(s) - and (b) NMA results in general.

Study design and setting: We conducted a systematic survey, searching MEDLINE (Ovid) for SRs with NMA published between January 1, 2020, and December 31, 2021. We planned to include a random sample of publications, with predefined mechanisms in place for saturation, and included SRs that met prespecified quality criteria and extracted data on presentation formats that reported: (a) estimates of effects, (b) certainty of the evidence, or (c) rating of interventions.

Results: The 200 eligible SRs, from 158 unique Journals, utilized 1133 presentation formats. We found structured evidence summaries in 10 publications (5.0%), with 3 (1.5%) reporting structured evidence summaries across all outcomes, including benefits and harms. Sixteen of the 133 SRs (11.7%) reporting dichotomous outcomes included estimates of absolute effects. Seventy-six SRs (38.0%) reported both benefits and harms and 26 SRs (13.0%) reported certainty ratings in presentation formats, 20 (76.9%) used Grading of Recommendations Assessment, Development and Evaluation and 6 (23.1%) used Confidence In Network Meta-analysis. Surface Under the Cumulative Ranking Curve was the most common method to rate interventions (69 SRs, 34.5%). NMA results were most often reported using forest plots (108 SRs, 54.0%) and league tables (93 SRs, 46.5%).

Conclusion: Most SRs with NMA do not report structured evidence summaries and only rarely do such summaries include reporting of both benefits and harms; those that do offer effective user-friendly communication and provide models for optimal NMA presentation practice.

Keywords: Certainty of the evidence; Evidence summaries; Network meta-analysis; Presentation formats; Summary of findings tables; Systematic reviews.

MeSH terms

Evidence-Based Medicine / methods
Evidence-Based Medicine / standards
Humans
Network Meta-Analysis*
Systematic Reviews as Topic* / methods
Systematic Reviews as Topic* / standards