Systematic reviews supporting practice guideline recommendations lack protection against bias

Objective: To evaluate the quality of systematic reviews (SRs) affecting clinical practice in endocrinology.

Study design and setting: We identified all SRs cited in The Endocrine Society's Clinical Practice Guidelines published between 2006 and January 2012. We evaluated the methodological and reporting quality of the SRs in duplicate using the Assessment of Multiple Systematic Reviews (AMSTAR) tool. We also noted if the guidelines recommendations that are clearly supported by SRs acknowledged their quality.

Results: During the 5-year period of study, endocrine guidelines cited 69 SRs. These SRs had a mean AMSTAR score of 6.4 (standard deviation, 2.5) of a maximum score of 11, with scores improving over time. SRs of randomized trials had higher AMSTAR scores than those of observational studies. Low-quality SRs (methodological AMSTAR score 1 or 2 of 5, n = 24, 35%) were cited in 24 different recommendations and were the main evidentiary support for five recommendations, of which only one acknowledged the quality of SRs.

Conclusion: Few recommendations in endocrinology are supported by SRs. The quality of SRs is suboptimal and is not acknowledged by guideline developers.