AGREE II appraisals of clinical practice guidelines in rehabilitation showed poor reporting and moderate variability in quality ratings when users apply different cuff-offs: a methodological study

Objectives: To analyze the reporting characteristics of Appraisal of Guidelines Research and Evaluation (AGREE) II appraisals in rehabilitation and explore how much quality ratings of Clinical Practice Guidelines (CPGs) vary applying different cut-offs.

Study design and setting: We conducted a methodological study re-analyzing data of an overview of AGREE II CPG appraisals in rehabilitation. Reporting characteristics of appraisals and methods used for quality rating were abstracted. We applied the most frequent cut-offs retrieved on all CPG sample to explore changes in quality ratings (i.e., high/low).

Results: We included 40 appraisals (n = 544 CPGs).The AGREE II overall assessment 1 (overall CPG quality) was reported in 26 appraisals (65%) and the overall assessment 2 (recommendation for use) in 17 (42.5%). Twenty-five appraisals (62.5%) reported the use of cut-offs based on domains and/or overall assessments. Application of the most reported cut-offs led to variability in quality ratings in 26% of the CPGs, of which 92% CPGs shifted their rating from low to high-quality and 8% shifted from high to low-quality.

Conclusion: Rehabilitation stakeholders should take care to select the highest quality CPG in view of the poor reporting of AGREE II overall assessment 1 and 2 and moderate variability of quality ratings.