[Development of a Ranking Tool for Scientificity, Transparency and Applicability of Clinical Practice Guidelines]

N Yang; W Zhao; Y Pan; X Z Lyu; X Y Hao; W A Qi; L Du; E M Liu; T Chen; W S Zhang; C F Zhang; G N Zhu; Q M Wang; W B Meng; Y B Liang; Y H Jin; W Wang; D Xing; J H Tian; B Ma; X H Wang; X P Song; L Ge; K H Yang; X Q Liu; J M Wei; Yaolong Chen

doi:10.3760/cma.j.cn112137-20220219-00340

[Development of a Ranking Tool for Scientificity, Transparency and Applicability of Clinical Practice Guidelines]

Zhonghua Yi Xue Za Zhi. 2022 May 31:102:1-10. doi: 10.3760/cma.j.cn112137-20220219-00340. Online ahead of print.

[Article in Chinese]

Authors

N Yang¹, W Zhao², Y Pan³, X Z Lyu⁴, X Y Hao⁵, W A Qi⁶, L Du⁷, E M Liu⁸, T Chen⁹, W S Zhang¹⁰, C F Zhang¹¹, G N Zhu¹², Q M Wang¹³, W B Meng¹⁴, Y B Liang¹⁵, Y H Jin¹⁶, W Wang¹⁷, D Xing¹⁸, J H Tian¹, B Ma¹, X H Wang¹⁹, X P Song¹⁹, L Ge¹⁹, K H Yang¹, X Q Liu²⁰, J M Wei²¹, Yaolong Chen²²

Affiliations

¹ Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China.
² General Editorial Office, Chinese Medical Association Publishing House, Beijing 100052, China.
³ Marketing and Sales Department, Chinese Medical Association Publishing House, Beijing 100052, China.
⁴ Editorial Department, Chinese Medical Journal, Chinese Medical Association Publishing House, Beijing 100052, China.
⁵ Editorial Department, Chinese Medical Journal (English Edition), Chinese Medical Association Publishing House, Beijing 100052, China.
⁶ Editorial Department, British Medical Journal (Chinese Edition), Chinese Medical Association Publishing House, Beijing 100052, China.
⁷ Evidence-Based Medicine Center, West China School of Medicine, West China Hospital, Sichuan University, Chengdu 610041.
⁸ Department of Respiratory Medicine, Children's Hospital of Chongqing Medical University, Chongqing 400014.
⁹ Department of Hematology, Huashan Hospital, Fudan University, Shanghai 200040, China.
¹⁰ Department of Obstetrics and Gynecology, Xiangya Hospital, Central South University, Changsha 410008, China.
¹¹ Children's Hospital of Fudan University, Shanghai 201102, China.
¹² Department of Dermatology, Xijing Hospital, Xi'an 710032, China.
¹³ Department of Medical Oncology, Henan Cancer Hospital, Zhengzhou 450008, China.
¹⁴ Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou 730000, China.
¹⁵ Department of Ophthalmology, The Eye Hospital of Wenzhou Medical University, Wenzhou 325027, China.
¹⁶ Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430072, China.
¹⁷ Department of Urology, The Fifth People's Hospital of Shanghai, Fudan University, Shanghai 200240, China.
¹⁸ Department of Trauma and Orthopaedics, Peking University People's Hospital, Beijing 100044, China.
¹⁹ School of Public Health, Lanzhou University, Lanzhou 730000.
²⁰ Department of Infectious Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union & Peking Union Medical College, Beijing 100730.
²¹ Chinese Medical Association Publishing House, Beijing 100052, China.
²² Research Unit of Evidence-Based Evaluation and Guidelines (2021RU017), Chinese Academy of Medical Sciences, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China Guidelines and Standards Research Center, Chinese Medical Association Publishing House, Beijing 100052, China.

PMID: 35701091
DOI: 10.3760/cma.j.cn112137-20220219-00340

Abstract
in English, Chinese

Objective: To address the limitations of existing methods and tools for evaluating clinical practice guidelines, we aimed to develop a comprehensive instrument focusing on the three main dimensions of guideline development: scientificity, transparency, applicability. We will use it to rank the guidelines according to the scores. We abbreviated it as STAR, and its reliability, validity and usability were also tested. Methods: A multidisciplinary expert working group was set up, including methodologists, statisticians, journal editors, medical professionals, and others. Scoping review, Delphi methods and hierarchical analysis were used to determine the final checklist of STAR. Results: The new instrument contained 11 domains and 39 items. Intrinsic reliability of each domain was indicated by Cronbach's α coefficient, with a average value of 0.646. The Cohen's kappa coefficients for methodological evaluators and clinical evaluators were 0.783 and 0.618. The overall content validity index was 0.905. The R² for the criterion validity analysis was 0.76. The average score for usability of the items was 4.6, and the mean time spent to evaluate each guideline was 20 minutes. Conclusion: The instrument has good reliability, validity and evaluating efficiency, and can be used for evaluating and ranking guidelines more comprehensively.

目的： 针对现有临床实践指南评价方法和工具的不足，研发针对临床实践指南科学性（Scientificity）、透明性（Transparency）和适用性（Applicability）的评级（Rankings）工具，缩写为STAR，并对其信效度进行验证和对易用性进行评估。 方法： 成立包含指南方法学家、统计学家、期刊编辑、医务人员等多学科的专家工作组，基于概况性评价、德尔菲法和层次分析法确定评级领域条目和相应分值，通过共识会议确定最终工具清单，并对工具进行信度、效度和易用性验证。 结果： 形成11个领域39个条目和相应分值的综合性评价工具。工具内在信度分析显示各领域平均Cronbach′s α系数值0.646；方法学评级人员和临床评级人员之间的信度Cohen′s kappa系数分别为0.783和0.618；条目整体内容效度指数0.905；效标效度分析决定系数R²为0.76；条目的易用性平均得分4.6，评价1部指南中位用时20分钟。 结论： STAR具有良好的信效度和评级效率，可针对中国指南的科学性、透明性和适用性进行综合评级。.

Publication types

English Abstract

Abstract in English, Chinese

Publication types

Abstract
in English, Chinese