Flying Blind: How Thorough are IRBs when Assessing Scientific Value?

Background: Institutional Review Boards (IRBs) in the United States play a crucial role in ensuring the ethical conduct of clinical trials, including assessing the scientific merit of studies to justify the risks to participants. However, prior research suggests that many IRBs do not systematically evaluate scientific merit, raising concerns about the approval of low-quality trials.

Objective: To investigate whether IRBs provide adequate guidance on assessing scientific merit in their Standard Operating Procedures (SOPs) and other relevant materials.

Design: A systematic pilot investigation of IRB SOPs and related guidance documents from a sample of U.S.-based non-profit institutions.

Participants: IRB materials from 35 U.S.-based non-profit institutions selected from the FDA's Bioresearch Monitoring Information System database, representing 39.9% of submissions between 2018 and 2021. Additionally, materials from one U.S.-based for-profit IRB were included.

Interventions: Not applicable.

Main measures: The presence of guidance on 15 dimensions of scientific merit, clustered into four PICO (Population, Intervention, Comparator, Outcome) categories, was assessed by reviewing IRB SOPs and related documents. Evidence of guidance was determined by mention of keywords related to each dimension.

Key results: Most IRB materials mentioned basic study elements such as study design (99%), subject recruitment (90%), and intervention justification (97%). However, critical aspects related to study quality, such as bias reduction (53%) and outcome measurement tools (57%), were less frequently mentioned. The least represented dimension was confounder control (10%).

Conclusions: IRB guidance materials vary in their coverage of scientific merit dimensions, with significant gaps in areas critical for assessing study quality. Strengthening guidance materials by including comprehensive instructions for all 15 dimensions could improve IRB assessments of scientific merit, thereby enhancing the ethical oversight of clinical trials.