Methodological standards in the design and reporting of pilot and feasibility studies in emergency medicine literature: a systematic review

Objective: Pilot and feasibility studies are intended to ensure that subsequent randomised controlled trials (RCTs) are feasible, economical and rigorous, especially in a challenging research environment such as emergency medicine (EM). We aimed to evaluate the methodological quality in conducting and reporting randomised pilot and feasibility studies in the EM literature and propose recommendations to improve their quality.

Design: Methodological systematic review.

Data sources and eligibility: We searched MEDLINE and Embase (2018-29 September 2023) for pilot or feasibility RCTs published as full texts in the five top-ranked and other first-quartile EM journals according to Scimago.

Data extraction and analysis: We assessed their methodological features and reporting quality primarily based on the Consolidated Standards of Reporting Trials (CONSORT) extension.

Results: A total of 24 randomised trials identified as pilot (n=13), feasibility (n=3) or both (n=8) were included. At least one feasibility outcome was assessed in 9 trials (feasibility trials), while 15 others only focused on treatment efficacy (efficacy trials). Only three (12.5%) studies progressed to the main trials. Among 12 feasibility trials, 55.6% reported their outcomes with uncertainty estimates, and 33.3% had clear progression criteria. Efficacy trials tended to draw clinical implications on their results. Studies from the five top-ranked journals had better methodological and reporting quality than those from other first-quartile journals.

Conclusion: Main methodological concerns for pilot and feasibility studies in first-quartile EM literature include misconceptions, misuses and suboptimal design and reporting quality. These issues were more prominent in lower-ranked first-quartile journals. Our findings highlight the need for resources and training for researchers, journal editors and peer reviewers on the value, objectives and appropriate conduct of pilot and feasibility studies. The conceptual framework and standardised methodological components should be emphasised. EM journals should reinforce the reporting standards and support their publication. These actions can lead to more methodologically rigorous pilot and feasibility studies in EM.

Prospero registration number: CRD42023468437.