A systematic review of the use of unannounced standardized patients (USPs) in clinical settings: A call for more detailed quality and fidelity descriptions and expansion to new areas

Background: Unannounced standardized patients (USPs) have long been used to measure clinical performance in situ. These incognito actors capture data on clinician skills during an encounter, as well as patient experience more broadly. A robust USP program requires extensive preparation and standardization efforts. Given the widespread expansion of USPs for education, research, and improvement efforts, we conducted a systematic review with the goal of capturing the breadth of uses of USPs across settings, along with the standardization measures employed across studies.

Methods: In collaboration with a medical librarian, we conducted systematic searches across six databases. Two independent researchers screened each report for inclusion. Three coders extracted and reviewed study characteristics and data from the studies deemed eligible for inclusion. We extracted data on: target population, setting, and assessed skills. We also captured the reliability and fidelity measures described in each study, including USP detection, USP training methods, and assessment measures.

Results: 128 articles were included. Individual clinicians were the most frequently targeted population (n = 114, 89 %). Common clinician roles included physicians (n = 92, 72 %) and pharmacists (n = 12, 9 %). The collective care team was the target in two studies (2 %), and systems and larger healthcare facilities were targeted in only 1 (1 %) and 13 (10 %) studies, respectively. Studies were primarily conducted in ambulatory settings (n = 118, 92 %). History gathering (n = 76, 59 %), communication (n = 55, 43 %), counseling (n = 51, 40 %), and patient education (n = 49, 38 %) were commonly assessed, as were correct diagnosis (n = 34, 27 %), appropriate ordering of labs/tests (n = 30, 23 %), referrals (n = 35, 27 %), and prescriptions (n = 36, 28 %). USP detection reporting was variable across studies; however, no detection information was provided for 48 studies. 62 % of articles reported incorporating a measure of reliability or fidelity into their study, while the remainder either failed to provide adequate information on use of these measures.

Conclusions: We explored USP use across settings and describe the scope and limitations of the literature. USPs capture a range of data domains but a lack uniform report of reliability measures can potentially undermine findings. Future studies should incorporate and uniformly report out on detection, training, and assessment.