Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysis

Background: Discrete choice experiments (DCEs) are increasingly used to inform the design of health products and services. It is essential to understand the extent to which DCEs provide reliable predictions outside of experimental settings in real-world decision-making situations. We aimed to compare the prediction accuracy of stated preferences with real-world choices, as modelled from DCE data.

Methods: We searched six databases for health-related studies that used DCE to assess external validity and reported on predicted versus real-world choices, up to July 2024. A generalised linear mixed model was used for a meta-analysis to jointly pool the sensitivity and specificity. Heterogeneity was assessed using the I ² statistic, and sources of heterogeneity using meta-regression. This study is registered with PROSPERO (CRD42023451545).

Findings: We identified 14 relevant studies, of which 10 were included in the meta-analysis. Most studies were conducted in high-income countries (11/14, 79%) from the European region (9/14, 64%) and analysed using mixed logit models (5/14, 36%). Pooled sensitivity and specificity estimates were 89% (95% CI:77-95, I ² = 97%) and 52% (95% CI:32-72, I ² = 95%), respectively. The area under the SROC curve (AUC) was 0.81 (95% CI:0.77-0.84). Our meta-regression found that DCEs for prevention-related choices had higher sensitivity than treatment-related choices. DCEs conducted under clinical settings and analysed using the heteroskedastic multinomial logit model, incorporating systematic preference heterogeneity and random opt-out utility, had higher specificity than non-clinical settings and alternative models.

Interpretation: DCEs are valuable for capturing health-related preferences and possess reasonable external validity to predict health-related behaviours, particularly for opt-in choices. Contextual factors (e.g., type of intervention, study setting, analysis method) influenced the predictive accuracy.

Funding: JJO is supported by an Australian National Health and Medical Research Council Emerging Leadership Investigator Grant (GNT1193955). EBG is supported by the Dutch Research Council (NWO-Talent-Scheme-Vidi-Grant No, 09150171910002). YZ is supported by an Australian Government Research Training Program (RTP) scholarship.