Testing the External Validity of a Discrete Choice Experiment Method: An Application to Latent Tuberculosis Infection Treatment

Objectives: To explore the external validity and predictive power of stated preferences obtained from a discrete choice experiment (DCE) by comparing the predicted behavior of respondents to their actual choices at an individual level.

Methods: A DCE was performed in patients before being offered treatment for latent tuberculosis infection. A mixed logit model was estimated using hierarchical Bayes. The individual-specific preference coefficients were used to calculate the expected probability of choosing the treatment by each patient. The predicted choice using this probability was compared with their actual decision. We used a receiver-operating characteristic curve and different thresholds to convert probabilities into the predicted choices. The comparability of different distributions for the random parameters was also examined.

Results: Our results identified significant heterogeneity in preferences for all attributes among respondents. The best model correctly predicted actual treatment decisions for 83% of the participants. The results from using different thresholds and a receiver-operating characteristic curve also confirmed the compatibility between predicted and actual choices. We showed that individual-specific coefficients reflected respondents' actual choices more closely compared with the aggregate-level estimates.

Conclusions: The results of this study provided support for the external validity of DCEs on the basis of their power to predict actual behavior in this setting. Future investigations are, however, required to establish the external validity of DCEs in different settings.