How Many Monte Carlo Samples Are Needed for Probabilistic Cost-Effectiveness Analyses?

Objectives: Probabilistic sensitivity analysis (PSA) is conducted to account for the uncertainty in cost and effect of decision options under consideration. PSA involves obtaining a large sample of input parameter values (N) to estimate the expected cost and effect of each alternative in the presence of parameter uncertainty. When the analysis involves using stochastic models (eg, individual-level models), the model is further replicated P times for each sampled parameter set. We study how N and P should be determined.

Methods: We show that PSA could be structured such that P can be an arbitrary number (say, P=1). To determine N, we derive a formula based on Chebyshev's inequality such that the error in estimating the incremental cost-effectiveness ratio (ICER) of alternatives (or equivalently, the willingness-to-pay value at which the optimal decision option changes) is within a desired level of accuracy. We described 2 methods to confirm, visually and quantitatively, that the N informed by this method results in ICER estimates within the specified level of accuracy.

Results: When N is arbitrarily selected, the estimated ICERs could be substantially different from the true ICER (even as P increases), which could lead to misleading conclusions. Using a simple resource allocation model, we demonstrate that the proposed approach can minimize the potential for this error.

Conclusions: The number of parameter samples in probabilistic cost-effectiveness analyses should not be arbitrarily selected. We describe 3 methods to ensure that enough parameter samples are used in probabilistic cost-effectiveness analyses.