Sensitivity and identification quantification by a relative latent model complexity perturbation in Bayesian meta-analysis

In recent years, Bayesian meta-analysis expressed by a normal-normal hierarchical model (NNHM) has been widely used for combining evidence from multiple studies. Data provided for the NNHM are frequently based on a small number of studies and on uncertain within-study standard deviation values. Despite the widespread use of Bayesian NNHM, it has always been unclear to what extent the posterior inference is impacted by the heterogeneity prior (sensitivity $S$ ) and by the uncertainty in the within-study standard deviation values (identification $I$ ). Thus, to answer this question, we developed a unified method to simultaneously quantify both sensitivity and identification ( $S$ - $I$ ) for all model parameters in a Bayesian NNHM, based on derivatives of the Bhattacharyya coefficient with respect to relative latent model complexity (RLMC) perturbations. Three case studies exemplify the applicability of the method proposed: historical data for a conventional therapy, data from which one large study is first included and then excluded, and two subgroup meta-analyses specified by their randomization status. We analyzed six scenarios, crossing three RLMC targets with two heterogeneity priors (half-normal, half-Cauchy). The results show that $S$ - $I$ explicitly reveals which parameters are affected by the heterogeneity prior and by the uncertainty in the within-study standard deviation values. In addition, we compare the impact of both heterogeneity priors and quantify how $S$ - $I$ values are affected by omitting one large study and by the randomization status. Finally, the range of applicability of $S$ - $I$ is extended to Bayesian NtHM. A dedicated R package facilitates automatic $S$ - $I$ quantification in applied Bayesian meta-analyses.