The impact of database restriction on pharmacovigilance signal detection of selected cancer therapies

Background: The aim of this study was to investigate whether database restriction can improve oncology drug pharmacovigilance signal detection performance.

Methods: We used spontaneous adverse event (AE) reports in the United States (US) Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) database. Positive control (PC) drug medical concept (DMC) pairs were selected from safety information not included in the product's first label but subsequently added as label changes. These medical concepts (MCs) were mapped to the Medical Dictionary for Regulatory Activities (MedDRA) preferred terms (PTs) used in FAERS to code AEs. Negative controls (NC) were MCs with circumscribed PTs not included in the corresponding US package insert (USPI). We calculated shrinkage-adjusted observed-to-expected (O/E) reporting frequencies for the aforementioned drug-PT pairs. We also formulated an adjudication framework to calculate performance at the MC level. Performance metrics [sensitivity, specificity, positive and negative predictive value (PPV, NPV), signal/noise (S/N), F and Matthews correlation coefficient (MCC)] were calculated for each analysis and compared.

Results: The PC reference set consisted of 11 drugs, 487 PTs, 27 MCs, 37 drug-MC combinations and 638 drug-event combinations (DECs). The NC reference set consisted of 11 drugs, 9 PTs, 5 MCs, 40 drug-MC combinations and 67 DECs. Most drug-event pairs were not highlighted by either analysis. A small percentage of signals of disproportionate reporting were lost, more noise than signal, with no gains. Specificity and PPV improved whereas sensitivity, NPV, F and MCC decreased, but all changes were small relative to the decrease in sensitivity. The overall S/N improved.

Conclusion: This oncology drug restricted analysis improved the S/N ratio, removing proportionately more noise than signal, but with significant credible signal loss. Without broader experience and a calculus of costs and utilities of correct versus incorrect classifications in oncology pharmacovigilance such restricted analyses should be optional rather than a default analysis.