Exploring Completeness of Adverse Event Reports as a Tool for Signal Detection in Pharmacovigilance

Background: Completeness of adverse event (AE) reports is an important component of quality for good pharmacovigilance practices. We aimed to evaluate the impact of incorporating a measure of completeness of AE reports on quantitative signal detection.

Methods: An internal safety database from a global pharmaceutical company was used in the analysis. vigiGrade, an index score of completeness, was derived for each AE report. Data from various patient support programs (PSPs) were categorized based on average vigiGrade score per PSP. Performance of signal detection was compared between: (1) weighting and not weighting by vigiGrade score; and, (2) well documented and poorly documented PSPs using sensitivity, specificity, area under the receiver operating characteristics curve (AUC) and time-to-signal detection.

Results: The ability to detect signals did not differ significantly when weighting by vigiGrade score [sensitivity (50% vs. 45%, p = 1), specificity (82.8% vs. 82.8%, p = 1), AUC (0.66 vs. 0.63, p = 0.051) or time-to-signal detection (HR 0.81, p = 0.63)] compared to not weighting. Well documented PSPs were better at detecting signals than poorly documented PSPs (AUC 0.66 vs. 0.52; p = 0.041) but time-to-signal detection did not differ significantly (HR 1.54, p = 0.42).

Conclusion: Completeness of AE reports did not significantly impact the ability to detect signals when weighting by vigiGrade score or restricting the database based on the level of completeness. While the vigiGrade helps provide quality assessments of AE reports and prioritize cases for review, our findings indicate the tool might not be useful for quantitative signal detection when used by itself.