Test negative design for vaccine effectiveness estimation in the context of the COVID-19 pandemic: A systematic methodology review

Background: During the height of the global COVID-19 pandemic, the test-negative design (TND) was extensively used in many countries to evaluate COVID-19 vaccine effectiveness (VE). Typically, the TND involves the recruitment of care-seeking individuals who meet a common clinical case definition. All participants are then tested for an infection of interest.

Objectives: To review and describe the variation in TND methodology, and disclosure of potential biases, as applied to the evaluation of COVID-19 VE during the early vaccination phase of the pandemic.

Methods: We conducted a systematic review by searching four biomedical databases using defined keywords to identify peer-reviewed articles published between January 1, 2020, and January 25, 2022. We included only original articles that employed a TND to estimate VE of COVID-19 vaccines in which cases and controls were evaluated based on SARS-CoV-2 laboratory test results.

Results: We identified 96 studies, 35 of which met the defined criteria. Most studies were from North America (16 studies) and targeted the general population (28 studies). Outcome case definitions were based primarily on COVID-19-like symptoms; however, several papers did not consider or specify symptoms. Cases and controls had the same inclusion criteria in only half of the studies. Most studies relied upon administrative or hospital databases assembled for a different (non-evaluation) clinical purpose. Potential unmeasured confounding (20 studies), misclassification of current SARS-CoV-2 infection (16 studies) and selection bias (10 studies) were disclosed as limitations by some studies.

Conclusion: We observed potentially meaningful deviations from the validated design in the application of the TND during the COVID-19 pandemic.