Over the last two decades, hundreds of new psychoactive substances (NPSs), also known as "designer drugs", have emerged on the illicit drug market. The toxic and potentially fatal effects of these compounds oblige laboratories around the world to screen for NPS in seized materials and biological samples, commonly using high-resolution mass spectrometry. However, unambiguous identification of a NPS by mass spectrometry requires comparison to data from analytical reference materials, acquired on the same instrument. The sheer number of NPSs that are available on the illicit market, and the pace at which new compounds are introduced, means that forensic laboratories must make difficult decisions about which reference materials to acquire. Here, we asked whether retrospective suspect screening of population-scale mass spectrometry data could provide a data-driven platform to prioritize emerging NPSs for assay development. We curated a suspect database of precursor and diagnostic fragment ion masses for 83 emerging NPSs and used this database to retrospectively screen mass spectrometry data from 12,727 urine drug screens from one Canadian province. We developed integrative computational strategies to prioritize the most reliable identifications and tracked the frequency of these identifications over a 3 year study period between August 2019 and August 2022. The resulting data were used to guide the acquisition of new reference materials, which were in turn used to validate a subset of the retrospective identifications. Last, we took advantage of matching clinical reports for all 12,727 samples to systematically benchmark the accuracy of our retrospective data analysis approach. Our work opens up new avenues to enable the rapid detection of emerging illicit drugs through large-scale reanalysis of mass spectrometry data.