Assessing the article screening efficiency of artificial intelligence for Systematic Reviews

Objectives: Artificial intelligence (AI) tools utilizing machine learning (ML) have gained increasing utility in medicine and academia as a means of enhancing efficiency. ASReview is one such AI program designed to streamline the systematic review process through the automated prioritization of relevant articles for screening. This study examined the screening efficiency of ASReview when conducting systematic reviews and the potential factors that could influence its efficiency.

Methods: Six distinct topics within the field of periodontics were searched in PubMed and Web of Science to obtain articles for screening within ASReview. Through a "training" process, relevant and irrelevant articles were manually incorporated to develop "prior knowledge" and facilitate ML optimization. Screening was then conducted following ASReview's algorithmically-generated relevance rankings. Screening efficiency was evaluated based on the normalized number of articles not requiring detailed review and on the total time expenditure.

Results: Across the six topics, an average of 60.2 % of articles did not warrant extensive screening, given that all relevant articles were discovered within the first 39.8 % of publication reviewed. No significant variations in efficiencies were observed with differing methods of assembling prior knowledge articles or via modifications in article ratios and numbers.

Conclusions: On average, ASReview conferred a 60.2 % improvement in screening efficiency, largely attributed to its dynamic ML capabilities. While advanced technologies like ASReview promise enhanced efficiencies, the accurate human discernment of article relevancy and quality remains indispensable when training these AI tools.

Clinical significance: Using ASReview has the potential to save approximately 60 % of time and effort required for screening articles.