Artificial intelligence outperforms human students in conducting neurosurgical audits

Objectives: Neurosurgical audits are an important part of improving the safety, efficiency and quality of care but require considerable resources, time, and funding. To that end, the advent of the Artificial Intelligence-based algorithms offered a novel, more economically viable solution. The aim of the study was to evaluate whether the algorithm can indeed outperform humans in that task.

Patients & methods: Forty-six human students were invited to inspect the clinical notes of 45 medical outliers on a neurosurgical ward. The aim of the task was to produce a report containing a quantitative analysis of the scale of the problem (e.g. time to discharge) and a qualitative list of suggestions on how to improve the patient flow, quality of care, and healthcare costs. The Artificial Intelligence-based Frideswide algorithm (FwA) was used to analyse the same dataset.

Results: The FwA produced 44 recommendations whilst human students reported an average of 3.89. The mean time to deliver the final report was 5.80 s for the FwA and 10.21 days for humans. The mean relative error for factual inaccuracy for humans was 14.75 % for total waiting times and 81.06 % for times between investigations. The report produced by the FwA was entirely factually correct. 13 out of 46 students submitted an unfinished audit, 3 out of 46 made an overdue submission. Thematic analysis revealed numerous internal contradictions of the recommendations given by human students.

Conclusion: The AI-based algorithm can produce significantly more recommendations in shorter time. The audits conducted by the AI are more factually accurate (0 % error rate) and logically consistent (no thematic contradictions). This study shows that the algorithm can produce reliable neurosurgical audits for a fraction of the resources required to conduct it by human means.