Background: Up to half of all surgical adverse events are due to non-technical errors, making non-technical skill assessment and improvement a priority. No specific tools are available to retrospectively identify non-technical errors that have occurred in surgical patient care. This original study aimed to develop and provide evidence of validity and inter-rater reliability for the System for Identification and Categorization of Non-technical Error in Surgical Settings (SICNESS).
Methods: A literature review, modified Delphi process, and two pilot phases were used to develop and test the SICNESS tool. For each pilot, 12 months of surgical mortality data from the Australian and New Zealand Audit of Surgical Mortality were assessed by two independent reviewers using the SICNESS tool. Main outcomes included tool validation through modified Delphi consensus, and inter-rater reliability for: non-technical error identification and non-technical error categorization using Cohen's κ coefficient, and overall agreement using Fleiss' κ coefficient.
Results: Version 1 of the SICNESS was used for pilot 1, including 412 mortality cases, and identified and categorized non-technical errors with strong-moderate inter-rater reliability. Non-technical error exemplars were created and validated through Delphi consensus, and a novel mental model was developed. Pilot 2 included an additional 432 mortality cases. Inter-rater reliability was near perfect for leadership (κ 0.92, 95% c.i. 0.82 to 1.00); strong for non-technical error identification (κ 0.89, 0.84 to 0.93), communication and teamwork (κ 0.89, 0.79 to 0.99), and decision-making (κ 0.85, 0.79 to 0.92); and moderate for situational awareness (κ 0.79, 0.71 to 0.87) and overall agreement (κ 0.69, 0.66 to 0.73).
Conclusion: The SICNESS is a reliable and valid tool, enabling retrospective identification and categorization of non-technical errors associated with death, occurring in real surgical patient interactions.
Many errors in surgery occur because of poor non-technical skills. The aim of this study was to create a tool to identify this type of error using patient data so future errors may be prevented. The tool was designed through expert opinion and literature review. It was tested using surgical patient death data from Australia and New Zealand. The final tool was able to identify and group non-technical errors reliably. This tool makes it possible to identify non-technical errors so future errors may be reduced.
© The Author(s) 2024. Published by Oxford University Press on behalf of BJS Foundation Ltd.