Using Failure Mode and Effects Analysis to Evaluate Risk in the Clinical Adoption of Automated Contouring and Treatment Planning Tools

Purpose: In this study, we applied the failure mode and effects analysis (FMEA) approach to an automated radiation therapy contouring and treatment planning tool to assess, and subsequently limit, the risk of deploying automated tools.

Methods and materials: Using an FMEA, we quantified the risks associated with the Radiation Planning Assistant (RPA), an automated contouring and treatment planning tool currently under development. A multidisciplinary team identified and scored each failure mode, using a combination of RPA plan data and experience for guidance. A 1-to-10 scale for severity, occurrence, and detectability of potential errors was used, following American Association of Physicists in Medicine Task Group 100 recommendations. High-risk failure modes were further explored to determine how the workflow could be improved to reduce the associated risk.

Results: Of 290 possible failure modes, we identified 126 errors that were unique to the RPA workflow, with a mean risk priority number (RPN) of 56.3 and a maximum RPN of 486. The top 10 failure modes were caused by automation bias, operator error, and software error. Twenty-one failure modes were above the action threshold of RPN = 125, leading to corrective actions. The workflow was modified to simplify the user interface and better training resources were developed, which highlight the importance of thorough review of the output of automated systems. After the changes, we rescored the high-risk errors, resulting in a final mean and maximum RPN of 33.7 and 288, respectively.

Conclusions: We identified 126 errors specific to the automated workflow, most of which were caused by automation bias or operator error, which emphasized the need to simplify the user interface and ensure adequate user training. As a result of changes made to the software and the enhancement of training resources, the RPNs subsequently decreased, showing that FMEA is an effective way to assess and reduce risk associated with the deployment of automated planning tools.