The optimal labelling method for artificial intelligence-assisted polyp detection in colonoscopy

Background: The methodology in colon polyp labeling in establishing database for ma-chine learning is not well-described and standardized. We aimed to find out the best annotation method to generate the most accurate model in polyp detection.

Methods: 3542 colonoscopy polyp images were obtained from endoscopy database of a tertiary medical center. Two experienced endoscopists manually annotated the polyp with (1) exact outline segmentation and (2) using a standard rectangle box close to the polyp margin, and extending 10%, 20%, 30%, 40% and 50% longer in both width and length of the standard rectangle for AI modeling setup. The images were randomly divided into training and validation sets in 4:1 ratio. U-Net convolutional network architecture was used to develop automatic segmentation machine learning model. Another unrelated verification set was established to evaluate the performance of polyp detection by different segmentation methods.

Results: Extending the bounding box to 20% of the polyp margin represented the best performance in accuracy (95.42%), sensitivity (94.84%) and F1-score (95.41%). Exact outline segmentation model showed the excellent performance in sensitivity (99.6%) and the worst precision (77.47%). The 20% model was the best among the 6 models. (confidence interval = 0.957-0.985; AUC = 0.971).

Conclusions: Labelling methodology affect the predictability of AI model in polyp detection. Extending the bounding box to 20% of the polyp margin would result in the best polyp detection predictive model based on AUC data. It is mandatory to establish a standardized way in colon polyp labeling for comparison of the precision of different AI models.