Accurate instance segmentation of surgical instruments in robotic surgery: model refinement and cross-dataset evaluation

Purpose: Automatic segmentation of surgical instruments in robot-assisted minimally invasive surgery plays a fundamental role in improving context awareness. In this work, we present an instance segmentation model based on refined Mask R-CNN for accurately segmenting the instruments as well as identifying their types.

Methods: We re-formulate the instrument segmentation task as an instance segmentation task. Then we optimize the Mask R-CNN with anchor optimization and improved Region Proposal Network for instrument segmentation. Moreover, we perform cross-dataset evaluation with different sampling strategies.

Results: We evaluate our model on a public dataset of the MICCAI 2017 Endoscopic Vision Challenge with two segmentation tasks, and both achieve new state-of-the-art performance. Besides, cross-dataset training improved the performance on both segmentation tasks compared with those tested on the public dataset.

Conclusion: Results demonstrate the effectiveness of the proposed instance segmentation network for surgical instruments segmentation. Cross-dataset evaluation shows our instance segmentation model presents certain cross-dataset generalization capability, and cross-dataset training can significantly improve the segmentation performance. Our empirical study also provides guidance on how to allocate the annotation cost for surgeons while labelling a new dataset in practice.