Three-Dimensional Surgical Needle Localization and Tracking Using Stereo Endoscopic Image Streams

IEEE Int Conf Robot Autom. 2018 May:2018:6617-6624. doi: 10.1109/icra.2018.8460867. Epub 2018 Sep 13.

Abstract

This paper presents algorithms for three-dimensional tracking of surgical needles using the stereo endoscopic camera images obtained from the da Vinci ® Surgical Robotic System. The proposed method employs Bayesian state estimation, computer vision techniques, and robot kinematics. A virtual needle rendering procedure is implemented to create simulated images of the surgical needle under the da Vinci ® robot endoscope, which makes it possible to measure the similarity between the rendered needle image and the real needle. A particle filter algorithm using the mentioned techniques is then used for tracking the surgical needle. The performance of the tracking is experimentally evaluated using an actual da Vinci ® surgical robotic system and quantitatively validated in a ROS/Gazebo simulation thereof.