Nested Semi-Transparent Isosurface Simulated Volume-Rendering (NESTIS-VR) - An efficient on-device rendering approach for Augmented Reality headsets increasing surgeon confidence of kidney donor arterial anatomy

Fabian N Necker; Marc L Melcher; Stephan Busque; Christoph W Leuze; Pejman Ghanouni; Chris Le Castillo; Elizabeth Nguyen; Bruce L Daniel

doi:10.1016/j.compbiomed.2024.109267

Nested Semi-Transparent Isosurface Simulated Volume-Rendering (NESTIS-VR) - An efficient on-device rendering approach for Augmented Reality headsets increasing surgeon confidence of kidney donor arterial anatomy

Comput Biol Med. 2024 Oct 12:183:109267. doi: 10.1016/j.compbiomed.2024.109267. Online ahead of print.

Authors

Fabian N Necker¹, Marc L Melcher², Stephan Busque², Christoph W Leuze³, Pejman Ghanouni⁴, Chris Le Castillo⁵, Elizabeth Nguyen³, Bruce L Daniel⁶

Affiliations

¹ Incubator for Medical Mixed Reality at Stanford (IMMERS), Department of Radiology, Stanford University, Palo Alto, CA, USA; Institute of Functional and Clinical Anatomy, Faculty of Medicine, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany; Department of Radiology, Stanford University, Palo Alto, CA, USA. Electronic address: [email protected].
² Stanford Medicine, Department of Surgery - Multi-Organ Transplantation, Stanford University, Palo Alto, CA, USA.
³ Incubator for Medical Mixed Reality at Stanford (IMMERS), Department of Radiology, Stanford University, Palo Alto, CA, USA.
⁴ Department of Radiology, Stanford University, Palo Alto, CA, USA.
⁵ 3D and Quantitative Imaging Lab, Department of Radiology, Stanford University, Palo Alto, CA, USA.
⁶ Incubator for Medical Mixed Reality at Stanford (IMMERS), Department of Radiology, Stanford University, Palo Alto, CA, USA; Department of Radiology, Stanford University, Palo Alto, CA, USA.

PMID: 39405728
DOI: 10.1016/j.compbiomed.2024.109267

Abstract

Background and objective: Volume-renderings of computed tomography or magnetic resonance angiograms (MRAs) are routinely used by surgeons in the preoperative assessment of vascular anatomy in kidney donors. Stereoscopic headsets (OST-HMD) like Microsoft HoloLens allow intuitive interaction with three-dimensional content for more intuitive comprehension, but do not allow real-time ray-casting volume-rendering of medical volume datasets on-device due to computational limitations.

Methods: We introduce NEsted Semi-Transparent Isosurface Simulated Volume-Rendering (NESTIS-VR), as an on-device alternative to ray-casting volume-rendering and developed an application for HoloLens to render kidney donor MRAs with interactive control of fundamental rendering parameters. We compared NESTIS-VR with current standard pre-calculated 2D ray-cast volume-renderings in an observational study with 2 expert kidney transplant surgeons, measuring their confidence in pre-operatively assessing the kidney pedicle arterial anatomy in 20 potential donors. We also compared it against other 3D rendering techniques to understand which features contributed most to any improvements.

Results: Real-time stereoscopic three-dimensional (3D) NESTIS-VR in Augmented Reality significantly improves surgeons' confidence compared with pre-calculated conventional two-dimensional (2D) ray-casting volume-rendered images (p = 0.0415/p = 0.00003). 2D non-stereoscopic NESTIS-VR was significantly superior to pre-calculated 2D ray-casting volume-rendered images for both surgeons (p = 0.044/p = 0.0003). Single isosurface 2D rendering was significantly superior than pre-calculated 2D volume-rendered images for one surgeon. There was no significant difference between binocular 3D display over 2D views with NESTIS-VR or between constrained and unconstrained vantage points for 2D viewing.

Conclusion: NESTIS-VR provides a new approach to rendering medical datasets in computationally limited OST-HMD headsets and significantly increases surgeons' confidence of kidney donor arterial anatomy. The principal confidence benefit arises from providing surgeons interactive control over rendering parameters compared to pre-calculated renderings at preset parameters whilst rendering on-device and keeping the OST-HMD untethered from a workstation.

Keywords: Augmented Reality; Computer-aided surgery; Depth cues; Rendering techniques; Surgical planning; Vascular anatomy.