Design of Interactive Augmented Reality Functions for Robotic Surgery and Evaluation in Dry-Lab Lymphadenectomy

2022

Article

hi

Augmented reality (AR) has been widely researched for use in healthcare. Prior AR for robot-assisted minimally invasive surgery has mainly focused on superimposing preoperative 3D images onto patient anatomy. This paper presents alternative interactive AR tools for robotic surgery. We designed, built, and evaluated four voice-controlled functions: viewing a live video of the operating room, viewing two-dimensional preoperative images, measuring 3D distances, and warning about out-of-view instruments. This low-cost system was developed on a da Vinci Si, and it can be integrated into surgical robots equipped with a stereo camera and a stereo viewer. Eight experienced surgeons performed dry-lab lymphadenectomies and reported that the functions improved the procedure. They particularly appreciated the possibility of accessing the patient's medical records on demand, measuring distances intraoperatively, and interacting with the functions using voice commands. The positive evaluations garnered by these alternative AR functions and interaction methods provide support for further exploration.

@article{Forte22-IJMRCAS-Design,
  title = {Design of Interactive Augmented Reality Functions for Robotic Surgery and Evaluation in Dry-Lab Lymphadenectomy},
  author = {Forte, Maria-Paola and Gourishetti, Ravali and Javot, Bernard and Engler, Tobias and Gomez, Ernest D. and Kuchenbecker, Katherine J.},
  journal = {The International Journal of Medical Robotics and Computer Assisted Surgery},
  volume = {18},
  number = {2},
  pages = {e2351},
  month = apr,
  year = {2022},
  doi = { 10.1002/rcs.2351},
  month_numeric = {4}
}