We are investigating technical approaches for improving training in robotic surgery. From left to right: Vibration sensors are attached to the arms of a da Vinci surgical system, and a force sensor is placed below the task. Vibration and force signals are used for surgical skill evaluation. Trainees also feel the vibrations as vibrotactile feedback played by voice-coil actuators, while force feedback is rendered through tactile actuators. Visual augmented reality allows training with pre-recorded expert performances.
Minimally invasive surgery (MIS) allows surgeons to perform procedures through tiny incisions, thus reducing healing time compared to traditional open surgery. In robot-assisted MIS (RMIS), surgeons control the instruments from a console. Consequently, besides the advantages of using a robotic platform, RMIS has one main drawback: the lack of haptic feedback, which is critical in manipulation tasks.
We explored the effects of haptic feedback while training in RMIS. First, we investigated vibrotactile feedback using VerroTouch, a previously developed system that allows the surgeon to feel the instruments’ vibrations at the operator’s manipulators. The vibrotactile feedback did not increase or decrease the trainee's workload [ ]; ongoing work is examining impacts on learning. We also explored force feedback and created bracelets that squeeze the operator’s wrists proportionally to the force applied by the surgical instruments. Participants applied significantly less force on the task materials when receiving the feedback [ ].
We identified two additional challenges when training in RMIS: the quantitative evaluation of surgical skills and the limits of existing simulators.
Currently, surgical skill assessment is mainly conducted through manual video evaluation, which is time-consuming and subject to bias. For MIS, we developed a motion-tracking system and machine-learning algorithm to evaluate trainee performance during suturing tasks. The automatic ratings closely matched human expert ratings [ ]. For RMIS, we showed that surgical skill can be estimated through completion time, force applied to task materials, and vibrations generated at the surgical instruments [ ].
Finally, training with physical simulators requires experts to supervise the process. In RMIS, using a robotic console offers the possibility of evaluating surgical skills in real time and training in virtual and augmented reality. However, current virtual simulators result in lower skill transfer to actual surgeries. As such, we hypothesize that augmented reality simulators could combine the advantages of physical and virtual approaches. After an early prototype [ ], our final developed platform allows expert surgeons to record surgical procedures with multiple modalities and novice surgeons to replay and train with the multimodal recordings, including visual, auditory, and vibrotactile feedback [ ].