Sensory Substitution

Augmented reality display of tool-tissue interaction forces from low-cost sensors embedded onto surgical instruments.

With the high costs in the OR, creating surgical systems that integrate novel computer and human/machine interface technologies will revolutionize surgical procedures, extending the surgeon’s abilities to achieve better outcomes at lower costs.  Haptic (force and tactile feedback) has been proposed as a way to further enhance the performance of these systems. A limitation to the current generation of MIS robots is the lack of haptic feedback where the operator relies solely on visual feedback to determine the amount of force being applied. Implementing direct haptic feedback to the surgeon’s hands remains impractical for clinical application because of the cost and time of applying force sensors to disposable tools and the current limitations in sensing and control technologies. The goals of this work are to develop an intuitive augmented reality system for feedback of force information through sensory substitution, and to evaluate its performance in a surgical task such as knot tying. Work done in collaboration with graduate students Tope Akinibiyi and Sunipa Saha.

Funding Source: NIH Grant R01 EB002004, Whitaker R6-02-911
Special Thanks to the Minimally Invasive Surgery Training Center (MISTC) and Dr. David Yuh.