Title: Toward the design of a novel surgeon-computer interface using image processing of surgical tools in minimally invasive surgery

Authors: Shahram Payandeh; Jeff Hsu; Peter Doris

Addresses: Experimental Robotics and Imaging Laboratory, School of Engineering Science, Simon Fraser University, 8888 University Drive, Burnaby, British Columbia, V5A 1S6, Canada. ' Experimental Robotics and Imaging Laboratory, School of Engineering Science, Simon Fraser University, 8888 University Drive, Burnaby, British Columbia, V5A 1S6, Canada. ' Division of Minimally Invasive Surgery, Surrey Memorial Hospital, 13750 96 Avenue, Surrey, British Columbia, V3V 1Z2, Canada

Abstract: Minimally invasive surgery (or key-hole surgery) is an alternative to open surgery has been gaining popularity among patients and health delivery systems. In general, to view the surgical site, an endoscope is inserted into the abdominal cavity though natural or artificial incision. Long stem surgical tools are also inserted through supporting incisions. The surgeon can then perform the operation by indirectly viewing the scene and manipulating the surgical tools. While viewing the monitor, the surgeon does not have any automatic access to preoperative images or patient specific data or be able to manipulate superimpose them on the viewing monitor. This paper presents a novel approach based on image processing of the surgical site and neural network framework for classifying and identifying gestures of surgical tools and classification of their motions. Seven feature quantities were selected as an input to a feed-forward neural network. Experimental analysis of the classification was carried-out for single tools and multiple tool gestures in an in-vitro setting. Through a number of trails we were able to demonstrate the feasibility of our gesture recognition approaches.

Keywords: surgical tools; image processing; tool gesture recognition; surgeon-computer interface; SCI; human-computer interface; HCI; minimally invasive surgery; keyhole surgery; neural networks; classification; tool motion.

DOI: 10.1504/IJMEI.2012.045300

International Journal of Medical Engineering and Informatics, 2012 Vol.4 No.1, pp.1 - 24

Published online: 11 Aug 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article