Title: Development of pass-through augmented reality interface for human robot interaction

Authors: G. Madhumitha; M. Nandhini; R. Senthilnathan

Addresses: Department of Mechatronics Engineering, SRM Institute of Science and Technology, Kattankulathur, India ' Department of Mechatronics Engineering, SRM Institute of Science and Technology, Kattankulathur, India ' Department of Mechatronics Engineering, SRM Institute of Science and Technology, Kattankulathur, India

Abstract: In this technology driven world, human machine interaction is indispensable. Conventional human machine interface techniques pose the challenges of extensive training and increased cognitive load. Usage of hand gestures, face and speech recognition, eye ball tracking, etc. are in the upward trend to overcome these limitations and prove to be more intuitive and user friendly. In this paper, a vision-based pass-through augmented reality system is developed that enables the user to command the mobile robot for various mobility and navigation tasks. A pass-through augmented reality setup is achieved by a combination of a typical virtual reality headset and a stereo camera with depth perception. The stereo camera and a Leap Motion sensor mounted on a wearable VR headset acts as the vision system for the user and the gesture recognition system respectively. Necessary graphical user interface with the list of functions and information will appear on the VR headset display from which the user can select the option/command to be given to the mobile robot. The gesture made by the user in the GUI is detected and communicated to the robot through wireless means to perform the corresponding task.

Keywords: mixed reality; augmented reality; human-machine interaction.

DOI: 10.1504/IJAMECHS.2020.111307

International Journal of Advanced Mechatronic Systems, 2020 Vol.8 No.2/3, pp.102 - 108

Received: 29 Sep 2019
Accepted: 24 Apr 2020
Published online: 19 Nov 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article