Title: Natural interaction for online documentation in industrial maintenance

Authors: Michele Fiorentino; Antonio Emmanuele Uva; Giuseppe Monno; Rafael Radkowski

Addresses: DMMM, Politecnico di Bari, Viale Japigia 182, Bari, Italy ' DMMM, Politecnico di Bari, Viale Japigia 182, Bari, Italy ' DMMM, Politecnico di Bari, Viale Japigia 182, Bari, Italy ' Heinz Nixdorf Institute, University of Paderborn, Germany

Abstract: This paper presents a novel application of natural interaction techniques for technical documentation navigation in augmented reality maintenance. The objective is to provide technical information where and when needed in order to reduce the time spent in retrieving procedural data from manuals. The system integrates a video and depth camera for user gesture acquisition, a webcam for marker-based tracking and a see through head mounted display for digital information overlay. We designed a novel hand free gesture-based interface to access technical documentation while performing maintenance tasks. We implemented a specific flexible system to prototype natural interfaces using UML configurable state machines. It supports 2D and 3D content navigation, CAD assembly explosion, real time 3D sectioning and operation browsing. We differentiate the interaction metaphors for 2D documents and 3D CAD models in order to be intuitive and easy to learn. A case study and a user test demonstrated the feasibility and the good acceptance of the presented gesture interface.

Keywords: natural interfaces; 3D hand gestures; augmented reality; industrial maintenance; digital documentation; technical documentation; online documentation; document navigation; hand motions; information retrieval; gestural interface; interface design; user interfaces; video camera; depth camera; gesture acquisition; webcams; marker-based tracking; head mounted display; digital information overlay; content navigation; CAD assembly explosion; 3D sectioning; operation browsing; 2D documents; 3D CAD; 3D modelling.

DOI: 10.1504/IJCAET.2016.073269

International Journal of Computer Aided Engineering and Technology, 2016 Vol.8 No.1/2, pp.56 - 79

Published online: 30 Nov 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article