Int. J. of Arts and Technology   »   2015 Vol.8, No.1

 

 

Title: Finger musical gesture recognition in 3D space without any tangible instrument for performing arts

 

Authors: Sotiris Manitsaris; Apostolos Tsagaris; Kosmas Dimitropoulos; Athanasios Manitsaris

 

Addresses:
Robotics Lab, Mathématiques et Systèmes, Mines ParisTech, 60 Bd St Michel, 75272, Paris Cedex 06, France
Multimedia Technology and Computer Graphics Lab, Applied Informatics, University of Macedonia, 156 Egnatia Street, 54006, Thessaloniki, Greece
Informatics & Telematics Institute, Centre of Research & Technology, 6th km Xarilaou – Thermi, 57001, Thessaloniki, Greece
Multimedia Technology and Computer Graphics Lab, Applied Informatics, University of Macedonia, 156 Egnatia Street, 54006, Thessaloniki, Greece

 

Abstract: This paper proposes a marker-less computer vision methodology for the simultaneous recognition of complex finger musical gestures performed in space without any tangible musical instrument. Image analysis techniques are applied in order to detect and identify the fingertips on a video. Scale and rotation invariance techniques are also applied. The finger gesture recognition and prediction is based on the stochastic modelling of the extracted high-level features with the help of hidden Markov models and Gaussian mixture models. The applications concern directly the finger gesture control of sound in performing arts. The proposed system can be considered in the long term as an intangible musical instrument that implies 'everyday gestures' and reduces the gap between non-musicians or blind people and music.

 

Keywords: finger gesture recognition; computer vision; musical gestures; music interaction; intangible musical instruments; performing arts; image analysis; fingertip identification; video; stochastic modelling; feature extraction; hidden Markov models; HMM; Gaussian mixture models; sound control; everyday gestures.

 

DOI: 10.1504/IJART.2015.067390

 

Int. J. of Arts and Technology, 2015 Vol.8, No.1, pp.11 - 29

 

Available online: 08 Feb 2015

 

 

Editors Full text accessAccess for SubscribersPurchase this articleComment on this article