Title: Hybrid lip shape feature extraction and recognition for human-machine interaction

Authors: Yi Zhang; Jiao Liu; Yuan Luo; Huosheng Hu

Addresses: Engineering Research and Development Centre of Information Accessibility, Research Centre of Intelligent System and Robot, Chongqing University of Posts and Telecommunications, Chongqing 400065, China ' Engineering Research and Development Centre of Information Accessibility, Research Centre of Intelligent System and Robot, Chongqing University of Posts and Telecommunications, Chongqing 400065, China ' Engineering Research and Development Centre of Information Accessibility, Research Centre of Intelligent System and Robot, Chongqing University of Posts and Telecommunications, Chongqing 400065, China ' School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK

Abstract: Dumb and deaf people are unable to interact with robots using traditional voice-based human-machine interfaces (HMI). Lip motion is a useful way for these people to communicate with machines, even for normal people in extremely noisy environments. However, the recognition of lip motion is a difficult task since the region of interest (ROI) is non-linear and noisy. This paper proposes a novel lip shape feature extraction method to deal with the difficulty, based on hybrid dual-tree complex wavelet transform (DT-CWT) and discrete cosine transform (DCT). The approximate shift invariance of DT-CWT is utilised to make the same lip shape have the same feature vector when the lips are in different positions in the ROI. Then, DCT is used to extract coefficients from the feature vector generated by DT-CWT, and to choose the larger coefficients to obtain the key information of lip shape and reduce the dimensions of a feature vector. The experimental results show that this method can greatly improve the accuracy of lip shape recognition, and enhance the robustness of the lip shape-based HMI.

Keywords: human-machine interaction; HMI; lip shapes; feature extraction; feature recognition; dual-tree complex wavelet transform; DT-CWT; discrete cosine transform; DCT; adaboost; support vector machines; SVM; deaf people; dumb people; lip motion; lip shape recognition; hearing disability; speech disability.

DOI: 10.1504/IJMIC.2013.052812

International Journal of Modelling, Identification and Control, 2013 Vol.18 No.3, pp.191 - 198

Published online: 16 Aug 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article