Title: Arabic sign language recognition using vision and hand tracking features with HMM

Authors: Ala Addin I. Sidig; Hamzah Luqman; Sabri A. Mahmoud

Addresses: College of Computer Science and Engineering, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia ' College of Computer Science and Engineering, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia ' College of Computer Science and Engineering, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia

Abstract: Sign language employs signs made by hands and facial expressions to convey meaning. Sign language recognition facilitates the communication between community and hearing-impaired people. This work proposes a recognition system for Arabic sign language using four types of features, namely modified Fourier transform, local binary pattern, histogram of oriented gradients, and a combination of histogram of oriented gradients and histogram of optical flow. These features are evaluated using hidden Markov model on two databases. The best performance is achieved with modified Fourier transform and histogram of oriented gradients features with 99.11% and 99.33% accuracies, respectively. In addition, two algorithms are proposed, one for segmenting sign video streams captured by Microsoft Kinect V2 into signs and the second for hand detection in video streams. The obtained results show that our algorithms are efficient in segmenting sign video streams and detecting hands in video streams.

Keywords: Arabic sign language; ArSL; sign language recognition; SLR; video segmentation; histogram of oriented gradients; HOGs; hands detection; hidden Markov model; HMM.

DOI: 10.1504/IJISTA.2019.101951

International Journal of Intelligent Systems Technologies and Applications, 2019 Vol.18 No.5, pp.430 - 447

Received: 28 Aug 2017
Accepted: 21 Feb 2018
Published online: 02 Sep 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article