Title: A Kinect-based system for Arabic sign language to speech translation

Authors: Mohammad Fraiwan; Natheer Khasawneh; Hosam Ershedat; Ibrahim Al-Alali; Hamza Al-Kofahi

Addresses: Department of Computer Engineering, Jordan University of Science and Technology, P.O. Box 3030, Irbid 22110, Jordan ' Department of Software Engineering, Jordan University of Science and Technology, P.O. Box 3030, Irbid 22110, Jordan ' Department of Computer Engineering, Jordan University of Science and Technology, P.O. Box 3030, Irbid 22110, Jordan ' Department of Computer Engineering, Jordan University of Science and Technology, P.O. Box 3030, Irbid 22110, Jordan ' Department of Software Engineering, Jordan University of Science and Technology, P.O. Box 3030, Irbid 22110, Jordan

Abstract: Speech-based smart systems have come to play an increasingly diverse role in today's pervasive technology. Moreover, it is quite common to experience all kinds of innovations on daily basis, ranging from retina identifiers at banks to electronic fingerprints readers. Such proliferation presents an opportunity and a challenge to integrate speech and hearing-challenged individuals into society by designing sign language to speech translation systems. In this paper, we tackle the problem of Arabic Sign Language to Speech transformation. We make use of commercial off-the-shelf components to capture the Sign Language gestures. Graphical gestures were transformed into Arabic text, which in turn can be translated into any spoken language. Web services were used to generate the spoken sounds. The majority of this paper is dedicated to explaining hand and fingers identification. In addition, motion recognition is also detailed. The accuracy in identifying the implemented characters was shown to exceed 80%.

Keywords: ASL; Arabic sign language; Kinect; TTS; text-to-speech; web services; speech translation; Arabic text; sign language gestures; spoken sounds; speech sounds; hand identification; finger identification; motion recognition; biometrics; gesture recognition.

DOI: 10.1504/IJCAT.2015.071972

International Journal of Computer Applications in Technology, 2015 Vol.52 No.2/3, pp.117 - 126

Published online: 26 Sep 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article