Article: Finger gesture recognition by MediaPipe algorithm and advanced YOLOv7 network for deaf people Journal: International Journal of Biomedical Engineering and Technology (IJBET) 2025 Vol.48 No.1 pp.55 - 74 Abstract: Sign language is a challenge to be able to recognise and understand correctly its meaning. Therefore, with the deaf community, understanding sign language for daily activities is essential. For the support of the deaf community in interfacing together, researchers have represented different methods to support them in using sign languages. This article proposes using simple landmark hands with different finger shapes to detect ten letters in the American sign language (ASL) writing system. Therefore, an inference - MediaPipe algorithm for extracting hand features with the advanced 'you only look once version 7' (YOLOv7) network was applied for sign language recognition. The article used 2,000 hand images of five persons of different ages and genders with the YOLOv7 network to produce the high recognition performance. In particular, the average mAP accuracy reached 0.995, accuracy reached 99.4%, and recall parameter reached 99.2% using the confusion matrix. This algorithm can be developed with more letters. Inderscience Publishers - linking academia, business and industry through research

Title: Finger gesture recognition by MediaPipe algorithm and advanced YOLOv7 network for deaf people

Authors: Thanh-Hai Nguyen; Ba-Viet Ngo; Thanh-Long Nguyen; Chi-Cuong Vu

Addresses: Faculty of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology and Education, Vietnam ' Faculty of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology and Education, Vietnam ' Faculty of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology and Education, Vietnam ' Faculty of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology and Education, Vietnam

Abstract: Sign language is a challenge to be able to recognise and understand correctly its meaning. Therefore, with the deaf community, understanding sign language for daily activities is essential. For the support of the deaf community in interfacing together, researchers have represented different methods to support them in using sign languages. This article proposes using simple landmark hands with different finger shapes to detect ten letters in the American sign language (ASL) writing system. Therefore, an inference - MediaPipe algorithm for extracting hand features with the advanced 'you only look once version 7' (YOLOv7) network was applied for sign language recognition. The article used 2,000 hand images of five persons of different ages and genders with the YOLOv7 network to produce the high recognition performance. In particular, the average mAP accuracy reached 0.995, accuracy reached 99.4%, and recall parameter reached 99.2% using the confusion matrix. This algorithm can be developed with more letters.

Keywords: gesture recognition; YOLOv7 network; inference - MediaPipe algorithm; hand features; sign language.

DOI: 10.1504/IJBET.2025.146402

International Journal of Biomedical Engineering and Technology, 2025 Vol.48 No.1, pp.55 - 74

Received: 25 Feb 2024
Accepted: 26 Jun 2024
Published online: 28 May 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Finger gesture recognition by MediaPipe algorithm and advanced YOLOv7 network for deaf people

Keep up-to-date