Title: Computer vision-based approach for detecting arm-flapping as autism suspect behaviour

Authors: Esraa T. Sadek; Noha A. Seada; Said Ghoniemy

Addresses: Department of Computer Systems, Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt ' Department of Computer Systems, Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt ' Department of Computer Systems, Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt

Abstract: Autism spectrum disorder (ASD) is a neurodevelopmental condition that is characterised by frequent and restrictive motor activities, besides social and communicative disorders. It is considered one of the most rapidly evolving neurodevelopmental disorders in children recently. Repetitive motor behaviours, like arm-flapping and head rocking, may lead to attention distraction and self-injury in severe cases. In this research, a computer-vision-based neural network framework is proposed to automatically detect significant arm-flapping behaviour in autistics. The proposed framework goes through four main phases which are data pre-processing, pose estimation and skeleton representation, data post-processing, and action classification. The proposed framework was tested on three datasets and proved its applicability in real-world applications. The attained accuracy was better compared to that of the state-of-the-art methods. The proposed solution can be used to assist clinicians, and parents to automatically detect this behaviour to offer the child the appropriate medical care once a behavioural abnormality is detected.

Keywords: autism spectrum disorder; ASD; arm flapping; computer vision; neural networks.

DOI: 10.1504/IJMEI.2023.129350

International Journal of Medical Engineering and Informatics, 2023 Vol.15 No.2, pp.166 - 176

Received: 09 Oct 2020
Accepted: 08 Mar 2021
Published online: 07 Mar 2023 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article