Title: Facial action unit and its intensity detection using multi-network architecture

Authors: Rohan Appasaheb Borgalli; Sunil Surve

Addresses: Department of Electronics Engineering, Fr. Conceicao Rodrigues College of Engineering, Bandra, India; Department of Electronics and Telecommunication Engineering, Shah and Anchor Kutchhi Engineering College, Mumbai, India ' Department of Computer Engineering, Fr. Conceicao Rodrigues College of Engineering, Bandra, India; University of Mumbai, Mumbai – 400050, India

Abstract: Facial expressions recognition (FER) plays a significant role in applications like medicine, human-machine interface, e-education, video games, AI, distance psychotherapy, and security. In literature, solving the FER problem based on single static images is preferred due to the availability of the dataset, processing requires less memory, and the algorithm is not as complex as videos. In terms of techniques, deep learning, particularly convolution neural networks (CNNs) is favoured for its ability to learn high-level facial features. The proposed multi-network architecture uses modified Xceptionnet architecture by slightly changing a few final fully connected layers to detect facial action unit (FAU) intensity accurately. Using this modified architecture, we designed multi-network architecture for the DISFA+ Database, which consists of 12 networks, each trained separately on FAUs to detect action units and their intensity with reasonable accuracy of 89% and 64%, respectively, to be then intern mapped to find basic and compound facial emotions.

Keywords: facial expression; facial action unit; convolution neural network; action unit intensity; deep learning.

DOI: 10.1504/IJCVR.2025.148221

International Journal of Computational Vision and Robotics, 2025 Vol.15 No.5, pp.545 - 564

Accepted: 15 Nov 2023
Published online: 01 Sep 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article