Title: Spatiotemporal and region relationship network for micro-expression recognition

Authors: Zhiming Zhou; Ai Guan; Qiaoling Han; Qiuyan Zheng; Yue Zhao; Baoqing Zhu

Addresses: The School of Technology, Beijing Forestry University, Beijing 100083, China ' The School of Landscape Architecture, Beijing Forestry University, Beijing 100083, China ' The School of Technology, Beijing Forestry University, Beijing 100083, China ' The School of Technology, Beijing Forestry University, Beijing 100083, China ' The School of Technology, Beijing Forestry University, Beijing 100083, China ' The School of Biological Sciences and Technology, Beijing Forestry University, Beijing 100083, China

Abstract: Micro-expressions are subtle facial expressions that occur when a person fails to suppress his emotional response. A significant issue is the intricate correlation between micro-expression motion and various facial regions, which hinders the extraction of effective feature. This paper proposes a novel spatiotemporal and regional relationship network (STRNet) to address this issue. STRNet consists of two branches and a specialised feature fusion module. Specifically, one branch performs multi-level feature extraction. The other branch focuses on modelling the relationships between different facial regions. The outputs of the two branches are then combined through a feature fusion module; this module enhances the generalisation of the micro-expression features extracted by the network. Experiments on four public datasets (SAMM, CASMEII, SMIC and CASME3) validate STRNet's effectiveness. On CASMEII, it achieved UF1 and UAR scores of 0.9792 and 0.9764, respectively, and on CASME3, 0.5848 and 0.5601. STRNet outperformed existing methods, demonstrating superior performance.

Keywords: micro-expression recognition; affective computing; image classification; deep learning; convolutional neural network.

DOI: 10.1504/IJSCC.2025.145785

International Journal of Systems, Control and Communications, 2025 Vol.16 No.2, pp.104 - 118

Received: 23 Dec 2024
Accepted: 08 Feb 2025
Published online: 23 Apr 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article