Title: Bi-LSTM with attention pooling based on joint motion and difference entropy for action recognition

Authors: Lunzheng Tan; Chunping Huang; Limin Xia; Jiaxiao Li

Addresses: School of Information Engineering, Zhongshan Polytechnic, Zhongshan, Guangdong, 528400, China ' School of Information Engineering, Zhongshan Polytechnic, Zhongshan, Guangdong, 528400, China ' College of Information Science and Engineering, Central South University, Changsha, Hunan, 410075, China ' School of Information Engineering, Zhongshan Polytechnic, Zhongshan, Guangdong, 528400, China

Abstract: Human action recognition is one of the most challenging tasks in computer vision due to its complex background changes and redundancy of long-term video information. To tackle these issues, we propose a novel action recognition framework called Bi-LSTM with Attention Pooling based on Joint motion and difference Entropy (JEAP-BiLSTM). Firstly, we extract critical points of motion flow field as the key points of optical flow field, then compute the motion and difference entropy maps of the key points' optical flow as short-term features. On this basis, we then use Bi-LSTM to extract video long-term temporal features from forward and backward simultaneously. In order to solve the problem of background change, we introduce attention pooling to the extracted features to highlight the region of interest (ROI). Experiments demonstrate that the proposed JEAP-BiLSTM outperforms state-of-the-art action recognition methods.

Keywords: action recognition; attention mechanism; entropy map; Bi-LSTM.

DOI: 10.1504/IJAACS.2024.142198

International Journal of Autonomous and Adaptive Communications Systems, 2024 Vol.17 No.5, pp.436 - 448

Received: 10 Sep 2022
Accepted: 19 Nov 2022
Published online: 14 Oct 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article