Title: Mutually reinforcing motion-pose framework for pose invariant action recognition

Authors: Manoj Ramanathan; Wei-Yun Yau; Nadia Magnenat Thalmann; Eam Khwang Teoh

Addresses: Institute for Media Innovation, Nanyang Technological University, 50 Nanyang Drive, Research Techno Plaza, XFrontiers Block, Level 03-01, 637553, Singapore ' Institute of Infocomm Research, A-STAR, 1, Fusionopolis Way, #21-01, Connexis (South Tower), 138632, Singapore ' Institute for Media Innovation, Nanyang Technological University, 50 Nanyang Drive, Research Techno Plaza, XFrontiers Block, Level 03-01, 637553, Singapore ' School of Electrical and Electronics Engineering, Nanyang Technological University, 50, Nanyang Avenue, South Spine, Block S2-B2b-64, 639798, Singapore

Abstract: Action recognition from videos has many potential applications. However, there are many unresolved challenges, such as pose-invariant recognition, robustness to occlusion and others. In this paper, we propose to combine motion of body parts and pose hypothesis generation validated with specific canonical poses observed in a novel mutually reinforcing framework to achieve pose-invariant action recognition. To capture the temporal dynamics of an action, we introduce temporal stick features computed using the stick poses obtained. The combination of pose-invariant kinematic features from motion, pose hypothesis and temporal stick features are used for action recognition, thus forming a mutually reinforcing framework that repeats until the action recognition result converges. The proposed mutual reinforcement framework is capable of handling changes in posture of the person, occlusion and partial view-invariance. We perform experiments on several benchmark datasets which showed the performance of the proposed algorithm and its ability to handle pose variation and occlusion.

Keywords: action recognition; pose-invariant motion feature; canonical stick poses; mutual reinforcement framework.

DOI: 10.1504/IJBM.2019.099014

International Journal of Biometrics, 2019 Vol.11 No.2, pp.113 - 147

Received: 13 Apr 2018
Accepted: 04 Sep 2018

Published online: 12 Apr 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article