Article: Real-time human action recognition using depth motion maps and convolutional neural networks Journal: International Journal of High Performance Computing and Networking (IJHPCN) 2019 Vol.13 No.3 pp.312 - 320 Abstract: This paper presents an effective approach for recognising human actions from depth video sequences by employing depth motion maps (DMMs) and convolutional neural networks (CNNs). Depth maps are projected onto three orthogonal planes, and frame differences under each view (front/side/top) are then accumulated through an entire depth video sequence generating a DMM. We build a model architecture of multi-view convolutional neural network (MV-CNN) containing multiple networks to deal with three DMMs (DMMf, DMMs, DMMt). The output of full-connected layer under each view is integrated as feature representation, which is then learned in the last softmax regression layer to predict human actions. Experimental results on MSR-Action3D dataset and UTD-MHAD dataset indicate that the proposed approach achieves state-of-the-art recognition performance and is appropriate for real-time recognition. Inderscience Publishers - linking academia, business and industry through research

Title: Real-time human action recognition using depth motion maps and convolutional neural networks

Authors: Jiang Li; Xiaojuan Ban; Guang Yang; Yitong Li; Yu Wang

Addresses: School of Computer and Communication Engineering, University of Science and Technology Beijing, 30 Xueyuan Road, Haidian District, Beijing, 100083, China ' School of Computer and Communication Engineering, University of Science and Technology Beijing, 30 Xueyuan Road, Haidian District, Beijing, 100083, China ' School of Computer and Communication Engineering, University of Science and Technology Beijing, 30 Xueyuan Road, Haidian District, Beijing, 100083, China ' School of Computer and Communication Engineering, University of Science and Technology Beijing, 30 Xueyuan Road, Haidian District, Beijing, 100083, China ' North Electronic Instrument Institute, 100191, Beijing, China

Abstract: This paper presents an effective approach for recognising human actions from depth video sequences by employing depth motion maps (DMMs) and convolutional neural networks (CNNs). Depth maps are projected onto three orthogonal planes, and frame differences under each view (front/side/top) are then accumulated through an entire depth video sequence generating a DMM. We build a model architecture of multi-view convolutional neural network (MV-CNN) containing multiple networks to deal with three DMMs (DMM_f, DMM_s, DMM_t). The output of full-connected layer under each view is integrated as feature representation, which is then learned in the last softmax regression layer to predict human actions. Experimental results on MSR-Action3D dataset and UTD-MHAD dataset indicate that the proposed approach achieves state-of-the-art recognition performance and is appropriate for real-time recognition.

Keywords: real-time human action recognition; depth motion maps; DMMs; multi-view convolutional neural networks; MV-CNNs.

DOI: 10.1504/IJHPCN.2019.098572

International Journal of High Performance Computing and Networking, 2019 Vol.13 No.3, pp.312 - 320

Received: 24 Jun 2016
Accepted: 10 Oct 2016
Published online: 28 Mar 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Real-time human action recognition using depth motion maps and convolutional neural networks

Keep up-to-date