Title: New colour fusion deep learning model for large-scale action recognition

Authors: Yukhe Lavinia; Holly Vo; Abhishek Verma

Addresses: Department of Computer Science, California State University, Fullerton, CA 92831, USA ' Department of Computer Science, California State University, Fullerton, CA 92831, USA ' Department of Computer Science, New Jersey City University, Jersey City, NJ 07305, USA

Abstract: In this work we propose a fusion methodology that takes advantage of multiple deep convolutional neural network (CNN) models and two colour spaces RGB and oRGB to improve action recognition performance on still images. We trained our deep CNNs on both the RGB and oRGB colour spaces, extracted and fused all the features, and forwarded them to an SVM for classification. We evaluated our proposed fusion models on the Stanford 40 Action dataset and the People Playing Musical Instruments (PPMI) dataset using two metrics: overall accuracy and mean average precision (mAP). Our results prove to outperform the current state-of-the-arts with 84.24% accuracy and 83.25% mAP on Stanford 40 and 65.94% accuracy and 65.85% mAP on PPMI. Furthermore, we also evaluated the individual class performance on both datasets. The mAP for top 20 individual classes on Stanford 40 lies between 97% and 87%, on PPMI the individual mAP class performance lies between 87% and 34%.

Keywords: deep convolutional neural networks; deep learning fusion model; action recognition; VGGNet; GoogLeNet; ResNet; colour fusion; deep learning; computational vision robotics; computational vision; computational robotics; robotics; vision; inception v1; residual nets; deep learning fusion; action classification; image classification.

DOI: 10.1504/IJCVR.2020.104356

International Journal of Computational Vision and Robotics, 2020 Vol.10 No.1, pp.41 - 60

Received: 14 Jan 2019
Accepted: 03 Mar 2019
Published online: 06 Jan 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article