Article: Human behaviour analysis based on spatio-temporal dual-stream heterogeneous convolutional neural network Journal: International Journal of Computational Science and Engineering (IJCSE) 2023 Vol.26 No.6 pp.673 - 683 Abstract: At present, there are still many problems to be solved in human behaviour analysis, such as insufficient utilisation of behaviour characteristic information and slow operation rate. We propose a human behaviour analysis algorithm based on spatio-temporal dual-stream heterogeneous convolutional neural network (STDNet). The algorithm is improved on the basic structure of the traditional dual-stream network. When extracting spatial information, the DenseNet uses a hierarchical connection method to construct a dense network to extract the spatial feature of the video RGB image. When extracting motion information, BNInception is used to extract temporal features of video optical flow images. Finally, feature fusion is carried out by multi-layer perceptron and sent to Softmax classifier for classification. Experimental results on the UCF101 dataset show that the algorithm can effectively use the spatio-temporal feature information in video, reduce the amount of calculation of the network model, and greatly improve the ability to distinguish similar actions. Inderscience Publishers - linking academia, business and industry through research

Title: Human behaviour analysis based on spatio-temporal dual-stream heterogeneous convolutional neural network

Authors: Qing Ye; Yuqi Zhao; Haoxin Zhong

Addresses: School of Information Science and Technology, North China University of Technology, Beijing 100144, China ' School of Information Science and Technology, North China University of Technology, Beijing 100144, China ' State Grid Beijing Electric Power Construction Engineering Consulting Company, No. 188, Chengshousi Road, Fengtai District, Beijing 100164, China

Abstract: At present, there are still many problems to be solved in human behaviour analysis, such as insufficient utilisation of behaviour characteristic information and slow operation rate. We propose a human behaviour analysis algorithm based on spatio-temporal dual-stream heterogeneous convolutional neural network (STDNet). The algorithm is improved on the basic structure of the traditional dual-stream network. When extracting spatial information, the DenseNet uses a hierarchical connection method to construct a dense network to extract the spatial feature of the video RGB image. When extracting motion information, BNInception is used to extract temporal features of video optical flow images. Finally, feature fusion is carried out by multi-layer perceptron and sent to Softmax classifier for classification. Experimental results on the UCF101 dataset show that the algorithm can effectively use the spatio-temporal feature information in video, reduce the amount of calculation of the network model, and greatly improve the ability to distinguish similar actions.

Keywords: human behaviour analysis; STDNet; optical flow; feature extraction; dual-stream network.

DOI: 10.1504/IJCSE.2023.135277

International Journal of Computational Science and Engineering, 2023 Vol.26 No.6, pp.673 - 683

Received: 15 Oct 2021
Received in revised form: 21 Apr 2022
Accepted: 02 May 2022
Published online: 04 Dec 2023 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Human behaviour analysis based on spatio-temporal dual-stream heterogeneous convolutional neural network

Keep up-to-date