Title: A novel keyframe extraction technique systems using deep reinforcement learning simulation

Authors: M. Dhanushree; R. Priya; P. Aruna; R. Bhavani

Addresses: Department of Computer Science and Engineering, Faculty of Engineering and Technology, Annamalai University, Annamalainagar, 608002, Chidambaram, Tamil Nadu, India ' Department of Computer Science and Engineering, Faculty of Engineering and Technology, Annamalai University, Annamalainagar, 608002, Chidambaram, Tamil Nadu, India ' Department of Computer Science and Engineering, Faculty of Engineering and Technology, Annamalai University, Annamalainagar, 608002, Chidambaram, Tamil Nadu, India ' Department of Computer Science and Engineering, Faculty of Engineering and Technology, Annamalai University, Annamalainagar, 608002, Chidambaram, Tamil Nadu, India

Abstract: Smartphones and wearables make video capture easy, increasing video generation. The abundance of videos led to the development of a video summarisation study. The most important moments of a longer input video are selected to minimise its length while maintaining context. Video summary using keyframe extraction outputs key frames of key events. Its uses include anomaly detection, efficient video storage, indexing, and retrieval. Extracting semantically meaningful frames is difficult, and a big research gap exists. The keyframe extraction using deep reinforcement learning (KE_DRL) approach extracts representative and distinct semantically relevant keyframes. Frame-level and video-level characteristics are extracted. Frame-level characteristics are extracted using modified ResNet50 and I3Dnet. They are aggregated to generate a feature vector and global average pooled to get video-level features. attention-based video summariser network (AVSumnet) uses semantic video attributes as input. It is trained via reinforcement learning. A new summariser network training reward mechanism is proposed. Experimental findings show that the KE_DRL method creates better video keyframes than existing methods.

Keywords: keyframe extraction; video summarisation; bi-directional gated recurrent unit; BDGRU; deep reinforcement learning; attention mechanism; squeeze and excitation block; policy gradient; reward function; representativeness.

DOI: 10.1504/IJESMS.2025.149566

International Journal of Engineering Systems Modelling and Simulation, 2025 Vol.16 No.6, pp.354 - 368

Received: 20 Feb 2024
Accepted: 01 Jan 2025
Published online: 07 Nov 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article