Title: Integrated decision-making methodology based on reinforcement learning and imitation learning for automated commercial vehicles in the urban traffic environment

Authors: Weiming Hu; Jinchao Hu; Yan Liu; Jinying Zhou; Mengyue Su

Addresses: Vehicle Evaluation Research Center, China Automotive Engineering Research Institute Co. Ltd., Yubei, Chongqing, 401122, China; School of transportation, Southeast University, 2 Southeast University Road, Jiangsu Province, 211189, China ' School of Information Engineering Ningxia University, No. 489, West Helanshan Road, Xixia District, Yinchuan, 750021, China ' Vehicle Evaluation Research Center, China Automotive Engineering Research Institute Co. Ltd., Yubei, Chongqing 401122, China ' Vehicle Evaluation Research Center, China Automotive Engineering Research Institute Co. Ltd., Yubei, Chongqing 401122, China ' Vehicle Evaluation Research Center, China Automotive Engineering Research Institute Co. Ltd., Yubei, Chongqing 401122, China

Abstract: Driving decision-making plays an important role in determining the rationality and safety of automated commercial vehicles. Unlike extensively studied passenger vehicles, commercial vehicles exhibit longer braking distances, inferior roll stability, and increased visual blind spots. To achieve safe automated driving, with due consideration for low carbonisation and energy conservation, we present an integrated driving decision-making methodology based on multi-head attention (IDDM-MA). The IDDM-MA model comprises two interconnected networks. Initially, the soft actor-critic algorithm is employed to acquire driving strategies in hazardous scenarios via unsupervised learning. Subsequently, generative adversarial imitation learning is applied to simulate safe driving manoeuvres modelled after human drivers. Finally, the IDDM-MA model is trained and evaluated within the simulation of urban mobility (SUMO) environment. Experimental outcomes reveal that our proposal ensures reasonable clearances and lateral acceleration when the perception system detects obstacles within blind areas. Simultaneously, it demonstrates lower carbon emission intensity and improved fuel economy.

Keywords: automated commercial vehicle; deep reinforcement learning; imitation learning; safe driving decision-making; traffic simulation.

DOI: 10.1504/IJVSMT.2024.142150

International Journal of Vehicle Systems Modelling and Testing, 2024 Vol.18 No.3, pp.245 - 266

Received: 13 Dec 2023
Accepted: 15 Jun 2024
Published online: 10 Oct 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article