Title: Collaborative optimisation of lane change decision and trajectory based on double-layer deep reinforcement learning

Authors: Yang Shen; Jiankun Peng; Cheng Kang; Siyu Zhang; Fengyan Yi; Jinming Wan; Xingmao Wang

Addresses: School of Transportation, Southeast University, Nanjing, 211100, China; School of Automotive Engineering, Shandong JiaoTong University, Jinan, 250023, China ' School of Transportation, Southeast University, Nanjing, 211100, China ' CATARC Automotive Proving Ground Co., Ltd., Yancheng, 224100, China ' School of Transportation, Southeast University, Nanjing, 211100, China ' School of Automotive Engineering, Shandong JiaoTong University, Jinan, 250023, China ' CATARC Automotive Proving Ground Co., Ltd., Yancheng, 224100, China ' School of Automotive Engineering, Shandong JiaoTong University, Jinan, 250023, China

Abstract: In this paper, lane change process of vehicle is divided into two stages: lane change decision and lane change movement, and put forward the double-layer deep reinforcement learning architecture, the use of upper structure deep Q network (DQN) exchange way to control decision, and send the lane changing information to the lower deep deterministic policy gradient (DDPG) of vehicle trajectory control, After the lane change process the collaborative optimisation of DQN was completed through the feedback of vehicle position information before and after lane change., the collaborative optimisation of DQN is completed. The results show that, the proposed two-layer deep reinforcement learning architecture can increase the average velocity of the agent vehicle by 2-5% and reduce the average lateral speed and lateral acceleration by 12.5% and 12.2% respectively in the lane-changing process. Compared with no collaborative optimisation, the optimal lane change timing of the effectively cooptimised two-layer architecture is 34.64%.

Keywords: deep reinforcement learning; lane change decision; lane change action; collaborative optimisation; deep reinforcement learning; lane change decision; lane change action.

DOI: 10.1504/IJVD.2023.134751

International Journal of Vehicle Design, 2023 Vol.92 No.2/3/4, pp.336 - 356

Received: 22 Mar 2022
Received in revised form: 28 Nov 2022
Accepted: 21 Dec 2022
Published online: 09 Nov 2023 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article