Title: A welding manipulator path planning method combining reinforcement learning and intelligent optimisation algorithm

Authors: Junhua Zhang; Lianglun Cheng; Tao Wang; Wenya Xia; Dejun Yan; Zhiheng Wu; Xianyun Duan

Addresses: Guangdong University of Technology, Guangzhou City, 510000, Guangdong, China ' Guangdong University of Technology, Guangzhou City, 510000, Guangdong, China ' Guangdong University of Technology, Guangzhou City, 510000, Guangdong, China ' Huangpu Wenchong Shipbuilding Company Ltd., Guangzhou City, 510715, Guangdong, China ' Huangpu Wenchong Shipbuilding Company Ltd., Guangzhou City, 510715, Guangdong, China ' Guangdong Institute of Intelligent Manufacturing, Guangzhou City, 510070, Guangdong, China ' Guangdong Institute of Intelligent Manufacturing, Guangzhou City, 510070, Guangdong, China

Abstract: We present DDPG-AACO, a hierarchical method for welding manipulator path planning of complex components welding tasks that combines reinforcement learning (RL) with the intelligent optimisation algorithm. The RL agent, trained with the deep deterministic policy gradient (DDPG), learns local path planning policies that control the welding manipulator to safely move between two welding seam endpoints. Next, on a distance matrix constructed by the lengths of local paths between every two welding seam endpoints, the adaptive ant colony optimisation (AACO) algorithm with artificially changed value of pheromones is adopted to realise the global path planning that the welding manipulator traverses all welding seams under a welding direction constraint and has the shortest path length. The simulation results show the effectiveness of the method. The DDPG is better than the deep Q-learning-based methods when performing local path planning. Moreover, the length of the global path with direction constraint can converge to the minimum.

Keywords: complex component; welding manipulator; path planning; direction constraint; deep deterministic policy gradient; DDPG; adaptive ant colony optimisation; AACO.

DOI: 10.1504/IJMIC.2019.105972

International Journal of Modelling, Identification and Control, 2019 Vol.33 No.3, pp.261 - 270

Received: 26 Jan 2019
Accepted: 05 Jul 2019

Published online: 18 Mar 2020 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article