Title: Control of multi-legged robot using reinforcement learning with body image and application to rubble

Authors: Kazuya Nishigai; Hiroki Nakatsuka; Kazuyuki Ito

Addresses: Suzuki Motor Corporation, 300 Takatsuka-cho, Minami-ku, Hamamatu-shi, Shizuoka, 432-8611, Japan ' Fujitsu Frontech, 1776 Yanokuchi, Inagi-shi, Tokyo, 206-8555, Japan ' Department of Electrical and Electronics Engineering, Hosei University, 3-7-2, Kajinocho, Koganeishi, Tokyo, 184-8584, Japan

Abstract: In this paper, we address the autonomous control of a 6-legged robot using reinforcement learning, and apply it to rubbles. In conventional framework of reinforcement learning, many degrees of freedom of the 6-legged robot cause the state explosion problem, and it prevents the robot from real-time learning. Furthermore, even if learning process is conducted in simulation, the acquired policy is effective only for the simple simulated environment, and therefore, it is impossible to apply the acquired policy to unknown complex environment such as rubbles. To solve these problems, we focus on the flexibility of the body and its body image. The body image solves the state explosion problem and the flexibility of the body compensates the difference between the simple simulated environment and the complex real environment. We developed a prototype of the robot and conducted experiments to confirm the effectiveness and the validity of the proposed framework. As a result, effective locomotion was realised.

Keywords: multi-legged robots; legged locomotion; walking robots; body image; reinforcement learning; genetic algorithms; robot control; rubble; state explosion problem; flexible robots.

DOI: 10.1504/IJAMECHS.2013.057664

International Journal of Advanced Mechatronic Systems, 2013 Vol.5 No.4, pp.243 - 256

Published online: 12 Jul 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article