Title: Bootstrapping learning from abstract models in games

Authors: Purvag Patel; Normal Carver; Shahram Rahimi

Addresses: Department of Computer Science, Southern Illinois University, Carbondale, IL, 62901, USA ' Department of Computer Science, Southern Illinois University, Carbondale, IL, 62901, USA ' Department of Computer Science, Southern Illinois University, Carbondale, IL, 62901, USA

Abstract: Computer gaming environments are real time, dynamic, and complex, with incomplete knowledge of the world. Agents in such environments require detailed models of the world if they are to learn effective policies. Machine learning techniques such as reinforcement learning can become intractably large, detailed world models. In this paper we tackle the well-known problem of low convergence speed in reinforcement learning for the detailed model of the world, specifically for video games. We propose first training the agents with an abstract model of the world and then using the resulting policy to initialise the system prior to training the agent with the detailed model of the world. This paper reports on results from applying the proposed technique to the classic arcade game Asteroids. Our experiments show that an agent can quickly learn a policy with the abstract model, and that when this policy's learned values are used to initialise the detailed model, learning with the detailed model improves the rate of convergence.

Keywords: Q-learning; reinforcement learning; computer games; bots; convergence rate; bootstrapping; abstract models; modelling; video games; gaming; arcade games; Asteroids; learning agents; multi-agent systems; MAS; agent-based systems.

DOI: 10.1504/IJBIC.2013.055452

International Journal of Bio-Inspired Computation, 2013 Vol.5 No.4, pp.239 - 251

Received: 17 Nov 2012
Accepted: 20 Nov 2012
Published online: 31 Mar 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article