Title: Constitution of Ms.PacMan player with critical-situation learning mechanism

Authors: Hisashi Handa

Addresses: Graduate School of Natural Science and Technology, Okayama University, Okayama, 700-8530, Japan

Abstract: We have previously proposed evolutionary fuzzy systems of playing Ms.PacMan for the competitions. As a consequence of the evolution, reflective action rules such that PacMan tries to eat pills effectively until ghosts come close to PacMan are acquired. Such rules work well. However, sometimes it is too reflective so that PacMan goes toward ghosts by herself in longer corridors. In this paper, a critical situation learning module is combined with the evolved fuzzy systems, i.e., reflective action module. The critical situation learning module is composed of Q-learning with CMAC. Location information of surrounding ghosts and the existence of power-pills are given to PacMan as state. This module punishes if the PacMan is caught by ghosts. Therefore, this module learning which pairs of (state, action) causes her death. By using learnt Q-value, PacMan tries to survive much longer. Experimental results on Ms.PacMan elucidate the proposed method is promising since it can capture critical situations well. However, as a consequence of the large amount of memory required by CMAC, real-time responses tend to be lost.

Keywords: game players; CMAC; Q-learning; critical situation learning; fuzzy logic; gaming; Ms.PacMan; reflective action; game competitions; computer games; evolutionary fuzzy systems; PacMan.

DOI: 10.1504/IJKESDP.2010.035906

International Journal of Knowledge Engineering and Soft Data Paradigms, 2010 Vol.2 No.3, pp.237 - 250

Published online: 08 Oct 2010 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article