Title: A practical heterogeneous network optimisation algorithm based on reinforcement learning

Authors: Zhiyong Feng, Li Tan, Wei Li, T. Aaron Gulliver, Litao Liang

Addresses: Key Lab of Universal Wireless Communications, Beijing University of Posts and Telecommunications, 10 Xitucheng Road, Haidian District, Beijing, China. ' Key Lab of Universal Wireless Communications, Beijing University of Posts and Telecommunications, 10 Xitucheng Road, Haidian District, Beijing, China. ' School of Engineering and Computer Science, Victoria University of Wellington, P.O. 600, Wellington 6140, New Zealand. ' Department of Electrical and Computer Engineering, University of Victoria, P.O. Box 3055, STN CSC, Victoria, BC V8W 3P6, Canada. ' China Mobile Group Beijing Co., Ltd., No. 7 Dongzhimen South Street, Dongcheng District, Beijing, China

Abstract: Owing to the increasing and diversifying service requirements of wireless communications, wireless networks must coexist with heterogeneous radio systems. To realise the interconnection between different networks, it is important for the radio access network elements, such as the cellular network base stations (BSs) and the wireless local area network (WLAN) access points (APs) to be reconfigurable based on the real-time network environment. In this paper, we propose an efficient distributed reconfiguration algorithm for heterogeneous networks: the dynamic network self-optimisation algorithm (DNSA). This algorithm is based on the Q-learning algorithm and the self-optimisation of each network entity acting as independent agents. In the proposed algorithm, multiple agents perform the optimisation cooperatively to reduce the system blocking rate and improve network revenue. The dynamic network self-optimisation problem is transformed into a multiple-agent reinforcement learning problem which has much lower complexity and better performance.

Keywords: reconfigurable systems; Q-learning; reinforcement learning; multi-agent systems; agent-based systems; self-optimisation; universal mobile telecommunications; UMTS; wireless LANs; local area networks; WLAN; network optimisation; system blocking rate; network revenue.

DOI: 10.1504/IJCNDS.2011.040558

International Journal of Communication Networks and Distributed Systems, 2011 Vol.6 No.4, pp.357 - 372

Published online: 05 Jun 2011 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article