Title: Dynamic power management strategies for a sensor node optimised by reinforcement learning

Authors: Lei Zhou; Hao Tang; Huizi Li; Qi Jiang

Addresses: School of Computer and Information, Hefei University of Technology, Hefei 230009, China ' School of Computer and Information, Hefei University of Technology, Hefei 230009, China; School of Electrical Engineering and Automation, Hefei University of Technology, Hefei 230009, China ' School of Computer and Information, Hefei University of Technology, Hefei 230009, China ' School of Electrical Engineering and Automation, Hefei University of Technology, Hefei 230009, China

Abstract: Dynamic power management (DPM) is considered in wireless sensor networks for improving the energy efficiency of sensor nodes. DPM usually includes two classical problems, dynamic operating mode (OM) management and adaptive transmission mechanism. In this paper, we propose a new model of Markov decision process that combines dynamic OM management and adaptive transmission mechanism. In addition, a fragment transmission scheme is further integrated to reduce the probability of retransmission failure and improve the data transmission rate. The model takes into account the performance criteria being the expected cost of synthesising the per-packet energy consumption, the buffer overflow, the fragment cost and the energy consumption of operating mode switching. Reinforcement learning algorithm is subsequently proposed to search for the optimal strategies. Furthermore, a state-clustering approach is given to increase the learning speed and lessen the storage requirements. Finally, an example is presented to illustrate the effectiveness of the proposed method and show that the energy consumption is well-balanced dynamically under the optimised policy, while the throughput decreases only slightly. Therefore, the lifetime of the node can be extended under constrained resources.

Keywords: sensor nodes; dynamic power management; DPM; Markov decision process; MDP; fragment transmission; reinforcement learning; wireless sensor networks; WSNs; energy efficiency; adaptive transmission; buffer overflow; fragment cost; energy consumption; operating mode switching; state clustering.

DOI: 10.1504/IJCSE.2016.077730

International Journal of Computational Science and Engineering, 2016 Vol.13 No.1, pp.24 - 37

Received: 14 Aug 2013
Accepted: 27 Nov 2013

Published online: 14 Jul 2016 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article