Title: Availability and performability analysis for a service degradation process with condition-based preventive maintenance II - estimation and simulation

Authors: Tadashi Dohi

Addresses: Department of Information Engineering, Hiroshima University, 1-4-1 Kagamiyama, Higashi-Hiroshima 739-8527, Japan

Abstract: The preventive maintenance is very useful to improve effectively the service availability for software systems with service degradation. In this paper, we present a stochastic model to describe an operational software, which consists of one operating system and multiple applications and provides a service in continuous time. In this paper we examine adaptive estimation problems using a reinforcement learning algorithm which is called the Q-learning. We give the Q-learning algorithms to estimate the optimal software rejuvenation policies that minimise the service availability and maximise the expected service reward per unit time, respectively. Illustrative numerical examples are presented to investigate asymptotic properties of estimators of the optimal software rejuvenation policies.

Keywords: software services; service degradation; software rejuvenation; condition-based maintenance; semi-Markov decision process; optimality; preventive maintenance; service availability; software systems; statistics; adaptive estimation; reinforcement learning.

DOI: 10.1504/IJSEAM.2014.063880

International Journal of Strategic Engineering Asset Management, 2014 Vol.2 No.1, pp.98 - 115

Published online: 30 Aug 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article