Title: Efficient master production scheduling for manufacturing systems using an enhanced SARSA algorithm

Authors: Yuehan Liu; Haibo Shi; Chang Liu

Addresses: Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang – 110016, Liaoning, China; University of Chinese Academy of Sciences, Beijing – 100049, China ' Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang – 110016, Liaoning, China ' Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang – 110016, Liaoning, China

Abstract: Efficient master production scheduling is crucial in production planning, yet automated solutions remain scarce. This paper introduces a novel scheduling method, master production scheduling with improved state-action-reward-state-action (MPS-ISARSA) algorithm, based on an enhanced state-action-reward-state-action (SARSA) framework. Using a Markov decision process model, the method optimises scheduling through innovative reward shaping and a linearly decaying epsilon-greedy (ε_lin-greedy) policy to accelerate training. An overtime penalty ensures alignment between production capacity and demand. Extensive simulations validate the model's effectiveness, demonstrating superior convergence and efficiency compared to traditional methods and other reinforcement learning algorithms. This approach offers a scalable, intelligent framework for capacity-constrained production scheduling, with practical applications for manufacturing industries aiming to enhance operational efficiency.

Keywords: reinforcement learning; SARSA algorithm; production scheduling; master production scheduling; reward shaping; Markov decision process; capacity-constrained scheduling.

DOI: 10.1504/IJSPM.2025.148294

International Journal of Simulation and Process Modelling, 2025 Vol.22 No.1/2, pp.60 - 74

Received: 14 Dec 2024
Accepted: 25 Apr 2025
Published online: 01 Sep 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article