Title: Soft actor-critic with automatically adjusted entropy for autonomous exploring in unknown environments

Authors: Walid Jebrane; Nabil El Akchioui

Addresses: Faculty of Science and Technology, University Abdelmalek Essaadi, Al Hoceima – Tetouan, Morocco ' Faculty of Science and Technology, University Abdelmalek Essaadi, Al Hoceima – Tetouan, Morocco

Abstract: This paper presents an in-depth analysis of the soft actor-critic (SAC) algorithm, specifically focusing on its automatically adjusted temperature (alpha) variant, within the context of autonomous navigation in unexplored environments. We developed an exploration system that integrates the SAC-alpha algorithm to autonomously navigate and map unfamiliar terrains, employing a strategic waypoint selection mechanism to optimise movement toward target objectives. The SAC-alpha framework was customised and enhanced to address the challenges posed by the lack of pre-existing environmental maps and the presence of complex obstacles. The efficacy of SAC-alpha was rigorously evaluated in simulated environments, utilising metrics such as success rate, navigation efficiency, and obstacle avoidance performance. Results from these evaluations demonstrate that SAC-alpha outperforms the twin delayed deep deterministic policy gradient (TD3) algorithm, exhibiting superior capability in overcoming local optima, efficiently navigating complex environments, and ensuring safety throughout the navigation process.

Keywords: autonomous robot navigation; deep reinforcement learning; DRL; soft actor-critic; SAC; automatically adjusted entropy; exploring unknown environments.

DOI: 10.1504/IJVP.2025.144279

International Journal of Vehicle Performance, 2025 Vol.11 No.1, pp.79 - 104

Received: 25 Apr 2024
Accepted: 15 Oct 2024
Published online: 04 Feb 2025 *