Title: Optimising healthcare process efficiency with HyperBERT: embedding ICD code hierarchy for national healthcare systems simulation and improvement

Authors: Dereje Weyessa; Berhanu Beyene; Teklu Urgessa; Tiruveeduula Gopikirishna; Satyasis Mishra; Bijay Kumar Paikaray

Addresses: Department of Computer Science and Engineering, Adama Science and Technology University, Adama, Ethiopia ' President of Ethiopian Cybersecurity Association, Civil Service University of Ethiopia, Ababa, Ethiopia ' Department of Computer Science and Engineering, Adama Science and Technology University, Adama, Ethiopia ' Department of Computer Science and Engineering, Adama Science and Technology University, Adama, Ethiopia ' Department of ECE, Centurion University of Technology and Management, Odisha, India ' Centre for Data Science, Department of Computer Science and Engineering, Siksha 'O' Anusandhan (Deemed to be University), Bhubaneswar, Odisha, India

Abstract: This research study focuses on embedding the hierarchical structure of the International Classification of Diseases (ICD) codes for automatic code assignment in clinical texts. It explores: 1) incorporating diagnosis code hierarchy as auxiliary knowledge in a BERT training process; 2) how transformer-based models like BERT can serve as interpretable AI in Ethiopia's National Classification of Diseases (NCOD) and the MIMIC dataset. Utilising qualitative methods, including experimental and case study strategies, results indicate that a new model architecture called HyperBERT improved the macro average AUC by 0.003 and F1-score by 0.02. The model's interpretability was demonstrated through a code-wise label attention mechanism using the bidirectional encoder representation for transformer (BERT) visualisation tool. Riemannian optimisation and Poincaré loss were employed to validate a new document representation model combining BERT with hyperbolic embeddings, enabling effective ICD code predictions from discharge summaries in Ethiopia's healthcare system.

Keywords: ICD coding; interpretable AI; code hierarchical; deep neural network; bidirectional encoder representation for transformer; BERT; hyperbolic embedding; healthcare systems.

DOI: 10.1504/IJSPM.2025.148299

International Journal of Simulation and Process Modelling, 2025 Vol.22 No.1/2, pp.1 - 17

Received: 10 Dec 2024
Accepted: 27 Mar 2025
Published online: 01 Sep 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article