Article: <i>𝒟af</i>VBM: multimodal deep learning for predicting efficacy of salvage chemoradiotherapy Journal: International Journal of Bio-Inspired Computation (IJBIC) 2025 Vol.26 No.3 pp.181 - 192 Abstract: This study leverages multimodal deep learning to predict the efficacy of salvage chemoradiotherapy for esophageal squamous cell carcinoma. The target patients are those with regional lymph node recurrence after curative resection. Predicting efficacy is challenging due to limited data and the reliance on manual feature selection. Integrating multimodal data and optimising its utilisation can significantly support treatment prediction. However, there is little research on fully integrating multimodal data. To address this, we conduct a retrospective study and design a multimodal architecture to process patient data, including images, text, and tabular features. The model employs attention mechanisms and Fourier transform paths for seamless feature fusion. Our comparative analysis shows significant performance improvements with multimodal data, achieving an average precision, recall, and F1-score of 91.96%, 91.18%, and 90.86%, with best scores of 97.20%, 97.10%, and 97.00%. Inderscience Publishers - linking academia, business and industry through research

Title: 𝒟afVBM: multimodal deep learning for predicting efficacy of salvage chemoradiotherapy

Authors: Han Zhang; Xinwei Guo; Liang Gu; Zhenyu Lei; Masaaki Omura; Shangce Gao

Addresses: Faculty of Engineering, University of Toyama, Toyama, Japan; Nanjing Normal University, Taizhou College, Taizhou, China ' Department of Radiation Oncology, Affiliated Taixing People's Hospital of Yangzhou University, Taixing, China ' Department of Radiation Oncology, Affiliated Taixing People's Hospital of Yangzhou University, Taixing, China ' Faculty of Engineering, University of Toyama, Toyama, Japan ' Faculty of Engineering, University of Toyama, Toyama, Japan ' Faculty of Engineering, University of Toyama, Toyama, Japan

Abstract: This study leverages multimodal deep learning to predict the efficacy of salvage chemoradiotherapy for esophageal squamous cell carcinoma. The target patients are those with regional lymph node recurrence after curative resection. Predicting efficacy is challenging due to limited data and the reliance on manual feature selection. Integrating multimodal data and optimising its utilisation can significantly support treatment prediction. However, there is little research on fully integrating multimodal data. To address this, we conduct a retrospective study and design a multimodal architecture to process patient data, including images, text, and tabular features. The model employs attention mechanisms and Fourier transform paths for seamless feature fusion. Our comparative analysis shows significant performance improvements with multimodal data, achieving an average precision, recall, and F1-score of 91.96%, 91.18%, and 90.86%, with best scores of 97.20%, 97.10%, and 97.00%.

Keywords: multimodal model; feature fusion; efficacy prediction.

DOI: 10.1504/IJBIC.2025.149561

International Journal of Bio-Inspired Computation, 2025 Vol.26 No.3, pp.181 - 192

Received: 21 Oct 2024
Accepted: 09 Dec 2024
Published online: 07 Nov 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: 𝒟afVBM: multimodal deep learning for predicting efficacy of salvage chemoradiotherapy

Keep up-to-date