Title: Multi-task deep neural network models for learning COVID-19 disease representations from multimodal data

Authors: Veena Mayya; K. Karthik; Krishnananda Prabhu Karadka; S. Sowmya Kamath

Addresses: Healthcare Analytics and Language Engineering (HALE) Lab, Department of Information Technology, National Institute of Technology Karnataka, Surathkal, Mangalore, 575025, India; Department of Information and Communication Technology, Manipal Institute of Technology, Manipal Academy of Higher Education (MAHE), Manipal, 576104, India ' Healthcare Analytics and Language Engineering (HALE) Lab, Department of Information Technology, National Institute of Technology Karnataka, Surathkal, Mangalore, 575025, India ' Penzigo Technology Solutions Pvt. Ltd., NITK-Science and Technology Entrepreneurs' Park (STEP), NITK Surathkal, India ' Healthcare Analytics and Language Engineering (HALE) Lab, Department of Information Technology, National Institute of Technology Karnataka, Surathkal, Mangalore, 575025, India

Abstract: Over the continued course of the COVID-19 pandemic, a significant volume of expert-written diagnosis reports has been accumulated that capture a multitude of symptoms and observations on diagnosed COVID-19 cases, along with expert-validated chest X-ray scans. The utility of rich, latent information embedded in such unstructured expert-written diagnosis reports and its importance as a source of valuable disease-specific information has been explored to a very limited extent. In this work, a convolutional attention-based dense (CAD) neural model for COVID-19 prediction is proposed. The model is trained on the rich disease-specific parameters extracted from chest X-ray images and expert-written diagnostic text reports to support an evidence-based diagnosis. Scalability is ensured by incorporating content based learning models for automatically generating diagnosis reports of identified COVID-19 cases, reducing radiologists' cognitive burden. Experimental evaluation showed that multimodal patient data plays a vital role in diagnosing early-stage cases, thus helping hasten the diagnosis process.

Keywords: COVID-19 diagnosis; clinical decision support systems; multi-task learning; healthcare analytics; deep convolution neural networks.

DOI: 10.1504/IJMEI.2023.134537

International Journal of Medical Engineering and Informatics, 2023 Vol.15 No.6, pp.501 - 515

Received: 09 Mar 2021
Accepted: 24 Jul 2021
Published online: 27 Oct 2023 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article