Article: Transformer-based infrared and visible image fusion with language-driven loss Journal: International Journal of Ad Hoc and Ubiquitous Computing (IJAHUC) 2025 Vol.49 No.3 pp.143 - 152 Abstract: Infrared and visible image fusion aims to generate a fused image that highlights the target and encompasses detailed textural information. Deep learning-based image fusion algorithm enable enhanced multimodal sensing and decision-making across diverse environments. However, existing algorithms often focus only on visual quality while neglecting the image's semantic content. To address these issues, we propose an image fusion method that leverages the global feature extraction capabilities of the transformer model and optimises the fusion process through a loss function guided by the contrastive language-image pre-training (CLIP). Firstly, we develop a feature-guided transformer (FGT) module to extract and interact with local and global information in images. Subsequently, a feature dynamic fusion (FDF) module is designed to fuse the image adaptively from two different modalities. Comprehensive experiments conducted on three public datasets demonstrate that the proposed method outperforms state-of-the-art (SOTA) fusion methods subjectively and objectively. Inderscience Publishers - linking academia, business and industry through research

Title: Transformer-based infrared and visible image fusion with language-driven loss

Authors: Qihao Liu; Fei Lu; Ningxin Wang; Jungan Zhang; Yuanchao Hu

Addresses: State Grid Shandong Electric Power Company Huantai Power Supply Company, Zibo, Shandong, China ' State Grid Shandong Electric Power Company Huantai Power Supply Company, Zibo, Shandong, China ' State Grid Shandong Electric Power Company Huantai Power Supply Company, Zibo, Shandong, China ' State Grid Shandong Electric Power Company Huantai Power Supply Company, Zibo, Shandong, China ' Shandong University of Technology, Zibo, Shandong, China

Abstract: Infrared and visible image fusion aims to generate a fused image that highlights the target and encompasses detailed textural information. Deep learning-based image fusion algorithm enable enhanced multimodal sensing and decision-making across diverse environments. However, existing algorithms often focus only on visual quality while neglecting the image's semantic content. To address these issues, we propose an image fusion method that leverages the global feature extraction capabilities of the transformer model and optimises the fusion process through a loss function guided by the contrastive language-image pre-training (CLIP). Firstly, we develop a feature-guided transformer (FGT) module to extract and interact with local and global information in images. Subsequently, a feature dynamic fusion (FDF) module is designed to fuse the image adaptively from two different modalities. Comprehensive experiments conducted on three public datasets demonstrate that the proposed method outperforms state-of-the-art (SOTA) fusion methods subjectively and objectively.

Keywords: image fusion; language-vision pre-training; transformer; IoT network.

DOI: 10.1504/IJAHUC.2025.147567

International Journal of Ad Hoc and Ubiquitous Computing, 2025 Vol.49 No.3, pp.143 - 152

Received: 24 Oct 2024
Accepted: 26 Dec 2024
Published online: 21 Jul 2025 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Transformer-based infrared and visible image fusion with language-driven loss

Keep up-to-date