Title: Pyramid hierarchical network for multispectral pan-sharpening

Authors: Zenglu Li; Xiaoyu Guo; Songyang Xiang; Xiaohua Wu

Addresses: Network Center (Information Construction Office), Sanming University, 25 Jingdong Road, Sanming, Fujian, 365001, China ' School of Resources and Chemical Engineering, Sanming University, 25 Jingdong Road, Sanming, Fujian, 365001, China ' Geography and Ecological Environment Research Center, Fuzhou University, 2 Gakyuan Road, Fuzhou, Fujian, 353000, China ' School of Art and Design, Sanming University, 25 Jingdong Road, Sanming, Fujian, 365001, China

Abstract: Pan-sharpening aims to fuse high spatial-resolution panchromatic images (PAN) and low spatial-resolution multispectral images (MS) into high spatial-resolution multispectral images (HRMS).We propose a pyramid hierarchical multi-spectral fusion network, called PH-Net which can automatically fuse MS images and PAN images to generate corresponding HRMS images. The architecture is based on the U-Net network. First, a multi-level receptive field is realised by constructing an input pyramid. Then, hierarchical features are extracted from the encoder, decoder, and input pyramid. Finally, the rich hierarchical features are used to calculate the residual error between the MS image and the corresponding HRMS image. The learned residual error is inserted into the MS image to obtain the final high spatial-resolution multispectral image. To demonstrate the effectiveness of each component in the network architecture, we conducted an ablation study. In addition, thanks to the design of the multi-layer architecture, model training does not require a large dataset, which greatly improves the training speed and significantly improves the generalisability and ease of deployment of this work in the field of remote sensing images. Through qualitative and quantitative experiments, we proved that the proposed method is superior to current advanced methods.

Keywords: pan-sharpening; image fusion; pyramid attention; multispectral image; deep learning.

DOI: 10.1504/IJCSE.2024.137282

International Journal of Computational Science and Engineering, 2024 Vol.27 No.2, pp.142 - 158

Received: 20 Jul 2022
Received in revised form: 14 Sep 2022
Accepted: 24 Nov 2022
Published online: 11 Mar 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article