PromptHSI:

Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation

Chia-Ming Lee1, Ching-Heng Cheng1, Yu-Fan Lin1, Yi-Ching Cheng1, Wo-Ting Liao1, Chih-Chung Hsu1, Fu-En Yang2,3, Yu-Chiang Frank Wang2,3

Institute of Data Science, National Cheng Kung University1

Graduate Institute of Communication Engineering, National Taiwan University2 NVIDIA3

arXiv Supplementary Code Dataset Model

Performance comparison between All-in-One and HSI restoration methods on the AVIRIS dataset. The proposed PromptHSI offers leading performance and controllability while retaining a compact model size.

Abstract

Recent advances in All-in-One (AiO) RGB image restoration have demonstrated the effectiveness of prompt learning in handling multiple degradations within a single model. However, extending these approaches to hyperspectral image (HSI) restoration is challenging due to the domain gap between RGB and HSI features, information loss in visual prompts under severe composite degradations, and difficulties in capturing HSI-specific degradation patterns via text prompts. In this paper, we propose PromptHSI, the first universal AiO HSI restoration framework that addresses these challenges. By incorporating frequency-aware feature modulation—which utilizes frequency analysis to narrow down the restoration search space—and employing vision-language model (VLM)-guided prompt learning, our approach decomposes text prompts into intensity and bias controllers that effectively guide the restoration process while mitigating domain discrepancies. Extensive experiments demonstrate that our unified architecture excels at both fine-grained recovery and global information restoration across diverse degradation scenarios, highlighting its significant potential for practical remote sensing applications. The source code will be made publicly available.

A diagram illustrating our proposed strategy, where integrating prompt guidance with frequency modulation facilitates the adaptive selection of λ and μ, enhancing HSI restoration under various degradation types. This mechanism encourages the model to efficiently converge toward optimal parameters.

The architecture of the proposed PromptHSI, which is designed to tackle the composite degradations within a single universal model.

Visualization results for HSI restoration methods. top: the reconstructed HSI, bottom: the residual image derived from Ground-truth and reconstructed result. Most of state-of-the-arts HSI restoration methods fail to deal with composite degradations.

Overall performance evaluation for composite degradation and complexity comparison.

If you find our work helpful, please consider citing the following:
BibTeX
      @misc{lee2025prompthsiuniversalhyperspectralimage,
        title={PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation}, 
        author={Chia-Ming Lee and Ching-Heng Cheng and Yu-Fan Lin and Yi-Ching Cheng and Wo-Ting Liao and Fu-En Yang and Yu-Chiang Frank Wang and Chih-Chung Hsu},
        year={2025},
        eprint={2411.15922},
        archivePrefix={arXiv},
        primaryClass={eess.IV},
        url={https://arxiv.org/abs/2411.15922}, 
      }