PyTorch codes for "Satellite Video Super-resolution via Multi-Scale Deformable Convolution Alignment and Temporal Grouping Projection", IEEE Transactions on Geoscience and Remote Sensing (TGRS), 2022.

Authors: Yi Xiao, Xin Su, Qiangqiang Yuan*, Denghong Liu, Huanfeng Shen, and Liangpei Zhang
Wuhan University

As a new earth observation tool, satellite video has been widely used in remote-sensing field for dynamic analysis. Video super-resolution (VSR) technique has thus attracted increasing attention due to its improvement to spatial resolution of satellite video. However, the difficulty of remote-sensing image alignment and the low efficiency of spatial–temporal information fusion make poor generalization of the conventional VSR methods applied to satellite videos. In this article, a novel fusion strategy of temporal grouping projection and an accurate alignment module are proposed for satellite VSR. First, we propose a deformable convolution alignment module with a multiscale residual block to alleviate the alignment difficulties caused by scarce motion and various scales of moving objects in remote-sensing images. Second, a temporal grouping projection fusion strategy is proposed, which can reduce the complexity of projection and make the spatial features of reference frames play a continuous guiding role in spatial–temporal information fusion. Finally, a temporal attention module is designed to adaptively learn the different contributions of temporal information extracted from each group. Extensive experiments on Jilin-1 satellite video demonstrate that our method is superior to current state-of-the-art VSR methods.

git clone https://github.com/XY-boy/MSDTGP.git

• CUDA 10.0
• pytorch 1.x
• build DCNv2

Please download our dataset in

• Baidu Netdisk Jilin-189 Code:31ct
• Zenodo:

You can also train your dataset following the directory sturture below!

trainset--
 | train--
  | LR4x---
   | 000.png
   | ···.png
   | 099.png
  | GT---
  | Bicubic4x---

testset--
 | eval--
  | LR4x---
   | 000.png
   | ···.png
   | 099.png
  | GT---
  | Bicubic4x---

python main.py

python eval.py

If you have any questions or suggestions, feel free to contact me. 😊
Email: [email protected]; [email protected]

If you find our work helpful in your research, please consider citing it. Thank you! 😊😊

@ARTICLE{xiao2022msdtgp,  
author={Xiao, Yi and Su, Xin and Yuan, Qiangqiang and Liu, Denghong and Shen, Huanfeng and Zhang, Liangpei},  
journal={IEEE Transactions on Geoscience and Remote Sensing},  
title={Satellite Video Super-Resolution via Multiscale Deformable Convolution Alignment and Temporal Grouping Projection},   
year={2022},  
volume={60},  
number={},  
pages={1-19},  
doi={10.1109/TGRS.2021.3107352}
}

Our work is built upon RBPN and TDAN.
Thanks to the author for the source code !