Updates:
-
April 26, 2024: We released a new model trained on Vimeo-90K with DB degradation.
-
January 18, 2024: Check our updated paper with more illustrations, speed comparisons and additional qualitative results!
This is an offical PyTorch implementation of
[CVPR 2024 Highlight] Enhancing Video Super-Resolution via Implicit Resampling-based Alignment.
Kai Xu, Ziwei Yu, Xin Wang, Michael Bi Mi, Angela Yao
Computer Vision and Machine Learning group, NUS.
We test the code for python-3.9, torch-1.13.1 and cuda-11.7. Similar versions will also work.
conda create -n IART python==3.9
conda activate IART
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
pip install -r requirements.txtTo prepare the dataset, follow BasicSR. After completing the preparation, the directory structure should be as follows:
datasets/
├──REDS/
│ └──val_REDS4_sharp
│ └──val_REDS4_sharp_bicubic
Download models from this link and put them under experiments/.
# Run demo on frames under `demo/Vid4_BI`:
python demo.py
# Testing:
# For model trained on REDS dataset with BI degradation.
python test_scripts/BI/REDS/test_IART_REDS4_N6.py
python test_scripts/BI/REDS/test_IART_REDS4_N16.py
python test_scripts/BI/REDS/test_IART_Vid4_N6.py
python test_scripts/BI/REDS/test_IART_Vid4_N16.py
# For model trained on Vimeo-90K dataset with BI degradation.
python test_scripts/BI/Vimeo-90K/test_IART_Vid4.py
python test_scripts/BI/Vimeo-90K/test_IART_Vimeo-90K-T.py
# For model trained on Vimeo-90K dataset with BD degradation.
python test_scripts/BD/Vimeo-90K/test_IART_UDM10.py
python test_scripts/BD/Vimeo-90K/test_IART_Vid4.py
python test_scripts/BD/Vimeo-90K/test_IART_Vimeo-90K-T.pyThe following script is for training on BI degradation. You can simply change the data for BD degradation.
# VSR trained on REDS with 6 input frames, tested on REDS4
bash dist_train.sh 8 options/IART_REDS_N6_300K.yml
# VSR trained on REDS with 16 input frames, tested on REDS4
bash dist_train.sh 8 options/IART_REDS_N16_600K.yml
# video sr trained on Vimeo, validated on Vid4
bash dist_train.sh 8 options/IART_Vimeo_N14_300K.ymlDue to the incompatibility of pytorch checkpoint and distributed training, the training process will terminate after the first 5000 iterations. To resume the training, execute the training script again and the previously saved parameters will be automatically loaded. This will restore the normal training procedure.
We acknowledge the following contributors whose code served as the basis for our work: