Skip to content

Video Restoration Toolbox including FGST (ICML 2022), S2SVR (ICML 2022), etc.

License

Notifications You must be signed in to change notification settings

linjing7/VR-Baseline

Repository files navigation

A Toolbox for Video Restoration

jiedu jiedu

PWC PWC PWC

PWC PWC PWC

Authors

Jing Lin*, Yuanhao Cai*, Xiaowan Hu, Haoqian Wang, Youliang Yan, Xueyi Zou, Henghui Ding, Yulun Zhang, Radu Timofte, and Luc Van Gool

ntire

News

  • 2022.12.08 : Pretrained model, training/testing log, visual results of FGST on GoPro and DVD dataset are released. S2SVR will be provided later.🔥

  • 2022.11.30 : Data preparation codes of GoPro and DVD are provided. 🔆

  • 2022.08.05 : Pretrained model of FGST on GOPRO dataset is released. 💫

  • 2022.05.14 : Our FGST and S2SVR are accepted by ICML2022. 🚀

Super-Resolution Deblur Compressed Video Enhancement

Papers

Method Dataset Pretrained Model Training Log Testing Log Visual Result Quantitative Result
FGST GoPro Google Drive / Baidu Disk Google Drive / Baidu Disk Google Drive / Baidu Disk Google Drive / Baidu Disk 33.02 / 0.947
FGST DVD Google Drive / Baidu Disk Google Drive / Baidu Disk Google Drive / Baidu Disk Google Drive / Baidu Disk 33.50 / 0.945

Note: access code for Baidu Disk is VR11

1. Create Environment:

  • Python 3 (Recommend to use Anaconda)

  • NVIDIA GPU + CUDA

  • Python packages:

pip install torchvision==0.9.0  torch==1.8.0  torchaudio==0.8.0
pip install -r requirements.txt
pip install openmim
mim install mmcv-full==1.5.0
pip install -v -e .
pip install cupy-cuda101==7.7.0

2. Prepare Dataset:

Download the datasets (GOPRO,DVD,REDS,VIMEO,MFQE-v2) and and recollect them as the following form:

|--VR-Baseline
    |--data
    	|-- GoPro
    	    |-- test
    	    |-- train
    	|-- DVD
    	    |-- quantitative_datasets
    	      |-- GT
    	      |-- LQ
    	    |-- qualitative_datasets
    	|-- REDS
    	    |-- train_sharp_bicubic
    	    |-- train_sharp
    	|-- VIMEO
    	    |-- BIx4
    	    |-- GT
    	|-- MFQEV2
    	    |-- test
    	    |-- train

You can run the following command to recollect GoPro and DVD dataset:

cd VR-Baseline/data_preparation

# recollect GoPro dataset
python GoPro_Util.py --input_path INPUT_PATH --save_path SAVE_PATH

# recollect DVD dataset
python DVD_Util.py --input_path INPUT_PATH --save_path SAVE_PATH

You need to replace INPUT_PATH and SAVE_PATH with your own path.

3. Training:

cd VR_Baseline

# training FGST on GoPro dataset
bash tools/dist_train.sh configs/FGST_deblur_gopro.py 8

# training FGST on DVD dataset
bash tools/dist_train.sh configs/DVD_deblur_gopro.py 8

# training S2SVR on GoPro dataset
bash tools/dist_train.sh configs/S2SVR_deblur_gopro.py 8

# training S2SVR on REDS dataset
bash tools/dist_train.sh configs/S2SVR_sr_reds4.py 8

# training S2SVR on VIMEO dataset
bash tools/dist_train.sh configs/S2SVR_sr_vimeo.py 8

# training S2SVR on MFQEv2 dataset
bash tools/dist_train.sh configs/S2SVR_vqe_mfqev2.py 8

The training log, trained model will be available in VR-Baseline/experiments/ .

4. Testing:

Download pretrained model and run the following command.

To test on benchmark:

cd VR_Baseline

# testing FGST on GoPro dataset
bash tools/dist_train.sh configs/FGST_deblur_gopro_test.py 8

# testing FGST on DVD dataset
bash tools/dist_train.sh configs/FGST_deblur_dvd_test.py 8

5. TODO

These works are mostly done during the internship at HUAWEI Noah's Ark Lab. Due to the limitation of company regulations, the original pre-trained models can not be transferred and published here. We will retrain more models and open-source them when we have enough GPUs as soon as possible.

  • More data preparation codes
  • More Pretrained Models
  • Inference Results
  • MFQEv2 dataloader

6. Acknowledgement.

We refer to codes from BasicVSR++ and mmediting. Thanks for their awesome works.

7. Citation

If this repo helps you, please consider citing our works:

# FGST
@inproceedings{fgst,
  title={Flow-Guided Sparse Transformer for Video Deblurring},
  author={Lin, Jing and Cai, Yuanhao and Hu, Xiaowan and Wang, Haoqian and Yan, Youliang and Zou, Xueyi and Ding, Henghui and Zhang, Yulun and Timofte, Radu and Van Gool, Luc},
  booktitle={ICML},
  year={2022}
}


# S2SVR
@inproceedings{seq2seq,
  title={Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration},
  author={Lin, Jing  and Hu, Xiaowan and Cai, Yuanhao and Wang, Haoqian and Yan, Youliang and Zou, Xueyi and Zhang, Yulun and Van Gool, Luc},
  booktitle={ICML},
  year={2022}
}

About

Video Restoration Toolbox including FGST (ICML 2022), S2SVR (ICML 2022), etc.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages