From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal (ICCV 2023)
Yun Guo^, Xueyao Xiao^, Yi Chang*, Shumin Deng, Luxin Yan
Project website: [link] (Benchmark available now!)
Learning-based image deraining methods have made great progress. However, the lack of large-scale high-quality paired training samples is the main bottleneck to hamper the real image deraining (RID). To address this dilemma and advance RID, we construct a Large-scale High-quality Paired real rain benchmark (LHP-Rain), including 3000 video sequences with 1 million high-resolution (1920*1080) frame pairs. The advantages of the proposed dataset over the existing ones are three-fold: rain with higher-diversity and larger-scale, image with higher-resolution and higher quality ground-truth. Specifically, the real rains in LHP-Rain not only contain the classical rain streak/veiling/occlusion in the sky, but also the splashing on the ground overlooked by deraining community. Moreover, we propose a novel robust low-rank tensor recovery model to generate the GT with better separating the static background from the dynamic rain. In addition, we design a simple transformer-based single image deraining baseline, which simultaneously utilize the self-attention and cross-layer attention within the image and rain layer with discriminative feature representation. Extensive experiments verify the superiority of the proposed dataset and deraining method over state-of-the-art.
We provide full version, simple version and high-level annotations of LHP-Rain. The benchmark has been updated in Project website.
The project is built with PyTorch 1.9.0, Python3.7, CUDA11.1. For package dependencies, you can install them by:
pip install -r requirements.txtTo train SCD-Former, you can begin the training by:
python train/train_derain.py --arch Uformer_B --batch_size 8 --gpu '0,1' --train_ps 256 --train_dir ./train --val_ps 256 --val_dir ./test --env _derain --nepoch 3000 --checkpoint 500 --warmupTo evaluate SCD-Former, you can run:
python test_derain.pyIf you find this project useful in your research, please consider citing:
@InProceedings{Guo_2023_ICCV,
author = {Guo, Yun and Xiao, Xueyao and Chang, Yi and Deng, Shumin and Yan, Luxin},
title = {From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {12097-12107}
}
The code of SCD-Former is based on Uformer.
Please contact us if there is any question or suggestion(Yun Guo [email protected], Yi Chang [email protected]).