Skip to content

Commit

Permalink
demos update
Browse files Browse the repository at this point in the history
  • Loading branch information
mv-lab committed Oct 3, 2022
1 parent bcabd09 commit a1fe0cb
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 16 deletions.
63 changes: 47 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@

[![arXiv](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2209.11345)
![visitors](https://visitor-badge.glitch.me/badge?page_id=mv-lab/swin2sr)
[ <a href="https://colab.research.google.com/drive/1paPrt62ydwLv2U2eZqfcFsePI4X4WRR1?usp=sharing"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>](https://colab.research.google.com/drive/1paPrt62ydwLv2U2eZqfcFsePI4X4WRR1?usp=sharing)
[![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/jjourney1125/swin2sr)
[ <a href="https://www.kaggle.com/code/jesucristo/super-resolution-demo-swin2sr-official/"><img src="https://upload.wikimedia.org/wikipedia/commons/7/7c/Kaggle_logo.png?20140912155123" alt="kaggle logo" width=50></a>](https://www.kaggle.com/code/jesucristo/super-resolution-demo-swin2sr-official/)


<br>

Expand All @@ -20,15 +24,16 @@ MegaStudyEdu, South Korea

**News 🚀🚀**

- [09/2022] Ongoing website and multiple demos creation. Feel free to contact us. Paper will be presented at AIM, ECCV 2022.
- [10/2022] Demos on Kaggle, Collab and Huggingface Spaces 🤗 are ready!
- [09/2022] Ongoing website and multiple demos creation. Feel free to contact us. Paper will be presented at the [Advances in Image Manipulation (AIM) workshop](https://data.vision.ee.ethz.ch/cvl/aim22/), ECCV 2022, Tel Aviv.

------------------

This is the official repository and PyTorch implementation of Swin2SR. We provide the supplementary material, code, pretrained models and demos. Swin2SR represents a possible improvement of the famous [SwinIR](https://github.com/JingyunLiang/SwinIR/) by [Jingyun Liang](https://jingyunliang.github.io/) (kudos for such an amazing contribution ✋). Our model achieves state-of-the-art performance in:

- classical, lighweight and real-world image super-resolution (SR)
- color JPEG compression artifact reduction
- compressed input super-resolution: top solution at the "[AIM 2022 Challenge on Super-Resolution of Compressed Image and Video](https://codalab.lisn.upsaclay.fr/competitions/5076)"
- compressed input super-resolution: top solution at the "[AIM 2022 Challenge on Super-Resolution of Compressed Image and Video](https://codalab.lisn.upsaclay.fr/competitions/5076)" organized by [Ren Yang](https://scholar.google.de/citations?hl=en&user=3NgkOp0AAAAJ) and [Radu Timofte](https://scholar.google.de/citations?user=u3MwH5kAAAAJ&hl=en)


<p align="center">
Expand Down Expand Up @@ -56,6 +61,15 @@ In this paper, we explore the novel Swin Transformer V2, to improve SwinIR for i
The training code is at [KAIR](https://github.com/cszn/KAIR/). We follow the same training setup as [SwinIR](https://github.com/JingyunLiang/SwinIR/) by [Jingyun Liang](https://jingyunliang.github.io/). We are working on KAIR integration 👀
More details about the training setup in our [paper](https://arxiv.org/abs/2209.11345).


<details>
<summary><b>Why moving to Swin Transformer V2 ??</b></summary>
<img src="media/paper-why.png " alt="paper swinv2" width="800" border="0">
Especially in the case of lightweight super-resolution, we noticed how our model convergence was approximately x2 faster using the same experimental setup as SwinIR. We provide the details in the paper Section 3 and 4.2
</details>

<br>

Please check our **[demos](#demos) ready to run** 🚀

------
Expand All @@ -64,20 +78,20 @@ Please check our **[demos](#demos) ready to run** 🚀

We achieved state-of-the-art performance on classical, lightweight and real-world image Super-Resolution (SR), JPEG compression artifact reduction, and compressed input super-resolution. We use mainly the DIV2K Dataset and Flickr2K datasets for training, and for testing: RealSRSet, 5images/Classic5/Set5, Set14, BSD100, Urban100 and Manga109

🌎 **[All visual results of SwinIR can be downloaded here](https://github.com/mv-lab/swin2sr/releases)**. We also provide links to download the original datasets.
🌎 **[All visual results of Swin2SR can be downloaded here](https://github.com/mv-lab/swin2sr/releases)**. We also provide links to download the original datasets.
More details in our [paper](https://arxiv.org/abs/2209.11345).

<br>

<img src="media/chain-sample.png " alt="swin2sr makima demo" width="530" border="0">
<img src="media/gojou-sample.png " alt="swin2sr makima demo" width="530" border="0">


|Compressed inputs | Swin2SR output|
| :--- | :---: |
| <img src="media/frog_0.png" alt="frog_in" width="250" border="0"> | <img src="media/frog_1.png" alt="frog_swin2sr" width="250" border="0"> |
| <img src="media/0814_0.png" alt="div2k_in" width="250" border="0"> | <img src="media/0814_1.png" alt="div2k_swin2sr" width="250" border="0"> |
| <img src="media/buildings_0.png" alt="buildings_in" width="250" border="0"> | <img src="media/buildings_1.png" alt="buildings_swin2sr" width="250" border="0"> |
| <img src="media/frog_0.png" alt="frog_in" width="300" border="0"> | <img src="media/frog_1.png" alt="frog_swin2sr" width="300" border="0"> |
| <img src="media/0814_0.png" alt="div2k_in" width="300" border="0"> | <img src="media/0814_1.png" alt="div2k_swin2sr" width="300" border="0"> |
| <img src="media/buildings_0.png" alt="buildings_in" width="300" border="0"> | <img src="media/buildings_1.png" alt="buildings_swin2sr" width="300" border="0"> |

<img src="media/chain-sample.png " alt="swin2sr makima demo" width="620" border="0">
<img src="media/gojou-sample.png " alt="swin2sr makima demo" width="620" border="0">


<br>

Expand All @@ -89,7 +103,7 @@ More details in our [paper](https://arxiv.org/abs/2209.11345).

1. create a folder `inputs` and put there the input images. The model expects low-quality and low-resolution JPEG compressed images.

2. select `--scale` standard is 4, this means we will increase the resolution of the image x4 times. For example for a 1MP image (1000x1000) we will upscale it to near 4K (4000x4000). We also allow scale 2 (x2 upscaling), for further resolutions please contact us and we will provide the models.
2. select `--scale` standard is 4, this means we will increase the resolution of the image x4 times. For example for a 1MP image (1000x1000) we will upscale it to near 4K (4000x4000).

3. run our model using `main_test_swin2sr.py` and `--save_img_only`. The pre-trained models are included in [our repo](https://github.com/mv-lab/swin2sr), you can download them from [here](https://github.com/mv-lab/swin2sr/releases) or check the repo [releases](https://github.com/mv-lab/swin2sr/releases). It is important to select the proper `--task`, by default we do compressed input super-resolution `compressed_s`.

Expand All @@ -106,14 +120,29 @@ to **reproduce results**, calculate metrics and further evaluation, please check

## Demos

🔥 🚀 ✅ **[Kaggle kernel demo](https://www.kaggle.com/code/jinssaa/swin2sr-dev/) ready to run!** easy to follow includes testing for multiple SR applications.

[Super-Resolution Demo Swin2SR Official](https://www.kaggle.com/code/jesucristo/super-resolution-demo-swin2sr-official/)
🔥 🚀 ✅ **[Kaggle kernel demo](https://www.kaggle.com/code/jesucristo/super-resolution-demo-swin2sr-official/) ready to run!** easy to follow includes testing for multiple SR applications.

<details>
<summary>Clicke here to see how the Kaggle demo looks like</summary>
<p align="center">
<img src="media/kaggle-demo.png " alt="kaggle demo" width="800" border="0">
</p>
</details>

<br>

[Super-Resolution Demo Swin2SR Official](https://www.kaggle.com/code/jesucristo/super-resolution-demo-swin2sr-official/) is also available in [<a href="https://colab.research.google.com/drive/1paPrt62ydwLv2U2eZqfcFsePI4X4WRR1?usp=sharing">Google Colab <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>](https://colab.research.google.com/drive/1paPrt62ydwLv2U2eZqfcFsePI4X4WRR1?usp=sharing)

We also have an **interactive demo, no login required!** in [Huggingface Spaces 🤗](https://huggingface.co/spaces/jjourney1125/swin2sr) just click and upload images.


<img src="media/hug-demo.png" alt="swin2sr huggingface demo" width="800" border="0">

We are working on more interactive demos 👀 Contact us if you have ideas!



----
<br>

Expand Down Expand Up @@ -144,6 +173,7 @@ We follow the same evaluation setup as [SwinIR](https://github.com/JingyunLiang/
### ClassicalSR
```
python main_test_swin2sr.py --task classical_sr --scale 2 --training_patch_size 64 --model_path model_zoo/swin2sr/Swin2SR_ClassicalSR_X2_64.pth --folder_lq testsets/Set5/LR_bicubic/X2 --folder_gt testsets/Set5/HR
python main_test_swin2sr.py --task classical_sr --scale 4 --training_patch_size 64 --model_path model_zoo/swin2sr/Swin2SR_ClassicalSR_X4_64.pth --folder_lq testsets/Set5/LR_bicubic/X4 --folder_gt testsets/Set5/HR
```

Expand Down Expand Up @@ -177,11 +207,12 @@ python main_test_swin2sr.py --task jpeg_car --jpeg 40 --model_path model_zoo/swi

**[SwinIR: Image Restoration Using Swin Transformer](https://arxiv.org/abs/2108.10257) by Liang et al, ICCVW 2021.**

[Swin Transformer V2: Scaling Up Capacity and Resolution](https://arxiv.org/abs/2111.09883) by Liu et al, CVPR 2022.
**[AISP: AI Image Signal Processing](https://github.com/mv-lab/AISP) by Marcos Conde, Radu Timofte and collaborators, 2022.**

[AIM 2022 Challenge on Super-Resolution of Compressed Image and Video](https://arxiv.org/abs/2208.11184) organized by Ren Yang.

**[AISP: AI Image Signal Processing](https://github.com/mv-lab/AISP) by Marcos Conde, Radu Timofte and collaborators, 2022.**
[Swin Transformer V2: Scaling Up Capacity and Resolution](https://arxiv.org/abs/2111.09883) by Liu et al, CVPR 2022.


-----------------

Expand All @@ -208,4 +239,4 @@ This project is released under the Apache 2.0 license. The codes are heavily bas

## Contact

Marcos Conde ([email protected]) and Ui-Jin Choi are the contact persons. Please add in the email subject "swin2sr".
Marcos Conde ([email protected]) and Ui-Jin Choi ( [email protected]) are the contact persons. Please add in the email subject "swin2sr".
Binary file added media/hug-demo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/paper-why.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit a1fe0cb

Please sign in to comment.