Add Web Demo & Docker environment

[chenxwh]

This pull request makes it possible to run your model inside a Docker environment, which makes it easier for other people to run it. We're using an open source tool called Cog to make this process easier.

This also means we can make a web page where other people can try out your model! We have implemented Image Denoising, Deblurring and Stereo Image Super-resolution in one demo, view it here: https://replicate.com/megvii-research/nafnet. You can find the docker file under the tab ‘run model with docker’.

We have added some examples to the page, but do claim the page so you can own the page, customise the Example gallery as you like, push any future update to the web demo, and we'll feature it on our website and tweet about it too. You can find the 'Claim this model' button on the top of the page. Any member of the megvii-research organization on GitHub can claim the model ~

In case you're wondering who I am, I'm from Replicate, where we're trying to make machine learning reproducible. We got frustrated that we couldn't run all the really interesting ML work being done. So, we're going round implementing models we like. 😊

[mayorx]

Thanks for your contribution, LGTM!

[chenxwh]

Hi @mayorx :),

Glad to hear that you like the demo, if you are happy with it we'd appreciate if you could kindly claim the page (the claim button is at the top of the page) so we can feature the demo in our website, thank you!

[mayorx]

Hi, chenxwh,
Thanks! I have claimed the page.

But what should I do if I want to add more tasks (e.g., deblurring with the REDS pre-trained model, and set it as defaul) in the web demo?
I can modify the predict.py in this repo, but I did not find where to upload the pre-trained model in replicate.com

[chenxwh]

Hi @mayorx,

A tool called Cog is used for the replicate, you can find the detailed documentation[ here, but in a nutshell, the process goes like:

1. Install cog with these command in terminal (docker should be installed prior to it)
2. Edit your cog.yaml file for setting up environment and dependencies, and predict.py for inference
3. Create web demo (which is https://replicate.com/megvii-research/nafnet in this case)
4. Optionally, check if all works as expected locally with something like cog predict -i task='xx' -i [email protected]
5. Push to web demo with cog push r8.im/megvii-research/nafnet

Then the changes will be synced on the web. That's it! Let me know if you need any further clarification :)

Chenxi

[mayorx]

HI, Chenxi,

Thanks!

I tried the process, but it seems missing some steps:
1). I do not have a 'local environment' yet, thus it might be necessary to create a local environment from https://replicate.com/megvii-research/nafnet

2). How to build a local environment from the web demo? It seems not contained in the doc.
Is it docker pull r8.im/megvii-research/nafnet@sha256:e116b6df8437d9c562f9de2a86cea6fd76a96705e502f091457926bbe989436c ? It is over 10GB, downloading ...

3). How to modify the local docker image, is there anything that needs to be cared about?

4). After the modification, should I just run cog push .. and it will do the other things (e.g. web page modification) automatically?

[chenxwh]

Hi @mayorx,

I am not sure I understand what you meant by 'building' local environment, but the environment specified in cog.yaml will be built for the replicate demo when you run prediction on the web or during cog predict for locally, so you do not need the 'local environment' prior to it to make the demo work.

The docker image is big indeed, it wraps up everything needed for the model to run, to start with, the base images from nvidia are several gigabytes with cuda libraries etc, sometimes if the checkpoints are pre-downloaded (so you do not need to do gdown or something for the web demo to speed things up), the image can be even bigger. But you do not need to pull the image if you want to make changes to the demo.

The steps described before is for when you wish to start a new model (sorry if it was confusing). With the existing one, you simply need to update cog.yaml and predict.py (and add the weights to the corresponding directory if they are pre-downloaded, which I do not think is the case for NAFNet), and then with cog push, the demo will be updated (as well as the image that is built for the demo) :)

To summarise, you do not need to work with docker directly, docker is the wrapper behind cog that makes it possible. Hope it helps!

[mayorx]

Hi, Chenxi,

With the existing one, you simply need to update cog.yaml and predict.py (and add the weights to the corresponding directory if they are pre-downloaded, which I do not think is the case for NAFNet), and then with cog push, the demo will be updated (as well as the image that is built for the demo)

Thanks for your reply. What I want to do is add a deblur demo (REDS pretrain model, as a default option), as it is in our colab demo. So it may be the case of "add the weights to the corresponding directory if they are pre-downloaded" ?
But building a local environment (run the demo by docker) seems to be not practical in my case, as nvidia-docker is required, but it is not supported in MacOS (my local environment is MacOS) or "docker in docker" case (my remote environment is built in docker).
And I tried several ways to push the modified docker image directly into the repo (i.e. r8.im/megvii-research/nafnet), including cog push but meets different problems.

My question is, what is the best practice to modify the existing web demo (https://replicate.com/megvii-research/nafnet) if nvidia-docker is not supported in my case?

[chenxwh]

Ah I see, MacOS shouldn't be an issue, but do you have GPU in your local machine? since in cog.yaml line 3 it specifies gpu: true, it is expecting local GPU available to be able to push. If GPU is not required, maybe can try change it in cog.yaml and see? Or for now I could add the REDS model for you on Replicate, if that is easier?

[chenxwh]

In case GPU is necessary, I have updated demo with REDS model as default on https://replicate.com/megvii-research/nafnet (GoPro is also kept) :)

[mayorx]

Chenxi,
Woo! Thanks a lot, really appreciate it.

[zeke]

Hi @mayorx 👋🏼

I'm Zeke from the @replicate team.

This model is really impressive. The deblurred license plate is really surprising. Almost like science fiction. "ENHANCE!"

I just wanted to let you know that the arXiv.org website now includes a link to your model on Replicate.

See the "Demos" tab at the bottom of the page here: https://arxiv.org/abs/2204.04676

[mayorx]

Wooo!
I see it at arxiv, and thanks for your work! It's very convenient.