A free and open-source inpainting tool powered by SOTA AI model.
- Completely free and open-source
- Fully self-hosted
- Classical image inpainting algorithm powered by cv2
- Multiple SOTA AI models
- Support CPU & GPU
- Various inpainting strategy
- Run as a desktop APP
| Usage | Before | After |
|---|---|---|
| Remove unwanted things |  |
 |
| Remove unwanted person |  |
 |
| Remove Text |  |
 |
| Remove watermark |  |
 |
| Fix old photo |  |
 |
| Text Driven Inpainting |  |
Prompt: a fox sitting on a bench |
pip install lama-cleaner
# Model will be downloaded automatically
lama-cleaner --model=lama --device=cpu --port=8080
# Lama Cleaner is now running at https://localhost:8080Available arguments:
| Name | Description | Default |
|---|---|---|
| --model | lama/ldm/zits/mat/fcf/sd1.4 See details in Inpaint Model | lama |
| --hf_access_token | stable-diffusion(sd) model need huggingface access token to download model | |
| --sd-run-local | Once the model as downloaded, you can pass this arg and remove --hf_access_token |
|
| --sd-disable-nsfw | Disable stable-diffusion NSFW checker. | |
| --sd-cpu-textencoder | Always run stable-diffusion TextEncoder model on CPU. | |
| --device | cuda, mps or cpu | cuda |
| --port | Port for backend flask web server | 8080 |
| --gui | Launch lama-cleaner as a desktop application | |
| --gui_size | Set the window size for the application | 1200 900 |
| --input | Path to image you want to load by default | None |
| --debug | Enable debug mode for flask web server |
| Model | Description | Config |
|---|---|---|
| cv2 | 👍 No GPU is required, and for simple backgrounds, the results may even be better than AI models. | |
| LaMa | 👍 Generalizes well on high resolutions(~2k) |
|
| LDM | 👍 Possible to get better and more detail result 👍 The balance of time and quality can be achieved by adjusting steps 😐 Slower than GAN model 😐 Need more GPU memory |
Steps: You can get better result with large steps, but it will be more time-consuming Sampler: ddim or plms. In general plms can get better results with fewer steps |
| ZITS | 👍 Better holistic structures compared with previous methods 😐 Wireframe module is very slow on CPU |
Wireframe: Enable edge and line detect |
| MAT | TODO | |
| FcF | 👍 Better structure and texture generation 😐 Only support fixed size (512x512) input |
|
| SD1.4 | 👍 SOTA text-to-image diffusion model |
| Original Image | LaMa | LDM |
|---|---|---|
 |
 |
 |
| Original Image | ZITS | LaMa |
|---|---|---|
 |
 |
 |
Image is from ZITS paper. I didn't find a good example to show the advantages of ZITS and let me know if you have a good example. There can also be possible problems with my code, if you find them, please let me know too!
| Original Image | Lama | FcF |
|---|---|---|
 |
 |
 |
Lama Cleaner provides three ways to run inpainting model on images, you can change it in the settings dialog.
| Strategy | Description | VRAM |
|---|---|---|
| Original | Use the resolution of the original image | 🎉 |
| Resize | Resize the image to a smaller size before inpainting. Lama Cleaner will make sure that the area of the image outside the mask is not degraded. | 🎉 🎉 |
| Crop | Crop masking area from the original image to do inpainting | 🎉 🎉 🎉 |
If you have problems downloading the model automatically when lama-cleaner start,
you can download it manually. By default lama-cleaner will load model from TORCH_HOME=~/.cache/torch/hub/checkpoints/,
you can set TORCH_HOME to other folder and put the models there.
Only needed if you plan to modify the frontend and recompile yourself.
Frontend code are modified from cleanup.pictures, You can experience their great online services here.
- Install dependencies:
cd lama_cleaner/app/ && yarn - Start development server:
yarn start - Build:
yarn build
To run Stable Diffusion Inpaint on Apple Silicon gpu run lama-cleaner with --device=mps
lama-cleaner --model=sd1.4 --device=mps --port=8080
Run within a Docker container. Set the CACHE_DIR to models location path. Optionally add a -d option to
the docker run command below to run as a daemon.
docker build -f Dockerfile -t lamacleaner .
docker run -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner \
python3 main.py --device=cpu --port=8080 --host=0.0.0.0
docker run --gpus all -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner \
python3 main.py --device=cuda --port=8080 --host=0.0.0.0
Then open https://localhost:8080