Add sdxl controlnet reference community pipeline #9893
Conversation
examples/community/README.md
Outdated
|  | ||
|  |
There was a problem hiding this comment.
It seems to me that the url at https://github.com/zideliu/diffusers/assets/34944964/743848da-a215-48f9-ae39-b5e2ae49fb13 cannot be found (404). The diffusers repository fork seems to have been removed by their author. Do you confirm?
Actually, I've updated the stable_diffusion_xl_reference.py file with the latest modifications from StableDiffusionXLPipeline on my side. If you agree, I propose to create a new PR:
- Upgrading the
StableDiffusionXLReferencePipelineto use the latest modifications fromStableDiffusionXLPipeline(IP adapters...), - Replacing the human example of this updated
StableDiffusionXLReferencePipelineby a non-human one and fixing the 404 url at the same time.
There was a problem hiding this comment.
Maybe you could open a PR to https://hf.co/datasets/huggingface/documentation-images and host the images there?
There was a problem hiding this comment.
Yes, PR https://huggingface.co/datasets/huggingface/documentation-images/discussions/393 has been created accordingly.
examples/community/README.md
Outdated
| Output Image | ||
|
|
||
| `prompt: 1 girl` | ||
|
|
||
| `reference_attn=True, reference_adain=True, num_inference_steps=20, style_fidelity=1.0` | ||
|
|
||
|  | ||
|
|
||
| `reference_attn=False, reference_adain=True, num_inference_steps=20, style_fidelity=1.0` | ||
|
|
||
|  | ||
|
|
||
| `reference_attn=True, reference_adain=False, num_inference_steps=20, style_fidelity=1.0` | ||
|
|
||
|  |
There was a problem hiding this comment.
Can we avoid using human references in the examples and restrict ourselves to using non-human objects?
There was a problem hiding this comment.
Sure. I've pushed modifications to fix that.
What does this PR do?
It provides reference controlnet support for sdxl model used along other controlnet models. It is based on Stable Diffusion ControlNet Reference and Stable Diffusion XL Reference community pipelines, and inherits the Stable Diffusion XL ControlNet Pipeline.
Fixes #4675
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.