-
Notifications
You must be signed in to change notification settings - Fork 316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request]: Mask in shape of the face (not square) #654
Comments
When I started this project, it was hard to find a dataset that segmented only faces, but now there are many community-created models available. As an example, you can download several models from the links below and put them in the https://civitai.com/search/models?sortBy=models_v9&query=adetailer (NSFW) |
Thank you Bing-su. This is one of those type of searches where you can't find what you're looking for if you don't know what to call it. I guess what I am asking about is "segmentation" not masks. OK, so I installed a couple from Anzhc. One creates a mask for the entire head and includes hair (and a few things). Mix results with that depending on the image. I could see how that could be useful, if you prompt within adetailer for the entire head. Anzhc's Face Segmentation model is fantastic. It generates very good face masks. The results are far better than the square mask. I've been using this in several adetailer tests and getting very good results. The parameters I'm working with are (the only ones that even seem meaningful for mask quality):
After experiments, masked padding seems to adjust how much of the area outside of the mask is used in creating the new image. Something like providing more context for the new image, where 0 would mean ignore everything outside of the mask and 256 means use that many pixels outside of the mask as context for the new image. Then, when the final image is blended, it's only within the mask area. None of the padding area is actually seen in the final image. Setting this to 0 produced the worst results. 128 generally produced better quality inpainting of faces. For example, hairlines and ears are better blended. Yet 256 also works in many cases. Something that made this testing more difficult is how adetailer can save the mask previews, but it's the mask from the model not the resulting mask after applying adjustments. It would be very helpful to see the resulting mask. To try and determine this for myself I set the blur to 0, make adjustments, and then look at the before/after to trying and see what the mask might have been. I should write an article for Civitai about using adetailer with Anzhc's Face Segmentation. I hesitate since I can't explain what some of the inpainting parameters really only do. I'm just guessing, and that's not the type of article I would want to write. |
Thank you so much for your interest in this project. In the Settings section of the Stable Diffusion, there is an option to save the mask image used for inpainting. This should help you. The inpainting options in Adetailer are almost a direct copy of the inpainting options in stable-diffusion-webui. If you search for 'stable diffusion webui inpainting', you'll probably get several results. I hope you find some of these videos and webpages I found helpful, thanks. https://youtu.be/sF3POwPUWCE?si=3iouV_bLY6lkEXdM |
Is your feature request related to a problem? Please describe.
First, let me say this extension is a must-have. It's by far the one I use the most in Automatic1111.
The issue I'm raising is how faces are not squares. By creating a square mask adetailer ends up modifying parts of the image outside of the face. Adjusting the size of the mask, blur, or padding becomes a trade-off of not fixing part of the face or disturbing areas outside of the face.
I often have problems when the face and surrounding content are blended (bleeding or smearing of the image). For example, sometimes the hair around the face becomes blurred while the face is being "fixed."
The worst cases are when the angle of the face doesn't fit well within a square, such as a profile or angles where the nose extends beyond the face's outline.
PS - Square masks for hands is even worse. I've given up trying to use adetailer on hands.
Describe the solution you'd like
Generate a mask that is just the face area.
Even better, It could even be a contour (depth) map (if possible), then applying adjustments to the face as a 3D shape.
Describe alternatives you've considered
Dropping Automatic1111 and going to ComfyUI which has a node that can generate masks in the shape of the detected face.
The only other option I've got is to create my own mask by hand in img2img, then applying a fix manually. This is like getting into using Photoshop as a photographer (which I've done for decades, and it's too time consuming). Avoiding manual steps was the problem I think adetailer was designed to solve.
Additional context
No response
The text was updated successfully, but these errors were encountered: