Artifacts around the boundary

[tab881]

No description provided.

[Zhaoyi-Yan]

Hi,
Several tricks you can try:

1. Set the mask a little bit larger than your real mask, then only crop the pixels within the region of your real mask to the corresponding locations of your output image. If you are still not satisfied with it, you can try 2.
2. Try poisson matting(seamless_cloning in opencv) to make the boundary look seamless.
3. Finally, if you are wiling to explore and have sufficient GPU resources, you can try stack the mask as a channel, making the input 4 channels. At the same time, make the 'square mask' randomly appear of the whole spatial size. In this way, you may still add another small D to discriminate the patch. The whole loss will be 5: two construction losses, two discriminative losses and one guidance loss. It is a little bit hard to train, however, it should handle with arbitrary masks more adaptively.

As your 3D inpainting, I think it is probably possible. However, it may need you additional work on how to efficiently shift features out of the mask into the mask. Also, I am busy with other work these months, and have no time to optimize my code(It still has a lot of improvement space on either efficient and performance).