-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Great Project! Here are a few suggestions. #61
Comments
Hi, thanks for your efforts. We have merge your code to our repo, and it is a good implementation! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thanks a lot as UniAnimate is currently the best image animation project I have tested. I have so far noticed no artifact and the animation is quite smooth. I am using a 12GB VRAM without memory issues on a windows OS. Here are a few suggestions based on the errors I faced while trying to run the inference.
The first error was something like : The shape of the 2D attn_mask is torch.Size([77, 77]), but should be (1, 1).
This took some time to resolve especially because the error was not directly from your project, but from the open_clip dependency. It seems recent releases of open_clip included a change that can cause this error in some environments. In case any one encounters this error, simple navigate to Lib\site-packages\open_clip\transformer.py in your virtual environment and change batch_first: bool = True, on line 329 to batch_first: bool = False.
After fixing the first error, I ran the inference and got a second error which was something like: "list index out of range" & "local variable 'vit_frame' referenced before assignment."
The cause was that the use of reference videos or pose frames shorter than the max_frames set in UniAnimate\configs\UniAnimate_infer.yaml was not handled in the load_video_frames function in UniAnimate\tools\inferences\inference_unianimate_entrance.py. I suggest that the load_video_frames function be modified as shown below to handle all error cases related to the reference video or pose frame;
Or you can come up with a better implementation. Thank you.
The text was updated successfully, but these errors were encountered: