A n error about "The size of tensor a (96) must match the size of tensor b (288) at non-singleton dimension 0"

[ZhenyuLiu-SYSU]

Dear author:
I'm glad to see your excellent work.
As I'm a newcomer to the this field, when I try to generate the captions on COCO2017 dataset, I found this question?:

could you tell me how can I fix it？
Thank you！

[cookielovesdata]

Same issue! Have you figured out how?

[ZhenyuLiu-SYSU]

I tried to print the shape of each tensor and found that when is_cross_attention=True, mixed_query_layer shape: torch.Size([96, 4, 768]), but key_layer shape: torch.Size([288, 12, 577, 64] ), resulting in a difference in the dimension 0, so I performed the following operations：

The same problem will be encountered in two places later, and the solution is the same.

I tried to solve it but I don't know if it is correct, hope more people can help!

[kosarkazemi]

You need to downgrade the transformers library to 4.15.0.

[ZhenyuLiu-SYSU]

It works.
Thank you!

[Vinicius-ufsc]

for me 4.15 did not worked, but it worked with

!pip install transformers=='4.16'