Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S2 finetuning #9

Closed
xmu-xiaoma666 opened this issue Apr 30, 2024 · 1 comment
Closed

S2 finetuning #9

xmu-xiaoma666 opened this issue Apr 30, 2024 · 1 comment

Comments

@xmu-xiaoma666
Copy link

When you use S2 Finetining, the channel dimension of visual features will increase by three times. How to deal with the increase in the number of channels passed through?

@mmaaz60
Copy link
Member

mmaaz60 commented Apr 30, 2024

Hi @xmu-xiaoma666,

Thank you for your interest in our work. Yes you are right, while using S2, the channel dimensions will increase 3x and we have to accordingly adjust the MLP projector dimensions.

In summary now the MLP will be projecting from 1024*3 to 4096 instead of from 1024 to 4096. Further, note that we have to perform pretraining again as the projector changes in this case. I hope it will be helpful. Thank You.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants