Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification of ScienceQA score #2

Closed
Isaachhh opened this issue Apr 28, 2024 · 2 comments
Closed

Clarification of ScienceQA score #2

Isaachhh opened this issue Apr 28, 2024 · 2 comments

Comments

@Isaachhh
Copy link

I notice that your model achieve ~80 on ScienceQA. I wonder whether you report total accuracy other than image accuracy, because LLaVA reports only image accuracy (66.8 for 7B and 71.6 for 13B).
Thanks!

@mmaaz60
Copy link
Member

mmaaz60 commented Apr 28, 2024

Hi @Isaachhh,

Thank you for pointing it out, and apologies for any confusion it may have caused. We have updated the main results and added some additional models as well. I am pasting the table below for your reference.

image

Let us know if you have any questions. Thank You

@Isaachhh
Copy link
Author

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants