New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to calculate Avg Score mentioned in OpenVLM Leaderboard？ #176

Closed

la1n33 opened this issue Apr 26, 2024 · 0 comments

la1n33 commented Apr 26, 2024

Since OpenVLM Leaderboard describes its metrics like this:

Avg Score: The average score on all VLM Benchmarks (normalized to 0 - 100, the higher the better).

May I ask:

Do all VLM Benchmarks refer to all 19 benchmarks or only the 12 benchmarks displayed in the Main Leaderboard?
How to calculate Avg Score? Since MME and OCRBench need to be normalized, how do you normalize them?

la1n33 closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment