How to calculate the average rank?

[liuheng0111]

In the leaderboard, [LLaVA-InternLM2-20B (QLoRA)] get higher average score than Monkey-Chat, but Monkey-Chat rank higher. So how to calculate the Avg.Rank as the leaderboard shows?

[kennymckormick]

Hi, @liuheng0111 ,
Avg. Rank means the average ranks of an VLM on different benchmarks. For example, if there exist two benchmarks: MMBench and MMBench_CN, if GPT-4v ranked 1st on MMBench and ranked 3rd on MMBench_CN, it's Avg. Rank would be (1+3) / 2 = 2.

According to the definition, the Avg. Rank and Avg. Score may not have exactly the same partial order.