-
Notifications
You must be signed in to change notification settings - Fork 65
Issues: open-compass/VLMEvalKit
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Cannot reproduce llava v1.5 7b SEEDBench_IMG results
#42
by MagicGeek2
was closed Jan 5, 2024
updated Jan 5, 2024
How do you compute MME(normalized)?
#43
by simplelifetime
was closed Jan 8, 2024
updated Jan 8, 2024
Numpy compliation issue during installing
#34
by findalexli
was closed Jan 10, 2024
updated Jan 10, 2024
What is the estiamted runtime across benchmarks, and OpenAI api cost?
#33
by findalexli
was closed Jan 20, 2024
updated Jan 20, 2024
Get a 0 score in Evaluation of qwen chat in ScienceQA
#61
by YongLD
was closed Jan 24, 2024
updated Jan 24, 2024
These two download links are no longer accessible...
#78
by zhimin-z
was closed Jan 30, 2024
updated Jan 30, 2024
where is the vlmeval/utils/data_util.py
#83
by TousenKaname
was closed Jan 31, 2024
updated Jan 31, 2024
Error when evaluating mmbench_dev_en:The correct answer according to the 'answer' field in the table should be D, but the log says it is A.
#73
by jdy18
was closed Feb 1, 2024
updated Feb 10, 2024
[疑问] 实验分数对不上,使用llava_v1.5_7b在MMMU_DEV_VAL测试集上的分数对不上
#95
by mary-0830
was closed Feb 27, 2024
updated Feb 27, 2024
If i want to use model of LLaVa in VLMEval, which version should i installed for these?
#100
by EmberYS
was closed Mar 7, 2024
updated Mar 7, 2024
Enhancing Multi-Choice Question Handling with Case-Sensitive Matching
#99
by jdy18
was closed Mar 12, 2024
updated Mar 12, 2024
OPENAI_API_KEY is not set properly, will use exact matching for evaluation
#109
by StarCycle
was closed Mar 15, 2024
updated Mar 15, 2024
The md5 hashtest of the dataset DocVQA_VAL hasn't passed.
#103
by EmberYS
was closed Mar 16, 2024
updated Mar 16, 2024
The scores calculated by VLMEvalKit differ from the score calculated on the MMBench website
#121
by jdy18
was closed Mar 20, 2024
updated Mar 20, 2024
Previous Next
ProTip!
no:milestone will show everything without a milestone.