Skip to content

Issues: open-compass/VLMEvalKit

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
13 Open 78 Closed
13 Open 78 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Error about vlmeval/dataset/utils/multiple_choice.py
#290 by mary-0830 was closed Jul 19, 2024
11
How to create OpenVLM.json
#288 by amitbcp was closed Jul 18, 2024
4
Missing dataset_config.py
#282 by beichenzbc was closed Jul 15, 2024
4
[bug] The same sample was run twice in mp_utils.py
#272 by buptweixin was closed Jul 11, 2024
1
[BUG] local variable 'structs' referenced before assignment
#271 by Yunnglin was closed Jul 12, 2024
1
如何获得 opencompass 上评测模型的最新细分项结果？
#264 by jyC23333 was closed Jul 10, 2024
4
如何多卡并行测评？How to evaluate on multiple cards in parallel?
#263 by buptweixin was closed Jul 10, 2024
2
Math_Vista result is not summarized
#253 by ruifengma was closed Jul 8, 2024
4
SEEDBench2 and SEEDBench2_Plus give the same result
#247 by ruifengma was closed Jul 1, 2024
4
Image embeddings
#245 by chandrabhuma was closed Jun 30, 2024
3
MMBench_EN_V11 404 not found after evaluation
#239 by MonolithFoundation was closed Jun 26, 2024
2
evaluate TextVQA_VAL error
#237 by LRHstudy was closed Jun 27, 2024
1
Judge model seems not working
#236 by ruifengma was closed Jun 26, 2024
3
about vlmeval/evaluate/multiple_choice.py
#227 by mary-0830 was closed Jun 20, 2024
1
about vlmeval/evaluate/multiple_choice.py
#228 by mary-0830 was closed Jun 20, 2024
6
Multi-GPU inference issue
#224 by ruifengma was closed Jun 19, 2024
11
AesBench support
#222 by yipoh was closed Jun 27, 2024
1
支持GLM4v-9b-chat评测吗？
#220 by yinggoga was closed Jun 13, 2024
1
有关模型和数据集下载的问题
#213 by xioatian1 was closed May 29, 2024
5
How to calculate avg score after the leaderboard update to 8 multimodal benchmarks
#206 by wanxinzzz was closed May 21, 2024
1
Model name in demo
#199 by chandrabhuma was closed May 11, 2024
6
mmmu should only use mmmu val not MMMU_DEV_VAL
#198 by max-yue was closed May 24, 2024
3
[Request]Consider integrating MMT-Bench and CONTEXTUAL?
#196 by iamlockelightning was closed Jul 9, 2024
2
Question about LlavaBench Evaluation
#194 by justinphan3110 was closed May 9, 2024
4
About MMBench data source: Is there any difference from huggingface lmms-lab/MMBench?
#192 by Coobiw was closed May 6, 2024
2
ProTip! Updated in the last three days: updated:>2024-07-16.