Skip to content

Issues: open-compass/VLMEvalKit

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
27 Open 139 Closed
27 Open 139 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Please provide the dataset file or download script
#412 by lerogo was closed Aug 30, 2024
16
Change to GPT judgement
#305 by beichenzbc was closed Jul 23, 2024
16
Error about vlmeval/dataset/utils/multiple_choice.py
#290 by mary-0830 was closed Jul 19, 2024
11
Multi-GPU inference issue
#224 by ruifengma was closed Jun 19, 2024
11
[rank0]: KeyError: 'answer'
#268 by lucasjinreal was closed Aug 27, 2024
8
IndexError: index 1 is out of bounds for dimension 0 with size 1
#150 by lucasjinreal was closed Apr 21, 2024
8
A major problem with the multiple-choice evaluation
#141 by YongLD was closed Apr 9, 2024
7
Questions regarding the metrics for SEED bench
#20 by zjysteven was closed Dec 23, 2023
7
about vlmeval/evaluate/multiple_choice.py
#228 by mary-0830 was closed Jun 20, 2024
6
Model name in demo
#199 by chandrabhuma was closed May 11, 2024
6
InternVL2 Error
#289 by amitbcp was closed Aug 28, 2024
5
How to create OpenVLM.json
#288 by amitbcp was closed Jul 18, 2024
5
Does the generating params changed?
#269 by lucasjinreal was closed Aug 29, 2024
5
加载本地训练模型
#252 by 2499048212 was closed Sep 3, 2024
5
有关模型和数据集下载的问题
#213 by xioatian1 was closed May 29, 2024
5
[疑问] 实验分数对不上，使用llava_v1.5_7b在MMMU_DEV_VAL测试集上的分数对不上
#95 by mary-0830 was closed Feb 27, 2024
5
Error when evaluating mmbench_dev_en：The correct answer according to the 'answer' field in the table should be D, but the log says it is A.
#73 by jdy18 was closed Feb 1, 2024
5
InternVL2 Truncated Output
#466 by TJ-Ouyang was closed Sep 24, 2024
4
How to evaluate my own model
#413 by lixu6-alt was closed Sep 4, 2024
4
Cannot Reproduce Results of LLaVA-1.5 on MMBench_DEV_CN
#404 by ppalantir was closed Sep 16, 2024
4
配置数据集的vlmeval/utils/dataset_config.py文件不存在
#402 by scl-01 was closed Aug 23, 2024
4
Any pypi package for this toolkit?
#375 by zhimin-z was closed Aug 22, 2024
4
Request Support for VILA
#293 by amitbcp was closed Jul 29, 2024
4
Missing dataset_config.py
#282 by beichenzbc was closed Jul 15, 2024
4
如何获得 opencompass 上评测模型的最新细分项结果？
#264 by jyC23333 was closed Jul 10, 2024
4
ProTip! no:milestone will show everything without a milestone.