-
Notifications
You must be signed in to change notification settings - Fork 83
Issues: open-compass/VLMEvalKit
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Evaluation of custom models and datasets.
#108
by juxingyiwan
was closed Jul 19, 2024
updated Jul 19, 2024
Error about vlmeval/dataset/utils/multiple_choice.py
#290
by mary-0830
was closed Jul 19, 2024
updated Jul 19, 2024
ChartQA augmented & CMMMU
Feature Request
#69
by KosumosuL
was closed Jul 19, 2024
updated Jul 19, 2024
[BUG] local variable 'structs' referenced before assignment
#271
by Yunnglin
was closed Jul 12, 2024
updated Jul 12, 2024
[bug] The same sample was run twice in mp_utils.py
#272
by buptweixin
was closed Jul 11, 2024
updated Jul 11, 2024
如何多卡并行测评?How to evaluate on multiple cards in parallel?
#263
by buptweixin
was closed Jul 10, 2024
updated Jul 10, 2024
[Request]Consider integrating MMT-Bench and CONTEXTUAL?
#196
by iamlockelightning
was closed Jul 9, 2024
updated Jul 9, 2024
SEEDBench2 and SEEDBench2_Plus give the same result
#247
by ruifengma
was closed Jul 1, 2024
updated Jul 1, 2024
MMBench_EN_V11 404 not found after evaluation
#239
by MonolithFoundation
was closed Jun 26, 2024
updated Jun 26, 2024
about vlmeval/evaluate/multiple_choice.py
#228
by mary-0830
was closed Jun 20, 2024
updated Jun 20, 2024
about vlmeval/evaluate/multiple_choice.py
#227
by mary-0830
was closed Jun 20, 2024
updated Jun 20, 2024
The code for evaluating open question of MMMU is completely wrong
#110
by xwwu2015
was closed Jun 17, 2024
updated Jun 17, 2024
Questions regarding the metrics for SEED bench
#20
by zjysteven
was closed Dec 23, 2023
updated Jun 14, 2024
Previous Next
ProTip!
Exclude everything labeled
bug
with -label:bug.