-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Issues: EleutherAI/lm-evaluation-harness
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
IndexError: index out of bounds for multiple tasks of mmlu_pro
#2240
by lxning
was closed Aug 22, 2024
Can I incorporate my own benchmark as a new task within lm-evaluation-harness?
#2226
by lioooncoder
was closed Aug 19, 2024
Extremely SLOW, even slower than training
asking questions
For asking for clarification / support on library usage.
#2214
by AaronZLT
was closed Aug 22, 2024
AttributeError: module 'torch' has no attribute 'uint16'
#2199
by Shinning-Zhou
was closed Aug 22, 2024
It's hard to look at the code of this repository. Garbage.
#2194
by Runningwater2357
was closed Aug 6, 2024
Issue with
state-spaces/transformerpp-2.7b
when generating
#2182
by jhuang265
was closed Aug 22, 2024
[BUG] Huggingface and Neuron runs fail if model revision is an integer
#2167
by christyler3030
was closed Aug 5, 2024
When using the tmmluplus dataset, I'm encountering a TypeError: init() got an unexpected keyword argument 'group_alias'.
#2151
by zhuangyuan123
was closed Jul 31, 2024
Bug when hashing prompt when using openai-chatcompletions and log_samples
#2138
by eyuansu62
was closed Jul 29, 2024
Getting "jinja2.exceptions.TemplateError: System role not supported" exception with some tasks using --apply_chat_template
#2109
by chimezie
was closed Jul 16, 2024
Previous Next
ProTip!
Type g i on any issue or pull request to go back to the issue listing page.