Skip to content

Issues: EleutherAI/lm-evaluation-harness

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
7 Open 8 Closed
7 Open 8 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

ImportError: cannot import name 'HfApi' from 'huggingface_hub'
#1889 by baberabb was closed May 26, 2024
@haileyschoelkopf
2
exact_match computation very slow for multiple target tasks
#1264 by baberabb was closed Jan 10, 2024
1
Consolidate Reorderer, Grouper and chunks feature request A feature that isn't implemented yet.
#1140 by baberabb was closed Dec 23, 2023
1
4
HF generations - discrepancy with Llama bsz > 1 bug Something isn't working.
#1090 by baberabb was closed Dec 14, 2023
1
1
[Refactor] IndexError if label > num_of_choices for multiple choice
#811 by baberabb was closed Aug 31, 2023
@lintangsutawika
2
[Refactor] Add OpenAI Chat Completion API feature request A feature that isn't implemented yet.
#737 by baberabb was closed Dec 11, 2023
1
[Refactor] output_path
#681 by baberabb was closed Jul 17, 2023
1
[Refactor] bug(maybe) in Instance
#677 by baberabb was closed Jul 14, 2023
4
ProTip! Follow long discussions with comments:>50.