-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Issues: EleutherAI/lm-evaluation-harness
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
ValueError when task name collide with local directory names
#2122
opened Jul 20, 2024 by
alat-rights
eval gsm8k from local dataset folder with the bug info "ValueError: BuilderConfig 'main' not found."
#1829
opened May 12, 2024 by
Jp-17
TypeError: 'NoneType' object is not iterable when using cache and loglikelihood_rolling
#1821
opened May 10, 2024 by
mdocekal
ValueError: BuilderConfig 'pile_freelaw' not found., issue on running PILE eval
#1714
opened Apr 16, 2024 by
Harryalways317
List of A feature that isn't implemented yet.
until
tokens in cli args not supported
feature request
#1677
opened Apr 5, 2024 by
tianyi-chen
Add a docs FAQ section
documentation
Improvements or additions to documentation.
#1676
opened Apr 5, 2024 by
haileyschoelkopf
Clarification on API Endpoint: /v1/completions vs /v1/chat/completions
#1637
opened Mar 26, 2024 by
gerayking
Allow registering custom LM implementations without requiring loading and modifying lm_eval code
bug
Something isn't working.
#1621
opened Mar 22, 2024 by
apetrov-msk
Negative perplexity values
asking questions
For asking for clarification / support on library usage.
#1595
opened Mar 17, 2024 by
shikhar-srivastava
Make Adding New MCQA Metrics Easier
feature request
A feature that isn't implemented yet.
#1585
opened Mar 15, 2024 by
haileyschoelkopf
When using Something isn't working.
parallelize=True
, raise Runtime Error: expected all tensors to be on the same device
bug
#1575
opened Mar 14, 2024 by
feiba54
Add New Lambada Translations
good first issue
Good for newcomers
#1501
opened Feb 29, 2024 by
haileyschoelkopf
Previous Next
ProTip!
no:milestone will show everything without a milestone.