-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Issues: EleutherAI/lm-evaluation-harness
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Should num_fewshot be type list?
feature request
A feature that isn't implemented yet.
#837
opened Sep 6, 2023 by
Wehzie
Quac Dataset
feature request
A feature that isn't implemented yet.
good first issue
Good for newcomers
#827
opened Sep 4, 2023 by
RanchiZhao
Pile tasks on big-refactor use dataset_names from old dataset loader that don't exist on HF
bug
Something isn't working.
good first issue
Good for newcomers
help wanted
Contributors and extra help welcome.
#731
opened Aug 3, 2023 by
yeoedward
Add TheoremQA
feature request
A feature that isn't implemented yet.
good first issue
Good for newcomers
help wanted
Contributors and extra help welcome.
#518
opened May 23, 2023 by
StellaAthena
"Please select a token to use as
pad_token
" error for alpaca-lora-7b
model
#434
opened Apr 24, 2023 by
oshev
MNLI task giving (very) different results than the HuggingFace task accuracy metric
bug
Something isn't working.
good first issue
Good for newcomers
help wanted
Contributors and extra help welcome.
#320
opened May 8, 2022 by
JunShern
Implement TyDiQA
feature request
A feature that isn't implemented yet.
good first issue
Good for newcomers
help wanted
Contributors and extra help welcome.
#193
opened Jun 10, 2021 by
sdtblck
Implement MLQA
feature request
A feature that isn't implemented yet.
good first issue
Good for newcomers
help wanted
Contributors and extra help welcome.
#192
opened Jun 10, 2021 by
sdtblck
Implement XQuAD
feature request
A feature that isn't implemented yet.
good first issue
Good for newcomers
help wanted
Contributors and extra help welcome.
#191
opened Jun 10, 2021 by
sdtblck
Implement the SuperGLUE evaluation
feature request
A feature that isn't implemented yet.
#22
opened Sep 16, 2020 by
StellaAthena
1 of 2 tasks
Previous Next
ProTip!
Type g p on any issue or pull request to go back to the pull request listing page.