-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Issues: EleutherAI/lm-evaluation-harness
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
add meta tasks
feature request
A feature that isn't implemented yet.
#217
by leogao2
was closed Nov 21, 2022
No such file or directory: 'data/unscramble/mid_word_1_anagrams.jsonl.gz'
#249
by stas00
was closed Feb 13, 2022
Review all GLUE + SuperGLUE tasks.
opinions wanted
For discussing open questions.
#248
by zphang
was closed Nov 8, 2023
Multilingual StoryCloze
feature request
A feature that isn't implemented yet.
good first issue
Good for newcomers
#241
by StellaAthena
was closed May 11, 2023
Implement GPT-3 style contamination study
feature request
A feature that isn't implemented yet.
#231
by StellaAthena
was closed Nov 1, 2023
Implement SCAN
feature request
A feature that isn't implemented yet.
#224
by StellaAthena
was closed Nov 21, 2022
Implement the Contract Understanding Atticus Dataset (CUAD)
feature request
A feature that isn't implemented yet.
#221
by StellaAthena
was closed Nov 21, 2022
Citing this repo
documentation
Improvements or additions to documentation.
#211
by yuvalkirstain
was closed Dec 20, 2021
Add MKQA
feature request
A feature that isn't implemented yet.
#208
by sdtblck
was closed Nov 21, 2022
Add LAMA + MLAMA
feature request
A feature that isn't implemented yet.
#207
by sdtblck
was closed Nov 21, 2022
Add XNLI
feature request
A feature that isn't implemented yet.
#199
by sdtblck
was closed Mar 15, 2024
Implement the APPS evaluation
feature request
A feature that isn't implemented yet.
#196
by leogao2
was closed Jan 15, 2023
Implement the ASDiv Evaluation
feature request
A feature that isn't implemented yet.
#190
by leogao2
was closed Jan 4, 2022
ProTip!
Type g i on any issue or pull request to go back to the issue listing page.