Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
Clear current search query, filters, and sorts
76 Open 1,035 Closed
76 Open 1,035 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Fail gracefully upon tokenizer logging failure (#2035)
#2038 by haileyschoelkopf was merged Jun 29, 2024 Loading…
1
1
fix cache
#2037 by baberabb was merged Jun 28, 2024 Loading…
1
Fix strip whitespace filter
#2036 by NathanHB was closed Jun 28, 2024 Loading…
Add chat template to vllm
#2034 by baberabb was merged Jun 28, 2024 Loading…
1
6
Fix trust_remote_code-related test failures
#2024 by haileyschoelkopf was merged Jun 26, 2024 Loading…
1
adds leaderboard tasks
#2023 by NathanHB was closed Jun 26, 2024 Loading…
Add MMLU-ru based on MERA
#2019 by SpirinEgor was closed Jun 25, 2024 Loading…
1
Hotfix breaking import
#2015 by StellaAthena was merged Jun 24, 2024 Loading…
Remove LM dependency from build_all_requests
#2011 by baberabb was merged Jun 25, 2024 Loading…
1
Remove LM dependency from build_all_requests
#2009 by baberabb was closed Jun 22, 2024 Loading…
1
Fixes scrolls task bug with few_shot examples
#2003 by xksteven was merged Jun 28, 2024 Loading…
9
Fix Datasets --trust_remote_code
#1998 by haileyschoelkopf was merged Jun 19, 2024 Loading…
1
2
Log fewshot_as_multiturn in results files
#1995 by haileyschoelkopf was merged Jun 19, 2024 Loading…
1
Fix naming error; 'include: _paloma_template' -> 'include: paloma.yaml'
#1994 by LucWeber was closed Jun 19, 2024 Loading…
2
Fix Paloma Template yaml
#1993 by haileyschoelkopf was merged Jun 19, 2024 Loading…
1
added yaml and util file
#1991 by satyamshukl was closed Jun 25, 2024 Loading…
3
Fix self assignment in neuron_optimum.py
#1990 by LSinev was merged Jun 18, 2024 Loading…
2
Added ArabicMMLU
#1987 by Yazeed7 was merged Jun 19, 2024 Loading…
7
Added ArabicMMLU
#1986 by Yazeed7 was closed Jun 18, 2024 Loading…
1
add trust_remote_code for piqa
#1983 by changwangss was merged Jun 18, 2024 Loading…
1
Update interface.md
#1982 by johnwee1 was merged Jun 25, 2024 Loading…
3
Add Task: CBT
#1981 by ookkeeeee was closed Jun 25, 2024 Loading…
2
add persianmmlu benchmark for assessing Persian Language understanding
#1977 by MrzEsma was closed Jun 17, 2024 Loading…
1
added bias and stereotype classification tasks
#1974 by aditya20t was closed Jun 17, 2024 Loading…
1
Add GigaChat API
#1973 by seldereyy was closed Jun 19, 2024 • Draft
3
ProTip! Exclude everything labeled bug with -label:bug.