Skip to content

Issues: EleutherAI/lm-evaluation-harness

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort

Issues list

Evaluate Gemma with Chat Template
#2069 opened Jul 5, 2024 by pyf98
TinyBenchmark/TinyMMLU broken?
#2068 opened Jul 5, 2024 by skramer-dev
LLM leader board setting for mmlu.
#2066 opened Jul 5, 2024 by dsj96
Duplicate sample entries
#2025 opened Jun 26, 2024 by baberabb
Supporting Multimodality
#2014 opened Jun 24, 2024 by lintangsutawika
Implementing lessons from OLMES
#2002 opened Jun 20, 2024 by lintangsutawika
Long time testing Qwen2-72B bug Something isn't working.
#1984 opened Jun 18, 2024 by djstrong
Making torch dep optional?
#1959 opened Jun 12, 2024 by dlwh
OOM Issue
#1923 opened Jun 4, 2024 by zhentingqi
TypeError of scrolls_narrativeqa
#1891 opened May 27, 2024 by hicleo
Add more math evaluation tasks
#1869 opened May 22, 2024 by jordane95
ProTip! Follow long discussions with comments:>50.