Pull requests: EleutherAI/lm-evaluation-harness

75 Open 1,035 Closed

Author

Filter by author

Loading

author:

Filter by this user

Label

Filter by label

Loading

Projects

Filter by project

Loading

Milestones

Filter by milestone

Loading

Reviews

Filter by reviews

No reviews Review required Approved review Changes requested

Assignee

Filter by who’s assigned

Assigned to nobody Loading

assignee:

Filter by this user

Sort

Sort by

Newest Oldest Most commented Least commented Recently updated Least recently updated Best match

Most reactions

👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fix wandb logger module import in example

#2041 opened Jun 30, 2024 by ToluClassics Loading…

1

[Draft] Exploring multimodality

#2039 opened Jun 28, 2024 by haileyschoelkopf Loading…

5

add mmlusr tasks

#2032 opened Jun 28, 2024 by SkySuperCat Loading…

1

swahili_ARC_Challenge

#2031 opened Jun 27, 2024 by msamwelmollel Loading…

1

Use shell=False in subprocess Function Calls

#2030 opened Jun 27, 2024 by pixeeai Loading…

1

Update trust_remote_code for Hellaswag

#2029 opened Jun 27, 2024 by haileyschoelkopf Loading…

[add] multiple-choice-question versions of fld benchmark

#2022 opened Jun 26, 2024 by MorishT Loading…

3

Add Redlite tasks for safety benchmarking

#2020 opened Jun 25, 2024 by inno-simon Loading…

1

[Not For Merge] Enable chat-template for vLLM

#2017 opened Jun 25, 2024 by akjindal53244 Loading…

1

Fix regexp parsing for bbh_cot_fewshot

#2013 opened Jun 24, 2024 by arkapal3 Loading…

1

Added MedConceptsQA Benchmark

#2010 opened Jun 22, 2024 by Ofir408 Loading…

1

Refactor API models

#2008 opened Jun 22, 2024 by baberabb Loading…

Error Correction: Eliminate undefined parameter in function call

#2006 opened Jun 21, 2024 by zhabuye Loading…

2

make pytorch an optional dependency

#2004 opened Jun 20, 2024 by dlwh Loading…

1

5

Handle Empty openai response

#1999 opened Jun 19, 2024 by ciaranby Loading…

Fix partial caching of openai models

#1997 opened Jun 19, 2024 by ciaranby Loading…

1

Add Gigachat model

#1996 opened Jun 19, 2024 by seldereyy • Draft

Add HumanEval

#1992 opened Jun 19, 2024 by hjlee1371 Loading…

1

1

[Fix] Replace generic exception classes with a more specific ones

#1989 opened Jun 18, 2024 by LSinev Loading…

7

main

#1988 opened Jun 18, 2024 by msamwelmollel Loading…

5

add persianmmlu benchmark for assessing Persian Language understanding

#1979 opened Jun 17, 2024 by MrzEsma Loading…

3

Fix local completion huggingface tokenizer

#1975 opened Jun 17, 2024 by okdshin Loading…

1

mela

#1970 opened Jun 16, 2024 by Geralt-Targaryen Loading…

2

Fix OpenAI API discrepancies

#1969 opened Jun 14, 2024 by chimezie Loading…

Mmlu Pro

#1961 opened Jun 13, 2024 by ysjprojects Loading…

1

10