Pull requests: EleutherAI/lm-evaluation-harness

75 Open 1,035 Closed

Author

Filter by author

Loading

author:

Filter by this user

Label

Filter by label

Loading

Projects

Filter by project

Loading

Milestones

Filter by milestone

Loading

Reviews

Filter by reviews

No reviews Review required Approved review Changes requested

Assignee

Filter by who’s assigned

Assigned to nobody Loading

assignee:

Filter by this user

Sort

Sort by

Newest Oldest Most commented Least commented Recently updated Least recently updated Best match

Most reactions

👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

mlx Model (loglikelihood & generate_until)

#1902 opened May 29, 2024 by chimezie Loading…

5 👍 1

Vllm get tokenizer

#1794 opened May 6, 2024 by AguirreNicolas Loading…

1 👍 1

Alternative Worlds Prompts for Various Tasks and Benchmarks

#925 opened Oct 16, 2023 by lintangsutawika • Draft

3

[API] Add octoai back-end

#936 opened Oct 19, 2023 by vvchernov Loading…

10

Update scorer for gsm8k task

#943 opened Oct 24, 2023 by vvchernov • Draft

Update scorer for TriviaQA task

#944 opened Oct 24, 2023 by vvchernov • Draft

6

[API] Use private HF token for HF models and hide it when print to json file or console

#968 opened Nov 6, 2023 by vvchernov Loading…

4

Added no-softmax entries to MODEL_REGISTRY

#1052 opened Dec 2, 2023 by denizyuret Loading…

Add Selfcheckgpt evaluation to tasks

#1080 opened Dec 7, 2023 by PingNie1 Loading…

21

add all vlsp

#1123 opened Dec 14, 2023 by qnguyen3 • Draft

15

Standardize metrics

#1167 opened Dec 19, 2023 by lintangsutawika • Draft

1

9

Add various social bias tasks

#1185 opened Dec 21, 2023 by oskarvanderwal Loading…

1 task

11

Add task table

#1219 opened Dec 28, 2023 by baberabb • Draft

1

3

Add Cohere API as available language model

#395 opened Mar 10, 2023 by rdnfn Loading…

29

Deal with _encode_pair() / Llama token 29871 / SPIECE_UNDERLINE better bug Something isn't working.

#1322 opened Jan 19, 2024 by haileyschoelkopf • Draft

1

Add Group-Config

#1373 opened Jan 31, 2024 by lintangsutawika • Draft

fix wandb logger module import in example

#2041 opened Jun 30, 2024 by ToluClassics Loading…

1

Add parallel processing for OpenAI completion models

#1460 opened Feb 22, 2024 by pbevan1 Loading…

5

Transfer zero-shot BBH parsing improvements to few-shot BBH

#1481 opened Feb 26, 2024 by haileyschoelkopf • Draft

Adding new task: Boxes

#1557 opened Mar 11, 2024 by irafayabdul Loading…

8

add context-based requests processing

#1571 opened Mar 13, 2024 by artemorloff Loading…

1

17

Add natural questions in a the closedbook setup.

#1649 opened Mar 29, 2024 by OhadRubin • Draft

1

Physics GRE task added

#1655 opened Apr 1, 2024 by ShayekhBinIslam Loading…

1

10

Klokan-qa task

#1657 opened Apr 1, 2024 by hynky1999 Loading…

10

start working on test code for zeno

#1221 opened Dec 28, 2023 by Sparkier • Draft

1