Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Double check all of the zero/few-shot formats #34

Closed
leogao2 opened this issue Oct 5, 2020 · 0 comments
Closed

Double check all of the zero/few-shot formats #34

leogao2 opened this issue Oct 5, 2020 · 0 comments
Labels
documentation Improvements or additions to documentation.
Projects
New Features

Comments

@leogao2
Copy link
Contributor

leogao2 commented Oct 5, 2020

Once we have everything implemented, we should double check that we actually have the same zero/few shot format as was used for GPT3. The paper is, as expected, terribly vague, so we should try contacting OA for more info. Making sure this is done correctly is critical before proceeding further.
@leogao2 leogao2 changed the title Double check all of the sero/few-shot formats Double check all of the zero/few-shot formats Oct 5, 2020
@StellaAthena StellaAthena added the documentation Improvements or additions to documentation. label Oct 5, 2020
@StellaAthena StellaAthena added this to To do in Implementing Evaluations via automation Oct 23, 2020
@StellaAthena StellaAthena added this to To do in New Features Oct 23, 2020
New Features automation moved this from To do to Done Jan 4, 2021
pruksmhc pushed a commit to pruksmhc/lm-evaluation-harness that referenced this issue May 10, 2022
@jon-tow
Merge pull request EleutherAI#34 from EleutherAI/master
170b410 
Update from EAI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation.
Projects
No open projects
New Features
  
Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@StellaAthena @leogao2