-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement arithmetic evaluations #25
Comments
I'll take this! |
@VitamintK Hey Kevin, how's this coming? Still planning on doing it? |
I am, but I've been putting it off and haven't started yet. |
No worries! Just wanted to check in and make sure it was on your radar. Do you think you'll be able to get it done in the next week or two? Finishing implementing the eval datasets is currently the major project blocker; we need to finish this before we can dedupe and we need to do that before we can start training models. |
Yup, I'll try to get it done soon! |
@VitamintK What's happened? Is this something you have done? Should I assign it to someone else? |
@StellaAthena hey, sorry for not following up with this until now, I am the worst. Finally got around to it, though! |
Adding CrowsPairs task for English and French
Adding CrowsPairs task for English and French
Adding CrowsPairs task for English and French
From the GPT-3 paper:
This should be modeled after the
BoolQ
task inlm_eval/tasks/suerglue.py
The text was updated successfully, but these errors were encountered: