Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alghafa benchmark #1946

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

khalil-Hennara
Copy link
Contributor

This is a continue process of adding the AlGhafa benchmark, within this request I've add the BoolQ and OpenBookQ

@lintangsutawika
Copy link
Contributor

Would this complete all the tasks the comprise of the AlGhafa benchmark?

@khalil-Hennara
Copy link
Contributor Author

khalil-Hennara commented Jun 11, 2024

No there are another 5 datasets

@lintangsutawika
Copy link
Contributor

I think it would be great the have those in this PR as well.

@khalil-Hennara
Copy link
Contributor Author

I can add all the other data in one pull request but this would take some time, if it's ok I can add it on this pull the request, or I can make a new pull request with the other

@lintangsutawika
Copy link
Contributor

I think it's fine to have them in 1 PR. So that its easier to track.

@khalil-Hennara
Copy link
Contributor Author

image
I didn't face any problem with the pre-commit command but I don't know what is the problem with faling unit Test

@khalil-Hennara
Copy link
Contributor Author

khalil-Hennara commented Jun 27, 2024

Can you please help me with this failed test @lintangsutawika , I have used the same format and code from original task Toxigen https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/toxigen , I would appreciate your help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants