Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Winogrande evaluation #123

Merged
merged 1 commit into from
Feb 3, 2021

Conversation

jon-tow
Copy link
Member

@jon-tow jon-tow commented Feb 3, 2021

Reference Issue: #13

@codecov
Copy link

codecov bot commented Feb 3, 2021

Codecov Report

Merging #123 (2ba0e70) into master (e12d007) will increase coverage by 2.02%.
The diff coverage is 79.31%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #123      +/-   ##
==========================================
+ Coverage   59.30%   61.33%   +2.02%     
==========================================
  Files          31       31              
  Lines        1848     1857       +9     
==========================================
+ Hits         1096     1139      +43     
+ Misses        752      718      -34     
Impacted Files Coverage Δ
lm_eval/tasks/__init__.py 92.30% <ø> (ø)
lm_eval/tasks/wsc273.py 74.60% <74.60%> (+32.29%) ⬆️
lm_eval/tasks/winogrande.py 92.85% <91.66%> (+45.12%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19b0f52...485e45b. Read the comment docs.

@jon-tow jon-tow self-assigned this Feb 3, 2021
@jon-tow jon-tow requested a review from leogao2 February 3, 2021 05:23
@leogao2 leogao2 merged commit a75a948 into EleutherAI:master Feb 3, 2021
@jon-tow jon-tow deleted the winogrande-evaluation branch February 3, 2021 23:49
qmdnls pushed a commit to qmdnls/lm-evaluation-harness that referenced this pull request Aug 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants