Skip to content

Commit

Permalink
update eval harness
Browse files Browse the repository at this point in the history
  • Loading branch information
@zhangir-azerbayev
zhangir-azerbayev committed Oct 16, 2023
1 parent 51adb78 commit c2c2202
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lm-evaluation-harness
Submodule lm-evaluation-harness updated 24 files
+93 −194 README.md
+35 −35 configs/majk.json
+18 −0 eval_scripts/env.sh
+43 −0 eval_scripts/eval_34b_maj1.sh
+50 −0 eval_scripts/eval_34b_majk.sh
+42 −0 eval_scripts/eval_7b_maj1.sh
+50 −0 eval_scripts/eval_7b_majk.sh
+0 −19 eval_scripts/generic_run.sh
+0 −22 eval_scripts/generic_run_majk.sh
+0 −11 eval_scripts/llemma_7b.sh
+0 −11 eval_scripts/llemma_7b_majk.sh
+36 −8 lm_eval/mixins.py
+3 −2 lm_eval/models/vllm.py
+8 −7 lm_eval/tasks/hendrycks_test_cot.py
+2 −2 lm_eval/tasks/math_sat_cot.py
+41 −35 lm_eval/tasks/math_tasks.py
+212,225 −0 output/codellama_34b_maj1.json
+212,225 −0 output/codellama_7b_maj1.json
+212,225 −0 output/llama-2_7b_maj1.json
+212,225 −0 output/llemma_34b_maj1.json
+212,225 −0 output/llemma_7b_maj1.json
+32 −0 score_math.py
+28 −0 score_mmlu.py
+184 −0 unsafe_score_minerva_math.py

0 comments on commit c2c2202

Please sign in to comment.