You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi~Referring to the doc, I think if I add '--output' and '--log_samples' argument I could get every inference result from one doc. I have executed the basic demo with these two arguments like this:lm_eval --model hf --model_args pretrained=EleutherAI/gpt-j-6B --tasks hellaswag --device cuda:0 --batch_size 8 --output_path /home/work/ais-harness-data-test/harness-res/0418/ --log_samples
I have found 'pretrained=EleutherAI__gpt-j-6B_hellaswag.jsonl' under the result output dir but I am confused of the result. The attachment is a result demo of one doc. result_demo.txt
I have found the dataset doc details and some results after computation. However, I do not find the inference result of the model. Has it been provided? If not, is there any way to obtain it or will it be supported in the future. I am looking forward to your reply, thanks a lot
The text was updated successfully, but these errors were encountered:
Hi~Referring to the doc, I think if I add '--output' and '--log_samples' argument I could get every inference result from one doc. I have executed the basic demo with these two arguments like this:
lm_eval --model hf --model_args pretrained=EleutherAI/gpt-j-6B --tasks hellaswag --device cuda:0 --batch_size 8 --output_path /home/work/ais-harness-data-test/harness-res/0418/ --log_samples
I have found 'pretrained=EleutherAI__gpt-j-6B_hellaswag.jsonl' under the result output dir but I am confused of the result. The attachment is a result demo of one doc.
result_demo.txt
I have found the dataset doc details and some results after computation. However, I do not find the inference result of the model. Has it been provided? If not, is there any way to obtain it or will it be supported in the future. I am looking forward to your reply, thanks a lot
The text was updated successfully, but these errors were encountered: