-
Notifications
You must be signed in to change notification settings - Fork 982
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gpt3small is broken #71
Comments
|
Interesting. Can you provide a link to the file? |
https://colab.research.google.com/drive/1IThG90kOdndybKuScNEZ1QG9nwnuj2l1?usp=sharing |
Ah yes, you cannot use parallel pipeline on a single GPU unit. The whole
point of parallel pipeline is to divide the pipeline between GPUs, which
you cannot do. You can manually set num_stages to 1, or you can use the
non-pipeline code.
…On Wed, Jan 20, 2021 at 11:15 PM srulikbd ***@***.***> wrote:
https://colab.research.google.com/drive/1IThG90kOdndybKuScNEZ1QG9nwnuj2l1?usp=sharing
there is some problem with the pipeline code. I'm trying fixing it.
something with
num_stages (2) must divide distributed world size (1)
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#71 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADZJVMBDQ76DLXFSFZZAWVDS26S75ANCNFSM4WHTG4SA>
.
|
ok, what I suspected. thanks! |
Have you checked out the [Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness)? That is extremely
important to both this project and other ones, and is more accessible to
people who can write code but are newcomers to AI research. Each task that
needs completing has an associated Issue.
…On Wed, Jan 20, 2021 at 11:53 PM srulikbd ***@***.***> wrote:
ok, what I suspected. thanks!
how can I help right now?
I saw that we need kubernetes skills, so I'm learning it.
anything else for a newcomer?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#71 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADZJVMBE72SG77DM3M7KYG3S26XMFANCNFSM4WHTG4SA>
.
|
gpt3small seems to have been left behind in some of our updates, and neither
scripts/train_gpt3small.sh
norscripts/train_gpt3small_pipeline.sh
run.The text was updated successfully, but these errors were encountered: