-
Notifications
You must be signed in to change notification settings - Fork 154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pytia or GPT-neox? #138
Comments
Hi! Please see the Pythia, page 18 for notes on this--we renamed the model sizes to count the embed+unembed parameters in the models. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the evals/bias-evals/ dir there are files starting with pythia 350m and pythia 1.3B, but those aren't sizes of pythia v0 nor v1, right? So is it 410m and 1.4B or is it not pythia?
Or am I missing something in the experimental setting?
The text was updated successfully, but these errors were encountered: