-
Notifications
You must be signed in to change notification settings - Fork 160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pythia v.s. GPT-neox #48
Comments
Hi, The Pythia models and GPT-NeoX-20b model were all trained in the same library, and use a very similar architecture. The Pythia models are primarily intended for research as a consistent set of models. If you’re only looking for performance, then the GPT-J-6B and GPT-NeoX-20b models have been trained on more data and will outperform Pythia models, though the Pythia models should have standard performance for models of their size! |
Thank you for the clarification! |
Hi, thanks for the awesome work.
I notice there are Pythia models and GPT-neox models on Eleuther's HF hub, both seem to be very recent. Wonder what are the the main differences between them, and if there is any preference on one of them.
Thank you.
The text was updated successfully, but these errors were encountered: