This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out most of the optimizations in GPT-NeoX for simplicity. It can load the pre-trained parameters.
This repository has been archived by the owner on Aug 11, 2022. It is now read-only.
labmlai/neox
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.