OPT (Open Pre-trained Transformers) is a family of NLP models trained on billions of tokens of text obtained from the internet.
For notes regarding the development of all these models, please refer to our chronicles.
| Model | Parameters | Pretrained weights |
|---|---|---|
| OPT-125M | 125M | part0, part1 |
| OPT-350M | 350M | part0 |
| OPT-1.3B | 1.3B | part0, part1 |
| OPT-2.7B | 2.7B | part0, part1, part2, part3 |
| OPT-6.7B | 6.7B | part0, part1 |
| OPT-13B | 13B | part0, part1 |
| OPT-30B | 30B | part0, part1 |
| OPT-66B | 66B | part0, part1, part2, part3, part4, part5, part6, part7 |
| OPT-175B | 175B | request access here |
We are including a model card (Mitchell et al., 2018) and data card (Gebru et al., 2021) to help with transparency and accountability in model development.
The use of OPT model weights is subject to the Model License.