An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

Plug and play modules to optimize the performances of your AI systems 🚀

Flexible components pairing 🤗 Transformers with Pytorch Lightning

All lecture notes, slides and assignments from CS224n: Natural Language Processing with Deep Learning class by Stanford