🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference,…

Machine Learning Engineering Open Book

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Official PyTorch implementation of StyleGAN3

A framework for few-shot evaluation of language models.

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Just playing with getting VQGAN+CLIP running locally, rather than having to use colab.

Toolkit for creating, sharing and using natural language prompts.

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

LLM vulnerability scanner

A modular framework for neural networks with Euclidean symmetry

v objective diffusion inference code for PyTorch.

Locating and editing factual associations in GPT (NeurIPS 2022)

MAGMA - a GPT-style multimodal model that can understand any combination of images and language. NOTE: The freely available model from this repo is only a demo. For the latest multimodal and multil…

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch

Tools for understanding how transformer predictions are built layer-by-layer

Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network

Privacy Testing for Deep Learning

State of the Art Magic: the Gathering Draft and DeckBuilder AI.

Code repository for our paper "Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift": https://arxiv.org/abs/1810.11953

A framework for few-shot evaluation of autoregressive language models.

Implementation of Marge, Pre-training via Paraphrasing, in Pytorch

CLOOB training (JAX) and inference (JAX and PyTorch)

Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing

Implementation of LogAvgExp for Pytorch