Stars
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Data and tools for generating and inspecting OLMo pre-training data.
Modeling, training, eval, and inference code for OLMo
A curated list of engineering blogs
The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
Evolutionary Scale Modeling (esm): Pretrained language models for proteins
A modular RL library to fine-tune language models to human preferences
A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
Hackable and optimized Transformers building blocks, supporting a composable construction.
The simplest, fastest repository for training/finetuning medium-sized GPTs.
A prize for finding tasks that cause large language models to show inverse scaling
DALL·E Mini - Generate images from a text prompt
Code for ACL 2022 Paper: Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons
Search Engines with Autoregressive Language models
Jupyter notebooks for the Natural Language Processing with Transformers book
Graph Data Augmentation Library for PyTorch Geometric
CodeGen is a family of open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.
The first place solution for the NeurIPS 2021 Nethack Challenge -- https://www.aicrowd.com/challenges/neurips-2021-the-nethack-challenge
This repo is meant to serve as a guide for Machine Learning/AI technical interviews.