Modeling, training, eval, and inference code for OLMo

Data and tools for generating and inspecting OLMo pre-training data.

A framework for few-shot evaluation of language models.

The hub for EleutherAI's work on interpretability and learning dynamics

Generate Linked Document Pretraining data for LLMs

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Seminar on Large Language Models (COMP790-101 at UNC Chapel Hill, Fall 2022)

Paper List for Contrastive Learning for Natural Language Processing

A declarative drawing API in Python

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

A set of Python classes that transport OpenCV images from one computer to another using PyZMQ messaging.

A system for assigning and grading notebooks

A complete computer science study plan to become a software engineer.