Improving Alignment and Robustness with Circuit Breakers

A Native-PyTorch Library for LLM Fine-tuning

Machine Learning Engineering Open Book

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

Minimalistic large language model 3D-parallelism training

Friends don't let friends make certain types of data visualization - What are they and why are they bad.

A simple but complete full-attention transformer with a set of promising experimental features from various papers

The hub for EleutherAI's work on interpretability and learning dynamics

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Cramming the training of a (BERT-type) language model into limited compute.

Algorithmically create or extend categorical colour palettes

An autoregressive character-level language model for making more things

Figure sizes, font sizes, fonts, and more configurations at minimal overhead. Fix your journal papers, conference proceedings, and other scientific publications.

Code to run the TILT transfer learning experiments

NAACL 2022: MCSE: Multimodal Contrastive Learning of Sentence Embeddings

A library for parameter-efficient and composable transfer learning for NLP with sparse fine-tunings.

An original implementation of "Noisy Channel Language Model Prompting for Few-Shot Text Classification"

This code accompanies the paper "Bayesian Framework for Information-Theoretic Probing" published in EMNLP 2021.

Train Dense Passage Retriever (DPR) with a single GPU

Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

An Open-Source Framework for Prompt-Learning.

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

Mistral: A strong, northwesterly wind: Framework for transparent and accessible large-scale language model training, built with Hugging Face 🤗 Transformers.

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Model parallel transformers in JAX and Haiku