A framework for few-shot evaluation of language models.

Efficiently computing & storing token n-grams from large corpora

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Adds GaLore style projection wrappers to optax optimizers

The hub for EleutherAI's work on interpretability and learning dynamics

Keeping language models honest by directly eliciting knowledge encoded in their activations.

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.