Doing simple retrieval from LLM models at various context lengths to measure accuracy

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).

MLX: An array framework for Apple silicon

Find all the fundamental UXI guidelines and pattern-based web components to build brand driven, consistent and intuitive designs for digital Porsche products.

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

Inference Llama 2 in one file of pure 🔥

Python pdb for multiple processes

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Inference code for CodeLlama models

[EMNLP 2022] Training Language Models with Memory Augmentation https://arxiv.org/abs/2205.12674

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines

commaVQ is a dataset of compressed driving video

Tools for building GPU clusters

LLMs for your CLI

A Data Streaming Library for Efficient Neural Network Training

A high-throughput and memory-efficient inference and serving engine for LLMs

The Memory layer for your AI apps

Write scalable load tests in plain Python 🚗💨

CUDA on non-NVIDIA GPUs

It's React, but in Python

Implementation of Flash Attention in Jax

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”

A simulation framework for RLHF and alternatives. Develop your RLHF method without collecting human data.

Tevatron - A flexible toolkit for neural retrieval research and development.

The Mojo Programming Language

Tiny data-over-sound library

The RedPajama-Data repository contains code for preparing large datasets for training large language models.