🔍 A Hex Editor for Reverse Engineers, Programmers and people who value their retinas when working at 3 AM.

SOTA Weight-only Quantization Algorithm for LLMs. This is official implementation of "Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs"

Named Tensors for Legible Deep Learning in JAX

QUIC and HTTP/3 implementation in Python

The official PyTorch implementation of Google's Gemma models

Animation engine for explanatory math videos

LLM training in simple, raw C/HIP for AMD GPUs

An OpenAI API compatible API for chat with image input and questions about the images. aka Multimodal.

📚 Download the full collection of Paul Graham essays in EPUB, PDF & Markdown for easy reading.

The implementation for paper "Token-wise Influential Training Data Retrieval for Large Language Models" (Accepted on ACL 2024).

TORAX: Tokamak transport simulation in JAX

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

Experimental q[X]ora kernel development code

Scalable toolkit for efficient model alignment

Fast lexical search library implementing BM25 in Python using Scipy (on average 2x faster than Elasticsearch in single-threaded setting)

Implementation for MatMul-free LM.

Code for simulating vLLM

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frame…

Proof-of-concept of global switching between numpy/jax/pytorch in a library.

A high-throughput and memory-efficient inference and serving engine for LLMs

Latent Large Language Models

MambaOut: Do We Really Need Mamba for Vision?

Display and control your Android device

GPU & Accelerator process monitoring for AMD, Apple, Huawei, Intel, NVIDIA and Qualcomm

Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"

Collective communications library with various primitives for multi-machine training.

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization