Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

Structured Text Generation

Materials for learning SGLang

A Bulletproof Way to Generate Structured JSON from Language Models

AI powered one-click comprehensive docs from transcripts and text.

OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models

Material for gpu-mode lectures

Entropy Based Sampling and Parallel CoT Decoding

extensible collectives library in triton

LLM training code for Databricks foundation models

A native PyTorch Library for large model training

Because it's there.

Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding

16-fold memory access reduction with nearly no loss

Simple and fast low-bit matmul kernels in CUDA / Triton

A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 and reasoning techniques.

An implementation of the Llama architecture, to instruct and delight

Long context evaluation for large language models

A throughput-oriented high-performance serving framework for LLMs

Code for Palu: Compressing KV-Cache with Low-Rank Projection

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on infer…

Applied AI experiments and examples for PyTorch

Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

Efficient Triton Kernels for LLM Training

A curated collection of LLM reasoning and planning resources, including key papers, limitations, benchmarks, and additional learning materials.