Efficient Triton Kernels for LLM Training

Helpful tools and examples for working with flex-attention

The official implementation of Self-Play Preference Optimization (SPPO)

Schedule-Free Optimization in PyTorch

[ICML 2024] LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery

🙌 OpenHands: Code Less, Make More

Code of NeurIPS paper: arxiv.org/abs/2302.08224

Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision

Simple and efficient pytorch-native transformer training and inference (batched)

Granite Code Models: A Family of Open Foundation Models for Code Intelligence

The simplest, fastest repository for training/finetuning medium-sized GPTs. Now, with kittens!

Tile primitives for speedy kernels

A ChatGPT(GPT-3.5) & GPT-4 Workload Trace to Optimize LLM Serving Systems

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

Linear algebra foundation for the Rust programming language

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

CoreNet: A library for training deep neural networks

Mesh TensorFlow: Model Parallelism Made Easier

Arena-Hard-Auto: An automatic LLM benchmark.

A JAX research toolkit for building, editing, and visualizing neural networks.

Turn jitted jax functions back into python source code

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

The official Meta Llama 3 GitHub site

LLM training in simple, raw C/CUDA

Open weights language model from Google DeepMind, based on Griffin.