A high-throughput and memory-efficient inference and serving engine for LLMs

Latent Large Language Models

MambaOut: Do We Really Need Mamba for Vision?

Display and control your Android device

GPU & Accelerator process monitoring for AMD, Apple, Huawei, Intel, NVIDIA and Qualcomm

Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"

Collective communications library with various primitives for multi-machine training.

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization

YaFSDP: Yet another Fully Sharded Data Parallel

Neural Networks: Zero to Hero

This project is a implementation in PyTorch for ZO-AdaMU optimization: Adapting Perturbation with the Momentum and Uncertainty in Zeroth-order Optimization.

Gorilla: An API store for LLMs

LLM Analytics

Examples and tutorials on using SOTA computer vision models and techniques. Learn everything from old-school ResNet, through YOLO and object-detection transformers like DETR, to the latest models l…

CUDA accelerated rasterization of gaussian splatting

Zero-effort CLI interfaces & config objects, from types

🐟 Python profile viewer

GPT-2 (124M) quality in 5B tokens

Python logging made (stupidly) simple

Evaluate your LLM's response with Prometheus and GPT4 💯

🚪✊Knock Knock: Get notified when your training ends with only two additional lines of code

📸 Snapshot plugin with rich features that can make pretty code snapshots for Neovim

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

Make triton easier

Open-source observability for your LLM application, based on OpenTelemetry