SGLang is yet another fast serving framework for large language models and vision language models.

Utilities intended for use with Llama models.

The Tensor (or Array)

QQQ is an innovative and hardware-optimized W4A8 quantization solution.

AMD’s C++ library for accelerating tensor primitives

Debug print operator for cudagraph debugging

RAND library for HIP programming language

ms-swift: Use PEFT or Full-parameter to finetune 300+ LLMs or 50+ MLLMs. (Qwen2, GLM4v, Internlm2.5, Yi, Llama3.1, Llava-Video, Internvl2, MiniCPM-V, Deepseek, Baichuan2, Gemma2, Phi3-Vision, ...)

Integrate MS-AMP into nanoGPT (https://github.com/karpathy/nanoGPT)

Ongoing research training transformer models at scale

PyTorch bindings for CUTLASS grouped GEMM.

KvikIO - High Performance File IO

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

Fast and memory-efficient exact attention

This is a cross-chip platform collection of operators and a unified neural network library.

BizyAir: Comfy Nodes that can run in any environment.

Brand new TTS solution

A fast communication-overlapping library for tensor parallelism on GPUs.

NVIDIA Math Libraries for the Python Ecosystem

Efficient operation implementation based on the Cambricon Machine Learning Unit (MLU) .

CloudAI Benchmark Framework

PaddleAPEX：Paddle Accuracy and Performance EXpansion pack

A large-scale simulation framework for LLM inference

YaRN: Efficient Context Window Extension of Large Language Models

LLM101n: Let's build a Storyteller

Shared Middle-Layer for Triton Compilation