pytorch memory track code

PyTorch native quantization and sparsity for training and inference

⏰ Collaboratively track deadlines of conferences recommended by CCF (Website, Python Cli, Wechat Applet) / If you find it useful, please star this project, thanks~

Infinity is a high-throughput, low-latency REST API for serving text-embeddings, reranking models and clip

The simplest, fastest repository for training/finetuning medium-sized GPTs.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

A high-throughput and memory-efficient inference and serving engine for LLMs

A massively parallel, high-level programming language

教科书《计算机体系结构基础》（胡伟武等，第三版）的开源版本

Evaluate custom and HuggingFace text-to-image/zero-shot-image-classification models like CLIP, SigLIP, DFN5B, and EVA-CLIP. Metrics include Zero-shot accuracy, Linear Probe, Image retrieval, and KN…

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

GeoGuessrAI, Predict the coordinates of a given street view image 🗺️

Tower of the Sorcerer for Windows Kai (改): Modifier of game variables and improvement of game experience

TinyChatEngine: On-Device LLM Inference Library

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

What would you do with 1000 H100s...

Decompilation of The Legend of Zelda: Breath of the Wild (Switch 1.5.0)

Official PyTorch implementation of "TinySAM: Pushing the Envelope for Efficient Segment Anything Model"

ViT Prisma is a mechanistic interpretability library for Vision Transformers (ViTs).

Image2Emoji, Predict the most relevant emoji for a given image 🐱

For peering into the brain of CLIP and other vision models!

Minimalist ML framework for Rust

🦜🔗 Build context-aware reasoning applications

Machine Learning Engineering Open Book

Puzzles for learning Triton

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.

MiniCPM-2B: An end-side LLM outperforming Llama2-13B.