A high-throughput and memory-efficient inference and serving engine for LLMs

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

A high-throughput and memory-efficient inference and serving engine for LLMs

Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes

Sparsity-aware deep learning inference runtime for CPUs

Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

Contour Location Via Entropy Reduction (NeurIPS 2018)