🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)

计图大模型推理库，具有高性能、配置要求低、中文支持好、可移植等特点

oneCCL Bindings for Pytorch*

A model compilation solution for various hardware

LLM101n: Let's build a Storyteller

Hands-On Practical MLIR Tutorial

CodeGeeX4-ALL-9B, a versatile model for all AI software development scenarios, including code completion, code interpreter, web search, function calling, repository-level Q&A and much more.

Hands-On Practical MLIR Tutorial

Development repository for the Triton language and compiler

Run generative AI models in sophgo BM1684X

An innovative library for efficient LLM inference via low-bit quantization

Model compression for ONNX

一键命令下载飞书文档为 Markdown

[MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving

Tensor library for machine learning

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ, and export to onnx/onnx-runtime easily.

2021年最新整理， C++ 学习资料，含C++ 11 / 14 / 17 / 20 / 23 新特性、入门教程、推荐书籍、优质文章、学习笔记、教学视频等

Onboarding documentation source for the AMD Ryzen™ AI Software Platform. The AMD Ryzen™ AI Software Platform enables developers to take pretrained machine learning models in popular frameworks and …

Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.

Model Quantization Benchmark

Awesome LLM compression research papers and tools.

Open deep learning compiler stack for Kendryte AI accelerators ✨

[ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models"

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".