collection of benchmarks to measure basic GPU capabilities

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

使用 cutlass 实现 flash-attention 精简版，具有教学意义

小彭老师领衔编写，现代C++的中文百科全书

x86-64 SIMD矢量优化系列教程

Code for Palu: Compressing KV-Cache with Low-Rank Projection

Demonstration of various hardware effects on CUDA GPUs.

使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention

SGLang is yet another fast serving framework for large language models and vision language models.

Large Language Model Text Generation Inference

A way to use cuda to accelerate top k algorithm

深度学习系统笔记，包含深度学习数学基础知识、神经网络基础部件详解、深度学习炼丹策略、模型压缩算法详解，以及如何实现深度学习推理框架实战。

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>

LLM101n: Let's build a Storyteller

Source code for Twitter's Recommendation Algorithm

Using a four layer perceptron, The highest accuracy can reach 97%。

🏠 [ECCV 2024] Pytorch implementation of 'HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression'

Learning about CUDA by writing PTX code.

[ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs

Accessible large language models via k-bit quantization for PyTorch.

QQQ is an innovative and hardware-optimized W4A8 quantization solution.

[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling

A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

The best library for implementation of all Data Structures and Algorithms - Trees + Graph Algorithms too!

Lightning fast C++/CUDA neural network framework