Block or Report
Block or report rhmaaa
Contact GitHub support about this user’s behavior. Learn more about reporting abuse.
Report abuseStars
Language
Sort by: Recently starred
collection of benchmarks to measure basic GPU capabilities
Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.
使用 cutlass 实现 flash-attention 精简版,具有教学意义
Code for Palu: Compressing KV-Cache with Low-Rank Projection
Demonstration of various hardware effects on CUDA GPUs.
使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention
SGLang is yet another fast serving framework for large language models and vision language models.
Large Language Model Text Generation Inference
深度学习系统笔记,包含深度学习数学基础知识、神经网络基础部件详解、深度学习炼丹策略、模型压缩算法详解,以及如何实现深度学习推理框架实战。
Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
Source code for Twitter's Recommendation Algorithm
Using a four layer perceptron, The highest accuracy can reach 97%。
🏠 [ECCV 2024] Pytorch implementation of 'HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression'
[ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs
Accessible large language models via k-bit quantization for PyTorch.
QQQ is an innovative and hardware-optimized W4A8 quantization solution.
[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..
KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
The best library for implementation of all Data Structures and Algorithms - Trees + Graph Algorithms too!
Lightning fast C++/CUDA neural network framework