#
abduld
Follow
Starred repositories
A fast communication-overlapping library for tensor parallelism on GPUs.
AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and versatility of software and hardware.
Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
List of papers related to neural network quantization in recent AI conferences and journals.
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.
A framework for few-shot evaluation of language models.
A tool for generating information about the matrix multiplication instructions in AMD Radeon™ and AMD Instinct™ accelerators
A one stop repository for generative AI research updates, interview resources, notebooks and much more!
RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing …
Python bindings for the Transformer models implemented in C/C++ using GGML library.
Assembler and Decompiler for NVIDIA (Maxwell Pascal Volta Turing Ampere) GPUs.
Home of the JELOS Linux distribution.
A Easy-to-understand TensorOp Matmul Tutorial
WebUI for Fine-Tuning and Self-hosting of Open-Source Large Language Models for Coding
