LLM training in simple, raw C/CUDA

Tile primitives for speedy kernels

how to optimize some algorithm in cuda.

GPU database engine

[MICRO'23, MLSys'22] TorchSparse: Efficient Training and Inference Framework for Sparse Convolution on GPUs.

FlashInfer: Kernel Library for LLM Serving

Examples demonstrating available options to program multiple GPUs in a single node or a cluster

Yinghan's Code Sample

Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity

[ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference

play gemm with tvm

[EuroSys'24] Minuet: Accelerating 3D Sparse Convolutions on GPUs

Gallatin is a general-purpose memory manager for CUDA that allows for threads to quickly malloc and free memory of arbitrary size inside of kernels.