RDMA core userspace libraries and daemons

CUDA Library Samples

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Development repository for the Triton language and compiler

A high-performance, concurrent hash table

Brings SQL and AI together.

Inference Llama 2 in one file of pure C

A modern formatting library

An MLIR-based toy DL compiler for TVM Relay.

Hands-On Practical MLIR Tutorial

A C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.

Transformer related optimization, including BERT, GPT

LightSeq: A High Performance Library for Sequence Processing and Generation

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型

GLM (General Language Model)

C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & more LLMs

Fast and memory-efficient exact attention

Lightning fast C++/CUDA neural network framework

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

Portable C and C++ Development Kit for x64 (and x86) Windows

LLM inference in C/C++

Aligning pretrained language models with instruction data generated by themselves.

Inference code for Llama models

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

Tensors and Dynamic neural networks in Python with strong GPU acceleration