LLM inference in C/C++

📚 C/C++ 技术面试基础知识总结，包括语言、程序库、数据结构、算法、系统、网络、链接装载库等知识及面试经验、招聘、内推等信息。This repository is a summary of the basic knowledge of recruiting job seekers and beginners in the direction of C/C++ technology, in…

Port of OpenAI's Whisper model in C/C++

Fast C++ logging library.

A modern, C++-native, test framework for unit-tests, TDD and BDD - using C++14, C++17 and later (C++11 support is in v2.x branch, and C++03 on the Catch1.x branch)

Karabiner-Elements is a powerful utility for keyboard customization on macOS Sierra (10.12) or later.

Background Music, a macOS audio utility: automatically pause your music, set individual apps' volumes and record system audio.

Abseil Common Libraries (C++)

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing

Development repository for the Triton language and compiler

Tensor library for machine learning

A fast multi-producer, multi-consumer lock-free concurrent queue for C++11

cuDF - GPU DataFrame Library

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Transformer related optimization, including BERT, GPT

A Detailed Cplusplus Concurrency Tutorial 《C++ 并发编程指南》

Bear is a tool that generates a compilation database for clang tooling.

Static reflection for enums (to string, from string, iteration) for modern C++, work with any enum type without any macro or boilerplate code

cuML - RAPIDS Machine Learning Library

Simplify your onnx model

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

A dbg(…) macro for C++

KLEE Symbolic Execution Engine

Argument Parser for Modern C++

[ARCHIVED] The C++ Standard Library for your entire system. See https://github.com/NVIDIA/cccl

Submanifold sparse convolutional networks

Kokkos C++ Performance Portability Programming Ecosystem: The Programming Model - Parallel Execution and Memory Abstraction

Cista is a simple, high-performance, zero-copy C++ serialization & reflection library.

C++ compile-time enum to string, iteration, in a single header file