The most intelligent Siri powered by LLMs

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

SGLang is yet another fast serving framework for large language models and vision language models.

A guidance language for controlling large language models.

Accessible large language models via k-bit quantization for PyTorch.

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

LLM inference in C/C++

Inference code for Llama models

row-major matmul optimization

Universal LLM Deployment Engine with ML Compilation

Awesome AI Coding

a lightweight LLM model inference framework

整理开源的中文大语言模型，以规模较小、可私有化部署、训练成本较低的模型为主，包括底座模型，垂直领域微调及应用，数据集与教程等。

Python wrapper for isl, an integer set library

Decompiler of LLVM bitcode to C

pocl - Portable Computing Language

SAPFOR (System FOR Automated Parallelization)

Unicorn CPU emulator framework (ARM, AArch64, M68K, Mips, Sparc, PowerPC, RiscV, S390x, TriCore, X86)

An OpenCL device simulator and debugger

Everything we learnt from hacking Arm Mali GPUs.

Simple OpenCL Samples that Build with Khronos Headers and Libs

LLaMa/RWKV onnx models, quantization and testcase

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

计图大模型推理库，具有高性能、配置要求低、中文支持好、可移植等特点

open source driver project for adreno GPUs

一些阅读源码和Fuzzing 的经验,涵盖黑盒与白盒测试..

Extracts static code features from opencl kernels to be used for machine learning.

compiler learning resources collect.

基于 ChatGPT API 的划词翻译浏览器插件和跨平台桌面端应用 - Browser extension and cross-platform desktop application for translation based on ChatGPT API.