Implement some method of LLM KV Cache Sparsity

Android GPU Inspector

MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, m…

Distribute and run LLMs with a single file.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

Multi-Candidate Speculative Decoding

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, le…

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)

QLoRA: Efficient Finetuning of Quantized LLMs

Inference Llama 2 in one file of pure C

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

compiler learning resources collect.

TFLite Support is a toolkit that helps users to develop ML and deploy TFLite models onto mobile / ioT devices.

MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深度学习框架

ncnn is a high-performance neural network inference framework optimized for the mobile platform

Arm neon optimization practice

Apache NuttX is a mature, real-time embedded operating system (RTOS)

开发内功修炼

On-Device Training Under 256KB Memory [NeurIPS'22]

Models and examples built with TensorFlow

IEEE 802.11 a/g/p Transceiver

software center for hnd/axhnd/axhnd.675x routers

GNU Radio – the Free and Open Software Radio Ecosystem