Easy usage of Rockchip's NPUs found in RK3588 and similar chips

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Ubuntu 22.04 and 24.04 for Rockchip RK35XX Devices

A beautiful, simple, clean, and responsive Jekyll theme for academics

A reference application for a local AI assistant with LLM and RAG

FlashInfer: Kernel Library for LLM Serving

A Stream-based LLM Agent Framework for Continuous Context Sensing and Sharing

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting yo…

Pure C++ implementation of several models for real-time chatting on your computer (CPU)

Run generative AI models in sophgo BM1684X

Run Large Language Models on RK3588 with GPU-acceleration

Improve your Bilibili homepage by redesigning it, adding more features, and personalizing it to match your preferences. (English | 简体中文 | 正體中文 | 廣東話)

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

Sparsity-aware deep learning inference runtime for CPUs

Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

Reverse engineering the rk3588 npu

Run Neural networks on NationalChip NPU processor.

RK3399 Pro NPU support for Caffe SSD detector

DDK for Rockchip NPU

📋 A list of open LLMs available for commercial use.

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

Provides a set of benchmarks that can be used to measure the memory bandwidth performance of CPU's

Free monospaced font with programming ligatures

Stable Diffusion in NCNN with c++, supported txt2img and img2img

使用Android手机的CPU推理stable diffusion

Grok open release

Stable Diffusion AI client app for Android