LLM inference in C/C++

🤖 The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformer…

Distribute and run LLMs with a single file.

Development repository for the Triton language and compiler

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

Unlocker for VMWare macOS

Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM). Powers 👋 Jan

An open-source, high-performance SQL vector database built on ClickHouse.

ChatGPT Bot in CLI with long term memory support using Embeddings.

Knowhere is an open-source vector search engine, integrating FAISS, HNSW, etc.

Your customized AI assistant - Personal assistants on any hardware! With llama.cpp, whisper.cpp, ggml, LLaMA-v2.