A Blazing Fast AI Gateway. Route to 200+ LLMs with 1 fast & friendly API.

AI chat for every model.

Enhanced ChatGPT Clone: Features OpenAI, Assistants API, Azure, Groq, GPT-4 Vision, Mistral, Bing, Anthropic, OpenRouter, Vertex AI, Gemini, AI model switching, message search, langchain, DALL-E-3,…

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with …

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Inference code for CodeLlama models

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

Official Implementation of EAGLE-1 and EAGLE-2

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

Robust Speech Recognition via Large-Scale Weak Supervision

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and…

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting yo…

Self-hosted AI coding assistant

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Stable Diffusion web UI

🦜🔗 Build context-aware reasoning applications

Examples and guides for using the OpenAI API

LLM inference in C/C++

Inference code for Llama models

A natural language interface for computers

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

LlamaIndex is a data framework for your LLM applications

Port of OpenAI's Whisper model in C/C++

Structured Text Generation