A high-throughput and memory-efficient inference and serving engine for LLMs

EAGLE: Lossless Acceleration of LLM Decoding by Feature Extrapolation

Python actor framework for heterogeneous computing.

Fast inference from large lauguage models via speculative decoding

Scalable Python data science, in an API compatible & lightning fast way.

OpenAI's Code Interpreter in your terminal, running locally

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

One API for plugins and datasets, one interface for prompt engineering and visual operation, all for creating powerful AI applications.

An editor made for programming with AI 🤖

Examples and recipes for Llama 2 model

⚡ Building applications with LLMs through composability ⚡

Port of Facebook's LLaMA model in C/C++

Training and serving large-scale neural networks

⚡ Dynamically generated stats for your github readmes

A C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.

Xorbits Example Notebooks

Start a chat with any document with Ada Embedding and Davinci Completion

Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.

CnOCR: Awesome Chinese/English OCR toolkits based on PyTorch/MXNet, It comes with 20+ well-trained models for different application scenarios and can be used directly after installation. 【基于 PyTorc…

Apache Arrow DataFusion Python Bindings

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more