MSCCL++: A GPU-driven communication stack for scalable AI applications

FlashInfer: Kernel Library for LLM Serving

PyTorch for building ML systems. Iterable, debuggable, multi-cloud, 100% reproducible across research and production.

Introduction to WASM assembly

🚀 The leading Wasm Runtime supporting WASIX, WASI and Emscripten

Incremental bundler and build system optimized for JavaScript and TypeScript, written in Rust – including Turbopack and Turborepo.

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

ModelScope: bring the notion of Model-as-a-Service to life.

An ecosystem of Rust libraries for working with large language models

Standalone pre-training recipe with JAX+Flax

Accelerate your training with this open-source library. Optimize performance with streamlined training and serving options with JAX. 🚀

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelization, and has demonstrated industry lead…

🐉 Making Rust a first-class language and ecosystem for GPU shaders 🚧

Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.

A C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.

Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/

🔍 Tiny, full-text search engine for static websites built with Rust and Wasm

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting yo…

This is the Rust course used by the Android team at Google. It provides you the material to quickly teach Rust.

Elegant easy-to-use neural networks + scientific computing in JAX. https://docs.kidger.site/equinox/

🔍 LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your d…

The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

LLM inference in C/C++

🦜🔗 Build context-aware reasoning applications

Get up and running with Llama 3, Mistral, Gemma, and other large language models.

🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23