Stars
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
A high-throughput and memory-efficient inference and serving engine for LLMs
Fast and memory-efficient exact attention
AirLLM 70B inference with single 4GB GPU
TEN Agent is the world’s first real-time multimodal agent integrated with the OpenAI Realtime API and RTC. In addition, TEN Agent also has vision and RAG capabilities.
Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
21 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/
A resource for learning about Machine learning & Deep Learning
🦜🔗 Build context-aware reasoning applications
LlamaIndex is a data framework for your LLM applications
Open weights LLM from Google DeepMind.
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
A curated list of awesome End-to-End Autonomous Driving resources (continually updated)
[CVPR 2023] ReasonNet: End-to-End Driving with Temporal and Global Reasoning
Implementation of paper - YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
[IEEE T-PAMI 2024] All you need for End-to-end Autonomous Driving
A JAX-based simulator for autonomous driving research.