A high-throughput and memory-efficient inference and serving engine for LLMs

A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)

Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

Get up and running with Llama 3, Mistral, Gemma 2, and other large language models.

Sampling profiler for Python programs

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Cost monitoring for Kubernetes workloads and cloud costs

💫 Industrial-strength Natural Language Processing (NLP) in Python

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Podman: A tool for managing OCI containers and pods.

A command-line benchmarking tool

Bandit is a tool designed to find common security issues in Python code.

⚡ A Fast, Extensible Progress Bar for Python and CLI

LlamaIndex is a data framework for your LLM applications

18 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines

the AI-native open-source embedding database

OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (v2, v3)

Multilingual Sentence & Image Embeddings with BERT

Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/

Examples and guides for using the OpenAI API

Kubernetes IN Docker - local clusters for testing Kubernetes

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Declarative Continuous Deployment for Kubernetes

Numbers every LLM developer should know

TensorFlow code and pre-trained models for BERT

An extremely fast Python linter and code formatter, written in Rust.

Code for the paper "Language Models are Unsupervised Multitask Learners"

Python packaging and dependency management made easy