cgmemtime measures the high-water RSS+CACHE memory usage of a process and its descendant processes.

Ultra low overhead NVIDIA GPU telemetry plugin for telegraf with memory temperature readings.

Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

Linux​ based GDDR6/GDDR6X VRAM temperature reader for NVIDIA RTX 3000/4000 series GPUs.

NVIDIA GPU metrics exporter for Prometheus leveraging DCGM

Module, Model, and Tensor Serialization/Deserialization

All Algorithms implemented in Python

Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚

A python module to repair invalid JSON, commonly used to parse the output of LLMs

Efficient Triton Kernels for LLM Training

Gpu benchmark

A guidance language for controlling large language models.

Utils for streaming large files (S3, HDFS, gzip, bz2...)

A hack to make MPI over Infiniband to work on Docker and Singularity containers

Pragmatic approach to parsing import profiles for CI's

A sample pattern for running CI tests on Modal

An extremely fast Python package and project manager, written in Rust.

A sample pattern for running CI tests on Modal

Making Reddit data accessible to researchers, moderators and everyone else. Interact with the data through large dumps, an API or web interface.

Applied AI experiments and examples for PyTorch

The official Python client for the Huggingface Hub.

CUDA checkpoint and restore utility

Checkpoint/Restore tool

Hardware locality (hwloc)

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

A PyTorch Native LLM Training Framework

🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.