GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

A PyTorch Native LLM Training Framework

The Modern C++ Challenge, published by Packt

a lightweight LLM model inference framework

MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深度学习框架

A communication library for deep learning

What would you do with 1000 H100s...

The full minitorch student suite.

LeetCode题解，151道题完整版。

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Provides symbolic API for model creation in PyTorch.

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

This repository houses VERY simple example code for each STL algorithm and explains what each does.

📚 Modern C++ Tutorial: C++11/14/17/20 On the Fly | https://changkun.de/modern-cpp/

Open source FPGA-based NIC and platform for in-network compute

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…

Prints values and types during compilation!

Fast C++ logging library.

Lightning fast C++/CUDA neural network framework

MLX: An array framework for Apple silicon

A toolkit for 3D computer vision tasks.

ring-attention experiments

Transformers with Arbitrarily Large Context

📚 技术面试必备基础知识、Leetcode、计算机操作系统、计算机网络、系统设计

The road to hack SysML and become an system expert

tiny ring attention implement for learning purpose

flash attention tutorial written in python, triton, cuda, cutlass