Microsoft Collective Communication Library

Ongoing research training transformer models at scale

华中科技大学 Beamer 模板

System for AI Education Resource.

Here are my personal paper reading notes (including cloud computing, resource management, systems, machine learning, deep learning, and other interesting stuffs).

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Materials for the Learn PyTorch for Deep Learning: Zero to Mastery course.

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

A PyTorch Native LLM Training Framework

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Awesome-LLM: a curated list of Large Language Model

Simple examples to introduce PyTorch

PyTorch Tutorial for Deep Learning Researchers

cluster data collected from production clusters in Alibaba for cluster management research

Lucid: A Non-Intrusive, Scalable and Interpretable Scheduler for Deep Learning Training Jobs

A MoE impl for PyTorch, [ATC'23] SmartMoE

PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021

Tutel MoE: An Optimized Mixture-of-Experts Implementation

PyTorch extensions for high performance and large scale training.

🧠💬 Articles I wrote about machine learning, archived from MachineCurve.com.

Seamless operability between C++11 and Python

🧑‍🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gan…