🦾💻🌐 distributed training & serverless inference at scale on RunPod
-
Updated
May 26, 2024 - Jupyter Notebook
🦾💻🌐 distributed training & serverless inference at scale on RunPod
META LLAMA3 GENAI Real World UseCases End To End Implementation Guide
A script for training the ConvNextV2 on CIFAR10 dataset using the FSDP technique for a distributed training scheme.
Fast and easy distributed model training examples.
Minimal yet high performant code for pretraining llms. Attempts to implement some SOTA features. Implements training through: Deepspeed, Megatron-LM, and FSDP. WIP
Framework, Model & Kernel Optimizations for Distributed Deep Learning - Data Hack Summit
Fully Sharded Data Parallel (FSDP) implementation of Transformer XL
Add a description, image, and links to the fsdp topic page so that developers can more easily learn about it.
To associate your repository with the fsdp topic, visit your repo's landing page and select "manage topics."