Pinned

1. NVIDIA/Megatron-LM NVIDIA/Megatron-LM Public

   Ongoing research training transformer models at scale

   Python 8.9k 2k

2. NVIDIA/TransformerEngine NVIDIA/TransformerEngine Public

   A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…

   Python 1.5k 231

3. lit-llama lit-llama Public

   Forked from Lightning-AI/lit-llama

   Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

   Python

4. huggingface/accelerate huggingface/accelerate Public

   🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

   Python 7.2k 838

5. state-spaces/mamba state-spaces/mamba Public

   Python 10.2k 798

6. NVIDIA/NeMo NVIDIA/NeMo Public

   A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

   Python 10.3k 2.2k