Commits

History for

Commits on Jun 26, 2024

Disable nvtx decorator to avoid graph break (#5697 )

tohtana
and
loadams
committedJun 26, 2024
Universal checkpoint for zero stage 3 (#5475 )

committedJun 26, 2024

Commits on Jun 17, 2024

Remove compile wrapper to simplify access to model attributes (#5581 )

tohtana
and
tjruwase
committedJun 17, 2024

Commits on Jun 5, 2024

fix sequence parallel(Ulysses) grad scale for zero0 (#5555 )
inkcherry
committedJun 5, 2024

Commits on May 22, 2024

Add throughput timer configuration (#5363 )

committedMay 22, 2024

Commits on May 15, 2024

Improve _configure_optimizer() final optimizer log (#5528 )

nelyahu
and
loadams
committedMay 15, 2024

Commits on May 8, 2024

Fix compile wrapper (#5455 )

committedMay 8, 2024

Commits on Apr 16, 2024

Update engine.py to avoid torch warning (#5408 )

committedApr 16, 2024

Commits on Apr 8, 2024

Support MoE for pipeline models (#5338 )

mosheisland
and
misland-habana
committedApr 8, 2024

Commits on Apr 5, 2024

Auto convert moe param groups (#5354 )

jeffra
and
sfc-gh-jrasley
committedApr 5, 2024

Commits on Mar 28, 2024

Improve universal checkpoint (#5289 )
tohtana
committedMar 28, 2024

Commits on Mar 27, 2024

support bf16_optimizer moe expert parallel training and moe EP grad_scale/grad_norm fix (#5259 )

inkcherry
and
tjruwase
committedMar 27, 2024

Commits on Mar 13, 2024

Fix loading a universal checkpoint (#5263 )
tohtana
committedMar 13, 2024

Commits on Feb 21, 2024

Fix allreduce for BF16 and ZeRO0 (#5170 )

tohtana
and
loadams
committedFeb 21, 2024
MOE: Fix save checkpoint when TP > 1 (#5157 )

mosheisland
and
misland-habana
committedFeb 21, 2024

Commits on Feb 15, 2024

tensorboard logging: avoid item() outside gas to improve performance (#5135 )

nelyahu
and
tjruwase
committedFeb 15, 2024

Commits on Feb 12, 2024

BF16 optimizer: Improve device utilization by immediate grad update (#4975 )

deepcharm
and
tjruwase
committedFeb 12, 2024
Stop tracking backward chain of broadcast (ZeRO3) (#5113 )
tohtana
committedFeb 12, 2024

Commits on Feb 6, 2024

Enable torch.compile with ZeRO (Experimental) (#4878 )

committedFeb 6, 2024

Commits on Feb 5, 2024

Stop tracking backward chain of broadcast in initialization (#5075 )

tohtana
and
mrwyattii
committedFeb 5, 2024

Commits on Jan 25, 2024

Support exclude_frozen_parameters for save_16bit_model (#4999 )

LZHgrla
and
tjruwase
committedJan 25, 2024

Commits on Jan 18, 2024

Fixes for training models with bf16 + freshly initialized optimizer via load_module_only (#4141 )

committedJan 18, 2024

Commits on Jan 6, 2024

Nvme offload checkpoint (#4707 )

committedJan 6, 2024

Commits on Jan 5, 2024

Release overlap_comm & contiguous_gradients restrictions for ZeRO 1 (#4887 )

li-plus
and
tjruwase
committedJan 5, 2024
Support cpu tensors without direct device invocation (#3842 )

committedJan 5, 2024

Commits on Jan 3, 2024

Pipeline: Add support to eval micro bs configuration (#4859 )

nelyahu
and
loadams
committedJan 3, 2024

Commits on Dec 30, 2023

Remove hooks on gradient accumulation on engine/optimizer destroy (#4858 )
chiragjn
committedDec 30, 2023

Commits on Dec 26, 2023

deepspeed engine: fp16 support validation on init (#4843 )
nelyahu
committedDec 26, 2023

Commits on Dec 20, 2023

engine.py: remove unused _curr_save_path (#4844 )

nelyahu
and
mrwyattii
committedDec 20, 2023
Capture short kernel sequences to graph (#4318 )

inkcherry
and
tjruwase
committedDec 20, 2023

Commits on Dec 15, 2023

mv DeepSpeedEngine param_names dict init post _configure_distributed_model (#4803 )

nelyahu
and
mrwyattii
committedDec 15, 2023

Commits on Dec 8, 2023

BF16_Optimizer: add support for bf16 grad acc (#4713 )

nelyahu
and
tjruwase
committedDec 8, 2023

Commits on Nov 21, 2023

Communication Optimization for Large-Scale Training (#4695 )

RezaYazdaniAminabadi
and
tjruwase
committedNov 21, 2023

Commits on Nov 9, 2023

Universal ckp fixes (#4588 )

committedNov 9, 2023

Commits on Nov 8, 2023

Enhance the robustness of module_state_dict (#4587 )
LZHgrla
committedNov 8, 2023