Commits

Commits on Apr 24, 2024

Fix flash config for llama2/70B.yml config (#1206 )
Quentin-Anthony
committedApr 24, 2024

Commits on Apr 1, 2024

[ZeRO-3] Ensured passing neox deepspeed_config when using partitioned init (#1191 )

R0n12
and
Quentin-Anthony
committedApr 1, 2024
[AMD] Supporting fused kernels build using JIT (#1188 )
R0n12
committedApr 1, 2024
making PR triggered CPU test for changes to megatron (#1195 )

committedApr 1, 2024

Commits on Mar 26, 2024

Update README.md
StellaAthena
committedMar 26, 2024
Added more papers
StellaAthena
committedMar 26, 2024
Merge pull request #1196 from edouardoyallon/typo_readme
StellaAthena
committedMar 26, 2024
Small typo in the README
edouardoyallon
committedMar 26, 2024

Commits on Mar 19, 2024

[ZeRO-3] Partitioned init with deepspeed.zero.Init() (#1190 )

R0n12
and
Quentin-Anthony
committedMar 19, 2024

Commits on Mar 15, 2024

Mamba + Tensor Parallel Support (#1184 )

committedMar 15, 2024

Commits on Mar 13, 2024

Switch to using Cuda Flash Attn for Alibi (#1183 )

haileyschoelkopf
and
github-actions
committedMar 13, 2024

Commits on Mar 10, 2024

Add Mamba Architecture (#1157 )

committedMar 10, 2024

Commits on Mar 8, 2024

Commits on Mar 7, 2024

Add MoE (#1129 )

committedMar 7, 2024

Commits on Mar 4, 2024

Support Lion with Zero Optimizer (#1166 )

DayOfThePenguin
and
Quentin-Anthony
committedMar 4, 2024
Make rotary freqs buffer non-persistent (#1168 )

committedMar 4, 2024
Ignore markdown for pre-commit (#1171 )

Quentin-Anthony
and
github-actions
committedMar 4, 2024
Remove unnecessary fp32/bf16 conversion (#1169 )

DayOfThePenguin
and
Quentin-Anthony
committedMar 4, 2024
Single node Pythia 14M training on ngc pytorch 24.02 container (#1170 )

tf-nv
and
Quentin-Anthony
committedMar 4, 2024

Commits on Mar 2, 2024

Improve argument validation for Flash-attn + SWA (#1162 )

haileyschoelkopf
and
github-actions
committedMar 2, 2024

Commits on Feb 29, 2024

Update cpu_ci.yml (#1159 )

jaimemcc-intel
and
github-actions
committedFeb 29, 2024

Commits on Feb 26, 2024

[Bug?] Fix profiling argument names (#1155 )

committedFeb 26, 2024

Commits on Feb 23, 2024

Draft PR Adding mistral 0.1 (#1131 )

committedFeb 23, 2024
Minor changes (#1125 )
segyges
committedFeb 23, 2024
Python version update (#1122 )
segyges
committedFeb 23, 2024
add profiling to readme (#1154 )

jahatef
and
github-actions
committedFeb 23, 2024

Commits on Feb 21, 2024

Memory profiling (#1153 )

committedFeb 21, 2024
Fixes distributed tests, and skips tests that are broken. (#1149 )

committedFeb 21, 2024

Commits on Feb 8, 2024

Improve Conversion Utilities (#1124 )

committedFeb 8, 2024

Commits on Feb 5, 2024

Remove gas (beano) (#1144 )
segyges
committedFeb 5, 2024