Commits

History for

Commits on Jun 7, 2024

Fix changed behavior of pipe_parallel (#1219 )

committedJun 7, 2024

Commits on May 21, 2024

Small tidying (#1222 )

committedMay 21, 2024
fixed fused_rope naming in JIT + added readme for amd support (#1224 )
R0n12
committedMay 21, 2024
Add Torch Profiler Support (#1226 )

committedMay 21, 2024
Rwkv pipeline parallelism (#1221 )

committedMay 21, 2024

Commits on May 6, 2024

add rwkv support (#1198 )

committedMay 6, 2024

Commits on May 4, 2024

Add megablocks dropless MoE (#1192 )

yang
and
Quentin-Anthony
committedMay 4, 2024

Commits on Apr 25, 2024

Fixes a weird typo (#1207 )

StellaAthena
and
github-actions
committedApr 25, 2024

Commits on Apr 1, 2024

[ZeRO-3] Ensured passing neox deepspeed_config when using partitioned init (#1191 )

R0n12
and
Quentin-Anthony
committedApr 1, 2024
[AMD] Supporting fused kernels build using JIT (#1188 )
R0n12
committedApr 1, 2024

Commits on Mar 19, 2024

[ZeRO-3] Partitioned init with deepspeed.zero.Init() (#1190 )

R0n12
and
Quentin-Anthony
committedMar 19, 2024

Commits on Mar 15, 2024

Mamba + Tensor Parallel Support (#1184 )

committedMar 15, 2024

Commits on Mar 13, 2024

Switch to using Cuda Flash Attn for Alibi (#1183 )

haileyschoelkopf
and
github-actions
committedMar 13, 2024

Commits on Mar 10, 2024

Add Mamba Architecture (#1157 )

committedMar 10, 2024

Commits on Mar 8, 2024

Commits on Mar 7, 2024

Add MoE (#1129 )

committedMar 7, 2024

Commits on Mar 4, 2024

Support Lion with Zero Optimizer (#1166 )

DayOfThePenguin
and
Quentin-Anthony
committedMar 4, 2024
Make rotary freqs buffer non-persistent (#1168 )

committedMar 4, 2024
Remove unnecessary fp32/bf16 conversion (#1169 )

DayOfThePenguin
and
Quentin-Anthony
committedMar 4, 2024

Commits on Mar 2, 2024

Improve argument validation for Flash-attn + SWA (#1162 )

haileyschoelkopf
and
github-actions
committedMar 2, 2024

Commits on Feb 26, 2024

[Bug?] Fix profiling argument names (#1155 )

committedFeb 26, 2024

Commits on Feb 23, 2024

Draft PR Adding mistral 0.1 (#1131 )

committedFeb 23, 2024

Commits on Feb 21, 2024

Memory profiling (#1153 )

committedFeb 21, 2024

Commits on Feb 5, 2024

Remove gas (beano) (#1144 )
segyges
committedFeb 5, 2024

Commits on Feb 1, 2024

Update lm_eval v0.4 to PyPI dependencies (#1141 )

committedFeb 1, 2024

Commits on Jan 30, 2024

moved eval import and added to docs (#1139 )
R0n12
committedJan 30, 2024

Commits on Jan 29, 2024

Contributing Guide (#1138 )

committedJan 29, 2024

Commits on Jan 26, 2024

fused layernorm (#1105 )

yang
and
jahatef
committedJan 26, 2024

Commits on Jan 24, 2024

update copyrights (#1128 )

jahatef
and
github-actions
committedJan 24, 2024

Commits on Jan 19, 2024

Enable passing of --account to srun / SlurmLauncher (#1126 )

haileyschoelkopf
and
github-actions
committedJan 19, 2024

Commits on Jan 5, 2024

Fused Rotary Embeddings (fixed) (#1108 )

committedJan 5, 2024

Commits on Jan 4, 2024

Fix repo for CI (#1106 )
yang
committedJan 4, 2024

Commits on Dec 26, 2023

Update neox_args.py (#1107 )

StellaAthena
and
github-actions
committedDec 26, 2023

Commits on Dec 23, 2023

Lm eval 0.4.0 support (#1101 )

committedDec 23, 2023