Commits

History for

Commits on Dec 22, 2023

Add QK Normalization (#1100 )

committedDec 22, 2023

Commits on Dec 6, 2023

Corrects FLOPs formula as per 1093 (#1094 )

StellaAthena
and
github-actions
committedDec 6, 2023

Commits on Dec 4, 2023

Extend ci suite (#1080 )
mkerin
committedDec 4, 2023

Commits on Nov 16, 2023

Update neox_args.py (#1081 )

jahatef
and
github-actions
committedNov 16, 2023

Commits on Nov 1, 2023

fixed case when ntasks_per_node is used instead (#1069 )
AIproj
committedNov 1, 2023

Commits on Oct 31, 2023

fix lion optimizer documentation (#1067 )

jahatef
and
github-actions
committedOct 31, 2023

Commits on Oct 20, 2023

Lion Optimizer (#1062 )

andylolu2
and
adi-kmt
committedOct 20, 2023

Commits on Oct 17, 2023

LR scheduler fix no longer breaks inference (#1060 )

dashstander
and
github-actions
committedOct 17, 2023

Commits on Oct 4, 2023

Add documentation about using labelled datasets (#1056 )

dashstander
and
github-actions
committedOct 4, 2023

Commits on Sep 27, 2023

Improve FLOPS Calculation (#1044 )

committedSep 27, 2023
Patch LR Annealing Bug (#1046 )

committedSep 27, 2023

Commits on Sep 25, 2023

Pre-compute RoPE embeddings in fp32 (#1041 )

committedSep 25, 2023
Remove the NeoX implementation of GPT2Tokenizer (#1042 )

dashstander
and
github-actions
committedSep 25, 2023

Commits on Sep 23, 2023

Fix final value of exponential decay lr (#1040 )
Quentin-Anthony
committedSep 23, 2023
Fixed final value of linear decay lr (#1039 )
foggy-frost-forest
committedSep 23, 2023
Add s3 checkpoint syncing (#1010 )

committedSep 23, 2023

Commits on Sep 20, 2023

integrated flash attention 2 (#1035 )
a663E-36z1120
committedSep 20, 2023

Commits on Sep 18, 2023

Fix SequentialWrapper Generation (pipe_parallel_size = 0) (#1031 )
xu-song
committedSep 18, 2023
Remove support for lazy dataset implementation (#1033 )

committedSep 18, 2023
Fix bf16 for zero > 0 and pipeline parallelism > 0 (#1032 )

dashstander
and
github-actions
committedSep 18, 2023

Commits on Sep 15, 2023

Fix Generation with Sequential Model (#1026 )
xu-song
committedSep 15, 2023

Commits on Aug 12, 2023

Fixed final value of cosine decay lr (#1011 )

kshitijkg
and
github-actions
committedAug 12, 2023

Commits on Aug 7, 2023

Fixed AnnealingLR Class and Cosine Decay Schedule (#1008 )

kshitijkg
and
github-actions
committedAug 7, 2023

Commits on Jul 17, 2023

changed all instances of torch.concat to torch.cat
mariebiscuit
committedJul 17, 2023

Commits on Jul 8, 2023

default tokenizer_type should be string
mayankjobanputra
committedJul 8, 2023

Commits on Jul 5, 2023

Update training.py
StellaAthena
committedJul 5, 2023
Update data_utils.py
StellaAthena
committedJul 5, 2023

Commits on Jun 23, 2023

Merge branch 'main' into dataloadFix
Quentin-Anthony
committedJun 23, 2023

Commits on Jun 22, 2023

Update gpt2_dataset.py (#974 )

StellaAthena
and
github-actions
committedJun 22, 2023

Commits on Jun 21, 2023

fixes
Quentin TastyRice
committedJun 21, 2023
fix issues with dataloading
Quentin TastyRice
committedJun 21, 2023
fix issue with data
Quentin TastyRice
committedJun 21, 2023

Commits on Jun 7, 2023

Adding the possibility of passing a label dataset (#958 )

honglu2875
and
haileyschoelkopf
committedJun 7, 2023

Commits on Jun 5, 2023

Use minimal softmax mask values based on the datatype (#966 )

bentherien
and
Quentin-Anthony
committedJun 5, 2023

Commits on May 23, 2023

Return to broadcasting megatron and deepspeed configs as training.py arguments (#948 )

committedMay 23, 2023