Commits

History for

Commits on Aug 31, 2021

Merge branch 'main' into soft_prompt_2
StellaAthena
authoredAug 31, 2021
Char level ppl (#400 )

sdtblck
and
StellaAthena
authoredAug 31, 2021
add char level perplexity
sdtblck
committedAug 31, 2021

Commits on Aug 30, 2021

replace init_from_vocab option with init_string
sdtblck
committedAug 30, 2021
add soft prompt tuning
sdtblck
committedAug 30, 2021

Commits on Aug 26, 2021

add alibi pos embed
sdtblck
committedAug 26, 2021

Commits on Aug 25, 2021

get rid of reset_position_ids / reset_attention_mask (never used...)
sdtblck
committedAug 25, 2021
remove apply_residual_connection_post_ln arg
sdtblck
committedAug 25, 2021

Commits on Jul 7, 2021

Merge remote-tracking branch 'origin/gptj_residual' into main
sdtblck
committedJul 7, 2021

Commits on Jun 22, 2021

spelling error
sdtblck
committedJun 22, 2021
add gptj_residual arg
sdtblck
committedJun 22, 2021

Commits on Jun 20, 2021

Merge branch 'main' into _bf16
StellaAthena
authoredJun 20, 2021

Commits on Jun 15, 2021

fix microbatching + add parameters for evaluation
Samuel Weinbach
committedJun 15, 2021

Commits on May 19, 2021

Add GMLP (#339 )
sdtblck
authoredMay 19, 2021

Commits on May 17, 2021

Add new sparse attention options (#332 )
sdtblck
authoredMay 17, 2021
python 3.7 compatibility (#333 )
sdtblck
authoredMay 17, 2021

Commits on May 14, 2021

Merge branch 'main' into _bf16
sdtblck
authoredMay 14, 2021
add apexlayernorm back in for backward compatibility
sdtblck
committedMay 14, 2021

Commits on May 13, 2021

Checkpoint validation (#324 )

sweinbach
and
Samuel Weinbach
authoredMay 13, 2021

Commits on May 12, 2021

update name to scaled_orthogonal
sdtblck
authoredMay 12, 2021
fix(args): add nestler_orthogonal and small_init
ClashLuke
authoredMay 12, 2021

Commits on May 11, 2021

update docstring
sdtblck
committedMay 11, 2021
Add ability to weight datasets by number of documents
sdtblck
committedMay 11, 2021
Add blended datasets
sdtblck
committedMay 11, 2021
Added wang_init and small_init.
connor
committedMay 11, 2021

Commits on May 10, 2021

Merge remote-tracking branch 'origin/bf16' into zero-3
StellaAthena
committedMay 10, 2021
Add option to change init methods. Also add orthogonal and xavier init methods.
connor
committedMay 10, 2021

Commits on May 8, 2021

Merge branch 'gptneox-custom-train-test-val-datapath' of https://github.com/EleutherAI/gpt-neox into gptneox-custom-train-test-val-datapath
sdtblck
committedMay 8, 2021

Commits on May 7, 2021

scale d_ff for equal parameter count with geglu + fix geglu bias
sdtblck
committedMay 7, 2021
Update neox_args.py
sdtblck
authoredMay 7, 2021
add / change comments for activation functions
Samuel Weinbach
committedMay 7, 2021
Merge branch 'main' into relu
sdtblck
authoredMay 7, 2021
Added custom train, test and valid data-path functionality
gali
committedMay 7, 2021
activation functions initial commit
Samuel Weinbach
committedMay 7, 2021

Commits on May 6, 2021

include wand params for host and project
Samuel Weinbach
committedMay 6, 2021