-
Notifications
You must be signed in to change notification settings - Fork 976
Issues: EleutherAI/gpt-neox
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
[BUG] Inconsistent loss between Something isn't working
overlap_comm=true
and overlap_comm=false
bug
#1004
opened Jul 27, 2023 by
0x6b64
Convert HF Llama Checkpoints to Neox Checkpoints
feature request
New feature or request
#994
opened Jul 10, 2023 by
sxthunder
AssertionError: zero stage 1 requires an optimizer
bug
Something isn't working
good first issue
Good for newcomers
help wanted
This issue needs assistance
#987
opened Jul 4, 2023 by
yonglianglan
How to preserve Pythia's sampling order but for different batch size.
bug
Something isn't working
#984
opened Jul 3, 2023 by
lintangsutawika
Why we need to average LayerNorm values over mp ranks when converting to HFformat checkpoint?
#983
opened Jun 26, 2023 by
forceshorty
Bias weights are multi-added when using Something isn't working
good first issue
Good for newcomers
gpt_j_residual
in model-parallel execution
bug
#962
opened May 31, 2023 by
cbcase
Can't finetune 20B model from slim weights with zero optimizer enabled
bug
Something isn't working
#926
opened May 5, 2023 by
coreystatendet
Fine-tuning gpt-neox on 8 A100s
feature request
New feature or request
#892
opened Apr 20, 2023 by
rajhans
OOM error when training on a 220G Memory machine with 8 V100.
feature request
New feature or request
#867
opened Apr 2, 2023 by
SefaZeng
Add support for pytorch 2.0 ?
deprioritized
Issues that are not closed, but are low priority and unlikely to be solved soon
feature request
New feature or request
#858
opened Mar 27, 2023 by
guozhiyao
Finetuning loss explode when not loading deepspeed zero optimal states
bug
Something isn't working
#843
opened Mar 19, 2023 by
sxthunder
Implement Prefix-LM attention masking
feature request
New feature or request
#805
opened Mar 1, 2023 by
TokyoExpress
Unable to load model checkpoint with model parallelism
feature request
New feature or request
#773
opened Jan 20, 2023 by
RaoNikitha
Multi-node training without shared memory
deprioritized
Issues that are not closed, but are low priority and unlikely to be solved soon
feature request
New feature or request
#765
opened Jan 6, 2023 by
VHellendoorn
In interactive mode prompt length more than one word causes to crash
bug
Something isn't working
deprioritized
Issues that are not closed, but are low priority and unlikely to be solved soon
#758
opened Dec 27, 2022 by
ahmedavid
Training speed in bf16 mode is slow.
bug
Something isn't working
#660
opened Aug 29, 2022 by
frankang
RuntimeError: Error(s) in loading state_dict for EmbeddingPipe: size mismatch for word_embeddings.weight
bug
Something isn't working
good first issue
Good for newcomers
help wanted
This issue needs assistance
#645
opened Jul 7, 2022 by
mcao516
Text generation yields different outputs despite temperature = 0.0
bug
Something isn't working
good first issue
Good for newcomers
#643
opened Jul 5, 2022 by
ScTof
getting error saying __init__ () got an unexpected keyword argument 'checkpointable_layers' when i started training
bug
Something isn't working
#632
opened Jun 13, 2022 by
whoislimshady
OOM issues running inference with large contexts on 2x3090 system
bug
Something isn't working
#631
opened Jun 9, 2022 by
fpgaminer
ProTip!
Type g i on any issue or pull request to go back to the issue listing page.