-
Notifications
You must be signed in to change notification settings - Fork 998
Issues: EleutherAI/gpt-neox
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Add support for pytorch 2.0 ?
deprioritized
Issues that are not closed, but are low priority and unlikely to be solved soon
feature request
New feature or request
#858
opened Mar 27, 2023 by
guozhiyao
DeepSpeed Sparse Attention is Broken
bug
Something isn't working
#863
opened Mar 29, 2023 by
dashstander
OOM error when training on a 220G Memory machine with 8 V100.
feature request
New feature or request
#867
opened Apr 2, 2023 by
SefaZeng
Fine-tuning gpt-neox on 8 A100s
feature request
New feature or request
#892
opened Apr 20, 2023 by
rajhans
Add StableLM as an example to the README
documentation
Improvements or additions to documentation
#896
opened Apr 22, 2023 by
StellaAthena
20B pretrained model inference OOM on 8xA100 40GB
bug
Something isn't working
good first issue
Good for newcomers
#901
opened Apr 23, 2023 by
Mutinifni
Can't finetune 20B model from slim weights with zero optimizer enabled
bug
Something isn't working
#926
opened May 5, 2023 by
coreystatendet
The plot got from muP coord_check seems not horizontal, which may indicates there exits a bug in the muP implementation?
bug
Something isn't working
#956
opened May 28, 2023 by
BaoYu0721
Bias weights are multi-added when using Something isn't working
good first issue
Good for newcomers
gpt_j_residual
in model-parallel execution
bug
#962
opened May 31, 2023 by
cbcase
Why we need to average LayerNorm values over mp ranks when converting to HFformat checkpoint?
#983
opened Jun 26, 2023 by
forceshorty
How to preserve Pythia's sampling order but for different batch size.
bug
Something isn't working
#984
opened Jul 3, 2023 by
lintangsutawika
AssertionError: zero stage 1 requires an optimizer
bug
Something isn't working
good first issue
Good for newcomers
help wanted
This issue needs assistance
#987
opened Jul 4, 2023 by
yonglianglan
Convert HF Llama Checkpoints to Neox Checkpoints
feature request
New feature or request
#994
opened Jul 10, 2023 by
sxthunder
[BUG] Inconsistent loss between Something isn't working
overlap_comm=true
and overlap_comm=false
bug
#1004
opened Jul 27, 2023 by
0x6b64
Support for Mosaic Models
feature request
New feature or request
#1057
opened Oct 6, 2023 by
rajveer43
Interoperability and GPT-NeoX
documentation
Improvements or additions to documentation
question
#1058
opened Oct 12, 2023 by
StellaAthena
Integrate TransformerEngine
feature request
New feature or request
#1098
opened Dec 21, 2023 by
Quentin-Anthony
Create Singularity Container
feature request
New feature or request
good first issue
Good for newcomers
help wanted
This issue needs assistance
#1119
opened Jan 11, 2024 by
Quentin-Anthony
[BUG?] Higher "gradient_accumulation_steps" still increases memory usage a lot
bug
Something isn't working
#1123
opened Jan 15, 2024 by
exnx
Tests fail when run with pytest --forked
bug
Something isn't working
#1132
opened Jan 25, 2024 by
segyges
PyTorch Lightning Fused optimizer step
feature request
New feature or request
#1160
opened Feb 29, 2024 by
jahatef
My servers used for multi-node training do not have ssh. How can I launch multi-node training using the torchrun command?
feature request
New feature or request
#1203
opened Apr 23, 2024 by
dingning97
LoRA Support
feature request
New feature or request
#1204
opened Apr 23, 2024 by
Quentin-Anthony
4 tasks
ProTip!
no:milestone will show everything without a milestone.