Skip to content

Pull requests: EleutherAI/gpt-neox

New pull request New
Clear current search query, filters, and sorts
1 Open 20 Closed
1 Open 20 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Adds the proper DeepSpeed bf16 config
#786 by dashstander was closed Feb 14, 2023 Loading…
ALibi & Flash Attention
#864 by dashstander was merged Apr 11, 2023 Loading…
1
Return to broadcasting megatron and deepspeed configs as training.py arguments
#948 by dashstander was merged May 23, 2023 Loading…
1
Bump transformers version and update enwik8 link
#1024 by dashstander was merged Sep 13, 2023 Loading…
1
Fix bf16 for zero > 0 and pipeline parallelism > 0
#1032 by dashstander was merged Sep 18, 2023 Loading…
1
Remove support for lazy dataset implementation
#1033 by dashstander was merged Sep 18, 2023 Loading…
1
Remove the NeoX implementation of GPT2Tokenizer
#1042 by dashstander was merged Sep 25, 2023 Loading…
1
Improve FLOPS Calculation
#1044 by dashstander was merged Sep 27, 2023 Loading…
1
Patch LR Annealing Bug
#1046 by dashstander was merged Sep 27, 2023 Loading…
1
Add section to the README detailing how to start distributed jobs
#1048 by dashstander was merged Sep 29, 2023 Loading…
1
Organize the tools directory
#1055 by dashstander was merged Oct 2, 2023 Loading…
1
Add documentation about using labelled datasets
#1056 by dashstander was merged Oct 4, 2023 Loading…
1
LR scheduler fix no longer breaks inference
#1060 by dashstander was merged Oct 17, 2023 Loading…
1
Save checkpoint metadata when using SlurmRunner
#733 by dashstander was merged Dec 8, 2022 Loading…
1
3
Updates bf16 demo config and mixed precision docutmentation.
#941 by dashstander was merged May 18, 2023 Loading…
1
1
Pre-compute RoPE embeddings in fp32
#1041 by dashstander was merged Sep 25, 2023 Loading…
1
1
SLURM Multi-Node Support
#680 by dashstander was merged Sep 22, 2022 Loading…
6
Slurm Fix and formatting
#729 by dashstander was merged Dec 6, 2022 Loading…
4
Implement DeepSpeed Main autotuning for NeoX
#739 by dashstander was merged Mar 9, 2023 Loading… Release V2
@nsarka @dashstander
2
Add DeepSpeed bf16 configuration
#787 by dashstander was merged May 16, 2023 Loading…
8
ProTip! Add no:assignee to see everything that’s not assigned.