Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate the full power of ZeRo into the code #19

Closed
StellaAthena opened this issue Jan 1, 2021 · 1 comment
Closed

Integrate the full power of ZeRo into the code #19

StellaAthena opened this issue Jan 1, 2021 · 1 comment
Labels
feature request New feature or request
Projects
1T or BUST

Comments

@StellaAthena
Copy link
Member

No description provided.

@StellaAthena StellaAthena added the feature request New feature or request label Jan 1, 2021
@StellaAthena StellaAthena added this to To do in 1T or BUST via automation Jan 1, 2021
@sdtblck
Copy link
Contributor

sdtblck commented Jan 5, 2021

ZeRo is handled from the deepspeed config and deepspeed initialize

@sdtblck sdtblck closed this as completed Jan 5, 2021
1T or BUST automation moved this from To do to Done Jan 5, 2021
leogao2 pushed a commit that referenced this issue Feb 13, 2021
@StellaAthena
Merge pull request #19 from EleutherAI/sparse_attn
c68f0cd 
add sparse attn
leogao2 pushed a commit that referenced this issue Feb 13, 2021
@StellaAthena
Merge pull request #19 from EleutherAI/sparse_attn
9532b8f 
add sparse attn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request New feature or request
Projects
1T or BUST
Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@StellaAthena @sdtblck