Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Apply new fused rotary embedding #1077

Closed
Quentin-Anthony opened this issue Nov 11, 2023 · 10 comments · Fixed by #1108
Closed

Apply new fused rotary embedding #1077

Quentin-Anthony opened this issue Nov 11, 2023 · 10 comments · Fixed by #1108
Assignees
@yang
Labels
feature request New feature or request

Comments

@Quentin-Anthony
Copy link
Member

This could really help us: NVIDIA/apex#1746
@Quentin-Anthony Quentin-Anthony added the feature request New feature or request label Nov 11, 2023
@StellaAthena StellaAthena self-assigned this Nov 15, 2023
@Quentin-Anthony
Copy link
Member Author

May also need NVIDIA/apex#1750 applied in order to test FYI

@StellaAthena
Copy link
Member

This is naively ported but untested.

@Quentin-Anthony
Copy link
Member Author

This is naively ported but untested.

Huh? Please clarify.

@StellaAthena
Copy link
Member

This is naively ported but untested.

Huh? Please clarify.

Sorry, that was a progress update. I've ported the code changes to GPT-NeoX but haven't had a chance to test them yet.

@Quentin-Anthony
Copy link
Member Author

This is naively ported but untested.

Huh? Please clarify.

Sorry, that was a progress update. I've ported the code changes to GPT-NeoX but haven't had a chance to test them yet.

Gotcha.

@Quentin-Anthony
Copy link
Member Author

@StellaAthena -- Can you write up what you have into a draft PR? We can offload testing from you.

@StellaAthena
Copy link
Member

@Quentin-Anthony They're not ready for testing yet, but I can still open a draft PR if you'd like. Right now I'm in a place where I think I've copied over all of the core code but the kernels aren't building and I haven't been able to debug why that is yet.

I had hoped to make more progress on this last week, but got swamped with some other stuff. I'm happy to hand it off if someone else wants to take it over.

@Quentin-Anthony
Copy link
Member Author

@Quentin-Anthony They're not ready for testing yet, but I can still open a draft PR if you'd like. Right now I'm in a place where I think I've copied over all of the core code but the kernels aren't building and I haven't been able to debug why that is yet.

I had hoped to make more progress on this last week, but got swamped with some other stuff. I'm happy to hand it off if someone else wants to take it over.

Yes please make a draft PR

@StellaAthena StellaAthena mentioned this issue Dec 23, 2023
Closed
@StellaAthena StellaAthena linked a pull request Dec 23, 2023 that will close this issue
Fused Rotary Embeddings #1103
Closed
@yang
Copy link
Contributor

yang commented Dec 25, 2023

@StellaAthena Still wanting help getting this to work / testing this?

@StellaAthena
Copy link
Member

@StellaAthena Still wanting help getting this to work / testing this?

That would be excellent, thank you.

@StellaAthena StellaAthena assigned yang and unassigned StellaAthena Dec 25, 2023
@yang yang mentioned this issue Jan 2, 2024
Merged
@StellaAthena StellaAthena linked a pull request Jan 2, 2024 that will close this issue
Fused Rotary Embeddings (fixed) #1108
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yang yang

Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fused Rotary Embeddings (fixed) yang/gpt-neox
Fused Rotary Embeddings EleutherAI/gpt-neox
3 participants
@yang @Quentin-Anthony @StellaAthena