New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port NVIDIA Nsight profiling to gpt-neox #1134

Closed

1 of 2 tasks

Quentin-Anthony opened this issue Jan 27, 2024 · 0 comments · Fixed by #1154

Labels

feature request

Member

Quentin-Anthony commented Jan 27, 2024 •

edited

Loading

Megatron supports an easy way to profile a few steps in training: https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/arguments.py#L834

Currently I'm just manually running nsight, but this would make things easier.

Port Megatron-LM nsight changes to gpt-neox (done: Memory profiling #1153)
Document the change in the README

Quentin-Anthony added the feature request label

Quentin-Anthony changed the title ~~Port NVIDA Nsight profiling to gpt-neox~~ Port NVIDIA Nsight profiling to gpt-neox

jahatef mentioned this issue

add profiling to readme #1154

Merged

Quentin-Anthony closed this as completed in #1154

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment