Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug?] Fix profiling argument names #1155

Merged
merged 21 commits into from
Feb 26, 2024
Merged

[Bug?] Fix profiling argument names #1155

merged 21 commits into from
Feb 26, 2024

Conversation

haileyschoelkopf
Copy link
Contributor

I believe some of the profiling neox args had their names changed, which seems to be causing trouble when I try to run on the latest codebase commit:

Traceback (most recent call last):
  File "/weka/hailey/mistral-support-neox/gpt-neox/train.py", line 34, in <module>
    main()
  File "/weka/hailey/mistral-support-neox/gpt-neox/train.py", line 30, in main
    pretrain(neox_args=neox_args)
  File "/weka/hailey/mistral-support-neox/gpt-neox/megatron/training.py", line 228, in pretrain
    iteration = train(
  File "/weka/hailey/mistral-support-neox/gpt-neox/megatron/training.py", line 884, in train
    loss_dict, skipped_iter = train_step(
  File "/weka/hailey/mistral-support-neox/gpt-neox/megatron/training.py", line 775, in train_step
    neox_args.profiling
AttributeError: 'NeoXArgs' object has no attribute 'profiling'

I've adjusted neox_args.profile -> neox_args.profiling to fix this.

Also wondering when the flops_profiler arg gets used? But I may just be missing that.

@Quentin-Anthony
Copy link
Member

I spoke with @jahatef and the actual typo was the neox_args.profiling call. I prefer neox_args.profile and will update this branch accordingly.

@Quentin-Anthony Quentin-Anthony merged commit 9663802 into main Feb 26, 2024
2 checks passed
@Quentin-Anthony Quentin-Anthony deleted the fix-profiling branch February 26, 2024 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants