Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] DreamerV3: Make 200M (XL model) work; mixed float16 option #38461

Merged
merged 85 commits into from
Aug 25, 2023
Merged

[RLlib] DreamerV3: Make 200M (XL model) work; mixed float16 option #38461

merged 85 commits into from
Aug 25, 2023

Conversation

sven1977
Copy link
Contributor

@sven1977 sven1977 commented Aug 15, 2023 •
edited
Loading

DreamerV3: Make 200M (XL model) work; mixed float16 option

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

sven1977 and others added 30 commits August 10, 2023 16:21
@sven1977
wip
7e8460a 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
e352970 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
a6615d1 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
6baf773 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
7cb7c85 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
df06e3c 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
ff82335 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
604884c 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
dbb2d9e 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
941201f 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
fcd2b6b 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
f20a044 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
4df58db 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
4dc773f 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into drea…
6463282 
…mer_v3_06_make_200M_work
@sven1977
wip
dc604e7 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
e6a8a0a 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
400d277 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
ae3e471 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
68bcdeb 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
4e86fb1 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
0511b22 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
6950357 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
0c63a29 
Signed-off-by: sven1977 <[email protected]>
hacked working XL model
38396f9
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into drea…
7b5eab0 
…mer_v3_06_make_200M_work
@sven1977
wip
cf73fca 
Signed-off-by: sven1977 <[email protected]>
@sven1977
working XL model training w/o gRAM OOMs
0969377 
* computing all gradients
* NOT postprocessing
* NOT applying

Signed-off-by: sven1977 <[email protected]>
working XL model: compute gradients only for critic, but postprocess …
810fbe8 
…AND apply them as well
@sven1977
wip
3a0e4df 
Signed-off-by: sven1977 <[email protected]>
@sven1977 sven1977 added release-blocker P0 Issue that blocks the release rllib RLlib related issues rllib-newstack labels Aug 24, 2023
@sven1977
wip
dcf302c 
Signed-off-by: sven1977 <[email protected]>
krfricke
krfricke approved these changes Aug 24, 2023
View reviewed changes
Copy link
Contributor

@krfricke krfricke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stamp

@zhe-thoughts zhe-thoughts self-assigned this Aug 24, 2023
zhe-thoughts
zhe-thoughts approved these changes Aug 24, 2023
View reviewed changes
Copy link
Collaborator

@zhe-thoughts zhe-thoughts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is critical PR to merge

@zhe-thoughts
Copy link
Collaborator

@sven1977 @krfricke @kouroshHakha : could one of you confirm that CI is OK, before I merge?

@sven1977
LINT
40217a5 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Copy link
Contributor Author

Thanks @zhe-thoughts ! I'll let you know. Just waiting for tests now to all pass.

@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into drea…
6811a9f 
…mer_v3_06_2_make_200M_work_plus_float16
@sven1977
LINT
5c2a133 
Signed-off-by: sven1977 <[email protected]>
@sven1977
LINT
19ff410 
Signed-off-by: sven1977 <[email protected]>
@sven1977
fix
b73b8c5 
Signed-off-by: sven1977 <[email protected]>
@sven1977
doctests
945c292 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
2d0f3d0 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
fe2749e 
Signed-off-by: sven1977 <[email protected]>
@sven1977
LINT
5d0eaa3 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Copy link
Contributor Author

Hey @zhe-thoughts , this would be good to merge. All tests are passing now. Thank you so much!

@zhe-thoughts zhe-thoughts added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Aug 25, 2023
@zhe-thoughts zhe-thoughts merged commit b004579 into ray-project:master Aug 25, 2023
1 of 2 checks passed
arvind-chandra pushed a commit to lmco/ray that referenced this pull request Aug 31, 2023
@sven1977 @arvind-chandra
[RLlib] DreamerV3: Make 200M (XL model) work; mixed float16 option (r…
f3b654d 
…ay-project#38461)

Signed-off-by: sven1977 <[email protected]>
Signed-off-by: e428265 <[email protected]>
LeonLuttenberger pushed a commit to jaidisido/ray that referenced this pull request Sep 5, 2023
@sven1977 @LeonLuttenberger
[RLlib] DreamerV3: Make 200M (XL model) work; mixed float16 option (r…
3f867e1 
…ay-project#38461)

Signed-off-by: sven1977 <[email protected]>
jimthompson5802 pushed a commit to jimthompson5802/ray that referenced this pull request Sep 12, 2023
@sven1977 @jimthompson5802
[RLlib] DreamerV3: Make 200M (XL model) work; mixed float16 option (r…
a9f2779 
…ay-project#38461)

Signed-off-by: sven1977 <[email protected]>
Signed-off-by: Jim Thompson <[email protected]>
vymao pushed a commit to vymao/ray that referenced this pull request Oct 11, 2023
@sven1977
[RLlib] DreamerV3: Make 200M (XL model) work; mixed float16 option (r…
064d352 
…ay-project#38461)

Signed-off-by: sven1977 <[email protected]>
Signed-off-by: Victor <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maxpumperla maxpumperla maxpumperla approved these changes

@zhe-thoughts zhe-thoughts zhe-thoughts approved these changes

@krfricke krfricke krfricke approved these changes

@kouroshHakha kouroshHakha kouroshHakha approved these changes

@gjoliver gjoliver Awaiting requested review from gjoliver

@avnishn avnishn Awaiting requested review from avnishn

@ArturNiederfahrenhorst ArturNiederfahrenhorst Awaiting requested review from ArturNiederfahrenhorst

@smorad smorad Awaiting requested review from smorad

Assignees

@ArturNiederfahrenhorst ArturNiederfahrenhorst

@zhe-thoughts zhe-thoughts

@kouroshHakha kouroshHakha

Labels
release-blocker P0 Issue that blocks the release rllib RLlib related issues rllib-newstack tests-ok The tagger certifies test failures are unrelated and assumes personal liability.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@sven1977 @zhe-thoughts @maxpumperla @krfricke @kouroshHakha @ArturNiederfahrenhorst