Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] Remove DataContext variables for legacy execution backend / stages optimizer #42780

Merged
merged 10 commits into from
Jan 29, 2024
Merged

[Data] Remove DataContext variables for legacy execution backend / stages optimizer #42780

merged 10 commits into from
Jan 29, 2024

Conversation

scottjlee
Copy link
Contributor

@scottjlee scottjlee commented Jan 28, 2024 •
edited
Loading

Why are these changes needed?

Removes DataContext attributes related to the legacy execution backend and stages optimizer:

  • optimize_fuse_stages
  • optimize_fuse_read_stages
  • optimize_fuse_shuffle_stages
  • optimize_reorder_stages
  • optimizer_enabled
  • new_execution_backend
  • use_streaming_executor

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@scottjlee scottjlee marked this pull request as ready for review January 29, 2024 17:31
@c21
Copy link
Contributor

c21 commented Jan 29, 2024

Need to rebase to latest master.

@scottjlee
merge
bb49537 
Signed-off-by: Scott Lee <[email protected]>
@scottjlee
clean up
ecc9b28 
Signed-off-by: Scott Lee <[email protected]>
@c21 c21 merged commit 3173487 into ray-project:master Jan 29, 2024
9 checks passed
This was referenced Jun 5, 2024
Closed
Merged
bveeramani added a commit that referenced this pull request Jun 5, 2024
@bveeramani
[Data] Clean up some uses of check_num_computed in test_parquet.py (
2e22b28 
#45758)

check_num_computed counts the number of read tasks executed within a LazyBlockList. With the removal of the legacy execution backend (#42780), this value is typically 0, and it's not useful to test against.

Signed-off-by: Balaji Veeramani <[email protected]>
ryanaoleary pushed a commit to ryanaoleary/ray that referenced this pull request Jun 6, 2024
@bveeramani @ryanaoleary
[Data] Clean up some uses of check_num_computed in test_parquet.py (
886c7da 
ray-project#45758)

check_num_computed counts the number of read tasks executed within a LazyBlockList. With the removal of the legacy execution backend (ray-project#42780), this value is typically 0, and it's not useful to test against.

Signed-off-by: Balaji Veeramani <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>
yucai pushed a commit to yucai/ray that referenced this pull request Jun 6, 2024
@bveeramani @yucai
[Data] Clean up some uses of check_num_computed in test_parquet.py (
2a2afe2 
ray-project#45758)

check_num_computed counts the number of read tasks executed within a LazyBlockList. With the removal of the legacy execution backend (ray-project#42780), this value is typically 0, and it's not useful to test against.

Signed-off-by: Balaji Veeramani <[email protected]>
Signed-off-by: yucai <[email protected]>
ryanaoleary pushed a commit to ryanaoleary/ray that referenced this pull request Jun 6, 2024
@bveeramani @ryanaoleary
[Data] Clean up some uses of check_num_computed in test_parquet.py (
028c15f 
ray-project#45758)

check_num_computed counts the number of read tasks executed within a LazyBlockList. With the removal of the legacy execution backend (ray-project#42780), this value is typically 0, and it's not useful to test against.

Signed-off-by: Balaji Veeramani <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>
richardsliu pushed a commit to richardsliu/ray that referenced this pull request Jun 12, 2024
@bveeramani @richardsliu
[Data] Clean up some uses of check_num_computed in test_parquet.py (
4256024 
ray-project#45758)

check_num_computed counts the number of read tasks executed within a LazyBlockList. With the removal of the legacy execution backend (ray-project#42780), this value is typically 0, and it's not useful to test against.

Signed-off-by: Balaji Veeramani <[email protected]>
Signed-off-by: Richard Liu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@c21 c21 c21 approved these changes

@ericl ericl Awaiting requested review from ericl

@scv119 scv119 Awaiting requested review from scv119

@amogkam amogkam Awaiting requested review from amogkam

@bveeramani bveeramani Awaiting requested review from bveeramani bveeramani is a code owner

@raulchen raulchen Awaiting requested review from raulchen raulchen is a code owner

@stephanie-wang stephanie-wang Awaiting requested review from stephanie-wang stephanie-wang is a code owner

@Zandew Zandew Awaiting requested review from Zandew

Assignees

@c21 c21

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@scottjlee @c21