Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Serve] Make the checkpoint and recover only from GCS #26753

Merged
merged 9 commits into from
Jul 25, 2022
Merged

[Serve] Make the checkpoint and recover only from GCS #26753

merged 9 commits into from
Jul 25, 2022

Conversation

sihanwang41
Copy link
Contributor

@sihanwang41 sihanwang41 commented Jul 20, 2022 •
edited
Loading

Why are these changes needed?

Make the GCS as only way to recover. (Not well tested yet for external storage)

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@shrekris-anyscale shrekris-anyscale changed the title [Serve] Make the checkpint and recover only from GCS [Serve] Make the checkpoint and recover only from GCS Jul 20, 2022
@simon-mo
Copy link
Contributor

this is also mentioned in documentation. Can you take care of that?

edoakes
edoakes reviewed Jul 20, 2022
View reviewed changes
Copy link
Contributor

@edoakes edoakes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also get rid of most of the code under serve/storage.

simon-mo
simon-mo approved these changes Jul 21, 2022
View reviewed changes
Copy link
Contributor

@simon-mo simon-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also feel free to remove s3 and other stores in storage folder! since they are now no longer used.

python/ray/serve/api.py Outdated Show resolved Hide resolved
@edoakes
Copy link
Contributor

edoakes commented Jul 21, 2022

@sihanwang41 DCO build failed. Need to signoff on your commits. Use this to squash the changes here into one signed-off commit:

# From your branch.
git reset $(git merge-base master $(git branch --show-current))
git commit -s -m <your message>
git push -f

@edoakes edoakes added the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Jul 21, 2022
@simon-mo simon-mo merged commit 8ecd928 into ray-project:master Jul 25, 2022
ddelange added a commit to ddelange/ray that referenced this pull request Jul 26, 2022
@ddelange
Merge branch 'master' of https://github.com/ray-project/ray into pyto…
6f86ce4 
…rch-extra-index-url

* 'master' of https://github.com/ray-project/ray: (26 commits)
  [runtime env] plugin refactor[6/n]: java api refactor (ray-project#26783)
  easy test? (ray-project#26905)
  [core] Introduce a flag which allows a longer timeout for raylet when GCS restarts. (ray-project#26919)
  [RLLib] Record framework and algorithm used by an RLlib run. (ray-project#26956)
  [train] set split locality_hints (ray-project#26973)
  [Serve] Make the checkpoint and recover only from GCS (ray-project#26753)
  [AIR DOC] minor tweaks to checkpoint user guide for clarity and consistency subheadings (ray-project#26937)
  [tune] Only sync down from cloud if needed (ray-project#26725)
  [Core] Refactoring Ray DAG object scanner (ray-project#26917)
  [air] Raise error on path-like access for Checkpoints (ray-project#26970)
  [AIR] Enable other notebooks previously marked with # REGRESSION (ray-project#26896)
  [RLlib] Simplify agent collector (ray-project#26803)
  [Datasets] Automatically cast tensor columns when building Pandas blocks. (ray-project#26924)
  [Workflow] Fix flaky example(ray-project#26960)
  [dashboard] Update cluster_activities endpoint to use pydantic. (ray-project#26609)
  [air] Un-revert "[air] remove unnecessary logs + improve repr for result" (ray-project#26942)
  [setup-dev] Add flag to skip symlink certain folders (ray-project#26899)
  [air/tune/docs] Cont. convert Tune examples to use Tuner.fit() (ray-project#26959)
  [air/tune/docs] Change Tuner() occurences in rest of ray/tune (ray-project#26961)
  [data] set iter_batches default batch_size (ray-project#26955)
  ...

Signed-off-by: ddelange <[email protected]>
@scv119 scv119 mentioned this pull request Jul 26, 2022
Closed
klwuibm pushed a commit to yuanchi2807/ray that referenced this pull request Jul 27, 2022
@sihanwang41 @klwuibm
[Serve] Make the checkpoint and recover only from GCS (ray-project#26753
12e1213 
)

Signed-off-by: klwuibm <[email protected]>
Catch-Bull pushed a commit to alipay/ant-ray that referenced this pull request Jul 27, 2022
@sihanwang41 @Catch-Bull
[Serve] Make the checkpoint and recover only from GCS (ray-project#26753
c03ce5d 
)

Signed-off-by: Catch-Bull <[email protected]>
Rohan138 pushed a commit to Rohan138/ray that referenced this pull request Jul 28, 2022
@sihanwang41 @Rohan138
[Serve] Make the checkpoint and recover only from GCS (ray-project#26753
46571fb 
)

Signed-off-by: Rohan138 <[email protected]>
franklsf95 pushed a commit to franklsf95/ray that referenced this pull request Aug 2, 2022
@sihanwang41 @franklsf95
[Serve] Make the checkpoint and recover only from GCS (ray-project#26753
29e63ff 
)

Signed-off-by: Frank Luan <[email protected]>
gramhagen pushed a commit to gramhagen/ray that referenced this pull request Aug 15, 2022
@sihanwang41
[Serve] Make the checkpoint and recover only from GCS (ray-project#26753
ba39e5f 
)

Signed-off-by: Scott Graham <[email protected]>
gramhagen pushed a commit to gramhagen/ray that referenced this pull request Aug 15, 2022
@sihanwang41
[Serve] Make the checkpoint and recover only from GCS (ray-project#26753
351fe55 
)
Stefan-1313 pushed a commit to Stefan-1313/ray_mod that referenced this pull request Aug 18, 2022
@sihanwang41
[Serve] Make the checkpoint and recover only from GCS (ray-project#26753
3fb2b44 
)

Signed-off-by: Stefan van der Kleij <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edoakes edoakes edoakes approved these changes

@simon-mo simon-mo simon-mo approved these changes

@jiaodong jiaodong Awaiting requested review from jiaodong

@shrekris-anyscale shrekris-anyscale Awaiting requested review from shrekris-anyscale shrekris-anyscale is a code owner

Assignees

@edoakes edoakes

@simon-mo simon-mo

Labels
@author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@sihanwang41 @simon-mo @edoakes