-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Serve] Use Async Handle for DAG Execution #27411
Conversation
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
python/ray/dag/py_obj_scanner.py
Outdated
to internal data structures, preventing actually writing them to | ||
the buffer. | ||
""" | ||
if obj is _get_object or obj is _get_object: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
duplicate
Signed-off-by: simon-mo <[email protected]>
index = len(self._found) | ||
self._found.append(obj) | ||
return _get_node, (id(self), index) | ||
else: | ||
index = len(self._objects) | ||
self._objects.append(obj) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how do we do the dict object to do append?
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thank you for adding unit tests to the PyObjScanner, that class makes me a bit nervous in general.
Only concern is exposing this RayServeLazyAsyncHandle
as a user-facing class... if we can fix that for the release, great, but not a complete dealbreaker.
python/ray/dag/py_obj_scanner.py
Outdated
self._found = None | ||
# List of other objects found during the serializatoin pass. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# List of other objects found during the serializatoin pass. | |
# List of other objects found during the serialization pass. |
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: simon-mo <[email protected]>
Signed-off-by: Stefan van der Kleij <[email protected]>
Signed-off-by: simon-mo [email protected]
Why are these changes needed?
This PR replaces the sync handle usage in deployment graph with async handle. It uses the following method
.remote
return value as aasyncio.Task
instead of acoroutine
. This doesn't change the UXref = await handle.remote()
but it allows the result to be awaited twice by DAG code..execute
and get theasyncio.Task
back..remote
call.PyObjScanner
to find all occurrence ofasyncio.Task
and resolve them in batch.PyObjScanner
to handle arbitrary source type for replacement.PyObjScanner
so it will not serialize input objects.This PR is code complete and ready for review. Before merge, I'll update the documentation and example code. Add more tests.
Related issue number
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.