-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[serve][streaming] No replicas available blocks the HTTP proxy event loop #36460
Comments
Merged
8 tasks
edoakes
added a commit
that referenced
this issue
Jun 15, 2023
…ocking proxy event loop (#36459) `replica_updated_event` was not being cleared. It seems that even though we were awaiting the event, it was not yielding the loop because the event was always set. This caused the `while replica is not None` loop to busy-spin, blocking the HTTP proxy event loop. Closes #36460
arvind-chandra
pushed a commit
to lmco/ray
that referenced
this issue
Aug 31, 2023
…ocking proxy event loop (ray-project#36459) `replica_updated_event` was not being cleared. It seems that even though we were awaiting the event, it was not yielding the loop because the event was always set. This caused the `while replica is not None` loop to busy-spin, blocking the HTTP proxy event loop. Closes ray-project#36460 Signed-off-by: e428265 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The loop waiting for a replica is not properly yielding. Repro:
In one terminal:
In second terminal:
The
/-/routes
call hangs until the replica finishes initializing.The text was updated successfully, but these errors were encountered: