Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stop the queue-proxy drain if we don't receive a TERM signal #13229

Merged
merged 1 commit into from
Aug 20, 2022

Conversation

dprotaso
Copy link
Member

@dprotaso dprotaso commented Aug 18, 2022

Fixes #12571

Proposed Changes

  • Queue proxy will stop draining if it has not received a TERM signal. This implies the user container is being restarted and not the pod.

Release Note

Readiness probes no longer fail if the user container is restarted (due to a liveness probe failure)

- this implies the user-container is being restarted but not the entire pod
@knative-prow knative-prow bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Aug 18, 2022
@knative-prow
Copy link

knative-prow bot commented Aug 18, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dprotaso

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. area/autoscale area/networking area/test-and-release It flags unit/e2e/conformance/perf test issues for product features labels Aug 18, 2022
@dprotaso
Copy link
Member Author

/assign @nader-ziada
/assign @psschwei

@codecov
Copy link

codecov bot commented Aug 18, 2022

Codecov Report

Merging #13229 (26d7d93) into main (d60e5e7) will decrease coverage by 0.08%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main   #13229      +/-   ##
==========================================
- Coverage   86.62%   86.53%   -0.09%     
==========================================
  Files         196      196              
  Lines       14495    14505      +10     
==========================================
- Hits        12556    12552       -4     
- Misses       1640     1653      +13     
- Partials      299      300       +1     
Impacted Files Coverage Δ
pkg/queue/sharedmain/main.go 0.61% <0.00%> (-0.02%) ⬇️
pkg/autoscaler/statforwarder/processor.go 88.00% <0.00%> (-6.00%) ⬇️
pkg/autoscaler/statforwarder/forwarder.go 90.66% <0.00%> (-5.34%) ⬇️
pkg/reconciler/configuration/configuration.go 85.20% <0.00%> (+1.53%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

go func() {
select {
case <-ctx.Done():
case <-time.After(time.Second):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so you check for this every 1 second the whole time the server is running?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - there's no for loop and it's in the /wait-for-drain handler.

@nader-ziada
Copy link
Member

/lgtm

@knative-prow knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Aug 19, 2022
@dprotaso
Copy link
Member Author

{ init failed to get project from boskos: failed to get a "gke-project" from boskos: resources not found}

/retest

@dprotaso
Copy link
Member Author

/retest

2 similar comments
@krsna-m
Copy link
Contributor

krsna-m commented Aug 19, 2022

/retest

@nader-ziada
Copy link
Member

/retest

@dprotaso
Copy link
Member Author

Pending: Unschedulable (message: no nodes available to schedule pods)

/retest

@knative-prow knative-prow bot merged commit ea5c535 into knative:main Aug 20, 2022
@dprotaso dprotaso deleted the serving-probe branch August 22, 2022 15:14
bradbeam added a commit to coreweave/serving that referenced this pull request Nov 16, 2023
Signed-off-by: Brad Beam <[email protected]>
bradbeam added a commit to coreweave/serving that referenced this pull request Nov 17, 2023
This takes the spirit of knative#13229 and backports it to our version.
Upstream change was made after the refactoring/moving of this code in

To avoid bumping the knative/pkg dependency, an updated Drainer was
copied over.

Relevent commits:
Queue Proxy: knative@ea5c535
Drainer: https://github.com/knative/pkg/blob/97c7258e3a98b81459936bc7a29dc6a9540fa357/network/handlers/drain.go

Signed-off-by: Brad Beam <[email protected]>
bradbeam added a commit to coreweave/serving that referenced this pull request Nov 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/autoscale area/networking area/test-and-release It flags unit/e2e/conformance/perf test issues for product features lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Readiness probe fails if the container restarts after a liveness probe fail
4 participants