Allow graceful interruption of worker sleep #928
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, unless `Worker.raise_signal_exceptions` was set, the worker process ignored INT and TERM signals while sleeping after detection of an empty queue. That behavior was specially noticeable when `sleep_delay` was much greater than the default of 5 seconds: a daemonized job, for example, would end up being killed because of not responding in a reasonable time. Now, when the worker needs to sleep, it does so waiting on an event with a timeout. Stopping the job from a signal sets the event and aborts the wait immediately.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When daemonized through
Delayed::CommandDelayed::Workerwill ignore signals every time it sleeps after detecting an empty queue. That's specially problematic when using a longsleep_delay(30+ seconds) with a predominantly idle queue, because the Daemons gem will forcefully kill the process if it's not responsive.Another way the issue manifests is when executing something like the following:
Trying to stop the command with
Ctrl-Coutputs a messageExiting..., but nothing else happens, apparently. The command does stop, however, after the fullsleep_delayinterval elapses.The root cause is a
sleepcall inDelayed::Worker, which cannot be interrupted unless a signal is raised. Since raising signals is often undesirable because it might also interrupt unfinished jobs we need to address only the case whenDelayed::Workeris sleeping.This patch offers a solution, which consists in substituting the sleep call with a wait on an event. Setting the event from the trap enables the worker to exit immediately. Otherwise, if the timeout given to the wait call elapses, everything behaves as if sleep was called.
This resolves the issue #593 and may also be related to #459 and #916.