Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Copy-on-Write forking (logic copied from Puma) #1160

Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
54eba86
Adds support for running as a forking process in the foreground
johnnyshields Apr 4, 2021
3401c24
Cleanup behavior for Command#launch and #daemonize
johnnyshields Apr 4, 2021
d328cb8
Prevent specs from creating stray directories
johnnyshields Apr 5, 2021
dd0db79
Fix rake tasks
johnnyshields Apr 5, 2021
40d9f04
More fixes to rake command parsing
johnnyshields Apr 5, 2021
0a7145f
- Add tests for Rake tasks
johnnyshields Apr 5, 2021
9d0d3fe
Make life easy and require all files at boot time
johnnyshields Apr 5, 2021
246936c
Revert require of Command
johnnyshields Apr 5, 2021
ea83992
Fix edge case where queues are nil
johnnyshields Apr 6, 2021
2c87350
Improve log message
johnnyshields Apr 6, 2021
853efa9
Fix CI
johnnyshields Apr 6, 2021
ed855a4
Try again
johnnyshields Apr 6, 2021
226d6d8
Fix spec
johnnyshields Apr 6, 2021
7f284da
Add continue-on-error to steps
johnnyshields Apr 6, 2021
6f225da
Fix spec
johnnyshields Apr 6, 2021
a17c488
Ignore process messages from non-workers
johnnyshields Apr 6, 2021
0d06a33
Fix rubocop
johnnyshields Apr 6, 2021
293522b
Shutdown after child killed with 30 sec grace period.
johnnyshields Apr 7, 2021
60ccd9a
Fix bug
johnnyshields Apr 7, 2021
6ab0d17
Cleanup logging
johnnyshields Apr 7, 2021
e52508a
Support Copy-on-Write forking from first worker. Code is borrowed fro…
johnnyshields Jan 9, 2022
cc26386
Port more code from Puma
johnnyshields Jan 9, 2022
1e4b105
Fix specs
johnnyshields Jan 9, 2022
d1bef03
Log via queue
johnnyshields Jan 9, 2022
0b57172
Remove time from log
johnnyshields Jan 9, 2022
ebbc778
Run worker on thread
johnnyshields Jan 9, 2022
e6efe21
Fix Class name
johnnyshields Jan 9, 2022
20bac34
Fix logger in child class
johnnyshields Jan 9, 2022
98225b4
Fix ping method
johnnyshields Jan 9, 2022
759b67d
Fix spec
johnnyshields Jan 9, 2022
aa2a012
Fix one more bug
johnnyshields Jan 9, 2022
92eee8d
Fix bug
johnnyshields Jan 9, 2022
7713000
Fix issue with ping
johnnyshields Jan 9, 2022
8eef0c2
Fix phased_restart
johnnyshields Jan 9, 2022
813fe78
Fix single mode
johnnyshields Jan 9, 2022
6a13532
Fix logging
johnnyshields Jan 9, 2022
e6c8a9d
Fix logger
johnnyshields Jan 9, 2022
b904f12
Port more signal handling from Puma
johnnyshields Jan 10, 2022
2573b46
Add one more line
johnnyshields Jan 10, 2022
f8166f9
Fix graceful stop on term
johnnyshields Jan 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Port more code from Puma
  • Loading branch information
johnnyshields committed Jan 9, 2022
commit cc26386d59b1c2f8ff3fb6db8ccd8c4697cf0266
14 changes: 10 additions & 4 deletions lib/delayed/launcher/cluster.rb
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,13 @@ def cull_childs

logger.debug "Culling #{diff.inspect} workers"

handles_to_cull = @child_handles[-diff, diff]
handles_to_cull =
case @options[:worker_culling_strategy]
when :youngest
@child_handles.sort_by(&:started_at)[-diff, diff]
when :oldest
@child_handles.sort_by(&:started_at)[0, diff]
end
logger.debug "Workers to cull: #{handles_to_cull.inspect}"

handles_to_cull.each do |handle|
Expand Down Expand Up @@ -387,10 +393,10 @@ def spawn_childs
end

def next_child_index
all_positions = 0...@child_count
occupied_positions = @child_handles.map(&:index)
available_positions = all_positions.to_a - occupied_positions
available_positions.first
idx = 0
idx += 1 until !occupied_positions.include?(idx)
idx
end

def spawn_child(idx, parent)
Expand Down
8 changes: 5 additions & 3 deletions lib/delayed/launcher/forking.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,13 @@ def initialize(options)
options.delete(:args)

# Set default options
options[:worker_count] ||= 1
options[:worker_check_interval] ||= DEFAULT_WORKER_CHECK_INTERVAL
options[:worker_timeout] ||= DEFAULT_WORKER_TIMEOUT
options[:worker_boot_timeout] ||= DEFAULT_WORKER_TIMEOUT
options[:worker_shutdown_timeout] ||= DEFAULT_WORKER_SHUTDOWN_TIMEOUT
options[:worker_count] ||= 1
# TODO: need phased restart timeout
options[:worker_culling_strategy] ||= :youngest
options.delete(:pools) if options[:pools] == []
options[:pid_dir] ||= "#{Delayed.root}/tmp/pids"
options[:log_dir] ||= "#{Delayed.root}/log"
Expand All @@ -49,9 +51,9 @@ def run
end

# Begin graceful shutdown of the workers
def stop(timeout = nil)
def stop
# @status = :stop
@runner.stop(timeout)
@runner.stop
end

# Begin forced shutdow nof the workers
Expand Down