Sidekiq::Throttler is a middleware for Sidekiq that adds the ability to rate limit job execution on a per-worker basis.
Sidekiq::Throttler supports Sidekiq versions 2 and 3 and is actively tested against Ruby versions 2.0.0, 2.1, and 2.2.
Add this line to your application's Gemfile:
gem 'sidekiq-throttler'
And then execute:
$ bundle
Or install it yourself as:
$ gem install sidekiq-throttler
In a Rails initializer or wherever you've configured Sidekiq, add Sidekiq::Throttler to your server middleware:
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add Sidekiq::Throttler
end
end
Sidekiq::Throttler defaults to in-memory storage of job execution times. If
you have multiple worker processes, or frequently restart your processes, this
will be unreliable. Instead, specify the :redis
storage option:
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add Sidekiq::Throttler, storage: :redis
end
end
In a worker, specify a threshold (maximum jobs) and period for throttling:
class MyWorker
include Sidekiq::Worker
sidekiq_options throttle: { threshold: 50, period: 1.hour }
def perform(user_id)
# Do some heavy API interactions.
end
end
In the above example, when the number of executed jobs for the worker exceeds 50 in an hour, remaining jobs will be delayed.