Skip to content

A job queue backend for the Flow framework, based on Doctrine

License

Notifications You must be signed in to change notification settings

Flowpack/jobqueue-doctrine

Repository files navigation

Flowpack.JobQueue.Doctrine

A job queue backend for the Flowpack.JobQueue.Common package based on Doctrine.

Usage

Install the package using Composer:

composer require flowpack/jobqueue-doctrine

If not already installed, that will fetch its requirements, namely the jobqueue-common package.

Now the queue can be configured like this:

Flowpack:
  JobQueue:
    Common:
      queues:
        'some-queue':
          className: 'Flowpack\JobQueue\Doctrine\Queue\DoctrineQueue'
          executeIsolated: true
          options:
            defaultTimeout: 50
          releaseOptions:
            priority: 512
            delay: 120

The required tables can be created executing:

./flow queue:setup some-queue

Boost Mode

The poll interval should be short enough to process messages in time, and long enough to minimize resource consumption for the database. Boost mode is a solution which automatically handles spikes by processing messages in quick succession. When no new messages appear for a specified time, boost mode is disabled again.

The frequency by which the queue loop will look for new messages is the configured pollInterval. In boost mode, the option boostPollInterval is used instead. boostTime defines the time since the last processed message after which boost mode is deactivated again.

Specific options

The DoctrineQueue supports following options:

Option Type Default Description
defaultTimeout integer 60 Number of seconds new messages are waited for before a timeout occurs, this is overridden by a "timeout" argument in the waitAndTake() and waitAndReserve() methods
pollInterval float 1 Number of seconds between SQL lookups for new messages
boostPollInterval float 0.5 Number of seconds between SQL lookups for new messages when in "boost mode"
boostTime float 10 Maximum number of seconds since last processed message to activate "boost mode"
tableName string flowpack_jobqueue_messages_ Name of the database table for this queue. By default this is the queue name prefixed with "flowpack_jobqueue_messages_"
backendOptions array - Doctrine-specific connection params (see Doctrine reference)

NOTE: The DoctrineQueue should work with any database supported by Doctrine DBAL. It has been tested on MySQL, PostgreSQL, SQL Server and SQLite. You can specify the backend via the backendOptions. If you omit this setting, the current connection will be re-used (i.e. the currently active Flow database).

Submit options

Additional options supported by JobManager::queue(), DoctrineQueue::submit() and the Job\Defer annotation:

Option Type Default Description
delay integer 0 Number of seconds before a message is marked "ready" after submission. This can be useful to prevent premature execution of jobs (i.e. before entities are persisted)

Release options

Additional options to be specified via releaseOptions for the queue:

Option Type Default Description
delay integer 0 Number of seconds before a message is marked "ready" after it has been released.

License

This package is licensed under the MIT license

Contributions

Pull-Requests are more than welcome. Make sure to read the Code Of Conduct.