Introduce PayloadNotFoundError (subclass of DeserializationError)

[johnnyshields]

Currently, there are two types of DeserializationErrors:

1. The handler code itself is malformed, e.g. triggered by a Psych::SyntaxError

2. The payload object is not found in the database (refer to Delayed::PsychExt class, e.g. the below)

        when %r{^!ruby/ActiveRecord:(.+)$}
          klass = resolve_class(Regexp.last_match[1])
          payload = Hash[*object.children.map { |c| accept c }]
          id = payload['attributes'][klass.primary_key]
          id = id.value if defined?(ActiveRecord::Attribute) && id.is_a?(ActiveRecord::Attribute)
          begin
            klass.unscoped.find(id)
          rescue ActiveRecord::RecordNotFound => error
            raise Delayed::PayloadNotFoundError, "ActiveRecord::RecordNotFound, class: #{klass}, primary key: #{id} (#{error.message})"
          end

Case 1 makes sense to handle by immediately permanently failing the job, if the yaml is malformed the job will never run.

Case 2 however, there is a possibility that the job will run at a later time, as the object could appear later in the database. This is especially true for Mongoid, which is non-transactional and uses replication with some minor timelag. When triggering a delayed job (e.g. a mailer) on new object creation, the job can possibly be invoked and query a replica set member BEFORE the data has replicated, raising a Mongoid::Errors::DocumentNotFound which is re-raised as a Delayed::DeserializationError.

To give power users like myself more flexibility, I've introduced a new config Delayed::Worker.fail_if_payload_not_found. The default is the current behavior (immediate failure) however it can be tweaked to retry.

[johnnyshields]

Spec failure here is not due to this PR, please check Travis.