-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recurring Tasks #4
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #4 +/- ##
==========================================
+ Coverage 72.68% 77.36% +4.67%
==========================================
Files 12 18 +6
Lines 681 1109 +428
==========================================
+ Hits 495 858 +363
- Misses 144 192 +48
- Partials 42 59 +17
Continue to review full report at Codecov.
|
lloydmeta
force-pushed
the
feature/recurring-tasks
branch
from
February 27, 2020 20:10
4590d75
to
86832da
Compare
We'll need to this to decide how we want to run things inside the recurring jobs scheduling loop. Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
* Move some shared models around from tasks.Service Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
* Add parsing of ScheduleExpressions with tests Signed-off-by: lloydmeta <[email protected]>
…l, so that we can see newly-deleted recurring tasks and unschedule them. * update implementation and add tests * Begin implementing recurring.Manager Signed-off-by: lloydmeta <[email protected]>
* Add logging statements to scheduler Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
…Functions for clarity Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
* Do not send loaded if the array is zero sized * Bulk load error handling Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
* Add thing wrappers around tracers and transactions * Use them for background tasks Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
race conditions due to asynchronous scheduling Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
…ssion Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
lloydmeta
force-pushed
the
feature/recurring-tasks
branch
from
February 27, 2020 20:12
86832da
to
0e4980f
Compare
* Make service.All reflect "realtime" data by refreshing the index first * In `enforceSync`, try an optimistic sync of just the problematic things before doing a full sync * More logs tightening Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
* Make a shared mock recurring.Service; make the existing test use it * Add a test for recurring.NewManager Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
Signed-off-by: lloydmeta <[email protected]>
…Recurring Task caused it to be spawned. * This required moving the recurring.Id model to task.RecurringTaskId because Go doesn't allow "circular" imports * Note that the user-facing API model does not have this field at all when creating Tasks. * Update ES template and Kibana index pattern to have this field Signed-off-by: lloydmeta <[email protected]>
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds initial support for recurring Tasks.
Recurring Tasks are Tasks that can be defined to automatically enqueue according to a given schedule expression.
Currently, the standard Cron parser from github.com/robfig/cron is used and so it supports basic macros.
The current implementation involves having a given leader server poll for changes, and (re-using the existing leader lock mechanism) depends on mark-as-dirty and soft-deletes of RecurringTasks. The periodic poll for changed-but-unseen data and a secondary polling sync-enforcer (check against all the recurring tasks defined in the data store) ensures that changes to recurring Tasks (creates, deletes, or updates), are taken into account. The polling intervals are configurable.
Each Task enqueued from a Recurring Task also has a
recurring_task_id
for tracking.At the moment, there is no support for "do not enqueue if there is a previous un-handled Task" for Recurring Tasks.