Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce configurable PULL_BATCH_COUNT and DELTA_COUNT #57

Merged
merged 1 commit into from
Feb 26, 2022

Conversation

shayonj
Copy link
Owner

@shayonj shayonj commented Feb 26, 2022

On tables with high volume workloads,
having the ability to tune PULL_BATCH_COUNT
and DELTA_COUNT is nice for faster catchup and swap.

The default values are 1000 and 20 respectively. Which means,
in one pass pg-osc will replay 1000 rows and if the count is <20
then it will perform the swap and replay.

By introducing new cli flags (--pull-batch-count, --delta-count),
these figures can be tuned further

Fixes: #21

On tables with high volume workloads,
having the ability to tune PULL_BATCH_COUNT
and DELTA_COUNT is nice for faster catchup and swap.

The default values are 1000 and 20 respectively. Which means,
in one pass pg-osc will replay 1000 rows and if the count is <20
then it will perform the swap and replay.

By introducing new cli flags (--pull-batch-count, --delta-count),
these figures can be tuned further
@shayonj shayonj merged commit 2fb94c3 into main Feb 26, 2022
@shayonj shayonj deleted the s/pull-delta-count branch February 26, 2022 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make PULL_BATCH_COUNT AND DELTA_COUNT configurable by a flag
1 participant