Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

feat(bulk_load): support clear last bulk load state rpc #1103

Merged
merged 5 commits into from
May 12, 2022

Conversation

hycdong
Copy link
Contributor

@hycdong hycdong commented May 11, 2022

Pegasus#872.
Meta server will store last bulk load state for client to query last bulk load result, this pull request adds a clear rpc interface to clear this state.

Copy link
Member

@acelyc111 acelyc111 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Is there a mechanism that the bulk load state can be auto sweep up?

@hycdong
Copy link
Contributor Author

hycdong commented May 12, 2022

LGTM Is there a mechanism that the bulk load state can be auto sweep up?

Meta server will reserve last bulk load state, including success and failure, and will clear it when next bulk load process is triggered. I think it is a way to auto sweep up state. This rpc provides the manual interface, which is facility for user-tools.
In our production environment, user will always retry whole bulk load process if they got error, which will sometimes lead to performance downgrade. As a result, our user tool will restrict error retry time, and new bulk load can not be triggered through tools, can only be triggered by admin-cli. To make it more flexible, we finally decide to add this manual interface.

@hycdong hycdong merged commit 3eb720b into XiaoMi:master May 12, 2022
@hycdong hycdong deleted the add_clear_bulk_load branch May 12, 2022 06:58
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants