-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add batch split #6
Changes from 1 commit
27080ea
49f184a
ac7096d
8e6e698
300f8b3
918d432
b31552c
095d561
84a7a16
d9fa785
4529dda
c193676
fbd7230
872f84a
dec91b9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
Signed-off-by: Connor1996 <[email protected]>
- Loading branch information
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -130,33 +130,15 @@ old command type `Split` still needs to be preserved. | |
|
||
### How to produce multiple split keys | ||
|
||
This part mainly focuses on `SplitChecker`. | ||
|
||
First of all, adjust `trait` so that it can return multiple split keys. | ||
|
||
```rust | ||
pub trait SplitChecker { | ||
// ... | ||
|
||
// before: fn split_key(&mut self) -> Option<Vec<u8>> | ||
fn split_keys(&mut self) -> Vec<Vec<u8>>; | ||
|
||
// before: fn approximate_split_key(&self, _: &Region, _: &DB) -> Result<Option<Vec<u8>>> | ||
fn approximate_split_keys(&self, _: &Region, _: &DB) -> Result<Vec<Vec<u8>>>; | ||
} | ||
``` | ||
|
||
Then add one config `batch_split_limit` to limit the number of produced split | ||
keys in a batch. If it is unlimited, for a once split check, it scans all over | ||
the Region's range, and in some extreme case, this would cause performance issue. | ||
|
||
Now we have four split-checkers: half, keys, size, and table. SizeChecker and | ||
KeysChecker can be rewritten to produce multiple keys, and other checkers' | ||
logic stays unchanged. | ||
|
||
The general logic of SizeChecker and KeysChecker are similar. The only | ||
difference between them is one splits Region based on size and the other splits | ||
Region based on the number of keys. So here we mainly describe the logic of | ||
First introducing one config `batch_split_limit` to limit the number of produced | ||
split keys in a batch. If it is unlimited, for a once split check, it scans all | ||
over the Region's range, and in some extreme case, this would cause performance | ||
issue. | ||
|
||
SizeChecker and KeysChecker can be rewritten to produce multiple keys, and the | ||
general logic of SizeChecker and KeysChecker are similar. The only difference | ||
between them is one splits Region based on size and the other splits Region | ||
based on the number of keys. So here we mainly describe the logic of | ||
SizeChecker: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually, I think the following two paragraphs are too verbose and not necessary. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IMO, the algorithm should discussed in RFC stage. |
||
|
||
- before: it scans key-value pairs in a Region's range sequentially to | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, this section is too detailed and not necessary.