-
Notifications
You must be signed in to change notification settings - Fork 208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
storage: using concurrent data structure to incrementally maintain storage snapshot #620
Comments
how to achieve this? make a snapshot of each epoch and only track the different from last? |
Exactly, but I didn't come up with a detailed way to achieve this... Indeed, every operation to RisingLight's storage manifest is simply a diff operation. |
I'm sorry, i have no relevant experience about this. Unable to give helpful advice :) |
No worries 🤪 |
Currently, every operation on storage snapshot will clone and create a new full snapshot, which contains a list of rowsets. Indeed, we can use some concurrent data structure to maintain this, so as to reduce the need of cloning.
risinglight/src/storage/secondary/version_manager.rs
Lines 257 to 265 in 3e59c36
https://github.com/ibraheemdev/seize looks like a good choice.
The text was updated successfully, but these errors were encountered: