This repository has been archived by the owner on Oct 23, 2022. It is now read-only.
Use Key-value DB for DataStore and PinStore #439
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is for using key-value db like sled as DataStore and PinSotre, so that it is more efficient for pinning and easy to realize MFS.
The FsDatastore is inefficiency now, when checking a indirect pinned cid, all file in the FsDatastore will be visited and read, what more important is that it is hard to distinguish the data for pinning and the data for DataStore. The key-value db like sled will bring convenience and efficiency in the next work with its high efficiency and easy key/value operation. Leveldb may be the first choice of key-value dbs, but now there are no suitable rust leveldb crates; Sled is fast and simple, and it is usable on servers and phones from any C-compatible language.
This PR is the reopen of PR434. For there is some verbose commits like "replaceing leveldb with sled", the master branch of fetchadd/rust-ipfs in the PR434 is deprecated. Now the same changes are applied to the staging branch which is synced with the master of rust-ipfs/rust-ipfs. So Thanks for the PR of fetchadd/rust-ipfs from @koivunej which uses mutex to work around init(&self) and unsafe, but as the master is deprecated, I can only copy the code to staging branch.
Accorrding to the review from @ koivunej in PR434, the
pinstore_interface_tests
is added to KvDataStore, and now all tests are passed.Closes #434.