Keep track of obsolete blob files in VersionSet #6755
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
@ltamasi has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
… the obsolete list
…was ineffective otherwise) and add one more to make sure a file is only returned once
2b2f11c
to
f0a08ea
Compare
|
@ltamasi has updated the pull request. Re-import the pull request |
There was a problem hiding this comment.
@ltamasi has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
riversand963
approved these changes
Apr 30, 2020
|
@ltamasi has updated the pull request. Re-import the pull request |
There was a problem hiding this comment.
@ltamasi has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
pushed a commit
that referenced
this pull request
May 4, 2020
Summary: The patch adds logic that returns the set of live blob files from `Version::AddLiveFiles` and `VersionSet::AddLiveFiles` (in addition to live table files), and also cleans up the code a bit, for example, by exposing only the numbers of table files as opposed to the earlier `FileDescriptor`s that no clients used. Moreover, the patch extends the `GetLiveFiles` API so that it also exposes blob files in the current version. Similarly to #6755, this is a building block for identifying and purging obsolete blob files. Pull Request resolved: #6785 Test Plan: `make check` Reviewed By: riversand963 Differential Revision: D21336210 Pulled By: ltamasi fbshipit-source-id: fc1aede8a49eacd03caafbc5f6f9ce43b6270821
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
The patch adds logic to keep track of obsolete blob files. A blob file becomes
obsolete when the last
shared_ptrthat points to the correspondingSharedBlobFileMetaDataobject goes away, which, in turn, happens when thelast
Versionthat contains the blob file is destroyed. No longer needed blobfiles are added to the obsolete list in
VersionSetusing a custom deleter toavoid unnecessary coupling between
SharedBlobFileMetaDataandVersionSet.Obsolete blob files are returned by
VersionSet::GetObsoleteFilesand storedin
JobContext.Test Plan:
make check