refactor(inverted_index): integrate puffin manager with sst indexer #4285
Conversation
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
This reverts commit 81ea1b6.
|
Note Reviews pausedUse the following commands to manage reviews:
WalkthroughThe update introduces a new configuration parameter Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
There was a problem hiding this comment.
Actionable comments posted: 3
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (32)
- config/config.md (2 hunks)
- config/datanode.example.toml (1 hunks)
- config/standalone.example.toml (1 hunks)
- src/mito2/src/access_layer.rs (3 hunks)
- src/mito2/src/cache/write_cache.rs (11 hunks)
- src/mito2/src/compaction/compactor.rs (5 hunks)
- src/mito2/src/config.rs (2 hunks)
- src/mito2/src/engine/basic_test.rs (1 hunks)
- src/mito2/src/error.rs (2 hunks)
- src/mito2/src/flush.rs (3 hunks)
- src/mito2/src/read/scan_region.rs (1 hunks)
- src/mito2/src/sst/index.rs (5 hunks)
- src/mito2/src/sst/index/indexer.rs (1 hunks)
- src/mito2/src/sst/index/indexer/abort.rs (1 hunks)
- src/mito2/src/sst/index/indexer/finish.rs (1 hunks)
- src/mito2/src/sst/index/indexer/update.rs (1 hunks)
- src/mito2/src/sst/index/inverted_index.rs (1 hunks)
- src/mito2/src/sst/index/inverted_index/applier.rs (1 hunks)
- src/mito2/src/sst/index/inverted_index/applier/builder.rs (1 hunks)
- src/mito2/src/sst/index/inverted_index/applier/builder/between.rs (2 hunks)
- src/mito2/src/sst/index/inverted_index/applier/builder/comparison.rs (2 hunks)
- src/mito2/src/sst/index/inverted_index/applier/builder/eq_list.rs (2 hunks)
- src/mito2/src/sst/index/inverted_index/applier/builder/in_list.rs (2 hunks)
- src/mito2/src/sst/index/inverted_index/applier/builder/regex_match.rs (2 hunks)
- src/mito2/src/sst/index/inverted_index/creator.rs (11 hunks)
- src/mito2/src/sst/index/puffin_manager.rs (2 hunks)
- src/mito2/src/sst/parquet.rs (2 hunks)
- src/mito2/src/sst/parquet/reader.rs (1 hunks)
- src/mito2/src/sst/parquet/writer.rs (2 hunks)
- src/mito2/src/test_util.rs (2 hunks)
- src/mito2/src/worker.rs (4 hunks)
- tests-integration/tests/http.rs (1 hunks)
Files skipped from review due to trivial changes (6)
- src/mito2/src/error.rs
- src/mito2/src/sst/index/indexer.rs
- src/mito2/src/sst/index/inverted_index/applier/builder/between.rs
- src/mito2/src/sst/index/inverted_index/applier/builder/comparison.rs
- src/mito2/src/sst/index/inverted_index/applier/builder/in_list.rs
- src/mito2/src/sst/index/inverted_index/applier/builder/regex_match.rs
Additional comments not posted (72)
src/mito2/src/sst/index/inverted_index.rs (2)
1-13: File header is well-defined.The file header includes the copyright notice and license information, which is correctly formatted and necessary for compliance.
15-19: Modules and constants are appropriately defined.The new modules (
applier,creator, andcodec) and the constantINDEX_BLOB_TYPEare clearly defined and appropriately named. Ensure that these modules are correctly implemented and integrated.src/mito2/src/sst/index/indexer/update.rs (3)
1-13: File header is well-defined.The file header includes the copyright notice and license information, which is correctly formatted and necessary for compliance.
15-29: Methoddo_updateis well-structured.The method
do_updatecorrectly handles empty batches and calls the appropriate update and abort methods. Ensure that thedo_update_inverted_indexanddo_abortmethods are correctly implemented.
31-54: Methoddo_update_inverted_indexhandles errors appropriately.The method
do_update_inverted_indexcorrectly handles the absence of an inverted indexer and errors during the update process. The use ofcfg!(any(test, feature = "test"))for conditional panic is a good practice for testing. Ensure that thecreator.updatemethod is correctly implemented.src/mito2/src/sst/index/indexer/abort.rs (4)
1-13: File header is well-defined.The file header includes the copyright notice and license information, which is correctly formatted and necessary for compliance.
15-24: Methoddo_abortis well-structured.The method
do_abortcorrectly calls the appropriate abort methods for the inverted index and puffin writer. Ensure that thedo_abort_inverted_indexanddo_abort_puffin_writermethods are correctly implemented.
26-45: Methoddo_abort_inverted_indexhandles errors appropriately.The method
do_abort_inverted_indexcorrectly handles the absence of an inverted indexer and errors during the abort process. The use ofcfg!(any(test, feature = "test"))for conditional panic is a good practice for testing. Ensure that theindexer.abortmethod is correctly implemented.
47-68: Methoddo_abort_puffin_writerhandles errors appropriately.The method
do_abort_puffin_writercorrectly handles the absence of a puffin writer and errors during the abort process. The use ofcfg!(any(test, feature = "test"))for conditional panic is a good practice for testing. Ensure that thepuffin_writer.finishmethod is correctly implemented.src/mito2/src/sst/index/indexer/finish.rs (5)
1-13: File header is well-defined.The file header includes the copyright notice and license information, which is correctly formatted and necessary for compliance.
15-41: Methoddo_finishis well-structured.The method
do_finishcorrectly handles the finishing of operations, including taking the puffin writer, updating the inverted index, and handling errors. Ensure that thedo_finish_inverted_indexanddo_finish_puffin_writermethods are correctly implemented.
43-62: Methoddo_finish_puffin_writerhandles errors appropriately.The method
do_finish_puffin_writercorrectly handles errors during the finish process. The use ofcfg!(any(test, feature = "test"))for conditional panic is a good practice for testing. Ensure that thepuffin_writer.finishmethod is correctly implemented.
64-100: Methoddo_finish_inverted_indexhandles errors appropriately.The method
do_finish_inverted_indexcorrectly handles the absence of an inverted indexer and errors during the finish process. The use ofcfg!(any(test, feature = "test"))for conditional panic is a good practice for testing. Ensure that theindexer.finishmethod is correctly implemented.
102-119: Methodfill_inverted_index_outputis well-structured.The method
fill_inverted_index_outputcorrectly fills the output with the relevant information about the inverted index. Ensure that thecolumn_idsmethod of theInvertedIndexeris correctly implemented.src/mito2/src/sst/index/puffin_manager.rs (3)
25-25: Imports look good.The imports for
BlobGuardandPuffinManagerare necessary for the Puffin Manager integration.
45-45: Type alias definition looks good.The type alias
SstPuffinWriteris correctly defined and necessary for the Puffin Manager integration.
Line range hint
147-157:
IndexerBuilder initialization looks good.The initialization of
IndexerBuilderincludes all required fields for the integration of the Puffin Manager with the SST Indexer.src/mito2/src/access_layer.rs (4)
25-25: Import looks good.The import of
InvertedIndexConfigis necessary for the changes made to theSstWriteRequeststruct.
145-157: IndexerBuilder initialization looks good.The initialization of
IndexerBuilderincludes all required fields for the integration of the Puffin Manager with the SST Indexer.
187-193: Enum definition looks good.The
OperationTypeenum is necessary to represent the origin of theSstWriteRequest.
196-206: Struct modification looks good.The modifications to the
SstWriteRequeststruct, includingop_typeandinverted_index_config, are necessary for the integration of the Puffin Manager with the SST Indexer.src/mito2/src/sst/index/inverted_index/applier.rs (2)
33-33: Import looks good.The import of
INDEX_BLOB_TYPEfrominverted_indexis necessary for the changes made to theSstIndexApplierstruct.
33-33: Imports look good.The imports of
BlobReaderandPuffinManagerFactoryare necessary for the changes made to theSstIndexApplierstruct.src/mito2/src/sst/parquet/writer.rs (2)
137-137: Method modification looks good.The modification to the
write_allmethod, including the call toself.indexer.finish(), is necessary for the integration of the Puffin Manager with the SST Indexer.
166-166: Method modification looks good.The modification to the
write_allmethod, includingindex_metadatain theSstInfostruct, is necessary for the integration of the Puffin Manager with the SST Indexer.src/mito2/src/sst/index/inverted_index/applier/builder/eq_list.rs (1)
23-23: Import Path UpdateThe import path for
SstIndexApplierBuilderhas been updated. Ensure that the new path is correct and consistent with the updated module structure.src/mito2/src/sst/index/inverted_index/applier/builder.rs (2)
39-40: Import Path UpdateThe import paths for
SstIndexApplierandIndexValueCodechave been updated. Ensure that the new paths are correct and consistent with the updated module structure.
Line range hint
237-237:
Import Path UpdateThe import path for
PuffinManagerFactoryhas been added. Ensure that this addition is correct and consistent with the rest of the codebase.src/mito2/src/sst/index.rs (14)
15-19: Module Imports UpdateThe module imports have been updated to include
indexer,intermediate,inverted_index,puffin_manager, andstatistics. Ensure that these updates are correct and consistent with the updated module structure.
25-26: Import Path UpdateThe import paths for
PuffinManager,SstPuffinManager, andSstPuffinWriterhave been added. Ensure that these additions are correct and consistent with the rest of the codebase.
38-38: Import Path UpdateThe import path for
SstIndexCreatorhas been updated toinverted_index::creator::SstIndexCreator as InvertedIndexer. Ensure that this update is correct and consistent with the rest of the codebase.
40-52: New Struct DefinitionsNew struct definitions for
IndexOutputandInvertedIndexOutputhave been added. Ensure that these definitions are correct and consistent with the rest of the codebase.
61-62: New Fields in Indexer StructNew fields
inverted_indexerandpuffin_writerhave been added to theIndexerstruct. Ensure that these additions are correct and consistent with the rest of the codebase.
66-94: New Methods in Indexer StructNew methods
update,finish,abort, andmemory_usagehave been added to theIndexerstruct. Ensure that these methods are correct and consistent with the rest of the codebase.
99-107: New Fields in IndexerBuilder StructNew fields
op_type,file_id,file_path,metadata,row_group_size,puffin_manager,intermediate_manager,index_options, andinverted_index_confighave been added to theIndexerBuilderstruct. Ensure that these additions are correct and consistent with the rest of the codebase.
111-134: New Method in IndexerBuilder StructA new method
buildhas been added to theIndexerBuilderstruct. Ensure that this method is correct and consistent with the rest of the codebase.
Line range hint
136-204:
New Method in IndexerBuilder StructA new method
build_inverted_indexerhas been added to theIndexerBuilderstruct. Ensure that this method is correct and consistent with the rest of the codebase.
206-224: New Method in IndexerBuilder StructA new method
build_puffin_writerhas been added to theIndexerBuilderstruct. Ensure that this method is correct and consistent with the rest of the codebase.
304-325: New Test for IndexerBuilderA new test
test_build_indexer_basichas been added for theIndexerBuilderstruct. Ensure that this test is correct and covers relevant scenarios.
328-352: New Test for IndexerBuilderA new test
test_build_indexer_disable_createhas been added for theIndexerBuilderstruct. Ensure that this test is correct and covers relevant scenarios.
355-376: New Test for IndexerBuilderA new test
test_build_indexer_no_taghas been added for theIndexerBuilderstruct. Ensure that this test is correct and covers relevant scenarios.
379-400: New Test for IndexerBuilderA new test
test_build_indexer_zero_row_grouphas been added for theIndexerBuilderstruct. Ensure that this test is correct and covers relevant scenarios.src/mito2/src/config.rs (2)
357-359: New Field in InvertedIndexConfig StructA new field
compresshas been added to theInvertedIndexConfigstruct. Ensure that this addition is correct and consistent with the rest of the codebase.
376-378: Default Value for Compress FieldThe default value for the
compressfield in theInvertedIndexConfigstruct has been set totrue. Ensure that this default value is appropriate and consistent with the intended behavior.src/mito2/src/cache/write_cache.rs (5)
48-49: Field Addition ApprovedThe addition of the
puffin_manager_factoryfield to theWriteCachestruct looks good.
Line range hint
64-73:
Method Update ApprovedThe integration of the
puffin_manager_factoryparameter in thenewmethod looks good.
Line range hint
84-95:
Method Update ApprovedThe integration of the
puffin_manager_factoryparameter in thenew_fsmethod looks good.
122-135: Method Update ApprovedThe usage of the
puffin_manager_factoryin thewrite_and_upload_sstmethod and the updates to theIndexerBuilderinitialization look good.
262-262: Import Addition ApprovedThe addition of the
OperationTypeimport in the test module looks good.config/datanode.example.toml (1)
434-436: Field Addition ApprovedThe addition of the
compressfield under theregion_engine.mito.inverted_indexsection looks good.config/standalone.example.toml (1)
457-459: Field Addition ApprovedThe addition of the
compressfield under theregion_engine.mito.inverted_indexsection looks good.src/mito2/src/compaction/compactor.rs (2)
27-27: Field Addition ApprovedThe addition of the
inverted_index_configfield to theSstWriteRequeststruct looks good.
Line range hint
277-318:
Method Update ApprovedThe usage of the
inverted_index_configin themerge_sstsmethod and the updates to theSstWriteRequestinitialization look good.src/mito2/src/sst/index/inverted_index/creator.rs (6)
404-404: Verify test coverage forcompressfield inSstIndexCreator.Ensure that the tests in this module cover the new
compressfield inSstIndexCreator.Verification successful
Verify test coverage for
compressfield inSstIndexCreator.The search results indicate that the
compressfield is indeed covered in the test files.
- Files with relevant test coverage:
tests-integration/tests/http.rstests-integration/src/cluster.rssrc/puffin/src/puffin_manager/tests.rssrc/mito2/src/manifest/tests/checkpoint.rsScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify test coverage for `compress` field in `SstIndexCreator`. # Test: Search for the usage of `compress` in tests. Expect: Proper test coverage for the new field. rg --type rust --test $'compress'Length of output: 463
Script:
#!/bin/bash # Description: Verify test coverage for `compress` field in `SstIndexCreator`. # Test: Search for the usage of `compress` in tests. Expect: Proper test coverage for the new field. rg --type rust --text 'compress'Length of output: 43672
249-251: Verify usage ofcompressfield inSstIndexCreator::finish.Ensure that the
SstIndexCreator::finishmethod correctly uses the newcompressfield inPutOptions. Verify that it is properly documented and tested.Verification successful
Verify usage of
compressfield inSstIndexCreator::finish.The
compressfield in theSstIndexCreator::finishmethod is correctly used and referenced in the codebase. ThePutOptionsstruct andcompressfield are also present in tests, ensuring proper testing.
src/mito2/src/sst/index/inverted_index/creator.rs: Definition and usage ofcompressinSstIndexCreator::finishsrc/puffin/src/puffin_manager/tests.rs: Tests involvingPutOptionsandcompressScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify usage of `compress` field in `SstIndexCreator::finish`. # Test: Search for the usage of `SstIndexCreator::finish`. Expect: Proper usage of the new field. rg --type rust $'SstIndexCreator::finish'Length of output: 40
Script:
#!/bin/bash # Description: Locate the definition of `SstIndexCreator` and its methods, including `finish`. # Search for the definition of `SstIndexCreator`. ast-grep --lang rust --pattern 'struct SstIndexCreator { $$$ }' # Search for the `finish` method within `SstIndexCreator`. ast-grep --lang rust --pattern 'impl SstIndexCreator { $$$ fn finish(&self) -> $_ { $$$ } $$$ }' # Search for references to `PutOptions` to verify usage of `compress`. rg --type rust 'PutOptions' # Search for references to `compress` within the repository. rg --type rust 'compress'Length of output: 46020
77-77: Verify the integration and usage ofcompressinSstIndexCreator.Ensure that the new field
compressis correctly integrated and used throughout the codebase. Verify that it is properly documented and tested.
147-149: Verify handling ofcompressfield inSstIndexCreator::update.Ensure that the
SstIndexCreator::updatemethod correctly handles the newcompressfield. Verify that it is properly documented and tested.
92-92: Verify the integration and usage ofcompressparameter inSstIndexCreator::new.Ensure that the new
compressparameter is correctly integrated and used throughout the codebase. Verify that it is properly documented and tested.
123-124: Verify the integration and usage ofcompressfield inSstIndexCreatorinitialization.Ensure that the new
compressfield is correctly integrated and used throughout the codebase. Verify that it is properly documented and tested.Verification successful
The
compressfield is correctly integrated and used in theSstIndexCreatorinitialization.
- The
compressfield is included in theSstIndexCreatorstruct.- The
compressparameter is correctly handled in thenewmethod ofSstIndexCreator.- The instantiation of
SstIndexCreatorcorrectly passes a boolean value for thecompressfield.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the integration and usage of `compress` field in `SstIndexCreator` initialization. # Test: Search for the usage of `SstIndexCreator` initialization. Expect: Proper usage of the new field. rg --type rust $'SstIndexCreator'Length of output: 610
Script:
#!/bin/bash # Description: Verify the integration and usage of `compress` field in `SstIndexCreator` initialization. # Retrieve the definition of SstIndexCreator to confirm the presence of the compress field. ast-grep --lang rust --pattern $'struct SstIndexCreator { compress: $_, $$$ }' # Retrieve the implementation of the constructor to see how compress is initialized. ast-grep --lang rust --pattern $'impl SstIndexCreator { pub fn new($$$) -> Self { $$$ } }' --debug # Search for instances where SstIndexCreator is instantiated to verify the use of compress. rg --type rust $'SstIndexCreator::new'Length of output: 545
Script:
#!/bin/bash # Description: Verify the integration and usage of `compress` field in `SstIndexCreator` initialization. # Retrieve the definition of SstIndexCreator to confirm the presence of the compress field. ast-grep --lang rust --pattern $'struct SstIndexCreator { compress: $_, $$$ }' # Retrieve the implementation of the constructor to see how compress is initialized. ast-grep --lang rust --pattern $'impl SstIndexCreator { pub fn new($$$) -> Self { $$$ } }' # Retrieve the specific instance of SstIndexCreator::new in creator.rs to verify the use of compress. rg --type rust --context 5 $'SstIndexCreator::new' src/mito2/src/sst/index/inverted_index/creator.rsLength of output: 664
Script:
#!/bin/bash # Description: Verify the integration and usage of `compress` field in `SstIndexCreator` initialization. # Retrieve the full implementation of the new method to verify the initialization of the compress field. ast-grep --lang rust --pattern $'impl SstIndexCreator { pub fn new($_, $_, $_, $_, $_) -> Self { $$$ } }' # Search for other instances of SstIndexCreator instantiation to ensure the compress field is correctly handled. rg --type rust $'SstIndexCreator::new'Length of output: 250
Script:
#!/bin/bash # Description: Verify the integration and usage of `compress` field in `SstIndexCreator` initialization. # Manually search for the new method in creator.rs to inspect its implementation. rg --type rust --context 10 $'pub fn new' src/mito2/src/sst/index/inverted_index/creator.rs # Retrieve more context around the current instance of SstIndexCreator::new in creator.rs to verify the use of compress. rg --type rust --context 10 $'SstIndexCreator::new' src/mito2/src/sst/index/inverted_index/creator.rsLength of output: 1742
src/mito2/src/engine/basic_test.rs (1)
583-583: Verify the correctness ofregion_stat.sst_usageassertion value.Ensure that the new assertion value
3026is correct and reflects the expected behavior.src/mito2/src/read/scan_region.rs (2)
48-49: New imports for SstIndexApplierBuilder and SstIndexApplierRef.The added imports are necessary for the integration of the Puffin Manager with the SST indexer.
Line range hint
144-159:
Ensure correct integration of Puffin Manager with SST indexer.The
build_index_appliermethod correctly integrates the Puffin Manager usingSstIndexApplierBuilder. It handles errors by logging a warning and returningNoneif the index applier cannot be built.tests-integration/tests/http.rs (1)
Line range hint
1-1:
No changes to review.This file does not contain any annotated changes.
src/mito2/src/flush.rs (2)
327-327: EnsureOperationType::Flushis appropriate.The operation type is set to
Flush. Verify if this is the correct operation type for this context.
334-334: LGTM!The addition of
inverted_index_configto theSstWriteRequeststruct aligns with the refactor objective.src/mito2/src/sst/parquet/reader.rs (1)
55-55: Verify the usage ofSstIndexApplierRef.The import statement
use crate::sst::index::inverted_index::applier::SstIndexApplierRef;was added. Ensure that this import is necessary and used within the file.Verification successful
Verified: The import statement is necessary and used within the file.
The import statement
use crate::sst::index::inverted_index::applier::SstIndexApplierRef;is used within the filesrc/mito2/src/sst/parquet/reader.rsin both a struct field and a method signature.
index_applier: Option<SstIndexApplierRef>pub(crate) fn index_applier(mut self, index_applier: Option<SstIndexApplierRef>) -> SelfScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the usage of `SstIndexApplierRef` in the file. # Test: Search for the usage of `SstIndexApplierRef`. Expect: At least one occurrence. rg --type rust 'SstIndexApplierRef' src/mito2/src/sst/parquet/reader.rsLength of output: 281
src/mito2/src/worker.rs (3)
Line range hint
52-52:
Verify the usage ofPuffinManagerFactory.The import statement
use crate::sst::index::puffin_manager::PuffinManagerFactory;was added. Ensure that this import is necessary and used within the file.Verification successful
The usage of
PuffinManagerFactoryis verified.The import statement
use crate::sst::index::puffin_manager::PuffinManagerFactory;is necessary and correctly used within the filesrc/mito2/src/worker.rs.
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the usage of `PuffinManagerFactory` in the file. # Test: Search for the usage of `PuffinManagerFactory`. Expect: At least one occurrence. rg --type rust 'PuffinManagerFactory' src/mito2/src/worker.rsLength of output: 400
152-152: Verify the integration ofpuffin_manager_factoryinwrite_cache_from_config.The
puffin_manager_factoryhas been added to thewrite_cache_from_configcall. Ensure that this integration is correct and necessary.Verification successful
The integration of
puffin_manager_factoryinwrite_cache_from_configis correct.The
puffin_manager_factoryis appropriately passed as an argument in the calls towrite_cache_from_config.
src/mito2/src/worker.rs:
- Line 152:
puffin_manager_factory.clone()Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the integration of `puffin_manager_factory` in the `write_cache_from_config` call. # Test: Search for the usage of `write_cache_from_config`. Expect: At least one occurrence with `puffin_manager_factory`. rg --type rust 'write_cache_from_config' src/mito2/src/worker.rsLength of output: 200
Script:
#!/bin/bash # Description: Check the arguments passed to `write_cache_from_config` in `src/mito2/src/worker.rs`. # Extract lines containing `write_cache_from_config` and the surrounding lines to examine the arguments. rg --type rust --context 3 'write_cache_from_config' src/mito2/src/worker.rsLength of output: 1035
284-284: Verify the integration ofpuffin_manager_factoryandintermediate_managerinWorkerStarter.The
puffin_manager_factoryandintermediate_managerhave been added to theWorkerStarterstruct. Ensure that this integration is correct and necessary.Verification successful
Verification successful: Integration of
puffin_manager_factoryandintermediate_managerinWorkerStarteris correct and necessary.The
puffin_manager_factoryandintermediate_managerhave been correctly integrated and are actively used within theWorkerStarterstruct.
- Instances of
puffin_manager_factoryinitialization and usage have been found.- Instances of
intermediate_managerinitialization and usage have been found.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the integration of `puffin_manager_factory` and `intermediate_manager` in the `WorkerStarter` struct. # Test: Search for the usage of `WorkerStarter`. Expect: At least one occurrence with `puffin_manager_factory` and `intermediate_manager`. rg --type rust 'WorkerStarter' src/mito2/src/worker.rsLength of output: 181
Script:
#!/bin/bash # Description: Verify the initialization and usage of `puffin_manager_factory` and `intermediate_manager` in `WorkerStarter`. # Test: Search for `puffin_manager_factory` and `intermediate_manager` within `src/mito2/src/worker.rs`. rg --type rust 'puffin_manager_factory' src/mito2/src/worker.rs rg --type rust 'intermediate_manager' src/mito2/src/worker.rsLength of output: 1376
src/mito2/src/test_util.rs (2)
72-72: Import looks good!The import of
PuffinManagerFactoryis consistent with the PR objective of integrating the Puffin Manager with the SST Indexer.
608-626: Changes increate_write_cachelook good!The initialization of
PuffinManagerFactoryandIntermediateManagerand their inclusion inWriteCache::newalign with the PR objective of integrating the Puffin Manager with the SST Indexer. The code is correct and follows best practices.config/config.md (1)
414-414: The new configuration parameter is well-documented.The addition of
region_engine.mito.inverted_index.compressis clear and consistent with the existing documentation.
Signed-off-by: Zhenchi <[email protected]>
56c3ba4
to
afb3786
Compare
Signed-off-by: Zhenchi <[email protected]>
|
@coderabbitai pause |
Actions performedReviews paused. |
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4285 +/- ##
==========================================
+ Coverage 84.96% 85.02% +0.05%
==========================================
Files 1061 1066 +5
Lines 188124 188473 +349
==========================================
+ Hits 159845 160251 +406
+ Misses 28279 28222 -57 |
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
#4246
What's changed and what's your intention?
...to make room for the addition of full-text indexer
Checklist
Summary by CodeRabbit
New Features
Deprecations
intermediate_pathsetting in favor ofregion_engine.mito.index.aux_path.Refactor
Bug Fixes
Documentation