feat: active pbss #1881
Merged
feat: active pbss #1881
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
buddh0
reviewed
Sep 19, 2023
cmd/geth/snapshot.go
Outdated
| @@ -541,6 +545,8 @@ func traverseState(ctx *cli.Context) error { | |||
| defer stack.Close() | |||
|
|
|||
| chaindb := utils.MakeChainDatabase(ctx, stack, true, false) | |||
| triedb := trie.NewDatabase(chaindb, nil) | |||
There was a problem hiding this comment.
defer chaindb.Close() before new triedb
buddh0
reviewed
Sep 19, 2023
fynnss
reviewed
Sep 20, 2023
fynnss
reviewed
Sep 20, 2023
buddh0
reviewed
Sep 20, 2023
| log.Warn("The flag --txlookuplimit is deprecated and will be removed, please use --history.transactions") | ||
| cfg.TransactionHistory = ctx.Uint64(TxLookupLimitFlag.Name) | ||
| } | ||
| if ctx.String(GCModeFlag.Name) == "archive" && cfg.TransactionHistory != 0 { |
There was a problem hiding this comment.
if ctx.String(GCModeFlag.Name) == "archive" && ctx.Uint64(TxLookupLimitFlag.Name) != 0 {
ctx.Set(TxLookupLimitFlag.Name, "0")
log.Warn("Disable transaction unindexing for archive node")
}
if ctx.IsSet(LightServeFlag.Name) && ctx.Uint64(TxLookupLimitFlag.Name) != 0 {
log.Warn("LES server cannot serve old transaction status and cannot connect below les/4 protocol version if transaction lookup index is limited")
}
then should these lines need to be deleted?
There was a problem hiding this comment.
the latest commit has deleted this part.
buddh0
reviewed
Sep 20, 2023
joeylichang
force-pushed
the
pbss_active
branch
from
d8546c0
to
f9abd31
Compare
fynnss
reviewed
Sep 20, 2023
joeylichang
force-pushed
the
pbss_active
branch
from
381dda6
to
bff171f
Compare
* all: activate pbss * core/rawdb: fix compilation error * cma, core, eth, les, trie: address comments * cmd, core, eth, trie: polish code * core, cmd, eth: address comments * cmd, core, eth, les, light, tests: address comment * cmd/utils: shorten log message * trie/triedb/pathdb: limit node buffer size to 1gb * cmd/utils: fix opening non-existing db * cmd/utils: rename flag name * cmd, core: group chain history flags and fix tests * core, eth, trie: fix memory leak in snapshot generation * cmd, eth, internal: deprecate flags * all: enable state tests for pathdb, fixes * cmd, core: polish code * trie/triedb/pathdb: limit the node buffer size to 256mb --------- Co-authored-by: Martin Holst Swende <[email protected]> Co-authored-by: Péter Szilágyi <[email protected]>
joeylichang
force-pushed
the
pbss_active
branch
from
bff171f
to
ad0d995
Compare
joeylichang
force-pushed
the
pbss_active
branch
from
e1ff74f
to
4145f32
Compare
fix: lint error fix: ut error fix: code review comments
joeylichang
force-pushed
the
pbss_active
branch
from
4145f32
to
e4b69f7
Compare
brilliant-lx
approved these changes
Sep 26, 2023
fynnss
approved these changes
Sep 26, 2023
This was referenced Oct 18, 2023
Merged
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
ETH proposed Path-based state scheme (PBSS) and released it in v1.1.2. PBSS has obvious advantages in inline prune, disk IO, and disk capacity, it can improve the performance of the import block and reduce disk costs. This PR is to activate PBSS on BSC. PBSS design can be found in Ethereum Path-Based Storage Model and Newly Inline State Prune.
Rationale
Performance
Prior to this PR, our team did a quick performance test based on the released BSC v1.2.11. Following is the 13 days comparison of BSC v1.2.11 + PBSS node(172.22.43.76) and a BSC v1.2.11 node(172.22.42.157)
Hardware: 16core, 64G, 3000 IOPS 125M/S SSD.
Head Blcok
BSC v1.2.11 + PBSS catches up to 1200w+ block.
BSC v1.2.11 catches up to 730w+ block.
Import Block Cost
Chain validation
Chain execution
Chain commit
ETHDB size
BSC v1.2.11 + PBSS node no trie nodes history, only store the latest world state.
Example
--state.scheme: Scheme to use for storing ethereum state ('hash' or 'path')", default is hash.--history.state: Number of recent blocks to retain state history for (default = 90,000 blocks, 0 = entire chain).--history.transactions: Number of recent blocks to maintain transactions index for (default = about one year, 0 = entire chain),Changes
Notable changes: