-
Notifications
You must be signed in to change notification settings - Fork 107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tested eth/65 support #772
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What was the criteria used to decide which log messages were commented out?
EDIT: Nevermind, this was explained here:
46d022f
This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <[email protected]>
Turns out `{.rlpInline.}` doesn't do anything. It's documented but not implemented. Due to this, whenever a peer sent us a `NewBlock` message, we had an RLP decoding error processing it, and disconnected the peer thinking it was the peer's error. These messages are sent often by good peers, so whenever we connected to a really good peer, we'd end up disconnecting from it due to this. Because a block body is a list of transactions, the parse errors looked suspiciously like EIP-2718/2976/2930/1559 typed transaction RLP errors. But it was a failure to parse `BlockBody` inline. Conveniently, the `EthBlock` type defined for another reason is encoded exactly the way `NewBlockAnnounce` needs to be, so we can reuse that type. This didn't stand out before updating to `eth/65`, because with old protocols we tend to only connect to old peers, which may be out of date themselves and have no new blocks to send. Also, we didn't really investigate occasional disconnects before, we assumed they're just part of P2P life. Signed-off-by: Jamie Lokier <[email protected]>
Signed-off-by: Jamie Lokier <[email protected]>
Use a hard-coded number instead of the same expression as the client, so that bugs introduced via that expression are detected. Using the same expression as the client can hide issues when the value is wrong in both places. When the expected value genuinely changes, it'll be obvious. Just change this number. Signed-off-by: Jamie Lokier <[email protected]>
Use a hard-coded number instead of the same expression as the client, so that bugs introduced via that expression are detected. Using the same expression as the client can hide issues when the value is wrong in both places. When the expected value genuinely changes, it'll be obvious. Just change this number. Signed-off-by: Jamie Lokier <[email protected]>
While looking at the RPC `eth_protocolVersion` to see exactly what it should report when there are multiple `eth/NN` versions supported by a node, I found some differences in the documentation: - The old Ethereum wiki documents it as returning a decimal string. - But Infura documents it as returning 0x-prefixed hex string. - Geth 1.10.0 has removed this call entirely "as it makes no sense". https://eth.wiki/json-rpc/API#eth_protocolversion https://infura.io/docs/ethereum/json-rpc/eth-protocolVersion https://blog.ethereum.org/2021/03/03/geth-v1-10-0/#compatibility Signed-off-by: Jamie Lokier <[email protected]>
In RPC `eth_protocolVersion`, look at the live `EthereumNode` to find which version of `eth/NN` protocol is active, instead of trusting a compile-time constant. It's better to check dynamically. GraphQL already does this. As a result, the RPC code doesn't depend on `eth_protocol` any more. To make sure there are no more accidental users of the old constant, `protocolVersion` is no longer exported from `protocol_eth65`. (The simplest way to support `eth/65` was to make `eth_protocolVersion` use `protocol_eth65.protocolVersion`, to get 65. But that's silly. More seriously, when we add another version (`eth/66`) running alongside `eth/65`, that expression would still compile ok yet return the wrong value, while still passing the RPC test suite.) Signed-off-by: Jamie Lokier <[email protected]>
This constant shouldn't be used outside `protocol_eth65`. When we support multiple `eth/NN` versions side by side, or even just have multiple code files, there's a risk some code would import just one of the files (e.g. `protocol_eth65`), use `protocolVersion`, and incorrectly act as though that version is the one active on the node. In fact that happened, and now it can't happen. Other code needs to query the `EtheruemNode` to find what versions are really active. Signed-off-by: Jamie Lokier <[email protected]>
Move `blockchain_sync.nim` from `nim-eth` to `nimbus-eth1`. This lets `blockchain_sync` use the `eth/65` protocol to synchronise with more modern peers than before. Practically, the effect is the sync process runs more quickly and reliably than before. It finds usable peers, and they are up to date. Note, this is mostly old code, and it mostly performs "classic sync", the original Ethereum method. Here's a summary of this code: - It decides on a blockchain canonical head by sampling a few peers. - Starting from block 0 (genesis), it downloads each block header and block, mostly in order. - After it downloads each block, it executes the EVM transactions in that block and updates state trie from that, before going to the next block. - This way the database state is updated by EVM executions in block order, and new state is persisted to the trie database after each block. Even though it mentions Geth "fast sync" (comments near end of file), and has some elements, it isn't really. The most obvious missing part is this code _doesn't download a state trie_, it calculates all state from block 0. Geth "fast sync" has several parts: 1. Find an agreed common chain among several peers to treat as probably secure, and a sufficiently long suffix to provide "statistical economic consensus" when it is validated. 2. Perform a subset of PoW calculations, skipping forward over a segment to verify some of the PoWs according to a pattern in the relevant paper. 3. Download the state trie from the block at the start of that last segment. 4. Execute only the blocks/transactions in that last segment, using the downloaded state trie, to fill out the later states and properly validate the blocks in the last segment. Some other issues with `blockchain_sync` code: - If it ever reaches the head of the chain, it doesn't follow new blocks with increasing block numbers, at least not rapidly. - If the chain undergoes a reorg, this code won't fetch a block number it has already fetched, so it can't accept the reorg. It will end up conflicted with peers. This hasn't mattered because the development focus has been on the bulk of the catching up process, not the real-time head and reorgs. - So it probably doesn't work correctly when it gets close to the head due to many small reorgs, though it might for subtle reasons. - Some of the network message handling isn't sufficiently robust, and it discards some replies that have valid data according to specification. - On rare occasions the initial query mapping block hash to number can fail (because the peer's state changes). - It makes some assumptions about the state of peers based on their responses which may not be valid (I'm not convinced they are). The method for working out "trusted" peers that agree a common chain prefix is clever. It compares peers by asking each peer if it has the header matching another peer's canonical head block by hash. But it's not clear that merely knowing about a block constitutes agreement about the canonical chain. (If it did, query by block number would give the same answer more authoritatively.) Nonetheless, being able to run this sync process on `eth/65` is useful. <# interactive rebase in progress; onto 66532e8 Signed-off-by: Jamie Lokier <[email protected]>
Disable some trace messages which appeared a lot in the output and probably aren't so useful any more, when block processing is functioning well at high speed. Turning on the trace level globally is useful to get a feel for what's happening, but only if each category is kept to a reasonable amount. As well as overwhelming the output so that it's hard to see general activity, some of these messages happen so much they severely slow down processing. Ones called every time an EVM opcode uses some gas are particularly extreme. These messages have all been chosen as things which are probably not useful any more (the relevant functionality has been debugged and is tested plenty). These have been commented out rather than removed. It may be that turning trace topics on/off, or other selection, is a better longer term solution, but that will require better command line options and good defaults for sure. (I think higher levels `tracev` and `tracevv` levels (extra verbose) would be more useful for this sort of deep tracing on request.) For now, enabling `--log-level:TRACE` on the command line is quite useful as long as we keep each category reasonable, and this patch tries to keep that balance. - Don't show "has transactions" on virtually every block imported. - Don't show "Sender" and "txHash" lines on every transaction processed. - Don't show "GAS CONSUMPTION" on every opcode executed", this is way too much. - Don't show "GAS RETURNED" and "GAS REFUND" on each contract call. - Don't show "op: Stop" on every Stop opcode, which means every transaction. - Don't show "Insufficient funds" whenever a contract can't call another. - Don't show "ECRecover", "SHA256 precompile", "RIPEMD160", "Identity" or even "Call precompile" every time a precompile is called. These are very well tested now. - Don't show "executeOpcodes error" whenever a contract returns an error. (This is changed to `trace` too, it's a normal event that is well tested.) Signed-off-by: Jamie Lokier <[email protected]>
The format is reasonably useful and not too large, when looking at the behaviour of sync processes. It doesn't try to show all the details of packets, just something at a useful level of detail to see what's going on. The consistent presentation has proven helpful too, e.g. when grepping. Signed-off-by: Jamie Lokier <[email protected]>
Using the same packet tracing format to match `protocol_eth65`. There aren't many calls, and this makes them clear. Signed-off-by: Jamie Lokier <[email protected]>
Signed-off-by: Jamie Lokier <[email protected]>
Sounds like there should be a corresponding eth/63 removal somewhere? also, we tend to squashmerge things because rolling back lots of small broken commits is messy |
I try to arrange multi-repo PRs to be decoupled (when that is easy), so that the interesting place (e.g. where the new feature or bugfix arrives) can be used on its own merits without being linked quasi-atomically with a PR in other library-like repos. In this case it means If the result is now unused code in the library-like repos, do the cleanup there after the dust has settled. Rather like updating well-maintained libraries and applications that use them. Especially when the other, library-like repos have multiple users, as I'll want to check the change in that doesn't break any of them. In this case I'm worried about knock on effects on Waku and any other repo I don't know about that's using RLPx, because removing |
This is an intentionally simple PR, designed to not break, change or depend on other functionality much, so that the "old sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of
sync algorithms), but it is one of the foundations.
For a while now Nimbus Eth1 only supported protocol
eth/63
. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it.The current specification is
eth/66
, and the functionality we really need is ineth/64
.So why
eth/65
?eth/64
is essential because of theforkId
feature. Buteth/64
is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talketh/64
any more.eth/66
is the current specification, but some clients don't talketh/66
yet. We'd like to have the best peer connectivity during tests, and currently everything that talkseth/66
also talkseth/65
.Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add
eth/64..eth/66
for greater peer reach and testing theeth/66
behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.)What are
eth/64
andeth/65
?eth/64
(EIP-2364) addedforkId
which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block.forkId
also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays.We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed.
eth/65
added more efficient transaction pool methods.Then a later version of
eth/65
(without a new number) added typed transactions, described in EIP-2976.Why it's moved to
nimbus-eth1
:eth/64
onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part ofnimbus-eth1
run-time state. These aren't available to purenim-eth
code. Although it would be possible to add an API to letnimbus-eth1
set these numbers, there isn't any point because the protocol would still only be useful tonimbus-eth1
.