Unified mode for undumping gzip-ed or era1-ed encoded block dumps #2198
Conversation
ackn: Built on Daniel's work
| dir*: string # Database folder | ||
| blocks*: KeyedQueue[uint64,Era1File] # Era1 block on disk | ||
| byBlkNum*: SortedSet[uint64,Era1DbBlocks] # File access info | ||
| ranges*: IntervalSetRef[uint64,uint64] # Covered ranges |
There was a problem hiding this comment.
an era1 file is exactly 8192 blocks (except the lasts one), so this code is not needed - in fact, the era files do not need to be indexed at all on startup (this is very slow) - all that's needed is the list of roots in the accumulator used for verifying era1 consistency (see getAccumulator in fluffy code) - from there, the names of the files and the ranges of blocks contained can be computed) cc @daniellm
There was a problem hiding this comment.
Yeah, this implementation seems overly complex to me. When I was planning on implementing this, it looked like I could do so very similar as EraDB is implemented, but with usage of the Accumulator instead.
|
Somewhat unfortunate that the binaries were added to the main repo - we tend to keep large files out from the main repos to keep cloning times reasonable - instead, a git submodule with a dedicated large-files repo should have been used. |
ackn:
Built on Daniel's work