Identified first roadblock for hive integration

[jangko]

hive integration requires client to be able to init and import chain from file.

• init: initialize the database with JSON genesis.
• import: importing RLP encoded block and try to validate it. if the block is valid, write it into db.

[jangko]

a custom genesis file is something like this if we init the database using --init:
the same json format is used by geth and generated by hive simulator.

{
  "nonce": "0x0000000000000000",
  "timestamp": "0x00",
  "extraData": "0x00",
  "gasLimit": "0x02540be400",
  "difficulty": "0x020000",
  "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "coinbase": "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
  "alloc": {
    "0x1000000000000000000000000000000000000000": {
      "balance": "0x00",
      "code": "0x5a465a905090036002900360015500",
      "nonce": "0x00",
      "storage": {}
    },
    "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b": {
      "balance": "0x3635c9adc5dea00000",
      "code": "0x",
      "nonce": "0x00",
      "storage": {}
    }
  },
  "config": {
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip158Block": 0,
    "byzantiumBlock": 0,
    "constantinopleBlock": 0,
    "petersburgBlock": 0,
    "istanbulBlock": 0,
    "berlinBlock": 2000,
    "chainId": 1
  }
}

next time we start syncing with network or importing blocks using --import, we need to read back the chain configuration(chainId, homesteadBlock-berlinBlock). but where is the best place to put this configuration? (db or config file in db folder/directory)
And should we support multiple database with different chain configuration or just replace the db with new configuration?
ping @jlokier, @mjfh

[jlokier]

For --init, we already have the --customnetwork /path/to/genesis.json option added in 45cda8b.

For single db or replacement db, we already have the --datadir:relative/path, so we already support multiple databases easily.

(It's unfortunate one of those takes a relative path to $HOME and the other doesn't, but that's just an ugly wart we can fix separately from this issue.)

You can pass both options each time nimbus is run, so what's missing from that behaviour? Is it that you want to store the recently used --customnetwork and --datadir in a configuration file so that you don't have to give the command line arguments in subsequent calls?

[jlokier]

Selecting the test networks --ropsten, --rinkeby etc. doesn't get remembered in a config file. So it would be new and inconsistent behaviour to automatically store a "last used config" in a config file for the custom genesis, but not for the chosen test network.

In other words, if we did use a config file it should be for automating more of the command line options, not just the genesis block. Until that's done, I think stick with giving --customnetwork and --datadir on the command line when running inside the Hive test suite, or even just skip --datadir because it's inside a custom Docker container already.

Alternatively it could make sense for certain things to be read from environment variables for Hive. That's also advocated by 12-factor app, but I think environment variables can get out of hand sometimes.

[jlokier]

When test networks are selected, nimbus should probably use a different database automatically, so different invocations don't interfere.

For example different paths under --datadir, or table/key prefixes inside the shared database. If it did that, then it could also do the same thing when given a custom genesis, perhaps using the hash of the genesis config file to choose the path or the table/key prefix.

But at the moment it doesn't automatically use a different database for testnets (unless you give --datadir too), so I think keep custom genesis consistent with testnet selection in this regard.

[jangko]

thanks for your feedback.
after a closer look, geth genesis file not only contains genesis data but also chain configuration which can be put into separate file.
the combination that can work with hive is --customnetwork together with --import, while --init is the short-lived version --customnetwork

[jlokier]

I don't see --import in Nimbus. Is this issue now about implementing --import?

[jangko]

fixed by #566, closing