-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hive: Nimbus startup and shutdown delays dominate testing time with ethereum/consensus
#591
Comments
Not sure this is part of the slow startup but assuming that it is the general nimbus binary that is being used for these Hive tests, it would be worth verifying if it is started with the default nat setting, which is To avoid delays due to that, start nimbus with for example the option |
trying Jacek suggestion in #591, I added nat setting to nimbus-eth1 hive shell script. visible difference after adding this flag is the nat library not complaining about "the gateway does not support nat-pmp" anymore. but the slow startup time described in #591 is yet to be measured again, although I already see improvement when executing ethereum/consensus category in hive.
trying Jacek suggestion in #591, I added nat setting to nimbus-eth1 hive shell script. visible difference after adding this flag is the nat library not complaining about "the gateway does not support nat-pmp" anymore. but the slow startup time described in #591 is yet to be measured again, although I already see improvement when executing ethereum/consensus category in hive.
progress in this area: |
... and the time to run
ethereum/consensus
Hive tests is important to us. It's currently too slow.Hive is the new Ethereum test suite, which may either displace or complement retesteth. Hive consists of a few different "simulators" and test suites, and one of those suites is by far the largest and slowest,
ethereum/consensus
, with about 28,000 tests.When run on Geth, it proceeds on both my fastest machines at about 1 test per second, taking roughly 6-7 hours (but we can parallelise a bit to speed it up). This is the command line:
Geth is running only a small fraction of that ~1 second. Most of the time goes in test setup and Dockerish things. (It's worth optimising the Docker setup to improve that, e.g. Docker is slower inside an LXD container or an old kernel. We can add some multi-core parallelism with
--sim.parallelism
for a decent speedup, although there's more I/O and space used too.)When run on
nimbus-eth1
, it is much slower at 7-8 seconds per test. This is the command line(*):That's a lot slower than Geth, ballpark 15 times slower, and it means Nimbus time is the dominant factor. dominant. That's too slow. It won't even complete the test suite in 24 hours, unless you have enough cores to throw at it.
Using
--docker.output
we can see most of the time is spent in slow startup or shutdown ofnimbus-eth1
, probably waiting for networking functions, but it could be unlogged database actions. It isn't slow when importing a genesis block. Whatever makes #585 slow might have something to do with this, but startup looks slow too.(*) After setting up Hive for
nimbus-eth1
according tonimbus-eth1/hive_integration/readme.md
.The text was updated successfully, but these errors were encountered: