Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test suite: arith.json, boolean.json, and mktx.json seem useless, perhaps worth skipping #89

Closed
tersec opened this issue Jul 30, 2018 · 0 comments

Comments

@tersec
Copy link
Contributor

tersec commented Jul 30, 2018

- arith.json                                                      Fail
- boolean.json                                                    Fail
- mktx.json                                                       Fail

The failures are due, essentially, to CALL being stubbed enough not to notice that it's supposed to fail.

All lack a post/log/etc section, and as such according to http:https://ethereum-tests.readthedocs.io/en/latest/test_types/vm_tests.html expect an exception. That'd be fine, but even had CALL not been partly stubbed out, the exception is uninformative in each case, and purely a testing artifact; as:

Since these tests are meant only as a basic test of VM operation, the CALL and CREATE instructions are not actually executed. To provide the possibility of testing to guarantee they were actually run at all, a separate portion callcreates details each CALL or CREATE operation in the order they would have been executed.

They all, further, lack callcreates, yet have a CALL instruction (and no flow control avoiding it), so are expected to experience exceptional halting the exact same way, regardless of nominally testing arithmetic, boolean operations, or transaction creation. They add nothing I can see to Nimbus's development or testing in a meaningful way.

Finally, they were removed from the official test suite 3 months ago: ethereum/tests@4faa8c5.

I don't see why they shouldn't all be skipped.

For reference, mktx.json runs 0x60006000600060006706f05b59d3b200003360c85a03f1, or:

[1] PUSH1 0x00
[3] PUSH1 0x00
[5] PUSH1 0x00
[7] PUSH1 0x00
[16] PUSH8 0x06f05b59d3b20000
[17] CALLER
[19] PUSH1 0xc8
[20] GAS
[21] SUB
[22] CALL

I won't quite the entire disassembly of boolean.json and arith.json's code, but they're 0x600160011615601a57600060006000600060023360c85a03f1505b600060011615603557600060006000600060033360c85a03f1505b600160001615605057600060006000600060043360c85a03f1505b600060001615606b57600060006000600060053360c85a03f1505b6001600117156086576000600060006000600c3360c85a03f1505b60006001171560a1576000600060006000600d3360c85a03f1505b60016000171560bc576000600060006000600e3360c85a03f1505b60006000171560d7576000600060006000600f3360c85a03f1505b and 0x600060006000600060026002600803036002600306600260020460046004600402026002600201010101013360c85a03f1 for https://etherscan.io/opcode-tool use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants