Having an overloaded method and then changing a class hierarchy not detected

[marek-parfianowicz]

Chenguang Zhu created an issue 2018-01-30
Summary: Consder the following simple project. There are three classes, A, C (which extends A), and T (which is a test class). Class A has method m(int), class C has method m(long), and test invokes m with int value. If we run the test with clover, it passes. If we then remove "extends A" from C and run the test with clover again, clover does not detect the actual invoked method changes, so it does not select the test to run, this is a safety issue.

Steps to Reproduce: Run the bash script ./s provided in this directory. The script (1) runs mvn test with clover, (2) removes 'extends A' from class C, (3) runs mvn clean, and (4) runs mvn test with clover again.

Expected Results: p.CTest runs both times (it passes in the first run and fails in the second run).

Actual Results: p.CTest runs only the first time (and does not run test the second time).

[marek-parfianowicz]

Hi Chenguang, thank you for reporting this test case. From what I see the problem lies in the test optimization, not in instrumentation itself (i.e. without the 'optimize' goal all tests are executed and the build fails).

When I ran with 'mvn -X' the snapshot shows:

All files: p/A.java Hitting tests: p.CTest.test p/CTest.java Hitting tests: p.CTest.test

so the C.java is unknown to CTest - it's because first version of the test called a method from A.java, not C.java - therefore modification of C.java and change of inheritance hierarchy is considered as unrelated to CTest class.

The workaround is to delete test optimization snapshot.

[marek-parfianowicz]

Original issue: https://bitbucket.org/openclover/clover/issues/58