Skip to content

Commit

Permalink
@testset for: avoid calling finish twice when it errors (#41715)
Browse files Browse the repository at this point in the history
Here is a MWE:
```julia
julia> using Test
       @testset "a" for i=1:2
           @test i != 1
       end
a: Test Failed at REPL[3]:3
  Expression: i != 1
   Evaluated: 1 != 1
Stacktrace:
[...]
Test Summary: | Fail  Total
a             |    1      1
Test Summary: | Fail  Total
a             |    1      1
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.

caused by: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.
```

The `finish` function is called twice, and for a toplevel testset, this means
throwing an error. This manifests in the above example by printing twice
the result of the testset (with "Test Summary"), and by having
"Error: ... caused by: ..." with the same message.

Co-authored-by: Kristoffer Carlsson <[email protected]>
  • Loading branch information
rfourquet and KristofferC committed May 17, 2022
1 parent 8a0a719 commit 4605704
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion stdlib/Test/src/Test.jl
Original file line number Diff line number Diff line change
Expand Up @@ -1425,7 +1425,10 @@ function testset_forloop(args, testloop, source)
# they can be handled properly by `finally` lowering.
if !first_iteration
pop_testset()
finish_errored = true
push!(arr, finish(ts))
finish_errored = false

# it's 1000 times faster to copy from tmprng rather than calling Random.seed!
copy!(RNG, tmprng)

Expand All @@ -1447,6 +1450,7 @@ function testset_forloop(args, testloop, source)
local arr = Vector{Any}()
local first_iteration = true
local ts
local finish_errored = false
local RNG = default_rng()
local oldrng = copy(RNG)
local oldseed = Random.GLOBAL_SEED
Expand All @@ -1458,7 +1462,7 @@ function testset_forloop(args, testloop, source)
end
finally
# Handle `return` in test body
if !first_iteration
if !first_iteration && !finish_errored
pop_testset()
push!(arr, finish(ts))
end
Expand Down

0 comments on commit 4605704

Please sign in to comment.