runtests: don't stop if exception is thrown with 1 worker

[rfourquet]

@amitmurthy in #13577 (comment), why did you decide to not run all the tests when there is an exception?
@kshyatt when you changed the line resp = e to resp = [e] in #18738, I assume leaving the test isa(resp, Exception) as-is was an oversight?

[yuyichao]

Having the test stops quickly makes debugging significantly easier.

[rfourquet]

Having the test stops quickly makes debugging significantly easier.

I don't disagree, but this isn't really the point here: this PR only changes the behavior of "runtests" when there is only 1 CPU to match that of when there are multi CPUs. In other words, most of us with Sys.CPU_CORES > 1 won't see any difference with this change. Stoping quickly could be the subject of another PR allowing to pass this option as a command-line parameter.

[yuyichao]

this PR only changes the behavior of "runtests" when there is only 1 CPU to match that of when there are multi CPU

Right, I know. And I'm 99.9% sure that when the multicore behavior was introduced it was regarded as a bug. The only way to workaround that bug was to run with one core which was at least why I didn't push hard on fixing it. (I'm pretty sure there was discussion on the github pr but can't really find it now.)

[rfourquet]

OK, I didn't know you could "run with one core" (EDIT: ok got it, JULIA_CPU_CORES). I made this change here to ease another PR, but I can look into implementing this "early-exit" option.

[yuyichao]

ok got it, JULIA_CPU_CORES

Yes. I think there's also sth like make test1 for it although I always just run the commands.

I can look into implementing this "early-exit" option.

That'll be nice. It's good enough as long as there's still a way to do it (without editting runtests.jl).

[rfourquet]

Note that because of the isa(resp, Exception) "bug" mentioned in the OP, it's already the case actually that even with 1 worker, the tests continue even when an exception is raised.