Speed up GPU tests by avoiding device reset

[upsj]

At the moment, all of our GPU tests store a device executor that gets destroyed after each test, which in turn executes cuda/hipDeviceReset. This causes each of our tests to take at least 800ms on NVIDIA GPUs (less on AMD GPUs), independent of the actual kernel runtime. I think this is not necessary and can be fixed by instantiating a second device executor in a global variable (other, cleaner solutions welcome!)

Somewhat related to #461

Before:

151: [==========] Running 24 tests from 1 test case.
151: [----------] Global test environment set-up.
151: [----------] 24 tests from Dense
151: [ RUN      ] Dense.SingleVectorHipScaleIsEquivalentToRef
151: [       OK ] Dense.SingleVectorHipScaleIsEquivalentToRef (1012 ms)
...
151: [ RUN      ] Dense.CalculateTotalColsIsEquivalentToRef
151: [       OK ] Dense.CalculateTotalColsIsEquivalentToRef (864 ms)
151: [----------] 24 tests from Dense (22293 ms total)

After:

151: [==========] Running 24 tests from 1 test case.
151: [----------] Global test environment set-up.
151: [----------] 24 tests from Dense
151: [ RUN      ] Dense.SingleVectorHipScaleIsEquivalentToRef
151: [       OK ] Dense.SingleVectorHipScaleIsEquivalentToRef (95 ms)
...
151: [ RUN      ] Dense.CalculateTotalColsIsEquivalentToRef
151: [       OK ] Dense.CalculateTotalColsIsEquivalentToRef (20 ms)
151: [----------] 24 tests from Dense (1783 ms total)

[thoasm]

What about creating a cuda/test/utils.hpp header (which includes core/test/utils.hpp), and do the creation of the executor there? That would at least hide the not so pleasant code inside a header and would allow us to improve / remove that in one line instead of in every test-file.
This would obviously require us changing all includes of core/test/utils.hpp to cuda/test/utils.hpp, but that is easy to automate (only needs to be done once).

[upsj]

@thoasm Good idea, I followed your suggestion and only had to patch a small compilation issue with assert and the comma operator in constexpr contexts for HIP.

[yhmtsai]

Does return GKO_ASSERT(...), value; -> GKO_ASSERT(...); return value; not work?
Or it can not be constexpr?

[upsj]

In C++11, constexpr functions are only allowed to consist of a single return statement, so that unfortunately doesn't work. The expected behavior for NDEBUG would be that assert is replaced by something like void(0), but this weird combination of includes seems to let some asserts vanish. I did not investigate this further yet, since the workaround works.

[yhmtsai]

LGTM.

[thoasm]

LGMT!

[tcojean]

LGTM.

[pratikvn]

LGTM!

[codecov]

Codecov Report

Merging #467 into develop will increase coverage by 1.49%.
The diff coverage is 91.35%.

@@             Coverage Diff             @@
##           develop     #467      +/-   ##
===========================================
+ Coverage    89.98%   91.47%   +1.49%     
===========================================
  Files          136      294     +158     
  Lines         6078    19551   +13473     
===========================================
+ Hits          5469    17884   +12415     
- Misses         609     1667    +1058     

Impacted Files	Coverage Δ
omp/solver/cgs_kernels.cpp	0.00% <0.00%> (-100.00%)	⬇️
omp/matrix/hybrid_kernels.cpp	0.00% <0.00%> (-98.83%)	⬇️
omp/solver/fcg_kernels.cpp	0.00% <0.00%> (-97.06%)	⬇️
omp/solver/cg_kernels.cpp	0.00% <0.00%> (-96.78%)	⬇️
omp/solver/gmres_kernels.cpp	0.00% <0.00%> (-95.63%)	⬇️
omp/matrix/coo_kernels.cpp	5.55% <0.00%> (-94.45%)	⬇️
omp/matrix/sellp_kernels.cpp	0.00% <0.00%> (-93.34%)	⬇️
omp/matrix/sparsity_csr_kernels.cpp	7.24% <0.00%> (-92.76%)	⬇️
omp/matrix/ell_kernels.cpp	0.00% <0.00%> (-91.67%)	⬇️
omp/solver/bicgstab_kernels.cpp	0.00% <0.00%> (-91.18%)	⬇️
... and 260 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3523945...0c1340a. Read the comment docs.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

No Coverage information
0.0% Duplication