Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parilu omp #315

Merged
merged 9 commits into from
Jun 19, 2019
Merged

Parilu omp #315

merged 9 commits into from
Jun 19, 2019

Conversation

hartwiganzt
Copy link
Collaborator

This pull request adds the ParILU functionality for the OMP executor. A challenge are the unit tests: the results will - depending on the parallelism - be very different from the reference executor results. Currently, I set a threshold of 0.3 for this specific test case.

@hartwiganzt hartwiganzt self-assigned this Jun 14, 2019
@hartwiganzt hartwiganzt added mod:openmp This is related to the OpenMP module. type:preconditioner This is related to the preconditioners 1:ST:ready-for-review This PR is ready for review labels Jun 14, 2019
@tcojean
Copy link
Member

tcojean commented Jun 14, 2019

If you do a huge amount of sweeps (say 100, could be less) is the precision better? In which case, we may prefer severely increasing the amount of sweeps specifically for the tests to ensure we have actual close values. Currently, 0.3 is very high as it is the order of most values in your test matrix. Values could be more than double the actual ones.

@codecov
Copy link

codecov bot commented Jun 14, 2019

Codecov Report

Merging #315 into develop will increase coverage by 0.04%.
The diff coverage is 98.63%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #315      +/-   ##
===========================================
+ Coverage    98.17%   98.21%   +0.04%     
===========================================
  Files          223      224       +1     
  Lines        17180    17321     +141     
===========================================
+ Hits         16866    17012     +146     
+ Misses         314      309       -5
Impacted Files Coverage Δ
omp/factorization/par_ilu_kernels.cpp 100% <100%> (+100%) ⬆️
omp/test/factorization/par_ilu_kernels.cpp 96.92% <96.92%> (ø)
core/base/mtx_io.cpp 93.5% <0%> (+0.64%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8bf33e0...80b6ce1. Read the comment docs.

@hartwiganzt
Copy link
Collaborator Author

It is not easy to say whether more sweeps increase the quality. In fact, the more sweeps, the higher is the chance of breakdown because of division-by-zero. For reference I below show a table with top-level iteration counts taken from the [ISC paper]((https://link.springer.com/chapter/10.1007/978-3-319-20119-1_1)
The results indicate that 3 sweeps may be a good choice....
Screen Shot 2019-06-14 at 4 58 24 PM

@thoasm
Copy link
Member

thoasm commented Jun 17, 2019

I am strongly in favor of using a file instead of this big, hard-coded matrix. Using a file has several benefits:

  • The code looks a lot nicer (no need for clang-format off / on)
  • We probably have more precise values (if you use the matrix market file)
  • Checking if the behavior is expected is a lot easier with a file since you can also read it with matlab
  • You can make changes inside the file a lot easier compared to inside the code
  • You can swap the matrix without a lot of effort

You can use the following patch to read the matrix from file (you have to copy the matrix to omp/test/factorization/ani1.mtx in order for this code to work):
patch_read_matrix_from_file.txt
(Note: All changes after the newly added SetUp() function are style changes caused by make format on my system)

We have one draw-back with the implementation I am proposing: we lose the ability to launch the test wherever we want because we have to be in the local directory to find the matrix-file.

We could get around this by creating a header file for the tests where we write the path of the build directory from cmake.
I think the proposed version should be fine, especially since there is an error message that explains why it crashes if it is called from the wrong place. Additionally, make test also works with this implementation.
But if it is requested, I can look into how to generate the header where the build directory is stored (or for a better alternative).

Copy link
Member

@thoasm thoasm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like the code duplication to be reduced by creating functions in the test.

omp/test/factorization/par_ilu_kernels.cpp Outdated Show resolved Hide resolved
omp/test/factorization/par_ilu_kernels.cpp Outdated Show resolved Hide resolved
omp/test/factorization/par_ilu_kernels.cpp Outdated Show resolved Hide resolved
omp/test/factorization/par_ilu_kernels.cpp Show resolved Hide resolved
omp/test/factorization/par_ilu_kernels.cpp Outdated Show resolved Hide resolved
omp/test/factorization/par_ilu_kernels.cpp Outdated Show resolved Hide resolved
Copy link
Member

@thoasm thoasm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just two small things.

omp/test/factorization/par_ilu_kernels.cpp Outdated Show resolved Hide resolved
omp/test/factorization/par_ilu_kernels.cpp Show resolved Hide resolved
Copy link
Member

@thoasm thoasm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@tcojean tcojean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I only have small comments.

In addition, I tried to see if there was any obvious way to improve the parallelization, but that was not fruitful. On the other end, making a reduction out of compute_nnz_l_u does not really worsen the computation time (on big enough matrices), although it does not really improve it either. You can decide whether you want to do it or not.

omp/factorization/par_ilu_kernels.cpp Outdated Show resolved Hide resolved
omp/test/factorization/ani1.mtx Show resolved Hide resolved
omp/test/factorization/ani1.mtx Show resolved Hide resolved
@hartwiganzt hartwiganzt merged commit a3307f0 into develop Jun 19, 2019
@hartwiganzt hartwiganzt deleted the parilu_omp branch June 19, 2019 04:19
tcojean added a commit that referenced this pull request Oct 20, 2019
The Ginkgo team is proud to announce the new minor release of Ginkgo version
1.1.0. This release brings several performance improvements, adds Windows support, 
adds support for factorizations inside Ginkgo and a new ILU preconditioner
based on ParILU algorithm, among other things. For detailed information, check the respective issue.

Supported systems and requirements:
+ For all platforms, cmake 3.9+
+ Linux and MacOS
  + gcc: 5.3+, 6.3+, 7.3+, 8.1+
  + clang: 3.9+
  + Intel compiler: 2017+
  + Apple LLVM: 8.0+
  + CUDA module: CUDA 9.0+
+ Windows
  + MinGW and CygWin: gcc 5.3+, 6.3+, 7.3+, 8.1+
  + Microsoft Visual Studio: VS 2017 15.7+
  + CUDA module: CUDA 9.0+, Microsoft Visual Studio
  + OpenMP module: MinGW or CygWin.


The current known issues can be found in the [known issues
page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues).


Additions:
+ Upper and lower triangular solvers ([#327](#327), [#336](#336), [#341](#341), [#342](#342)) 
+ New factorization support in Ginkgo, and addition of the ParILU
  algorithm ([#305](#305), [#315](#315), [#319](#319), [#324](#324))
+ New ILU preconditioner ([#348](#348), [#353](#353))
+ Windows MinGW and Cygwin support ([#347](#347))
+ Windows Visual studio support ([#351](#351))
+ New example showing how to use ParILU as a preconditioner ([#358](#358))
+ New example on using loggers for debugging ([#360](#360))
+ Add two new 9pt and 27pt stencil examples ([#300](#300), [#306](#306))
+ Allow benchmarking CuSPARSE spmv formats through Ginkgo's benchmarks ([#303](#303))
+ New benchmark for sparse matrix format conversions ([#312](#312))
+ Add conversions between CSR and Hybrid formats ([#302](#302), [#310](#310))
+ Support for sorting rows in the CSR format by column idices ([#322](#322))
+ Addition of a CUDA COO SpMM kernel for improved performance ([#345](#345))
+ Addition of a LinOp to handle perturbations of the form (identity + scalar *
  basis * projector) ([#334](#334))
+ New sparsity matrix representation format with Reference and OpenMP
  kernels ([#349](#349), [#350](#350))

Fixes:
+ Accelerate GMRES solver for CUDA executor ([#363](#363))
+ Fix BiCGSTAB solver convergence ([#359](#359))
+ Fix CGS logging by reporting the residual for every sub iteration ([#328](#328))
+ Fix CSR,Dense->Sellp conversion's memory access violation ([#295](#295))
+ Accelerate CSR->Ell,Hybrid conversions on CUDA ([#313](#313), [#318](#318))
+ Fixed slowdown of COO SpMV on OpenMP ([#340](#340))
+ Fix gcc 6.4.0 internal compiler error ([#316](#316))
+ Fix compilation issue on Apple clang++ 10 ([#322](#322))
+ Make Ginkgo able to compile on Intel 2017 and above ([#337](#337))
+ Make the benchmarks spmv/solver use the same matrix formats ([#366](#366))
+ Fix self-written isfinite function ([#348](#348))
+ Fix Jacobi issues shown by cuda-memcheck

Tools and ecosystem:
+ Multiple improvements to the CI system and tools ([#296](#296), [#311](#311), [#365](#365))
+ Multiple improvements to the Ginkgo containers ([#328](#328), [#361](#361))
+ Add sonarqube analysis to Ginkgo ([#304](#304), [#308](#308), [#309](#309))
+ Add clang-tidy and iwyu support to Ginkgo ([#298](#298))
+ Improve Ginkgo's support of xSDK M12 policy by adding the `TPL_` arguments
  to CMake ([#300](#300))
+ Add support for the xSDK R7 policy ([#325](#325))
+ Fix examples in html documentation ([#367](#367))
tcojean added a commit that referenced this pull request Oct 21, 2019
The Ginkgo team is proud to announce the new minor release of Ginkgo version
1.1.0. This release brings several performance improvements, adds Windows support,
adds support for factorizations inside Ginkgo and a new ILU preconditioner
based on ParILU algorithm, among other things. For detailed information, check the respective issue.

Supported systems and requirements:
+ For all platforms, cmake 3.9+
+ Linux and MacOS
  + gcc: 5.3+, 6.3+, 7.3+, 8.1+
  + clang: 3.9+
  + Intel compiler: 2017+
  + Apple LLVM: 8.0+
  + CUDA module: CUDA 9.0+
+ Windows
  + MinGW and Cygwin: gcc 5.3+, 6.3+, 7.3+, 8.1+
  + Microsoft Visual Studio: VS 2017 15.7+
  + CUDA module: CUDA 9.0+, Microsoft Visual Studio
  + OpenMP module: MinGW or Cygwin.


The current known issues can be found in the [known issues
page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues).


### Additions
+ Upper and lower triangular solvers ([#327](#327), [#336](#336), [#341](#341), [#342](#342)) 
+ New factorization support in Ginkgo, and addition of the ParILU
  algorithm ([#305](#305), [#315](#315), [#319](#319), [#324](#324))
+ New ILU preconditioner ([#348](#348), [#353](#353))
+ Windows MinGW and Cygwin support ([#347](#347))
+ Windows Visual Studio support ([#351](#351))
+ New example showing how to use ParILU as a preconditioner ([#358](#358))
+ New example on using loggers for debugging ([#360](#360))
+ Add two new 9pt and 27pt stencil examples ([#300](#300), [#306](#306))
+ Allow benchmarking CuSPARSE spmv formats through Ginkgo's benchmarks ([#303](#303))
+ New benchmark for sparse matrix format conversions ([#312](#312))
+ Add conversions between CSR and Hybrid formats ([#302](#302), [#310](#310))
+ Support for sorting rows in the CSR format by column idices ([#322](#322))
+ Addition of a CUDA COO SpMM kernel for improved performance ([#345](#345))
+ Addition of a LinOp to handle perturbations of the form (identity + scalar *
  basis * projector) ([#334](#334))
+ New sparsity matrix representation format with Reference and OpenMP
  kernels ([#349](#349), [#350](#350))

### Fixes
+ Accelerate GMRES solver for CUDA executor ([#363](#363))
+ Fix BiCGSTAB solver convergence ([#359](#359))
+ Fix CGS logging by reporting the residual for every sub iteration ([#328](#328))
+ Fix CSR,Dense->Sellp conversion's memory access violation ([#295](#295))
+ Accelerate CSR->Ell,Hybrid conversions on CUDA ([#313](#313), [#318](#318))
+ Fixed slowdown of COO SpMV on OpenMP ([#340](#340))
+ Fix gcc 6.4.0 internal compiler error ([#316](#316))
+ Fix compilation issue on Apple clang++ 10 ([#322](#322))
+ Make Ginkgo able to compile on Intel 2017 and above ([#337](#337))
+ Make the benchmarks spmv/solver use the same matrix formats ([#366](#366))
+ Fix self-written isfinite function ([#348](#348))
+ Fix Jacobi issues shown by cuda-memcheck

### Tools and ecosystem improvements
+ Multiple improvements to the CI system and tools ([#296](#296), [#311](#311), [#365](#365))
+ Multiple improvements to the Ginkgo containers ([#328](#328), [#361](#361))
+ Add sonarqube analysis to Ginkgo ([#304](#304), [#308](#308), [#309](#309))
+ Add clang-tidy and iwyu support to Ginkgo ([#298](#298))
+ Improve Ginkgo's support of xSDK M12 policy by adding the `TPL_` arguments
  to CMake ([#300](#300))
+ Add support for the xSDK R7 policy ([#325](#325))
+ Fix examples in html documentation ([#367](#367))


Related PR: #370
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1:ST:ready-for-review This PR is ready for review mod:openmp This is related to the OpenMP module. type:preconditioner This is related to the preconditioners
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants