Fix handling of missing diagonal in symbolic factorizations #1263

upsj · 2023-01-26T14:27:10Z

This artificially adds the diagonal entry for all symbolic factorizations by using a sentinel value (Cholesky) or adding the diagonal explicitly if missing (LU).

TODO:

fix cholesky sparsity pattern computation

upsj · 2023-02-15T15:17:35Z

format-rebase!

ginkgo-bot · 2023-02-15T15:19:12Z

Formatting rebase introduced changes, see Artifacts here to review them

yhmtsai

LGTM in other parts. the diag_postorder part is not reviewed yet

yhmtsai · 2023-02-20T09:07:41Z

common/cuda_hip/factorization/cholesky_kernels.hpp.inc

 {
 const auto row = thread::get_subwarp_id_flat<subwarp_size, IndexType>();
 if (row >= num_rows) {
 return;
 }
 const auto row_begin = row_ptrs[row];
+ const auto diag_postorder = inv_postorder[row];


Could you remind me what's inv_postoder and postorder?

postorder is the sequence of row numbers such that each child comes before its parents in the elimination tree (and thus a permutation), inv_postorder is the inverse permutation. diag_postorder is the rank of the row index in the postorder sequence, i.e. it corresponds to the (potentially missing) diagonal index in the post-order column index sequence. I'll add this explanation to the code

postorder[i]: i-th node is postorder[i]-th in original graph
row will be considered as inv_postorder[row] in new graph
is postorder_cols like inv_postorder[]?
col[i] will be considered as postorder_col[i] in the new graph

from build_postorder_cols, it seems to be.

reference/test/factorization/cholesky_kernels.cpp

fritzgoebel

LGTM!
A comment about the purpose of diag_postorder would be helpful.

yhmtsai · 2023-02-20T22:22:23Z

common/cuda_hip/factorization/cholesky_kernels.hpp.inc

 {
 const auto row = thread::get_subwarp_id_flat<subwarp_size, IndexType>();
 if (row >= num_rows) {
 return;
 }
 const auto row_begin = row_ptrs[row];
+ const auto diag_postorder = inv_postorder[row];


from build_postorder_cols, it seems to be.

ginkgo-bot · 2023-02-21T09:46:52Z

Note: This PR changes the Ginkgo ABI:

Functions changes summary: 44 Removed, 44 Changed (110 filtered out), 76 Added functions
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable

For details check the full ABI diff under Artifacts here

upsj · 2023-02-21T15:49:20Z

rebase!

ginkgo-bot · 2023-02-21T15:59:58Z

Error: PR already merged!

sonarcloud · 2023-02-21T20:44:05Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

63.6% Coverage
3.1% Duplication

Release 1.6.0 of Ginkgo. The Ginkgo team is proud to announce the new Ginkgo minor release 1.6.0. This release brings new features such as: - Several building blocks for GPU-resident sparse direct solvers like symbolic and numerical LU and Cholesky factorization, ..., - A distributed Schwarz preconditioner, - New FGMRES and GCR solvers, - Distributed benchmarks for the SpMV operation, solvers, ... - Support for non-default streams in the CUDA and HIP backends, - Mixed precision support for the CSR SpMV, - A new profiling logger which integrates with NVTX, ROCTX, TAU and VTune to provide internal Ginkgo knowledge to most HPC profilers! and much more. If you face an issue, please first check our [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues) and the [open issues list](https://github.com/ginkgo-project/ginkgo/issues) and if you do not find a solution, feel free to [open a new issue](https://github.com/ginkgo-project/ginkgo/issues/new/choose) or ask a question using the [github discussions](https://github.com/ginkgo-project/ginkgo/discussions). Supported systems and requirements: + For all platforms, CMake 3.13+ + C++14 compliant compiler + Linux and macOS + GCC: 5.5+ + clang: 3.9+ + Intel compiler: 2018+ + Apple Clang: 14.0 is tested. Earlier versions might also work. + NVHPC: 22.7+ + Cray Compiler: 14.0.1+ + CUDA module: CUDA 9.2+ or NVHPC 22.7+ + HIP module: ROCm 4.5+ + DPC++ module: Intel OneAPI 2021.3+ with oneMKL and oneDPL. Set the CXX compiler to `dpcpp`. + Windows + MinGW: GCC 5.5+ + Microsoft Visual Studio: VS 2019+ + CUDA module: CUDA 9.2+, Microsoft Visual Studio + OpenMP module: MinGW. ### Version Support Changes + ROCm 4.0+ -> 4.5+ after [#1303](#1303) + Removed Cygwin pipeline and support [#1283](#1283) ### Interface Changes + Due to internal changes, `ConcreteExecutor::run` will now always throw if the corresponding module for the `ConcreteExecutor` is not build [#1234](#1234) + The constructor of `experimental::distributed::Vector` was changed to only accept local vectors as `std::unique_ptr` [#1284](#1284) + The default parameters for the `solver::MultiGrid` were improved. In particular, the smoother defaults to one iteration of `Ir` with `Jacobi` preconditioner, and the coarse grid solver uses the new direct solver with LU factorization. [#1291](#1291) [#1327](#1327) + The `iteration_complete` event gained a more expressive overload with additional parameters, the old overloads were deprecated. [#1288](#1288) [#1327](#1327) ### Deprecations + Deprecated less expressive `iteration_complete` event. Users are advised to now implement the function `void iteration_complete(const LinOp* solver, const LinOp* b, const LinOp* x, const size_type& it, const LinOp* r, const LinOp* tau, const LinOp* implicit_tau_sq, const array<stopping_status>* status, bool stopped)` [#1288](#1288) ### Added Features + A distributed Schwarz preconditioner. [#1248](#1248) + A GCR solver [#1239](#1239) + Flexible Gmres solver [#1244](#1244) + Enable Gmres solver for distributed matrices and vectors [#1201](#1201) + An example that uses Kokkos to assemble the system matrix [#1216](#1216) + A symbolic LU factorization allowing the `gko::experimental::factorization::Lu` and `gko::experimental::solver::Direct` classes to be used for matrices with non-symmetric sparsity pattern [#1210](#1210) + A numerical Cholesky factorization [#1215](#1215) + Symbolic factorizations in host-side operations are now wrapped in a host-side `Operation` to make their execution visible to loggers. This means that profiling loggers and benchmarks are no longer missing a separate entry for their runtime [#1232](#1232) + Symbolic factorization benchmark [#1302](#1302) + The `ProfilerHook` logger allows annotating the Ginkgo execution (apply, operations, ...) for profiling frameworks like NVTX, ROCTX and TAU. [#1055](#1055) + `ProfilerHook::created_(nested_)summary` allows the generation of a lightweight runtime profile over all Ginkgo functions written to a user-defined stream [#1270](#1270) for both host and device timing functionality [#1313](#1313) + It is now possible to enable host buffers for MPI communications at runtime even if the compile option `GINKGO_FORCE_GPU_AWARE_MPI` is set. [#1228](#1228) + A stencil matrices generator (5-pt, 7-pt, 9-pt, and 27-pt) for benchmarks [#1204](#1204) + Distributed benchmarks (multi-vector blas, SpMV, solver) [#1204](#1204) + Benchmarks for CSR sorting and lookup [#1219](#1219) + A timer for MPI benchmarks that reports the longest time [#1217](#1217) + A `timer_method=min|max|average|median` flag for benchmark timing summary [#1294](#1294) + Support for non-default streams in CUDA and HIP executors [#1236](#1236) + METIS integration for nested dissection reordering [#1296](#1296) + SuiteSparse AMD integration for fillin-reducing reordering [#1328](#1328) + Csr mixed-precision SpMV support [#1319](#1319) + A `with_loggers` function for all `Factory` parameters [#1337](#1337) ### Improvements + Improve naming of kernel operations for loggers [#1277](#1277) + Annotate solver iterations in `ProfilerHook` [#1290](#1290) + Allow using the profiler hooks and inline input strings in benchmarks [#1342](#1342) + Allow passing smart pointers in place of raw pointers to most matrix functions. This means that things like `vec->compute_norm2(x.get())` or `vec->compute_norm2(lend(x))` can be simplified to `vec->compute_norm2(x)` [#1279](#1279) [#1261](#1261) + Catch overflows in prefix sum operations, which makes Ginkgo's operations much less likely to crash. This also improves the performance of the prefix sum kernel [#1303](#1303) + Make the installed GinkgoConfig.cmake file relocatable and follow more best practices [#1325](#1325) ### Fixes + Fix OpenMPI version check [#1200](#1200) + Fix the mpi cxx type binding by c binding [#1306](#1306) + Fix runtime failures for one-sided MPI wrapper functions observed on some OpenMPI versions [#1249](#1249) + Disable thread pinning with GPU executors due to poor performance [#1230](#1230) + Fix hwloc version detection [#1266](#1266) + Fix PAPI detection in non-implicit include directories [#1268](#1268) + Fix PAPI support for newer PAPI versions: [#1321](#1321) + Fix pkg-config file generation for library paths outside prefix [#1271](#1271) + Fix various build failures with ROCm 5.4, CUDA 12, and OneAPI 6 [#1214](#1214), [#1235](#1235), [#1251](#1251) + Fix incorrect read for skew-symmetric MatrixMarket files with explicit diagonal entries [#1272](#1272) + Fix handling of missing diagonal entries in symbolic factorizations [#1263](#1263) + Fix segmentation fault in benchmark matrix construction [#1299](#1299) + Fix the stencil matrix creation for benchmarking [#1305](#1305) + Fix the additional residual check in IR [#1307](#1307) + Fix the cuSPARSE CSR SpMM issue on single strided vector when cuda >= 11.6 [#1322](#1322) [#1331](#1331) + Fix Isai generation for large sparsity powers [#1327](#1327) + Fix Ginkgo compilation and test with NVHPC >= 22.7 [#1331](#1331) + Fix Ginkgo compilation of 32 bit binaries with MSVC [#1349](#1349)

upsj self-assigned this Jan 26, 2023

upsj force-pushed the fix_symbolic_diagonal branch from 024ddcc to 1f2a59d Compare February 15, 2023 15:16

upsj marked this pull request as ready for review February 15, 2023 15:16

upsj added the 1:ST:ready-for-review This PR is ready for review label Feb 15, 2023

upsj requested a review from a team February 15, 2023 15:17

ginkgo-bot force-pushed the fix_symbolic_diagonal branch from 1f2a59d to e8ac930 Compare February 15, 2023 15:19

yhmtsai reviewed Feb 20, 2023

View reviewed changes

upsj requested a review from fritzgoebel February 20, 2023 13:29

fritzgoebel approved these changes Feb 20, 2023

View reviewed changes

yhmtsai approved these changes Feb 20, 2023

View reviewed changes

upsj added 1:ST:ready-to-merge This PR is ready to merge. and removed 1:ST:ready-for-review This PR is ready for review labels Feb 21, 2023

upsj added 5 commits February 21, 2023 15:49

fix and test handling of missing diagonal in LU

2340fc2

add direct solvers to benchmark again

c05afc9

add ref test for missing diagonal in Cholesky

a272977

add tests with missing diagonal

d3ecb7d

fix handling of missing diagonal in Cholesky

c55df98

improve documentation

949d5b6

ginkgo-bot force-pushed the fix_symbolic_diagonal branch from 448b14c to 949d5b6 Compare February 21, 2023 15:49

upsj merged commit e0ce4a9 into develop Feb 21, 2023

upsj deleted the fix_symbolic_diagonal branch February 21, 2023 15:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix handling of missing diagonal in symbolic factorizations #1263

Fix handling of missing diagonal in symbolic factorizations #1263

upsj commented Jan 26, 2023 •

edited

Loading

upsj commented Feb 15, 2023

ginkgo-bot commented Feb 15, 2023

yhmtsai left a comment

yhmtsai Feb 20, 2023

upsj Feb 20, 2023 •

edited

Loading

yhmtsai Feb 20, 2023

yhmtsai Feb 20, 2023

fritzgoebel left a comment

yhmtsai Feb 20, 2023

ginkgo-bot commented Feb 21, 2023

upsj commented Feb 21, 2023

ginkgo-bot commented Feb 21, 2023

sonarcloud bot commented Feb 21, 2023

Fix handling of missing diagonal in symbolic factorizations #1263

Fix handling of missing diagonal in symbolic factorizations #1263

Conversation

upsj commented Jan 26, 2023 • edited Loading

upsj commented Feb 15, 2023

ginkgo-bot commented Feb 15, 2023

yhmtsai left a comment

Choose a reason for hiding this comment

yhmtsai Feb 20, 2023

Choose a reason for hiding this comment

upsj Feb 20, 2023 • edited Loading

Choose a reason for hiding this comment

yhmtsai Feb 20, 2023

Choose a reason for hiding this comment

yhmtsai Feb 20, 2023

Choose a reason for hiding this comment

fritzgoebel left a comment

Choose a reason for hiding this comment

yhmtsai Feb 20, 2023

Choose a reason for hiding this comment

ginkgo-bot commented Feb 21, 2023

upsj commented Feb 21, 2023

ginkgo-bot commented Feb 21, 2023

sonarcloud bot commented Feb 21, 2023

upsj commented Jan 26, 2023 •

edited

Loading

upsj Feb 20, 2023 •

edited

Loading