Add searching and merging functions #455

upsj · 2020-01-21T17:18:22Z

To simplify reviewing the ParILUT branch, this PR contains a few cherry-picked algorithms that are used within ParILUT, but are also rather general-purpose. The kernels are based on ideas from Green et al., “GPU merge path: a GPU merging algorithm”

Searching algorithms that, for a range and a corresponding predicate that is false on the lower part of the range and true on the upper part, identify the first index where the predicate is true. This is the most general form of a binary/k-ary search. There are three implementation variants as mentioned in the paper. (We could also potentially use them in our merge-path SpMV, but that's an idea for another time)
Merging algorithms that merge two sorted ranges into a single sorted range (via a callback function)

TODO:

Add references to the implementations
Remove unnecessary changes

common/components/merging.hpp.inc

tcojean

Comments on searching for now.

cuda/test/components/searching.cu

common/components/searching.hpp.inc

thoasm

Review Part 1.

common/components/merging.hpp.inc

cuda/components/merging.cuh

hip/components/merging.hip.hpp

cuda/components/searching.cuh

hip/components/searching.hip.hpp

cuda/test/components/merging.cu

tcojean

LGTM. Some minor comments.

common/components/merging.hpp.inc

cuda/test/components/searching.cu

thoasm

Mostly concerns about the use of auto in combination with integer literals.
Otherwise, it looks good to me.

common/components/merging.hpp.inc

common/components/searching.hpp.inc

hip/test/components/merging.hip.cpp

hip/test/components/searching.hip.cpp

hip/test/components/merging.hip.cpp

common/components/searching.hpp.inc

hip/test/components/merging.hip.cpp

common/components/merging.hpp.inc

cuda/test/components/merging.cu

cuda/test/components/searching.cu

hip/test/components/merging.hip.cpp

hip/test/components/searching.hip.cpp

common/components/merging.hpp.inc

yhmtsai · 2020-01-27T17:11:08Z

non self-contained hip file is hip/device_functions.h which needs to include hip/hip_runtime.h first.
We only use hip/device_functions.h in the config.hip.hpp to get warpsize.
For temporary fix, put the hip/hip_runtime.h into config.hip.hpp before hip/device_functions.h.
It should fix the problem, and I also did it in PR #447 sorting and config changes (https://github.com/ginkgo-project/ginkgo/pull/447/files/55a80da0a4247401db21d5ae4932930a3d01b176)
Could you also try it here?

thoasm

LGTM!

common/components/intrinsics.hpp.inc

Co-authored-by: Thomas Grützmacher <[email protected]> Co-authored-by: Terry Cojean <[email protected]>

Co-Authored-By: Thomas Grützmacher <[email protected]>

sonarcloud · 2020-01-29T00:08:56Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

No Coverage information
0.0% Duplication

This PR adds the kernel `add_diagonal_elements` to `ParIlu` which adds explicit zeros to the main diagonal if such element was not already present. This ensures that both `L` and `U` are always properly updated during the generation. Related issue: #455 Related PR: #456

The Ginkgo team is proud to announce the new minor release of Ginkgo version 1.2.0. This release brings full HIP support to Ginkgo, new preconditioners (ParILUT, ISAI), conversion between double and float for all LinOps, and many more features and fixes. Supported systems and requirements: + For all platforms, cmake 3.9+ + Linux and MacOS + gcc: 5.3+, 6.3+, 7.3+, all versions after 8.1+ + clang: 3.9+ + Intel compiler: 2017+ + Apple LLVM: 8.0+ + CUDA module: CUDA 9.0+ + HIP module: ROCm 2.8+ + Windows + MinGW and CygWin: gcc 5.3+, 6.3+, 7.3+, all versions after 8.1+ + Microsoft Visual Studio: VS 2017 15.7+ + CUDA module: CUDA 9.0+, Microsoft Visual Studio + OpenMP module: MinGW or CygWin. The current known issues can be found in the [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues). # Additions Here are the main additions to the Ginkgo library. Other thematic additions are listed below. + Add full HIP support to Ginkgo [#344](#344), [#357](#357), [#384](#384), [#373](#373), [#391](#391), [#396](#396), [#395](#395), [#393](#393), [#404](#404), [#439](#439), [#443](#443), [#567](#567) + Add a new ISAI preconditioner [#489](#489), [#502](#502), [#512](#512), [#508](#508), [#520](#520) + Add support for ParILUT and ParICT factorization with ILU preconditioners [#400](#400) + Add a new BiCG solver [#438](#438) + Add a new permutation matrix format [#352](#352), [#469](#469) + Add CSR SpGEMM support [#386](#386), [#398](#398), [#418](#418), [#457](#457) + Add CSR SpGEAM support [#556](#556) + Make all solvers and preconditioners transposable [#535](#535) + Add CsrBuilder and CooBuilder for intrusive access to matrix arrays [#437](#437) + Add a standard-compliant allocator based on the Executors [#504](#504) + Support conversions for all LinOp between double and float [#521](#521) + Add a new boolean to the CUDA and HIP executors to control DeviceReset (default off) [#557](#557) + Add a relaxation factor to IR to represent Richardson Relaxation [#574](#574) + Add two new stopping criteria, for relative (to `norm(b)`) and absolute residual norm [#577](#577) ### Example additions + Templatize all examples to simplify changing the precision [#513](#513) + Add a new adaptive precision block-Jacobi example [#507](#507) + Add a new IR example [#522](#522) + Add a new Mixed Precision Iterative Refinement example [#525](#525) + Add a new example on iterative trisolves in ILU preconditioning [#526](#526), [#536](#536), [#550](#550) ### Compilation and library changes + Auto-detect compilation settings based on environment [#435](#435), [#537](#537) + Add SONAME to shared libraries [#524](#524) + Add clang-cuda support [#543](#543) ### Other additions + Add sorting, searching and merging kernels for GPUs [#403](#403), [#428](#428), [#417](#417), [#455](#455) + Add `gko::as` support for smart pointers [#493](#493) + Add setters and getters for criterion factories [#527](#527) + Add a new method to check whether a solver uses `x` as an initial guess [#531](#531) + Add contribution guidelines [#549](#549) # Fixes ### Algorithms + Improve the classical CSR strategy's performance [#401](#401) + Improve the CSR automatical strategy [#407](#407), [#559](#559) + Memory, speed improvements to the ELL kernel [#411](#411) + Multiple improvements and fixes to ParILU [#419](#419), [#427](#427), [#429](#429), [#456](#456), [#544](#544) + Fix multiple issues with GMRES [#481](#481), [#523](#523), [#575](#575) + Optimize OpenMP matrix conversions [#505](#505) + Ensure the linearity of the ILU preconditioner [#506](#506) + Fix IR's use of the advanced apply [#522](#522) + Fix empty matrices conversions and add tests [#560](#560) ### Other core functionalities + Fix complex number support in our math header [#410](#410) + Fix CUDA compatibility of the main ginkgo header [#450](#450) + Fix isfinite issues [#465](#465) + Fix the Array::view memory leak and the array/view copy/move [#485](#485) + Fix typos preventing use of some interface functions [#496](#496) + Fix the `gko::dim` to abide to the C++ standard [#498](#498) + Simplify the executor copy interface [#516](#516) + Optimize intermediate storage for Composition [#540](#540) + Provide an initial guess for relevant Compositions [#561](#561) + Better management of nullptr as criterion [#562](#562) + Fix the norm calculations for complex support [#564](#564) ### CUDA and HIP specific + Use the return value of the atomic operations in our wrappers [#405](#405) + Improve the portability of warp lane masks [#422](#422) + Extract thread ID computation into a separate function [#464](#464) + Reorder kernel parameters for consistency [#474](#474) + Fix the use of `pragma unroll` in HIP [#492](#492) ### Other + Fix the Ginkgo CMake installation files [#414](#414), [#553](#553) + Fix the Windows compilation [#415](#415) + Always use demangled types in error messages [#434](#434), [#486](#486) + Add CUDA header dependency to appropriate tests [#452](#452) + Fix several sonarqube or compilation warnings [#453](#453), [#463](#463), [#532](#532), [#569](#569) + Add shuffle tests [#460](#460) + Fix MSVC C2398 error [#490](#490) + Fix missing interface tests in test install [#558](#558) # Tools and ecosystem ### Benchmarks + Add better norm support in the benchmarks [#377](#377) + Add CUDA 10.1 generic SpMV support in benchmarks [#468](#468), [#473](#473) + Add sparse library ILU in benchmarks [#487](#487) + Add overhead benchmarking capacities [#501](#501) + Allow benchmarking from a matrix list file [#503](#503) + Fix benchmarking issue with JSON and non-finite numbers [#514](#514) + Fix benchmark logger crashers with OpenMP [#565](#565) ### CI related + Improvements to the CI setup with HIP compilation [#421](#421), [#466](#466) + Add MacOSX CI support [#470](#470), [#488](#488) + Add Windows CI support [#471](#471), [#488](#488), [#510](#510), [#566](#566) + Use sanitizers instead of valgrind [#476](#476) + Add automatic container generation and update facilities [#499](#499) + Fix the CI parallelism settings [#517](#517), [#538](#538), [#539](#539) + Make the codecov patch check informational [#519](#519) + Add support for LLVM sanitizers with improved thread sanitizer support [#578](#578) ### Test suite + Add an assertion for sparsity pattern equality [#416](#416) + Add core and reference multiprecision tests support [#448](#448) + Speed up GPU tests by avoiding device reset [#467](#467) + Change test matrix location string [#494](#494) ### Other + Add Ginkgo badges from our tools [#413](#413) + Update the `create_new_algorithm.sh` script [#420](#420) + Bump copyright and improve license management [#436](#436), [#433](#433) + Set clang-format minimum requirement [#441](#441), [#484](#484) + Update git-cmake-format [#446](#446), [#484](#484) + Disable the development tools by default [#442](#442) + Add a script for automatic header formatting [#447](#447) + Add GDB pretty printer for `gko::Array` [#509](#509) + Improve compilation speed [#533](#533) + Add editorconfig support [#546](#546) + Add a compile-time check for header self-sufficiency [#552](#552) # Related PR: #583

upsj added mod:cuda This is related to the CUDA module. mod:hip This is related to the HIP module. labels Jan 21, 2020

upsj requested review from pratikvn, thoasm, yhmtsai and tcojean January 21, 2020 17:18

upsj self-assigned this Jan 21, 2020

upsj added the 1:ST:ready-for-review This PR is ready for review label Jan 21, 2020

upsj force-pushed the add_searching_merging_kernels branch 2 times, most recently from 0bfa721 to 2e724f5 Compare January 22, 2020 16:54

upsj commented Jan 23, 2020

View reviewed changes

common/components/merging.hpp.inc Outdated Show resolved Hide resolved

thoasm mentioned this pull request Jan 24, 2020

Fix parilu by adding explicit zeros to diagonal #456

Merged

2 tasks

tcojean reviewed Jan 24, 2020

View reviewed changes

cuda/test/components/searching.cu Outdated Show resolved Hide resolved

cuda/test/components/searching.cu Outdated Show resolved Hide resolved

common/components/searching.hpp.inc Show resolved Hide resolved

thoasm reviewed Jan 24, 2020

View reviewed changes

tcojean approved these changes Jan 27, 2020

View reviewed changes

common/components/merging.hpp.inc Outdated Show resolved Hide resolved

common/components/merging.hpp.inc Show resolved Hide resolved

common/components/merging.hpp.inc Outdated Show resolved Hide resolved

cuda/test/components/searching.cu Show resolved Hide resolved

thoasm reviewed Jan 27, 2020

View reviewed changes

upsj commented Jan 27, 2020

View reviewed changes

common/components/merging.hpp.inc Outdated Show resolved Hide resolved

thoasm approved these changes Jan 27, 2020

View reviewed changes

upsj force-pushed the add_searching_merging_kernels branch from 2fb111d to 4d36101 Compare January 28, 2020 13:25

upsj added 1:ST:ready-to-merge This PR is ready to merge. and removed 1:ST:ready-for-review This PR is ready for review labels Jan 28, 2020

thoasm reviewed Jan 28, 2020

View reviewed changes

common/components/intrinsics.hpp.inc Outdated Show resolved Hide resolved

upsj commented Jan 28, 2020

View reviewed changes

upsj added 5 commits January 28, 2020 22:20

add searching kernels and tests

fba361a

add merging algorithms and tests

94bc786

add references to algorithms

9719fdc

remove 3way merge, test all callback parameters

0fcdb21

unroll parallel merge loop by default

82542f2

upsj and others added 3 commits January 28, 2020 22:20

review updates

f3be9d5

Co-authored-by: Thomas Grützmacher <[email protected]> Co-authored-by: Terry Cojean <[email protected]>

improve merge callback interface

27cfdb8

replace inline by __forceinline__ for intrinsics

142d891

Co-Authored-By: Thomas Grützmacher <[email protected]>

upsj force-pushed the add_searching_merging_kernels branch from d374dc3 to 142d891 Compare January 28, 2020 21:21

upsj merged commit 11bc642 into develop Jan 29, 2020

upsj deleted the add_searching_merging_kernels branch January 29, 2020 07:03

tcojean mentioned this pull request Jun 23, 2020

Release/1.2.0 #576

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add searching and merging functions #455

Add searching and merging functions #455

upsj commented Jan 21, 2020 •

edited

Loading

tcojean left a comment

thoasm left a comment

tcojean left a comment

thoasm left a comment

yhmtsai commented Jan 27, 2020

thoasm left a comment

sonarcloud bot commented Jan 29, 2020

Add searching and merging functions #455

Add searching and merging functions #455

Conversation

upsj commented Jan 21, 2020 • edited Loading

tcojean left a comment

Choose a reason for hiding this comment

thoasm left a comment

Choose a reason for hiding this comment

tcojean left a comment

Choose a reason for hiding this comment

thoasm left a comment

Choose a reason for hiding this comment

yhmtsai commented Jan 27, 2020

thoasm left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Jan 29, 2020

upsj commented Jan 21, 2020 •

edited

Loading