Improve BENCHMARKING.md with more detailed info.
#619
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tcojean
added
is:enhancement
tcojean
force-pushed
the
improve_benchmarking_doc
branch
from
13befed
to
9aed0d9
Compare
Codecov Report
@@ Coverage Diff @@
## develop #619 +/- ##
========================================
Coverage 92.89% 92.89%
========================================
Files 303 303
Lines 21331 21331
========================================
Hits 19815 19815
Misses 1516 1516 Continue to review full report at Codecov.
|
pratikvn
approved these changes
Aug 12, 2020
There was a problem hiding this comment.
LGTM! Pretty comprehensive. It would also be nice to have something like a best-practice guide that explains what settings of Ginkgo the benchmarks should be run for. There we could mention some generic aspects such as:
- Compile your code in
Release - Make sure the machine has no competing jobs.
- The recommended number of warmup iterations (which might be different for solvers, preconds and SpMV)
And Ginkgo specific ones such as:
- For adaptive block jacobi, enable the
GINKGO_JACOBI_FULL_OPTIMIZATIONSflag with a warning that this uses a lot more memory. - Mention that we also have a overhead benchmarking setup.
and any other ones that I have forgotten
tcojean
force-pushed
the
improve_benchmarking_doc
branch
from
9aed0d9
to
409acef
Compare
|
Thanks for your comments @pratikvn, I integrated your suggestions. |
tcojean
force-pushed
the
improve_benchmarking_doc
branch
from
409acef
to
2c2185d
Compare
upsj
approved these changes
Aug 17, 2020
There was a problem hiding this comment.
LGTM! A general proposal: If we have interdependent pieces of documentation (BENCHMARKING.md + benchmark commandline options, ...), do you think it would make sense to add a note to the source code to check the dependent documentation? An example would be adding a new preconditioner, spmv algorithm or solver to the benchmarks
upsj
added
1:ST:ready-to-merge
Co-authored-by: Pratik Nayak <[email protected]>
Co-authored-by: Yuhsiang M. Tsai <[email protected]>
tcojean
force-pushed
the
improve_benchmarking_doc
branch
from
2c2185d
to
3ea7874
Compare
|
@upsj how do you suggest to do this? I don't think there is a central place in the source code where we could do this, so would you add a comment in every source file? |
|
I opened #623 to discuss the split documentation issue. |
|
Kudos, SonarCloud Quality Gate passed!
|
tcojean
added a commit
that referenced
this pull request
Aug 26, 2020
Release 1.3.0 of Ginkgo. The Ginkgo team is proud to announce the new minor release of Ginkgo version 1.3.0. This release brings CUDA 11 support, changes the default C++ standard to be C++14 instead of C++11, adds a new Diagonal matrix format and capacity for diagonal extraction, significantly improves the CMake configuration output format, adds the Ginkgo paper which got accepted into the Journal of Open Source Software (JOSS), and fixes multiple issues. Supported systems and requirements: + For all platforms, cmake 3.9+ + Linux and MacOS + gcc: 5.3+, 6.3+, 7.3+, all versions after 8.1+ + clang: 3.9+ + Intel compiler: 2017+ + Apple LLVM: 8.0+ + CUDA module: CUDA 9.0+ + HIP module: ROCm 2.8+ + Windows + MinGW and Cygwin: gcc 5.3+, 6.3+, 7.3+, all versions after 8.1+ + Microsoft Visual Studio: VS 2017 15.7+ + CUDA module: CUDA 9.0+, Microsoft Visual Studio + OpenMP module: MinGW or Cygwin. The current known issues can be found in the [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues). Additions: + Add paper for Journal of Open Source Software (JOSS). [#479](#479) + Add a DiagonalExtractable interface. [#563](#563) + Add a new diagonal Matrix Format. [#580](#580) + Add Cuda11 support. [#603](#603) + Add information output after CMake configuration. [#610](#610) + Add a new preconditioner export example. [#595](#595) + Add a new cuda-memcheck CI job. [#592](#592) Changes: + Use unified memory in CUDA debug builds. [#621](#621) + Improve `BENCHMARKING.md` with more detailed info. [#619](#619) + Use C++14 standard instead of C++11. [#611](#611) + Update the Ampere sm information and CudaArchitectureSelector. [#588](#588) Fixes: + Fix documentation warnings and errors. [#624](#624) + Fix warnings for diagonal matrix format. [#622](#622) + Fix criterion factory parameters in CUDA. [#586](#586) + Fix the norm-type in the examples. [#612](#612) + Fix the WAW race in OpenMP is_sorted_by_column_index. [#617](#617) + Fix the example's exec_map by creating the executor only if requested. [#602](#602) + Fix some CMake warnings. [#614](#614) + Fix Windows building documentation. [#601](#601) + Warn when CXX and CUDA host compiler do not match. [#607](#607) + Fix reduce_add, prefix_sum, and doc-build. [#593](#593) + Fix find_library(cublas) issue on machines installing multiple cuda. [#591](#591) + Fix allocator in sellp read. [#589](#589) + Fix the CAS with HIP and NVIDIA backends. [#585](#585) Deletions: + Remove unused preconditioner parameter in LowerTrs. [#587](#587) Related PR: #625
tcojean
added a commit
that referenced
this pull request
Aug 27, 2020
The Ginkgo team is proud to announce the new minor release of Ginkgo version 1.3.0. This release brings CUDA 11 support, changes the default C++ standard to be C++14 instead of C++11, adds a new Diagonal matrix format and capacity for diagonal extraction, significantly improves the CMake configuration output format, adds the Ginkgo paper which got accepted into the Journal of Open Source Software (JOSS), and fixes multiple issues. Supported systems and requirements: + For all platforms, cmake 3.9+ + Linux and MacOS + gcc: 5.3+, 6.3+, 7.3+, all versions after 8.1+ + clang: 3.9+ + Intel compiler: 2017+ + Apple LLVM: 8.0+ + CUDA module: CUDA 9.0+ + HIP module: ROCm 2.8+ + Windows + MinGW and Cygwin: gcc 5.3+, 6.3+, 7.3+, all versions after 8.1+ + Microsoft Visual Studio: VS 2017 15.7+ + CUDA module: CUDA 9.0+, Microsoft Visual Studio + OpenMP module: MinGW or Cygwin. The current known issues can be found in the [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues). Additions: + Add paper for Journal of Open Source Software (JOSS). [#479](#479) + Add a DiagonalExtractable interface. [#563](#563) + Add a new diagonal Matrix Format. [#580](#580) + Add Cuda11 support. [#603](#603) + Add information output after CMake configuration. [#610](#610) + Add a new preconditioner export example. [#595](#595) + Add a new cuda-memcheck CI job. [#592](#592) Changes: + Use unified memory in CUDA debug builds. [#621](#621) + Improve `BENCHMARKING.md` with more detailed info. [#619](#619) + Use C++14 standard instead of C++11. [#611](#611) + Update the Ampere sm information and CudaArchitectureSelector. [#588](#588) Fixes: + Fix documentation warnings and errors. [#624](#624) + Fix warnings for diagonal matrix format. [#622](#622) + Fix criterion factory parameters in CUDA. [#586](#586) + Fix the norm-type in the examples. [#612](#612) + Fix the WAW race in OpenMP is_sorted_by_column_index. [#617](#617) + Fix the example's exec_map by creating the executor only if requested. [#602](#602) + Fix some CMake warnings. [#614](#614) + Fix Windows building documentation. [#601](#601) + Warn when CXX and CUDA host compiler do not match. [#607](#607) + Fix reduce_add, prefix_sum, and doc-build. [#593](#593) + Fix find_library(cublas) issue on machines installing multiple cuda. [#591](#591) + Fix allocator in sellp read. [#589](#589) + Fix the CAS with HIP and NVIDIA backends. [#585](#585) Deletions: + Remove unused preconditioner parameter in LowerTrs. [#587](#587) Related PR: #627
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds more detailed information on how to benchmark Ginkgo.
Thanks @adam-m-jcbs for his useful review process for the JOSS paper which pointed to a lack of documentation on how to benchmark Ginkgo. I hope that with this PR we can fix this issue. See #597.
Summary:
ssgetand what to watch out for.run_all_benchmarks.shscript in order to give the proper rights to this file by default (executionxaccess).