Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Verify that concatenating columns does not overflow size_type #6809

Merged
merged 6 commits into from
Dec 2, 2020

Conversation

nvdbaranec
Copy link
Contributor

When concatenating large columns, it is possible to overflow the max size of a column (size_type). We need to verify this won't happen ahead of time. Previously, we were doing it for non-nested types, and only in the fused kernel concatenation case. This addresses both issues.

@nvdbaranec nvdbaranec requested a review from a team as a code owner November 19, 2020 20:17
@GPUtester
Copy link
Collaborator

Please update the changelog in order to start CI tests.

View the gpuCI docs here.

@codecov
Copy link

codecov bot commented Nov 20, 2020

Codecov Report

Merging #6809 (ea074fc) into branch-0.17 (f3c9322) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##           branch-0.17    #6809   +/-   ##
============================================
  Coverage        81.94%   81.94%           
============================================
  Files               96       96           
  Lines            16164    16164           
============================================
  Hits             13246    13246           
  Misses            2918     2918           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3c9322...ea074fc. Read the comment docs.

@kkraus14 kkraus14 added this to PR-WIP in v0.17 Release via automation Nov 23, 2020
@kkraus14 kkraus14 added 3 - Ready for Review Ready for review by team 4 - Needs Review Waiting for reviewer to review or respond libcudf Affects libcudf (C++/CUDA) code. labels Nov 23, 2020
v0.17 Release automation moved this from PR-WIP to PR-Reviewer approved Nov 30, 2020
Copy link
Contributor

@rgsl888prabhu rgsl888prabhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just have a small question.

cpp/src/copying/concatenate.cu Outdated Show resolved Hide resolved
Comment on lines 287 to 292
// these pragmas work around the nvcc issue where if a column_view is used
// inside of a __device__ code path, these functions will end up being created
// as __host__ __device__ because they are explicitly defaulted. However, if
// they then end up being called by a simple __host__ function
// (eg std::vector destructor) you get a compile error because you're trying to
// call a __host__ __device__ function from a __host__ function.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we able to use constexpr for these constructors, instead? that would allow them to be used interchangeably.

@nvdbaranec nvdbaranec added non-breaking Non-breaking change conda improvement Improvement / enhancement to an existing function and removed conda labels Dec 1, 2020
@harrism harrism changed the title [REVIEW] Verify that we will not overflow size_type when concatenating columns. [REVIEW] Verify that concatenating columns does not overflow size_type Dec 2, 2020
@harrism harrism added 6 - Okay to Auto-Merge and removed 3 - Ready for Review Ready for review by team 4 - Needs Review Waiting for reviewer to review or respond labels Dec 2, 2020
@rapids-bot rapids-bot bot merged commit 0ddba3d into rapidsai:branch-0.17 Dec 2, 2020
v0.17 Release automation moved this from PR-Reviewer approved to Done Dec 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
No open projects
v0.17 Release
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

8 participants