Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement compare operations for view types #5900

Merged
merged 6 commits into from
Jun 17, 2024

Conversation

XiangpengHao
Copy link
Contributor

Which issue does this PR close?

Closes #5897 .

Rationale for this change

Now we can compare two view type arrays.

Unfortunately, this PR becomes larger than I thought, partly because the test cases are rather comprehensive and the comparison is non-trivial.

The view type comparison code might be slightly more esoteric than one might expected, I hope the comments could help understanding/justify the complexity.

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added the arrow Changes to the arrow crate label Jun 16, 2024
alamb
alamb previously approved these changes Jun 17, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this PR becomes larger than I thought, partly because the test cases are rather comprehensive and the comparison is non-trivial.

👍

This PR is really nicely done in my opinion @XiangpengHao 🏆

The only things I think are required prior to merging is some additional test coverage (I left a note about what I think i smissing)

I also left some suggestions that might make the code easier to read, but I don't think they are necessary

Thanks again

cc @ariesdevil and @Weijun-H

@@ -538,6 +540,135 @@ impl<'a, T: ByteArrayType> ArrayOrd for &'a GenericByteArray<T> {
}
}

/// Comparing two ByteView types are non-trivial.
/// It takes a bit of patience to understand why we don't just compare two &[u8] directly.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😮 🏆 🥇 for the comments

arrow-ord/src/cmp.rs Show resolved Hide resolved
arrow-ord/src/cmp.rs Outdated Show resolved Hide resolved
arrow-ord/src/cmp.rs Show resolved Hide resolved
arrow-ord/src/comparison.rs Outdated Show resolved Hide resolved
arrow-ord/src/comparison.rs Show resolved Hide resolved
@alamb alamb dismissed their stale review June 17, 2024 17:38

Need some additional test coverage prior to merging

@alamb
Copy link
Contributor

alamb commented Jun 17, 2024

I also filed #5903 to track adding benchmarks for these kernels

@XiangpengHao
Copy link
Contributor Author

Added more tests and refactored a bit to make it more clear! can you take a look again? @alamb

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @XiangpengHao

@alamb alamb merged commit d0a88c6 into apache:master Jun 17, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement compare_op for GenericBinaryView
2 participants