-
Notifications
You must be signed in to change notification settings - Fork 689
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Write parquet statistics for IntervalDayTimeArray
, IntervalMonthDayNanoArray
and IntervalYearMonthArray
#5847
Labels
enhancement
Any new improvement worthy of a entry in the changelog
parquet
Changes to the parquet crate
Comments
alamb
added
enhancement
Any new improvement worthy of a entry in the changelog
parquet
Changes to the parquet crate
labels
Jun 5, 2024
for additional context and possibly code pointers where Interval support is missing. |
@marvinlanhenke also points out that to completely support this ticket probably requires completing #5849 first |
It is incorrect to write statistics for interval types, as per the format specification - https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#interval |
That makes sense. Thank you @tustvold |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
enhancement
Any new improvement worthy of a entry in the changelog
parquet
Changes to the parquet crate
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As we worked on filling out support for converting parquet statistics into ArrayRefs in DataFusion (see apache/datafusion#10453) @marvinlanhenke discovered that
Describe the solution you'd like
Support writing the following types of data with statistics in the parquet ArrowWriter
This might be the location:
arrow-rs/parquet/src/arrow/arrow_writer/mod.rs
Line 747 in fa8d350
Describe alternatives you've considered
Additional context
apache/datafusion#10801 (comment)
The text was updated successfully, but these errors were encountered: