-
Notifications
You must be signed in to change notification settings - Fork 707
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug(parquet): Disabling global statistics but enabling for particular column breaks reading #4587
Comments
tustvold
added a commit
that referenced
this issue
Aug 1, 2023
* Test disabling page index statistics (#4587) * Apply suggestions from code review Co-authored-by: Andrew Lamb <[email protected]> --------- Co-authored-by: Andrew Lamb <[email protected]>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If I write files with:
When I query it with datafusion or just
parquet::ParquetRecordBatchReaderBuilder
, it errors with: "missing offset index"Seems like it is skipping writing offset indices if page statistics are globally disabled?
I would expect, if it doesn't write offset indices then it shouldn't try to filter pages by statistics, also it should be documented that
set_column_statistics_enabled
doesn't override global settings in this way.The text was updated successfully, but these errors were encountered: