-
Notifications
You must be signed in to change notification settings - Fork 435
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[VL] Fix timestamp precision loss in serializer #5376
Conversation
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format?
See also: |
BTW, from this setting's comments, it's used for spilling also, not sure if this should be set to true for spilling also? |
Thanks for this fix. Looks like it's already set to true in Velox Spill https://github.com/facebookincubator/velox/blob/b1252f26e112e9632e5a3e11df536387282c0259/velox/exec/SpillFile.cpp#L197-L198 cc: @zhztheplayer |
a1f50ac
to
68c3bdc
Compare
Get it. Thanks for your info. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! cc @zhztheplayer
What changes were proposed in this pull request?
By default
PrestoVectorSerde
use millisecond precision for timestamp data. We observed timestamp prescision loseing when doing broadcast serialization. We need to fix this by settinguseLosslessTimestamp
forPrestoVectorSerde
.Wrong result:
Expected result:
How was this patch tested?
UT