Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support Polars datetime round expression #16226

Open
Tracked by #16481
beckernick opened this issue Jul 8, 2024 · 3 comments
Open
Tracked by #16481

[FEA] Support Polars datetime round expression #16226

beckernick opened this issue Jul 8, 2024 · 3 comments
Labels
cudf.polars Issues specific to cudf.polars feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.

Comments

@beckernick
Copy link
Member

beckernick commented Jul 8, 2024

We should eventually support plans that include datetime round.

import polars as pl
from functools import partial
from cudf_polars.callback import execute_with_cudf
from datetime import timedelta, datetime

use_cudf = partial(execute_with_cudf, raise_on_fail=True) # for testing

start = datetime(2001, 1, 1)
stop = datetime(2001, 1, 2)
s = pl.datetime_range(
    start, stop, timedelta(minutes=165), eager=True
).alias("datetime")
df = s.to_frame().lazy()

print(df.select(pl.col("datetime").dt.round("1h")).collect())
print(df.select(pl.col("datetime").dt.round("1h")).collect(post_opt_callback=use_cudf))
shape: (9, 1)
┌─────────────────────┐
│ datetime            │
│ ---                 │
│ datetime[μs]        │
╞═════════════════════╡
│ 2001-01-01 00:00:00 │
│ 2001-01-01 03:00:00 │
│ 2001-01-01 06:00:00 │
│ 2001-01-01 08:00:00 │
│ 2001-01-01 11:00:00 │
│ 2001-01-01 14:00:00 │
│ 2001-01-01 17:00:00 │
│ 2001-01-01 19:00:00 │
│ 2001-01-01 22:00:00 │
└─────────────────────┘
---------------------------------------------------------------------------
ComputeError                              Traceback (most recent call last)
Cell In[46], line 18
     15 df = s.to_frame().lazy()
     17 print(df.select(pl.col("datetime").dt.round("1h")).collect())
---> 18 print(df.select(pl.col("datetime").dt.round("1h")).collect(post_opt_callback=use_cudf))

File [/raid/nicholasb/miniconda3/envs/all_cuda-122_arch-x86_64/lib/python3.11/site-packages/polars/lazyframe/frame.py:1942](http://10.117.23.184:8882/lab/tree/raid/nicholasb/raid/nicholasb/miniconda3/envs/all_cuda-122_arch-x86_64/lib/python3.11/site-packages/polars/lazyframe/frame.py#line=1941), in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, cluster_with_columns, no_optimization, streaming, background, _eager, **_kwargs)
   1939 # Only for testing purposes atm.
   1940 callback = _kwargs.get("post_opt_callback")
-> 1942 return wrap_df(ldf.collect(callback))

ComputeError: 'cuda' conversion failed: TypeError: cannot unpack non-iterable builtins.TemporalFunction object
@beckernick beckernick added feature request New feature or request cudf.polars Issues specific to cudf.polars labels Jul 8, 2024
@wence-
Copy link
Contributor

wence- commented Jul 9, 2024

Easy to do, but also will want pola-rs/polars#17518

@vyasr
Copy link
Contributor

vyasr commented Jul 15, 2024

Just noting that the Polars PR linked above is merged.

@lithomas1
Copy link
Contributor

OK, this one is a bit annoying.

libcudf uses std::chrono::round which does banker's rounding
(i.e. round to even number when the value to round is in the middle of a frequency)

I think we'll need them to implement HALF_UP rounding for this to work well.

@lithomas1 lithomas1 added the libcudf Affects libcudf (C++/CUDA) code. label Aug 2, 2024
@lithomas1 lithomas1 removed their assignment Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cudf.polars Issues specific to cudf.polars feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.
Projects
Status: In Progress
Development

No branches or pull requests

4 participants