Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Data] Add
Dataset.to_dask()
parameter to toggle consistent metadat…
…a check (ray-project#37163) Currently, `Dataset.to_dask()` checks that the metadata be consistent across resulting partitions, via `DaskDataFrame.from_delayed(verify_meta=True)`. This can sometimes raise errors such as ``` ValueError: Metadata mismatch found in `from_delayed`. ``` To improve flexibility in using the method, we expose the `verify_meta` parameter to `Dataset.to_dask()`, which allows the user to skip the aforementioned metadata check. Signed-off-by: Scott Lee <[email protected]>
- Loading branch information