Skip to content

Commit

Permalink
[Data] Fix legacy do_write (ray-project#32661)
Browse files Browse the repository at this point in the history
With the new `write` added (from ray-project#32015 and ray-project#32440), Ray Data intends to support both the `write` and `do_write` functions for now. The check currently uses the `hasattr()` function to ensure the datasource object has a `write` method before using it.

However, this is insufficient for a custom datasource that inherits from `Datasource` since `Datasource` has the `write` method implemented. If the custom datasource only has `do_write` implemented, `hasattr(datasource, "write")` will return True since `hasattr()` will detect methods via inheritance.

The solution is to check if the `write` method was overwritten from `Datasource.write`. Any class that has not implemented `write` will have the equality check return True

Signed-off-by: Edward Oakes <[email protected]>
  • Loading branch information
matthew29tang authored and edoakes committed Mar 22, 2023
1 parent fab30a9 commit ddddc1b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion python/ray/data/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -2693,7 +2693,7 @@ def write_datasource(
soft=False,
)

if hasattr(datasource, "write"):
if type(datasource).write != Datasource.write:
plan = self._plan.with_stage(
OneToOneStage(
"write",
Expand Down

0 comments on commit ddddc1b

Please sign in to comment.