Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Behavior of output_required and materialization of downstream assets is unexpected #25958

Open
cmpadden opened this issue Nov 15, 2024 · 0 comments
Labels
type: bug Something isn't working

Comments

@cmpadden
Copy link
Contributor

cmpadden commented Nov 15, 2024

What's the issue?

From community member Mykola Palamarchuk.

­This part of documentation says that you can return None , so upstream assets will be skipped. But it doesn't work as described (latest Dagster 1.8.11).

from dagster import asset, Definitions, define_asset_job

@asset(output_required=False)
def x():
    return None

@asset
def plus1(x):  # should never materialize
    return x+1 


the_job = define_asset_job(
    name="the_job",
    selection=['*plus1'],
)

defs = Definitions(
    assets=[x, plus1],
    jobs=[the_job]
)

Materializing the_job leads to plus1 asset materialization with error unsupported operand type(s) for +: 'NoneType' and 'int' .
There is also a warning that says:

Value "None" returned for non-required output "result" of op "x". This value will be passed to downstream ops. For conditional execution, results must be yielded: ­https://docs.dagster.io/concepts/ops-jobs-graphs/graphs#with-conditional-branching

See original discussion in community Slack:

What did you expect to happen?

The expected behavior aligns with the docs of ​output­_required​.

Whether the decorated function will always materialize an asset. Defaults to True. If False, the function can return None, which will not be materialized to storage and will halt execution of downstream assets.

How to reproduce?

See example.

Dagster version

1.8.11

Deployment type

Local

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
By submitting this issue, you agree to follow Dagster's Code of Conduct.

@cmpadden cmpadden added the type: bug Something isn't working label Nov 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant