Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-9670] Fix nullability widening in CoGroup key resolution #11290

Merged
merged 2 commits into from
Apr 2, 2020

Conversation

reuvenlax
Copy link
Contributor

This PR also adds some zetasql unit tests that expose the previous bug.

@reuvenlax
Copy link
Contributor Author

run sql postcommit

@reuvenlax
Copy link
Contributor Author

Run Java PreCommit

+ leftSchema
+ " rhsSchema: "
+ rightSchema);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we still verify that the schemas have equivalent types without nullability?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The merge function itself now verifies this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. If there aren't any tests verifying this already could you add one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a test in CoGroupTest.java that verifies this. Should I add an additional one here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be nice to have something in ZetaSQLDialectSpecTest to verify the set operation just in case we break the logic there and not in CoGroup, but I guess it's fine either way

default:
result = fieldType1;
}
return result.withNullable(fieldType1.getNullable() || fieldType2.getNullable());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a general comment: do you think we should add a visitor pattern for this sort of thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe? I'm not sure visitor per se, as we don't require double dispatch. However we might consider generalizing the recursive tree walk.

Copy link
Member

@TheNeuralBit TheNeuralBit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I also ran some internal testing which looks good

@reuvenlax
Copy link
Contributor Author

@TheNeuralBit thanks. I'll merge once tests complete

@reuvenlax
Copy link
Contributor Author

Run Java PreCommit

1 similar comment
@reuvenlax
Copy link
Contributor Author

Run Java PreCommit

@reuvenlax reuvenlax merged commit 5504f32 into apache:master Apr 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants