Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-7916] Add ElasticsearchIO query parameter to take a ValueProvider #9285

Merged
merged 3 commits into from
Aug 8, 2019
Merged

[BEAM-7916] Add ElasticsearchIO query parameter to take a ValueProvider #9285

merged 3 commits into from
Aug 8, 2019

Conversation

oliverhenlich
Copy link
Contributor

@oliverhenlich oliverhenlich commented Aug 7, 2019

We need to be able to perform Elasticsearch queries that are dynamic. The problem is ElasticsearchIO.read().withQuery() only accepts a string which means the query must be known when the pipleline/Google Dataflow Template is built.

This change changes the withQuery() parameter from String to ValueProvider<String>.

R: @timrobertson100
R: @jbonofre
R: @echauchot


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • @jbonofre, @echauchot
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- Build Status --- --- Build Status
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status
--- Build Status
Build Status
Build Status --- --- Build Status

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@RyanSkraba
Copy link
Contributor

For info: @jbonofre and @echauchot are not available this week, you might want to find an alternate reviewer.

For what it's worth, LGTM but I would highly recommend the suggested change for compatibility.

@oliverhenlich
Copy link
Contributor Author

Thanks @RyanSkraba.
The only other person in the OWNERS file is @timrobertson100. Is that ok? How would I go about trying to find another reviewer?

@aromanenko-dev aromanenko-dev changed the title [BEAM-7916] - Change ElasticsearchIO query parameter to be a ValueProvider [BEAM-7916] Add ElasticsearchIO query parameter to be a ValueProvider Aug 8, 2019
@aromanenko-dev aromanenko-dev changed the title [BEAM-7916] Add ElasticsearchIO query parameter to be a ValueProvider [BEAM-7916] Add ElasticsearchIO query parameter to take a ValueProvider Aug 8, 2019
Copy link
Contributor

@aromanenko-dev aromanenko-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contribution! LGTM after @RyanSkraba requested changes.

@aromanenko-dev aromanenko-dev merged commit c545b8a into apache:release-2.14.0 Aug 8, 2019
@oliverhenlich
Copy link
Contributor Author

Hi @aromanenko-dev,
Thanks for merging this! I just realized that it's been merged into 2.14.0 which has already been released. Will that make it into the next version? What is the next version? On the Jira it says 2.16.0.

aromanenko-dev added a commit that referenced this pull request Aug 9, 2019
@aromanenko-dev
Copy link
Contributor

aromanenko-dev commented Aug 9, 2019

@oliverhenlich Yes, my bad, I didn't pay an attention that this PR was against release-2.14.0 branch instead of master. Thank you for pointing on this.
I'll revert this commit (since release 2.14 is already done) and could you create another PR against master branch?

aromanenko-dev pushed a commit to aromanenko-dev/beam that referenced this pull request Aug 12, 2019
@oliverhenlich
Copy link
Contributor Author

Hi @aromanenko-dev, sorry for the delay! Just got back to this. It looks like you've already taken care of this with PR: #9305 and #9314. Do I still need to do something?

What version would this fix then appear in 2.15?

PS: I noticed the last PR #9314 is failing but I don't think it has anything to do with my change.

@aromanenko-dev
Copy link
Contributor

Hi @oliverhenlich, yes, I reverted commit in release-2.14.0 and created new PR against master (I cherry-picked your commit, so your credits should be kept). So, I think there is nothing to do for now from your side. This fix will appear in 2.16 since 2.15 branch was already cut 2 weeks ago.

PS: Yes, you are right, it was not related to your changes, thanks to @RyanSkraba to rerun and it's ok now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants