Wire error handling into PubSubIO and add initial tests #30372
Conversation
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
Assigning reviewers. If you would like to opt out of this review, comment R: @m-trieu for label java. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
| read = preParse.apply(MapElements.into(typeDescriptor).via(getParseFn())); | ||
| } else { | ||
| // parse PubSub messages, separating out execptions |
| message = message.withTopic(topicFunction.apply(valueInSingleWindow).asPath()); | ||
| } catch (Exception e) { | ||
| badRecordRouter.route( | ||
| o, element, inputCoder, e, "Failed to determine PubSub topic using topic function"); |
There was a problem hiding this comment.
Should we just add valueInSingleWindow into the error message here so its easier to see what went wrong?
There was a problem hiding this comment.
In this case, I'm adding the input element, which is what should be meaningful to the user
| element, | ||
| inputCoder, | ||
| new IllegalArgumentException(e), | ||
| "PubSub message size limit exceeded"); |
There was a problem hiding this comment.
nit: to avoid confusion, can we make this message more generic? Looking at the validation code, it can fail for reasons other than message size (e.g. attribute key/value size too large)
* Wire error handling into PubSubIO and add initial tests * update comments to reflect that schema mismatches are not routed to DLQs * spotless * address nits
Please add a meaningful description for your change here
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.