Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-6431] Implement Execution Time metrics start,process,finish in the Java SDK #7676

Merged
merged 6 commits into from
Feb 5, 2019

Conversation

ajamato
Copy link

@ajamato ajamato commented Jan 30, 2019

[BEAM-6431] Implement Execution Time metrics start,process,finish in the Java SDK

Please add a meaningful description for your change here


Follow this checklist to help us incorporate your contribution quickly and easily:

  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Python Build Status --- Build Status
Build Status
Build Status --- --- ---

@ajamato ajamato force-pushed the execution_metric_implementation branch from e02fd40 to 3c4b47c Compare January 31, 2019 23:36
@ajamato ajamato force-pushed the execution_metric_implementation branch from 3c4b47c to 2aa8a9f Compare February 4, 2019 19:40
@ajamato
Copy link
Author

ajamato commented Feb 5, 2019

@ryan-williams @Ardagan
#7676
This PR is ready for review now. Uses the ExecutionStateTracker in the Java SDK to collect PTransformExecutionTimeRegistry.

I also started using matches for testing MonitoringInfos, we can add more matchers over time, this eliminates the need to cleanup/clear timestamps, values, etc when testing monitoring infos.

The usage of the states and state tracker mostly happen inside the PCollectionConsumerRegistry and PTransformFunctionRegistry. Once again I have changed its signature and ctors, which affects many tests which are using them (due to the repeated patterns in tests)

PTAL and let me know what you think of these changes.

@ajamato ajamato changed the title DO NOT SUBMIT [BEAM-6431] Implement Execution Time metrics start,process,finish in the Java SDK [BEAM-6431] Implement Execution Time metrics start,process,finish in the Java SDK Feb 5, 2019
@swegner swegner merged commit 696cf22 into apache:master Feb 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants