Skip to content

Commit

Permalink
[FLINK-11473][metrics][docs] Clarify documentation on Latency Tracking
Browse files Browse the repository at this point in the history
  • Loading branch information
knaufk authored and zentol committed Jan 31, 2019
1 parent 568d90c commit 999a04d
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions docs/monitoring/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -1628,16 +1628,18 @@ bypassing them. In particular the markers are not accounting for the time record
Only if operators are not able to accept new records, thus they are queuing up, the latency measured using
the markers will reflect that.

All intermediate operators keep a list of the last `n` latencies from each source to compute
a latency distribution.
The sink operators keep a list from each source, and each parallel source instance to allow detecting
latency issues caused by individual machines.
The `LatencyMarker`s are used to derive a distribution of the latency between the sources of the topology and each
downstream operator. These distributions are reported as histogram metrics. The granularity of these distributions can
be controlled in the [Flink configuration]({{ site.baseurl }}/ops/config.html#metrics-latency-interval. For the highest
granularity `subtask` Flink will derive the latency distribution between every source subtask and every downstream
subtask, which results in quadratic (in the terms of the parallelism) number of histograms.

Currently, Flink assumes that the clocks of all machines in the cluster are in sync. We recommend setting
up an automated clock synchronisation service (like NTP) to avoid false latency results.

<span class="label label-danger">Warning</span> Enabling latency metrics can significantly impact the performance
of the cluster. It is highly recommended to only use them for debugging purposes.
of the cluster (in particular for `subtask` granularity). It is highly recommended to only use them for debugging
purposes.

## REST API integration

Expand Down

0 comments on commit 999a04d

Please sign in to comment.