Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus exporter handles delta metrics differently from the current spec #5015

Closed
dashpole opened this issue Dec 2, 2022 · 2 comments · Fixed by #5062
Closed

Prometheus exporter handles delta metrics differently from the current spec #5015

dashpole opened this issue Dec 2, 2022 · 2 comments · Fixed by #5062
Labels
Bug Something isn't working

Comments

@dashpole
Copy link
Contributor

dashpole commented Dec 2, 2022

Describe the bug

The prometheus exporter appears to convert sums with aggregation temporality delta to gauges, and appears to ignore temporality for histograms.

What did you expect to see?

The spec for Sums says:

OpenTelemetry Sums follows this logic:

  • If the aggregation temporality is cumulative and the sum is monotonic, it MUST be converted to a Prometheus Counter.
  • If the aggregation temporality is cumulative and the sum is non-monotonic, it MUST be converted to a Prometheus Gauge.
  • If the aggregation temporality is delta and the sum is monotonic, it SHOULD be converted to a cumulative temporality and become a Prometheus Sum. The following behaviors are expected:
    • The new data point type must be the same as the accumulated data point type.
    • The new data point's start time must match the time of the accumulated data point. If not, see detecting alignment issues.
    • Otherwise, it MUST be dropped.

The spec for Histograms says:

OpenTelemetry Histograms with Delta aggregation temporality SHOULD be aggregated into a Cumulative aggregation temporality and follow the logic above, or MUST be dropped.

Basically, we need to either convert delta -> cumulative, or drop the metrics.

It is also definitely possible that i'm mistaken and it isn't possible for delta metrics to be passed to the Prometheus exporter. In that case, we should probably remove the aggregation temporality check from the metric -> type logic.

@dashpole dashpole added the Bug Something isn't working label Dec 2, 2022
@jack-berg
Copy link
Member

It is also definitely possible that i'm mistaken and it isn't possible for delta metrics to be passed to the Prometheus exporter.

This. Cumulative temporality for all instruments is specified here. It will be possible for MetricProducers other than the SDK to ignore the temporality, but the Prometheus reader won't be required to do anything about it:

The MetricReader is not required to ensure data points from a non-SDK MetricProducer are output in the configured aggregation temporality, as these data points are not collected using OpenTelemetry instruments.

@dashpole
Copy link
Contributor Author

dashpole commented Dec 2, 2022

Thanks. Based on the prometheus spec above, I would still expect delta temporality metrics to be dropped in the exporter. At minimum, I wouldn't expect delta sums to be converted to gauges. But this is moot until after #5011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants