Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[statsdreceiver] Counters are marked as monotonic #1789

Closed
mx-psi opened this issue Dec 8, 2020 · 4 comments
Closed

[statsdreceiver] Counters are marked as monotonic #1789

mx-psi opened this issue Dec 8, 2020 · 4 comments

Comments

@mx-psi
Copy link
Member

mx-psi commented Dec 8, 2020

Describe the bug

Statsd count metrics are being reported as monotonic when they are not. This can cause other components to incorrectly assume monotonicity (currently the SignalFx exporter and in the future the Datadog exporter).

Steps to reproduce

Run an OpenTelemetry Collector with the statsd receiver and send a count metric to it.

What did you expect to see?

Statsd metrics being reported as non-monotonic.

What did you see instead?

Statsd metrics being reported as monotonic. For a metric sent as counter.name:1|c.

Metric #28
Descriptor:
     -> Name: counter.name
     -> Description:
     -> Unit:
     -> DataType: IntSum
     -> IsMonotonic: true
     -> AggregationTemporality: AGGREGATION_TEMPORALITY_CUMULATIVE
IntDataPoints #0
StartTime: 0
Timestamp: 1607442068000000000
Value: 1

What version did you use?
313a5a5

What config did you use?
Config:

receivers:
  statsd:
    endpoint: "localhost:8125"

exporters:
  logging:
    loglevel: debug

service:
  pipelines:
    metrics:
      receivers: [statsd]
      exporters: [logging]

Environment
OS: macOS 10.15
Compiler(if manually compiled): go 1.15.5

Additional context

In StatsD, counters are not monotonic and work like deltas that are aggregated on the server side.

From b/statsd_spec:

A counter is a gauge calculated at the server. Metrics sent by the client increment or decrement the value of the gauge rather than giving its current value.

From statsd/statsd:

gorets:1|c This is a simple counter. Add 1 to the "gorets" bucket. At each flush the current count is sent and reset to 0.

They can increment or decrement and the instrumented application should report these increments or decrements rather than the whole value.

In the statsd receiver they are set as cumulative OpenCensus metrics:

case "c":
return buildCounterPoint(parsedMetric, now, func(parsedMetric *statsDMetric) {
parsedMetric.metricType = metricspb.MetricDescriptor_CUMULATIVE_INT64
})

Cumulative values are described by the OpenCensus proto as:

Floating [resp. Integer] point cumulative measurement. The value cannot decrease [...].
Recorded values are always >= 0.

The OpenCensus translator then maps this type into an IntSum with cumulative aggregation and monotonic.

I am not sure if they should be marked as cumulative either, given the OpenTelemetry proto description:

aggregation_temporality describes if the aggregator reports delta changes
since last report time, or cumulative changes since a fixed start time.

but this seems to be more common (e.g. some components on the core repo do this too, like the hostmetrics receiver), so I am not sure if this last part should be considered a bug.

@mx-psi mx-psi added the bug Something isn't working label Dec 8, 2020
@mx-psi
Copy link
Member Author

mx-psi commented Dec 9, 2020

cc @keitwb @jmacd since you are CODEOWNERS of this component.

@gavindoudou
Copy link
Contributor

gavindoudou commented Jan 12, 2021

Thanks for that, the new version has changed the type for counter already: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/master/receiver/statsdreceiver/protocol/statsd_parser.go#L220
Please take a look if the issue can be resolved.

@mx-psi
Copy link
Member Author

mx-psi commented Jan 22, 2021

Closing since I can no longer reproduce the issue. Thanks for the fix!

@mx-psi mx-psi closed this as completed Jan 22, 2021
@jmacd
Copy link
Contributor

jmacd commented Jan 22, 2021

Hmmm. I think the correct output point for a Statsd counter should be the non-monotonic delta sum, not the gauge, but we can revisit this.

dyladan referenced this issue in dynatrace-oss-contrib/opentelemetry-collector-contrib Jan 29, 2021
ljmsc referenced this issue in ljmsc/opentelemetry-collector-contrib Feb 21, 2022
* Add k8s.node.name and k8s.node.uid to semconv

According to the specification[1] they should exist.

[1] https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/resource/semantic_conventions/k8s.md#node

* Update changelog

* Update semconv/resource.go

Co-authored-by: Tyler Yahn <[email protected]>

* Update semconv/resource.go

Co-authored-by: Tyler Yahn <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Tyler Yahn <[email protected]>

* Update semconv/resource.go

Co-authored-by: Tyler Yahn <[email protected]>

Co-authored-by: Tyler Yahn <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants