Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prometheusexporter: metrics plummeted without prior notification. #30751

Closed
shaj13 opened this issue Jan 24, 2024 · 4 comments
Closed

prometheusexporter: metrics plummeted without prior notification. #30751

shaj13 opened this issue Jan 24, 2024 · 4 comments

Comments

@shaj13
Copy link

shaj13 commented Jan 24, 2024

Component(s)

No response

What happened?

Description

If the client sends, for instance, sum metrics without configuring AggregationTemporality, which is evidently incorrect, it leads to the otel prometheusexporter.Accumulator discarding those metrics without any prior notification.

example -> https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/prometheusexporter/accumulator.go#L168

Steps to Reproduce

send metrics without setting AggregationTemporality.

Expected Result

Warn/Info Log describes the dropped metrics including a reason.

Actual Result

The absence of logs necessitates debugging the entire metrics pipeline, encompassing otel and clients.

Collector version

latest

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

@shaj13 shaj13 added bug Something isn't working needs triage New item requiring triage labels Jan 24, 2024
Copy link
Contributor

Pinging code owners for exporter/prometheus: @Aneurysm9. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

I agree that it's good to log in this case to let users know why metrics are being dropped, but I would worry about log spam if it's anything more than debug. This isn't really an error or anything wrong in the collector, so it may be better suited to a lower log level.

If this log message occurs what can be done to resolve the issue? It seems like it would be useful the first time it's seen when debugging, but after that it would be known to the user and then just fill logs from then on.

@crobert-1 crobert-1 removed the needs triage New item requiring triage label Jan 25, 2024
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Mar 26, 2024
Copy link
Contributor

This issue has been closed as inactive because it has been stale for 120 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants