Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exporter Feature: OpenSearch Metrics #30556

Closed
edmondsiu0 opened this issue Jan 15, 2024 · 8 comments
Closed

Exporter Feature: OpenSearch Metrics #30556

edmondsiu0 opened this issue Jan 15, 2024 · 8 comments

Comments

@edmondsiu0
Copy link

Component(s)

exporter/opensearch

Is your feature request related to a problem? Please describe.

Similar to issue #23611, this Issue is to suggest OpenSearch exporter to convert Otel metric data directly into OpenSearch.

This will remove the need of using dataprepper otel_metrics processor to reprocess the information before being sent into OpenSearch.

Describe the solution you'd like

Similar to the linked issue, but specific to metrics, such as:

exporters:
  opensearch/metrics:
    http:
      endpoint: https://opensearch.example.com:9200
    index: my-metrics-index

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [opensearch/metrics]

Describe alternatives you've considered

Dataprepper otel_metrics processor is able to perform this task but it is an external component. It would be good if this can be transformed directly within Otel.

Additional context

No response

@edmondsiu0 edmondsiu0 added enhancement New feature or request needs triage New item requiring triage labels Jan 15, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

Just to confirm my understanding, this issue is a request for the OpenSearch exporter to add support for metrics. You're also saying that for this to work, it has to convert metrics into a format acceptable to OpenSearch. Am I understanding your request properly?

@edmondsiu0
Copy link
Author

That is correct, thanks!

@crobert-1
Copy link
Member

Does OpenSearch support any metric format other than Prometheus? I'm unfamiliar but trying to figure out if there are possible alternative options here.

From the help page I referenced, it looks like you may actually be able to just point OpenSearch at a collector instance using the Prometheus exporter. Would that work for your use case or am I missing something?

@scott-parillo
Copy link

We're in the very early phases of looking closely at Open Search as an alternative to Elastic and was just about to create a similar request when I found this entry. I know that Prometheus exporter could get the job done but we're hoping to avoid adding another service/backend to our solution.

Any idea how much effort it would take to enhance this exporter w/ metrics?

@crobert-1
Copy link
Member

I know that Prometheus exporter could get the job done but we're hoping to avoid adding another service/backend to our solution.

Out of ignorance, would adding the Prometheus exporter to your collector configuration require more of a backend change than adding the OpenSearch exporter for metrics?

How would the functionality of an OpenSearch exporter that supports metrics differ from the Prometheus exporter? What functionality is missing that you're looking for?

Converting metrics to the Prometheus format in the collector is quite a bit of effort and prone to bugs, so it would be good to avoid duplicating this effort as much as possible.

Copy link
Contributor

github-actions bot commented Apr 8, 2024

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

github-actions bot commented Jun 7, 2024

This issue has been closed as inactive because it has been stale for 120 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants