Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace the RemoteWriteQueue and WAL with the exporterhelper queue (sending_queue) in Prometheusremotewriteexporter #33137

Open
Kaivalya1997 opened this issue May 21, 2024 · 2 comments
Labels
enhancement New feature or request exporter/prometheusremotewrite needs triage New item requiring triage Stale

Comments

@Kaivalya1997
Copy link

Kaivalya1997 commented May 21, 2024

Component(s)

exporter/prometheusremotewrite

Is your feature request related to a problem? Please describe.

The Prometheusremotewriteexporter currently does not use the more general sending_queue in the exporterhelper (https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md) developed by the otel collector community for configuring in-memory or persistent queuing. Instead, it has a separate mechanism for setting up a persistent queue using the WAL config, which currently has some bugs (#15277). It also needs a different setting to configure parameters like queue size etc. in the RemoteWriteQueue config. It will instead be much easier to instead replace these two configs in the exporter by the exporterhelper sending_queue as it provides all these features as part of a single sending_queue config, and will also make the Prometheusremotewriteexporter settings conform with most other exporters which are using the exporterhelper queue.

Describe the solution you'd like

I propose that we replace the RemoteWriteQueue and WAL in the Prometheusremotewriteexporter config with exporterhelper.QueueSettings (sending_queue) so as to make the config better conform with most other exporters which are using the sending_queue. This will also remove the effort of separately maintaining the queuing related code for Prometheusremotewriteexporter, as the exporterhelper queue is maintained by the wider otel collector community.

Describe alternatives you've considered

No response

Additional context

I have made changes to the Prometheusremotewriteexporter code locally to use the exporterhelper queue instead of the RemoteWriteQueue and WAL and tested it. The changes do not require extensive code changes as exporterhelper.QueueSettings imports all the necessary code for the exporterhelper queue. The only changes needed would be in the config.go file to add some validations for the queue and some changes would be needed in the exporter.go file to remove the WAL related and RemoteWriteQueue related code, which should be straightforward.

@Kaivalya1997 Kaivalya1997 added enhancement New feature or request needs triage New item requiring triage labels May 21, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request exporter/prometheusremotewrite needs triage New item requiring triage Stale
Projects
None yet
Development

No branches or pull requests

1 participant