[receiver/mongodbatlas] Fix issue where storage client can timeout if both events and alerts are configured

[schmikei]

Description: Changes so that we only try to retrieve the storageClient once per start of the collector. Before they could compete for access and leave one timing out trying to access a filestorage extension.

Link to tracking Issue: Resolves #19434

Testing:

Tested with this config

receivers:
  mongodbatlas:
    public_key: <redacted>
    private_key: <redacted>
    alerts:
      enabled: true
      mode: poll
      projects:
      - name: Project 1
        include_clusters: [Cluster0]
      - name: Project 2
        include_clusters: [Cluster0]
      poll_interval: 5s
    events:
      projects:
        - name: "Project 1"
      poll_interval: 1m
    storage: file_storage

extensions:
  file_storage:
    directory: $OTEL_COLLECTOR_HOME/storage

exporters:
  logging:
service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers:
      - mongodbatlas
      exporters:
      - logging

Documentation:

[runforesight]

Foresight Summary

    
Major Impacts

build-and-test duration(33 minutes 2 seconds) has decreased 35 minutes 1 second compared to main branch avg(1 hour 8 minutes 3 seconds).

View More Details

⭕  build-and-test-windows workflow has finished in 5 seconds (43 minutes 37 seconds less than main branch avg.) and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
windows-unittest-matrix	-     🔗	N/A	See Details
windows-unittest	-     🔗	N/A	See Details

---

✅  check-links workflow has finished in 50 seconds (1 minute 11 seconds less than main branch avg.) and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
changed files	-     🔗	N/A	See Details
check-links	-     🔗	N/A	See Details

---

✅  telemetrygen workflow has finished in 1 minute 4 seconds (1 minute 33 seconds less than main branch avg.) and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
build-dev	-     🔗	N/A	See Details
publish-latest	-     🔗	N/A	See Details
publish-stable	-     🔗	N/A	See Details

---

✅  changelog workflow has finished in 2 minutes 31 seconds and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
changelog	-     🔗	N/A	See Details

---

✅  build-and-test workflow has finished in 33 minutes 2 seconds (35 minutes 1 second less than main branch avg.) and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
unittest-matrix (1.19, connector)	-     🔗	✅ 100  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, connector)	-     🔗	✅ 100  ❌ 0  ⏭ 0    🔗	See Details
correctness-metrics	-     🔗	✅ 2  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, internal)	-     🔗	✅ 594  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, internal)	-     🔗	✅ 594  ❌ 0  ⏭ 0    🔗	See Details
correctness-traces	-     🔗	✅ 17  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, extension)	-     🔗	✅ 538  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, processor)	-     🔗	✅ 1546  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, extension)	-     🔗	✅ 538  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, processor)	-     🔗	✅ 1546  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, receiver-0)	-     🔗	✅ 2580  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, receiver-0)	-     🔗	✅ 2580  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, exporter)	-     🔗	✅ 2474  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, exporter)	-     🔗	✅ 2474  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, receiver-1)	-     🔗	✅ 1940  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, receiver-1)	-     🔗	✅ 1940  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, other)	-     🔗	✅ 4728  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.20, other)	-     🔗	✅ 4728  ❌ 0  ⏭ 0    🔗	See Details
integration-tests	-     🔗	✅ 55  ❌ 0  ⏭ 0    🔗	See Details
setup-environment	-     🔗	N/A	See Details
check-collector-module-version	-     🔗	N/A	See Details
lint-matrix (receiver-0)	-     🔗	N/A	See Details
lint-matrix (receiver-1)	-     🔗	N/A	See Details
lint-matrix (processor)	-     🔗	N/A	See Details
lint-matrix (exporter)	-     🔗	N/A	See Details
lint-matrix (extension)	-     🔗	N/A	See Details
lint-matrix (connector)	-     🔗	N/A	See Details
lint-matrix (internal)	-     🔗	N/A	See Details
lint-matrix (other)	-     🔗	N/A	See Details
check-codeowners	-     🔗	N/A	See Details
build-examples	-     🔗	N/A	See Details
checks	-     🔗	N/A	See Details
lint	-     🔗	N/A	See Details
unittest (1.20)	-     🔗	N/A	See Details
unittest (1.19)	-     🔗	N/A	See Details
cross-compile (darwin, amd64)	-     🔗	N/A	See Details
cross-compile (darwin, arm64)	-     🔗	N/A	See Details
cross-compile (linux, 386)	-     🔗	N/A	See Details
cross-compile (linux, amd64)	-     🔗	N/A	See Details
cross-compile (linux, arm)	-     🔗	N/A	See Details
cross-compile (linux, arm64)	-     🔗	N/A	See Details
cross-compile (linux, ppc64le)	-     🔗	N/A	See Details
cross-compile (windows, 386)	-     🔗	N/A	See Details
cross-compile (windows, amd64)	-     🔗	N/A	See Details
build-package (deb)	-     🔗	N/A	See Details
build-package (rpm)	-     🔗	N/A	See Details
windows-msi	-     🔗	N/A	See Details
publish-check	-     🔗	N/A	See Details
publish-stable	-     🔗	N/A	See Details
publish-dev	-     🔗	N/A	See Details

---

✅  prometheus-compliance-tests workflow has finished in 3 minutes 14 seconds (5 minutes 6 seconds less than main branch avg.) and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
prometheus-compliance-tests	-     🔗	✅ 21  ❌ 0  ⏭ 0    🔗	See Details

---

✅  load-tests workflow has finished in 7 minutes 21 seconds (7 minutes 46 seconds less than main branch avg.) and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
loadtest (TestTraceAttributesProcessor)	-     🔗	✅ 3  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestIdleMode)	-     🔗	✅ 1  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile)	-     🔗	✅ 6  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs)	-     🔗	✅ 8  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS)	-     🔗	✅ 12  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs)	-     🔗	✅ 10  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestBallastMemory|TestLog10kDPS)	-     🔗	✅ 18  ❌ 0  ⏭ 0    🔗	See Details
setup-environment	-     🔗	N/A	See Details

---

✅  e2e-tests workflow has finished in 15 minutes 2 seconds and finished at 9th Mar, 2023.

---

Job	Failed Steps	Tests
kubernetes-test (v1.26.0)	-     🔗	N/A	See Details
kubernetes-test (v1.25.3)	-     🔗	N/A	See Details
kubernetes-test (v1.24.7)	-     🔗	N/A	See Details
kubernetes-test (v1.23.13)	-     🔗	N/A	See Details

---

🔎 See details on Foresight

^{*You can configure Foresight comments in your organization settings page.}

[mx-psi]

@zenmoto to take a look as a CODEOWNER

[schmikei]

@djaglowski do you mind also taking a look since you know quite a bit about the storage extension/adapter code?

[atoulme]

Changes look good. But this is beta component so you need to first deprecate the old settings, offer the new ones, and honor the old settings for a few releases.

[schmikei]

Changes look good. But this is beta component so you need to first deprecate the old settings, offer the new ones, and honor the old settings for a few releases.

Thanks! But I might be a tad confused, since this change does not affect users nor configurations i.e. the storage component was already configured per mongodbatlasreceiver component. It was just the multiple access of the same storage component by the same receiver was causing a timeout. Could you elaborate what would need to be feature gated for the following releases?

[atoulme]

Ah, I see it now. No changes to config. I was the one being confused, no worries :)