Use google cloud collector exporter, and feature-gate pdata-direct me…

…trics exporter. (#7177)

* use google cloud collector exporter, and feature-gate new metrics exporter

* update docs around new batching functionality

Co-authored-by: Bogdan Drutu <[email protected]>

CHANGELOG.md

@@ -4,8 +4,15 @@
 
 ### 💡 Enhancements 💡
 
+- `googlecloudexporter`: [Alpha] Translate metrics directly from OTLP to gcm using the `exporter.googlecloud.OTLPDirect` feature-gate (#7177)
 - `simpleprometheusreceiver`: Add support for static labels (#7908)
 
+### 🛑 Breaking changes 🛑
+
+### 🧰 Bug fixes 🧰
+
+### 🚀 New components 🚀
+
 ## v0.46.0
 
 ### 💡 Enhancements 💡

cmd/configschema/go.mod

@@ -18,7 +18,7 @@ require (
  bitbucket.org/atlassian/go-asap/v2 v2.6.0 // indirect
  cloud.google.com/go v0.100.2 // indirect
  cloud.google.com/go/compute v1.5.0 // indirect
- cloud.google.com/go/monitoring v1.1.0 // indirect
+ cloud.google.com/go/monitoring v1.2.0 // indirect
  cloud.google.com/go/spanner v1.29.0 // indirect
  cloud.google.com/go/trace v1.0.0 // indirect
  code.cloudfoundry.org/clock v1.0.0 // indirect
@@ -41,6 +41,7 @@ require (
  github.com/DataDog/datadog-agent/pkg/util/log v0.0.0-20201009092105-58e18918b2db // indirect
  github.com/DataDog/datadog-go v4.8.2+incompatible // indirect
  github.com/GehirnInc/crypt v0.0.0-20200316065508-bb7000b8a962 // indirect
+ github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector v0.26.0 // indirect
  github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.3.0 // indirect
  github.com/Microsoft/go-winio v0.5.1 // indirect
  github.com/ReneKroon/ttlcache/v2 v2.11.0 // indirect

exporter/googlecloudexporter/README.md

@@ -204,6 +204,44 @@ and [memory limiter](https://github.com/open-telemetry/opentelemetry-collector/t
 optimal network usage and avoiding memory overruns. You may also want to run an additional
 [sampler](../../processor/probabilisticsamplerprocessor), depending on your needs.
 
+## Features and Feature-Gates
+
+See the [Collector feature gates](https://github.com/open-telemetry/opentelemetry-collector/blob/main/service/featuregate/README.md#collector-feature-gates) for an overview of feature gates in the collector.
+
+**ALPHA**: `exporter.googlecloud.OTLPDirect`
+
+When enabled via `--feature-gates=exporter.googlecloud.OTLPDirect`, the googlecloud exporter translates pdata directly to google cloud monitoring's types, rather than first translating to opencensus. See the [Breaking Changes documentation](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/breaking-changes.md#breaking-changes-vs-old-googlecloud-exporter) for breaking changes that will occur as a result of enabling this feature.
+
+Additional configuration added for the metric exporter:
+
+- `metric.endpoint` (optional): Endpoint where metric data is going to be sent to. Replaces `endpoint`.
+- `metric.use_insecure` (optional): If true. use gRPC as their communication transport. Only has effect if Endpoint is not "". Replaces `use_insecure`.
+- `metric.known_domains` (optional): If a metric belongs to one of these domains it does not get a prefix.
+- `metric.instrumentation_library_labels` (optional): If true, set the instrumentation_source and instrumentation_version labels. Defaults to true.
+- `metric.create_service_timeseries` (optional): If true, this will send all timeseries using `CreateServiceTimeSeries`. Implicitly, this sets `SkipMetricDescriptor` to true.
+- `metric.create_metric_descriptor_buffer_size` (optional): Buffer size for the channel which asynchronously calls CreateMetricDescriptor. Default is 10.
+- `metric.service_resource_labels` (optional): If true, the exporter will copy OTel's service.name, service.namespace, and service.instance.id resource attributes into the GCM timeseries metric labels. Default is true.
+- `metric.resource_filters` (optional): If provided, resource attributes matching any filter will be included in metric labels. Defaults to empty, which won't include any additional resource labels.
+ - `prefix`: Match resource keys by prefix
+
+Additional configuration added for the trace exporter:
+
+- `trace.endpoint` (optional): Endpoint where trace data is going to be sent to. Replaces `endpoint`.
+- `trace.use_insecure` (optional): If true. use gRPC as their communication transport. Only has effect if Endpoint is not "". Replaces `use_insecure`.
+
+Removed configuration:
+
+- `endpoint`: replaced by `trace.endpoint` and `metric.endpoint`
+- `use_insecure`: replaced by `trace.use_insecure` and `metric.use_insecure`
+- `resource_mappings`: replacement is still under development
+
+Changes to defaults:
+
+- `prefix`: default is now `workload.googleapis.com`
+
+Additional Behavior:
+
+- Metrics are now batched in the exporter to handle edge cases around summar metrics. The batch processor is no longer needed in metrics processors.
 
 ## Deprecatations
 

exporter/googlecloudexporter/config.go

@@ -15,56 +15,22 @@
 package googlecloudexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/googlecloudexporter"
 
 import (
+ "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector"
  "go.opentelemetry.io/collector/config"
  "go.opentelemetry.io/collector/exporter/exporterhelper"
- "google.golang.org/api/option"
 )
 
 // Config defines configuration for Google Cloud exporter.
 type Config struct {
  config.ExporterSettings `mapstructure:",squash"`
- ProjectID string `mapstructure:"project"`
- UserAgent string `mapstructure:"user_agent"`
- Endpoint string `mapstructure:"endpoint"`
- // Only has effect if Endpoint is not ""
- UseInsecure bool `mapstructure:"use_insecure"`
+ collector.Config `mapstructure:",squash"`
 
  // Timeout for all API calls. If not set, defaults to 12 seconds.
  exporterhelper.TimeoutSettings `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct.
  exporterhelper.QueueSettings `mapstructure:"sending_queue"`
  exporterhelper.RetrySettings `mapstructure:"retry_on_failure"`
-
- ResourceMappings []ResourceMapping `mapstructure:"resource_mappings"`
- // GetClientOptions returns additional options to be passed
- // to the underlying Google Cloud API client.
- // Must be set programmatically (no support via declarative config).
- // Optional.
- GetClientOptions func() []option.ClientOption
-
- MetricConfig MetricConfig `mapstructure:"metric"`
 }
 
 func (cfg *Config) Validate() error {
  return nil
 }
-
-type MetricConfig struct {
- Prefix string `mapstructure:"prefix"`
- SkipCreateMetricDescriptor bool `mapstructure:"skip_create_descriptor"`
-}
-
-// ResourceMapping defines mapping of resources from source (OpenCensus) to target (Google Cloud).
-type ResourceMapping struct {
- SourceType string `mapstructure:"source_type"`
- TargetType string `mapstructure:"target_type"`
-
- LabelMappings []LabelMapping `mapstructure:"label_mappings"`
-}
-
-type LabelMapping struct {
- SourceKey string `mapstructure:"source_key"`
- TargetKey string `mapstructure:"target_key"`
- // Optional flag signals whether we can proceed with transformation if a label is missing in the resource.
- // When required label is missing, we fallback to default resource mapping.
- Optional bool `mapstructure:"optional"`
-}