[Orb Agent] Support to Opentelemetry Backend

[leoparente]

Hi!
I've started this project opentelemetry-infinity and I think it would be a nice backend for Orb Agent. The ideia is to allow the user to setup any receiver and/or processor policy through orb UI and get the data result into a ORB sink.

This feature would allow, for instance, to spin up a policy with SNMP Metrics Receiver and then get the results in a prometheus sink.

As otlp-inf is a wrapper over otelcol-contrib it can support all contrib receivers: receivers_list

Design Tips

• Orb should not allow user to setup the exporter of the otlp-inf policy
• Orb OtlpInf Agent Backend should spin up a OTLP receiver on its code like it is done for pktvisor/diode backends
• Orb OtlpInf Agent Backend should populate the policy that comes from the control plane (fleet) with its own exporter information.

[manrodrigues]

@weyrick , @lpegoraro , @etaques

[lpegoraro]

Hey @leoparente, we did a different approach on your backend, feel free to check it out in staging, it is still in dev, only metrics are being pushed, but it is going well so far.

[leoparente]

Nice job @lpegoraro! Do you have any specific docs to point me for that? how to build a policy to use snmp for instance!

[lpegoraro]

Config Yaml by default

receivers:
  snmp:
    collection_interval: 60s
    endpoint: udp:https://localhost:161
    version: v3
    security_level: auth_priv
    user: otel
    auth_type: "MD5"
    auth_password: ${env:SNMP_AUTH_PASSWORD}
    privacy_type: "DES"
    privacy_password: ${env:SNMP_PRIVACY_PASSWORD}

    resource_attributes:
      resource_attr.name.1:
        indexed_value_prefix: probe
      resource_attr.name.2:
        oid: "1.1.1.1"

    attributes:
      attr.name.1:
        value: a2_new_key
        enum:
          - in
          - out
      attr.name.2:
        indexed_value_prefix: device
      attr.name.3:
        oid: "2.2.2.2"

    metrics:
      # This metric will have multiple datapoints wil 1 attribute on each.
      # Each datapoint will have a (hopefully) different attribute value
      metric.name.1:
        unit: 1
        sum:
          aggregation: cumulative
          monotonic: true
          value_type: int
        column_oids:
          - oid: "2.2.2.1"
            attributes:
              - name: attr.name.3
      # This metric will have multiple datapoints with 2 attributes on each.
      # Each datapoint will have a guaranteed different attribute indexed value for 1 of the attributes.
      # Half of the datapoints will have the other attribute with a value of "in".
      # The other half will have the other attribute with a value of "out".
      metric.name.2:
        unit: "By"
        gauge:
          value_type: int
        column_oids:
          - oid: "3.3.3.3"
            attributes:
              - name: attr.name.2
              - name: attr.name.1
                value: in
          - oid: "2"
            attributes:
              - name: attr.name.2
              - name: attr.name.1
                value: out
      # This metric will have 2 datapoints with 1 attribute on each
      # One datapoints will have an attribute value of "in".
      # The other will have an attribute value of "out".
      metric.name.3:
        unit: "By"
        sum:
          aggregation: delta
          monotonic: false
          value_type: double
        scalar_oids:
          - oid: "4.4.4.4.0"
            attributes:
              - name: attr.name.1
                value: in
          - oid: "4.4.4.5.0"
            attributes:
              - name: aattr.name.1
                value: out
      # This metric will have metrics created with each attached to a different resource.
      # Each resource will have a resource attribute with a guaranteed unique value based on the index.
      metric.name.4:
        unit: "By"
        gauge:
          value_type: int
        column_oids:
          - oid: "5.5.5.5"
            resource_attributes:
              - resource_attr.name.1
      # This metric will have metrics created with each attached to a different resource.
      # Each resource will have a resource attribute with a hopefully unique value.
      metric.name.5:
        unit: "By"
        gauge:
          value_type: int
        column_oids:
          - oid: "1.1.1.2"
            resource_attributes:
              - resource_attr.name.2
extensions:
exporters:
service:
  pipelines:
    metrics:
      exporters:
      receivers:
        - httpcheck

You will need to run the orb-agent-full:develop

docker run --net=host \
-e ORB_TLS_VERIFY=false \
-v /<<your-directory>>/otel_docker.yaml:/opt/orb/agent.yaml \
-v /<<your-directory>>/local-agent.db:/opt/orb/orb-agent.db \
orbcommunity/orb-agent-full:develop run -d

Config for autoprovision.

version: "1.0"

# this section is used orb-agent
orb:
  db_file: "/opt/orb/orb.db"
  # these are arbitrary key value pairs used for dynamically define a group of agents by matching against agent group tags
  tags:
    region: LA
    local: dev
    otel: local
  cloud:
    config:
      # optionally specify an agent name to use during auto provisioning
      # hostname will be used if it's not specified here
      agent_name: docker-otel
      auto_provision: true
    api:
      address: https://stg.orb.live
      token: <yourtoken>
    mqtt:
      address: tls:https://stg.orb.live:8883
  backends:
    otel:
      config_file: "/opt/orb/agent.yaml"
      otlp_port: 4316

Have fun!

[lpegoraro]

btw, I used the example here for the policy
https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/snmpreceiver