Total Metric cycling values including 0 instead of giving consistent result

[daniel-anova]

I have this configuration:

    metricDefaults:
      aggregation:
        interval: 00:05:00
      scraping:
        schedule: "* * * * *"
    metrics:
      - name: azure_cosmos_db_data_usage
        description: Total data usage reported at 5 minutes granularity
        resourceType: CosmosDb
        azureMetricConfiguration:
          metricName: DataUsage
          aggregation:
            type: Total

But when looking at the metric i'm getting a constantly cycling set of values including zero instead of the expected slowly growing value.

I've done an az monitor metric list command and the returned metrics are the expected values minus the last one which seems to always contain an invalid data point.

{ 
  "timeseries": [
    {
      "data": [
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T09:26:00+00:00",
          "total": 84292960256.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T09:31:00+00:00",
          "total": 84292960256.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T09:36:00+00:00",
          "total": 84293025792.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T09:41:00+00:00",
          "total": 84292960256.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T09:46:00+00:00",
          "total": 84292960256.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T09:51:00+00:00",
          "total": 84289912832.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T09:56:00+00:00",
          "total": 84290043904.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T10:01:00+00:00",
          "total": 84289978368.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T10:06:00+00:00",
          "total": 84290043904.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T10:11:00+00:00",
          "total": 84293681152.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T10:16:00+00:00",
          "total": 84297678848.0
        },
        {
          "average": null,
          "count": null,
          "maximum": null,
          "minimum": null,
          "timeStamp": "2022-10-07T10:21:00+00:00",
          "total": 11983486976.0
        }
      ],
      "metadatavalues": []
    }
  ],
  "type": "Microsoft.Insights/metrics",
  "unit": "Bytes"
}

I tried playing with interval to see if it would help without any effect, so not sure what I'm doing wrong, if anyone could point me in the right direction I'd appreciate it.

[tomkerkhove]

Can you share what version you are running please?

I've done an az monitor metric list command and the returned metrics are the expected values minus the last one which seems to always contain an invalid data point.

That's normal, that is because the aggregation period has not exceeded yet.

[daniel-anova]

The versions are:
promitor-resource-discovery: 0.8.1 (helm release), 0.8.0 (container)
promitor-scraper: 2.9.0 (helm release), 2.8.0 (container)

[tomkerkhove]

Thanks! Would you mind posting a screenshot from the Azure Portal with aggregation of 5 min just to visually compare them please?

[daniel-anova]

The graph is mostly a straight line with a few spikes

for last 3 days with automatic granularity of 5m

and for last hour with automatic granularity of 5m

[daniel-anova]

Should I be opening an issue in promitor if this is not the expected behavior?

[tomkerkhove]

Sounds good and thanks for doing that, I am a bit busy at the moment and slow to respond - Sorry!