Scraper error for aks metrics

[krnayak2601]

For aks metrics collection from promitor 2.1.0 the below error is seen in scraper metrics. We tried using both auto and manul discovery both shows similar error

[05:16:05 FTL] Failed to scrape resource for metric 'Total_number_of_available_cpu_cores_in_a_managed_cluster'
System.ArgumentException: [details] cannot be Null, empty or white-space. (Parameter 'details')
   at GuardNet.Guard.For[TException](Func`1 predicate, TException exception)
   at Promitor.Integrations.AzureMonitor.Exceptions.MetricInformationNotFoundException..ctor(String name, String details, String dimension) in /src/Promitor.Integrations.AzureMonitor/MetricInformationNotFoundException.cs:line 36
   at Promitor.Integrations.AzureMonitor.AzureMonitorClient.QueryMetricAsync(String metricName, String metricDimension, AggregationType aggregationType, TimeSpan aggregationInterval, String resourceId, String metricFilter) in /src/Promitor.Integrations.AzureMonitor/AzureMonitorClient.cs:line 85
   at Promitor.Core.Scraping.AzureMonitorScraper`1.ScrapeResourceAsync(String subscriptionId, ScrapeDefinition`1 scrapeDefinition, TResourceDefinition resourceDefinition, AggregationType aggregationType, TimeSpan aggregationInterval) in /src/Promitor.Core.Scraping/AzureMonitorScraper.cs:line 69
   at Promitor.Core.Scraping.Scraper`1.ScrapeAsync(ScrapeDefinition`1 scrapeDefinition) in /src/Promitor.Core.Scraping/Scraper.cs:line 89
[05:16:05 INF] Found value 0.1240625 for metric azur_virtual_machines_percentage_cpu_average with aggregation interval 00:05:10
[05:16:05 INF] Found value 100 for metric Availability with aggregation interval 00:05:10
[05:16:05 INF] Found value 0.15125 for metric azur_virtual_machines_percentage_cpu_average with aggregation interval 00:05:10
[05:16:05 INF] Found value 3.107083333333333 for metric azur_virtual_machines_percentage_cpu_average with aggregation interval 00:05:10

"metrics-declaration.yaml" declaration

azureMetadata:
  subscriptionId: #####
  tenantId: #####
  resourceGroupName: abc
metricDefaults:
  aggregation:
    interval: 00:05:00
  scraping:
    schedule: 0 * * ? * *
metrics:
- name: azur_apiserver_current_inflight_requests
  description: "Inflight Requests"
  resourceType: Generic
  azureMetricConfiguration:
    metricName: apiserver_current_inflight_requests
    aggregation:
      type: Average
  resources:
  - resourceUri: urilink
version: v1

Did someone had similar issue for aks metrics using promitor

[tomkerkhove]

Thanks for asking, would you mind using our latest version? This is resolved, I think.

[krnayak2601]

Oh! It worked. Hmm..when you say it takes time, you mean promitor scaper takes time to get data from azure and aggregate the metrics? Although we have data for each min in azure.

Here is the scraper logs:
[16:39:00 INF] Discovered 2 resources for resource collection kubernetes-service-landscape.
[16:39:01 INF] Found value null for metric node_cpu_usage_percentage with aggregation interval 00:15:00
[16:39:02 INF] Found value 7.856617647058823 for metric node_cpu_usage_percentage with aggregation interval 00:15:00
[16:40:00 INF] Scraping resource collection kubernetes-service-landscape - 01/27/2022 16:40:00 +00:00

[krnayak2601]

It worked for a while, again have same errors
[17:16:00 INF] Discovered 2 resources for resource collection kubernetes-service-landscape.
[17:16:01 WRN] No metric information found for metric node_cpu_usage_percentage with dimension null. Details: No time series was found
[17:16:01 INF] Found value null for metric node_cpu_usage_percentage with aggregation interval 00:15:00
[17:16:01 WRN] No metric information found for metric node_cpu_usage_percentage with dimension null. Details: No time series was found
[17:16:01 INF] Found value null for metric node_cpu_usage_percentage with aggregation interval 00:15:00

[krnayak2601]

Hi Tom,

Is this something to to do with dimensions, I'm not sure what to check now..
I tried to increase aggregation and also change the type, none seems to be working

[tomkerkhove]

I will have to investigate this, sorry :/ Can you open a bug for this please?

Oh! It worked. Hmm..when you say it takes time, you mean promitor scaper takes time to get data from azure and aggregate the metrics? Although we have data for each min in azure.

I was more referring to #1290 where the API is not giving the information in time.

[krnayak2601]

no problem. #1946 is raised, I'll mark this one answered as the original issue is no longer seen in latest version