Azure CosmosDB partion key metrics

[matthieu-eck]

Hi! I would like to monitor the storage usage per partition key, can Promitor expose this metric?
I tried with :

• DataUsage: Gives me the usage for the whole database
• AvailableStorage: Gives me the available storage for the whole database

Azure documentation has no partition key metric, but maybe there is a way to get AvailableStorage by collection?

Thanks!

[tomkerkhove]

Hi!

As far as I know, you cannot split the Cosmos DB metrics in Azure Monitor per partition key, but maybe @markjbrown can correct me?

Is there an alternative for this and/or plans for this in Monitor, Mark?

[markjbrown]

We don't have any plans to monitor to that extent. Right now you can only get metrics on the top 3 largest partition keys.

The reason we don't enable monitoring across all of them is that it would quickly result in massive diagnostic logs and cost a ton of money.

Storage metrics, yes you can easily get those. check here for more details, https://docs.microsoft.com/en-us/azure/cosmos-db/monitor-cosmos-db-reference#storage-metrics

[tomkerkhove]

Thanks @markjbrown! I presume the documented metrics are the same from https://docs.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported#microsoftdocumentdbdatabaseaccounts?

[markjbrown]

Those are metrics that can be directly viewed in Azure Monitor. There is also our diagnostic logs which are here

[tomkerkhove]

That's what I figured, thanks @markjbrown!

@matthieu-eck Unfortunately there is no way for us to get the metrics based on your approach, sorry :/

[matthieu-eck]

Thanks @tomkerkhove and @markjbrown for your answers.

@markjbrown, you said that it was possible to get the top 3 largest partition keys, which could be already an helpful metric. I am not able to find this metric in Azure documentation, could you please tell me how I can get it? Thanks!

[markjbrown]

sure, take a look here.

https://docs.microsoft.com/en-us/azure/cosmos-db/cosmosdb-monitor-resource-logs#diagnostic-queries

[matthieu-eck]

Cool, thanks! I don't see how I can use this metric in promitor since they are not exposed here. From my understanding, I cannot use other metrics than the one described in monitor cosmos db reference. Any idea on how I can achieve that?

[tomkerkhove]

It's just a Kusto query so we could provide a Log Analytics provider which runs your query; over time we might do this for you but that's already a bridge further. Make sense to you @matthieu-eck?

Or are you planning on surfacing this in Azure Monitor as metrics soon @markjbrown?

[markjbrown]

There aren't any short term plans to do this but soemthing we will look at in the future. We've got a lot of work to do for our monitoring and metrics today.

[tomkerkhove]

Thanks for the quick reply & help @markjbrown!

@matthieu-eck I've created #1076 which will allow you to run those queries but won't be coming soon. Are you willing to contribute this or is it less priority?

[matthieu-eck]

I would like to contribute but since I don't speak C# at all and it does not have an high priority for me, I will wait the feature.
Thanks a lot @tomkerkhove and @markjbrown !