Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] Windows support #79

Open
jkroepke opened this issue Dec 18, 2023 · 16 comments
Open

[enhancement] Windows support #79

jkroepke opened this issue Dec 18, 2023 · 16 comments
Assignees
Labels
enhancement New feature or request

Comments

@jkroepke
Copy link
Contributor

Describe the enhancement you'd like

I have some cluster with Windows nodes enabled. I would like to ask if I can add windows support or do you think it out of context here?

Unlike kubernetes-mixin, which have separate dashboard, I would like to add the Windows queries into the existing one. Thats possible by using queries with OR, e.g.:

sum(container_memory_working_set_bytes{cluster="$cluster",namespace=~"$namespace", image!="", pod=~"${created_by}.*"}) by (pod)
OR
<WINDOWS Query>

Additional context

Since I'm running multiple OS hybrid clusters, I would like to add PRs for windows pods here. I'm not expecting that the maintainers here provide support for Windows. Before start to work here, I would like to know if its getting accepted?

@jkroepke jkroepke added the enhancement New feature or request label Dec 18, 2023
@dotdc
Copy link
Owner

dotdc commented Dec 18, 2023

Hi @jkroepke,

I'm not totally against it, but I have a few doubts...

Here are a few questions to start the discussion:

  • If I understand correctly, you have clusters with both GNU/Linux and Windows node pools.
    Could you share your use-case? (just curious)
  • Which dashboard(s) are you planning to add Windows support for?
  • Why do you think having the queries in here would be better than having them in separate dashboards?
  • Will there be any OS specific panels? And how do you plan to manage them (if any)?
  • After the initial work, would you be able to help Windows users if they have issues related to your change?

If anyone is also using Windows hosts, feel free to jump in the discussion with your thoughts!

@jkroepke
Copy link
Contributor Author

If I understand correctly, you have clusters with both GNU/Linux and Windows node pools.
Could you share your use-case? (just curious)

Sure! The Kubernetes Control Plane does not support Windows. I have to run Linux Node (e.g. for CoreDNS, ArgoCD) and Windows Node Pools for Application (Customer runs dotNET Application)

On Azure, a Kubernetes managed Services requires at-least one Linux Node and you can add additional Windows nodes. We are also quite common, that customers wants to use Standard infrastructure components like Redis, Solr or elasticsearch which are not running on Windows node. In general, we avoid Windows nodes as much as possible. Hybrid OS clusters are common, because mono-Windows cluster can't be exists.

Which dashboard(s) are you planning to add Windows support for?

Cluster, Namespace, Node, Pod

Why do you think having the queries in here would be better than having them in separate dashboards?

We have namespace which contains windows and linux containers. My personal opinion is that one dashboard provides a better user expericence.

Since kube-state-metrics provides request/limits for pods on any OS, Windows Pods are already included Pod CPU/Memory Request/Limits panels, but excluded on the usage panel. For separate dashboards, Requests/Limits from Windows Pods should be excluded then.

Example Panel: Real (linux only), Requests/Limits (any OS)

image

Will there be any OS specific panels?

No, except Windows is not providing metrics, some panels will be Linux specific. But I do not plan to add new panels.

After the initial work, would you be able to help Windows users if they have issues related to your change?

Yes, because out company has an upstream first culture. If other users have a bug, we may also have the bug, too. Of course, I will have.

The good thing is that the dashboard are providing only basic metrics. I don't plan to engineer the queries on my own. kubernetes-mixin provides some rules for windows which I will use as base. for the panel. I won't add a dependency against the kubernetes-mixin recording rules.


Since I expect some large changes with #15, I plan to start after the cluster variable work is finished.

@dotdc
Copy link
Owner

dotdc commented Dec 20, 2023

Hi @jkroepke,
I'm fine with the idea, we can try to add Windows support, as soon as it doesn't impact the experience on Linux.
Will just finish the work on the other issues before looking further into this.

@jkroepke
Copy link
Contributor Author

Will just finish the work on the other issues before looking further into this.

I would prefer to split the Windows integration into different, smaller PRs. At minimum a PR for each dashboard. Let me know, when I can start with the cluster dashboard.

@dotdc
Copy link
Owner

dotdc commented Jan 5, 2024

@jkroepke The other issues/PRs are done, so you can start working on this whenever you want.

@jkroepke
Copy link
Contributor Author

Hey @dotdc ,

In #103, I tired to split Linux and Windows queries when ever it was possible. Did you prefer this kind of solution or should we prefer more complex query as seen on the CPU by namespace queries?

@dotdc
Copy link
Owner

dotdc commented Apr 3, 2024

Hi @jkroepke,
I think I prefer the split version, but if it's not possible to have that everywhere, maybe go for the other one to have consistency.
What do you think ?

@jkroepke
Copy link
Contributor Author

jkroepke commented Apr 3, 2024

What do you think ?

The combind solution (one query for Windows and OS) results into very complex and un-understandable queries. Due complexity, external users may avoid to contribute improvement or they may destroy the Windows support by accident

@dotdc
Copy link
Owner

dotdc commented Apr 3, 2024

Agree.
Do you think it's possible to split the combined queries like "CPU by namespace" ?

@jkroepke
Copy link
Contributor Author

jkroepke commented Apr 3, 2024

Maybe, no idea yet. I may have to ask at Grafana slack.

@jkroepke
Copy link
Contributor Author

jkroepke commented Apr 8, 2024

Do you think it's possible to split the combined queries like "CPU by namespace" ?

In theory yes, but the make it complex at Grafana side:

image

@dotdc
Copy link
Owner

dotdc commented Apr 10, 2024

Can we manage all the cases using combined queries?

If it's the case, it might be better to go for the combined queries with comments to have consistency.
Here's an example :

image

What do you think ?

@dotdc
Copy link
Owner

dotdc commented Apr 10, 2024

Also, could be nice to add Windows support knowledge in the docs (README.md) for the other users.

@dotdc
Copy link
Owner

dotdc commented Jun 20, 2024

Hi @jkroepke,

Do you plan to add Windows support on the other dashboards?

@jkroepke
Copy link
Contributor Author

jkroepke commented Jun 20, 2024

Hi @dotdc,

in general, yes. But time is a bit limited and I had to take a focus on windows_exporter. Once I'm done, I will continue.

@dotdc
Copy link
Owner

dotdc commented Jun 21, 2024

That's fine, let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants