If you have a different setup, make sure the following metrics exist:

• container_cpu_usage_seconds_total
• container_memory_working_set_bytes
• kube_replicaset_owner
• kube_pod_owner
• kube_pod_status_phase

Note: If one of last three metrics is absent KRR will still work, but it will only consider currently-running pods when calculating recommendations. Historic pods that no longer exist in the cluster will not be taken into consideration.

1. Add our tap:

brew tap robusta-dev/homebrew-krr

2. Install KRR:

brew install krr

3. Check that installation was successful:

krr --help

4. Run KRR (first launch might take a little longer):

krr simple

You can install using brew (see above) on WSL2, or install from source (see below).

You can download pre-built binaries from Releases or use the prebuilt Docker container. For example, the container for version 1.8.3 is:

us-central1-docker.pkg.dev/genuine-flight-317411/devel/krr:v1.8.3

We do not recommend installing KRR from source in airgapped environments due to the headache of installing Python dependencies. Use one of the above methods instead and contact us (via Slack, GitHub issues, or email) if you need assistance.

1. Make sure you have Python 3.9 (or greater) installed
2. Clone the repo:

git clone https://github.com/robusta-dev/krr

3. Navigate to the project root directory (cd ./krr)
4. Install requirements:

pip install -r requirements.txt

5. Run the tool:

python krr.py --help

Notice that using source code requires you to run as a python script, when installing with brew allows to run krr. All above examples show running command as krr ..., replace it with python krr.py ... if you are using a manual installation.

Most helpful flags:

• --cpu-min Sets the minimum recommended cpu value in millicores
• --mem-min Sets the minimum recommended memory value in MB
• --history_duration The duration of the Prometheus history data to use (in hours)

More specific information on Strategy Settings can be found using

krr simple --help

If your Prometheus is not auto-connecting, you can use kubectl port-forward for manually forwarding Prometheus.

For example, if you have a Prometheus Pod called kube-prometheus-st-prometheus-0, then run this command to port-forward it:

kubectl port-forward pod/kube-prometheus-st-prometheus-0 9090

Then, open another terminal and run krr in it, giving an explicit Prometheus url:

krr simple -p https://127.0.0.1:9090

List as many namespaces as you want with -n (in this case, default and ingress-nginx)

krr simple -n default -n ingress-nginx

The -n flag also supports regex matches like -n kube-.*. To use regexes, you must have permissions to list namespaces in the target cluster.

krr simple -n default -n 'ingress-.*'

See example ServiceAccount and RBAC permissions

Use a label selector

python krr.py simple --selector 'app.kubernetes.io/instance in (robusta, ingress-nginx)'

By default krr will run in the current context. If you want to run it in a different context:

krr simple -c my-cluster-1 -c my-cluster-2

Currently KRR ships with a few formatters to represent the scan data:

• table - a pretty CLI table used by default, powered by Rich library
• json
• yaml
• pprint - data representation from python's pprint library
• csv - export data to a csv file in the current directory
• html

To run a strategy with a selected formatter, add a -f flag. Usually this should be combined with --fileoutput <filename> to write clean output to file without logs:

krr simple -f json --fileoutput krr-report.json

If you prefer, you can also use --logtostderr to get clean formatted output in one file and error logs in another:

krr simple --logtostderr -f json > result.json 2> logs-and-errors.log

See below on filtering output from a centralized prometheus, so it matches only one cluster

KRR supports all known authentication schemes for Prometheus, VictoriaMetrics, Coralogix, and other Prometheus compatible metric stores.

Refer to krr simple --help, and look at the flags --prometheus-url, --prometheus-auth-header, --prometheus-headers --prometheus-ssl-enabled, --coralogix-token, and the various --eks-* flags.

If you need help, contact us on Slack, email, or by opening a GitHub issue.

By default, KRR will try to auto-discover the running Prometheus Victoria Metrics and Thanos. For discovering Prometheus it scans services for those labels:

"app=kube-prometheus-stack-prometheus"
"app=prometheus,component=server"
"app=prometheus-server"
"app=prometheus-operator-prometheus"
"app=rancher-monitoring-prometheus"
"app=prometheus-prometheus"

For Thanos its these labels:

"app.kubernetes.io/component=query,app.kubernetes.io/name=thanos",
"app.kubernetes.io/name=thanos-query",
"app=thanos-query",
"app=thanos-querier",

And for Victoria Metrics its the following labels:

"app.kubernetes.io/name=vmsingle",
"app.kubernetes.io/name=victoria-metrics-single",
"app.kubernetes.io/name=vmselect",
"app=vmselect",

If none of those labels result in finding Prometheus, Victoria Metrics or Thanos, you will get an error and will have to pass the working url explicitly (using the -p flag).

(back to top)

If your Prometheus monitors multiple clusters we require the label you defined for your cluster in Prometheus.

For example, if your cluster has the Prometheus label cluster: "my-cluster-name", then run this command:

krr.py simple --prometheus-label cluster -l my-cluster-name

You may also need the -p flag to explicitly give Prometheus' URL.

For Azure managed Prometheus you need to generate an access token, which can be done by running the following command:

# If you are not logged in to Azure, uncomment out the following line
# az login
AZURE_BEARER=$(az account get-access-token --resource=https://prometheus.monitor.azure.com  --query accessToken --output tsv); echo $AZURE_BEARER

Than run the following command with PROMETHEUS_URL substituted for your Azure Managed Prometheus URL:

python krr.py simple --namespace default -p PROMETHEUS_URL --prometheus-auth-header "Bearer $AZURE_BEARER"

See here about configuring labels for centralized prometheus

(back to top)

Please find the detailed GMP usage instructions here

(back to top)

For Amazon Managed Prometheus you need to add your Prometheus link and the flag --eks-managed-prom and krr will automatically use your aws credentials

python krr.py simple -p "https://aps-workspaces.REGION.amazonaws.com/workspaces/..." --eks-managed-prom

Additional optional parameters are:

--eks-profile-name PROFILE_NAME_HERE # to specify the profile to use from your config
--eks-access-key ACCESS_KEY # to specify your access key
--eks-secret-key SECRET_KEY # to specify your secret key
--eks-service-name SERVICE_NAME # to use a specific service name in the signature
--eks-managed-prom-region REGION_NAME # to specify the region the Prometheus is in

See here about configuring labels for centralized prometheus

(back to top)

For Coralogix managed Prometheus you need to specify your Prometheus link and add the flag coralogix_token with your Logs Query Key

python krr.py simple -p "https://prom-api.coralogix..." --coralogix_token

See here about configuring labels for centralized prometheus

(back to top)

For Grafana Cloud managed Prometheus you need to specify Prometheus link, Prometheus user, and an access token of your Grafana Cloud stack. The Prometheus link and user for the stack can be found on the Grafana Cloud Portal. An access token with a metrics:read scope can also be created using Access Policies on the same portal.

Next, run the following command, after setting the values of PROM_URL, PROM_USER, and PROM_TOKEN variables with your Grafana Cloud stack's Prometheus link, Prometheus user, and access token.

python krr.py simple -p $PROM_URL --prometheus-auth-header "Bearer ${PROM_USER}:${PROM_TOKEN}" --prometheus-ssl-enabled

See here about configuring labels for centralized prometheus

(back to top)

By default, KRR will try to auto-discover the running Grafana Mimir.

For discovering Prometheus it scans services for those labels:

  "app.kubernetes.io/name=mimir,app.kubernetes.io/component=query-frontend"

(back to top)

We highly recommend using the free Robusta SaaS platform. You can:

• Understand individual app recommendations with app usage history

• Sort and filter recommendations by namespace, priority, and more

• Give dev's a YAML snippet to fix the problems KRR finds

• Analyze impact using KRR scan history

  

Put cost savings on autopilot. Get notified in Slack about recommendations above X%. Send a weekly global report, or one report per team.

• A Slack workspace

1. Install Robusta with Helm to your cluster and configure slack
2. Create your KRR slack playbook by adding the following to generated_values.yaml:

customPlaybooks:
# Runs a weekly krr scan on the namespace devs-namespace and sends it to the configured slack channel
customPlaybooks:
- triggers:
  - on_schedule:
      fixed_delay_repeat:
        repeat: -1 # number of times to run or -1 to run forever
        seconds_delay: 604800 # 1 week
  actions:
  - krr_scan:
      args: "--namespace devs-namespace" ## KRR args here
  sinks:
      - "main_slack_sink" # slack sink you want to send the report to here

3. Do a Helm upgrade to apply the new values: helm upgrade robusta robusta/robusta --values=generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

(back to top)

Install our k9s Plugin to get recommendations directly in deployments/daemonsets/statefulsets views.

Plugin: resource recommender

Installation instructions: k9s docs