To use the exporter as a python3 script, few packages needs to be installed. This can be done using

pip3 install -r pip_requirements.txt

Contents of requirements.txt

prometheus-client==0.8.0
requests==2.23.0
PyYAML==5.3.1
tenacity==6.2.0
urllib3==1.25.9

Now, create a folder /exporter and copy the metrics.json file to the folder. Finally, the exporter can be run as a python script using;

nohup python exporter.py [flags] &

where the flags are:

The exporter can be setup as given in the diagram using;

nohup python exporter.py --target-nsip=10.0.0.1 --port 8888 --config-file=config.yaml &

This directs the exporter container to scrape 10.0.0.1 IP, and the expose the stats it collects on port 8888.

File 'config.yaml' should contain username and password of the ADC to which exporter connects to. For reference, refer config.yaml.example which specifies the format for providing the login credentials.

Login credentials can also be provided using environment variables using NS_USER, NS_PASSWORD.Though config file input is the preferred method for security concerns.

The user can then access the exported metrics directly thorugh port 8888 on the machine where the exporter is running, or Prometheus and Grafana can be setup to view the exported metrics though their GUI.

As an optional configuration, Citrix ADC exporter allows you to validate the SSL server certificate provided by Citrix ADC.

For this:

-- In config.yaml, '--validate-cert' option should be set to 'yes', and certificate path should be provided using'--cacert-path' argument.

-- Please confirm that certificate and '--cert-path' provided is valid.

-- Additionaly, '--validate-cert(default='no')' and '--secure(default='yes')' options should be set to 'yes'.

In order to use the exporter as a container, the image quay.io/citrix/citrix-adc-metrics-exporter:1.4.5 will need to be pulled using;

docker pull quay.io/citrix/citrix-adc-metrics-exporter:1.4.5

NOTE: It can also be build locally using docker build -f Dockerfile -t <image_name>:<tag> ./

Now, the exporter can be run using:

docker run -dt -p <host_port>:<container_port> --mount type=bind,source=<host-path-for-config-file>,target=/exporter/config.yaml quay.io/citrix/citrix-adc-metrics-exporter:1.4.5 [flags] --config-file=/exporter/config.yaml

where the flags are:

To setup the exporter as given in the diagram, the following command can be used:

docker run -dt -p 8888:8888 --mount type=bind,source=/path/to/config.yaml,target=/exporter/config.yaml --name citrix-adc-exporter quay.io/citrix/citrix-adc-metrics-exporter:1.4.5 --target-nsip=10.0.0.1 --port=8888 --config-file=/exporter/config.yaml

This directs the exporter container to scrape the 10.0.0.1 IP, and the expose the stats it collects on port 8888.

File 'config.yaml' should contain username and password of the ADC to which exporter connects to. For reference, refer config.yaml.example which specifies the format for providing the login credentials. Config.yaml file then needs to be mounted to the container at fixed target specified('/exporter/config.yaml') only.

Login credentials can also be provided using environment variables using NS_USER, NS_PASSWORD.Though config file input is the preferred method for security concerns.

The user can then access the exported metrics directly thorugh port 8888 on the machine where the exporter is running, or Prometheus and Grafana can be setup to view the exported metrics though their GUI.

NOTE: In the command above, the value of the --port flag should be the same as the container_port.

As an optional configuration, Citrix ADC exporter allows you to validate the SSL server certificate provided by Citrix ADC.

For this:

-- In config.yaml, '--validate-cert' option should be set to 'yes', and certificate path should be provided using'--cacert-path' argument.

-- Please confirm that certificate and '--cert-path' provided is valid.

-- Additionaly, '--validate-cert(default='no')' and '--secure(default='yes')' options should be set to 'yes' for certificate to be considered.

Certificate should then be mounted at the '--cacert-path' provided. For instance, if cert is 'cacert.pem' and '--cacert-path' provided in 'config.yaml' is '/exporter/cacert.pem'

docker run -dt -p 8888:8888 --mount type=bind,source=/path/to/config.yaml,target=/exporter/config.yaml --mount type=bind,source=/path/to/cacert.pem,target=/exporter/cacert.pem --name citrix-adc-exporter quay.io/citrix/citrix-adc-metrics-exporter:1.4.5 --target-nsip=10.0.0.1 --port=8888 --config-file=/exporter/config.yaml

Cert validation options can also be provided using environment variables using NS_VALIDATE_CERT, NS_CACERT_PATH. Thoughconfig file input is the preferred method.

To provide the login credentials to access ADC, create a secret and mount the volume at mountpath "/mnt/nslogin".

kubectl create secret generic nslogin --from-literal=username=<citrix-adc-user> --from-literal=password=<citrix-adc-password> -n <namespace>

The following yaml file can be used to deploy the exporter as a pod in Kuberenetes and expose it as a service. Here, the necessary flags are provided as a list in the args: section of the yaml file.

apiVersion: v1
kind: Pod
metadata:
  name: exporter
  labels:
    app: exporter
spec:
  containers:
    - name: exporter
      image: quay.io/citrix/citrix-adc-metrics-exporter:1.4.5
      args:
        - "--target-nsip=10.0.0.1"
        - "--port=8888"
      imagePullPolicy: Always
      volumeMounts:
      - name: nslogin
        mountPath: "/mnt/nslogin"
        readOnly: true
      securityContext:
        readOnlyRootFilesystem: true
  volumes:
  - name: nslogin
    secret:
      secretName: nslogin
---
apiVersion: v1
kind: Service
metadata:
  name: exporter
  labels:
    app: exporter
spec:
  type: ClusterIP
  ports:
  - port: 8888
    targetPort: 8888
    name: exporter-port
  selector:
    app: exporter

Login Credentials can also be provided using environment variables NS_USER, NS_PASSWORD. Though, secret volume is the preffered method for security.

Flags which can be provided to the exporter in the args: section are:

As an optional configuration, Citrix ADC exporter allows you to validate the SSL server certificate provided by Citrix ADC. For this: User need to first mount the secret at any valid path inside pod. For instance, for CA certificate 'cacert.pem' and name give as 'exp-ca-cert'

kubectl create secret generic exp-ca-cert --from-file=./cacert.pem"

-- In args section of yaml, '--validate-cert' option should be set to 'yes', and certificate path should be provided using'--cacert-path' argument.

-- Please confirm that mounted certificate and '--cert-path' provided are valid.

-- Additionaly, '--validate-cert(default='no')' and '--secure(default='yes')' options should be set to 'yes' for certificate to be considered.

apiVersion: v1
kind: Pod
metadata:
  name: exporter
  labels:
    app: exporter
spec:
  containers:
    - name: exporter
      image: quay.io/citrix/citrix-adc-metrics-exporter:1.4.5
      args:
        - "--target-nsip=10.0.0.1"
        - "--port=8888"
        - "--validate-cert=yes"
        - "--cacert-path=/mnt/certs/cacert.pem"
      imagePullPolicy: Always
      volumeMounts:
      - name: nslogin
        mountPath: "/mnt/nslogin"
        readOnly: true
      - name: exp-ca-cert
        mountPath: "/mnt/certs/"
        readOnly: true
      securityContext:
        readOnlyRootFilesystem: true
  volumes:
  - name: nslogin
    secret:
      secretName: nslogin
  - name: exp-ca-cert
    secret:
      secretName: exp-ca-cert
---
apiVersion: v1
kind: Service
metadata:
  name: exporter
  labels:
    app: exporter
spec:
  type: ClusterIP
  ports:
  - port: 8888
    targetPort: 8888
    name: exporter-port
  selector:
    app: exporter

Cert Credentials can also be provided using environment variables NS_VALIDATE_CERT, NS_CACERT_PATH instead of args '--validate-cert', '--cacert-path' respectively.

These steps can be followed to setup a Prometheus container:

1. Pull the docker image: docker pull prom/prometheus.

2. Create the prometheus.cfg file as given below, and providie the Exporter's IP and Port in the place of <EXPORTER_IP> and <EXPORTER_PORT>. For example, the targets line might read - targets: ['10.100.200.3:8888'].

global:
  scrape_interval: 15s
  scrape_timeout:  10s

  external_labels:
    monitor: 'prometheus-monitor'

scrape_configs:
- job_name: prometheus

  static_configs:
  - targets: ['<EXPORTER_IP>:<EXPORTER_PORT>']

3. Run the Prometheus container while providing the location of the prometheus.cfg file created in the above step:
   docker run -dt -p 9090:9090 -v /path/to/prometheus.cfg:/etc/prometheus/prometheus.cfg prom/prometheus:latest --config.file=/etc/prometheus/prometheus.cfg

4. VERIFICATION: Go to the Prometheus web page and under the "Status" dropdown select "Targets". The exporter should appear as a target and in a few minutes time, must come up in UP state. This means that Prometheus is able to collect stats from the Exporter.

The steps bellow can be followed to setup up a Grafana container with a sample dashboard.

1. Pull grafana image: docker pull grafana/grafana:latest

2. Run grafana container: docker run -dt -p 3000:3000 grafana/grafana:latest

3. Import any of the sample grafana dashboard files: Login to Grafana using admin:admin, from the column on the left select the + symbol, select "Import", and select "upload .json file". Now, navigate to and upload "sample_service_stats.json" or "sample_system_stats.json".

4. To start seeing graphs and values in the dashboard, add the Prometheus datasource(s) to Grafana. While adding the datasource, ensure the name of the Prometheus datasource starts with the word "prometheus" (eg. prometheus_datasource1). Once added, datasources starting with the word "prometheus" will automatically get detected in the dropdown filters of the dashboard.

5. Usage of Dashboard: For K8s CIC set up(providing K8sCICPrefix), with "k8s_cic_ingress_services_stats.json",dashboard shows service requests, surque queue, RPS and Invalid Request/Response. User can select any of the configured ingress and then apply filter for a given ingress from the drop down menu for ingress.Fro non-k8s setup, one can use "sample_lb_stats.json" for all lbvservers configured on ADC for some counters. And with "sample_system_stats.json", dashboard shows CPU utilization, Memory Utilization and bandwidth capacity utilization where user can also set an alert. The dashboard can be expanded to include graphs of any other stats which the exporter is collecting. For more information on modifying the Grafana dashboard, please take a look at their documentation or demo videos.

If the graphs on the Grafana dashboards do not have any values plotted (not even a flat '0' value line), this means that Grafana is unable to obtain stats from its datasource. The following can be done:

i. Check if the prometheus datasource is saved and working properly. On saving the datasource after providing the Name and IP, a "Data source is working" message should appear in green indicating the datasource is reachable and detected.

ii. If the dashboard was created using sample_grafana_dashboard.json, ensure the name given to the Prometheus datasource begins with the word "prometheus", with all lowercase letters.

iii. Check the Targets page of prometheus to see if the requried target exporter is in DOWN state.

If this appears against any of the exporter target of Prometheus, this means that Prometheus is unable to connect to that exporter, or is unable to fetch all the metrics from that exporter within the given scrape_timeout.

i. If Prometheus Operator is being used scrape_timeout is usually adjusted automatically and such an error means that the exporter itself is not reachable

ii. If a standalone Promtheus container/pod is being used, try increasing the scrape_interval and scrape_timeout in the /etc/prometheus/prometheus.cfg file to allow for more time to collect the metrics.

It may be observed that some metrics are being received by Prometheus but have no value associated with them. This means that that the exporter is unable to collect that particular metric from the Citrix ADC. It could be either becuase:

i. The device provided as a --target-nsip is reachable on the IP and port but is not a Citrix ADC, or

ii. The metric being fetched does not exist in the Citrix ADC. Possibly due to it being an invalid metric name.