various fixes

CHANGELOG.md

@@ -11,6 +11,10 @@
 * [BREAKER!] Removed service.portNameAsCommand option, use kubernetes annotations
 * [FEATURE] Discover persistent volumes and create icinga services for those #4
 * [FIX] Fixed typo Definiton => Definition in default config example
+* [CHANGE] Added various new tests
+* [FIX] kube workes are in no namespace, remove icinga groups
+* [FIX] fixed Error: Invalid attribute specified: host_name\n for kube nodes
+* [FIX] fixed duplicate ingress objects (different path, same ingress names in different namespaces, ...)
 
 
 ## 1.0.1

README.md

@@ -8,9 +8,16 @@ kube-icinga automatically deploys icinga objects out of your kubernetes resource
 It has built-in autodiscovery and will work out of the box. However you can change various default configurations and deploy 
 custom icinga objects or disable/enable kubernetes objects to monitor.
 
+## Features
+
+* Autodiscovery
+* Icinga servicegroup support
+* Create services for kubernetes nodes, services, ingresses and persistent volumes
+* Completely customizable per resurce or per resource type
+
 ## How does it work?
 
-Multiple watchers are bootstraped and listen for any kubernetes changes. Those changes will reflect immediately on icinga.
+Multiple watchers are bootstraped and listen for any kubernetes changes. Those changes will reflect immediately on your icinga environment.
 
 * Kubernetes namespaces will result in icinga service groups and host groups
 * Nodes will result in host objects 
@@ -19,6 +26,7 @@ Multiple watchers are bootstraped and listen for any kubernetes changes. Those c
 * Persistent volumes will result in services attached to a dummy host
 
 >**Note**: [1] You may change this behaviour by attaching the services (ingress paths) to all kubernetes worker nodes by setting `kubernetes.ingresses.attachToNodes` to `true`. (This will result in many more services and checks depending on the size of your cluster!)
+
 >**Note**: [2] NodePort services will always be attached to each kubernetes worker node. See [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport) services for more information.
 
 Since there is no such thing as hosts in the world of moving and containers. The host object (kubernetes metadata.name) on icinga may just be a [dummy](https://www.icinga.com/docs/icinga2/latest/doc/10-icinga-template-library/#plugin-check-command-dummy) and will not get checked
@@ -62,19 +70,34 @@ kubectl -f https://raw.githubusercontent.com/gyselroth/kube-icinga/master/kube-i
 ```
 (Change the secret password and ICINGA_ADDRESS value accordingly)
 
+>**Note**: kube-icinga will be created as single pod deployment in the kubernetes kube-system namespace. You may changes this behaviour.
+
+### Resource visibility
+The resource yaml also contains a new cluster role `kube-icinga`. kube-icinga will create icinga objects for all visible namespaces which are by default all namespaces since it is a kubernetes cluster role.
+You may specify resources visibile to kube-icinga with custom RBAC rules.
 
 ## Advanced topics
 
 ### ClusterIP services
-Ingresses, NodePort and Load Balancer services will expose your kubernetes services to the public but not ClusterIP services. ClusterIP services (Which is the default) are only internal kubernetes services and not available from outside the cluster. If your icinga2 setup lives outside the cluster you only have to options, either disable deployment for those objects or setup an icinga [satelite](https://www.icinga.com/docs/icinga2/latest/doc/06-distributed-monitoring) container which will live in kubernetes. Of course you may also deploy an entire icinga2 setup on kubernetes if you do not have an existing one.
+Ingresses, NodePort and Load Balancer services will expose your kubernetes services to the public but not ClusterIP services. ClusterIP services (Which is the default) are only internal kubernetes services and not available from outside the cluster. If your icinga2 setup lives outside the cluster you only have two options, either disable deployment for those objects or setup an icinga [satelite](https://www.icinga.com/docs/icinga2/latest/doc/06-distributed-monitoring) container which will live on kubernetes. Of course you may also deploy an entire icinga2 setup on kubernetes if you do not have an existing one.
 
 ### Using icinga2 apply rules
-You certainly can use icinga2 apply rules. You may disable auto service deployments via `applyServices` for ingresses and services and define your own services via [apply rules](https://www.icinga.com/docs/icinga2/latest/doc/03-monitoring-basics/#using-apply).
+You certainly can use icinga2 apply rules. You may disable auto service deployments via `applyServices` for ingresses, services and volumes and define your own services via [apply rules](https://www.icinga.com/docs/icinga2/latest/doc/03-monitoring-basics/#using-apply).
 Of course you can also setup a mixed deployment. Automatically let kube-icinga deploy services and apply additional services via apply rules. 
 
 >**Note**: Since icinga apply rules are [not triggered](https://www.icinga.com/docs/icinga2/latest/doc/12-icinga2-api/#modifying-objects) if an object gets updated kube-icinga will delete and recreate those objects. 
 
-All icinga host object are created with all kubernetes data available packed in the variable `vars.kubernetes`. Therefore you can apply rules with this data, for example this will create an icinga service for all objects with a kubernetes label foo=bar.
+All icinga host objects are created with all kubernetes data available packed in the variable `vars.kubernetes`. Therefore you may apply rules with this data.
+
+For example kube-icinga will create a icinga host object for a kubernetes service:
+```
+
+```
+
+You may use this data to dynamically create icinga apply rules:
+```
+```
+
 
 ### Globally overwrite icinga object definitions
 It is possible to set custom options for the icinga objects during creation. You might set custom values via `kubernetes.ingresses.hostTemplates` or kubernetes.ingresses.serviceTemplate`. The same can also be done for services. For example it might be crucial to set a custom zone for ClusterIP services since they are only reachable within the cluster and shall be monitored by an icinga satelite node. Any valid setting for a specific icinga object can be set.
@@ -101,9 +124,13 @@ you may configure custom incinga attributes directly in the kubernetes resource
 ### Overwrite icinga object definitions directly in kubernetes resources
 
 kube-icinga is able to parse kubernetes annotations and merge those with its default settings.
+
+>**Note**: With annotations you may set specific settings for each kubernetes resources while configure options for kube-icinga set settings on a resource type basis.
+
 You may use the following annotations:
 
 | Name | Description |
+|-------|------------|
 | `kube-icinga/check_command` | Use a custom icinga check command. |
 | `kube-icinga/template` | Use a custom icinga template. |
 | `kube-icinga/definition` | JSON encoded icinga definiton which may contain advanced icinga options and gets merged with the defaults. |

config.json

@@ -34,7 +34,7 @@
  },
  "services": {
  "ClusterIP": {
- "discover": false,
+ "discover": true,
  "hostDefinition": {},
  "serviceTemplates": ["generic-service"],
  "hostTemplates": ["generic-host"],

docker-compose-dev.yml

@@ -6,9 +6,8 @@ services:
  # Set your hostname to the FQDN under which your
  # sattelites will reach this container
  hostname: icinga2
- env_file:
- - secrets_sql.env
  environment:
+ - MYSQL_ROOT_PASSWORD=<password>
  - ICINGA2_FEATURE_GRAPHITE=1
  # Important:
  # keep the hostname graphite the same as
@@ -55,8 +54,8 @@ services:
  - ./data/graphite/log/carbon:/var/log/carbon
  mysql:
  image: mariadb:10.1
- env_file:
-  - secrets_sql.env
+ environment:
+ - MYSQL_ROOT_PASSWORD=<password>
  volumes:
  - ./data/mysql/data:/var/lib/mysql
  # If you have previously used the container's internal DB use:

src/icinga.ts

@@ -177,11 +177,12 @@ export default class Icinga {
  this.logger.info(`delete service ${name} from host ${host}`);
 
  return this.icingaClient.deleteService(name, host, (err, result) => {
- resolve(true);
  if (err) {
  this.logger.error(`failed delete service ${name} from host ${host}`, {error: err});
+ resolve(false);
  } else {
  this.logger.info(`service ${name} was deleted successfully from host ${host}`, {result: result});
+ resolve(true);
  }
  });
  });
@@ -195,11 +196,12 @@ export default class Icinga {
  this.logger.info(`delete host ${name}`);
 
  this.icingaClient.deleteHost(name, (err, result) => {
- resolve(true);
  if (err) {
  this.logger.error(`failed delete host ${name}`, {error: err});
+ resolve(false);
  } else {
  this.logger.info(`host ${name} was deleted successfully`, {result: result});
+ resolve(true);
  }
  });
  });

src/kube/ingress.ts

@@ -4,12 +4,21 @@ import JSONStream from 'json-stream';
 import KubeNode from './node';
 import Resource from './abstract.resource';
 
+interface IngressOptions {
+ discover?: boolean;
+ applyServices?: boolean;
+ attachToNodes?: boolean;
+ hostDefinition?: object;
+ serviceDefinition?: object;
+ hostTemplates?: string[];
+ serviceTemplates?: string[];
+}
+
 /**
  * kubernetes ingresses
  */
 export default class Ingress extends Resource {
  protected logger: Logger;
- protected kubeClient;
  protected icinga: Icinga;
  protected jsonStream: JSONStream;
  protected kubeNode: KubeNode;
@@ -25,10 +34,9 @@ export default class Ingress extends Resource {
  /**
  * kubernetes hosts
  */
- constructor(logger: Logger, kubeNode: KubeNode, kubeClient, icinga: Icinga, jsonStream: JSONStream, options) {
+ constructor(logger: Logger, kubeNode: KubeNode, icinga: Icinga, jsonStream: JSONStream, options: IngressOptions) {
  super();
  this.logger = logger;
- this.kubeClient = kubeClient;
  this.icinga = icinga;
  this.jsonStream = jsonStream;
  this.kubeNode = kubeNode;
@@ -71,8 +79,9 @@ export default class Ingress extends Resource {
  * Preapre icinga object and apply
  */
  public async prepareObject(definition: any): Promise<any> {
+ let hostname = this.escapeName(['ingress', definition.metadata.namespace, definition.metadata.name].join('-'));
  if (!this.options.attachToNodes) {
- await this.applyHost(definition.metadata.name, definition.metadata.name, definition, this.options.hostTemplates);
+ await this.applyHost(hostname, definition.metadata.name, definition, this.options.hostTemplates);
  }
 
  let service = this.prepareResource(definition);
@@ -85,9 +94,10 @@ export default class Ingress extends Resource {
  for (const spec of definition.spec.rules) {
  for (const path of spec.http.paths) {
  let base = path.path || '/';
+ let name = this.escapeName([spec.host, 'http', base].join('-')); 
  let addition = {
  'check_command': 'http',
- 'display_name': `${spec.host}:http`,
+ 'display_name': `${spec.host}${base}:http`,
  'vars._kubernetes': true,
  'vars.kubernetes': definition,
  'vars.http_address': spec.host,
@@ -99,13 +109,14 @@ export default class Ingress extends Resource {
 
  Object.assign(addition, this.options.serviceDefinition);
  Object.assign(addition, service);
- this.applyService(definition.metadata.name, addition.display_name, addition, templates);
+ this.applyService(hostname, name, addition, templates);
 
  // tls secret set, also apply https service
  if (definition.spec.tls) {
+ name = this.escapeName([spec.host, 'https', base].join('-')); 
  addition.display_name += 's';
  addition['vars.http_ssl'] = true;
- this.applyService(definition.metadata.name, addition.display_name, addition, templates);
+ this.applyService(hostname, name, addition, templates);
  }
  }
  }
@@ -115,9 +126,9 @@ export default class Ingress extends Resource {
  /**
  * Start kube listener
  */
- public async kubeListener(): Promise<any> {
+ public async kubeListener(provider) {
  try {
- const stream = this.kubeClient.apis.extensions.v1beta1.watch.ingresses.getStream();
+ var stream = provider();
  stream.pipe(this.jsonStream);
  this.jsonStream.on('data', async (object) => {
  this.logger.debug('received kubernetes ingress resource', {object});
@@ -139,7 +150,7 @@ export default class Ingress extends Resource {
  });
 
  this.jsonStream.on('finish', () => {
- this.kubeListener();
+ this.kubeListener(provider);
  });
  } catch (err) {
  this.logger.error('failed start ingresses listener', {error: err});

src/kube/node.ts

@@ -3,28 +3,32 @@ import Icinga from '../icinga';
 import JSONStream from 'json-stream';
 import Resource from './abstract.resource';
 
+interface NodeOptions {
+ discover?: boolean;
+ hostDefinition?: object;
+ hostTemplates?: string[];
+}
+
 /**
  * kubernetes hosts
  */
 export default class Node extends Resource {
  protected logger: Logger;
- protected kubeClient;
  protected icinga: Icinga;
  protected jsonStream: JSONStream;
  protected nodes: string[] = [];
- protected options = {
- discovery: true,
+ protected options: NodeOptions = {
+ discover: true,
  hostDefinition: {},
  hostTemplates: [],
  };
 
  /**
  * kubernetes hosts
  */
- constructor(logger: Logger, kubeClient, icinga: Icinga, jsonStream: JSONStream, options) {
+ constructor(logger: Logger, icinga: Icinga, jsonStream: JSONStream, options: NodeOptions) {
  super();
  this.logger = logger;
- this.kubeClient = kubeClient;
  this.icinga = icinga;
  this.jsonStream = jsonStream;
  this.options = Object.assign(this.options, options);
@@ -33,22 +37,22 @@ export default class Node extends Resource {
  /**
  * Preapre icinga object and apply
  */
- protected async prepareObject(definition: any): Promise<boolean> {
+ public async prepareObject(definition: any): Promise<boolean> {
  let host = {
  'display_name': definition.metadata.name,
- 'host_name': definition.metadata.name,
+ 'address': definition.metadata.name,
  'vars._kubernetes': true,
  'vars.kubernetes': definition,
- 'groups': [definition.metadata.namespace],
+ 'check_command': 'ping', 
  };
 
  if (!definition.spec.unschedulable) {
  this.logger.debug('skip kube worker node '+definition.metadata.name+' since it is flagged as unschedulable');
  this.nodes.push(definition.metadata.name);
  }
-
- host = Object.assign(host, this.options.hostDefinition);
- return this.icinga.applyHost(host.host_name, host, this.options.hostTemplates);
+ 
+ Object.assign(host, this.options.hostDefinition);
+ return this.icinga.applyHost(definition.metadata.name, host, this.options.hostTemplates);
  }
 
  /**
@@ -61,9 +65,9 @@ export default class Node extends Resource {
  /**
  * Start kube listener
  */
- public async kubeListener(): Promise<any> {
+ public async kubeListener(provider) {
  try {
- const stream = this.kubeClient.apis.v1.watch.nodes.getStream();
+ const stream = provider();
  stream.pipe(this.jsonStream);
  this.jsonStream.on('data', async (object) => {
  // ignore MODIFIER for kube nodes
@@ -89,7 +93,7 @@ export default class Node extends Resource {
  });
 
  this.jsonStream.on('finish', () => {
- this.kubeListener();
+ this.kubeListener(provider);
  });
  } catch (err) {
  this.logger.error('failed start nodes listener', {error: err});