Icinga memory usage is increasing continuously.

[samta93]

Expected Behavior

Memory usage should not be keep inscreasing.

Current Behavior

When adding more 1k host objects the memory usage for icinga process is keep increasing.

Possible Solution

Steps to Reproduce (for bugs)

1. Create 1000 host objects.

Context

We are creating 1k host objects, we see that there is api call to icinga api in the logs ([2018-04-12 04:13:52 -0700] information/HttpServerConnection: Request: GET /v1/objects/hosts (from [127.0.0.1]:34770, user: admin)) and right after each call memory usage is increasing.

This might be happening because after each api calls, we see a thread being spawned which never dies, ps -m pidof icinga2 | wc -l | xargs

Your Environment

• Version used (icinga2 --version): (version: r2.5.4-1)
• Operating System and version:
  Platform: Ubuntu
  Platform version: 14.04.5 LTS, Trusty Tahr
  Kernel: Linux
  Kernel version: 3.13.0-141-generic
  Architecture: x86_64
• Enabled features (icinga2 feature list):
  Disabled features: compatlog gelf graphite influxdb livestatus opentsdb statusdata syslog
  Enabled features: api checker command debuglog ido-mysql mainlog notification perfdata
• Icinga Web 2 version and modules (System - About):
• Config validation (icinga2 daemon -C):
  information/cli: Icinga application loader (version: r2.5.4-1)
  information/cli: Loading configuration file(s).
  information/ConfigItem: Committing config item(s).
  information/ApiListener: My API identity: centralmsvm.englab.juniper.net

warning/ApplyRule: Apply rule 'disable-host-service-notifications' (in /etc/icinga2/conf.d/dependency.conf: 1:0-1:63) for type 'Dependency' does not match anywhere!
warning/ApplyRule: Apply rule 'disable-host-service-checks' (in /etc/icinga2/conf.d/dependency.conf: 6:1-6:57) for type 'Dependency' does not match anywhere!
warning/ApplyRule: Apply rule 'satellite-host' (in /etc/icinga2/conf.d/satellite.conf: 29:1-29:41) for type 'Dependency' does not match anywhere!
warning/ApplyRule: Apply rule 'sendnotify' (in /etc/icinga2/conf.d/notifications.conf: 11:1-11:39) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'sendnotify' (in /etc/icinga2/conf.d/notifications.conf: 17:1-17:42) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'sendnotify_ne' (in /etc/icinga2/conf.d/notifications.conf: 23:1-23:42) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'sendnotify_ne' (in /etc/icinga2/conf.d/notifications.conf: 29:1-29:45) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'autoheal-sendnotify' (in /etc/icinga2/conf.d/notifications.conf: 50:1-50:48) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'autoheal-sendnotify' (in /etc/icinga2/conf.d/notifications.conf: 56:1-56:51) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'backup-downtime' (in /etc/icinga2/conf.d/downtimes.conf: 5:1-5:52) for type 'ScheduledDowntime' does not match anywhere!
information/ConfigItem: Instantiated 2 ApiUsers.
information/ConfigItem: Instantiated 1 ApiListener.
information/ConfigItem: Instantiated 1 Zone.
information/ConfigItem: Instantiated 2 FileLoggers.
information/ConfigItem: Instantiated 1 Endpoint.
information/ConfigItem: Instantiated 10 NotificationCommands.
information/ConfigItem: Instantiated 23832 Notifications.
information/ConfigItem: Instantiated 201 CheckCommands.
information/ConfigItem: Instantiated 2754 Hosts.
information/ConfigItem: Instantiated 1 IcingaApplication.
information/ConfigItem: Instantiated 2784 HostGroups.
information/ConfigItem: Instantiated 1 EventCommand.
information/ConfigItem: Instantiated 1 UserGroup.
information/ConfigItem: Instantiated 1 User.
information/ConfigItem: Instantiated 21078 Services.
information/ConfigItem: Instantiated 3 TimePeriods.
information/ConfigItem: Instantiated 3 ServiceGroups.
information/ConfigItem: Instantiated 1 CheckerComponent.
information/ConfigItem: Instantiated 1 ExternalCommandListener.
information/ConfigItem: Instantiated 1 IdoMysqlConnection.
information/ConfigItem: Instantiated 1 NotificationComponent.
information/ConfigItem: Instantiated 1 PerfdataWriter.
information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
information/cli: Finished validating the configuration file(s)

• If you run multiple Icinga 2 instances, the zones.conf file (or icinga2 object list --type Endpoint and icinga2 object list --type Zone) from all affected nodes.

[dnsmichi]

v2.5.x is not supported anymore. In terms of the problem, this is a duplicate of #5148.

[samta93]

Hi @dnsmichi

Is this resolved #5148 with latest icinga, I have upgraded to r2.8.2-1 and still saw the same issue.

[dnsmichi]

The linked issue is still open and under investigation.

[samta93]

Thanks @dnsmichi I will wait for this issue to get resolved. Please let me know if need any help with test and verification.