-
Notifications
You must be signed in to change notification settings - Fork 570
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
icinga2 randomly does not reload objects after adding new objects via api #6957
Comments
Please see #6012. |
Nice. fast response! Looks like #5205/#6927 (or the mentioned parent task). So basically I need to trigger lots of restarts since kube-icinga may add lots of objects (also removing them first since there is no way to trigger apply rules for changed objects...) Only workaround is to trigger a restart? |
Up until the underlaying problem inside the IDO feature is fixed, a restart is the only workaround, yes. |
Whats the difference between a service reload via init and a POST /v1/actions/restart-process ? After sending a POST /v1/actions/restart-process my object list in icingaweb is empty and adding the same object again ends in error 500:
(Which is a different error compared to just do restart via systemd) Sending a POST /v1/actions/restart-process would be the only workaround for my app. Otherwise this is gonna be impossible with the actual version of the icinga api.
|
It misses the stage name after |
Argh my fault, I have removed content in /var/lib/icinga2/api/packages/_api manually during debuging and just noticed that files like active-stage.conf, active.conf were missing after restart. But if I create new objects via the api those get created in conf.d folder directly in _api, /var/lib/icinga2/api/packages/_api/conf.d/xxxx. And after restart the service the added services are gone again (But files still there). Maybe a check for that would be helpful (or a log entry somewhere that the stage folder is gone or not active.) Probably the api should respond with a 500 error and not accepting new objects in the first place. |
I've created #6959 as follow-up. I just don't have the time to code any further here, maybe you'd like to catch up on this. |
👍, yes as soon as I have some spare time. |
Will be superseded with IcingaDB, the old tracking for the IDO is #6012. |
Creating new objects does not reflect on the web interface (And i'm pretty sure they do not get checked as well).
The objects are visible after manually restarting icinga.
The problem look quite random (See steps to reproduce):
Again only restarting icinga solves this problem.
Expected Behaviour
New objects (10 test servicegroups (Can be any object types)) are visible in icingaweb.
Current Behaviour
Servicegroups are not visible in the servicegroup list in the icinga web ui.
As far as I can see the web ui fetches its information not from the api but from the mysql db directly.
GET https://localhost:5665/v1/objects/servicegroups
lists all those objects also doesicinga2 object list
.As soon as I restart icinga the objects are visible in the web ui.
Possible Solution
Not sure how this can happen but it looks like a major problem.
Steps to Reproduce (for bugs)
Create file /tmp/test:
cat /tmp/test | while read l; do sh -c "$l"; done
Objects are not visible in the web ui.
(You may need to do this a couple of time since this is not always the case but mostly)
Chances are higher to get some objects if waiting a short time between requests:
cat /tmp/test | while read l; do sh -c "$l"; sleep 2; done
Context
I have this issue in kube-icinga https://github.com/gyselroth/kube-icinga.
This async app does create many api calls within a short time and even async.
Your Environment
icinga2 --version
): r2.10.2-1icinga2 feature list
):icinga2 daemon -C
):zones.conf
file (oricinga2 object list --type Endpoint
andicinga2 object list --type Zone
) from all affected nodes.The text was updated successfully, but these errors were encountered: