Periodic service discovery - Only one rule per server carried out

[LaSoe (Lars Sörensen)]

I have noticed that only one periodic service discovery rule per server can be carried out.

Example:
I have a periodic service discovery rules that runs every Day at 8th o clock and adds/removes Service “A” on all Linux Server and a second rules that runs every Monday and adds/remove Service “B” on a Subgroup X of Linux servers.

On the Subgroup X of Linux servers, the Rule for Service “A” is never executed because only the first matching rule for Service “B” will be executed for this Servers.

When later somebody else defines another periodic service discovery rule that matches by random one member of this Subgroup X the chances are high that the Rule for Service “B” also won’t get executed anymore on this one Server because the new rule matches first. So in the end you never know which rule will be executed or you wonder why your rule gets suddenly not executed anymore on a server.

From my point of view, it makes no sense that only the first periodic service discovery rule per server is carried out. Is there any way to run diffrent periodic service discoveries for one server?

[andreas-doehler (Andreas)]

The discovery check is something like a classic Nagios check and cannot use timespecific parameters like “normal” CMK checks.

What is the real use case for such a setup? I cannot see any benefit if you split the discovery.

[LaSoe (Lars Sörensen)]

We have different services on different servers which should be added / removed automatically at different timesperiods.

When diffrent rules overlap, only the first matching rules applies and all other rules are not executed, even if different time periods and services are involved.

Another example:

• Rule1: add new Filesystem on all Server once a week
• Rule 2: add autodiscovered IBM Queues on all IBM MQ Server on Monday morning
• Rule 3: add autodiscovered Customer Services on IBM MQ Server “DEMO-123” each Thursday after deplayment window

Outcome:
Rule 3 (Customer Services) gets executetd on Server “DEMO-123”. Rule 1 and 2 won’t be executed even although the conditions are met because Rule 3 (first match).
Rule 2 (IBM Queues) gets executet on all IBM MQ Server except on “DEMO-123” because of Rule 3.
Rule 1 (Filesystem) will be executet on all Servers except on the MQ servers and “DEMO-123” because of Rule 2 or Rule 3.

And the Customer is keeping wondering why the new added Queues and Filesystems on “DEMO-123” won’t show up automaticly on the Service List of his Server.

[andreas-doehler (Andreas)]

I understand what you want but as i wrote the discovery check is a classic check where you cannot define time specific parameters.
But you have another option → CMK Rest API has some command regarding the discovery of services. update_discovery_phase should move some found services to the status monitored. You can define what should be monitored.

[LaSoe (Lars Sörensen)]

Ok, my example was a little constructed. Here a more real live example

Rule1: add new filesystem on all server every day at at 08:00
Rule 2: add autodiscovered IBM Queues on all IBM MQ Server every hour
Rule 3: add/remove autodiscovered Custom Service on Server “Custom-123” all 7 day at 20:00

But the Time of execution is not the point. The fact is that you can’t have more then one rule per server, and you are never sure which rule will be executed in the end.

Unfortunately, the examples in the API documentation are very poor. A more practical example how to do the same via REST-API would be very helpful since not all of us are API specialists and use the API every day. As example howto add and remove all “IBM MQ Queues” services on all MQ servers except Queues named “IBM MQ Queues.*TEST” would be a great help.

[andreas-doehler (Andreas)]

If you use the API you can do a

curl -X 'GET' \
  'https://MONSERVER/SITENAME/check_mk/api/1.0/domain-types/service/collections/services?host_name=HOSTNAME&discovery_phase=undecided' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic authstring'

This will output all the undecided services of a host.
Now you process the response an take the services you want to activate. The script should issue then the command.

curl -X 'PUT' \
  'https://MONSERVER/SITENAME/check_mk/api/1.0/objects/host/HOSTNAME/actions/update_discovery_phase/invoke' \
  -H 'accept: */*' \
  -H 'Authorization: Basic authstring' \
  -H 'Content-Type: application/json' \
  -d '{
            "target_phase": "monitored",
            "check_type": "CHECKTYPE-FROM-RESPONSE",
            "service_item": "ITEM-FROM-RESPONSE"
}'

You can also set services to ignored with a other target state.

I found at my test here only one problem that the “target_phase” “monitored” is not working at the moment, but set a service to ignored is working as expected. Strange !!!
I make another topic with this bug.

[LaSoe (Lars Sörensen)]

You will surely agree that the API cannot be a real replacement for the periodic service discovery rule.

For most users of CMK the API is not a valid replacement because it requires programming skills and you have to rebuild the whole logic of the periodic service discovery rule with all its facets.

For each discovery task a separate script has to be written. Especially if this has to be executed across multiple sites, it quickly becomes complicated and the implementation very time-consuming. This makes maintenance very cumbersome and makes it difficult to keep track of what is being done where.