Rule based notifications and hard states

General
1

Hello everyone,

we want to start with rule based notifications but we want only HARD States to get notified and now we have the following problem.
If we say only the 3rd and following notifications, we get the hard states for WARN, CRIT and UNKOWN but the hard state for OK ist the first one. Has anyone a hint for me?
Thank you

2

Hi @marass,

we have implemented this via the maximum number of check attempts for service and host and use just the normal change in state in our rule based notification rules.

3

When a service or host gets recovered from either WARN/CRIT/UNKW/UNREACHABLE/DOWN state to OK/UP, it’s automatically an hard state (OK).
What you can do is write your own notification script and add some logic to it.
Using for example the notification variables such as NOTIFY_NOTIFICATIONTYPE and NOTIFY_PREVIOUSSERVICEHARDSTATE to get your logic to work.

Still you “can’t run away” from OK/UP being an hard state.

More info here : https://checkmk.com/cms_notifications.html#scripts

Cheers,

1 Like
4

I don’t want to change the HARD state of OK / UP but how to manage only notify hard states via rule based notifications. At this point you have the difference of “max attempts” vs “1” and i don’t want to reduce max attempts to 1. So I think the only way is to create my own script as you @ricardoftribeiro or don’t use rule based notifications.

5

Only HARD states are reliable for notifications, so you shouldn’t have any problem.

For example.

If you have a 3 “Maximum number of check attempts for service” rule for a service:

A service is OK and gets CRITICAL (1st time) - This is SOFT State.
The service’s still CRITITAL (2nd Time) - This is still SOFT state.
The service is CRITICAL (3rd Time) - This is the HARD State -> You notification will be sent.

The service goes OK (Hard State) -> Your notification will be sent.

Cheers,

2 Likes
6

Full ACK. But with rule based notifications I see no option to only react on hard states. You can limit to the x-th notification or only specific services but not to limit to hard states.

7

You don’t see it, but it’s the only way CMK will “lauch” a notification.
You can’t notify anything in SOFT state.

1 Like
8

Ok. I checked my max attempts again and they were on 1. So I increased them and see if it now works like it should. Thanks for your help!

9

There is currently an issue with CMC not numbering the OK notification correctly with 1 but with N+1 following multiple problem notifications.
We already opened a developer ticket for that.