We’ve Checkmk setup to monitor some servers and we have alerting setup to call our support number between 7AM and 7PM based on some criticality rules in notifications. However, if a failure occurs outside of that window, Checkmk will not call us, which is expected, but we’re also missing the notifications. Is it possible to have these notifications queued up and when 7AM comes, we’d be notified about failures that occurred over night?
the problem with your setup is, that the notification itself is created outside of your service hours, forwarded into the notification system and your notification rule is not being matched by it because the notification rule has a condition for a timeperiod that does not reflect the current time.
So the notification will not match any notification rule and it will be lost in space.
By default checkmk will only create one notification for a state change, so you will not be notified any more, after the first notification was created.
What you can do is to set so called “notification periods” for your services and or hosts.
In this way the monitoring core is responsible to create the notification, when the notification period gets active and the error still persists and will create a new notification the beginning of your configured timeperiod.
From the docs : Monitoring events for an object that is not currently in its notification period will not be notified. Such notifications will be ‘reissued’ when the notification period is again active – if the host/service is still in a problem state. Only the latest state will be notified even if multiple changes to the object’s state have occurred during the time outside the notification period.
A second option would be to recreate notifications with the ruleset “periodic notifications during service problems” to recreate the notifications when the problem is not solved during a speciefied timeframe, but i think the first option solves your problem in a better way.
Hi Andre,
Thank you very much for taking the time to respond and for both solutions. Notification Periods sounds like the answer for our issue, will try that out and provide some feedback.
Have a great weekend!
If @aeckstein 's reply helped you, please consider marking it as a solution – that way you will say thank you to Andre and also show other community members that the question is resolved. Thank you!