[Check_mk (english)] Getting unexpected notifications when DOWNTIME Starts

Hi,
Have set up scheduled downtime for a
host and when the host goes down (within that period) I still get 2 notifications,
and I can’t work out why?
I get 2 notifications:
Check_MK: n000z306 - UP -> DOWN
and
Check_MK: n000z306/Check_MK OK ->
CRIT

The logs show

2016-10-20 14:38:48 Global rule ‘Email_on_host_problems’…

2016-10-20 14:38:48 Event type is ur

2016-10-20 14:38:48 -> matches!

2016-10-20 14:38:48 - adding
notification of xxxxxxxxxxxxx via mail

for both of these…Not sure why these
are not being flagged as inside the DOWNTIME period?

Note that most notifications for the
other services etc in this downtime are event type s for this so don’t
get sent.

In WATO notification for email, start
and end of a scheduled downtime is not ticked.

Any ideas/pointers?

OMD 1.30 Check_mk 1.2.6p12

Ben Shade

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

(1.2.8p12 CRE on RHEL 6)

Testing service notifications:
On one of my hosts:
Set host to scheduled downtime for 2 hours. (all services are also

in scheduled downtime)
chmod 444 /usr/bin/check_mk_agent
Verified that the host is showing a critical alert for the check_mk
service.
Waiting 30 minutes to see if a notification is generated.
No notification was generated (checking notify.log) after 40 minutes
(10 minutes past my configured service notification delay)

···

On 10/20/2016 10:08 AM, Ben Shade
wrote:

Hi,

          Have set up scheduled downtime

for a
host and when the host goes down (within that period) I still
get 2 notifications,
and I can’t work out why?

  I get 2 notifications:


  **          Check_MK: n000z306 - UP ->

DOWN**
and

  **          Check_MK: n000z306/Check_MK OK


CRIT**

  The logs show




          2016-10-20 14:38:48 Global rule

‘Email_on_host_problems’…

          2016-10-20 14:38:48 Event type is

ur

          2016-10-20 14:38:48 �->

matches!

          2016-10-20 14:38:48 � �- adding

notification of xxxxxxxxxxxxx via mail

          for both of these.....Not sure

why these
are not being flagged as inside the DOWNTIME period?

          Note that most notifications for

the
other services etc in this downtime are event type s for this so
don’t
get sent.

          In WATO notification for email,

start
and end of a scheduled downtime is not ticked.

  Any ideas/pointers?




  OMD 1.30 �Check_mk 1.2.6p12




  Ben Shade




    Unless stated otherwise above:

    IBM United Kingdom Limited - Registered in England and Wales

with number
741598.

    Registered office: PO Box 41, North Harbour, Portsmouth,

Hampshire PO6
3AU

_______________________________________________
checkmk-en mailing list

checkmk-en@lists.mathias-kettner.dehttp://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

Yes that is what I would expect.
I have just realised that I am using
flexible (flexible with max. duration) option…and
on testing that seems to be the ‘issue’ as normal downtime option seems
to work fine. I forgot that bit!
My notifications (with flexible on)
are generated immediately the server goes down ( I am shutting it down
as this is now using a test server), but I have also tried waiting a while
after the start of the scheduled shutdown (but with flexible on) before
actually shutting the system down but get the same.
With flexible on is it expected that
we get a notification or two? (In a previous life a few years ago
with ‘nagios’ I didn’t think I did, but may be wrong. I am expecting I
have configued something wrong still but can’t see what it is)
Thanks
Ben

···

(1.2.8p12 CRE on RHEL 6)
Testing service notifications:
On one of my hosts:
Set host to scheduled downtime for 2 hours. (all services are also in scheduled
downtime)
chmod 444 /usr/bin/check_mk_agent
Verified that the host is showing a critical alert for the check_mk service.
Waiting 30 minutes to see if a notification is generated.
No notification was generated (checking notify.log) after 40 minutes (10
minutes past my configured service notification delay)
On 10/20/2016 10:08 AM, Ben Shade wrote:
Hi,
Have set up scheduled downtime for a host and when the host goes down (within
that period) I still get 2 notifications, and I can’t work out why?
I get 2 notifications:
**
Check_MK: n000z306 - UP -> DOWN** and **
Check_MK: n000z306/Check_MK OK -> CRIT**

The logs show

2016-10-20 14:38:48 Global rule ‘Email_on_host_problems’…

2016-10-20 14:38:48 Event type is ur

2016-10-20 14:38:48 -> matches!

2016-10-20 14:38:48 - adding notification of xxxxxxxxxxxxx
via mail

for both of these…Not sure why these are not being flagged as inside
the DOWNTIME period?

Note that most notifications for the other services etc in this downtime
are event type s for this so don’t get sent.

In WATO notification for email, start and end of a scheduled downtime is
not ticked.

Any ideas/pointers?

OMD 1.30 Check_mk 1.2.6p12

Ben Shade

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

`_______________________________________________

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

[http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en`](http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en)

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

As I understand it, with flexible notifications, it’s not in downtime until

some event occurs and then it is probably too late to suppress the notification.

···

(Just guessing as I haven’t tried to look at the code.)

Sent from my iPad

On Oct 20, 2016, at 1:16 PM, Ben Shade benjamin_shade@uk.ibm.com wrote:

Yes that is what I would expect.
I have just realised that I am using
flexible (flexible with max. duration) option…and
on testing that seems to be the ‘issue’ as normal downtime option seems
to work fine. I forgot that bit!
My notifications (with flexible on)
are generated immediately the server goes down ( I am shutting it down
as this is now using a test server), but I have also tried waiting a while
after the start of the scheduled shutdown (but with flexible on) before
actually shutting the system down but get the same.
With flexible on is it expected that
we get a notification or two? (In a previous life a few years ago
with ‘nagios’ I didn’t think I did, but may be wrong. I am expecting I
have configued something wrong still but can’t see what it is)
Thanks
Ben

(1.2.8p12 CRE on RHEL 6)
Testing service notifications:
On one of my hosts:
Set host to scheduled downtime for 2 hours. (all services are also in scheduled
downtime)
chmod 444 /usr/bin/check_mk_agent
Verified that the host is showing a critical alert for the check_mk service.
Waiting 30 minutes to see if a notification is generated.
No notification was generated (checking notify.log) after 40 minutes (10
minutes past my configured service notification delay)
On 10/20/2016 10:08 AM, Ben Shade wrote:
Hi,
Have set up scheduled downtime for a host and when the host goes down (within
that period) I still get 2 notifications, and I can’t work out why?
I get 2 notifications:
**
Check_MK: n000z306 - UP → DOWN** and **
Check_MK: n000z306/Check_MK OK → CRIT**

The logs show

2016-10-20 14:38:48 Global rule ‘Email_on_host_problems’…

2016-10-20 14:38:48 Event type is ur

2016-10-20 14:38:48 → matches!

2016-10-20 14:38:48 - adding notification of xxxxxxxxxxxxx
via mail

for both of these…Not sure why these are not being flagged as inside
the DOWNTIME period?

Note that most notifications for the other services etc in this downtime
are event type s for this so don’t get sent.

In WATO notification for email, start and end of a scheduled downtime is
not ticked.

Any ideas/pointers?

OMD 1.30 Check_mk 1.2.6p12

Ben Shade

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

`_______________________________________________

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

[http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en`](http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en)

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

Also, if you are getting notifications immediately when the server
goes down,
then you probably don’t have any rules setup for host and service
notification delay.
You could create a couple of rules and restrict them to a single
host, one for host
notification delay, the other for service notification delay…say
for 5 minutes, then
see if that keeps the notifications from being generated since the
host should be
safely in scheduled downtime within 5 minutes of whatever event
occurred to
trigger the flexible downtime (assuming 1 minute polling intervals).

If that works, remove the host specific restriction and modify the

delay to suit your needs.

···

On 10/20/2016 01:16 PM, Ben Shade
wrote:

    Yes that is what I

would expect.

          I have just realised that I am

using
flexible �( flexible with max. duration)
option…and
on testing that seems to be the ‘issue’ as normal downtime
option seems
to work fine. I forgot that bit!

          My notifications (with flexible

on)
are generated immediately the server goes down ( I am shutting
it down
as this is now using a test server), but I have also tried
waiting a while
after the start of the scheduled shutdown (but with flexible on)
before
actually shutting the system down but get the same.

          With flexible on is it expected

that
we get a notification or two? �(In a previous life a few years
ago
with ‘nagios’ I didn’t think I did, but may be wrong. I am
expecting I
have configued something wrong still but can’t see what it is)

  Thanks


  Ben

  ---------------------------





  (1.2.8p12 CRE on RHEL 6)

    Testing service notifications:
    On one of my hosts:
    Set host to scheduled downtime for 2 hours. (all services are

also in scheduled
downtime)
chmod 444 /usr/bin/check_mk_agent
Verified that the host is showing a critical alert for the
check_mk service.
Waiting 30 minutes to see if a notification is generated.
No notification was generated (checking notify.log) after 40
minutes (10
minutes past my configured service notification delay)

  On 10/20/2016 10:08 AM, Ben Shade wrote:

  Hi,

    Have set up scheduled downtime for a host and when the host goes

down (within
that period) I still get 2 notifications, and I can’t work out
why?

    I get 2 notifications:
  **
      Check_MK: n000z306 - UP -> DOWN** and **
      Check_MK: n000z306/Check_MK OK -> CRIT**



    The logs show



    2016-10-20 14:38:48 Global rule 'Email_on_host_problems'...


    2016-10-20 14:38:48 Event type is ur

    2016-10-20 14:38:48 �-> matches!

    2016-10-20 14:38:48 � �- adding notification of xxxxxxxxxxxxx

via mail

    for both of these.....Not sure why these are not being flagged

as inside
the DOWNTIME period?

    Note that most notifications for the other services etc in this

downtime
are event type s for this so don’t get sent.

    In WATO notification for email, start and end of a scheduled

downtime is
not ticked.

    Any ideas/pointers?



    OMD 1.30 �Check_mk 1.2.6p12



    Ben Shade



    Unless stated otherwise above:

    IBM United Kingdom Limited - Registered in England and Wales

with number
741598.

    Registered office: PO Box 41, North Harbour, Portsmouth,

Hampshire PO6
3AU

  `_______________________________________________

      checkmk-en mailing list

    ``checkmk-en@lists.mathias-kettner.de``

    `[`http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en`](http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en)








    Unless stated otherwise above:

    IBM United Kingdom Limited - Registered in England and Wales

with number
741598.

    Registered office: PO Box 41, North Harbour, Portsmouth,

Hampshire PO6
3AU

Good plan, but still does the same. Thanks
for your ideas. I think you are right that with flex I will have to accept
a notification as that is how it work.
I think I will work with (non flex)
schedules for the moment as I can probably make it work as I need . Thanks
very much for you help.
Ben
Also, if you are getting notifications immediately when
the server goes down,
then you probably don’t have any rules setup for host and service notification
delay.
You could create a couple of rules and restrict them to a single host,
one for host
notification delay, the other for service notification delay…say for
5 minutes, then
see if that keeps the notifications from being generated since the host
should be
safely in scheduled downtime within 5 minutes of whatever event occurred
to
trigger the flexible downtime (assuming 1 minute polling intervals).
If that works, remove the host specific restriction and modify the delay
to suit your needs.

···

On 10/20/2016 01:16 PM, Ben Shade wrote:
Yes that is what I would expect.
I have just realised that I am using flexible (flexible
with max. duration) option…and on testing that seems to be the ‘issue’
as normal downtime option seems to work fine. I forgot that bit!
My notifications (with flexible on) are generated immediately the server
goes down ( I am shutting it down as this is now using a test server),
but I have also tried waiting a while after the start of the scheduled
shutdown (but with flexible on) before actually shutting the system down
but get the same.
With flexible on is it expected that we get a notification or two? (In
a previous life a few years ago with ‘nagios’ I didn’t think I did, but
may be wrong. I am expecting I have configued something wrong still but
can’t see what it is)
Thanks
Ben

(1.2.8p12 CRE on RHEL 6)
Testing service notifications:
On one of my hosts:
Set host to scheduled downtime for 2 hours. (all services are also in scheduled
downtime)
chmod 444 /usr/bin/check_mk_agent
Verified that the host is showing a critical alert for the check_mk service.
Waiting 30 minutes to see if a notification is generated.
No notification was generated (checking notify.log) after 40 minutes (10
minutes past my configured service notification delay)
On 10/20/2016 10:08 AM, Ben Shade wrote:
Hi,
Have set up scheduled downtime for a host and when the host goes down (within
that period) I still get 2 notifications, and I can’t work out why?
I get 2 notifications: **
Check_MK: n000z306 - UP -> DOWN** and **
Check_MK: n000z306/Check_MK OK -> CRIT**

The logs show

2016-10-20 14:38:48 Global rule ‘Email_on_host_problems’…

2016-10-20 14:38:48 Event type is ur

2016-10-20 14:38:48 -> matches!

2016-10-20 14:38:48 - adding notification of xxxxxxxxxxxxx
via mail

for both of these…Not sure why these are not being flagged as inside
the DOWNTIME period?

Note that most notifications for the other services etc in this downtime
are event type s for this so don’t get sent.

In WATO notification for email, start and end of a scheduled downtime is
not ticked.

Any ideas/pointers?

OMD 1.30 Check_mk 1.2.6p12

Ben Shade

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

`


checkmk-en mailing list`

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

If you configured rules for notification delay and are still getting
one or more notifications
immediately when the host or service goes down, then something is
wrong.

···

On 10/21/2016 6:19 AM, Ben Shade wrote:

    Good plan, but still

does the same. Thanks
for your ideas. I think you are right that with flex I will have
to accept
a notification as that is how it work.

          I think I will work with (non

flex)
schedules for the moment as I can probably make it work as I
need . Thanks
very much for you help.

  Ben






          Also, if you are getting notifications

immediately when
the server goes down,
then you probably don’t have any rules setup for host and
service notification
delay.
You could create a couple of rules and restrict them to a single
host,
one for host
notification delay, the other for service notification
delay…say for
5 minutes, then
see if that keeps the notifications from being generated since
the host
should be
safely in scheduled downtime within 5 minutes of whatever event
occurred
to
trigger the flexible downtime (assuming 1 minute polling
intervals).
If that works, remove the host specific restriction and modify
the delay
to suit your needs.

  On 10/20/2016 01:16 PM, Ben Shade wrote:

  Yes that is what I would expect.


    I have just realised that I am using flexible �(        flexible

with max. duration) option…and on testing that seems to be the
‘issue’
as normal downtime option seems to work fine. I forgot that bit!

    My notifications (with flexible on) are generated immediately

the server
goes down ( I am shutting it down as this is now using a test
server),
but I have also tried waiting a while after the start of the
scheduled
shutdown (but with flexible on) before actually shutting the
system down
but get the same.

    With flexible on is it expected that we get a notification or

two? �(In
a previous life a few years ago with ‘nagios’ I didn’t think I
did, but
may be wrong. I am expecting I have configued something wrong
still but
can’t see what it is)

    Thanks

    Ben
    ---------------------------




    (1.2.8p12 CRE on RHEL 6)

    Testing service notifications:
    On one of my hosts:
    Set host to scheduled downtime for 2 hours. (all services are

also in scheduled
downtime)
chmod 444 /usr/bin/check_mk_agent
Verified that the host is showing a critical alert for the
check_mk service.
Waiting 30 minutes to see if a notification is generated.
No notification was generated (checking notify.log) after 40
minutes (10
minutes past my configured service notification delay)

    On 10/20/2016 10:08 AM, Ben Shade wrote:
    Hi,

    Have set up scheduled downtime for a host and when the host goes

down (within
that period) I still get 2 notifications, and I can’t work out
why?

    I get 2 notifications: **

      Check_MK: n000z306 - UP -> DOWN** and **
      Check_MK: n000z306/Check_MK OK -> CRIT**



    The logs show



    2016-10-20 14:38:48 Global rule 'Email_on_host_problems'...


    2016-10-20 14:38:48 Event type is ur

    2016-10-20 14:38:48 �-> matches!

    2016-10-20 14:38:48 � �- adding notification of xxxxxxxxxxxxx

via mail

    for both of these.....Not sure why these are not being flagged

as inside
the DOWNTIME period?

    Note that most notifications for the other services etc in this

downtime
are event type s for this so don’t get sent.

    In WATO notification for email, start and end of a scheduled

downtime is
not ticked.

    Any ideas/pointers?



    OMD 1.30 �Check_mk 1.2.6p12



    Ben Shade



    Unless stated otherwise above:

    IBM United Kingdom Limited - Registered in England and Wales

with number
741598.

    Registered office: PO Box 41, North Harbour, Portsmouth,

Hampshire PO6
3AU

  `

      _______________________________________________

      checkmk-en mailing list`

    `checkmk-en@lists.mathias-kettner.de`

    [`http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en`](http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en)








    Unless stated otherwise above:

    IBM United Kingdom Limited - Registered in England and Wales

with number
741598.

    Registered office: PO Box 41, North Harbour, Portsmouth,

Hampshire PO6
3AU

    Unless stated otherwise above:

    IBM United Kingdom Limited - Registered in England and Wales

with number
741598.

    Registered office: PO Box 41, North Harbour, Portsmouth,

Hampshire PO6
3AU

No it worked as it ‘should’ i,e. The web
page had flagged the host and services as in scheduled downtime as soon
as it detected it (within a min of it being down) and the notification
arrived ~ 5 mins later - but the notification still arrived.
Thanks
If you configured rules for notification delay and are
still getting one or more notifications
immediately when the host or service goes down, then something is wrong.

···

On 10/21/2016 6:19 AM, Ben Shade wrote:
Good plan, but still does the same.
Thanks for your ideas. I think you are right that with flex I will have
to accept a notification as that is how it work.
I think I will work with (non flex) schedules for the moment as I can probably
make it work as I need . Thanks very much for you help.
Ben
Also, if you are getting notifications immediately when the server goes
down,
then you probably don’t have any rules setup for host and service notification
delay.
You could create a couple of rules and restrict them to a single host,
one for host
notification delay, the other for service notification delay…say for
5 minutes, then
see if that keeps the notifications from being generated since the host
should be
safely in scheduled downtime within 5 minutes of whatever event occurred
to
trigger the flexible downtime (assuming 1 minute polling intervals).
If that works, remove the host specific restriction and modify the delay
to suit your needs.
On 10/20/2016 01:16 PM, Ben Shade wrote:
Yes that is what I would expect.
I have just realised that I am using flexible (flexible
with max. duration) option…and on testing that seems to be the ‘issue’
as normal downtime option seems to work fine. I forgot that bit!
My notifications (with flexible on) are generated immediately the server
goes down ( I am shutting it down as this is now using a test server),
but I have also tried waiting a while after the start of the scheduled
shutdown (but with flexible on) before actually shutting the system down
but get the same.
With flexible on is it expected that we get a notification or two? (In
a previous life a few years ago with ‘nagios’ I didn’t think I did, but
may be wrong. I am expecting I have configued something wrong still but
can’t see what it is)
Thanks
Ben

(1.2.8p12 CRE on RHEL 6)
Testing service notifications:
On one of my hosts:
Set host to scheduled downtime for 2 hours. (all services are also in scheduled
downtime)
chmod 444 /usr/bin/check_mk_agent
Verified that the host is showing a critical alert for the check_mk service.
Waiting 30 minutes to see if a notification is generated.
No notification was generated (checking notify.log) after 40 minutes (10
minutes past my configured service notification delay)
On 10/20/2016 10:08 AM, Ben Shade wrote:
Hi,
Have set up scheduled downtime for a host and when the host goes down (within
that period) I still get 2 notifications, and I can’t work out why?
I get 2 notifications: **
Check_MK: n000z306 - UP -> DOWN** and **
Check_MK: n000z306/Check_MK OK -> CRIT**

The logs show

2016-10-20 14:38:48 Global rule ‘Email_on_host_problems’…

2016-10-20 14:38:48 Event type is ur

2016-10-20 14:38:48 -> matches!

2016-10-20 14:38:48 - adding notification of xxxxxxxxxxxxx
via mail

for both of these…Not sure why these are not being flagged as inside
the DOWNTIME period?

Note that most notifications for the other services etc in this downtime
are event type s for this so don’t get sent.

In WATO notification for email, start and end of a scheduled downtime is
not ticked.

Any ideas/pointers?

OMD 1.30 Check_mk 1.2.6p12

Ben Shade

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

`


checkmk-en mailing list`

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

Unless stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598.

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU