Errors in notify.log when "analyzing the notification rule" after updating to 2.4.0p14

Troubleshooting
1

image

CMK version: CRE 2.4.0p14
OS version: Ubuntu 24.04

Error message:

When clicking on “Setup > Events > Notifications > Events > Analyze recent notifications > Analyze Rule set with this notification,” the following errors appear in the notify.log file:

2025-10-22 15:30:43,951 [20] [cmk.base.events] Error on completing raw context: 'SITEOFHOST'
2025-10-22 15:30:43,951 [20] [cmk.base.notify] ----------------------------------------------------------------------
2025-10-22 15:30:43,951 [20] [cmk.base.notify] Analysing notification (myhost;Process cmk_test redis-server) context with 54 variables
2025-10-22 15:30:43,951 [15] [cmk.base.notify] Global rule 'notify name'...
2025-10-22 15:30:43,951 [15] [cmk.base.notify]  -> does not match: This rule is disabled
2025-10-22 15:30:43,951 [40] [cmk.base.notify] ERROR:
Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3/cmk/base/notify.py", line 493, in locally_deliver_raw_context
    return notify_rulebased(
           ^^^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/notify.py", line 763, in notify_rulebased
    why_not = rbn_match_rule(
              ^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/notify.py", line 1216, in rbn_match_rule
    return events.apply_matchers(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/events.py", line 490, in apply_matchers
    result = matcher(rule, context, analyse, all_timeperiods)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/notify.py", line 1219, in <lambda>
    lambda rule, context, analyse, all_timeperiods: events.event_match_rule(
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/events.py", line 503, in event_match_rule
    return apply_matchers(
           ^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/events.py", line 490, in apply_matchers
    result = matcher(rule, context, analyse, all_timeperiods)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/events.py", line 509, in <lambda>
    lambda rule, context, analyse, all_timeperiods: event_match_servicegroups_fixed(
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/events.py", line 634, in event_match_servicegroups_fixed
    return _event_match_servicegroups(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/mysite/lib/python3/cmk/base/events.py", line 664, in _event_match_servicegroups
    if context["WHAT"] != "SERVICE":
       ~~~~~~~^^^^^^^^
KeyError: 'WHAT'
2

Hello

  • Am I the only one seeing these logs in notify.log?
  • Does the “Analyzing the notification rule” button work for everyone?
    :thinking:
    Thanks
3

I just updated Monday (same version both CMK and OS) and see the ame problem.

2025-11-05 06:49:54,651 [20] [cmk.base.notify] Replaying notification 0 from backlog…

2025-11-05 06:49:54,651 [20] [cmk.base.events] Error on completing raw context: ‘SITEOFHOST’
2025-11-05 06:49:54,651 [20] [cmk.base.notify] ----------------------------------------------------------------------
2025-11-05 06:49:54,651 [20] [cmk.base.notify] Analysing notification (;File C:\Temp\FILENAME.csv) context with 54 variables
2025-11-05 06:49:54,651 [40] [cmk.base.notify] ERROR:
Traceback (most recent call last):
File “/omd/sites/monitoring/lib/python3/cmk/base/notify.py”, line 493, in locally_deliver_raw_context
return notify_rulebased(
^^^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/notify.py”, line 763, in notify_rulebased
why_not = rbn_match_rule(
^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/notify.py”, line 1216, in rbn_match_rule
return events.apply_matchers(
^^^^^^^^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/events.py”, line 490, in apply_matchers
result = matcher(rule, context, analyse, all_timeperiods)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/notify.py”, line 1219, in
lambda rule, context, analyse, all_timeperiods: events.event_match_rule(
^^^^^^^^^^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/events.py”, line 503, in event_match_rule
return apply_matchers(
^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/events.py”, line 490, in apply_matchers
result = matcher(rule, context, analyse, all_timeperiods)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/events.py”, line 509, in
lambda rule, context, analyse, all_timeperiods: event_match_servicegroups_fixed(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/events.py”, line 634, in event_match_servicegroups_fixed
return _event_match_servicegroups(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/omd/sites/monitoring/lib/python3/cmk/base/events.py”, line 664, in _event_match_servicegroups
if context[“WHAT”] != “SERVICE”:
~~~~~~~^^^^^^^^
KeyError: ‘WHAT’
4

Hello,
I can confirm this error on our p15 site when clicking on Analyze Rule set with this notification :

2025-11-05 07:58:02,706 [20] [cmk.base.notify] Replaying notification 0 from backlog...

2025-11-05 07:58:02,707 [20] [cmk.base.events] Error on completing raw context: 'SITEOFHOST'
2025-11-05 07:58:02,707 [20] [cmk.base.notify] ----------------------------------------------------------------------
2025-11-05 07:58:02,707 [20] [cmk.base.notify] Analysing notification (hostname.domain;Filesystem /var/log) context with 76 variables
2025-11-05 07:58:02,707 [40] [cmk.base.notify] ERROR:
Traceback (most recent call last):
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/notify.py", line 493, in locally_deliver_raw_context
    return notify_rulebased(
           ^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/notify.py", line 763, in notify_rulebased
    why_not = rbn_match_rule(
              ^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/notify.py", line 1216, in rbn_match_rule
    return events.apply_matchers(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/events.py", line 490, in apply_matchers
    result = matcher(rule, context, analyse, all_timeperiods)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/notify.py", line 1219, in <lambda>
    lambda rule, context, analyse, all_timeperiods: events.event_match_rule(
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/events.py", line 503, in event_match_rule
    return apply_matchers(
           ^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/events.py", line 490, in apply_matchers
    result = matcher(rule, context, analyse, all_timeperiods)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/events.py", line 509, in <lambda>
    lambda rule, context, analyse, all_timeperiods: event_match_servicegroups_fixed(
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/events.py", line 634, in event_match_servicegroups_fixed
    return _event_match_servicegroups(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk_test/lib/python3/cmk/base/events.py", line 664, in _event_match_servicegroups
    if context["WHAT"] != "SERVICE":
       ~~~~~~~^^^^^^^^
KeyError: 'WHAT'
5

I was able to reproduce the problem. It occurs when using the condition “service groups”.

I will create an internal ticket.

4 Likes
6

Hi Lars,

nice that you are able to reproduce this. In my company we are heavily use service groups for routing Alarms. :slight_smile:

7

Hi Christian,

I have to correct my last post a bit: according to our developers, it not only occurs when using the condition “service groups”.
But: It ONLY occurs, when replaying a previous notification.
Newly occurring notifications are handled correctly.

The developer has fixed it in the meantime, and the fix is under review.

Best regards
Lars

4 Likes
8

Problem will be fixed with werk #18862 in 2.4.0p16

3 Likes