After upgrade from 2.3.x to 2.4.x raw sporadically failing notifications with "jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'PREVIOUSSERVICEHARDSTATE'"

I’m on 2.4.0p15 and now I get notification errors such as “jinja2.exceptions.UndefinedError: ‘dict object’ has no attribute ‘SERVICESTATE’“

Well werk#18927 published in 2.4.0p15 fixes missing PREVIOUSHOSTHARDSTATE and PREVIOUSSERVICEHARDSTATE

Your issue is triggered by a different var.

SERVICESTATE would be a more elementary information then the PREVIOUS*HARDSTATE …

maybe you turn on var debugging for the notification log level
$SITEURL/check_mk/wato.py?mode=edit_configvar&varname=notification_logging
and investigate a little bit more here

Maybe posting more context might help here …

But I think you have a different issue …

Upgraded our Checkmk CCE from 2.4.0p10 → 2.4.0p15 and we are also experiencing these now instead of the “PREVIOUS..” attribute errors.

jinja2.exceptions.UndefinedError: ‘dict object’ has no attribute ‘SERVICESTATE’

while having a look at the diff

`git diff upstream/release/2.4.0p10…upstream/release/2.4.0p15 – notifications/templates/mail/`

I do not see any change that can explain this - so maybe a older issue taht was masked by the `PREVIOUS*HARDSTATE` issue?

can you configure full var dump and give some more context from the log file? maybe in a new topic?

someone from checkmk should have a look at it …

well - today the jinja2.exceptions.UndefinedError: ‘dict object’ has no attribute ‘SERVICESTATE’ catched us too …

Traceback (most recent call last): -- File "/omd/sites/$SITE$/share/check_mk/notifications/mail", line 12, in <module> -- mail.main() -- File "/omd/sites/$SITE$/lib/python3/cmk/notification_plugins/mail.py", line 746, in main -- BulkEmailContent(utils.read_bulk_contexts) -- File "/omd/sites/$SITE$/lib/python3/cmk/notification_plugins/mail.py", line 641, in __init__ -- txt, html, att = construct_content( -- ^^^^^^^^^^^^^^^^^^ -- File "/omd/sites/$SITE$/lib/python3/cmk/notification_plugins/mail.py", line 481, in construct_content -- TemplateRenderer().render_template( -- File "/omd/sites/$SITE$/lib/python3/cmk/notification_plugins/mail.py", line 279, in render_template -- return template.render(data) -- ^^^^^^^^^^^^^^^^^^^^^ -- File "/omd/sites/$SITE$/lib/python3.12/site-packages/jinja2/environment.py", line 1295, in render -- self.environment.handle_exception() -- File "/omd/sites/$SITE$/lib/python3.12/site-packages/jinja2/environment.py", line 942, in handle_exception -- raise rewrite_traceback_stack(source=source) -- File "/omd/sites/$SITE$/share/check_mk/notifications/templates/mail/base.html", line 91, in top-level template code -- {% include 'bulk.html' %} -- File "/omd/sites/$SITE$/share/check_mk/notifications/templates/mail/bulk.html", line 66, in top-level template code -- {{ entry.SERVICESTATE[:4] }} -- -- jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'SERVICESTATE'

+1
sporadically having this error since upgrading 2.4 raw from p13 to p15
but i started also to use bulk mail notifications at approx the same time.

Thank you for bringing this to our attention.

As @zerwes rightly pointed out, the Werk #18927 only handled “PREVIOUSHOSTHARDSTATE” and “PREVIOUSSERVICEHARDSTATE”. My hunch is also that these were other unhanded missing variables in the context passed to Jinja, but were masked due to the errors from the hard state variables.

I will create a ticket internally to look into this so that we can build more robust mail templates. Then I will update the thread with a Werk once a fix is applied.

2 Likes

From my point of view using jinja2 Environment(undefined=DebugUndefined) could be a quite easy and robust solution. It will keep the undefined var as string but raise no error.

2 Likes

We are also experiencing these failed notifications with 2.4.0p15 and integrated Nagios plugins. This is a critical issue for us as notifications for production systems are not being received.

I had a look, the problem is related to bulk mails containing host and service notifications.

Will provide a fix if i have feedback from a customer.

1 Like

Hi all!
Should be fixed with Werk #18869: HTML email: Fix possible exception of bulk notifications in 2.4.p17, thanks to @_rb and @logan-connolly :slight_smile:

1 Like