Checkmk Notification Spooler Problem on virtual Appliance

Hi Forum,
I currently have a strange problem with the state of the notification spooler. My Checkmk service reports that the notification spooler is not running, in fact that it is running and working. I tried to troubleshoot the problem with the logs but nothing special there.
I can see the process running in htop and it is doing it’s job because I receive e-mails.

Here is a screenshot of the service problems that are not representing the right state of the spooler:

The omd status output showed me that the MKNOTIFYD is running and OK. Like this:

OMD[SITENAME]:~$ omd status
mkeventd:       running
liveproxyd:     running
mknotifyd:      running
rrdcached:      running
cmc:            running
apache:         running
crontab:        running
-----------------------
Overall state:  running

As described on https://checkmk.com/cms_notifications.html
I took a look at …
var/log/notify.log
var/log/mknotifyd.log
… but nothing strange there even with the highest log level (debugging).

I also took a look at the agent output to find the problem. As you can see here.

<<<omd_status:cached(1579006293,60)>>>
[SITENAME]
mkeventd 0
liveproxyd 0
mknotifyd 0
rrdcached 0
cmc 0
apache 0
crontab 0
OVERALL 0
<<<mknotifyd:sep(0)>>>
[SITENAME]
Version:         1.5.0p23
Updated:         1579006331 (2020-01-14 13:52:11) 
Started:         1579005428 (2020-01-14 13:37:08, 903 sec ago) 
Configuration:   1579005428 (2020-01-14 13:37:08, 903 sec ago) 
Listening FD:    None

Spool:           New
Count:           0
Oldest:          
Youngest:        

Spool:           Deferred
Count:           0
Oldest:          
Youngest:        

Spool:           Corrupted
Count:           0
Oldest:          
Youngest:        

Queue:           mail
Waiting:         0
Processing:      0

(SITENAME,36508,11388,00:00:00/15:09,851) python /omd/sites/SITENAME/bin/mknotifyd

Here is a screenshot of my htop where you can clearly see that the MKNOTIFYD is running.

This is the state from the MKNOTIFYD:

I did a omd site restart but the problem was still showing up and I also rebooted the whole virtual appliance but that helped neither nor. The problem is persistent.

Setup:
Checkmk EE 1.5.0p23 running on the latest Checkmk Virtual Appliance Firmware (1.4.6) with baked agent installed and 1 distributed site connected.

Maybe it’s something related with the appliance or the serverside plugin is missunderstanding the agent output. idk ¯_(ツ)_/¯

I’m open for new ideas and help from you. Thanks in advance. :slightly_smiling_face:

I censored some output for security reasons.

1 Like

PUSH

Anyone have an idea? I still have this problem. :upside_down_face:
Thanks in advance.

1 Like

I still don’t know what caused this problem but an update to a higher Checkmk Version fixed the Problem.
In this case an upgrade to Checkmk CEE 1.6.0p8 fixed it. ¯_(ツ)_/¯

2 Likes