Checkmk RAW and Nagios Event Handler

matjaz.premerl · December 17, 2024, 2:34pm

Hi,

I want to open this topic, because in this forum or documentation I can’t find complete working example for the implementation of Nagios type of Event Handler for Checkmk RAW edition version 2.3.0. Currently I’m using 2.3.0p12.
For now I’ve tried like that:
I’ve created one custom cfg file in ~/etc/nagios/conf.d/gms_objects.cfg like below:

define service {
        host_name       *name_of_host*
        service_description     *name_of_service*
		event_handler   gmsrestart
        event_handler_enabled   1
}


define command {
        command_name   gmsrestart
        command_line   $USER2$/eventhandler_gms.sh $SERVICESTATE$ $SERVICESTATETYPE$ $HOSTADDRESS$
}

Command cmk -O did not find any problems with the upper syntax.

Script named eventhandler_gms.sh is in the path below:
~/local/lib/nagios/plugins

But when I push desired service to CRIT HARD event I did not see any action that the event handler run the provided sh script. Is there anyone who can share some advice if my steps and thinking are correct. Or if it’s possible that someone could share fully working example with the complete configuration.

After some time and additional testing, I’ve finally found working solution.

Here is my working example on a Checkmk monitor service named “Postfix status default”.
On site ~/etc/check_mk/conf.d I have created custom file alert.mk with configuration below:

extra_service_conf["event_handler_enabled"] = [
  {'condition': {'service_description': [{'$regex': 'Postfix status default'}], 'host_name': ['name_of_host_defined_in_checkmk']}, 'value': '1'},
]

extra_service_conf["event_handler"] = [
  {'condition': {'service_description': [{'$regex': 'Postfix status default'}], 'host_name': ['name_of_host_defined_in_checkmk']}, 'value': 'gmsrestart'},
]

In ~/etc/nagios/conf.d I’ve created custom file gms_objects.cfg. In this file I’ve just nagios definition of command.

define command{
        command_name   gmsrestart
        command_line   $USER2$/eventhandler_gms.sh $SERVICESTATE$ $SERVICESTATETYPE$ $HOSTADDRESS$
}

After that you can check the syntax with cmk -O and you should get not errors.

In the folder /omd/sites/name_of_checkmk_site/local/lib/nagios/plugins is action script eventhandler_gms.sh where is the logic and my custom command which I want to run on specific event. This script must have execute permission.

I’ve created this action script based on example I’ve found on github:

github.com

madic-creates/Scripts/blob/master/Linux/Nagios/plugins/eventhandler_cleanup.sh

#!/usr/bin/env bash
### Use with nagios event handler.
### Based on https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/3/en/eventhandlers.html
### Written for check_mk and https://www.geekbundle.org/selbstheilung-mit-check_mk-event-handler
###
### Written by Madic - madic@geekbundle.org 2018-06

SERVICESTATE="$1"
SERVICESTATETYPE="$2"
HOSTADDRESS="$3"
LOG=~/var/log/eventhandler.log

exec &>"$LOG"

CLEANUP() {
    echo "$(date +"%Y-%m-%d %H:%M:%S") Doing cleanup on $HOSTADDRESS through ${0##*/} Script..."
    ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no user@"$HOSTADDRESS" sudo /usr/local/sbin/cleanup.sh
}

case "$SERVICESTATE" in

This file has been truncated. show original

I was testing this with postfix service on my destination Linux system. I was stopping the service and after some time the monitored service went to CRIT status as HARD and the Event Hadler was triggered. You could check this in log file /var/log/nagios.log. Below is one example from my log file.

[1734611739] SERVICE ALERT: name_of_host_defined_in_checkmk;Postfix status default;CRITICAL;HARD;1;Status: the Postfix mail system is not running
[1734611739] SERVICE EVENT HANDLER: name_of_host_defined_in_checkmk;Postfix status default;CRITICAL;HARD;1;gmsrestart

From the log file you can see that the event handler named “gmsrestart” was started and with running of action script mentioned before, the postfix linux service was restarted on desination linux and Health of Postfix status went back to OK.

Regards,
Matjaž