Caching active checks

[jairelee]

Hi all,
Sorry if this topic was already raised, I couldn’t find anything related.
I was wondering if there is a way of using the last value generated by an active check when it fails.
Giving a bit more of background, I’m monitoring a list of customer routers which are connected to our network using VPN. I monitor them via the VPN IP and have created a custom SNMP check to pull a few OIDs that contain the public IP of the router, so if the VPN connection fails we can still connect using the public IP to restore the router’s configuration.
The problem is when the router fails to contact via VPN IP the new active check I created also fails, instead of showing the old value it just says the check has failed, which is the normal behaviour, but I’ve lost the public IP reported so can’t connect to it to restore the router config.

Is there a way for using the last output from the last successful check run?

Apologies if I’m asking something trivial.

Many thanks.

[jairelee]

Is not the best solution, by far, but given I couldn’t find any other alternative, I’ve written the following script and placed it under the omd user, ~/local/bin, location that gets replicated to the slaves I have configured for the distributed monitoring.

I call it from CheckMK as a “classical active and passive monitoring check”, using the following syntax:

routers_public_ip -H $HOSTADDRESS$ -C community_string -P 2c -o iso.3.6.1.2.1.31.1.1.1.18.25 -o iso.3.6.1.2.1.4.21.1.7.0.0.0.0 -o iso.3.6.1.2.1.31.1.1.1.18.36 -o iso.3.6.1.2.1.31.1.1.1.18.30 -o iso.3.6.1.2.1.31.1.1.1.18.24

An it gets applied to a folder with over 300 devices.

Script:

#!/bin/bash

# $2 is $HOSTADDRESS$ 

OUTPUT=`check_snmp $@`

if [ ! -d ~/var/tmp/routers_public_ip ]
then
        mkdir -p ~/var/tmp/routers_public_ip
fi

echo "$OUTPUT" | grep -q OK
if [ $? -eq 0 ]
then
        echo "$OUTPUT" | tee ~/var/tmp/routers_public_ip/$2.txt
else
        if [ -f ~/var/tmp/routers_public_ip/$2.txt ]
        then
                echo "`cat ~/var/tmp/routers_public_ip/$2.txt` - Cached"|sed 's/OK/FAILED/g'|sed 's/|//g'
                exit 1
        else
                echo "$OUTPUT"
                exit 2
        fi
fi

Not ideal but it does the job.

Cheers

[andreas-doehler (Andreas)]

I think the more common approach for a classic check is that this classic check writes a file for every host it is called and inside the file is the last success output. This file will only be overwritten by the result if the check result is ok. If the result is not ok it outputs the last result and don’t write the file.
There you only have one active check (the routers_public_ip) and not two.

[jairelee]

Hi Andreas,
Thank you for the reply.
I guess when you say I’m using two checks is because I’m calling check_snmp within my script.
On this I agree with you, I should use only one, although, it makes it much easier for me to parse the output when calling check_snmp than when calling snmpget with the list of OIDs.
Other than that, as I understand it, I’m already using the logic you suggest.
Thanks for the input
Regards.