CheckMK v2.2 - custom checks

Hi All,

In checkMK v1.6 we had a custom check that added additional information to the list of Filesystem checks. This was done by adding the information in the same way the standard check under the ‘< < < df > > >’ header was done.

Now that we’ve migrated to checkMK v2.2, the ‘< < < df > > >’ has now become ‘<<<df_v2>>>’ and seems to have the information of a df -T command on Linux. However, even when changing the information according to the new layout, the additional Filesystem information does not seem to be picked up and added to the rest of the Filesystem entries as was the case in v1.6.

I have the following questions based on the above situation:

  1. Is the ‘<<<df_v2>>>’ info indeed the output of a df -T command on Linux or are the additional parameters and/or formatting required to simulate the same output?

  2. Is there a different way to add this information to the local checks, so it is added to the Filesystem information collected by the regular checkMK checks?

Any information would be greatly appreciated.

Note: the df coding I had to add spaces to as this messaging system seems to either remove or turn it into empty space when you put these brackets <> around df.

Best regards,
Jacky

If you dig into the checkmk agent (/usr/bin/check_mk_agent), you’ll find the function section_df() that collects the data.

Here is an excerpt:

section_df() {
    ...
    excludefs="-x smbfs -x cifs -x iso9660 -x udf -x nfsv4 -x nfs -x mvfs -x prl_fs -x squashfs -x devtmpfs"
    ...
    
    echo '<<<df_v2>>>'
    df -PTlk ${excludefs} | sed 1d

    # df inodes information
    echo '[df_inodes_start]'
    df -PTli ${excludefs} | sed 1d
    echo '[df_inodes_end]'

    if inpath lsblk; then
        echo "[df_lsblk_start]"
        lsblk --list --paths --output NAME,UUID
        echo "[df_lsblk_end]"
    fi
}

So yes, it’s basically a df -T but with some extra options.

Hi Dirk,

Thanx for the quick reply. :+1:

In your example above the type of ‘Filesystem’ we’re adding is listed in the excludefs= line.
Would it be possible to modify that line to remove the type from the excludefs line?

Or should we copy the function to the local directory and modify it for the 2 nfs locations for which we do want the filesystem info in the checks? Would that be possible or will the overaall exclude list prevent it from showing up anyway?

Best regards,
Jacky

I wouldn’t modify the original check_mk_agent because the changes won’t survive an update. My suggestion would be to either completely disable the output of the section <<<df_v2>>>. This can be done by creating the file /etc/check_mk/exclude_sections.cfg with the content MK_SKIP_DF=yes. In this case the whole function section_df() is not called. Of course, you must then supply your own version of that function in a separate plugin below /usr/lib/check_mk_agent/plugins/.

The simpler (and preferred) approach is to put an own plugin in /usr/lib/check_mk_agent/plugins/ which simply adds the data that is missing from the built-in section <<<df_v2>>>:

#!/usr/bin/env bash

echo '<<<df_v2>>>'
df -PTlk --type devtmpfs | sed 1d

echo '[df_inodes_start]'
df -PTli --type devtmpfs | sed 1d
echo '[df_inodes_end]'

In this example df will only process filesystems of type devtmpfs (as an example). checkmk will merge the original section df_v2 and your section df_v2 into one big section on the server side. The check plugin won’t even notice.

But be aware that the -l option to the df means local filesystems only. The problem is that df will hang for ages if a remote filesystem is not accessible.

1 Like

Hi Dirk,

Thanx a lot! The script we’re currently using to select extra info is similar to what you have above, but is now located under local (since it was under local with v1.6), but I’ll move it to plugins and modify the selection to add the extra parameters you’ve indicated along with the -T. I’ll also play around with the -l to see if the 2 locations we want is selected when -l is added, otherwise I’ll leave that out as those locations are quite stable and we’ve never had hanging issues so far when we used v1.6.

Best regards,
Jacky

1 Like

@Dirk … for some reason it still seems to ignore the output, even though you can see it’s been added in the file produced for those servers.

Do you have any idea what could still be preventing it from adding the info to the UI host specific overview?

Best regards,
Jacky

It’s hard to tell without seeing your environment. If it’s a CIFS mount or NFS mount, then the df check is the wrong one anyway. For network filesystems exists a separate check. Look out for the sections <<<nfsmounts>>> end <<<cifsmounts>>>. The code to detect them is also built into the regular checkmk agent.

Hi Dirk,

We have the NFS mount standard checks on these nfs-shares we would like the filesystem checks on, but those only check whether the mount is connected. It doesn’t give you WARN or CRIT indications if the disk space on that mount is getting closer to capacity, whereas the Filesystem checks do let you know if the disk space is over 80% or 90% in use, which for those two nfs-shares that we also added as Filesystem checks in v1.6 works perfectly.

image
image

So it’s really frustrating when a newer version (2.2) causes checks that were possible in an older version (1.6) not to function anymore.

Best regards,
Jacky

I’m not sure if I understand you correctly. If you open the hamburger menu ☰ of your NFS filesystem service and then go to Paramters for this service:

… then you will be directed to the ruleset(s) to configure the filesystem levels of that service:

The ruleset is called Network filesystem - overall status and usage (e.g. NFS).

It’s a slightly different ruleset than that for the “regular” filesystems, but is already built into checkmk and requires no extra steps:

Hi Dirk,

Thanx! Didn’t know this part existed! Will set this up instead of the custom check.

Best regards,
Jacky

1 Like