/mnt/tmp/{foo,bar} not recognized by checkmk

Troubleshooting
,
1

CMK version: 2.2.0p12
OS version: RHEL7

We do have new mounts on almost every host now but those are not recognized by checkmk it seems.
As written in the topic title, these mounts are missing in the monitoring. Is there maybe a hidden setting that excludes these ‘/mnt/whatever’ somewhere?

cheers

2

The description of the behaviour is hidden in the manpage. It says:

One service is created for each filesystem which is not of the type tmpfs. The inventory uses the current mount options as check parameter. Note: the agent sends only filesystems which have a device below /dev, so network filesystems and virtual filesystems are excluded by the agent.

So maybe one of the restrictions apply to your filesystems?

3

These mounts are ‘lustre’ mounts and I did not find any restrictions in our rules so far.

4

The restrictions I mentioned do not stem from any rules. They are hardcoded in the check plugin.

5

I never heard of ‘lustre’ mount. Could you please paste output of command df -h /mnt/your-mountpoint?

Following are mount types hardcoded in agent to exclude:

excludefs="-x smbfs -x cifs -x iso9660 -x udf -x nfsv4 -x nfs -x mvfs -x prl_fs -x squashfs -x devtmpfs -x autofs -x beegfs"

df -PTlk ${excludefs} | sed 1d

Isn’t lustre mount shown as localhost in df output? Like this for MapR filesystem:

[root@node29 ~]# df -h /mapr
Filesystem       Size  Used Avail Use% Mounted on
localhost:/mapr  100G     0  100G   0% /mapr

Therefore the -l argument in df -PTlk is excluding it.

1 Like
6 
Filesystem                 Size  Used Avail Use% Mounted on
10.0.50.102@tcp:/ugmpjbmv  1.1T  738G  377G  67% /mnt/tmp/mc

‘mount’ output for one of the mounts:
10.0.50.102@tcp:/ugmpjbmv on /mnt/tmp/mc type lustre (rw,noatime,flock,lazystatfs)

7

Please try now df -PTlk /mnt/tmp/mc, sorry forgot to mention it before, to compare outputs.

This is something new for me 10.0.50.102@tcp. I never saw “@tcp” in df output.

8 
df -PTlk /mnt/tmp/mc
df: no file systems processed
9

Yeah, then I think only way to monitor it is to manually edit agent and remove l parameter from df -PTlk. From df manual:

-l, --local
limit listing to local file systems

Or, second way, monitor usage on source machine. I think I was struggling with it also in past on some other topic here.

This is comment in agent:

# These should generally be monitored on the NFS server, not on the clients.
10

I see. But thats not easily done. Do you see a way to implement this in checkmk as I would assume ‘lustre’ mounts to be a thing anyway in cloud-based systems.

11

Yeah, if it is just one mountpoint, then create some simple bash script with df command, like it is in official agent. And distribute it within agent using Agent Bakery (if you use it), if it is on many servers.

12

Can you pinpoint me to something where thats documented? I’m not good at the insights of all that, sorry.

13

Here is docu how to write custom checks.