Failed to obtain ICMP socket: Operation not permitted , hosts are DOWN

[Darksice (Thomas Amaro)]

CMK version: OMD - Open Monitoring Distribution Version 2.2.0p10.cre
OS version: Rocky Linux release 8.8 (Green Obsidian)

Error message: check_icmp: Failed to obtain ICMP socket: Operation not permitted

When I’m adding a host to check_mk, the host appears DOWN and says “check_icmp: Failed to obtain ICMP socket: Operation not permitted”. After checking forum I can’t figure out why it’s not working.

Site infos :

OMD[SITENAME]:~/lib/nagios/plugins$ id
uid=381(SITENAME) gid=1009(SITENAME) groups=1009(SITENAME),391(omd)

check infos :

OMD[SITENAME]:~/lib/nagios/plugins$ ls -l check_icmp 
-rwxr-x--- 1 root omd 65240 Sep 19 11:18 check_icmp*

caps info: (as I read, it now works with cap and no longer with suid)

OMD[SITENAME]:~/lib/nagios/plugins$ getcap check_icmp
check_icmp cap_net_raw=ep

Trying to run the check:

OMD[SITENAME]:~/lib/nagios/plugins$ ./check_icmp -vvvvv -4 -H 192.168.0.1
Warning: no SO_TIMESTAMP support
check_icmp: Failed to obtain ICMP socket: Operation not permitted

When I’m running the check as root user it’s working

At first I thought this was the only problem but here you ask to run “cmk --debug -vvn hostname” command which returns an error (on each host) :

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

Files: ([], None)
Traceback (most recent call last):
  File "/omd/sites/SITENAME/lib/python3/cmk/base/config.py", line 2264, in _extract_agent_and_snmp_sections
    create_snmp_section_plugin_from_legacy(
  File "/omd/sites/SITENAME/lib/python3/cmk/base/api/agent_based/register/section_plugins_legacy/__init__.py", line 245, in create_snmp_section_plugin_from_legacy
    detect_spec = create_detect_spec(
                  ^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/SITENAME/lib/python3/cmk/base/api/agent_based/register/section_plugins_legacy/convert_scan_functions.py", line 419, in create_detect_spec
    _compute_detect_spec(
  File "/omd/sites/SITENAME/lib/python3/cmk/base/api/agent_based/register/section_plugins_legacy/convert_scan_functions.py", line 387, in _compute_detect_spec
    scan_func_ast = _get_scan_function_ast(section_name, scan_function, fallback_files)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/SITENAME/lib/python3/cmk/base/api/agent_based/register/section_plugins_legacy/convert_scan_functions.py", line 110, in _get_scan_function_ast
    assert source != "", "Files: %r" % ((read_files, src_file_name),)
           ^^^^^^^^^^^^
AssertionError: Files: ([], None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/omd/sites/SITENAME/bin/cmk", line 97, in <module>
    errors = config.load_all_agent_based_plugins(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/SITENAME/lib/python3/cmk/base/config.py", line 1681, in load_all_agent_based_plugins
    errors.extend(load_checks(get_check_api_context, filelist))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/SITENAME/lib/python3/cmk/base/config.py", line 1822, in load_checks
    return _extract_agent_and_snmp_sections(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/SITENAME/lib/python3/cmk/base/config.py", line 2285, in _extract_agent_and_snmp_sections
    raise MKGeneralException(exc) from exc
cmk.utils.exceptions.MKGeneralException: Files: ([], None)

I don’t know if the two problems are related.

If you need more information I’d be glad to give them to you.

Thanks,

[aeckstein (Andre Eckstein)]

Hi Thomas,

the permission and capabilities look correct so far.

i would try to

• fully disable selinux
• test what happens, when you create a new empty test site and execute the commands there
• make sure that you installed the correct checkmk package for rocky linux
• test with the latest checkmk version

[Darksice (Thomas Amaro)]

Hi,

Thanks for reply,

• fully disable selinux

SELinux status: disabled

• test what happens, when you create a new empty test site and execute the commands there

Same thing happens “Failed to obtain ICMP socket: Operation not permitted”

• make sure that you installed the correct checkmk package for rocky linux

I installed the el8 package (which is the version for Rocky 8)

• test with the latest checkmk version

After updating, the command cmk --debug -vvn hostname works but it didn’t resolve the check_icmp error :

OMD[SITENAME]:~/lib/nagios/plugins$ ./check_icmp -vvvvv -4 -H 192.168.0.1
Warning: no SO_TIMESTAMP support
check_icmp: Failed to obtain ICMP socket: Operation not permitted

I tried to use strace but it just shows that it doesn’t have the necessary privileges :

socket(AF_INET, SOCK_RAW, IPPROTO_ICMP) = -1 EPERM (Operation not permitted)
setsockopt(-1, SOL_SOCKET, SO_TIMESTAMP_OLD, [1], 4) = -1 EBADF (Bad file descriptor)
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x9), ...}) = 0
write(1, "Warning: no SO_TIMESTAMP support"..., 33Warning: no SO_TIMESTAMP support
) = 33
getuid()                                = 381
setuid(381)                             = 0
openat(AT_FDCWD, "/usr/share/locale/C.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/C/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(1, "check_icmp: Failed to obtain ICM"..., 66check_icmp: Failed to obtain ICMP socket: Operation not permitted
) = 66

[Darksice (Thomas Amaro)]

Finally it works, to help people who are stuck here, check if your partition has the nosuid flag in /etc/fstab. I had nosuid on the /opt partition, after deleting it and reloading with mount -a, it now works.