Agent only listening on IPv6

General
, ,
1

Hi all

Due to specific vendor requirements, we have disabled IPv6 in our environment. Most clients with the CheckMk are working fine a some are only listening on IPv6 dispite the fact that IPv6 is disabled.

All affected clients are a mix of RHEL versions from 7 through to 9.

image
image998×194 8.41 KB

Any ideas how I can force the client to listen on the IPv4 interface?

Thank you

2

Does it not respond to IPv4 queries?

3

The process that listens on port 6556 for incoming requests is the cmk-agent-ctl daemon which is controlled by the systemd unit cmk-agent-ctl-daemon.service.

I’m sorry to ask but have you tried restarting that unit since you disabled IPv6?

4

Nope. I did a tcpdump on the CheckMk server and I could see the requests going to the client on tcp/6556 but there was no response back to the server.

5

IPv6 was disabled at build time (many months ago) and I’ve only just started with CheckMk a few weeks ago.

systemctl status cmk-agent-ctl-daemon.service
● cmk-agent-ctl-daemon.service - Checkmk agent controller daemon
     Loaded: loaded (/usr/lib/systemd/system/cmk-agent-ctl-daemon.service; enabled; preset: disabled)
     Active: active (running) since Wed 2024-03-20 11:42:20 AWST; 1 month 5 days ago
       Docs: https://docs.checkmk.com/latest/en/agent_linux.html
   Main PID: 2036459 (cmk-agent-ctl)
      Tasks: 4 (limit: 48888)
     Memory: 4.4M
        CPU: 1min 32.875s
     CGroup: /system.slice/cmk-agent-ctl-daemon.service
             └─2036459 /usr/bin/cmk-agent-ctl daemon

Apr 25 05:11:49 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:33166: Request failed. (tls handshake eof)
Apr 25 07:11:46 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:46710: Request failed. (tls handshake eof)
Apr 25 09:11:46 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:38424: Request failed. (tls handshake eof)
Apr 25 10:24:59 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:47438: Request failed. (tls handshake eof)
Apr 25 10:25:19 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:54258: Request failed. (tls handshake eof)
Apr 25 10:25:32 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:35270: Request failed. (received corrupt message)
Apr 25 11:11:46 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:42818: Request failed. (tls handshake eof)
Apr 25 13:11:48 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:56840: Request failed. (tls handshake eof)
Apr 25 15:11:47 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:59298: Request failed. (tls handshake eof)
Apr 25 16:36:52 some_client cmk-agent-ctl[2036459]: WARN [cmk_agent_ctl::modes::pull] [::ffff:172.xx.xx.11]:51720: Request failed. (tls handshake eof)

And /var/lib/cmk-agent/registered_connections.json shows the correct server so the client is registered.

6

Yes, I felt a little embarrassed to ask. :grimacing:

But what you can try is this: stop the systemd unit and start the daemon manually as user cmk-agent in the foreground with -vv (see cmk-agent-ctl daemon --help for further options). Maybe it gives you some hints:

sudo systemctl stop cmk-agent-ctl-daemon.service
sudo -u cmk-agent cmk-agent-ctl daemon -vv
INFO [cmk_agent_ctl] starting
INFO [cmk_agent_ctl] Loaded config from '"/var/lib/cmk-agent/cmk-agent-ctl.toml"', legacy pull 'LegacyPullMarker("/var/lib/cmk-agent/allow-legacy-pull")' absent
DEBUG [cmk_agent_ctl::modes::push] Sleeping 0s to avoid DDOSing of sites
INFO [cmk_agent_ctl::modes::pull] Start listening for incoming pull requests
INFO [cmk_agent_ctl::modes::pull] Listening on [::]:6556 for incoming pull connections (IPv6 & IPv4 if activated)

If you are done, hit CTRL-C and re-start the unit again.

7

Nah, all good. I’m such a noob here that nothing is off the table.

However, I’ve just noticed that the CheckMk agent is not the only service doing this. Other services are also listening on IPv6.

I’ll raise a ticket with Red Hat Support to see if they have any ideas.

Thanks

1 Like