[agent] Host is registered for TLS but not using it, Got no information from host

CMK server version:
OMD - Open Monitoring Distribution Version 2.1.0p12.cre

OS version:
Debian GNU/Linux 11 (bullseye)

OS Host:
Debian GNU/Linux 10 (buster)

Error message:
[agent] Host is registered for TLS but not using it, Got no information from host

cmk-agent-ctl version:
Version: 2.1.0p12

╰─$ systemd --version                                                                                                                                                                    1 ↵
systemd 241 (241)

I have seen that many people have the same problem as me, but I have not found a solution for it, and these people are also waiting for solutions. I tried on several Debian hosts, but all hosts have the same problem. As soon as I register them, I get the error TLS message. Does anyone know how to solve this or how to fix the error and find out what is causing it?

I dont think that the 6556 port is importent for now because the TLS communication could be with 443 so 443 is already open on host and server side. But still somehow the Host is not using the TLS. The data collection and transport channel in the agent host should be established between systemd socket and cmk-agent-ctl-daemon on the host side. Is it possible that the port of cmk-agent-ctl-daemon is in use and not running, so that the tcp socket on cmk-agent-ctl cannot establish a TLS connection with the checkmk server? If this is the case, why should the port be used and how can it be solved without restarting the operating system. Thanks everyone!

╰─$ sudo cmk-agent-ctl status
Version: 2.1.0p12
Agent socket: operational
IP allowlist: any


Connection: xxxxx
        UUID: xxxxxx
        Local:
                Connection type: pull-agent
                Certificate issuer: Site 'xxx' local CA
                Certificate validity: Wed, 05 Oct 2022 12:04:40 +0000 - Mon, 05 Feb 3021 12:04:40 +0000
        Remote:
                Connection type: pull-agent
                Registration state: operational
                Host name: Nextcloud
╰─$ sudo systemctl status check-mk-agent.socket
● check-mk-agent.socket - Local Checkmk agent socket
   Loaded: loaded (/lib/systemd/system/check-mk-agent.socket; enabled; vendor preset: enabled)
   Active: active (listening) since Wed 2022-10-05 10:22:20 CEST; 4h 12min ago
   Listen: /run/check-mk-agent.socket (Stream)
 Accepted: 2; Connected: 0;
    Tasks: 0 (limit: 4915)
   Memory: 0B
   CGroup: /system.slice/check-mk-agent.socket

Oct 05 10:22:20 nextcloud systemd[1]: Starting Local Checkmk agent socket.
Oct 05 10:22:20 nextcloud systemd[1]: Listening on Local Checkmk agent socket.
╰─$ sudo systemctl status cmk-agent-ctl-daemon.service
● cmk-agent-ctl-daemon.service - Checkmk agent controller daemon
   Loaded: loaded (/lib/systemd/system/cmk-agent-ctl-daemon.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2022-10-05 12:19:05 CEST; 1h 46min ago
  Process: 11265 ExecStart=/usr/bin/cmk-agent-ctl daemon (code=exited, status=1/FAILURE)
 Main PID: 11265 (code=exited, status=1/FAILURE)

Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Failed with result 'exit-code'.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Service RestartSec=100ms expired, scheduling restart.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Scheduled restart job, restart counter is at 5.
Oct 05 12:19:05 nextcloud systemd[1]: Stopped Checkmk agent controller daemon.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Start request repeated too quickly.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Failed with result 'exit-code'.
Oct 05 12:19:05 nextcloud systemd[1]: Failed to start Checkmk agent controller daemon.
╰─$ sudo journalctl -u cmk-agent-ctl-daemon.service 
Oct 05 12:19:05 nextcloud cmk-agent-ctl[11265]: ERROR [cmk_agent_ctl] Failed to listen on TCP socket for incoming pull connections.
Oct 05 12:19:05 nextcloud cmk-agent-ctl[11265]: Error with IPV6:
Oct 05 12:19:05 nextcloud cmk-agent-ctl[11265]: Address in use (os error 98)
Oct 05 12:19:05 nextcloud cmk-agent-ctl[11265]: Error with IPV4:
Oct 05 12:19:05 nextcloud cmk-agent-ctl[11265]: Address in use (os error 98)
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Failed with result 'exit-code'.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Service RestartSec=100ms expired, scheduling restart.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Scheduled restart job, restart counter is at 5.
Oct 05 12:19:05 nextcloud systemd[1]: Stopped Checkmk agent controller daemon.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Start request repeated too quickly.
Oct 05 12:19:05 nextcloud systemd[1]: cmk-agent-ctl-daemon.service: Failed with result 'exit-code'.
Oct 05 12:19:05 nextcloud systemd[1]: Failed to start Checkmk agent controller daemon.

╰─$ sudo ss -tulpn | grep 6556
tcp LISTEN 0 128 *:6556 : users:((“systemd”,pid=1,fd=40))

So I ask for help to solve the problem because everything in the documentation I did but it still does not work or can I be missing something. The problem with the port 6556 that is used by systemd. I have seen some people on other blogs reboot the OS and the port 6556 has been fixed. Right now I can’t reboot because it is a live server and users are using it. I need another way to solve this problem. I hope I get a good answer and an expert can solve my problem.

What have I done so far?

  • Completely reinstall the agent host, but no fix.
  • Delete the UUID registration and re-register, but no solution.
  • Re-registered the host with IP instead of the domain name, but no fix
  • Hostname and IP address are already entered in the host portal, but no fix.
  • The registration was done successfully and I have a verify trust relationship and also operational and hostname there, but still no fix.

Thanks
Nabisadah

Hi. Did you check yourt firewall for port 8000? TLS over the agent ctl use port 8000 instead of 6556. 6556 is only used now only internaly on agent host.

Rg, Christian

Hi Christian, thank you for your time. I have solved the problem. Please also add this to your troubleshooting documents, as many people have the same problem. Before you install the agent on Linux. Linux administrator should disable the “Xinetd service”.

The following guide will hopefully help others.

  1. Stop Xinetd service
  2. Stop all agent controller services
  3. Uninstall the agent controller on the host
  4. Delete all directories/files of the agent controller from the host
  5. Install a clean installation of the agent controller and follow

The new agent controls over port 6556 and runs a service on systemd. Some operating systems also have a Xinetd service. The old version of Agent used Xinetd, and the new version uses systemd. So before you install the new version, you must stop the older service. If you do not, the new Agent Controller Deamon will not run actively because the old service uses port 6556 somehow.

1.1 Stop the Xinetd service

╰─$ sudo systemctl stop xinetd

1.2 Verify the xinetd stop service

╰─$ sudo systemctl status xinetd
● xinetd.service - LSB: Starts or stops the xinetd daemon.
   Loaded: loaded (/etc/init.d/xinetd; generated)
   Active: inactive (dead) since Wed 2022-10-05 15:14:25 CEST; 17h ago
     Docs: man:systemd-sysv-generator(8)

2.1 Stop all agent controller services

╰─$ sudo cmk-agent-ctl delete-all                                                                                                                                                        
╰─$ sudo systemctl stop check-mk-agent.socket
╰─$ sudo systemctl stop check-mk-agent-async.service
╰─$ sudo systemctl stop cmk-agent-ctl-daemon.service

3.1 Uninstall the agent controller on host

╰─$ sudo dpkg -r check-mk-agent
(Reading database ... 56008 files and directories currently installed.)
Removing check-mk-agent (2.1.0p12-1) ...
Removing deployed systemd units: cmk-agent-ctl-daemon.service, check-mk-agent@.service, check-mk-agent-async.service, check-mk-agent.socket
Deactivating systemd unit 'cmk-agent-ctl-daemon.service' (if active)...
Deactivating systemd unit 'check-mk-agent-async.service' (if active)...
Deactivating systemd unit 'check-mk-agent.socket' (if active)...
╰─$ sudo apt purge check-mk-agent
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Die folgenden Pakete werden ENTFERNT:
  check-mk-agent*
0 aktualisiert, 0 neu installiert, 1 zu entfernen und 2 nicht aktualisiert.
Nach dieser Operation werden 0 B Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren? [J/n] y
(Lese Datenbank ... 54393 Dateien und Verzeichnisse sind derzeit installiert.)
Löschen der Konfigurationsdateien von check-mk-agent (2.1.0p12-1) ...
dpkg: Warnung: Während Entfernens von check-mk-agent ist Verzeichnis »/var/lib/check_mk_agent« nicht leer, wird daher nicht gelöscht
dpkg: Warnung: Während Entfernens von check-mk-agent ist Verzeichnis »/usr/lib/check_mk_agent/plugins« nicht leer, wird daher nicht gelöscht

4.1 Delete all configuration and dependent packages of the agent from the host

  • You can uninstall or removes an installed check-mk-agent package

sudo apt-get remove check-mk-agent

  • If you would like to remove check-mk-agent and it’s dependent packages which are no longer needed

sudo apt-get remove --auto-remove check-mk-agent

  • If you use with purge options to check-mk-agent package all the configuration and dependent packages will be removed.

sudo apt-get purge check-mk-agent

  • If you use purge options along with auto remove, will be removed everything regarding the package, It’s really useful when you want to reinstall again.

sudo apt-get purge --auto-remove check-mk-agent

4.2 Delete all directories/files of the agent controller from the host


cd /etc
sudo rm -r check_mk  

cd /var/lib
sudo rm -r check_mk_agent
sudo rm -r cmk-agent

cd /usr/lib
sudo rm -r check_mk_agent

sudo systemctl daemon-reload 

When you have done all this, install the agent again and it will work properly.

8 Likes

helps a lot, thanx for sharing this solution!

1 Like

I found you don’t actually need to uninstall the agent.
These are the steps I took to resolve this issue.

Remove the xinetd.d script and reload xinetd (if you installed it like I did)
# rm -f /etc/xinetd.d/check-mk-agent; systemctl restart xinetd

Delete all connections and stop the services
# sudo cmk-agent-ctl delete-all
# systemctl stop check-mk-agent.socket
# systemctl stop check-mk-agent-async.service
# systemctl stop cmk-agent-ctl-daemon.service

Remove any files in the rtc_remotes folder
# rm -f /var/lib/check_mk_agent/rtc_remotes/*

Restart the services
# systemctl start cmk-agent-ctl-daemon.service
# systemctl start check-mk-agent-async.service
# systemctl start check-mk-agent.socket

Re-register the agent
# cmk-agent-ctl register --hostname .....

Hope this helps someone

1 Like

Do not forget to remove the account cmk-agent with sudo deluser cmk-agent.

This account gets created when the agent package is installed but will not be removed even with an apt purge.

2 Likes