Details: We have a few sites (see below for site details). The VMs in the DMZ cannot talk to our central site, but can talk to our DMZ site. The DMZ site can talk to our central site over port 80, 443, 6556, and 6557. I believe we have the site settings and agent updater agent settings set so that the distributed sites know to reach out to the central site to pull agent updates, but I’m seeing the following error (see cmk-update-agent -v output). These two sites have existed since before CMK v1.6 so it’s possible that something is missing that is normally there on more recent installs, I just don’t know what that is.
Site Details
(Hostnames are not publicly accessible)
prod: Central CheckMK Server
prod_ext: DMZ CheckMK Server
cmk-update-agent register output
cmk-update-agent register -s vdmz-checkmk.vermeermfg.com -i prod_ext -H $(hostname | awk -F '.' '{print $1}') -U 'cmkautomation' -S $cmkSecret --insecure
All HTTPS connections are done insecurely, as you requested. As a consequence, no TLS verification will be done, i.e. the authenticity of the Checkmk server cannot be guaranteed. However, HTTPS connections are still TLS-encrypted while using the “–insecure” option.
Successfully registered agent of host “dv-netiqag1” for deployment.
cmk-update-agent -v output
cmk-update-agent -v
Updated the certificate store “/var/lib/check_mk_agent/cas/all_certs.pem” with 4 certificate(s)
±------------------------------------------------------------------+
| |
| Checkmk Agent Updater v2.2.0-2025.05.15 - Update |
| |
±------------------------------------------------------------------+
Getting target agent configuration for host ‘dv-netiqag1’ from deployment server
Agent Bakery: Unhandled exception: FileNotFoundError: [Errno 2] No such file or directory: ‘/omd/sites/prod_ext/var/check_mk/agents/linux_deb/packages’
See ~/var/log/web.log on site ‘prod_ext’ for details
See syslog or Logfile at /var/lib/check_mk_agent/cmk-update-agent.log for details.
I recently migrated our automatic updates to a similar setup due to an isolated network in the cloud using a private distributed node.
I found that using the agent updater rules to provide the update URL didn’t work as I expected with similar errors. I believe it embeds the information in the config of a dedicated package and probably overrides the global setting behavior.
This setup wasn’t as easy to follow on the Automatic Updates > Distrbuted Node documentation but did work, once I figured out the intent behind some of the wording.
In short, I had to:
Remove the URL components from the baked agent rule
Set the value(s) in the global settings under Automatic Updates.
Add a settings override for the remote URL for each distributed node.
Bake and sign the new package(s) for future deployments.
From there, I was able to use the cmk-update-agent register command and variables to register to the local distributed node. It would then pull the appropriate agent on check-in or ckm-update-agent -v to force it. If it was an older package installed or complains about signature, I could force the reinstall with --force option.
It’s complaining that /omd/sites/prod_ext/var/check_mk/agents/linux_deb/packages doesn’t exist on vdmz-checkmk.
On a brand new site & CheckMK server I created a month ago, the file exists (although it appears to be an outdated version) but on vdmz-checkmk it doesn’t exist. Any ideas how to get CheckMK to sync the agents over?
One item of note. I am running 2.3.0p33 CME at this time. Potentially a difference in behavior between releases but it sounds like it might not be the exact same scenario I was up against. Hopefully I’m not causing too much confusion.
An additional item I ran into was SSL between my distributed node reaching the central site via HTTPS. While browsers worked fine to both, the check_mk components on the distributed node had an issue.
I was including the SSL certificate and intermediates in the agent package similar to your screenshot but the actual agent wasn’t the issue. In the server side web logs, it did show the distributed node attempting to retrieve the package from the central site and failing.
I identified my problem with the openssl troubleshooting steps further down the documentation page. I saw errors about the intermediate cert(s) from the distributed node back to the central site. My fix was to update the SSL cert file on the central site to include the cert + intermediates and restart apache. Then the agent checked for an update, the remote node retrieved the package from the central site and the host agent updater successfully pulled it down and installed.
If you try openssl s_client -connect v-checkmk.vermeermfg.com:443 on your distributed node, it might indicate a similar issue.
If not this then hopefully someone else can chime in to assist.
