CMK version: 2.4.0P27 CME
**OS version: CheckMK Appliance 1.7.18
**
When trying to upgrade a distributed Checkmk environment from 2.4.0p27 to 2.5.0, the upgrade fails on the remote sites during `cmk-update-config --conflict abort --dry-run`.
Environment:
- Distributed setup with one central site and four remote sites
- Current version: Checkmk 2.4.0p27
- Target version: Checkmk 2.5.0
- The error occurs on the remote sites
- The affected ruleset is `agent_config:cmk_update_agent`
Error output:
Executing ‘cmk-update-config --conflict abort --dry-run’
Update aborted with:
Ruleset: agent_config:cmk_update_agent
Title: Agent updater (Linux, Windows, Solaris)
Folder: main
Rule nr: 1
Exception: Validation errors: [ValidationMessage(location=[‘signature_keys’], message=‘The minimum allowed length is 1.’, replacement_value=[])]
The update is then aborted and rolled back successfully.
Troubleshooting performed so far:
- Verified that the environment is already on Checkmk 2.4.0p27 before attempting the upgrade.
- Checked for empty signature key lists with:
grep -RIn --include=“*.mk” “signature_keys.*\[\]” ~/etc/check_mk/conf.d/wato/
This returned no results.
- Edited the Agent updater rule in the GUI.
- Explicitly deselected all signature keys, saved the rule, edited it again, selected the current key, and saved again.
- Performed Activate Changes / Config push.
- Performed Bake and Sign agents again.
- Retried but the same validation error remained.
- Removed the Agent updater rule completely from the GUI and recreated it.
- Retried the update dry-run, but the same validation error still occurs.
Current observation:
Even after removing/recreating the Agent updater, the 2.5.0 update validation still fails on the remote site with:
location=[‘signature_keys’]
message=‘The minimum allowed length is 1.’
replacement_value=[]
Question:
Could this be a migration issue in distributed environments, stale WATO/distributed configuration data on remote sites?
Full Error message:
Error during update:
Cannot execute omd command omd -V 2.5.0.ultimatemt update --pre-flight=abort --skeleton=install --confirm-version --confirm-edition SITE_NAME: Cleaning up tmp directory...OK
2026-05-04 11:01:44 - Updating site 'SITE_NAME' from version 2.4.0p27.cme to 2.5.0.ultimatemt...
* Installed dir .restore_working_dir
* Updated .profile
* Installed dir etc/clickhouse-server
* Installed dir etc/agent-receiver
* Updated etc/mk-livestatus/nagios.cfg
* Updated etc/init.d/mknotifyd
* Updated etc/init.d/agent-receiver
* Updated etc/init.d/liveproxyd
* Updated etc/init.d/cmc
* Updated etc/init.d/automation-helper
* Installed file etc/init.d/metric-backend
* Updated etc/init.d/otel-collector
* Updated etc/rabbitmq/conf.d/00-default.conf
* Updated etc/rabbitmq/definitions.d/00-default.json
* Installed dir etc/clickhouse-server/config.d
* Installed dir etc/clickhouse-server/users.d
* Installed file etc/clickhouse-server/users.xml
* Installed file etc/clickhouse-server/config.xml
* Updated etc/ssl/openssl.cnf.dist
* Updated etc/ssl/openssl.cnf
* Permissions 0640 -> 0750 etc/ssl/misc/CA.pl
* Updated etc/ssl/misc/tsget.pl
* Updated etc/nagios/conf.d/check_mk_templates.cfg
* Installed file etc/agent-receiver/gunicorn.conf.py
* Updated etc/check_mk/apache.conf
* Updated etc/apache/apache.conf
* Updated etc/apache/conf.d/check_mk.conf
* Updated etc/apache/conf.d/security.conf
* Updated etc/apache/conf.d/nagvis.conf
* Installed link etc/rc.d/92-metric-backend
* Installed file etc/cron.d/cmk_transform_inventory_trees
* Updated etc/otel-collector/fallback.yaml
* Installed dir var/clickhouse-server
* Installed dir var/log/clickhouse-server
* Vanished var/check_mk/persisted
* Vanished var/monitoring-plugins
* Vanished etc/default/cmc
* Vanished etc/default
* Vanished .cache/.gitkeep
Creating temporary filesystem /omd/sites/SITE_NAME/tmp...OK
Executing 'cmk-update-config --conflict abort --dry-run'
-| ATTENTION
-| Some steps may take a long time depending on your installation.
-| Please be patient.
-|
-| Cleanup precompiled host and folder files
-| Verifying Checkmk configuration...
-| 01/12 Legacy check plug-ins...
-| 02/12 Rulesets...
-| WARNING: Invalid rule configuration detected
-| Ruleset: agent_config:cmk_update_agent
-| Title: Agent updater (Linux, Windows, Solaris)
-| Folder: main
-| Rule nr: 1
-| Exception: Validation errors: [ValidationMessage(location=['signature_keys'], message='The minimum allowed length is 1.', replacement_value=[])]
-|
-|
-| Update aborted with Error: failed ruleset validation.
-| The update can be retried after the error has been fixed.
The update was rolled back successfully.
