Update to 2.4: WARNING: Invalid rule configuration detected

Current: Checkmk Raw Edition 2.3.0p31
Updating to: 2.4.0 cre

I guess deleting this ruleset and adding it after the upgrade might work, but why is this happening at all?

Executing 'cmk-update-config --conflict ask --dry-run'
-| ATTENTION
-|   Some steps may take a long time depending on your installation.
-|   Please be patient.
-| 
-| Cleanup precompiled host and folder files
-| Verifying Checkmk configuration...
-|  01/08 Legacy check plug-ins...
-|  02/08 Rulesets...
-| WARNING: Invalid rule configuration detected
-| Ruleset: active_checks:sftp
-| Title: Check SFTP Service
-| Folder: main
-| Rule nr: 1
-| Exception: ('sftpgobn.misoft.de', 'checkmk', ('password', 'something'), {})
-| 
-| Traceback (most recent call last):
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/plugins/pre_actions/rulesets.py", line 134, in _validate
_rule_values
-|     transformed_value = ruleset.rulespec.valuespec.transform_value(rule.value)
-|                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-|   File "/omd/sites/misoft/lib/python3/cmk/gui/valuespec/definitions.py", line 6634, in transform_value
-|     return self.from_valuespec(self._valuespec.transform_value(self.to_valuespec(value)))
-|                                                                ^^^^^^^^^^^^^^^^^^^^^^^^
-|   File "/omd/sites/misoft/lib/python3/cmk/plugins/collection/rulesets/sftp.py", line 38, in _migrate_from_tuple
-|     raise ValueError(params)
-| ValueError: ('sftpgobn.misoft.de', 'checkmk', ('password', 'something'), {})
-| 
-| During handling of the above exception, another exception occurred:
-| 
-| Traceback (most recent call last):
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/main.py", line 104, in main_check_config
-|     check_config(logger, conflict)
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/main.py", line 239, in check_config
-|     pre_action(logger, conflict_mode)
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/plugins/pre_actions/rulesets.py", line 50, in __call__
-|     result = _validate_rule_values(
-|              ^^^^^^^^^^^^^^^^^^^^^^
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/plugins/pre_actions/rulesets.py", line 171, in _validate
_rule_values
-|     if _continue_on_invalid_rule(conflict_mode).is_abort():
-|        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/plugins/pre_actions/rulesets.py", line 95, in _continue_
on_invalid_rule
-|     return continue_per_users_choice(
-|            ^^^^^^^^^^^^^^^^^^^^^^^^^^
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/plugins/pre_actions/utils.py", line 174, in continue_per
_users_choice
-|     while (response := _prompt(prompt_text).lower()) not in [
-|                        ^^^^^^^^^^^^^^^^^^^^
-|   File "/omd/sites/misoft/lib/python3/cmk/update_config/plugins/pre_actions/utils.py", line 43, in _prompt
-|     tcflush(sys.stdin, TCIFLUSH)
-| termios.error: (25, 'Inappropriate ioctl for device')
-| 
-| Update aborted with Error: (25, 'Inappropriate ioctl for device').
-| Your site has not been modified.
-| The update can be retried after the error has been fixed.

Thanks for reporting this. Sorry for the inconvenience.
We have opened an internal ticket.

2 Likes

Still the same problem with 2.4.0p1 ;(

I have had the same issue. I created another rule with the same values, worked fine. Before, I’ve tried changing each and every field of the original rule, but upon saving, it always errored…

Hi @Jay2k1,

thank your adding your experience.

Still, I would like to understand it better:

I created another rule with the same values, worked fine.
Before, I’ve tried changing each and every field of the original rule, but upon saving, it always errored…

Was both AFTER upgrading to 2.4.0p1?

Sunny Greetings and thank you
Hartmut

Hi Hartmut,

short answer: These error messages appeared both during the upgrade from 2.3 to 2.4 (where I’ve ignored them) and again from 2.4.0 to 2.4.0p1.
After upgrading to p1, I did my “experiments”. So yes, after going to p1, I’ve tried

  • fixing the problem by changing each rule value separately and saving, to no avail
  • replicating the error by basically cloning the problematic rule manually with identical values which just resulted in a working rule

Long answer:
During the update to 2.4, I’ve had a lot of issues - an incompatible notification mkp for telegram, three or four incompatible other mkps, warnings about the old check_http rules and some more. The right thing to do would’ve been to roll back and try to fix everything before the upgrade so that it runs through flawlessly, but because I’ve had lots of warnings and python errors during most of my incremental 2.3 upgrades, asking me to either a) abort or continue, or b) abort or disable the mkp, and I’ve always just continued, disabling mkps where necessary, and then re-enabling them afterwards (they kept working), my mindset was to try and preserve as much as possible, try to get 2.4 to a working state and then fixing what’s necessary.
I noticed the SFTP rule messages during both 2.3->2.4 and 2.4.0->2.4.0p1 upgrades but forgot to check for the rules in the GUI after the upgrade initially. The p1 upgrade reminded me that there was something I should look at, which is when I arrived at the conclusion from my above message.

This, I think, is hence a bug in transitioning check_sftp rules from 2.3 to 2.4 somehow.

I have a snapshot from before fixing the SFTP rule and upgrading to p1 (so, 2.4.0.cre). I just rolled back to test some more, and when going to the “check SFTP service” rules page, it tells me “the value of this rules is not valid”:

image

If I then click the clone icon, I get this:

image

I then went ahead and copy pasted all values from the yellow warning message into the fields again and saved, which worked.

I’ve then looked at the generated .mk file, the rules looked like this:

active_checks['sftp'] = [
{'condition': {'host_folder': '/%s/' % FOLDER_PATH,
               'host_name': ['pterodactyl']},
 'id': '9e856574-1783-4633-8578-76f1045baf99',
 'options': {'description': 'THIS IS THE BROKEN RULE', 'disabled': False},
 'value': ('pterodactyl.hostname.redacted',
           'username.redacted',
           ('password', 'password.redacted'),
           {'description': 'SFTP Wings redacted', 'port': 2022})},
{'condition': {'host_folder': '/%s/' % FOLDER_PATH,
               'host_name': ['pterodactyl']},
 'id': 'e13b05db-efb8-48b7-91c0-ffddbe653e7c',
 'options': {'description': 'THIS IS THE WORKING MANUAL CLONE RULE', 'disabled': False},
 'value': {'description': 'SFTP Wings redacted',
           'host': 'pterodactyl.hostname.redacted',
           'port': 2022,
           'secret': ('cmk_postprocessed',
                      'explicit_password',
                      ('uuidda4f144a-63a4-40b0-8588-acb6dd150248',
                       'password.redacted')),
           'user': 'username.redacted'}},
] + active_checks['sftp']

If I then put them next to each other and manually pretty-print them, adding line breaks and tabs, it becomes clear that the value format is different with a newly created rule (old and broken on the left, new and working on the right):

Again, this debugging with these screenshots was just done on 2.4.0.cre.

It seems to me that in 2.4, a slightly different, more reasonable data format was used than in 2.3, but there’s no (or bugged) backwards compatibility/automatic transitioning for old existing rules.

Hope this helps!

EDIT: Here’s the output of the updater going 2.4.0->2.4.0p1 when having both the broken and the new rule in place:

2025-05-19 17:37:26 - Updating site 'monitoring' from version 2.4.0.cre to 2.4.0p1.cre...

Creating temporary filesystem /omd/sites/monitoring/tmp...OK
Executing 'cmk-update-config --conflict ask --dry-run'
-| ATTENTION
-|   Some steps may take a long time depending on your installation.
-|   Please be patient.
-|
-| Cleanup precompiled host and folder files
-| Verifying Checkmk configuration...
-|  01/08 Legacy check plug-ins...
-|  02/08 Rulesets...
-| WARNING: Invalid rule configuration detected
-| Ruleset: active_checks:sftp
-| Title: Check SFTP Service
-| Folder: privat
-| Rule nr: 1
-| Exception: ('pterodactyl.hostname.redacted', 'username.redacted', ('password', 'password.redacted'), {'description': 'SFTP Wings redacted', 'port': 2022})
-|
-| You can abort the update process (A) or continue (c) the update. Abort update? [A/c]
c
-|  03/08 UI extensions...
-|  04/08 Migrate Azure Databases...
-|  05/08 Agent based plugins...
-|  06/08 Autochecks...
-|  07/08 Invalid hosts labels...
-|  08/08 Deprecated .mk configuration of plugins...
-| Done (success)
-|

Completed verifying site configuration. Your site now has version 2.4.0p1.cre.
Executing update-pre-hooks script "01_mkp-disable-outdated"...
-| [proxmox_provisioned 1.3.1]: enabled
-| [sas2ircu 1.1]: enabled
-| [check_calculate 0.2.2]: disabled
OK
Executing update-pre-hooks script "02_cmk-update-config"...
-| ATTENTION
-|   Some steps may take a long time depending on your installation.
-|   Please be patient.
-|
-| Cleanup precompiled host and folder files
-| Verifying Checkmk configuration...
-|  01/08 Legacy check plug-ins...
-|  02/08 Rulesets...
-| WARNING: Invalid rule configuration detected
-| Ruleset: active_checks:sftp
-| Title: Check SFTP Service
-| Folder: privat
-| Rule nr: 1
-| Exception: ('pterodactyl.hostname.redacted', 'username.redacted', ('password', 'password.redacted'), {'description': 'SFTP Wings redacted', 'port': 2022})
-|
-|  03/08 UI extensions...
-|  04/08 Migrate Azure Databases...
-|  05/08 Agent based plugins...
-|  06/08 Autochecks...
-|  07/08 Invalid hosts labels...
-|  08/08 Deprecated .mk configuration of plugins...
-| Done (success)
-|
-| Updating Checkmk configuration...
-|  01/30 User attributes...
-|  02/30 Create precompiled host and folder files...
-|  03/30 Update dashboards...
-|  04/30 Global settings...
-|  05/30 Convert counter files...
-|  06/30 Migrate Azure Databases...
-|  07/30 Message broker port of site connections...
-|  08/30 Rulesets...
-| ERROR: Failed to transform rule: (Ruleset: active_checks:sftp, Folder: privat, Rule: 0, Value: ('pterodactyl.hostname.redacted', 'username.redacted', ('password', 'password.redacted'), {'description': 'SFTP Wings redacted', 'port': 2022}): ('pterodactyl.hostname.redacted', 'username.redacted', ('password', 'password.redacted'), {'description': 'SFTP Wings redacted', 'port': 2022})
-|  WARNING: Exception in ruleset implementation detected (Ruleset: active_checks:sftp, Title: Check SFTP Service, Folder: privat,
-| Rule nr: 0, Exception: ('pterodactyl.hostname.redacted', 'username.redacted', ('password', 'password.redacted'), {'description': 'SFTP Wings redacted', 'port': 2022}))
-| Traceback (most recent call last):
-|   File "/omd/sites/monitoring/lib/python3/cmk/update_config/plugins/actions/rulesets.py", line 36, in validate_rule_values
-|     ruleset.rulespec.valuespec.validate_datatype(
-|   File "/omd/sites/monitoring/lib/python3/cmk/gui/valuespec/definitions.py", line 6628, in validate_datatype
-|     self._valuespec.validate_datatype(self.to_valuespec(value), varprefix)
-|                                       ^^^^^^^^^^^^^^^^^^^^^^^^
-|   File "/omd/sites/monitoring/lib/python3/cmk/plugins/collection/rulesets/sftp.py", line 38, in _migrate_from_tuple
-|     raise ValueError(params)
-| ValueError: ('pterodactyl.hostname.redacted', 'username.redacted', ('password', 'password.redacted'), {'description': 'SFTP Wings redacted', 'port': 2022})
-| A crash report was generated with ID: 41f3846c-34c7-11f0-bb71-bc2411e0664e
-|  Detected 1 issue(s) in loaded rulesets. This is a problem with the plug-in implementation. It needs to be addressed by the maintainers. Please review the crashes in the crash reports page to help fix the issues. Until all issues are resolved, we recommend disabling the affected rules.
-|  09/30 Notification host tag conditions...
-|  10/30 Autochecks...
-|  11/30 Migrate CLI parent scan config...
-|  12/30 Process discovery for self monitoring...
-|  13/30 Update background jobs...
-|  14/30 Remove leftovers of user profile cleanup background job...
-|  15/30 Migrate notifications...
-|        Wrote notification configuration backup to
-|        /omd/sites/monitoring/notifications_backup.mk.
-|
-|        Please check if the notification pages in the GUI work as expected.
-|        In case of problems you can copy the backup files back to
-|        /omd/sites/monitoring/etc/check_mk/conf.d/wato/notifications.mk.
-|        If everything works as expected you can remove the backup.
-|
-|  16/30 Cleanup version specific caches...
-|  17/30 Host attribute topics...
-|  18/30 Terminating all existing user sessions...
-|  19/30 Update LDAP connections...
-|  20/30 Remove unreadable prediction files...
-|  21/30 Topics...
-|  22/30 Event Console: Migrate history files to sqlite...
-|  23/30 Remove invalid hosts labels...
-|  24/30 Migrate etc/diskspace.conf...
-|  25/30 Reset deprecations scheduling...
-|  26/30 Remove zipped status data tree files...
-|  27/30 Ensure message broker certs are ready...
-| Certificate found.
-|  28/30 Check for deprecated check_http plug-in rules...
-| WARNING: You have 4 rules using the ruleset Check HTTP service deprecated.
-| This ruleset will be deprecated along with the old HTTP monitoring plug-in in the next version(s) of Checkmk.
-| Rules must therefore be migrated to the new ruleset which is used by the httpv2 plugin.
-| Rule migration can be done manually or by calling cmk-migrate-http as site user. See cmk-migrate-http --help for more information on this helper script.
-| For additional information on the deprecation of the HTTP plug-in see the werk #17665.
-|  29/30 Validating configuration files...
-|  30/30 Update core config...
-| Generating configuration for core (type nagios)...
-| Precompiling host checks...OK
-| Done (success)
OK
Finished update.
1 Like

2.4.0p2.cre still produces this error…

Good Morning @Jay2k1,

thank you for the extensive answer.

I will take this internally and ask a developer to have a look at issue in the next days (as we’re heading for the Checkmk conference in the next hour).
I will keep you posted.

Sunny Greetings
Hartmut

2 Likes

2.4.0p3.cre still produces this error…

Hello @peterge,

I am sorry - yes, it’s still persists in 2.4.0p3
We have a potential fix now, but still need to verify it.

Sunny Greetings
Hartmut

1 Like

Any news? :open_mouth:
some more characters…

Good Morning @peterge,

I am sorry, that it takes so long - that’s way longer than it should be.
Due to high load and some vacations, the bugfix hadn’t been verified yet - I reached out again internally last Friday. I hope, we can finish this topic in the next days.

Sunny Greetings
Hartmut

1 Like

Already fixed as part of Werk #18102: check_sftp: Fix invalid rule configuration when updating

2 Likes