Error running automation call try-inventory (exit code 2), error for esx_vsphere_vm.running_on

CMK version: CEE 2.1.0p2
OS version: Ubuntu 20.04

Error message:

Error running automation call try-inventory (exit code 2), error:

Invalid characters in 'esx_vsphere_vm.running_on' for SectionName: '.'

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

~$ cmk --debug -vvI <hostname>
Discovering services and host labels on: <hostname>
<hostname>:
+ FETCHING DATA
  Source: SourceType.HOST/FetcherType.TCP
[cpu_tracking] Start [7f5ec4843910]
[TCPFetcher] Fetch with cache settings: DefaultAgentFileCache(<hostname>, base_path=/omd/sites/prod/tmp/check_mk/cache, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Using data from cache file /omd/sites/prod/tmp/check_mk/cache/<hostname>
Got 29021 bytes data from cache
[TCPFetcher] Use cached data
[cpu_tracking] Stop [7f5ec4843910 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
  Source: SourceType.HOST/FetcherType.PIGGYBACK
[cpu_tracking] Start [7f5ec48430d0]
[PiggybackFetcher] Fetch with cache settings: NoCache(<hostname>, base_path=/omd/sites/prod/tmp/check_mk/data_source_cache/piggyback, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=True, use_outdated=False, simulation=False)
Not using cache (Cache usage disabled)
[PiggybackFetcher] Execute data source
Piggyback file '/omd/sites/prod/tmp/check_mk/piggyback/<hostname>/<vcenter host>': Successfully processed from source '<vcenter host>'
No piggyback files for '<host ip>'. Skip processing.
Not using cache (Cache usage disabled)
[cpu_tracking] Stop [7f5ec48430d0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
+ PARSE FETCHER RESULTS
  Source: SourceType.HOST/FetcherType.TCP
<<<check_mk>>> / Transition NOOPParser -> HostSectionParser
<<<cmk_agent_ctl_status:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<wmi_cpuload:sep(124)>>> / Transition HostSectionParser -> HostSectionParser
<<<uptime>>> / Transition HostSectionParser -> HostSectionParser
<<<mem>>> / Transition HostSectionParser -> HostSectionParser
<<<fileinfo:sep(124)>>> / Transition HostSectionParser -> HostSectionParser
<<<df:sep(9)>>> / Transition HostSectionParser -> HostSectionParser
<<<winperf_phydisk>>> / Transition HostSectionParser -> HostSectionParser
<<<winperf_if>>> / Transition HostSectionParser -> HostSectionParser
<<<winperf_processor>>> / Transition HostSectionParser -> HostSectionParser
<<<services>>> / Transition HostSectionParser -> HostSectionParser
<<<logwatch>>> / Transition HostSectionParser -> HostSectionParser
<<<checkmk_agent_plugins_win:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<ps:sep(9)>>> / Transition HostSectionParser -> HostSectionParser
<<<wmi_webservices:sep(124)>>> / Transition HostSectionParser -> HostSectionParser
<<<dotnet_clrmemory:sep(124)>>> / Transition HostSectionParser -> HostSectionParser
Transition HostSectionParser -> NOOPParser
<<<iis_app_pool_state:sep(124)>>> / Transition NOOPParser -> HostSectionParser
<<<check_mk:cached(1655195226,3600)>>> / Transition HostSectionParser -> HostSectionParser
<<<windows_updates:cached(1655195165,14400)>>> / Transition HostSectionParser -> HostSectionParser
Transition HostSectionParser -> NOOPParser
<<<systemtime>>> / Transition NOOPParser -> HostSectionParser
Trying to acquire lock on /omd/sites/prod/var/check_mk/persisted/<hostname>
Got lock on /omd/sites/prod/var/check_mk/persisted/<hostname>
Releasing lock on /omd/sites/prod/var/check_mk/persisted/<hostname>
Released lock on /omd/sites/prod/var/check_mk/persisted/<hostname>
Stored persisted sections: win_cpuinfo, win_os, win_bios, win_system, win_computersystem, win_disks, win_video, win_networkadapter, win_ip_r, win_wmi_software, win_wmi_updates, win_reg_uninstall, win_exefiles
Using persisted section SectionName('win_cpuinfo')
Using persisted section SectionName('win_os')
Using persisted section SectionName('win_bios')
Using persisted section SectionName('win_system')
Using persisted section SectionName('win_computersystem')
Using persisted section SectionName('win_disks')
Using persisted section SectionName('win_video')
Using persisted section SectionName('win_networkadapter')
Using persisted section SectionName('win_ip_r')
Using persisted section SectionName('win_wmi_software')
Using persisted section SectionName('win_wmi_updates')
Using persisted section SectionName('win_reg_uninstall')
Using persisted section SectionName('win_exefiles')
  -> Add sections: ['check_mk', 'checkmk_agent_plugins_win', 'cmk_agent_ctl_status', 'df', 'dotnet_clrmemory', 'fileinfo', 'iis_app_pool_state', 'logwatch', 'mem', 'ps', 'services', 'systemtime', 'uptime', 'win_bios', 'win_computersystem', 'win_cpuinfo', 'win_disks', 'win_exefiles', 'win_ip_r', 'win_networkadapter', 'win_os', 'win_reg_uninstall', 'win_system', 'win_video', 'win_wmi_software', 'win_wmi_updates', 'windows_updates', 'winperf_if', 'winperf_phydisk', 'winperf_processor', 'wmi_cpuload', 'wmi_webservices']
  Source: SourceType.HOST/FetcherType.PIGGYBACK
<<<esx_vsphere_vm:cached(1655197805,90)>>> / Transition NOOPParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
No persisted sections
  -> Add sections: ['esx_vsphere_vm', 'labels']
Received no piggyback data
Received no piggyback data
+ ANALYSE DISCOVERED HOST LABELS
Trying host label discovery with: check_mk, checkmk_agent_plugins_win, cmk_agent_ctl_status, df, dotnet_clrmemory, esx_vsphere_vm, fileinfo, iis_app_pool_state, labels, logwatch, mem, ps, services, systemtime, uptime, win_bios, win_computersystem, win_cpuinfo, win_disks, win_exefiles, win_ip_r, win_networkadapter, win_os, win_reg_uninstall, win_system, win_video, win_wmi_software, win_wmi_updates, windows_updates, winperf_if, winperf_phydisk, winperf_processor, wmi_cpuload, wmi_webservices
  cmk/os_family: windows (check_mk)
  cmk/vsphere_object: vm (esx_vsphere_vm)
  cmk/piggyback_source_<vcenter host>: yes (labels)
Trying host label discovery with:
Trying to acquire lock on /omd/sites/prod/var/check_mk/crashes/base/fd3bb99e-ebc1-11ec-afb0-6b2b3191696c/crash.info
Got lock on /omd/sites/prod/var/check_mk/crashes/base/fd3bb99e-ebc1-11ec-afb0-6b2b3191696c/crash.info
Releasing lock on /omd/sites/prod/var/check_mk/crashes/base/fd3bb99e-ebc1-11ec-afb0-6b2b3191696c/crash.info
Released lock on /omd/sites/prod/var/check_mk/crashes/base/fd3bb99e-ebc1-11ec-afb0-6b2b3191696c/crash.info
Traceback (most recent call last):
  File "/omd/sites/prod/bin/cmk", line 92, in <module>
    exit_status = modes.call(mode_name, mode_args, opts, args)
  File "/omd/sites/prod/lib/python3/cmk/base/modes/__init__.py", line 69, in call
    return handler(*handler_args)
  File "/omd/sites/prod/lib/python3/cmk/base/modes/check_mk.py", line 1699, in mode_discover
    discovery.commandline_discovery(
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/__init__.py", line 182, in commandline_discovery
    _commandline_discovery_on_host(
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/__init__.py", line 249, in _commandline_discovery_on_host
    host_labels = analyse_node_labels(
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/_host_labels.py", line 85, in analyse_node_labels
    existing_host_labels=_load_existing_host_labels(host_key.hostname) if load_labels else (),
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/_host_labels.py", line 201, in _load_existing_host_labels
    return [HostLabel.from_dict(name, value) for name, value in raw_label_dict.items()]
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/_host_labels.py", line 201, in <listcomp>
    return [HostLabel.from_dict(name, value) for name, value in raw_label_dict.items()]
  File "/omd/sites/prod/lib/python3/cmk/base/discovered_labels.py", line 59, in from_dict
    plugin_name = None if raw_name is None else SectionName(raw_name)
  File "/omd/sites/prod/lib/python3/cmk/utils/type_defs/pluginname.py", line 58, in __new__
    cls._validate_args(plugin_name)
  File "/omd/sites/prod/lib/python3/cmk/utils/type_defs/pluginname.py", line 48, in _validate_args
    raise ValueError(
ValueError: Invalid characters in 'esx_vsphere_vm.running_on' for SectionName: '.'

I just updated from 2.0.0p24 to 2.1.0p2. Most worked out fine, but couldn’t identify this issue in the werks. How do I fix this without doing a cmk -II hostname for all affected hosts?

Is it possible that there is a file “esx_vsphere_vm” inside the folder “~/local/share/check_mk/checks/”?
If yes then this is a modified version of the old 2.0 esx_vsphere_vm check. You need to remove the modified version.

No, there are files in there but none of them contain esx inside the file name or inside the file

Just noticed, the “Check_MK inventory” fails in a slightly different way, but maybe the the additional information helps:

Exception:	
ValueError (Invalid characters in 'esx_vsphere_vm.running_on' for SectionName: '.')

Traceback:
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/decorator.py", line 43, in wrapped_check_func
    status, output_text = _combine_texts(check_func(hostname, *args, **kwargs))
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/__init__.py", line 563, in active_check_discovery
    host_labels = analyse_host_labels(
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/_host_labels.py", line 45, in analyse_host_labels
    else analyse_node_labels(
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/_host_labels.py", line 85, in analyse_node_labels
    existing_host_labels=_load_existing_host_labels(host_key.hostname) if load_labels else (),
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/_host_labels.py", line 201, in _load_existing_host_labels
    return [HostLabel.from_dict(name, value) for name, value in raw_label_dict.items()]
  File "/omd/sites/prod/lib/python3/cmk/base/agent_based/discovery/_host_labels.py", line 201, in <listcomp>
    return [HostLabel.from_dict(name, value) for name, value in raw_label_dict.items()]
  File "/omd/sites/prod/lib/python3/cmk/base/discovered_labels.py", line 59, in from_dict
    plugin_name = None if raw_name is None else SectionName(raw_name)
  File "/omd/sites/prod/lib/python3/cmk/utils/type_defs/pluginname.py", line 58, in __new__
    cls._validate_args(plugin_name)
  File "/omd/sites/prod/lib/python3/cmk/utils/type_defs/pluginname.py", line 48, in _validate_args
    raise ValueError(

Local Variables:
{'cls': <class 'cmk.utils.type_defs.pluginname.SectionName'>,
 'invalid': '.',
 'plugin_name': 'esx_vsphere_vm.running_on'}

So, I had a deeper look into it. The offending part is the discovered host label:

	'cmk/vsphere_object': 
	{
		'value': 'vm', 
		'plugin_name': 'esx_vsphere_vm.running_on'
	}

The plugin_name contains a non valid character, thats where the ValueError comes from.

I don’t know when this label got added, but it is in a lot of our hosts. I guess it’s a 1.6 legacy thing? I used sed to remove the .running_on in var/check_mk/discovered_host_labels and checkmk is now happy again!

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.