Now the service in question is a mount on a DRBD device managed by Pacemaker/Corosync and it really only is active on the first node, so I don’t see why this check complains that it gets additional results.
I’d like to set the aggregation option to Failover, but at the moment this doesn’t work because of this problem.
CMK version: 2.1.0p9.cre
OS version: Debian GNU/Linux 10 (buster)
Error message:
Best: [nfsrefer-01], 5.77% used (228.59 MB of 3.87 GB), trend: +16.91 B / 24 hours, Additional results from: [nfsrefer-02]
Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)
Checkmk version 2.1.0p9
Try license usage history update.
Trying to acquire lock on /omd/sites/zdv/var/check_mk/license_usage/next_run
Got lock on /omd/sites/zdv/var/check_mk/license_usage/next_run
Trying to acquire lock on /omd/sites/zdv/var/check_mk/license_usage/history.json
Got lock on /omd/sites/zdv/var/check_mk/license_usage/history.json
Next run time has not been reached yet. Abort.
Releasing lock on /omd/sites/zdv/var/check_mk/license_usage/history.json
Released lock on /omd/sites/zdv/var/check_mk/license_usage/history.json
Releasing lock on /omd/sites/zdv/var/check_mk/license_usage/next_run
Released lock on /omd/sites/zdv/var/check_mk/license_usage/next_run
+ FETCHING DATA
Source: SourceType.HOST/FetcherType.TCP
[cpu_tracking] Start [7f880ecadeb0]
[TCPFetcher] Fetch with cache settings: DefaultAgentFileCache(nfsrefer-01, base_path=/omd/sites/zdv/tmp/check_mk/cache, max_age=MaxAge(checking=90, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Using data from cache file /omd/sites/zdv/tmp/check_mk/cache/nfsrefer-01
Got 106381 bytes data from cache
[TCPFetcher] Use cached data
[cpu_tracking] Stop [7f880ecadeb0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
Source: SourceType.HOST/FetcherType.PIGGYBACK
[cpu_tracking] Start [7f880e940820]
[PiggybackFetcher] Fetch with cache settings: NoCache(nfsrefer-01, base_path=/omd/sites/zdv/tmp/check_mk/data_source_cache/piggyback, max_age=MaxAge(checking=90, discovery=120, inventory=120), disabled=True, use_outdated=False, simulation=False)
Not using cache (Cache usage disabled)
[PiggybackFetcher] Execute data source
Piggyback file '/omd/sites/zdv/tmp/check_mk/piggyback/nfsrefer-01/backup-01': Successfully processed from source 'backup-01'
No piggyback files for '10.94.23.114'. Skip processing.
Not using cache (Cache usage disabled)
[cpu_tracking] Stop [7f880e940820 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
Source: SourceType.HOST/FetcherType.TCP
[cpu_tracking] Start [7f880e9400a0]
[TCPFetcher] Fetch with cache settings: DefaultAgentFileCache(nfsrefer-02, base_path=/omd/sites/zdv/tmp/check_mk/cache, max_age=MaxAge(checking=90, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Using data from cache file /omd/sites/zdv/tmp/check_mk/cache/nfsrefer-02
Got 86322 bytes data from cache
[TCPFetcher] Use cached data
[cpu_tracking] Stop [7f880e9400a0 - Snapshot(process=posix.times_result(user=0.010000000000000231, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
Source: SourceType.HOST/FetcherType.PIGGYBACK
[cpu_tracking] Start [7f880e940e50]
[PiggybackFetcher] Fetch with cache settings: NoCache(nfsrefer-02, base_path=/omd/sites/zdv/tmp/check_mk/data_source_cache/piggyback, max_age=MaxAge(checking=90, discovery=120, inventory=120), disabled=True, use_outdated=False, simulation=False)
Not using cache (Cache usage disabled)
[PiggybackFetcher] Execute data source
Piggyback file '/omd/sites/zdv/tmp/check_mk/piggyback/nfsrefer-02/backup-01': Successfully processed from source 'backup-01'
No piggyback files for '10.94.23.115'. Skip processing.
Not using cache (Cache usage disabled)
[cpu_tracking] Stop [7f880e940e50 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.009999997913837433))]
+ PARSE FETCHER RESULTS
Source: SourceType.HOST/FetcherType.TCP
<<<check_mk>>> / Transition NOOPParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<systemd_units>>> / Transition HostSectionParser -> HostSectionParser
<<<nfsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<cifsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<mounts>>> / Transition HostSectionParser -> HostSectionParser
<<<ps_lnx>>> / Transition HostSectionParser -> HostSectionParser
<<<mem>>> / Transition HostSectionParser -> HostSectionParser
<<<cpu>>> / Transition HostSectionParser -> HostSectionParser
<<<uptime>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if:sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<tcp_conn_stats>>> / Transition HostSectionParser -> HostSectionParser
<<<diskstat>>> / Transition HostSectionParser -> HostSectionParser
<<<kernel>>> / Transition HostSectionParser -> HostSectionParser
<<<md>>> / Transition HostSectionParser -> HostSectionParser
<<<vbox_guest>>> / Transition HostSectionParser -> HostSectionParser
<<<drbd>>> / Transition HostSectionParser -> HostSectionParser
<<<heartbeat_crm>>> / Transition HostSectionParser -> HostSectionParser
<<<fileinfo:sep(124)>>> / Transition HostSectionParser -> HostSectionParser
<<<chrony:cached(1660041746,30)>>> / Transition HostSectionParser -> HostSectionParser
<<<mrpe>>> / Transition HostSectionParser -> HostSectionParser
<<<mrpe>>> / Transition HostSectionParser -> HostSectionParser
<<<mrpe:cached(1660041569,300)>>> / Transition HostSectionParser -> HostSectionParser
<<<local:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<cfengine3>>> / Transition HostSectionParser -> HostSectionParser
<<<debian_lts_status>>> / Transition HostSectionParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<logwatch>>> / Transition HostSectionParser -> HostSectionParser
<<<netstat>>> / Transition HostSectionParser -> HostSectionParser
<<<logfiles:cached(1660041569,300)>>> / Transition HostSectionParser -> HostSectionParser
<<<apt:cached(1660039388,3600):sep(0)>>> / Transition HostSectionParser -> HostSectionParser
Trying to acquire lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-01
Got lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-01
Releasing lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-01
Released lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-01
Stored persisted sections: lnx_packages, lnx_distro, lnx_cpuinfo, dmidecode, lnx_uname, lnx_video, lnx_ip_r, lnx_sysctl, lnx_block_devices
Using persisted section SectionName('lnx_packages')
Using persisted section SectionName('lnx_distro')
Using persisted section SectionName('lnx_cpuinfo')
Using persisted section SectionName('dmidecode')
Using persisted section SectionName('lnx_uname')
Using persisted section SectionName('lnx_video')
Using persisted section SectionName('lnx_ip_r')
Using persisted section SectionName('lnx_sysctl')
Using persisted section SectionName('lnx_block_devices')
-> Add sections: ['apt', 'cfengine3', 'check_mk', 'chrony', 'cifsmounts', 'cpu', 'debian_lts_status', 'df', 'diskstat', 'dmidecode', 'drbd', 'fileinfo', 'heartbeat_crm', 'kernel', 'labels', 'lnx_block_devices', 'lnx_cpuinfo', 'lnx_distro', 'lnx_if', 'lnx_ip_r', 'lnx_packages', 'lnx_sysctl', 'lnx_uname', 'lnx_video', 'local', 'logfiles', 'logwatch', 'md', 'mem', 'mounts', 'mrpe', 'netstat', 'nfsmounts', 'ps_lnx', 'systemd_units', 'tcp_conn_stats', 'uptime', 'vbox_guest']
Source: SourceType.HOST/FetcherType.PIGGYBACK
<<<burp:cached(1660041510,300)>>> / Transition NOOPParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
No persisted sections
-> Add sections: ['burp', 'labels']
Source: SourceType.HOST/FetcherType.TCP
<<<check_mk>>> / Transition NOOPParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<systemd_units>>> / Transition HostSectionParser -> HostSectionParser
<<<nfsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<cifsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<mounts>>> / Transition HostSectionParser -> HostSectionParser
<<<ps_lnx>>> / Transition HostSectionParser -> HostSectionParser
<<<mem>>> / Transition HostSectionParser -> HostSectionParser
<<<cpu>>> / Transition HostSectionParser -> HostSectionParser
<<<uptime>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if:sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<tcp_conn_stats>>> / Transition HostSectionParser -> HostSectionParser
<<<diskstat>>> / Transition HostSectionParser -> HostSectionParser
<<<kernel>>> / Transition HostSectionParser -> HostSectionParser
<<<md>>> / Transition HostSectionParser -> HostSectionParser
<<<vbox_guest>>> / Transition HostSectionParser -> HostSectionParser
<<<drbd>>> / Transition HostSectionParser -> HostSectionParser
<<<heartbeat_crm>>> / Transition HostSectionParser -> HostSectionParser
<<<fileinfo:sep(124)>>> / Transition HostSectionParser -> HostSectionParser
<<<chrony:cached(1660041716,30)>>> / Transition HostSectionParser -> HostSectionParser
<<<mrpe>>> / Transition HostSectionParser -> HostSectionParser
<<<mrpe>>> / Transition HostSectionParser -> HostSectionParser
<<<mrpe:cached(1660041655,300)>>> / Transition HostSectionParser -> HostSectionParser
<<<local:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<cfengine3>>> / Transition HostSectionParser -> HostSectionParser
<<<debian_lts_status>>> / Transition HostSectionParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<logwatch>>> / Transition HostSectionParser -> HostSectionParser
<<<netstat>>> / Transition HostSectionParser -> HostSectionParser
<<<logfiles:cached(1660041475,300)>>> / Transition HostSectionParser -> HostSectionParser
<<<apt:cached(1660041184,3600):sep(0)>>> / Transition HostSectionParser -> HostSectionParser
Trying to acquire lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-02
Got lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-02
Releasing lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-02
Released lock on /omd/sites/zdv/var/check_mk/persisted/nfsrefer-02
Stored persisted sections: lnx_packages, lnx_distro, lnx_cpuinfo, dmidecode, lnx_uname, lnx_video, lnx_ip_r, lnx_sysctl, lnx_block_devices
Using persisted section SectionName('lnx_packages')
Using persisted section SectionName('lnx_distro')
Using persisted section SectionName('lnx_cpuinfo')
Using persisted section SectionName('dmidecode')
Using persisted section SectionName('lnx_uname')
Using persisted section SectionName('lnx_video')
Using persisted section SectionName('lnx_ip_r')
Using persisted section SectionName('lnx_sysctl')
Using persisted section SectionName('lnx_block_devices')
-> Add sections: ['apt', 'cfengine3', 'check_mk', 'chrony', 'cifsmounts', 'cpu', 'debian_lts_status', 'df', 'diskstat', 'dmidecode', 'drbd', 'fileinfo', 'heartbeat_crm', 'kernel', 'labels', 'lnx_block_devices', 'lnx_cpuinfo', 'lnx_distro', 'lnx_if', 'lnx_ip_r', 'lnx_packages', 'lnx_sysctl', 'lnx_uname', 'lnx_video', 'local', 'logfiles', 'logwatch', 'md', 'mem', 'mounts', 'mrpe', 'netstat', 'nfsmounts', 'ps_lnx', 'systemd_units', 'tcp_conn_stats', 'uptime', 'vbox_guest']
Source: SourceType.HOST/FetcherType.PIGGYBACK
<<<burp:cached(1660041510,300)>>> / Transition NOOPParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
No persisted sections
-> Add sections: ['burp', 'labels']
Received no piggyback data
Received no piggyback data
Received no piggyback data
Received no piggyback data
[cpu_tracking] Start [7f880dd48b50]
value store: synchronizing
Trying to acquire lock on /omd/sites/zdv/tmp/check_mk/counters/nfsrefer
Got lock on /omd/sites/zdv/tmp/check_mk/counters/nfsrefer
value store: loading from disk
Releasing lock on /omd/sites/zdv/tmp/check_mk/counters/nfsrefer
Released lock on /omd/sites/zdv/tmp/check_mk/counters/nfsrefer
Filesystem /data Best: [nfsrefer-01], 5.77% used (228.59 MB of 3.87 GB), trend: +16.92 B / 24 hours, Additional results from: [nfsrefer-02]
Filesystem /uni-mainz.de Best: [nfsrefer-01], 11.63% used (453.05 MB of 3.80 GB), trend: +41.01 kB / 24 hours, Additional results from: [nfsrefer-02]
[cpu_tracking] Stop [7f880dd48b50 - Snapshot(process=posix.times_result(user=0.009999999999999787, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.019999999552965164))]
Trying to acquire lock on /omd/sites/zdv/var/check_mk/crashes/base/250762ae-17d0-11ed-9298-a0369ff0d398/crash.info
Got lock on /omd/sites/zdv/var/check_mk/crashes/base/250762ae-17d0-11ed-9298-a0369ff0d398/crash.info
Releasing lock on /omd/sites/zdv/var/check_mk/crashes/base/250762ae-17d0-11ed-9298-a0369ff0d398/crash.info
Released lock on /omd/sites/zdv/var/check_mk/crashes/base/250762ae-17d0-11ed-9298-a0369ff0d398/crash.info
Traceback (most recent call last):
File "/omd/sites/zdv/bin/cmk", line 98, in <module>
exit_status = modes.call("--check", None, opts, args)
File "/omd/sites/zdv/lib/python3/cmk/base/modes/__init__.py", line 69, in call
return handler(*handler_args)
File "/omd/sites/zdv/lib/python3/cmk/base/modes/check_mk.py", line 1804, in mode_check
checking.commandline_checking(
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/decorator.py", line 43, in wrapped_check_func
status, output_text = _combine_texts(check_func(hostname, *args, **kwargs))
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/__init__.py", line 121, in commandline_checking
return _execute_checkmk_checks(
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/__init__.py", line 174, in _execute_checkmk_checks
num_success, plugins_missing_data = check_host_services(
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/__init__.py", line 322, in check_host_services
success = _execute_check(
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/__init__.py", line 382, in _execute_check
submittable = get_aggregated_result(
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/__init__.py", line 470, in get_aggregated_result
result = _aggregate_results(
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/__init__.py", line 578, in _aggregate_results
perfdata, results = _consume_and_dispatch_result_types(subresults)
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/__init__.py", line 622, in _consume_and_dispatch_result_types
for subr in subresults:
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/_cluster_modes.py", line 145, in _cluster_check
yield from summarizer.secondary_results(
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/_cluster_modes.py", line 225, in secondary_results
yield from (
File "/omd/sites/zdv/lib/python3/cmk/base/agent_based/checking/_cluster_modes.py", line 226, in <genexpr>
Result(
File "/omd/sites/zdv/lib/python3/cmk/base/api/agent_based/checking_classes.py", line 367, in __new__
state, summary, details = _create_result_fields(**kwargs)
File "/omd/sites/zdv/lib/python3/cmk/base/api/agent_based/checking_classes.py", line 405, in _create_result_fields
raise ValueError(f"'{name}' must be non-empty str or None, got {var}")
ValueError: 'notice' must be non-empty str or None, got
(Ignore the crash at the end - that’s a different problem: Cluster Checks crashing (Apache and MRPE) - #5 by schlarbm)
Thanks,
Moritz