CMK version: 2.2.0p[16 works, 32 doesn’t seem to]
OS version: ubuntu 22/24
Error message:
no error message, just doesn’t end up in discovery.
Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)
Checkmk version 2.2.0p32
- FETCHING DATA
Source: SourceInfo(hostname=’ f.q.d.n ', ipaddress='i.p', ident='agent', fetcher_type=<FetcherType.TCP: 8>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7f8cdee086d0]
Read from cache: AgentFileCache(f.q.d.n, path_template=/omd/sites/site/tmp/check_mk/cache/{hostname}, max_age=MaxAge(checking=0, discovery=180.0, inventory=180.0), simulation=False, use_only_cache=False, file_cache_mode=6)
Not using cache (Too old. Age is 110 sec, allowed is 0 sec)
[TCPFetcher] Execute data source
Connecting via TCP to 10.1.31.31:6556 (5.0s timeout)
Detected transport protocol: TransportProtocol.PLAIN (b'<<')
Reading data from agent
Closing TCP connection to 10.1.31.31:6556
Write data to cache file /omd/sites/wopub/tmp/check_mk/cache/f.q.d.n
Trying to acquire lock on /omd/sites/site/tmp/check_mk/cache/f.q.d.n
Got lock on /omd/sites/site/tmp/check_mk/cache/f.q.d.n
Releasing lock on /omd/sites/site/tmp/check_mk/cache/f.q.d.n
Released lock on /omd/sites/site/tmp/check_mk/cache/f.q.d.n
[cpu_tracking] Stop [7f8cdee086d0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=1.2399999983608723))]
Source: SourceInfo(hostname=' f.q.d.n ', ipaddress='i.p', ident='piggyback', fetcher_type=<FetcherType.PIGGYBACK: 4>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7f8cdee086d0]
Read from cache: NoCache(f.q.d.n, path_template=/dev/null, max_age=MaxAge(checking=0.0, discovery=0.0, inventory=0.0), simulation=False, use_only_cache=False, file_cache_mode=1)
[PiggybackFetcher] Execute data source
No piggyback files for ' f.q.d.n '. Skip processing.
No piggyback files for 'i.p'. Skip processing.
[cpu_tracking] Stop [7f8cdee086d0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
+ PARSE FETCHER RESULTS
<<<check_mk>>> / Transition NOOPParser -> HostSectionParser
<<<cmk_agent_ctl_status:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<checkmk_agent_plugins_lnx:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<df_v2>>> / Transition HostSectionParser -> HostSectionParser
<<<df_v2>>> / Transition HostSectionParser -> HostSectionParser
<<<systemd_units>>> / Transition HostSectionParser -> HostSectionParser
<<<nfsmounts_v2:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<cifsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<mounts>>> / Transition HostSectionParser -> HostSectionParser
<<<ps_lnx>>> / Transition HostSectionParser -> HostSectionParser
<<<mem>>> / Transition HostSectionParser -> HostSectionParser
<<<cpu>>> / Transition HostSectionParser -> HostSectionParser
<<<uptime>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if:sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<tcp_conn_stats>>> / Transition HostSectionParser -> HostSectionParser
<<<multipath>>> / Transition HostSectionParser -> HostSectionParser
<<<diskstat>>> / Transition HostSectionParser -> HostSectionParser
<<<kernel>>> / Transition HostSectionParser -> HostSectionParser
<<<md>>> / Transition HostSectionParser -> HostSectionParser
<<<vbox_guest>>> / Transition HostSectionParser -> HostSectionParser
<<<postfix_mailq>>> / Transition HostSectionParser -> HostSectionParser
<<<postfix_mailq_status:sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<job>>> / Transition HostSectionParser -> HostSectionParser
<<<chrony:cached(1736455195,120)>>> / Transition HostSectionParser -> HostSectionParser
<<<local:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<logwatch>>> / Transition HostSectionParser -> HostSectionParser
HostKey(hostname=’f.q.d.n’, source_type=<SourceType.HOST: 1>) -> Add sections: ['check_mk', 'checkmk_agent_plugins_lnx', 'chrony', 'cifsmounts', 'cmk_agent_ctl_status', 'cpu', 'df_v2', 'diskstat', 'job', 'kernel', 'labels', 'lnx_if', 'local', 'logwatch', 'md', 'mem', 'mounts', 'multipath', 'nfsmounts_v2', 'postfix_mailq', 'postfix_mailq_status', 'ps_lnx', 'systemd_units', 'tcp_conn_stats', 'uptime', 'vbox_guest']
HostKey(hostname=' f.q.d.n', source_type=<SourceType.HOST: 1>) -> Add sections: []
Received no piggyback data
[cpu_tracking] Start [7f8cdee2e4d0]
value store: synchronizing
Trying to acquire lock on /omd/sites/site/tmp/check_mk/counters/f.q.d.n
Got lock on /omd/sites/site/tmp/check_mk/counters/f.q.d.n
value store: loading from disk
Releasing lock on /omd/sites/site/tmp/check_mk/counters/f.q.d.n
Released lock on /omd/sites/site/tmp/check_mk/counters/ f.q.d.n
CPU load 15 min load: 0.00, 15 min load per core: 0.00 (2 cores)
CPU utilization Total CPU (5 min average): 1.67%
Check_MK Agent Version: 2.2.0p32, OS: linux, TLS is not activated on monitored host (see details), Agent plugins: 2, Local checks: 0
Disk IO SUMMARY Read: 219 B/s, Write: 14.9 kB/s, Latency: 1 millisecond
Filesystem / Used: 32.94% - 6.43 GiB of 19.5 GiB, trend per 1 day 0 hours: +8.97 MiB, trend per 1 day 0 hours: +0.04%, Time left until disk full: 4 years 34 days
Filesystem /home Used: 6.92% - 67.4 MiB of 973 MiB, trend per 1 day 0 hours: -0 B, trend per 1 day 0 hours: -0.00%
Filesystem /var Used: 18.41% - 1.43 GiB of 7.78 GiB, trend per 1 day 0 hours: +4.11 MiB, trend per 1 day 0 hours: +0.05%, Time left until disk full: 4 years 122 days
Filesystem /var/lib Used: 35.56% - 1.37 GiB of 3.86 GiB, trend per 1 day 0 hours: +971 KiB, trend per 1 day 0 hours: +0.02%, Time left until disk full: 7 years 129 days
Interface 2 [ens160], (up), MAC: 00:50:56:BE:C9:33, Speed: 10 GBit/s, In: 105 B/s (<0.01%), Out: 17.1 B/s (<0.01%)
Interface 3 [ens192], (up), MAC: 00:50:56:BE:8B:B7, Speed: 10 GBit/s, In: 2.49 kB/s (<0.01%), Out: 3.08 kB/s (<0.01%)
Interface 4 [ens224], (up), MAC: 00:50:56:BE:45:10, Speed: 10 GBit/s, In: 275 B/s (<0.01%), Out: 93.9 B/s (<0.01%)
Kernel Performance Process Creations: 2.02/s, Context Switches: 770.47/s, Major Page Faults: 0.01/s, Page Swap in: 0.01/s, Page Swap Out: 0.00/s
Memory Total virtual memory: 20.22% - 1.19 GiB of 5.88 GiB, 9 additional details available
Mount options of / Mount options exactly as expected
Mount options of /home Mount options exactly as expected
Mount options of /var Mount options exactly as expected
Mount options of /var/lib Mount options exactly as expected
NTP Time Offset: 0.0199 ms, Stratum: 2, Time since last sync: 1 minute 24 seconds
Number of threads 419, Usage: 2.89%
Postfix Queue Deferred queue length: 0, Active queue length: 0
Postfix status Status: the Postfix mail system is running, PID: 1886
Systemd Service Summary Total: 165, Disabled: 4, Failed: 1
Systemd Socket Summary Total: 21, Disabled: 0, Failed: 0
Uptime Up since Sep 10 2024 08:35:39, Uptime: 121 days 8 hours
No piggyback files for ' f.q.d.n '. Skip processing.
No piggyback files for 'i.p'. Skip processing.
[cpu_tracking] Stop [7f8cdee2e4d0 - Snapshot(process=posix.times_result(user=0.020000000000000018, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.019999999552965164))]
[agent] Success, [piggyback] Success (but no data found for this host), execution time 1.3 sec | execution_time=1.260 user_time=0.020 system_time=0.000 children_user_time=0.000 children_system_time=0.000 cmk_time_agent=1.240
OMD[site]:~$
if I pull the data directly from the agent, I see what seems like it should be working.
(tail of the data):
<<<chrony:cached(1736455615,120)>>>
Reference ID : D8EF2300 (216.239.35.0)
Stratum : 2
Ref time (UTC) : Thu Jan 09 20:43:17 2025
System time : 0.000006577 seconds fast of NTP time
Last offset : -0.000011220 seconds
RMS offset : 0.000018661 seconds
Frequency : 24.255 ppm slow
Residual freq : -0.000 ppm
Skew : 0.008 ppm
Root delay : 0.002335537 seconds
Root dispersion : 0.000361471 seconds
Update interval : 257.3 seconds
Leap status : Normal
<<<local:sep(0)>>>
<<<logwatch>>>
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736455664-078226075170161185153065004154013048098202106029
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736455223-110200038157226143021058076234241203041168063239
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736455475-204208194050077029227023253030007128177022253046
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736455232-070042004242186151222232184054011122020191072214
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736454990-057212183254117228044203140085026182014122081000
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736454868-225023125067151047237087077146251164070240175230
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736455354-121082074106230176213085175182194139244053103141
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736455111-174134133137221211078153039074037054245149060002
[[[/var/log/x/test.log:missing]]]
[[[/var/tmp/cmkck.log]]]
BATCH: 1736455597-089068060197178075080213214037203214241207056057
I see others having this same issues, but I haven’t yet found someone’s answer to fix it. it seems to work fine on a seperate site/install, I can’t quite fathom how it’s not working on this one.