Systemd-timesyncd check: "Cannot reasonably calculate time since last synchronization (hosts time is running ahead)"

qupfer · August 14, 2024, 5:00pm

CMK version:
This is Check_MK version 2.3.0p12 CRE

OS version:
“Ubuntu 22.04.4 LTS” (server and client)

Error message:
Cannot reasonably calculate time since last synchronization (hosts time is running ahead)
for the systemd-timesyncd check.
Its go away with the next check and come back several time a day.

Any ideas? Run on all machines systemd-timesyncd and set all to the same zone (Europe/Berlin).

Thanks for any help.

Output of “cmk --debug -vvn hostname”:

OMD[monitoring]:~$ cmk --debug -vvn strato2.07q.de
value store: synchronizing
Trying to acquire lock on /omd/sites/monitoring/tmp/check_mk/counters/strato2.07q.de
Got lock on /omd/sites/monitoring/tmp/check_mk/counters/strato2.07q.de
value store: loading from disk
Releasing lock on /omd/sites/monitoring/tmp/check_mk/counters/strato2.07q.de
Released lock on /omd/sites/monitoring/tmp/check_mk/counters/strato2.07q.de
Checkmk version 2.3.0p12
+ FETCHING DATA
  Source: SourceInfo(hostname='strato2.07q.de', ipaddress='85.215.168.155', ident='agent', fetcher_type=<FetcherType.TCP: 8>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7a5aa9e28fb0]
Read from cache: AgentFileCache(strato2.07q.de, path_template=/omd/sites/monitoring/tmp/check_mk/cache/{hostname}, max_age=MaxAge(checking=0, discovery=90.0, inventory=90.0), simulation=False, use_only_cache=False, file_cache_mode=6)
Not using cache (Too old. Age is 20 sec, allowed is 0 sec)
Connecting via TCP to 85.215.168.155:6556 (5.0s timeout)
Detected transport protocol: TransportProtocol.TLS
Reading data from agent via TLS socket
Reading data from agent
Detected transport protocol: TransportProtocol.PLAIN
Closing TCP connection to 85.215.168.155:6556
Write data to cache file /omd/sites/monitoring/tmp/check_mk/cache/strato2.07q.de
Trying to acquire lock on /omd/sites/monitoring/tmp/check_mk/cache/strato2.07q.de
Got lock on /omd/sites/monitoring/tmp/check_mk/cache/strato2.07q.de
Releasing lock on /omd/sites/monitoring/tmp/check_mk/cache/strato2.07q.de
Released lock on /omd/sites/monitoring/tmp/check_mk/cache/strato2.07q.de
[cpu_tracking] Stop [7a5aa9e28fb0 - Snapshot(process=posix.times_result(user=0.010000000000000009, system=0.0, children_user=0.0, children_system=0.0, elapsed=1.2600000016391277))]
  Source: SourceInfo(hostname='strato2.07q.de', ipaddress='85.215.168.155', ident='piggyback', fetcher_type=<FetcherType.PIGGYBACK: 4>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7a5aab195730]
Read from cache: NoCache(strato2.07q.de, path_template=/dev/null, max_age=MaxAge(checking=0.0, discovery=0.0, inventory=0.0), simulation=False, use_only_cache=False, file_cache_mode=1)
No piggyback files for 'strato2.07q.de'. Skip processing.
No piggyback files for '85.215.168.155'. Skip processing.
Get piggybacked data
[cpu_tracking] Stop [7a5aab195730 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
[cpu_tracking] Start [7a5aab25d670]
+ PARSE FETCHER RESULTS
<<<check_mk>>> / Transition NOOPParser -> HostSectionParser
<<<cmk_agent_ctl_status:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<checkmk_agent_plugins_lnx:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<df_v2>>> / Transition HostSectionParser -> HostSectionParser
<<<df_v2>>> / Transition HostSectionParser -> HostSectionParser
<<<systemd_units>>> / Transition HostSectionParser -> HostSectionParser
<<<nfsmounts_v2:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<cifsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<mounts>>> / Transition HostSectionParser -> HostSectionParser
<<<ps_lnx>>> / Transition HostSectionParser -> HostSectionParser
<<<mem>>> / Transition HostSectionParser -> HostSectionParser
<<<cpu>>> / Transition HostSectionParser -> HostSectionParser
<<<uptime>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if:sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<tcp_conn_stats>>> / Transition HostSectionParser -> HostSectionParser
<<<multipath>>> / Transition HostSectionParser -> HostSectionParser
<<<diskstat>>> / Transition HostSectionParser -> HostSectionParser
<<<kernel>>> / Transition HostSectionParser -> HostSectionParser
<<<md>>> / Transition HostSectionParser -> HostSectionParser
<<<vbox_guest>>> / Transition HostSectionParser -> HostSectionParser
<<<job>>> / Transition HostSectionParser -> HostSectionParser
<<<timesyncd>>> / Transition HostSectionParser -> HostSectionParser
<<<timesyncd_ntpmessage:sep(10)>>> / Transition HostSectionParser -> HostSectionParser
<<<local:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
  HostKey(hostname='strato2.07q.de', source_type=<SourceType.HOST: 1>)  -> Add sections: ['check_mk', 'checkmk_agent_plugins_lnx', 'cifsmounts', 'cmk_agent_ctl_status', 'cpu', 'df_v2', 'diskstat', 'job', 'kernel', 'labels', 'lnx_if', 'local', 'md', 'mem', 'mounts', 'multipath', 'nfsmounts_v2', 'ps_lnx', 'systemd_units', 'tcp_conn_stats', 'timesyncd', 'timesyncd_ntpmessage', 'uptime', 'vbox_guest']
  HostKey(hostname='strato2.07q.de', source_type=<SourceType.HOST: 1>)  -> Add sections: []
Received no piggyback data
CPU load             15 min load: 0.08, 15 min load per core: 0.02 (4 cores)
CPU utilization      Total CPU: 1.79%
Check_MK Agent       Version: 2.3.0p12, OS: linux, Agent plug-ins: 0, Local checks: 0
Disk IO SUMMARY      Read: 0.00 B/s, Write: 62.9 kB/s, Latency: 38 microseconds
Filesystem /         Used: 11.13% - 4.20 GiB of 37.8 GiB, trend per 1 day 0 hours: +13.5 MiB, trend per 1 day 0 hours: +0.04%, Time left until disk full: 6 years 346 days
Filesystem /boot     Used: 19.71% - 173 MiB of 880 MiB, trend per 1 day 0 hours: +1.56 KiB, trend per 1 day 0 hours: +<0.01%, Time left until disk full: 1273 years 11 days
Filesystem /boot/efi Used: 5.85% - 6.10 MiB of 104 MiB, trend per 1 day 0 hours: +0 B, trend per 1 day 0 hours: +0%
Filesystem btrfs /dev/vda2 Used: 1.00% - 2.60 GiB of 260 GiB, trend per 1 day 0 hours: +5.29 MiB, trend per 1 day 0 hours: +<0.01%, Time left until disk full: 136 years 182 days
Interface 2          [ens6], (up), MAC: 02:01:65:40:BA:4D, Speed: unknown, In: 1.44 kB/s, Out: 2.89 kB/s
Interface 3          [lxdbr0], (up), MAC: 00:16:3E:F4:7F:2B, Speed: 10 GBit/s, In: 0.00 B/s (0%), Out: 0.00 B/s (0%)
Kernel Performance   Process Creations: 6.73/s, Context Switches: 338.45/s, Major Page Faults: 0.00/s, Page Swap in: 0.00/s, Page Swap Out: 0.00/s
Memory               Total virtual memory: 8.40% - 663 MiB of 7.71 GiB, 8 additional details available
Mount options of /   Mount options exactly as expected
Mount options of /boot Mount options exactly as expected
Mount options of /boot/efi Mount options exactly as expected
Mount options of /mnt/btrfs Mount options exactly as expected
Number of threads    284, Usage: 0.45%
Systemd Service Summary Total: 176, Disabled: 5, Failed: 0
Systemd Socket Summary Total: 26, Disabled: 3, Failed: 0
Systemd Timesyncd Time Offset: 3 milliseconds, Cannot reasonably calculate time since last synchronization (hosts time is running ahead)(!!), Cannot reasonably calculate time since last synchronization (hosts time is running ahead)(!!), Stratum: 1.00, Jitter: 3 milliseconds, Synchronized on 2001:638:610:be01::108
TCP Connections      Established: 5
Uptime               Up since 2024-08-14 10:47:13, Uptime: 7 hours 57 minutes
No piggyback files for 'strato2.07q.de'. Skip processing.
No piggyback files for '85.215.168.155'. Skip processing.
[cpu_tracking] Stop [7a5aab25d670 - Snapshot(process=posix.times_result(user=0.030000000000000027, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.029999999329447746))]
[agent] Success, [piggyback] Success (but no data found for this host), execution time 1.3 sec | execution_time=1.290 user_time=0.040 system_time=0.000 children_user_time=0.000 children_system_time=0.000 cmk_time_agent=1.250
OMD[monitoring]:~$

r.sander · August 14, 2024, 5:15pm

Replace systemd-timesyncd with chronyd or ntpd.

robin.gierse · August 16, 2024, 11:03am

Is the monitored system actually in sync with the time on the Checkmk server?

j.demmerle · October 2, 2024, 10:35am

We currently experience the same problem.
The instance is running on CEE 2.3.0p16 and all servers, including the checkmk server, are using the first PTB timeserver.
Currently it looks like the problem only exists on servers running Debian 12. On servers running Debian 11 everything looks fine.
What is interesting is that the checkmk instance is also running on Debian 12 and this is the only server where this problem does not exist.

r.sander · October 2, 2024, 11:18am

systemd-timesyncd may be OK on desktop systems but I would not trust it to sync time on servers.

robin.gierse · October 31, 2024, 7:46am

What is your reasoning for this position?

Edit: I accidentally edited this post instead of replying, so the above might be slightly different from the original question.

r.sander · October 31, 2024, 8:47am

It may just be a gut feeling but I have more trust in ntpd and chronyd.

systemd-timesyncd only implements SNTP as the man page tells.

robin.gierse · November 12, 2024, 8:08am

Fair points. I will keep using it on clients, but on servers I am still on good ol’ NTPD.
I was just curious if there is more to it than preference and gut feeling.

Thanks!

system · November 12, 2025, 8:09am

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.