OMD 'my_site' status unknown

**CMK version: 2.4.0p2
**OS version: Debian 12

**Error message: OMD ‘my_site’ status: unknown → Summary: Item not found in monitoring data

This morning I find the “OMD ‘my_site’ status” in the Unknown state concerning the self monitoring of my Checkmk server.
I don’t know if it’s really important since everything else seems to work normally, but on Friday when I left my office it was not like that.
Please note that I recently set up a crontab task running on the server that updates/upgrades and reboots it every nigh. So I suspect a very recent Debian update, but I don’t know what exactly. If it is the case, I’m probably not the only one with this issue.

If I run a “omd status” command, everything seems fine:

agent-receiver:     running
mkeventd:           running
rrdcached:          running
redis:              running
npcd:               running
automation-helper:  running
ui-job-scheduler:   running
nagios:             running
apache:             running
crontab:            running
---------------------------
Overall state:      running

So it’s really not a big issue as everything else works perfectly but I find it weird because it’s the first time I see that.

The reason can be an older agent on the monitoring server itself.
Is the agent version there also 2.4.0p2?

Hello, thanks for your answer. But yes, the agent is up to date (2.4.0p2).

Same issue in another thread for me. Just never found a solution for it. Mine runs but stays in a stopped state according to checkmk for hours and hours.

On possible solution - as it is a cached output, please have a look at the “/var/lib/check_mk_agent/cache/” folder.
There should be only files with a current date.
In my case it looks like this.

-rw-r--r-- 1 root root 164 May 26 15:33 omd_status.cache
-rw-r--r-- 1 root root 372 May 26 13:07 plugins_cmk-update-agent.cache
-rw-r--r-- 1 root root  53 May 26 06:58 plugins_mk_apt.cache

All older files can be removed. If the systems need them it creates it new :slight_smile:

1 Like

I have 2 files with the current date in this folder.

-rw-r--r-- 1 root root  156 26 mai   04:33 omd_status.cache
-rw-r--r-- 1 root root   24 26 mai   04:35 omd_status.cache.new.1360148

The new file should be gone after some seconds.
The content of the file then should look like.

<<<omd_status>>>
[SITENAME]
agent-receiver 0
mkeventd 0
mknotifyd 0
rrdcached 0
redis 0
automation-helper 0
ui-job-scheduler 0
cmc 0
apache 0
dcd 0
crontab 0
OVERALL 0

My thread is here though the file stamp(s) were always within a minute or two. I could always delete them for the heck of it.
I believe mine had something to do with the time zone on it for one server I had an issue with, since it was seemingly always an 8-hour problem. I changed the time zone on the server since I forgot…and this addressed the issue on one server, but not another.
The other server had its time zone redone on the OS itself and has been rebooted and gets rebooted on a daily basis, but still the same issue.
Maybe one day I’ll figure it out. :smiley:

This is what’s inside the omd_status.cache file:


<<<omd_status>>>
[SITENAME]
agent-receiver 0
mkeventd 0
rrdcached 0
redis 0
npcd 0
automation-helper 0
ui-job-scheduler 0
nagios 0
apache 0
crontab 0
OVERALL 0

It’s not exactly the same as yours, could it be the reason?

@andreas-doehler I’m not sure my issue is the same as the one you got as the status for OMD is not “stopped” in my case but “Item not found in monitoring data” with an “unknown” status.

Everything has been working flawlessly for several years, I don’t understand why it changed during this weekend… :thinking:

The only difference i see is, you use the RAW edition and in my case it’s is from a Cloud/Enterprise edition.
But if it says item not found then it has a problem with the “[SITENAME]”.
I would do a “cmk --debug -vvn CMKHOST” on the command line where “CMKHOST” is the name of your monitoring host. It is possible that you get an error message beside the “item not found”.

It can not be the check itself as this check was last time touched last year. If it was working with 2.4.0 or p1, i have no clue what is the reason.

Hi! Yes it has been working fine until now. It was working with version 2.4.0 (I didn’t install p1) and it was working with 2.3.0p*. The issue showed up during the weekend while I was not at the office. The last things I did on the server last week was installing 2.4.0p2 and setting up a crontab task to perform this short task every night:

apt update
apt full-upgrade
apt autoremove
/sbin/reboot

I just did a “cmk --debug -vvn CMKHOST”, this is the result (I just replaced my site name by [my_site], my MAC address by [my_MAC_address] and my IP addresses for confidentiality reasons):

Summary

value store: loading from disk
Checkmk version 2.4.0p2

  • FETCHING DATA
    Source: SourceInfo(hostname=‘srvcheckmk’, ipaddress=‘[checkmk_IP]’, ident=‘agent’, fetcher_type=<FetcherType.TCP: 8>, source_type=<SourceType.HOST: 1>)
    [cpu_tracking] Start [7f68b51b5520]
    Read from cache: AgentFileCache(path_template=/omd/sites/[my_site]/tmp/check_mk/cache/srvcheckmk, max_age=MaxAge(checking=0, discovery=90.0, inventory=90.0), simulation=False, use_only_cache=False, file_cache_mode=6)
    Not using cache (Too old. Age is 1 sec, allowed is 0 sec)
    Connecting via TCP to [checkmk_IP]:6556 (5.0s timeout)
    Detected transport protocol: TransportProtocol.TLS
    Reading data from agent via TLS socket
    Reading data from agent
    Detected transport protocol: TransportProtocol.PLAIN
    Closing TCP connection to [checkmk_IP]:6556
    Write data to cache file /omd/sites/[my_site]/tmp/check_mk/cache/srvcheckmk
    Trying to acquire lock on /omd/sites/[my_site]/tmp/check_mk/cache/srvcheckmk
    Got lock on /omd/sites/[my_site]/tmp/check_mk/cache/srvcheckmk
    Releasing lock on /omd/sites/[my_site]/tmp/check_mk/cache/srvcheckmk
    Released lock on /omd/sites/[my_site]/tmp/check_mk/cache/srvcheckmk
    [cpu_tracking] Stop [7f68b51b5520 - Snapshot(process=posix.times_result(user=0.009999999999999787, system=0.0, children_user=0.0, children_system=0.0, elapsed=1.8599999994039536))]
    Source: SourceInfo(hostname=‘srvcheckmk’, ipaddress=‘[checkmk_IP]’, ident=‘piggyback’, fetcher_type=<FetcherType.PIGGYBACK: 4>, source_type=<SourceType.HOST: 1>)
    [cpu_tracking] Start [7f68b51b5520]
    Read from cache: NoCache(path_template=/dev/null, max_age=MaxAge(checking=0.0, discovery=0.0, inventory=0.0), simulation=False, use_only_cache=False, file_cache_mode=1)
    0 piggyback files for ‘srvcheckmk’.
    0 piggyback files for ‘[checkmk_IP]’.
    Get piggybacked data
    [cpu_tracking] Stop [7f68b51b5520 - Snapshot(process=posix.times_result(user=0.010000000000000231, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
    [cpu_tracking] Start [7f68b570c9e0]
  • PARSE FETCHER RESULTS
    <<<check_mk>>> / Transition NOOPParser → HostSectionParser
    <<<cmk_agent_ctl_status:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<checkmk_agent_plugins_lnx:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<labels:sep(0)>> / Transition HostSectionParser → HostSectionParser
    <<<df_v2>>> / Transition HostSectionParser → HostSectionParser
    <<<df_v2>>> / Transition HostSectionParser → HostSectionParser
    <<<systemd_units>>> / Transition HostSectionParser → HostSectionParser
    <<<nfsmounts_v2:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<ps_lnx>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<lnx_if>>> / Transition HostSectionParser → HostSectionParser
    <<<lnx_if:sep(58)>>> / Transition HostSectionParser → HostSectionParser
    <<<tcp_conn_stats>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<vbox_guest>>> / Transition HostSectionParser → HostSectionParser
    <<<livestatus_status:sep(59)>>> / Transition HostSectionParser → HostSectionParser
    <<<livestatus_ssl_certs:sep(124)>>> / Transition HostSectionParser → HostSectionParser
    <<<mkeventd_status:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<cmk_site_statistics:sep(59)>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<lnx_thermal:sep(124)>>> / Transition HostSectionParser → HostSectionParser
    <<<omd_broker_status:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<omd_broker_queues:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<omd_broker_shovels:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<omd_apache:sep(124)>>> / Transition HostSectionParser → HostSectionParser
    <<<omd_diskusage:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<omd_info:sep(59)>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<timesyncd_ntpmessage:sep(10)>>> / Transition HostSectionParser → HostSectionParser
    <<local:sep(0)>> / Transition HostSectionParser → HostSectionParser
    HostKey(hostname=‘srvcheckmk’, source_type=<SourceType.HOST: 1>) → Add sections: [‘check_mk’, ‘checkmk_agent_plugins_lnx’, ‘cifsmounts’, ‘cmk_agent_ctl_status’, ‘cmk_site_statistics’, ‘cpu’, ‘df_v2’, ‘diskstat’, ‘job’, ‘kernel’, ‘labels’, ‘livestatus_ssl_certs’, ‘livestatus_status’, ‘lnx_if’, ‘lnx_thermal’, ‘local’, ‘md’, ‘mem’, ‘mkbackup’, ‘mkeventd_status’, ‘mounts’, ‘nfsmounts_v2’, ‘omd_apache’, ‘omd_broker_queues’, ‘omd_broker_shovels’, ‘omd_broker_status’, ‘omd_diskusage’, ‘omd_info’, ‘ps_lnx’, ‘systemd_units’, ‘tcp_conn_stats’, ‘timesyncd’, ‘timesyncd_ntpmessage’, ‘uptime’, ‘vbox_guest’]
    HostKey(hostname=‘srvcheckmk’, source_type=<SourceType.HOST: 1>) → Add sections:
    Received no piggyback data
    CPU load 15 min load: 1.22, 15 min load per core: 0.30 (4 cores)
    CPU utilization Total CPU: 48.89%
    Check_MK Agent Version: 2.4.0p2, OS: linux, Agent plug-ins: 0, Local checks: 0
    Disk IO SUMMARY Read: 15.0 kB/s, Write: 789 kB/s, Latency: 41 microseconds
    Filesystem / Used: 17.48% - 95.6 GiB of 547 GiB, trend per 1 day 0 hours: -16.2 MiB, trend per 1 day 0 hours: -0.00%
    Filesystem /opt/omd/sites/[my_site]/tmp Used: 0.05% - 3.79 MiB of 7.78 GiB, trend per 1 day 0 hours: -45.8 KiB, trend per 1 day 0 hours: -0.00%
    Interface 2 [eno1], (up), MAC: [my_MAC_address], Speed: 1 GBit/s, In: 24.2 kB/s (0.02%), Out: 7.69 kB/s (<0.01%)
    Kernel Performance Process Creations: 83.00/s, Context Switches: 2683.00/s, Major Page Faults: 3.67/s, Page Swap in: 0.00/s, Page Swap Out: 0.00/s
    Memory Total virtual memory: 9.96% - 1.65 GiB of 16.5 GiB, 9 additional details available
    Mount options of / Mount options exactly as expected
    Number of threads 219, Usage: 0.17%
    OMD [my_site] Event Console Current events: 0, Virtual memory: 206 MiB, Overall event limit inactive, No hosts event limit active, No rules event limit active, Received messages: 0.00/s, Rule hits: 0.00/s, Rule tries: 0.00/s, Message drops: 0.00/s, Created events: 0.00/s, Client connects: 0.33/s, Rule hit ratio: -, Processing time per message: -, Time per client request: 0.42 ms
    OMD [my_site] apache No activity since last check
    OMD [my_site] backup mybackup Backup completed, it was running for 22 seconds from 2025-05-07 13:57:15 till 2025-05-07 13:57:37, Size: 48.5 MiB
    OMD [my_site] disk usage Total: 2.72 GiB, Agents: 0 B, Core: 1.27 MiB, History: 28.0 KiB, Inventory: 435 KiB, Local: 312 KiB, Logs: 9.29 MiB, PNP4Nagios: 2.11 GiB, RRDs: 0 B, Tmp: 3.59 MiB
    OMD [my_site] performance Livestatus version: 2.4.0p2, Host checks: 0.5/s, Service checks: 13.1/s
    OMD [my_site] status Item not found in monitoring data
    Process [my_site] agent receiver Processes: 2, Virtual memory: 116 MiB, Resident memory: 84.3 MiB, CPU: 0%, Running for: 5 hours 3 minutes
    Process [my_site] apache Processes: 5, Virtual memory: 857 MiB, Resident memory: 634 MiB, CPU: 0%, Youngest running for: 17 minutes 37 seconds, Oldest running for: 5 hours 3 minutes
    Process [my_site] automation helpers Processes: 3, Virtual memory: 2.74 GiB, Resident memory: 341 MiB, CPU: 0%, Running for: 5 hours 3 minutes
    Process [my_site] event console Processes: 1, Virtual memory: 206 MiB, Resident memory: 40.0 MiB, CPU: 0%, Running for: 5 hours 3 minutes
    Process [my_site] redis-server Processes: 1, Virtual memory: 65.4 MiB, Resident memory: 15.5 MiB, CPU: 0%, Running for: 5 hours 3 minutes
    Process [my_site] rrdcached Processes: 1, Virtual memory: 564 MiB, Resident memory: 36.8 MiB, CPU: 0%, Running for: 5 hours 3 minutes
    Site [my_site] statistics Total hosts: 29, Problem hosts: 1, Total services: 922, Problem services: 142
    Systemd Service Summary Total: 149, Disabled: 2, Failed: 0
    Systemd Socket Summary Total: 12, Disabled: 1, Failed: 0
    Systemd Timesyncd Time Offset: 270 microseconds, Time since last sync: 30 minutes 20 seconds, Time since last NTPMessage: 30 minutes 20 seconds, Stratum: 4.00, Jitter: 16 milliseconds, Synchronized on [DC_IP]
    TCP Connections Established: 5
    Temperature Zone 0 Temperature: 50.0 °C
    Uptime Up since 2025-05-27 04:38:37, Uptime: 5 hours 3 minutes
  • EXECUTING INVENTORY PLUGINS
    cisco_meraki_power_supplies: skipped (no data)
    cisco_meraki_power_supplies: skipped (no data)
    cisco_meraki_org_device_info: skipped (no data)
    cisco_meraki_org_device_info: skipped (no data)
    fritz: skipped (no data)
    fritz: skipped (no data)
    oracle_performance: skipped (no data)
    oracle_performance: skipped (no data)
    oracle_dataguard_stats: skipped (no data)
    oracle_dataguard_stats: skipped (no data)
    oracle_tablespaces: skipped (no data)
    oracle_tablespaces: skipped (no data)
    oracle_instance: skipped (no data)
    oracle_instance: skipped (no data)
    oracle_recovery_area: skipped (no data)
    oracle_recovery_area: skipped (no data)
    esx_systeminfo: skipped (no data)
    esx_systeminfo: skipped (no data)
    mssql_instance: skipped (no data)
    mssql_instance: skipped (no data)
    redfish_firmware: skipped (no data)
    redfish_firmware: skipped (no data)
    redfish_firmware_hpe_ilo4: skipped (no data)
    redfish_firmware_hpe_ilo4: skipped (no data)
    ibm_mq_queues: skipped (no data)
    ibm_mq_queues: skipped (no data)
    win_reg_uninstall: skipped (no data)
    win_reg_uninstall: skipped (no data)
    winperf_if: skipped (no data)
    winperf_if: skipped (no data)
    lparstat_aix: skipped (no data)
    lparstat_aix: skipped (no data)
    lnx_if: ok
    lnx_if: skipped (no data)
    win_computersystem: skipped (no data)
    win_computersystem: skipped (no data)
    infoblox_systeminfo: skipped (no data)
    infoblox_systeminfo: skipped (no data)
    aix_baselevel: skipped (no data)
    aix_baselevel: skipped (no data)
    inventory_esx_vsphere_virtual_machines: skipped (no data)
    inventory_esx_vsphere_virtual_machines: skipped (no data)
    inventory_docker_container_node_name: skipped (no data)
    inventory_docker_container_node_name: skipped (no data)
    perle_psmu: skipped (no data)
    perle_psmu: skipped (no data)
    suseconnect: skipped (no data)
    suseconnect: skipped (no data)
    citrix_state: skipped (no data)
    citrix_state: skipped (no data)
    solaris_pkginfo: skipped (no data)
    solaris_pkginfo: skipped (no data)
    dmidecode: skipped (no data)
    dmidecode: skipped (no data)
    win_video: skipped (no data)
    win_video: skipped (no data)
    inventory_checkmk: ok
    inventory_checkmk: skipped (no data)
    win_exefiles: skipped (no data)
    win_exefiles: skipped (no data)
    snmp_extended_info: skipped (no data)
    snmp_extended_info: skipped (no data)
    win_ip_r: skipped (no data)
    win_ip_r: skipped (no data)
    kube_statefulset: skipped (no data)
    kube_statefulset: skipped (no data)
    aix_lparstat_inventory: skipped (no data)
    aix_lparstat_inventory: skipped (no data)
    checkmk_agent_plugins: ok
    checkmk_agent_plugins: skipped (no data)
    kube_daemonset: skipped (no data)
    kube_daemonset: skipped (no data)
    lnx_sysctl: skipped (no data)
    lnx_sysctl: skipped (no data)
    couchbase_nodes_ports: skipped (no data)
    couchbase_nodes_ports: skipped (no data)
    win_wmi_software: skipped (no data)
    win_wmi_software: skipped (no data)
    ibm_mq_managers: skipped (no data)
    ibm_mq_managers: skipped (no data)
    inventory_esx_vsphere_clusters: skipped (no data)
    inventory_esx_vsphere_clusters: skipped (no data)
    mssql_clusters: skipped (no data)
    mssql_clusters: skipped (no data)
    kube_namespace: skipped (no data)
    kube_namespace: skipped (no data)
    fortisandbox_software: skipped (no data)
    fortisandbox_software: skipped (no data)
    dell_hw_info: skipped (no data)
    dell_hw_info: skipped (no data)
    lnx_block_devices: skipped (no data)
    lnx_block_devices: skipped (no data)
    lnx_distro: skipped (no data)
    lnx_distro: skipped (no data)
    win_wmi_updates: skipped (no data)
    win_wmi_updates: skipped (no data)
    check_mk: ok
    check_mk: skipped (no data)
    juniper_info: skipped (no data)
    juniper_info: skipped (no data)
    ibm_mq_channels: skipped (no data)
    ibm_mq_channels: skipped (no data)
    kube_cluster: skipped (no data)
    kube_cluster: skipped (no data)
    citrix_controller: skipped (no data)
    citrix_controller: skipped (no data)
    hp_proliant_da_phydrv: skipped (no data)
    hp_proliant_da_phydrv: skipped (no data)
    solaris_prtdiag: skipped (no data)
    solaris_prtdiag: skipped (no data)
    win_os: skipped (no data)
    win_os: skipped (no data)
    win_system: skipped (no data)
    win_system: skipped (no data)
    win_computersystemproduct: skipped (no data)
    win_computersystemproduct: skipped (no data)
    docker_node_images: skipped (no data)
    docker_node_images: skipped (no data)
    snmp_quantum_storage_info: skipped (no data)
    snmp_quantum_storage_info: skipped (no data)
    lnx_ip_r: skipped (no data)
    lnx_ip_r: skipped (no data)
    solaris_uname: skipped (no data)
    solaris_uname: skipped (no data)
    win_networkadapter: skipped (no data)
    win_networkadapter: skipped (no data)
    inv_esx_vsphere_hostsystem: skipped (no data)
    inv_esx_vsphere_hostsystem: skipped (no data)
    solaris_addresses: skipped (no data)
    solaris_addresses: skipped (no data)
    docker_container_labels: skipped (no data)
    docker_container_labels: skipped (no data)
    solaris_routes: skipped (no data)
    solaris_routes: skipped (no data)
    fortinet_firewall_network_interfaces: skipped (no data)
    fortinet_firewall_network_interfaces: skipped (no data)
    hp_proliant_mem: skipped (no data)
    hp_proliant_mem: skipped (no data)
    lnx_video: skipped (no data)
    lnx_video: skipped (no data)
    mem: ok
    mem: skipped (no data)
    docker_container_network: skipped (no data)
    docker_container_network: skipped (no data)
    fireeye_sys_status: skipped (no data)
    fireeye_sys_status: skipped (no data)
    allnet_ip_sensoric: skipped (no data)
    allnet_ip_sensoric: skipped (no data)
    azure_load_balancer: skipped (no data)
    azure_load_balancer: skipped (no data)
    docker_node_network: skipped (no data)
    docker_node_network: skipped (no data)
    fortigate_ha: skipped (no data)
    fortigate_ha: skipped (no data)
    fortimail_system: skipped (no data)
    fortimail_system: skipped (no data)
    aix_service_packs: skipped (no data)
    aix_service_packs: skipped (no data)
    mem_used: skipped (no data)
    mem_used: skipped (no data)
    oracle_systemparameter: skipped (no data)
    oracle_systemparameter: skipped (no data)
    kyocera_printer: skipped (no data)
    kyocera_printer: skipped (no data)
    aix_packages: skipped (no data)
    aix_packages: skipped (no data)
    aruba_wlc_aps: skipped (no data)
    aruba_wlc_aps: skipped (no data)
    fortiauthenticator_system: skipped (no data)
    fortiauthenticator_system: skipped (no data)
    kube_node: skipped (no data)
    kube_node: skipped (no data)
    mobileiron_inventory: skipped (no data)
    mobileiron_inventory: skipped (no data)
    win_disks: skipped (no data)
    win_disks: skipped (no data)
    lnx_cpuinfo: skipped (no data)
    lnx_cpuinfo: skipped (no data)
    checkpoint_vpn_tunnels: skipped (no data)
    checkpoint_vpn_tunnels: skipped (no data)
    win_cpuinfo: skipped (no data)
    win_cpuinfo: skipped (no data)
    enviromux_micro_information: skipped (no data)
    enviromux_micro_information: skipped (no data)
    win_bios: skipped (no data)
    win_bios: skipped (no data)
    docker_node_info: skipped (no data)
    docker_node_info: skipped (no data)
    kube_pod: skipped (no data)
    kube_pod: skipped (no data)
    lnx_uname: skipped (no data)
    lnx_uname: skipped (no data)
    hp_proliant_systeminfo: skipped (no data)
    hp_proliant_systeminfo: skipped (no data)
    inv_if: ok
    inv_if: skipped (no data)
    infoblox_osinfo: skipped (no data)
    infoblox_osinfo: skipped (no data)
    perle_chassis_slots: skipped (no data)
    perle_chassis_slots: skipped (no data)
    snmp_info: skipped (no data)
    snmp_info: skipped (no data)
    lnx_packages: skipped (no data)
    lnx_packages: skipped (no data)
    inv_cisco_vlans: skipped (no data)
    inv_cisco_vlans: skipped (no data)
    primekey: skipped (no data)
    primekey: skipped (no data)
    solaris_cpus: skipped (no data)
    solaris_cpus: skipped (no data)
    kube_deployment: skipped (no data)
    kube_deployment: skipped (no data)
    perle_chassis: skipped (no data)
    perle_chassis: skipped (no data)
    prtconf: skipped (no data)
    prtconf: skipped (no data)
    fortisandbox_system: skipped (no data)
    fortisandbox_system: skipped (no data)
    ipmi_firmware: skipped (no data)
    ipmi_firmware: skipped (no data)
    kube_cronjob: skipped (no data)
    kube_cronjob: skipped (no data)
    netapp_ontap_info: skipped (no data)
    netapp_ontap_info: skipped (no data)
    netapp_ontap_disk: skipped (no data)
    netapp_ontap_disk: skipped (no data)
    Trying to acquire lock on /omd/sites/[my_site]/tmp/check_mk/status_data/srvcheckmk
    Got lock on /omd/sites/[my_site]/tmp/check_mk/status_data/srvcheckmk
    Releasing lock on /omd/sites/[my_site]/tmp/check_mk/status_data/srvcheckmk
    Released lock on /omd/sites/[my_site]/tmp/check_mk/status_data/srvcheckmk
    0 piggyback files for ‘srvcheckmk’.
    [cpu_tracking] Stop [7f68b570c9e0 - Snapshot(process=posix.times_result(user=0.16000000000000014, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.1600000001490116))]
    [agent] Success, [piggyback] Success (but no data found for this host), execution time 2.0 sec | execution_time=2.020 user_time=0.180 system_time=0.000 children_user_time=0.000 children_system_time=0.000 cmk_time_agent=1.840

I don’t understand much about it, but if you can deduce things from it…

Is yours related to time at all? Once I fixed my time issues, mine has been solid. I would only see the problems when the server was initially rebooted. Then I’d see the problem for 8 hours (the time difference) and then it would resolve. Just an idea.

Hello @bbinder and thank you for your answer, but no, I don’t think time is the problem here. The “date” command on the system gives me the correct date and time, and the “Systemd Timesyncd Time” check is in the green on the interface.

Ok, my time was correct too when I would run those commands. It would just shift during the boot process, but then shift back seconds later. It’s why it made it difficult for me to track down. You would probably see an error like mine though, where the time check is in the future, so if you don’t see that, then it’s likely not the case.