Hi,
this issue has been long struggle with checkmk agent on Windows Server 2022 with WSUS, DFS and PRINTER roles only. Other 60 Windows agents with various versions of Windows OS and pre / on 2.3.0.p7 versions of agents are functioning without problems.
I have 2 machines (this is one of them) where agent’s WinPerf, Physical disk and CPU utilization services fell out out of CheckMK with error message below. Restarting agent does not help, Re-installing agent is impossible due to corruption errors, nothing helps except one thing - reboot of operating system. After that, everything is normal for about a week, same thing all over again. Event Viewer, except Event ID 35, is not showing anything pertaining to OS nor checkmk service.
I had reinstalled OS from scratch, only for the issue to re-appear. I am suspecting it may have something to do with .NET or one of the roles or its components at play, impacting state of Performance Monitor OS facility, but I am unable to confirm it via manual query of it using the console it provides. Performance Monitor is functioning without issues. This issue has been there even on version 2.2.
Can someone please explain or shed some light on relationship of Performance Monitor and checkmk agent? Why only physical disk and CPU utilization, but not Memory or other metrics are impacted?
Why does OS reboot help the issue for a week, only to re-appear in the same manner?
any information is appreciated!
thanks
The description for Event ID 35 from source checkmk cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
checkmk
checkmk service uses log path 'C:\ProgramData\checkmk\agent\log'
The message resource is present but the message was not found in the message table
CMK version:
Server:
OMD - Open Monitoring Distribution Version 2.3.0p7.cre
OS version:
NAME=“Rocky Linux”
VERSION=“9.4 (Blue Onyx)”
ID=“rocky”
ID_LIKE=“rhel centos fedora”
VERSION_ID=“9.4”
PLATFORM_ID=“platform:el9”
PRETTY_NAME=“Rocky Linux 9.4 (Blue Onyx)”
ANSI_COLOR=“0;32”
LOGO=“fedora-logo-icon”
CPE_NAME=“cpe:/o:rocky:rocky:9::baseos”
HOME_URL=“https://rockylinux.org/”
BUG_REPORT_URL=“https://bugs.rockylinux.org/”
SUPPORT_END=“2032-05-31”
ROCKY_SUPPORT_PRODUCT=“Rocky-Linux-9”
ROCKY_SUPPORT_PRODUCT_VERSION=“9.4”
REDHAT_SUPPORT_PRODUCT=“Rocky Linux”
REDHAT_SUPPORT_PRODUCT_VERSION=“9.4”
Agent 2.2p26:
on
PS C:\Windows\system32> Get-WindowsFeature | Where-Object {$_. installstate -eq "installed"}
Display Name Name Install State
------------ ---- -------------
[X] File and Storage Services FileAndStorage-Services Installed
[X] File and iSCSI Services File-Services Installed
[X] File Server FS-FileServer Installed
[X] DFS Namespaces FS-DFS-Namespace Installed
[X] Storage Services Storage-Services Installed
[X] Print and Document Services Print-Services Installed
[X] Print Server Print-Server Installed
[X] Web Server (IIS) Web-Server Installed
[X] Web Server Web-WebServer Installed
[X] Common HTTP Features Web-Common-Http Installed
[X] Default Document Web-Default-Doc Installed
[X] Static Content Web-Static-Content Installed
[X] Performance Web-Performance Installed
[X] Dynamic Content Compression Web-Dyn-Compression Installed
[X] Security Web-Security Installed
[X] Request Filtering Web-Filtering Installed
[X] Windows Authentication Web-Windows-Auth Installed
[X] Application Development Web-App-Dev Installed
[X] .NET Extensibility 4.8 Web-Net-Ext45 Installed
[X] ASP.NET 4.8 Web-Asp-Net45 Installed
[X] ISAPI Extensions Web-ISAPI-Ext Installed
[X] ISAPI Filters Web-ISAPI-Filter Installed
[X] Management Tools Web-Mgmt-Tools Installed
[X] IIS Management Console Web-Mgmt-Console Installed
[X] IIS 6 Management Compatibility Web-Mgmt-Compat Installed
[X] IIS 6 Metabase Compatibility Web-Metabase Installed
[X] Windows Server Update Services UpdateServices Installed
[X] WID Connectivity UpdateServices-WidDB Installed
[X] WSUS Services UpdateServices-Services Installed
[X] .NET Framework 4.8 Features NET-Framework-45-Fea... Installed
[X] .NET Framework 4.8 NET-Framework-45-Core Installed
[X] ASP.NET 4.8 NET-Framework-45-ASPNET Installed
[X] WCF Services NET-WCF-Services45 Installed
[X] HTTP Activation NET-WCF-HTTP-Activat... Installed
[X] TCP Port Sharing NET-WCF-TCP-PortShar... Installed
[X] Azure Arc Setup AzureArcSetup Installed
[X] Group Policy Management GPMC Installed
[X] Microsoft Defender Antivirus Windows-Defender Installed
[X] Remote Server Administration Tools RSAT Installed
[X] Role Administration Tools RSAT-Role-Tools Installed
[X] Windows Server Update Services Tools UpdateServices-RSAT Installed
[X] API and PowerShell cmdlets UpdateServices-API Installed
[X] User Interface Management Console UpdateServices-UI Installed
[X] File Services Tools RSAT-File-Services Installed
[X] DFS Management Tools RSAT-DFS-Mgmt-Con Installed
[X] Print and Document Services Tools RSAT-Print-Services Installed
[X] System Data Archiver System-DataArchiver Installed
[X] Windows Internal Database Windows-Internal-Dat... Installed
[X] Windows PowerShell PowerShellRoot Installed
[X] Windows PowerShell 5.1 PowerShell Installed
[X] Windows Process Activation Service WAS Installed
[X] Process Model WAS-Process-Model Installed
[X] Configuration APIs WAS-Config-APIs Installed
[X] WoW64 Support WoW64-Support Installed
[X] XPS Viewer XPS-Viewer Installed
Error message:
[agent] Success, [piggyback] Success (but no data found for this host), Missing monitoring data for plugins, winperf_if WARN , winperf_phydisk WARN , winperf_processor_util WARN , execution time 138.4 sec
Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)
OMD[monitoring]:~$ cmk --debug -vvn cainfmgtapp02
value store: synchronizing
Trying to acquire lock on /omd/sites/monitoring/tmp/check_mk/counters/cainfmgtapp02
Got lock on /omd/sites/monitoring/tmp/check_mk/counters/cainfmgtapp02
value store: loading from disk
Releasing lock on /omd/sites/monitoring/tmp/check_mk/counters/cainfmgtapp02
Released lock on /omd/sites/monitoring/tmp/check_mk/counters/cainfmgtapp02
Checkmk version 2.3.0p7
Updating IPv4 DNS cache for cainfmgtapp02: 10.101.22.5
Trying to acquire lock on /omd/sites/monitoring/var/check_mk/ipaddresses.cache
Got lock on /omd/sites/monitoring/var/check_mk/ipaddresses.cache
Releasing lock on /omd/sites/monitoring/var/check_mk/ipaddresses.cache
Released lock on /omd/sites/monitoring/var/check_mk/ipaddresses.cache
- FETCHING DATA
Source: SourceInfo(hostname=‘cainfmgtapp02’, ipaddress=‘10.101.22.5’, ident=‘piggyback’, fetcher_type=<FetcherType.PIGGYBACK: 4>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7f4a2fa6b080]
Read from cache: NoCache(cainfmgtapp02, path_template=/dev/null, max_age=MaxAge(checking=0.0, discovery=0.0, inventory=0.0), simulation=False, use_only_cache=False, file_cache_mode=1)
Piggyback file ‘/omd/sites/monitoring/tmp/check_mk/piggyback/cainfmgtapp02/cainfprdhpv04’: Successfully processed from source ‘cainfprdhpv04’
No piggyback files for ‘10.101.22.5’. Skip processing.
Get piggybacked data
[cpu_tracking] Stop [7f4a2fa6b080 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
[cpu_tracking] Start [7f4a30282ae0] - PARSE FETCHER RESULTS
<<<hyperv_vmstatus:cached(1720539086,90)>>> / Transition NOOPParser → HostSectionParser
<<<hyperv_checkpoints:cached(1720539086,90)>>> / Transition HostSectionParser → HostSectionParser
<<labels:sep(0)>> / Transition HostSectionParser → HostSectionParser
HostKey(hostname=‘cainfmgtapp02’, source_type=<SourceType.HOST: 1>) → Add sections: [‘hyperv_checkpoints’, ‘hyperv_vmstatus’, ‘labels’]
Received no piggyback data
Piggyback file ‘/omd/sites/monitoring/tmp/check_mk/piggyback/cainfmgtapp02/cainfprdhpv04’: Successfully processed from source ‘cainfprdhpv04’
No piggyback files for ‘10.101.22.5’. Skip processing.
[cpu_tracking] Stop [7f4a30282ae0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
[piggyback] Successfully processed from source ‘cainfprdhpv04’, execution time 0.0 sec | execution_time=0.000 user_time=0.000 system_time=0.000 children_user_time=0.000 children_system_time=0.000 cmk_time_agent=0.000
