CMK version: 2.5.0p3
OS version: Debian 13 Checkmk installed in Docker
Error message: Total CPU: 100.00%, total is under high load for: 4 days 0 hours (warn/crit at 10 minutes 0 seconds/15 minutes 0 seconds)CRIT
Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)
value store: loading from disk
Checkmk version 2.5.0p3
- FETCHING DATA
Source: SourceInfo(hostname=‘REDACTED’, ipaddress=‘REDACTED’, ident=‘snmp’, fetcher_type=<FetcherType.SNMP: 7>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7fc911208050]
Read from cache: SNMPFileCache(base_path=/omd/sites/cmk, relative_path_template=tmp/check_mk/data_source_cache/snmp/{mode}/REDACTED, max_age=MaxAge(checking=0, discovery=90.0, inventory=90.0), simulation=False, use_only_cache=False, file_cache_mode=6)
Not using cache (Too old. Age is 16 sec, allowed is 0 sec)
hr_cpu: Fetching data (SNMP walk cache is enabled: Use any locally cached information)
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.25.3.3.1.2’
hr_fs: Fetching data (SNMP walk cache is enabled: Use any locally cached information)
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.25.2.3.1.2’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.25.2.3.1.3’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.25.2.3.1.4’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.25.2.3.1.5’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.25.2.3.1.6’
hr_mem: Fetching data (SNMP walk cache is enabled: Use any locally cached information)
Already fetched OID: .1.3.6.1.2.1.25.2.3.1.2
Already fetched OID: .1.3.6.1.2.1.25.2.3.1.3
Already fetched OID: .1.3.6.1.2.1.25.2.3.1.4
Already fetched OID: .1.3.6.1.2.1.25.2.3.1.5
Already fetched OID: .1.3.6.1.2.1.25.2.3.1.6
if64: Fetching data (SNMP walk cache is enabled: Use any locally cached information)
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.1.3’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.25.1.1’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.1’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.2’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.3’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.5’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.8’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.6’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.7’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.8’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.9’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.13’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.14’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.10’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.11’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.12’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.13’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.19’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.20’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.21’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.18’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.2.2.1.6’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.31.1.1.1.15’
snmp_info: Fetching data (SNMP walk cache is enabled: Use any locally cached information)
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.1.1’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.1.2’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.1.4’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.1.5’
Running ‘snmpbulkwalk -Cr10 -v2c -c REDACTED -m “” -M “” -Cc -OQ -OU -On -Ot REDACTED .1.3.6.1.2.1.1.6’
snmp_uptime: Fetching data (SNMP walk cache is enabled: Use any locally cached information)
Already fetched OID: .1.3.6.1.2.1.1.3
Already fetched OID: .1.3.6.1.2.1.25.1.1
Write data to cache file /omd/sites/cmk/tmp/check_mk/data_source_cache/snmp/checking/REDACTED
[cpu_tracking] Stop [7fc911208050 - Snapshot(process=posix.times_result(user=0.010000000000000009, system=0.010000000000000009, children_user=0.07, children_system=0.11, elapsed=0.24000000208616257))]
Source: SourceInfo(hostname=‘REDACTED’, ipaddress=‘REDACTED’, ident=‘piggyback’, fetcher_type=<FetcherType.PIGGYBACK: 4>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7fc91142a490]
Read from cache: NoCache(base_path=/dev/null, relative_path_template=, max_age=MaxAge(checking=0.0, discovery=0.0, inventory=0.0), simulation=False, use_only_cache=False, file_cache_mode=1)
[cpu_tracking] Stop [7fc91142a490 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
[cpu_tracking] Start [7fc91142a490] - PARSE FETCHER RESULTS
Get piggybacked data
0 piggyback files for ‘REDACTED’.
0 piggyback files for ‘REDACTED’.
HostKey(hostname=‘REDACTED’, source_type=<SourceType.HOST: 1>) → Add sections: [‘hr_cpu’, ‘hr_fs’, ‘hr_mem’, ‘if64’, ‘snmp_info’, ‘snmp_uptime’]
HostKey(hostname=‘REDACTED’, source_type=<SourceType.HOST: 1>) → Add sections:
Received no piggyback data
CPU utilization Total CPU: 1.47%
Filesystem / Used: 12.05% - 12.0 GiB of 100.0 GiB, trend per 1 day 0 hours: -9.24 MiB, trend per 1 day 0 hours: -0.01%
Filesystem /boot Used: 14.09% - 143 MiB of 1014 MiB, trend per 1 day 0 hours: +0 B, trend per 1 day 0 hours: +0%
Filesystem /dev/shm Used: <0.01% - 16.0 KiB of 46.4 GiB, trend per 1 day 0 hours: +0 B, trend per 1 day 0 hours: +0%
Filesystem /mnt/rd0 Used: <0.01% - 436 KiB of 46.4 GiB, trend per 1 day 0 hours: -20 B, trend per 1 day 0 hours: -0.00%
Filesystem /run Used: <0.01% - 3.35 MiB of 46.4 GiB, trend per 1 day 0 hours: -234 KiB, trend per 1 day 0 hours: -0.00%
Filesystem /run/user/1000 Used: 0% - 0 B of 9.29 GiB, trend per 1 day 0 hours: +0 B, trend per 1 day 0 hours: +0%
Filesystem /scratch Used: 2.58% - 7.28 GiB of 282 GiB, trend per 1 day 0 hours: -33.3 KiB, trend per 1 day 0 hours: -0.00%
Filesystem /REDACTED Used: 50.62% - REDACTED TiB of REDACTED TiB, trend per 1 day 0 hours: +1.44 GiB, trend per 1 day 0 hours: +<0.01%
Filesystem /REDACTED/REDACTED/bin Used: 12.05% - 12.0 GiB of 100.0 GiB, trend per 1 day 0 hours: -9.24 MiB, trend per 1 day 0 hours: -0.01%
Filesystem /REDACTED/REDACTED/dev Used: 12.05% - 12.0 GiB of 100.0 GiB, trend per 1 day 0 hours: -9.24 MiB, trend per 1 day 0 hours: -0.01%
Filesystem /REDACTED/REDACTED/lib Used: 12.05% - 12.0 GiB of 100.0 GiB, trend per 1 day 0 hours: -9.24 MiB, trend per 1 day 0 hours: -0.01%
Filesystem /REDACTED/REDACTED/lib64 Used: 12.05% - 12.0 GiB of 100.0 GiB, trend per 1 day 0 hours: -9.24 MiB, trend per 1 day 0 hours: -0.01%
Filesystem /REDACTED/REDACTED/sbin Used: 12.05% - 12.0 GiB of 100.0 GiB, trend per 1 day 0 hours: -9.24 MiB, trend per 1 day 0 hours: -0.01%
Filesystem /REDACTED/REDACTED/usr Used: 12.05% - 12.0 GiB of 100.0 GiB, trend per 1 day 0 hours: -9.24 MiB, trend per 1 day 0 hours: -0.01%
Filesystem /REDACTED/REDACTED/var/log/REDACTEDBackup Used: 40.83% - 40.8 GiB of 100.0 GiB, trend per 1 day 0 hours: -114 MiB, trend per 1 day 0 hours: -0.11%
Filesystem /sys/fs/cgroup Used: 0% - 0 B of 46.4 GiB, trend per 1 day 0 hours: +0 B, trend per 1 day 0 hours: +0%
Filesystem /var Used: 40.83% - 40.8 GiB of 100.0 GiB, trend per 1 day 0 hours: -114 MiB, trend per 1 day 0 hours: -0.11%
Interface 6 [p2p1], (up), MAC: REDACTED, Speed: 10 GBit/s, In: 1.35 MB/s (0.11%), Out: 16.7 kB/s (<0.01%)
Interface 7 [p2p2], (up), MAC: REDACTED, Speed: 10 GBit/s, In: 7.29 B/s (<0.01%), Out: 24.5 kB/s (<0.01%)
Interface 8 [bond0], (up), MAC: REDACTED, Speed: 20 GBit/s, In: 1.35 MB/s (0.05%), Out: 41.2 kB/s (<0.01%)
Memory Total (RAM + Swap): 57.26% - 53.2 GiB of 92.9 GiB RAM, RAM: 57.21% - 53.1 GiB of 92.9 GiB, Swap: 0.29% - 46.8 MiB of 16.0 GiB
SNMP Info REDACTED, REDACTED, "", REDACTED
Uptime Up since REDACTED, Uptime: 108 days 2 hours
0 piggyback files for ‘REDACTED’.
0 piggyback files for ‘REDACTED’.
[cpu_tracking] Stop [7fc91142a490 - Snapshot(process=posix.times_result(user=0.010000000000000009, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.019999999552965164))]
[snmp] Success, [piggyback] Success (but no data found for this host), execution time 0.3 sec | execution_time=0.260 user_time=0.020 system_time=0.010 children_user_time=0.070 children_system_time=0.110 cmk_time_snmp=0.040 cmk_time_agent=0.000
I am monitoring a host via SNMP. When CPU usage spikes for about 3 minutes, the service status changes to CRIT. The alert notification incorrectly states that the CPU usage has been high for days.
As you can see in the graph below, it is only a brief spike:
Here are the check parameters:
