This is how it looks, when the “Host Check Command” Rule is not set:
As we have restriced ping we usually use the agent state as host state and not ping.
2024-05-13 08:15:54 [5] [core 1222] Executing external command: LOG;SERVICE NOTIFICATION: cmkadmin;deagstest010;Check_MK Agent;WARNING;log;Version: 2.3.0p1, OS: linux, TLS is not activated on monitored host (see details)(!), Agent plug-ins: 9, Local checks: 0
2024-05-13 08:15:54 [7] [core 1222] [livestatus external] spooled command 'LOG;SERVICE NOTIFICATION RESULT: cmkadmin;deagstest010;Check_MK Agent;OK;log;;'
2024-05-13 08:15:54 [5] [core 1222] Executing external command: LOG;SERVICE NOTIFICATION RESULT: cmkadmin;deagstest010;Check_MK Agent;OK;log;;
2024-05-13 08:15:55 [6] [core 1222] [state saver] saved state to "/omd/sites/spcm/var/check_mk/core/state.pb" in 0.938977 ms
2024-05-13 08:15:57 [5] [core 1222] [config cleaner] removed dead configuration directory "/omd/sites/spcm/var/check_mk/core/helper_config/5"
2024-05-13 08:16:53 [7] [core 1222] [fetcher pool scheduler] unscheduling service "deagstest010;Check_MK" at 2024-05-13 08:16:53
2024-05-13 08:16:53 [7] [core 1222] obtained SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:2}
2024-05-13 08:16:58 [7] [core 1222] released SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:1}
2024-05-13 08:16:58 [7] [core 1222] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:17:58 with commandline [@cmk deagstest010 10.19.68.10]
2024-05-13 08:16:58 [7] [core 1222] [fetcher pool scheduler] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:17:58
2024-05-13 08:17:58 [7] [core 1222] [fetcher pool scheduler] unscheduling service "deagstest010;Check_MK" at 2024-05-13 08:17:58
2024-05-13 08:17:58 [7] [core 1222] obtained SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:2}
2024-05-13 08:18:04 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": hard state change to CRITICAL
2024-05-13 08:18:04 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": setting up, problem ID 16, delay 0sec
2024-05-13 08:18:04 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:04 [7] [core 1222] released SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:1}
2024-05-13 08:18:04 [7] [core 1222] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:19:04 with commandline [@cmk deagstest010 10.19.68.10]
2024-05-13 08:18:04 [7] [core 1222] [fetcher pool scheduler] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:19:04
2024-05-13 08:18:04 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:05 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:06 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:07 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:08 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:09 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:10 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:11 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:12 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:13 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:14 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:15 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:16 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:17 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:18 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:19 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:20 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:21 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:22 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:23 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:24 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:25 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:26 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:27 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:28 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:29 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:30 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:31 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:32 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:33 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:34 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:35 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:36 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:37 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:38 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:39 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:40 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:41 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:42 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:43 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:44 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:45 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:46 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:47 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:48 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:49 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:50 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:51 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:52 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:53 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:54 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:55 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:56 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:57 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:58 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:18:59 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:19:00 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:19:01 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:19:02 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:19:03 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": postponing, last host check not recent enough
2024-05-13 08:19:04 [7] [core 1222] [fetcher pool scheduler] unscheduling service "deagstest010;Check_MK" at 2024-05-13 08:19:04
2024-05-13 08:19:04 [7] [core 1222] obtained SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:2}
2024-05-13 08:19:04 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": sending PROBLEM notification to its contacts
2024-05-13 08:19:04 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": spooling notification to rule based notifications
2024-05-13 08:19:04 [7] spooled 1885 byte event: 'CONTACTS=
NOTIFICATIONTYPE=PROBLEM
NOTIFICATIONCOMMENT=
NOTIFICATIONAUTHOR=
NOTIFICATIONAUTHORNAME=
NOTIFICATIONAUTHORALIAS=
HOSTACKAUTHOR=
HOSTACKCOMMENT=
SERVICEACKAUTHOR=
SERVICEACKCOMMENT=
MICROTIME=1715588344755339
HOSTNOTIFICATIONNUMBER=0
LASTHOSTPROBLEMID=12
HOSTPROBLEMID=12
HOSTATTEMPT=1
MAXHOSTATTEMPTS=1
HOSTNAME=deagstest010
HOSTALIAS=dev-ca-1
HOSTADDRESS=10.19.68.10
HOSTOUTPUT=Packet received via smart PING
HOSTPERFDATA=
HOSTSTATE=UP
HOSTSTATEID=0
LASTHOSTSTATE=UP
LASTHOSTSTATEID=0
PREVIOUSHOSTHARDSTATE=DOWN
PREVIOUSHOSTHARDSTATEID=1
LONGHOSTOUTPUT=
HOSTCHECKCOMMAND=check-mk-host-smart
LASTHOSTSTATECHANGE=1715587090
LASTHOSTUP=1715588344
HOSTDOWNTIME=0
HOSTGROUPNAMES=check_mk
HOSTCONTACTGROUPNAMES=all
HOSTTAGS=/wato/ auto-piggyback checkmk-agent cmk-agent ip-v4 ip-v4-only lan no-snmp prod site:spcm tcp
HOST_SL=
HOST_EC_CONTACT=
HOSTNOTESURL=
HOSTNOTES=
HOST_FILENAME=/wato/hosts.mk
HOST_ADDRESS_6=
HOST_TAGS=/wato/ auto-piggyback checkmk-agent cmk-agent ip-v4 ip-v4-only lan no-snmp prod site:spcm tcp
HOST_ADDRESSES_6=
HOST_ADDRESS_4=10.19.68.10
HOST_ADDRESS_FAMILY=4
HOST_ADDRESSES_4=
SERVICEDESC=HAProxy Server ca-adm-stateful/ca-1
SERVICEOUTPUT=Status: DOWN(!!), Active, Layer Check: L7STS, Up since 15 seconds
SERVICEPERFDATA=
SERVICENOTIFICATIONNUMBER=1
SERVICEPROBLEMID=16
LASTSERVICEPROBLEMID=16
SERVICEATTEMPT=1
MAXSERVICEATTEMPTS=1
SERVICESTATE=CRITICAL
SERVICESTATEID=2
LASTSERVICESTATE=OK
LASTSERVICESTATEID=0
PREVIOUSSERVICEHARDSTATE=OK
PREVIOUSSERVICEHARDSTATEID=0
LASTSERVICESTATECHANGE=1715588284
LASTSERVICEOK=1715588218
SERVICEDOWNTIME=0
LONGSERVICEOUTPUT=Status: DOWN(!!)\\nActive\\nLayer Check: L7STS\\nUp since 15 seconds
SERVICECHECKCOMMAND=check_mk-haproxy_server
SERVICEGROUPNAMES=
SERVICECONTACTGROUPNAMES=all
SVC_SL=
SERVICE_SL=
SERVICE_EC_CONTACT=
SERVICENOTESURL=
SERVICENOTES=
SERVICEDISPLAYNAME=HAProxy Server ca-adm-stateful/ca-1
'
2024-05-13 08:19:04 [7] [notification helper 1275] service "deagstest010;HAProxy Server ca-adm-stateful/ca-1": cancelled: no more notifications necessary
2024-05-13 08:19:05 [7] unspooled 1885 byte event: 'CONTACTS=
NOTIFICATIONTYPE=PROBLEM
NOTIFICATIONCOMMENT=
NOTIFICATIONAUTHOR=
NOTIFICATIONAUTHORNAME=
NOTIFICATIONAUTHORALIAS=
HOSTACKAUTHOR=
HOSTACKCOMMENT=
SERVICEACKAUTHOR=
SERVICEACKCOMMENT=
MICROTIME=1715588344755339
HOSTNOTIFICATIONNUMBER=0
LASTHOSTPROBLEMID=12
HOSTPROBLEMID=12
HOSTATTEMPT=1
MAXHOSTATTEMPTS=1
HOSTNAME=deagstest010
HOSTALIAS=dev-ca-1
HOSTADDRESS=10.19.68.10
HOSTOUTPUT=Packet received via smart PING
HOSTPERFDATA=
HOSTSTATE=UP
HOSTSTATEID=0
LASTHOSTSTATE=UP
LASTHOSTSTATEID=0
PREVIOUSHOSTHARDSTATE=DOWN
PREVIOUSHOSTHARDSTATEID=1
LONGHOSTOUTPUT=
HOSTCHECKCOMMAND=check-mk-host-smart
LASTHOSTSTATECHANGE=1715587090
LASTHOSTUP=1715588344
HOSTDOWNTIME=0
HOSTGROUPNAMES=check_mk
HOSTCONTACTGROUPNAMES=all
HOSTTAGS=/wato/ auto-piggyback checkmk-agent cmk-agent ip-v4 ip-v4-only lan no-snmp prod site:spcm tcp
HOST_SL=
HOST_EC_CONTACT=
HOSTNOTESURL=
HOSTNOTES=
HOST_FILENAME=/wato/hosts.mk
HOST_ADDRESS_6=
HOST_TAGS=/wato/ auto-piggyback checkmk-agent cmk-agent ip-v4 ip-v4-only lan no-snmp prod site:spcm tcp
HOST_ADDRESSES_6=
HOST_ADDRESS_4=10.19.68.10
HOST_ADDRESS_FAMILY=4
HOST_ADDRESSES_4=
SERVICEDESC=HAProxy Server ca-adm-stateful/ca-1
SERVICEOUTPUT=Status: DOWN(!!), Active, Layer Check: L7STS, Up since 15 seconds
SERVICEPERFDATA=
SERVICENOTIFICATIONNUMBER=1
SERVICEPROBLEMID=16
LASTSERVICEPROBLEMID=16
SERVICEATTEMPT=1
MAXSERVICEATTEMPTS=1
SERVICESTATE=CRITICAL
SERVICESTATEID=2
LASTSERVICESTATE=OK
LASTSERVICESTATEID=0
PREVIOUSSERVICEHARDSTATE=OK
PREVIOUSSERVICEHARDSTATEID=0
LASTSERVICESTATECHANGE=1715588284
LASTSERVICEOK=1715588218
SERVICEDOWNTIME=0
LONGSERVICEOUTPUT=Status: DOWN(!!)\\nActive\\nLayer Check: L7STS\\nUp since 15 seconds
SERVICECHECKCOMMAND=check_mk-haproxy_server
SERVICEGROUPNAMES=
SERVICECONTACTGROUPNAMES=all
SVC_SL=
SERVICE_SL=
SERVICE_EC_CONTACT=
SERVICENOTESURL=
SERVICENOTES=
SERVICEDISPLAYNAME=HAProxy Server ca-adm-stateful/ca-1
'
2024-05-13 08:19:05 [7] [notification helper 1275] successfully sent event
2024-05-13 08:19:09 [7] [core 1222] [livestatus external] spooled command 'LOG;SERVICE NOTIFICATION: cmkadmin;deagstest010;HAProxy Server ca-adm-stateful/ca-1;CRITICAL;log;Status: DOWN(!!), Active, Layer Check: L7STS, Up since 15 seconds'
2024-05-13 08:19:09 [5] [core 1222] Executing external command: LOG;SERVICE NOTIFICATION: cmkadmin;deagstest010;HAProxy Server ca-adm-stateful/ca-1;CRITICAL;log;Status: DOWN(!!), Active, Layer Check: L7STS, Up since 15 seconds
2024-05-13 08:19:09 [7] [core 1222] [livestatus external] spooled command 'LOG;SERVICE NOTIFICATION RESULT: cmkadmin;deagstest010;HAProxy Server ca-adm-stateful/ca-1;OK;log;;'
2024-05-13 08:19:09 [5] [core 1222] Executing external command: LOG;SERVICE NOTIFICATION RESULT: cmkadmin;deagstest010;HAProxy Server ca-adm-stateful/ca-1;OK;log;;
2024-05-13 08:19:09 [7] [core 1222] released SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:1}
2024-05-13 08:19:09 [7] [core 1222] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:20:09 with commandline [@cmk deagstest010 10.19.68.10]
2024-05-13 08:19:09 [7] [core 1222] [fetcher pool scheduler] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:20:09
2024-05-13 08:20:09 [7] [core 1222] [fetcher pool scheduler] unscheduling service "deagstest010;Check_MK" at 2024-05-13 08:20:09
2024-05-13 08:20:09 [7] [core 1222] obtained SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:2}
2024-05-13 08:20:15 [7] [core 1222] released SerialToken{Request[deagstest010],6} => SerialTokenFactory{6:1}
2024-05-13 08:20:15 [7] [core 1222] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:21:15 with commandline [@cmk deagstest010 10.19.68.10]
2024-05-13 08:20:15 [7] [core 1222] [fetcher pool scheduler] scheduling service "deagstest010;Check_MK" at 2024-05-13 08:21:15