ILO7 Abfrage über Redfish dauert sehr lange bis Timeouts

Hallo zusammen,

Die Fehlerbeschreibung:

Die Abfrage von HPE Servern mit iLO7(neuste Firmware) dauert bis zu 60 Sekunden

OMD[nms]:~$ cmk --debug -vvI esxi04-ilo
Discovering services and host labels on: esxi04-ilo
esxi04-ilo:
+ FETCHING DATA
  Source: SourceInfo(hostname='esxi04-ilo', ipaddress='10.1.2.77', ident='special_redfish', fetcher_type=<FetcherType.SPECIAL_AGENT: 6>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [75e154a33380]
Read from cache: AgentFileCache(path_template=/omd/sites/nms/tmp/check_mk/data_source_cache/special_redfish/esxi04-ilo, max_age=MaxAge(checking=0, discovery=90.0, inventory=90.0), simulation=False, use_only_cache=False, file_cache_mode=1)
Calling: /omd/sites/nms/lib/python3/cmk/plugins/redfish/libexec/agent_redfish -u ilo_checkmk --password-id ilo_checkmk:/omd/sites/nms/var/check_mk/passwords_merged -p 443 -P https -m Memory,Power,Processors,Thermal,FirmwareInventory,NetworkAdapters,NetworkInterfaces,EthernetInterfaces,Storage,ArrayControllers,SmartStorage,HostBusAdapters,PhysicalDrives,LogicalDrives,Drives,Volumes,SimpleStorage -n '' -c FirmwareInventory-600,NetworkAdapters-600,NetworkInterfaces-600 --timeout 15 --retries 2 10.1.2.77
Get data from program


//ab jetzt > 50 Sek.

[cpu_tracking] Stop [77fe32343bf0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.64, children_system=0.08, elapsed=166.99000000022352))]
  Source: SourceInfo(hostname='esxi04-ilo', ipaddress='10.1.2.77', ident='piggyback', fetcher_type=<FetcherType.PIGGYBACK: 4>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [77fe323430e0]
Read from cache: NoCache(path_template=/dev/null, max_age=MaxAge(checking=0.0, discovery=0.0, inventory=0.0), simulation=False, use_only_cache=False, file_cache_mode=1)
0 piggyback files for 'esxi04-ilo'.
0 piggyback files for '10.1.2.77'.
Get piggybacked data
[cpu_tracking] Stop [77fe323430e0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.009999999776482582))]
+ PARSE FETCHER RESULTS
<<<labels:sep(0)>>> / Transition NOOPParser -> HostSectionParser
<<<redfish_manager:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_system:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_chassis:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_firmwareinventory:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_storage:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_memory:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_networkinterfaces:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_processors:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_ethernetinterfaces:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_drives:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_volumes:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_networkadapters:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_power:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<redfish_thermal:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
  HostKey(hostname='esxi04-ilo', source_type=<SourceType.HOST: 1>)  -> Add sections: ['labels', 'redfish_chassis', 'redfish_drives', 'redfish_ethernetinterfaces', 'redfish_firmwareinventory', 'redfish_manager', 'redfish_memory', 'redfish_networkadapters', 'redfish_networkinterfaces', 'redfish_power', 'redfish_processors', 'redfish_storage', 'redfish_system', 'redfish_thermal', 'redfish_volumes']
  HostKey(hostname='esxi04-ilo', source_type=<SourceType.HOST: 1>)  -> Add sections: []
Received no piggyback data
+ ANALYSE DISCOVERED HOST LABELS
Trying host label discovery with: labels, redfish_chassis, redfish_drives, redfish_ethernetinterfaces, redfish_firmwareinventory, redfish_manager, redfish_memory, redfish_networkadapters, redfish_networkinterfaces, redfish_power, redfish_processors, redfish_storage, redfish_system, redfish_thermal, redfish_volumes
  cmk/os_family: redfish (labels)
  cmk/os_name: iLO 7 (labels)
  cmk/os_platform: HPE (labels)
  cmk/os_type: redfish (labels)
  cmk/os_version: 1.20 (labels)
Trying host label discovery with:
SUCCESS - Found 5 host labels
+ ANALYSE DISCOVERED SERVICES
+ EXECUTING DISCOVERY PLUGINS (14)
  Trying discovery with: redfish_voltage, redfish_firmware, redfish_fans, redfish_memory, redfish_networkadapters, redfish_temperatures, redfish_volumes, redfish_ethernetinterfaces, redfish_psu, redfish_storage, redfish_memory_summary, redfish_processors, redfish_system, redfish_drives
SUCCESS - Found no new services


und die Agent-Ausgabe in der GUI:

[special_redfish] Agent exited with code 1: ERROR: too many retries for connection attemptCRIT, [piggyback] Success (but no data found for this host), Missing monitoring data for all pluginsWARN, execution time 47.6 sec

Number of connection retries: 2

Timeout for connection: 15s

Zum Vergleich, in gleichem VLAN: iLO6 und iLo5: < 8 Sek

Die Umgebung:

Checkmk Enterprise Edition 2.4.0p22

HPE-Model: HPE DL380 G12(neuste Firmware) mit iLO 7(neuste Firmware)

Konnte jemand ähnliches Verhalten beobachten?

Vielen Dank im Voraus

Diese Command Line einfach mal per Hand ausführen und die Option “–debug” anhängen.

3 Minuten ist viel zu lang. Mit der Debug Option erzeugt er ein ausführliches Log was alles so passiert und an welcher Stelle die Zeit gebraucht wird.

…. –debug

Die Ausgabe:

INFO 2026-03-04 17:00:24 redfish: Redfish API
INFO 2026-03-04 17:00:24 redfish.rest.v1: Attempt 1 of /redfish/v1
INFO 2026-03-04 17:00:26 redfish.rest.v1: Response Time for GET to /redfish/v1:

die komplette Ausgabe im Anhang

debug_ilo7.txt (134.3 KB)

PS. Die neuen Server haben 2TB RAM:

Auffällig:

….

INFO 2026-03-04 17:01:27 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory: 0.014009062986588106 seconds.
INFO 2026-03-04 17:01:27 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm1
INFO 2026-03-04 17:01:27 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm1: 0.0310341709991917 seconds.
INFO 2026-03-04 17:01:27 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm2
INFO 2026-03-04 17:01:27 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm2: 0.028957645990885794 seconds.
INFO 2026-03-04 17:01:27 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm3
INFO 2026-03-04 17:01:27 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm3: 0.03260615299222991 seconds.
INFO 2026-03-04 17:01:27 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm4
INFO 2026-03-04 17:01:27 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm4: 0.030516171013005078 seconds.
INFO 2026-03-04 17:01:27 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm5
INFO 2026-03-04 17:01:27 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm5: 0.03137825502199121 seconds.
INFO 2026-03-04 17:01:27 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm6
INFO 2026-03-04 17:01:27 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm6: 0.03162535300361924 seconds.
INFO 2026-03-04 17:01:27 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm7
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm7: 0.032569667004281655 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm8
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm8: 0.035766246990533546 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm9
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm9: 0.03968867100775242 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm10
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm10: 0.04613818897632882 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm11
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm11: 0.04503690201090649 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm12
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm12: 0.08032506599556655 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm13
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm13: 0.04922755801817402 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm14
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm14: 0.05333906100713648 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm15
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm15: 0.07729856699006632 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc1dimm16
INFO 2026-03-04 17:01:28 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc1dimm16: 0.06828273402061313 seconds.
INFO 2026-03-04 17:01:28 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm1
INFO 2026-03-04 17:01:38 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm1: 10.135284149990184 seconds.
INFO 2026-03-04 17:01:38 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm2
INFO 2026-03-04 17:01:48 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm2: 10.067554240988102 seconds.
INFO 2026-03-04 17:01:48 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm3
INFO 2026-03-04 17:01:59 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm3: 10.748558747989591 seconds.
INFO 2026-03-04 17:01:59 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm4
INFO 2026-03-04 17:02:09 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm4: 10.071254783018958 seconds.
INFO 2026-03-04 17:02:09 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm5
INFO 2026-03-04 17:02:19 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm5: 10.07412231300259 seconds.
INFO 2026-03-04 17:02:19 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm6
INFO 2026-03-04 17:02:29 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm6: 10.163512782019097 seconds.
INFO 2026-03-04 17:02:29 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm7
INFO 2026-03-04 17:02:39 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm7: 10.08299342400278 seconds.
INFO 2026-03-04 17:02:39 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm8
INFO 2026-03-04 17:02:49 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm8: 10.138024714018684 seconds.
INFO 2026-03-04 17:02:49 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm9
INFO 2026-03-04 17:03:00 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm9: 10.111263820988825 seconds.
INFO 2026-03-04 17:03:00 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm10
INFO 2026-03-04 17:03:10 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm10: 10.191929174005054 seconds.
INFO 2026-03-04 17:03:10 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm11
INFO 2026-03-04 17:03:20 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm11: 10.139657788007753 seconds.
INFO 2026-03-04 17:03:20 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm12
INFO 2026-03-04 17:03:30 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm12: 10.146931735012913 seconds.
INFO 2026-03-04 17:03:30 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm13
INFO 2026-03-04 17:03:40 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm13: 10.092934354004683 seconds.
INFO 2026-03-04 17:03:40 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm14
INFO 2026-03-04 17:03:50 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm14: 10.113085449993378 seconds.
INFO 2026-03-04 17:03:50 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm15
INFO 2026-03-04 17:04:00 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1/Memory/proc2dimm15: 10.14078086000518 seconds.
INFO 2026-03-04 17:04:00 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1/Memory/proc2dimm16

Den Speicher Check für die einzelnen Module abschalten und nur den Memory Summary nutzen.

Das Abschalten erfolgt in der Special Agent Konfiguration.

Der kommt automatisch mit den System Daten.

Hallo

Danke, hat etwas geholfen, aber nicht komplett:

Redfish Compatible Management Controller->Fetching setting for individual sections->Memory Modules:Never

[special_redfish] Fetcher for host "esxi06-ilo" timed out after 60 secondsCRIT, [piggyback] Fetcher for host "esxi06-ilo" timed out after 60 secondsCRIT,


debug_ilo7.txt (134.3 KB)

Die Ausgabe, siehe Anhang.

Auf dem iLO ist was richtig kaputt - hab auch schon iLO7 im Monitoring und keine Probleme.

Hier allein die Anmeldung dauert viel viel zu lange.

INFO 2026-03-04 17:00:26 redfish.rest.v1: Attempt 1 of /redfish/v1/SessionService/Sessions
INFO 2026-03-04 17:00:36 redfish.rest.v1: Response Time for GET to /redfish/v1/SessionService/Sessions: 10.158979480998823 seconds.
INFO 2026-03-04 17:00:36 redfish.rest.v1: Attempt 1 of /redfish/v1
INFO 2026-03-04 17:00:47 redfish.rest.v1: Response Time for GET to /redfish/v1: 10.111598446994321 seconds.
INFO 2026-03-04 17:00:47 redfish.rest.v1: Attempt 1 of /redfish/v1/Managers?$expand=.
INFO 2026-03-04 17:00:57 redfish.rest.v1: Response Time for GET to /redfish/v1/Managers?$expand=.: 10.188599655986764 seconds.
<<<labels:sep(0)>>>
{"cmk/os_family": "redfish", "cmk/os_name": "iLO 7", "cmk/os_platform": "HPE", "cmk/os_type": "redfish", "cmk/os_version": "1.20"}
INFO 2026-03-04 17:00:57 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems
INFO 2026-03-04 17:01:07 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems: 10.076941022998653 seconds.
INFO 2026-03-04 17:01:07 redfish.rest.v1: Attempt 1 of /redfish/v1/Systems/1
INFO 2026-03-04 17:01:17 redfish.rest.v1: Response Time for GET to /redfish/v1/Systems/1: 10.210448865982471 seconds.

Jeder Aufruf 10 Sekunden das passt nicht.

auch die aktuellste FW-Version?

1.20.00 Feb 12 2026

Das müsste ich erstmal nachschauen bei Gelegenheit.

Eine neue Erkenntnis:

Nach dem wir gestern das Betriebssystem(vmware esxi) installiert haben, laufen die ILO-Abfragen schnell.

1 Like