Piggyback Daten / SAP Überwachung

fbotte · March 8, 2021, 4:03pm

Moin,

ich verzweifele gerade am SAP Monitoring mit CheckMK. Ich kriege die Werte die im SAP Plugin konfiguriert sind in der GUI einfach nicht zu sehen, der Agent liefert die Daten aber.

Ich hab im SAP Agenten bei den CCMS Pfaden u.A diese Zeile mit drin:
SAP CCMS Admin Workplace/Entire System/KA*/Database/DatabaseClient/DBConnection/DBServer

Wenn ich jetzt mit telnet meine Agneten direkt Abfrage ist die Zeile in der Antwort drin:

[root@lxomd2 lxhhsapdev]# telnet lxhhsapdev 6556 |grep DatabaseClient
sapkax_KAX_03 1 50 SAP CCMS Admin Workplace/Entire System/KAX/Database/DatabaseClient/DBConnection/DBServer - - 2021-03-08 16:46:32: lxhhsapdev Details: connected to database server lxhhsapdev, database KAX (ORACLE, OCI_112, 19.5.0.0.0,)
sapkax_KAX_03 1 50 SAP CCMS Admin Workplace/Entire System/KAX/Database/DatabaseClient/DBConnection/DBServer - - 2021-03-08 16:46:32: lxhhsapdev Details: connected to database server lxhhsapdev, database KAX (ORACLE, OCI_112, 19.5.0.0.0,)

Meine Erwartung war eigentlich, das CheckMK via Dynamic Host einen neunen Host Namnes lxhhsapdev_KAX anlegt unter dem die ganzen CCMS Check eingehängt sind.

Den Host legt er auch an, aber das sind keine Services drunter.
Im Verzeichnis /omd/sites//tmp/check_mk/piggyback/lxhhsapdev_KAX finde ich auch eine Datei lxhhsapdev die die CCMS Checks enthält, aber in der GUI krieg ich die ums ver****en nicht zu sehen.

Irgendjemand 'ne Idee was ich da falsch mache ?

Gruß
Frank

andreas-doehler · March 9, 2021, 7:16am

Was passiert wenn du auf der Shell ein “cmk --debug -vvII lxhhsapdev_KAX” machst.
Damit wird ja das Inventory für den Piggyback Host angeschubst.
Falls hier kein Fehler kommt und nix gefunden wird bedeutet dies, dass die Daten welche für den Host vorhanden sind nicht zu einem Check passen.

fbotte · March 9, 2021, 9:34am

Moin Andreas,

danke erst mal für die Antwort

Ich hab das mal ausgeführt. Da werden jede Menge Serivces gefunden, das sind aber alles die von dem SAP System selber und nicht die gesuchten CCMS Checks.

Ich vermute mal das das daran liegt, das bei dem lxhhsapdev_KAX als IP auch die vom SAP System eigetragen ist.

Aber das Piggybackfile mit den CCMS Checks wird auch verarbeitet: /omd/sites/test/tmp/check_mk/piggyback/lxhhsapdev_KAX/lxhhsapdev

Das File sieht so aus:
<<sap:sep(9):cached(1615281653,90)>>
sapkax_KAX_03 1 50 SAP CCMS Admin Workplace/Entire System/KAX/Database/DatabaseClient/DBConnection/DBServer - - 2021-03-09 10:16:32: lxhhsapdev Details: connected to database server lxhhsapdev, database KAX (ORACLE, OCI_112, 19.5.0.0.0,)
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Dialog Response Time/ResponseTime 5 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Network Time/FrontEndNetTime 0 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Standardized Response Time/ResponseTime(StandardTran.) 4 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Users Logged On/UsersLoggedIn 1 -
Background 1 50 SAP CCMS Monitor Templates/Background Processing/System Wide/Background/BackgroundService/SystemWideFreeBPWP 5 WPs
sapkax_KAX_03 1 60 SAP CCMS Monitor Templates/Entire System/KAX/Application Server/sapkax_KAX_03/R3Services/Update/Update/AbapErrorInUpdate - - None:
Spool 3 90 SAP CCMS Monitor Templates/Spool System/System Wide/Spool/SpoolService/sapkap_KAP_00/Status - - 2021-03-09 09:16:32: Spool server is not active. Output request waiting!
Spool 1 90 SAP CCMS Monitor Templates/Spool System/System Wide/Spool/SpoolService/sapkax_KAX_03/Status - - 2021-03-09 09:16:32: Spool server is active
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/ErrorsInWpSPO 0 -
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/ErrorFreqInWpSPO 0.0 /min
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/QueueLength 0 %
sapkax_KAX_03 1 60 SAP CCMS Technical Expert Monitors/All Contexts on Local Application Server/sapkax_KAX_03/InstanceAsTask/Status - - 2021-03-09 09:20:49: Active since 26.11.2020, 15:01:32: Last test: 09.03.2021, 10:20:49

Trotzdem sehe ich die Services nicht, weder bei lxhhsapdev_KAX noch bei dem SAP Host lxhhsapdev.
Richtig gut wäre es wenn ich am Ende nur einen Host hab und alles unter lxhhsapdev auftaucht, aber eins nach dem anderen erstmal muss ich die Serivces überhaupt irgendwo sehen.

Und hier ist noch der komplette Output vom cmk Kommando.
OMD[test]:~$ cmk --debug -vvII lxhhsapdev_KAX
Discovering services on: lxhhsapdev_KAX
lxhhsapdev_KAX:

FETCHING DATA
[agent] No persisted sections loaded
[agent] Not using cache (Does not exist)
[agent] Execute data source
[agent] Connecting via TCP to lxhhsapdev:6556 (5.0s timeout)
[agent] Reading data from agent
[agent] Write data to cache file /omd/sites/test/tmp/check_mk/cache/lxhhsapdev_KAX
Try aquire lock on /omd/sites/test/tmp/check_mk/cache/lxhhsapdev_KAX
Got lock on /omd/sites/test/tmp/check_mk/cache/lxhhsapdev_KAX
Releasing lock on /omd/sites/test/tmp/check_mk/cache/lxhhsapdev_KAX
Released lock on /omd/sites/test/tmp/check_mk/cache/lxhhsapdev_KAX
[piggyback] No persisted sections loaded
[piggyback] Execute data source
Piggyback file ‘/omd/sites/test/tmp/check_mk/piggyback/lxhhsapdev_KAX/lxhhsapdev’: Successfully processed from source ‘lxhhsapdev’.
Piggyback file ‘/omd/sites/test/tmp/check_mk/piggyback/lxhhsapdev/lxvcarz’: Successfully processed from source ‘lxvcarz’.
Storing piggyback data for: KAX
Try aquire lock on /omd/sites/test/tmp/check_mk/piggyback/KAX/lxhhsapdev_KAX
Got lock on /omd/sites/test/tmp/check_mk/piggyback/KAX/lxhhsapdev_KAX
Releasing lock on /omd/sites/test/tmp/check_mk/piggyback/KAX/lxhhsapdev_KAX
Released lock on /omd/sites/test/tmp/check_mk/piggyback/KAX/lxhhsapdev_KAX
EXECUTING DISCOVERY PLUGINS (88)
Trying discovery with: oracle_tablespaces, esx_vsphere_vm.snapshots, cpu.loads, ntp.time, oracle_undostat, check_mk.only_from, postfix_mailq_status, sap_hana_filesystem, sap_hana_license, sap_hana_backup, local, labels, oracle_rman_backups, md, df, sap_hana_mem.database, esx_vsphere_vm.heartbeat, mem.vmalloc, mem.win, systemd_units.services, oracle_sessions, mounts, kernel, sap_hana_diskusage, sap.dialog, mrpe, systemd_units, postfix_mailq, cpu.threads, ps_lnx, sap_hana_events, esx_vsphere_vm.name, oracle_instance, oracle_logswitches, ntp, check_mk.agent_update, job, sap_hana_process_list, esx_vsphere_vm.guest_tools, esx_vsphere_vm.datastores, sap_hana_version, ps.perf, sap.value, lnx_if, diskstat, ps, sap.value-groups, oracle_processes, esx_vsphere_vm.mem_usage, oracle_performance, sap_hana_memrate, oracle_recovery_status, oracle_rman, esx_vsphere_vm.mounted_devices, sap.value_groups, oracle_dataguard_stats, sap_hana_data_volume, kernel.util, sap_hana_ess, esx_vsphere_vm.running_on, lvm_lvs, sap_hana_full_backup, mem.linux, cifsmounts, sap_hana_mem, sap_hana_status, esx_vsphere_vm.cpu, df_netapp32, oracle_asm_diskgroup, tcp_conn_stats, uptime, df_netapp, nfsmounts, saprouter_cert, df_netscaler, sap_state, oracle_jobs, nfsiostat, lvm_vgs, mem.used, sap_hana_replication_status, systemd_units.services_summary, oracle_longactivesessions, vbox_guest, oracle_recovery_area, sap_hana_proc, oracle_locks, df_zos
ps.perf does not support discovery. Skipping it.
systemd_units does not support discovery. Skipping it.
ps_lnx does not support discovery. Skipping it.
Try aquire lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Got lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Releasing lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Released lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Try aquire lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
Got lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
Releasing lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
Released lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
1 check_mk.agent_update
1 cifsmounts
1 cpu.loads
1 cpu.threads
19 df
1 diskstat
1 esx_vsphere_vm.cpu
1 esx_vsphere_vm.datastores
1 esx_vsphere_vm.guest_tools
1 esx_vsphere_vm.heartbeat
1 esx_vsphere_vm.mem_usage
1 esx_vsphere_vm.mounted_devices
1 esx_vsphere_vm.name
1 esx_vsphere_vm.running_on
1 esx_vsphere_vm.snapshots
3 kernel
1 kernel.util
1 lnx_if
1 local
4 lvm_vgs
1 mem.linux
19 mounts
2 mrpe
2 nfsiostat
2 nfsmounts
1 ntp.time
1 oracle_instance
20 oracle_jobs
1 oracle_locks
1 oracle_logswitches
1 oracle_longactivesessions
1 oracle_performance
1 oracle_processes
1 oracle_recovery_area
1 oracle_recovery_status
1 oracle_rman
1 oracle_sessions
7 oracle_tablespaces
1 oracle_undostat
1 postfix_mailq
1 postfix_mailq_status
1 sap.dialog
1 sap_state
1 systemd_units.services_summary
1 tcp_conn_stats
1 uptime
SUCCESS - Found 115 services, 3 host labels

andreas-doehler · March 9, 2021, 12:22pm

Wenn du nur testen willst ob die per Piggyback übermittelten Daten für was gut sind würde ich auch hier mal einen Dummy Host verwenden welcher nur den Namen trägt und als Agent Type “no Agent” gesetzt hat.

fbotte · March 9, 2021, 12:47pm

Der Host wurde so mit der Konfiguration von der Dynamic Host Configuration erzeugt.
Um den auf “no-agent” zu kriegen musste ich den komplett löschen und danach manuell anlegen.
Ich hab ihn jetzt mal mit “No Agent”, “Always use and expect Piggybackdata” und ohne IP konfiguriert.

Die vermissten Checks sehe ich leider immer noch nicht
OMD[test]:~$ cmk --debug -vvII lxhhsapdev_KAX
Discovering services on: lxhhsapdev_KAX
lxhhsapdev_KAX:

FETCHING DATA
[piggyback] No persisted sections loaded
[piggyback] Execute data source
Piggyback file ‘/omd/sites/test/tmp/check_mk/piggyback/lxhhsapdev_KAX/lxhhsapdev’: Successfully processed from source ‘lxhhsapdev’.
EXECUTING DISCOVERY PLUGINS (23)
Trying discovery with: sap_hana_diskusage, sap_hana_mem, sap_hana_status, labels, sap.dialog, sap.value-groups, sap_hana_filesystem, sap_hana_license, sap_hana_backup, saprouter_cert, sap_state, sap_hana_memrate, sap.value_groups, sap_hana_process_list, sap_hana_mem.database, sap_hana_replication_status, sap_hana_data_volume, sap_hana_version, sap_hana_ess, sap.value, sap_hana_events, sap_hana_proc, sap_hana_full_backup
Try aquire lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Got lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Releasing lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Released lock on /omd/sites/test/var/check_mk/autochecks/lxhhsapdev_KAX.mk
Try aquire lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
Got lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
Releasing lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
Released lock on /omd/sites/test/var/check_mk/discovered_host_labels/lxhhsapdev_KAX.mk
1 sap.dialog
SUCCESS - Found 1 services, 1 host labels

Wenn ich das ganze allerdings auf unserem Produktiven System mach findet das die ganzen Services
OMD[kalo]:~$ cmk --debug -vvII KAX
Discovering services on: KAX
KAX:

FETCHING DATA
[piggyback] No persisted sections loaded
[piggyback] Execute data source
Piggyback file ‘/omd/sites/kalo/tmp/check_mk/piggyback/KAX/lxhhsapdev’: Successfully processed from source ‘lxhhsapdev’.
No piggyback files for ‘10.10.70.242’. Skip processing.
Loading autochecks from /omd/sites/kalo/var/check_mk/autochecks/KAX.mk
EXECUTING DISCOVERY PLUGINS (23)
Trying discovery with: sap_hana_diskusage, sap_hana_mem, sap_hana_status, labels, sap.dialog, sap.value-groups, sap_hana_filesystem, sap_hana_license, sap_hana_backup, saprouter_cert, sap_state, sap_hana_memrate, sap.value_groups, sap_hana_process_list, sap_hana_mem.database, sap_hana_replication_status, sap_hana_data_volume, sap_hana_version, sap_hana_ess, sap.value, sap_hana_events, sap_hana_proc, sap_hana_full_backup
Try aquire lock on /omd/sites/kalo/var/check_mk/autochecks/KAX.mk
Got lock on /omd/sites/kalo/var/check_mk/autochecks/KAX.mk
Releasing lock on /omd/sites/kalo/var/check_mk/autochecks/KAX.mk
Released lock on /omd/sites/kalo/var/check_mk/autochecks/KAX.mk
Try aquire lock on /omd/sites/kalo/var/check_mk/discovered_host_labels/KAX.mk
Got lock on /omd/sites/kalo/var/check_mk/discovered_host_labels/KAX.mk
Releasing lock on /omd/sites/kalo/var/check_mk/discovered_host_labels/KAX.mk
Released lock on /omd/sites/kalo/var/check_mk/discovered_host_labels/KAX.mk
1 sap.dialog
13 sap.value
SUCCESS - Found 14 services, 1 host labels

Der Host heißt hier zwar KAX, das liegt aber daran das der aus einer Zeit stammt als es noch keine Dynamic Configuration gab.

andreas-doehler · March 9, 2021, 8:12pm

Sehen die beiden Piggyback Daten Files ähnlich aus?
Weil du selbst sagst scheint der zweite Host ja ok zu sein.
Interessant ist nur wie viele SAP Plugins er versucht zu discovern und nur Value und Dialog findet.

fbotte · March 9, 2021, 9:54pm

Ja die beiden Dateien sind bis auf die Timestamps und ein paar Messwerte identisch:

Produktion (funktioniert)
<<sap:sep(9):cached(1615326470,90)>>
sapkax_KAX_03 1 50 SAP CCMS Admin Workplace/Entire System/KAX/Database/DatabaseClient/DBConnection/DBServer - - 2021-03-09 22:46:32: lxhhsapdev Details: connected to database server lxhhsapdev, database KAX (ORACLE, OCI_112, 19.5.0.0.0,)
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Dialog Response Time/ResponseTime 5 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Network Time/FrontEndNetTime 0 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Standardized Response Time/ResponseTime(StandardTran.) 4 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Users Logged On/UsersLoggedIn 2 -
Background 1 50 SAP CCMS Monitor Templates/Background Processing/System Wide/Background/BackgroundService/SystemWideFreeBPWP 19 WPs
sapkax_KAX_03 1 60 SAP CCMS Monitor Templates/Entire System/KAX/Application Server/sapkax_KAX_03/R3Services/Update/Update/AbapErrorInUpdate - - None:
Spool 3 90 SAP CCMS Monitor Templates/Spool System/System Wide/Spool/SpoolService/sapkap_KAP_00/Status - - 2021-03-09 21:46:32: Spool server is not active. Output request waiting!
Spool 1 90 SAP CCMS Monitor Templates/Spool System/System Wide/Spool/SpoolService/sapkax_KAX_03/Status - - 2021-03-09 21:46:32: Spool server is active
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/ErrorsInWpSPO 0 -
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/ErrorFreqInWpSPO 0.0 /min
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/QueueLength 0 %
sapkax_KAX_03 1 60 SAP CCMS Technical Expert Monitors/All Contexts on Local Application Server/sapkax_KAX_03/InstanceAsTask/Status - - 2021-03-09 21:47:47: Active since 26.11.2020, 15:01:32: Last test: 09.03.2021, 22:47:47

Neues CheckMK (tut nicht)
<<sap:sep(9):cached(1615326491,90)>>
sapkax_KAX_03 1 50 SAP CCMS Admin Workplace/Entire System/KAX/Database/DatabaseClient/DBConnection/DBServer - - 2021-03-09 22:46:32: lxhhsapdev Details: connected to database server lxhhsapdev, database KAX (ORACLE, OCI_112, 19.5.0.0.0,)
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Dialog Response Time/ResponseTime 80 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Network Time/FrontEndNetTime 149 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Standardized Response Time/ResponseTime(StandardTran.) 4 msec
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Dialog Overview/Users Logged On/UsersLoggedIn 2 -
Background 1 50 SAP CCMS Monitor Templates/Background Processing/System Wide/Background/BackgroundService/SystemWideFreeBPWP 19 WPs
sapkax_KAX_03 1 60 SAP CCMS Monitor Templates/Entire System/KAX/Application Server/sapkax_KAX_03/R3Services/Update/Update/AbapErrorInUpdate - - None:
Spool 3 90 SAP CCMS Monitor Templates/Spool System/System Wide/Spool/SpoolService/sapkap_KAP_00/Status - - 2021-03-09 21:46:32: Spool server is not active. Output request waiting!
Spool 1 90 SAP CCMS Monitor Templates/Spool System/System Wide/Spool/SpoolService/sapkax_KAX_03/Status - - 2021-03-09 21:46:32: Spool server is active
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/ErrorsInWpSPO 0 -
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/ErrorFreqInWpSPO 0.0 /min
sapkax_KAX_03 1 50 SAP CCMS Monitor Templates/Spool System/Spool Servers/KAX\sapkax_KAX_03/Spool/SpoolService/QueueLength 0 %
sapkax_KAX_03 1 60 SAP CCMS Technical Expert Monitors/All Contexts on Local Application Server/sapkax_KAX_03/InstanceAsTask/Status - - 2021-03-09 21:48:07: Active since 26.11.2020, 15:01:32: Last test: 09.03.2021, 22:48:07

Soweit ich den output verstehe, verwendet er ja in beien Fällen “sap.value” in beiden Fällen für die discovery. Aber auf dem neuen System findet er nix.

andreas-doehler · March 10, 2021, 6:38am

Falls der Check auf dem Produktionssystem nicht angepasst wurde bin ich hier auch erstmal mit meinem Latein am Ende warum er nix findet.

fbotte · March 10, 2021, 3:48pm

Nicht das ich wüsste das da was angepasst ist.
Ich hab jetzt mal ein Ticket beim Support aufgemacht.
Ich schreib dann was das Problem war.

fbotte · March 29, 2021, 10:18am

Ich bin noch die Antwort von CheckMK Support schuldig
Um es kurz zu machen, das Problem saß mal wieder vor der Tastatur.
Neben dem Plugin (SAP R/3 Monitoring plugin) durch das die Daten im Piggybackfile bereit gestellt werden, muss auch noch eine Display Regel (SAP R/3 Single Value Inventory) konfiguriert werden.
Die hatte ich nicht drin, weil das bei unserem alten System so scheinbar nicht notwendig war.

Unter Strich war die Lösung eine SAP R/3 Single Value Inventory Regel zu bauen die ALLE vom Agent gelieferten Daten auch anzeigt:

Gruß
Frank

system · March 29, 2022, 10:19am

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact @fayepal if you think this should be re-opened.