Python local checks services don't show on GUI

Troubleshooting
1

CMK version: 2.1.0-2022.04.13 CRE (Docker check-mk-raw)

Error message:
N/A

Description of the issue

I have the bash script testbash.sh with the following content:

#!/bin/bash

echo "0 \"testbash\" - Just a test"

and the python script testpython.sh with the following content:

#!/usr/bin/env python3

print("0 \"testpython\" - Just a test")

On my terminal I can see the following information:

OMD[cmk]:/usr/lib/check_mk_agent/local$ ls -la
total 32
drwxr-xr-x 1 root root 4096 Apr 13 14:56 ./
drwxr-xr-x 1 root root 4096 Apr 13 00:16 ../
-rwxrwxrwx 1 cmk  cmk  3139 Apr 13 14:40 check-adtran-mosaic-elb.sh*
-rwxrwxrwx 1 cmk  cmk  2023 Apr 13 14:40 check-aws-direct-connect.sh*
-rwxrwxrwx 1 cmk  cmk    49 Apr 13 14:56 testbash.sh*
-rwxrwxrwx 1 cmk  cmk    64 Apr 13 14:55 testpython.py*

OMD[cmk]:/usr/lib/check_mk_agent/local$ check_mk_agent | grep -v grep | grep -A10 "<<<local"
<<<local:sep(0)>>>
2 "Adtran Mosaic ELB BizOps Dev" - ELB elb-Adtran doesn't exist or Checkmk couldn't get Adtran Mosaic ELB BizOps Dev instance(s) states from the AWS API.
2 "AWS Direct Connect IPV6" - IPV6 Virtual Interface XXXXXX is missing!
2 "AWS Direct Connect IPV4" - IPV4 Virtual Interface XXXXXX is missing!
0 "testbash" - Just a test
0 "testpython" - Just a test

OMD[cmk]:/usr/lib/check_mk_agent/local$ cmk -IIv --detect-plugins=local localhost
Discovering services and host labels on: localhost
localhost:
+ FETCHING DATA
[TCPFetcher] Execute data source
[PiggybackFetcher] Execute data source
No piggyback files for 'localhost'. Skip processing.
No piggyback files for '127.0.0.1'. Skip processing.
+ ANALYSE DISCOVERED HOST LABELS
SUCCESS - Found no host labels
+ ANALYSE DISCOVERED SERVICES
+ EXECUTING DISCOVERY PLUGINS (1)
  4 local
SUCCESS - Found 4 services

OMD[cmk]:/usr/lib/check_mk_agent/local$ cmk -nv --detect-plugins=local localhost
+ FETCHING DATA
[TCPFetcher] Execute data source
[PiggybackFetcher] Execute data source
No piggyback files for 'localhost'. Skip processing.
No piggyback files for '127.0.0.1'. Skip processing.
AWS Direct Connect IPV4 IPV4 Virtual Interface XXXXXX is missing!
AWS Direct Connect IPV6 IPV6 Virtual Interface XXXXXX is missing!
testbash             Just a test
[agent] Success, execution time 0.0 sec | execution_time=0.000 user_time=0.000 system_time=0.000 children_user_time=0.000 children_system_time=0.000

According to my understanding check_mk_agent provide the right output from both plugins but for some reason cmk can only detect the bash plugin. On the GUI service discovery I can also only see the “testbash” service and NOT the “testpython” service.

Why it can’t detect it?

Output of “cmk --debug -vvn hostname”:~

Checkmk version 2.1.0-2022.04.13
Try license usage history update.
Trying to acquire lock on /omd/sites/cmk/var/check_mk/license_usage/next_run
Got lock on /omd/sites/cmk/var/check_mk/license_usage/next_run
Trying to acquire lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Got lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Next run time has not been reached yet. Abort.
Releasing lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Released lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Releasing lock on /omd/sites/cmk/var/check_mk/license_usage/next_run
Released lock on /omd/sites/cmk/var/check_mk/license_usage/next_run

  • FETCHING DATA
    Source: SourceType.HOST/FetcherType.TCP
    [cpu_tracking] Start [7fde571535e0]
    [TCPFetcher] Fetch with cache settings: DefaultAgentFileCache(localhost, base_path=/omd/sites/cmk/tmp/check_mk/cache, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
    Not using cache (Too old. Age is 3 sec, allowed is 0 sec)
    [TCPFetcher] Execute data source
    Connecting via TCP to 127.0.0.1:6556 (5.0s timeout)
    Detected transport protocol: TransportProtocol.PLAIN (b’<<')
    Reading data from agent
    Write data to cache file /omd/sites/cmk/tmp/check_mk/cache/localhost
    Trying to acquire lock on /omd/sites/cmk/tmp/check_mk/cache/localhost
    Got lock on /omd/sites/cmk/tmp/check_mk/cache/localhost
    Releasing lock on /omd/sites/cmk/tmp/check_mk/cache/localhost
    Released lock on /omd/sites/cmk/tmp/check_mk/cache/localhost
    Closing TCP connection to 127.0.0.1:6556
    [cpu_tracking] Stop [7fde571535e0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=5.329999998211861))]
    Source: SourceType.HOST/FetcherType.PIGGYBACK
    [cpu_tracking] Start [7fde57153190]
    [PiggybackFetcher] Fetch with cache settings: NoCache(localhost, base_path=/omd/sites/cmk/tmp/check_mk/data_source_cache/piggyback, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=True, use_outdated=False, simulation=False)
    Not using cache (Cache usage disabled)
    [PiggybackFetcher] Execute data source
    No piggyback files for ‘localhost’. Skip processing.
    No piggyback files for ‘127.0.0.1’. Skip processing.
    Not using cache (Cache usage disabled)
    [cpu_tracking] Stop [7fde57153190 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.010000001639127731))]
  • PARSE FETCHER RESULTS
    Source: SourceType.HOST/FetcherType.TCP
    <<<check_mk>>> / Transition NOOPParser → HostSectionParser
    <<<cmk_agent_ctl_status:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<checkmk_agent_plugins_lnx:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<labels:sep(0)>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<ps_lnx>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<lnx_if:sep(58)>>> / Transition HostSectionParser → HostSectionParser
    <<<tcp_conn_stats>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<vbox_guest>>> / Transition HostSectionParser → HostSectionParser
    <<<postfix_mailq>>> / Transition HostSectionParser → HostSectionParser
    <<<postfix_mailq_status:sep(58)>>> / Transition HostSectionParser → HostSectionParser
    <<<livestatus_status:sep(59)>>> / Transition HostSectionParser → HostSectionParser
    <<<livestatus_ssl_certs:sep(124)>>> / Transition HostSectionParser → HostSectionParser
    <<<mkeventd_status:sep(0)>>> / Transition HostSectionParser → HostSectionParser
    <<<cmk_site_statistics:sep(59)>>> / Transition HostSectionParser → HostSectionParser
    <<>> / Transition HostSectionParser → HostSectionParser
    <<<omd_status:cached(1649861979,60)>>> / Transition HostSectionParser → HostSectionParser
    <<mknotifyd:sep(0)>> / Transition HostSectionParser → HostSectionParser
    <<<omd_apache:sep(124)>>> / Transition HostSectionParser → HostSectionParser
    <<<omd_info:sep(59)>>> / Transition HostSectionParser → HostSectionParser
    <<local:sep(0)>> / Transition HostSectionParser → HostSectionParser
    No persisted sections
    → Add sections: [‘check_mk’, ‘checkmk_agent_plugins_lnx’, ‘cifsmounts’, ‘cmk_agent_ctl_status’, ‘cmk_site_statistics’, ‘cpu’, ‘df’, ‘diskstat’, ‘job’, ‘kernel’, ‘labels’, ‘livestatus_ssl_certs’, ‘livestatus_status’, ‘lnx_if’, ‘local’, ‘md’, ‘mem’, ‘mkeventd_status’, ‘mknotifyd’, ‘mounts’, ‘nfsmounts’, ‘omd_apache’, ‘omd_info’, ‘omd_status’, ‘postfix_mailq’, ‘postfix_mailq_status’, ‘ps_lnx’, ‘tcp_conn_stats’, ‘uptime’, ‘vbox_guest’]
    Source: SourceType.HOST/FetcherType.PIGGYBACK
    No persisted sections
    → Add sections:
    Received no piggyback data
    Received no piggyback data
    [cpu_tracking] Start [7fde57153310]
    value store: synchronizing
    Trying to acquire lock on /omd/sites/cmk/tmp/check_mk/counters/localhost
    Got lock on /omd/sites/cmk/tmp/check_mk/counters/localhost
    value store: loading from disk
    Releasing lock on /omd/sites/cmk/tmp/check_mk/counters/localhost
    Released lock on /omd/sites/cmk/tmp/check_mk/counters/localhost
    AWS Direct Connect IPV4 IPV4 Virtual Interface XXXXXX is missing!
    AWS Direct Connect IPV6 IPV6 Virtual Interface XXXXXX is missing!
    CPU load 15 min load: 1.14, 15 min load per core: 0.28 (4 cores)
    CPU utilization Total CPU: 24.65%
    Disk IO SUMMARY Read: 1.37 kB/s, Write: 297 kB/s, Latency: 1 millisecond
    Filesystem / 58.51% used (28.69 of 49.03 GB), trend: +18.94 MB / 24 hours
    Interface 2 [eth0], (up), MAC: 6E:6C:91:43:72:74, Speed: 10 GBit/s, In: 1.31 kB/s (<0.01%), Out: 2.54 kB/s (<0.01%)
    Kernel Performance Process Creations: 56.33/s, Context Switches: 15070.56/s, Major Page Faults: 0.00/s, Page Swap in: 0.00/s, Page Swap Out: 0.00/s
    Memory Total virtual memory: 52.9% - 8.03 GB of 15.18 GB, Committed: 138% - 21.02 GB of 15.18 GB virtual memory (warn/crit at 100.00%/150.00% used)(!)
    Mount options of /dev/termination-log Mount options exactly as expected
    Number of threads 1862, Usage: 1.50%
    OMD cmk Event Console Current events: 0, Virtual memory: 190.82 MB, Overall event limit inactive, No hosts event limit active, No rules event limit active, Received messages: 0.00/s, Rule hits: 0.00/s, Rule tries: 0.00/s, Message drops: 0.00/s, Created events: 0.00/s, Client connects: 0.11/s, Rule hit ratio: -, Processing time per message: -, Time per client request: 0.29 ms
    OMD cmk apache 0.69 Requests/s, 0.01 Seconds serving/s, 512.63 B Sent/s
    OMD cmk performance Livestatus version: 2.1.0-2022.04.13, Host checks: 0.0/s, Service checks: 2.4/s
    OMD cmk status running
    Postfix Queue Deferred queue length: 0, Active queue length: 0
    Site cmk statistics Total hosts: 2, Problem hosts: 0, Total services: 67, Problem services: 7
    TCP Connections Established: 2
    Uptime Up since Mar 16 2022 22:16:15, Uptime: 27 days 16 hours
    ingress2 Item not found in monitoring data
    No piggyback files for ‘localhost’. Skip processing.
    No piggyback files for ‘127.0.0.1’. Skip processing.
    [cpu_tracking] Stop [7fde57153310 - Snapshot(process=posix.times_result(user=0.020000000000000018, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.009999997913837433))]
    [agent] Success, execution time 0.0 sec | execution_time=0.010 user_time=0.020 system_time=0.000 children_user_time=0.000 children_system_time=0.000
2

Hi,

did you also test with sys.stdout.write in your python code? Looks like that print only works in an iteractive shell.

Regards, Christian

3

@danielserrao Did you activate the changes (cmk -R) after the discovery (cmk -IIv ...) ? In the output of the discovery it says that there were 4 services found.
When you ran the next command (cmk -nv ...) only 3 local checks are listed.
When discovering services you need to activate the changes, otherwise the new services are not actively monitored.
The output of the agent however looks fine to me.

4

Hi @ChristianM and @lkoenig

Sorry for the late response.

I tried “sys.stdout.write” and got the same result, but it worked after executing “cmk -R” on the CLI. Thanks for the help :).

I just wonder why “cmk -R” needs to be executed when creating new python plugins but for for bash plugins?

5

Glad to hear you got it working.
The local check in bash was probably already inventorized before, meaning you already activated the changes / executed cmk -R when then bash-check was present but the python-check wasn‘t. Maybe the python-check wasn’t executable at some point when the bash-check was. This has nothing to do with the language the check was written in. If the check is executable and produces the right output checkmk will pick it up. But you always have to discover it first and activate the changes to get it into the active monitoring.

6

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.