Kubernetes Monitoring

Micles · May 31, 2022, 7:13pm

**CMK version: 2.1.0b5.cre **
OS version: Ubuntu 20.04

We try to monitor a kubernetes cluster, but we always get this errors.
Is it realy a license problem?

When we try to search for the kubernetes service at the host, there is no output.
We used this manual
(Monitoring Kubernetes)

cmk --debug -vvn

Precompile /omd/sites/mysite/local/share/check_mk/checks/license.include to /omd/sites/mysite/var/check_mk/precompiled_checks/local/license.include
Error in check include file /omd/sites/mysite/local/share/check_mk/checks/license.include: [Errno 2] No such file or directory: '/omd/sites/mysite/local/share/check_mk/checks/license.include'
Error in plugin file /omd/sites/mysite/local/share/check_mk/checks/veeam_o365licenses: [Errno 2] No such file or directory: '/omd/sites/mysite/local/share/check_mk/checks/license.include'
Trying to acquire lock on /omd/sites/mysite/var/check_mk/crashes/base/fb1a164a-e114-11ec-b076-9755f0b37a35/crash.info
Got lock on /omd/sites/mysite/var/check_mk/crashes/base/fb1a164a-e114-11ec-b076-9755f0b37a35/crash.info
Releasing lock on /omd/sites/mysite/var/check_mk/crashes/base/fb1a164a-e114-11ec-b076-9755f0b37a35/crash.info
Released lock on /omd/sites/mysite/var/check_mk/crashes/base/fb1a164a-e114-11ec-b076-9755f0b37a35/crash.info
Traceback (most recent call last):
  File "/omd/sites/mysite/bin/cmk", line 79, in <module>
    errors = config.load_all_agent_based_plugins(check_api.get_check_api_context)
  File "/omd/sites/mysite/lib/python3/cmk/base/config.py", line 1526, in load_all_agent_based_plugins
    errors.extend(load_checks(get_check_api_context, filelist))
  File "/omd/sites/mysite/lib/python3/cmk/base/config.py", line 1606, in load_checks
    did_compile |= load_check_includes(f, check_context)
  File "/omd/sites/mysite/lib/python3/cmk/base/config.py", line 1723, in load_check_includes
    did_compile |= load_precompiled_plugin(include_file_path, check_context)
  File "/omd/sites/mysite/lib/python3/cmk/base/config.py", line 1897, in load_precompiled_plugin
    py_compile.compile(path, precompiled_path, doraise=True)
  File "/omd/sites/mysite/lib/python3.9/py_compile.py", line 142, in compile
    source_bytes = loader.get_data(file)
  File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/omd/sites/mysite/local/share/check_mk/checks/license.include'

web.log

2022-05-31 20:53:47,419 [30] [cmk.web.automations 3004147] 'check_mk --automation try-inventory @noscan myhost.de' returned 'Error in check include file /omd/sites/mysite/local/share/check_mk/checks/license.include: [Errno 2] No such file or directory: '/omd/sites/mysite/local/share/check_mk/checks/license.include'

davidwayne · May 31, 2022, 8:39pm

Why don’t you use 2.1 stable version ?

Micles · May 31, 2022, 9:34pm

This was just a try… rolled back to 2.1 stable.
And found the same line while up/downgrading:
“[Errno 2] No such file or directory: '/omd/sites/mysite/local/share/check_mk/checks/license.include”

Deleted all items in this folder and now i have a better error for kubernetes monitoring.


+ FETCHING DATA
  Source: SourceType.HOST/FetcherType.PROGRAM
[cpu_tracking] Start [7fb76f3a7d60]
[ProgramFetcher] Fetch with cache settings: DefaultAgentFileCache(checkmk.services.staging.mysite.de, base_path=/omd/sites/mysite/tmp/check_mk/data_source_cache/special_kube, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Not using cache (Does not exist)
[ProgramFetcher] Execute data source
Calling: /omd/sites/mysite/share/check_mk/agents/special/agent_kube --pwstore=4@0@stagingkubernetes '--cluster' 'staging' '--token' '**********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************' '--monitored-objects' 'deployments' 'daemonsets' 'statefulsets' 'namespaces' 'nodes' 'pods' '--cluster-aggregation-exclude-node-roles' 'control-plane' 'infra' '--api-server-endpoint' 'https://checkmk.services.staging.mysite.de:443' '--verify-cert-api' '--api-server-proxy' 'FROM_ENVIRONMENT' '--cluster-collector-endpoint' 'http://checkmk.services.staging.mysite.de:30035' '--verify-cert-collector' '--cluster-collector-proxy' 'FROM_ENVIRONMENT'
[cpu_tracking] Stop [7fb76f3a7d60 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.75, children_system=0.11, elapsed=1.030000001192093))]
  Source: SourceType.HOST/FetcherType.PIGGYBACK
[cpu_tracking] Start [7fb76f3a7460]
[PiggybackFetcher] Fetch with cache settings: NoCache(checkmk.services.staging.mysite.de, base_path=/omd/sites/mysite/tmp/check_mk/data_source_cache/piggyback, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=True, use_outdated=False, simulation=False)
Not using cache (Cache usage disabled)
[PiggybackFetcher] Execute data source
No piggyback files for 'checkmk.services.staging.mysite.de'. Skip processing.
Not using cache (Cache usage disabled)
[cpu_tracking] Stop [7fb76f3a7460 - Snapshot(process=posix.times_result(user=0.0, system=0.010000000000000009, children_user=0.0, children_system=0.0, elapsed=0.0))]
+ PARSE FETCHER RESULTS
  Source: SourceType.HOST/FetcherType.PROGRAM
  -> Not adding sections: Agent exited with code 1: 404, Reason: Not Found, Message: None
  Source: SourceType.HOST/FetcherType.PIGGYBACK
No persisted sections
  -> Add sections: []
Received no piggyback data
Received no piggyback data
[cpu_tracking] Start [7fb76f36cc40]
value store: synchronizing
Trying to acquire lock on /omd/sites/mysite/tmp/check_mk/counters/checkmk.services.staging.mysite.de
Got lock on /omd/sites/mysite/tmp/check_mk/counters/checkmk.services.staging.mysite.de
value store: loading from disk
Releasing lock on /omd/sites/mysite/tmp/check_mk/counters/checkmk.services.staging.mysite.de
Released lock on /omd/sites/mysite/tmp/check_mk/counters/checkmk.services.staging.mysite.de
No piggyback files for 'checkmk.services.staging.mysite.de'. Skip processing.
[cpu_tracking] Stop [7fb76f36cc40 - Snapshot(process=posix.times_result(user=0.020000000000000018, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.019999999552965164))]
[special_kube] Agent exited with code 1: 404, Reason: Not Found, Message: None(!!), execution time 1.1 sec | execution_time=1.050 user_time=0.020 system_time=0.010 children_user_time=0.750 children_system_time=0.110 cmk_time_ds=0.170 cmk_time_agent=0.000

There should be something wrong with thr kubernetes cluster…

davidwayne · May 31, 2022, 10:05pm

Could be. Or try on a fresh 2.1.0 site?

r.sander · June 1, 2022, 7:37am

This hints to a file in your local hierarchy. Do a grep -r license.include $OMD_ROOT/local to find the file that tries to include license.include.

system · June 1, 2023, 7:37am

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.