We are using Kubernetes with Rancher. We are trying to use CheckMK to monitor our production Kubernetes Clusters.
I am getting the following error in CheckMK:
Cannot update tree(!), Found 1 inventory entries, [special_kubernetes] Agent exited with code 1: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({‘Server’: ‘nginx/1.19.2’, ‘Date’: ‘Fri, 23 Apr 2021 07:20:40 GMT’, ‘Content-Type’: ‘application/json’, ‘Content-Length’: ‘1331’, ‘Connection’: ‘keep-alive’, ‘Vary’: ‘Accept-Encoding’, ‘Audit-Id’: ‘0213a5a6-8c67-4f42-a9b2-de0182a364c1’, ‘Cache-Control’: ‘no-cache, private’, ‘X-Content-Type-Options’: ‘nosniff’, ‘Strict-Transport-Security’: ‘max-age=15724800; includeSubDomains’}) HTTP response body: { “paths”: [ “/apis”, “/apis/”, “/apis/apiextensions.k8s.io”, “/apis/apiextensions.k8s.io/v1”, “/apis/apiextensions.k8s.io/v1beta1”, “/healthz”, “/healthz/etcd”, “/healthz/log”, “/healthz/ping”, “/healthz/poststarthook/crd-informer-synced”, “/healthz/poststarthook/generic-apiserver-start-informers”, “/healthz/poststarthook/max-in-flight-filter”, “/healthz/poststarthook/start-apiextensions-controllers”, “/healthz/poststarthook/start-apiextensions-informers”, “/livez”, “/livez/etcd”, “/livez/log”, “/livez/ping”, “/livez/poststarthook/crd-informer-synced”, “/livez/poststarthook/generic-apiserver-start-informers”, “/livez/poststarthook/max-in-flight-filter”, “/livez/poststarthook/start-apiextensions-controllers”, “/livez/poststarthook/start-apiextensions-informers”, “/metrics”, “/openapi/v2”, “/readyz”, “/readyz/etcd”, “/readyz/informer-sync”, “/readyz/log”, “/readyz/ping”, “/readyz/poststarthook/crd-informer-synced”, “/readyz/poststarthook/generic-apiserver-start-informers”, “/readyz/poststarthook/max-in-flight-filter”, “/readyz/poststarthook/start-apiextensions-controllers”, “/readyz/poststarthook/start-apiextensions-informers”, “/readyz/shutdown”, “/version” ] }(!!)
I tried to run the script from the filesystem
bin/python3 share/check_mk/agents/special/agent_kubernetes --debug --token TOKEN --infos nodes --no-cert-check --api-server-endpoint URL
I got a similar error message as I did in the GUI.
kubernetes.client.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({‘Server’: ‘nginx/1.19.2’, ‘Date’: ‘Fri, 23 Apr 2021 08:14:53 GMT’, ‘Content-Type’: ‘application/json’, ‘Content-Length’: ‘1331’, ‘Connection’: ‘keep-alive’, ‘Vary’: ‘Accept-Encoding’, ‘Audit-Id’: ‘59aa21c8-990a-46a2-92f1-0ed052f4353e’, ‘Cache-Control’: ‘no-cache, private’, ‘X-Content-Type-Options’: ‘nosniff’, ‘Strict-Transport-Security’: ‘max-age=15724800; includeSubDomains’})
HTTP response body: {
“paths”: [
“/apis”,
“/apis/”,
…
It is able to authenticate; it has access to them; but doesn’t know how to properly query them and generate services from them.
Found some related threads from 2020 and 2019:
- Kubernetes plugins fails with kubernetes > 1.18 - #10 by arthurlogilab
- Kubernetes plugins upgrade for kubernetes > 1.18? - #9 by martin.schwarz
- Support for kubernetes 1.18.+ · Issue #1153 · kubernetes-client/python · GitHub
- ApiException and 404 response when calling CustomObjectsApi · Issue #948 · kubernetes-client/python · GitHub
Is there any update about support for Kubernetes versions 1.19, 1.20, 1.21 ?
We will soon upgrade Kubernetes to 1.20.
Kubernetes:
v1.19.6 RKE
Check_MK:
Enterprise Edition
2.0.0p1
