Node-collector-machine-sections Pods getting in CrashbackLoopOff in Kubernetes

CMK version: 2.1.0
**OS version:ubuntu for worker nodes

Error message:

nigoyal7@master:~/checkmk_kube_agent/deploy/kubernetes$ kubectl get pods -n checkmk-monitoring
NAME                                     READY   STATUS             RESTARTS      AGE
cluster-collector-58888c97bb-bdx87       1/1     Running            0             5m19s
node-collector-container-metrics-tcbwh   2/2     Running            0             5m19s
node-collector-container-metrics-xnfg7   2/2     Running            0             5m19s
node-collector-machine-sections-d4g98    0/1     CrashLoopBackOff   5 (57s ago)   5m19s
node-collector-machine-sections-k9v4x    0/1     CrashLoopBackOff   5 (77s ago)   5m19s
nigoyal7@master:~/checkmk_kube_agent/deploy/kubernetes$ kubectl logs node-collector-machine-sections-d4g98 -n checkmk-monitoring
Traceback (most recent call last):
  File "/usr/local/bin/checkmk-machine-sections-collector", line 8, in <module>
    sys.exit(main_machine_sections())
  File "/usr/local/lib/python3.10/site-packages/checkmk_kube_agent/send_metrics.py", line 471, in _main
    worker(session, cluster_collector_base_url, headers, verify)
  File "/usr/local/lib/python3.10/site-packages/checkmk_kube_agent/send_metrics.py", line 376, in machine_sections_worker
    returncode = process.wait(5)
  File "/usr/local/lib/python3.10/subprocess.py", line 1204, in wait
    return self._wait(timeout=timeout)
  File "/usr/local/lib/python3.10/subprocess.py", line 1930, in _wait
    raise TimeoutExpired(self.args, timeout)
subprocess.TimeoutExpired: Command '['/usr/local/bin/check_mk_agent']' timed out after 5 seconds

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

Hi,

I am trying to deploy checkmk in Kubernetes cluster, I have followed below guide to setup monitoring in Kubernetes…DaemonSet node-collector-machine-sections is getting into Crashbackloop Off…

As per error, i think i need to follow additional guide to deploy check_mk_agent…

I am using ubuntu OS, for worker and master nodes…in kubernetes cluster…

Please let me know if i need to install check_mk_agent in ubuntu OS(worker and master nodes) or i can deploy check_mk_agent as pod in kubernetes cluster and connect with checkmk collectors

Please provide the guide link, which i need to follow to deploy check_mk_agent in kubernetes to solve this issue

Thanks
Nitin Goyal

You can follow the steps as mentioned here:

There is a link to the YouTube video where the configuration is explained in full detail.

Hi Sudhir,

This issue is resolved after following steps explained in below you tube link by using helm method of deploying…

Thanks & Regards,
Nitin Goyal

Good to know that it worked for you. Please mark this as solved.

Hi Sudhir,

Can you please guide where i can mark this thread as solved …

Thanks
Nitin Goyal

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.