Error: Not authenticated in Browser when trying with http://external_ip:nodeport

CMK version:
2.1.0

OS version:
Kubernetes 1.23.1

Error message:

When trying to open checkmk url with external ip of node and node port number, it giving below error
http://34.100.157.53:30035/

{"detail":"Not authenticated"}

I was able to open checkmk application url with version: 2.0.0 when service is exposed as nodePort..However in this case its giving above error..please guide how to solve this issue..

I have followed steps as explained in below video link using helm..
[(247) Episode 26: Monitoring Kubernetes with Checkmk - YouTube](https://www.youtube.com/watch?v=H9AlO98afUE&list=PL8DfRO2DvOK1slgjfTu0hMOnepf1F7ssh&index=29)

All pods are running fine 

nigoyal7@master:~/checkmk-2.1.0-helm$ helm upgrade --install --create-namespace -n checkmk-monitor checkmk tribe29/checkmk -f values.yaml
W0528 14:34:04.053908   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.056895   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.071334   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.073938   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.076297   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.079906   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.082159   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.084740   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0528 14:34:04.088943   12396 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Release "checkmk" has been upgraded. Happy Helming!
NAME: checkmk
LAST DEPLOYED: Sat May 28 14:34:02 2022
NAMESPACE: checkmk-monitor
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
You can access the checkmk `cluster-collector` via:
NodePort:
  export NODE_PORT=$(kubectl get --namespace checkmk-monitor -o jsonpath="{.spec.ports[0].nodePort}" services checkmk-cluster-collector);
  export NODE_IP=$(kubectl get nodes --namespace checkmk-monitor -o jsonpath="{.items[0].status.addresses[0].address}");
  echo http://$NODE_IP:$NODE_PORT
  # Cluster-internal DNS of `cluster-collector`: checkmk-cluster-collector.checkmk-monitor
With the token of the service account named `checkmk-checkmk` in the namespace `checkmk-monitor` you can now issue queries against the `cluster-collector`.
Run the following to fetch its token, resp. ca-certificate:
  export TOKEN=$(kubectl get secret $(kubectl get serviceaccount checkmk-checkmk -o=jsonpath='{.secrets[*].name}' -n checkmk-monitor) -n checkmk-monitor -o=jsonpath='{.data.token}' | base64 --decode);
  export CA_CRT="$(kubectl get secret $(kubectl get serviceaccount checkmk-checkmk -o=jsonpath='{.secrets[*].name}' -n checkmk-monitor) -n checkmk-monitor -o=jsonpath='{.data.ca\.crt}' | base64 --decode)";
  # Note: Quote the variable when echo'ing to preserve proper line breaks: `echo "$CA_CRT"`
To test access you can run:
  curl -H "Authorization: Bearer $TOKEN" http://$NODE_IP:$NODE_PORT/metadata | jq
nigoyal7@master:~/checkmk-2.1.0-helm$ export NODE_PORT=$(kubectl get --namespace checkmk-monitor -o jsonpath="{.spec.ports[0].nodePort}" services checkmk-cluster-collector);
nigoyal7@master:~/checkmk-2.1.0-helm$   export NODE_IP=$(kubectl get nodes --namespace checkmk-monitor -o jsonpath="{.items[0].status.addresses[0].address}");
nigoyal7@master:~/checkmk-2.1.0-helm$ export TOKEN=$(kubectl get secret $(kubectl get serviceaccount checkmk-checkmk -o=jsonpath='{.secrets[*].name}' -n checkmk-monitor) -n checkmk-monitor -o=jsonpath='{.data.token}' | base64 --decode);
nigoyal7@master:~/checkmk-2.1.0-helm$   export CA_CRT="$(kubectl get secret $(kubectl get serviceaccount checkmk-checkmk -o=jsonpath='{.secrets[*].name}' -n checkmk-monitor) -n checkmk-monitor -o=jsonpath='{.data.ca\.crt}' | base64 --decode)";
nigoyal7@master:~/checkmk-2.1.0-helm$ echo http://$NODE_IP:$NODE_PORT
http://10.160.0.12:30035
nigoyal7@master:~/checkmk-2.1.0-helm$ curl -H "Authorization: Bearer $TOKEN" http://$NODE_IP:$NODE_PORT/metadata | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2527  100  2527    0     0  66500      0 --:--:-- --:--:-- --:--:-- 66500
{
  "cluster_collector_metadata": {
    "node": "worker-1",
    "host_name": "checkmk-cluster-collector-f76777f88-7qzwd",
    "container_platform": {
      "os_name": "alpine",
      "os_version": "3.15.0",
      "python_version": "3.10.2",
      "python_compiler": "GCC 10.3.1 20211027"
    },
    "checkmk_kube_agent": {
      "project_version": "1.0.0"
    }
  },
  "node_collector_metadata": [
    {
      "node": "worker-3",
      "host_name": "checkmk-node-collector-machine-sections-plfwz",
      "container_platform": {
        "os_name": "alpine",
        "os_version": "3.15.0",
        "python_version": "3.10.2",
        "python_compiler": "GCC 10.3.1 20211027"
      },
      "checkmk_kube_agent": {
        "project_version": "1.0.0"
      },
      "collector_type": "Machine Sections",
      "components": {
        "cadvisor_version": null,
        "checkmk_agent_version": "2.1.0b1"
      }
    },
    {
      "node": "worker-3",
      "host_name": "checkmk-node-collector-container-metrics-ljftl",
      "container_platform": {
        "os_name": "alpine",
        "os_version": "3.15.0",
        "python_version": "3.10.2",
        "python_compiler": "GCC 10.3.1 20211027"
      },
      "checkmk_kube_agent": {
        "project_version": "1.0.0"
      },
      "collector_type": "Container Metrics",
      "components": {
        "cadvisor_version": "v0.43.0",
        "checkmk_agent_version": null
      }
    },
    {
      "node": "worker-2",
      "host_name": "checkmk-node-collector-machine-sections-wcxc4",
      "container_platform": {
        "os_name": "alpine",
        "os_version": "3.15.0",
        "python_version": "3.10.2",
        "python_compiler": "GCC 10.3.1 20211027"
      },
      "checkmk_kube_agent": {
        "project_version": "1.0.0"
      },
      "collector_type": "Machine Sections",
      "components": {
        "cadvisor_version": null,
        "checkmk_agent_version": "2.1.0b1"
      }
    },
    {
      "node": "worker-1",
      "host_name": "checkmk-node-collector-machine-sections-jgzrk",
      "container_platform": {
        "os_name": "alpine",
        "os_version": "3.15.0",
        "python_version": "3.10.2",
        "python_compiler": "GCC 10.3.1 20211027"
      },
      "checkmk_kube_agent": {
        "project_version": "1.0.0"
      },
      "collector_type": "Machine Sections",
      "components": {
        "cadvisor_version": null,
        "checkmk_agent_version": "2.1.0b1"
      }
    },
    {
      "node": "worker-1",
      "host_name": "checkmk-node-collector-container-metrics-q8845",
      "container_platform": {
        "os_name": "alpine",
        "os_version": "3.15.0",
        "python_version": "3.10.2",
        "python_compiler": "GCC 10.3.1 20211027"
      },
      "checkmk_kube_agent": {
        "project_version": "1.0.0"
      },
      "collector_type": "Container Metrics",
      "components": {
        "cadvisor_version": "v0.43.0",
        "checkmk_agent_version": null
      }
    },
    {
      "node": "worker-2",
      "host_name": "checkmk-node-collector-container-metrics-bbx9t",
      "container_platform": {
        "os_name": "alpine",
        "os_version": "3.15.0",
        "python_version": "3.10.2",
        "python_compiler": "GCC 10.3.1 20211027"
      },
      "checkmk_kube_agent": {
        "project_version": "1.0.0"
      },
      "collector_type": "Container Metrics",
      "components": {
        "cadvisor_version": "v0.43.0",
        "checkmk_agent_version": null
      }
    }
  ]
}
nigoyal7@master:~/checkmk-2.1.0-helm$ kubectl get pods -n checkmk-monitor
NAME                                             READY   STATUS    RESTARTS      AGE
checkmk-cluster-collector-f76777f88-7qzwd        1/1     Running   1 (11m ago)   174m
checkmk-node-collector-container-metrics-bbx9t   2/2     Running   0             10m
checkmk-node-collector-container-metrics-ljftl   2/2     Running   0             10m
checkmk-node-collector-container-metrics-q8845   2/2     Running   2 (11m ago)   174m
checkmk-node-collector-machine-sections-jgzrk    1/1     Running   1 (11m ago)   174m
checkmk-node-collector-machine-sections-plfwz    1/1     Running   0             10m
checkmk-node-collector-machine-sections-wcxc4    1/1     Running   0             10m
nigoyal7@master:~/checkmk-2.1.0-helm$
nigoyal7@master:~/checkmk-2.1.0-helm$ kubectl get svc -n checkmk-monitor
NAME                        TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
checkmk-cluster-collector   NodePort   10.97.189.177   <none>        8080:30035/TCP   3h9m


Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

Can you try http://external_ip:nodeport/docs ?

This should give you the Fast API endpoints page where you can authorize with the token.

Hi David,

Please guide how to configure at site http://external_ip:nodeport/docs

Is giving page as attached in screenshot,

Which Token, kubernetes_service_host and kubernetes_service_port_https i need to use,
as i try with default secret token from kubernetes cluster but its giving me error as shown in image attached…

Also please guide how i can access checkmk site further…

Thanks
Nitin Goyal

As per the status of the pods here, all looks okay. The helm chart was deployed successfully.
Now, you just need to do the configuration in Checkmk as described in Chapter 3.2 Monitoring Kubernetes
This should work out of the box provided the Checkmk server can also access those IP/Port/URL.

Hi David,

Mentioned link for chapter 3.2 is to configure the checkmk on GUI, However i am not able to open checkmk site using http://external_ip:nodeport,

Getting Error:
When trying to open checkmk url with external ip of node and node port number, it giving below error
http://34.100.157.53:30035/

{“detail”:“Not authenticated”}

Please guide how to solve this issue, authenticate with checmk and access checkmk GUI…

Thanks
Nitin Goyal

Mentioned link for chapter 3.2 is to configure the checkmk on GUI, However i am not able to open checkmk site using http://external_ip:nodeport,

I think there is a misunderstanding here. “http://external_ip:nodeport” will not give you a Checkmk site. Its an API endpoint for cluster-collector to which your Checkmk site should connect.

Your Checkmk site URL should be separate. Where have you installed Checkmk ? On-prem ?

Hi David,

I have followed steps to configure monitoring in Kubernetes as explained in below link and you tube video link…

New tab (youtube.com)

I want to setup checkmk in kubernetes cluster, please guide if any additional document i need to follow to configure checkmk server in kubernetes cluster to access GUI and connect with collectors to get the metrics…

Thanks
Nitin Goyal

I want to setup checkmk in kubernetes cluster, please guide if any additional document i need to follow to configure checkmk server in kubernetes cluster to access GUI and connect with collectors to get the metrics…
In that case, you have to follow this Deploying Checkmk in Kubernetes | Checkmk . Once Checkmk monitoring instance has been deployed in your K8 setup (please read the disclaimer as well) then you just need to follow the Youtube tutorial to configure the connection to the cluster-collector.

Hi David,

Which latest version image i need to use for checkmk to deploy in kubernetes cluster…

in mentioned link check-mk-enterprise:2.0.0p5

Also in checkmk below download site, latest version: 2.1.0 on OS ubuntu
Download the Checkmk

Please confirm which image version i need to proceed to deploy in kubernetes cluster and connect with collectors as mentioned in chapter 3.1, 3.2 of Kubernetes monitoring…

Thanks
Nitin Goyal

The new Kubernetes monitoring only works with 2.1.0.

Hi Sudhir,

Is there any docker image for checkmk version 2.1.0, to deploy as pods in kubernetes cluster, please share link to download
or do i need to download 2.1.0 for ubuntu OS which is not docker image i guess, in this case please provide documentation link to install the package on ubuntu OS, so that i can install on ubuntu OS on one of kubernetes master or worker node which is hosted on ubuntu OS and further will able to connect with collector pods of checkmk…

Thanks
Nitin Goyal

Are you using CEE or CRE?
The version list for CEE is available here
https://registry.checkmk.com/v2/enterprise/check-mk-enterprise/tags/list

I am using Docker (Contain Runtime Engine)

I need to test checkmk on enterprise free edition image,
Please confirm if i can download docker image: checkmk/check-mk-free:2.1.0 as mentioned in below link

and further follow below guide to deploy as pod in kubernetes cluster

Please confirm if like this way, checkmk server will be able to connect with checkmk collector pods…and i will be able to access checkmk GUI and also able to get kubernetes cluster metrics

Thanks
Nitin Goyal

The images are located here for CFE edition:
https://hub.docker.com/r/checkmk/check-mk-free/tags

Hi ,

I am to access checkmk GUI, however getting below error, services discovery getting failed…after configuration as explained in below You tube link:

I have checked pods and services are running fine…

nigoyal7@master:~$ kubectl get pods -n checkmk
NAME                       READY   STATUS    RESTARTS      AGE
checkmk-76776c8477-bb76x   1/1     Running   1 (14h ago)   16h
nigoyal7@master:~$ kubectl get pods -n checkmk-monitor
NAME                                             READY   STATUS    RESTARTS       AGE
checkmk-cluster-collector-f76777f88-c8wh4        1/1     Running   6 (14h ago)    2d23h
checkmk-node-collector-container-metrics-q8845   2/2     Running   21 (74s ago)   3d16h
checkmk-node-collector-machine-sections-jgzrk    1/1     Running   14 (57s ago)   3d16h
nigoyal7@master:~$ kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   19h
nigoyal7@master:~$ kubectl get svc -n checkmk
NAME      TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
checkmk   NodePort   10.102.88.223   <none>        80:32038/TCP,443:31118/TCP   10d
nigoyal7@master:~$ kubectl get svc -n checkmk-monitor
NAME                        TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
checkmk-cluster-collector   NodePort   10.97.189.177   <none>        8080:30035/TCP   3d16h

Please help to solve this issue…

Thanks
Nitin Goyal

Can you login to Checkmk container and do a simple ping or telnet to IP and port ?

Hi David,

Please specify which container and which checkmk pod, i need to test ping or telnet from…

Below is checkmk server pod…please confirm if i need to test from there…and also confirm to which IP and Port, i need to test ping or telnet

nigoyal7@master:~/edureka-project$ kubectl get pods -n checkmk
NAME                       READY   STATUS    RESTARTS      AGE
checkmk-76776c8477-bb76x   1/1     Running   2 (44m ago)   19h

Thanks
Nitin Goyal

This depends on what have configured in the Checkmk Kubernetes rule.

Hi David,

I have tested ping and telnet with kubernetes clusterIP service: 10.96.0.1 , port: 443 and API server endpoint: 10.160.0.12 port: 6443, I tried configuring both in kubernetes rule under API server connection server → endpoint as shown in attached screenshot, however either of them are not working…please check the output from checkmk server pod and screenshot attached…


nigoyal7@master:~/edureka-project$ kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   35m

nigoyal7@master:~/edureka-project$ sudo cat /etc/kubernetes/admin.conf | grep 10.
    server: https://10.160.0.12:6443

nigoyal7@master:~/edureka-project$ kubectl get  pods -n checkmk
NAME                       READY   STATUS    RESTARTS      AGE
checkmk-76776c8477-bb76x   1/1     Running   2 (57m ago)   19h

nigoyal7@master:~/edureka-project$ kubectl exec -it checkmk-76776c8477-bb76x -n checkmk -- bash
root@checkmk-76776c8477-bb76x:/# ping 10.96.0.1
PING 10.96.0.1 (10.96.0.1) 56(84) bytes of data.
^C
--- 10.96.0.1 ping statistics ---
13 packets transmitted, 0 received, 100% packet loss, time 287ms

root@checkmk-76776c8477-bb76x:/# ping 10.160.0.12
PING 10.160.0.12 (10.160.0.12) 56(84) bytes of data.
64 bytes from 10.160.0.12: icmp_seq=1 ttl=63 time=1.56 ms
64 bytes from 10.160.0.12: icmp_seq=2 ttl=63 time=0.336 ms
^C
--- 10.160.0.12 ping statistics ---

root@checkmk-76776c8477-bb76x:/# telnet 10.96.0.1  443
Trying 10.96.0.1...
Connected to 10.96.0.1.
Escape character is '^]'.
^C
root@checkmk-76776c8477-bb76x:/# telnet 10.160.0.12  6443
Trying 10.160.0.12...
Connected to 10.160.0.12.
Escape character is '^]'.
^C


Thanks
Nitin Goyal

Have you also uploaded the certificate to the trust-store ? If not, then try disabling certificate verification.