Mk_docker.py not discovering anything?(continue)

Troubleshooting
1

I have exactly same issue where check_mk server 1.6.0p16 raw version,
will not show docker info.
like past locked post-

docker plugins on monitored host work from cli ok.
and check_mk server return data ok from
“OMD[host]:~$ cmk -d host4” command.

but in check_mk server gui panel i get these errors:
Error: mk_docker requires the docker library. Please install it on the monitored system (pip install docker). CRIT
and
UNKN - Containers: count not present in agent output UNKN , Running: count not present in agent output UNKN , Paused: count not present in agent output UNKN , Stopped: count not present in agent output UNKN

any idea?

docker-error
docker-error1402×83 19.4 KB

2

Hi,
did you install the needed python docker modul (not only the plugin) on your target?
Please find the needed information here:
https://checkmk.de/cms_monitoring_docker.html
Cheers,
Christian

3

yes i did installed it (pip install docker)
and did try python import docker all is fine(else i assume the plugin wasn’t working from cli…)

4

If you have the same issues as in the mentioned post it would be good to see your output from the “mk_docker.py” script.
Or the agent output as you should get it with “cmk -d host4”

5

from the cmk -h host4
<<<check_mk>>>
Version: 1.6.0p16
AgentOS: linux
Hostname: host4
AgentDirectory: /etc/check_mk
DataDirectory: /var/lib/check_mk_agent
SpoolDirectory: /var/lib/check_mk_agent/spool
PluginsDirectory: /usr/lib/check_mk_agent/plugins
LocalDirectory: /usr/lib/check_mk_agent/local

<<<docker_node_info:sep(124)>>>
@docker_version_info|{}
{“Critical”: “Error: mk_docker requires the docker library. Please install it on the monitored system (pip install docker).”}
<<<>>>
<<<docker_container_status>>>
{“Status”:“exited”,“Running”:false,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:0,“ExitCode”:100,“Error”:"",“StartedAt”:“2020-07-21T21:01:18.318045916Z”,“FinishedAt”:“2020-07-21T21:01:21.933180076Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“4579c70945248736753d91337a8121e0eb142a06223cb79a4e363bd1e14d93f5”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{},“SandboxKey”:"/var/run/docker/netns/4579c7094524",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:"",“Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:"",“IPPrefixLen”:0,“IPv6Gateway”:"",“MacAddress”:"",“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:"",“Gateway”:"",“IPAddress”:"",“IPPrefixLen”:0,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:"",“DriverOpts”:null}}}

<<<<>>>>
<<<<367ec1ac0625>>>>
<<<docker_container_status>>>
{“Status”:“exited”,“Running”:false,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:0,“ExitCode”:100,“Error”:"",“StartedAt”:“2020-07-21T20:55:53.035956717Z”,“FinishedAt”:“2020-07-21T20:55:56.94505554Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“8d7611224ff0510c9a6ae04fc7cc73d472fab51e891110748d68696e3119aa4d”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{},“SandboxKey”:"/var/run/docker/netns/8d7611224ff0",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:"",“Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:"",“IPPrefixLen”:0,“IPv6Gateway”:"",“MacAddress”:"",“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:"",“Gateway”:"",“IPAddress”:"",“IPPrefixLen”:0,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:"",“DriverOpts”:null}}}

<<<<>>>>
<<<>>>
<<<docker_container_status>>>
{“Status”:“running”,“Running”:true,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:29519,“ExitCode”:0,“Error”:"",“StartedAt”:“2020-04-29T12:07:42.953994418Z”,“FinishedAt”:“0001-01-01T00:00:00Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{“com.docker.compose.config-hash”:“3b6821a7ca74520e5b578b26f44ca8cc93ae136e74d8b003f6aa23c420d6e6f5”,“com.docker.compose.container-number”:“1”,“com.docker.compose.oneoff”:“False”,“com.docker.compose.project”:“infrastructure”,“com.docker.compose.service”:“cron”,“com.docker.compose.version”:“1.24.0”}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“375109dd8e44d76db2ef86cc1d3942483fec268f627f7f21ca68ab01a8e6543c”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{},“SandboxKey”:"/var/run/docker/netns/375109dd8e44",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:“8e176dacf0dbe5fb12d3d51e9a60c6201cb2bb7fd5870634f71349f616a120e2”,“Gateway”:“172.17.0.1”,“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:“172.17.0.7”,“IPPrefixLen”:16,“IPv6Gateway”:"",“MacAddress”:“02:42:ac:11:00:07”,“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:“8e176dacf0dbe5fb12d3d51e9a60c6201cb2bb7fd5870634f71349f616a120e2”,“Gateway”:“172.17.0.1”,“IPAddress”:“172.17.0.7”,“IPPrefixLen”:16,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:“02:42:ac:12:00:04”,“DriverOpts”:null}}}

<<<<>>>>
<<<<2d501225438d>>>>
<<<docker_container_status>>>
{“Status”:“running”,“Running”:true,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:15877,“ExitCode”:0,“Error”:"",“StartedAt”:“2020-04-09T11:39:40.61448215Z”,“FinishedAt”:“0001-01-01T00:00:00Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{“com.docker.compose.config-hash”:“067dab4b74c3bf2047a3262a1aa1773ea31314d24ba7baa9c43f996f23c90aff”,“com.docker.compose.container-number”:“1”,“com.docker.compose.oneoff”:“False”,“com.docker.compose.project”:“infrastructure”,“com.docker.compose.service”:“php”,“com.docker.compose.version”:“1.24.0”}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“ce07f5b2cb99d63175d9aa94e4a937867de4580b27b5338e32843ed1a4c80b26”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{“9000/tcp”:[{“HostIp”:“0.0.0.0”,“HostPort”:“9000”}]},“SandboxKey”:"/var/run/docker/netns/ce07f5b2cb99",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:“40fe8bdd244544c535310ee9b6a2eae7f6eefba70709712379aae35a210b72b5”,“Gateway”:“172.17.0.1”,“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:“172.17.0.5”,“IPPrefixLen”:16,“IPv6Gateway”:"",“MacAddress”:“02:42:ac:11:00:05”,“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:“40fe8bdd244544c535310ee9b6a2eae7f6eefba70709712379aae35a210b72b5”,“Gateway”:“172.17.0.1”,“IPAddress”:“172.17.0.5”,“IPPrefixLen”:16,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:“02:42:ac:11:00:05”,“DriverOpts”:null}}}

<<<<>>>>
<<<>>>
<<<docker_container_status>>>
{“Status”:“running”,“Running”:true,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:14553,“ExitCode”:0,“Error”:"",“StartedAt”:“2020-04-01T18:31:19.111802855Z”,“FinishedAt”:“0001-01-01T00:00:00Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{“com.docker.compose.config-hash”:“95396c828165f61db8df40a7a1961f5a1dc65c8f790435980421291f1da53c03”,“com.docker.compose.container-number”:“1”,“com.docker.compose.oneoff”:“False”,“com.docker.compose.project”:“infrastructure”,“com.docker.compose.service”:“phpmautic”,“com.docker.compose.version”:“1.24.0”}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“b926cf2899ad1c414fec59a00e722a5885c4f4595c72c60c6075b226b5351837”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{“9000/tcp”:null},“SandboxKey”:"/var/run/docker/netns/b926cf2899ad",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:“bb524548dd131f63a40fa86d5ec511b09e83ad09c79fb1a413e74eb9a4b2b253”,“Gateway”:“172.17.0.1”,“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:“172.17.0.4”,“IPPrefixLen”:16,“IPv6Gateway”:"",“MacAddress”:“02:42:ac:11:00:04”,“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:“bb524548dd131f63a40fa86d5ec511b09e83ad09c79fb1a413e74eb9a4b2b253”,“Gateway”:“172.17.0.1”,“IPAddress”:“172.17.0.4”,“IPPrefixLen”:16,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:“02:42:ac:11:00:09”,“DriverOpts”:null}}}

<<<<>>>>
<<<>>>
<<<docker_container_status>>>
{“Status”:“running”,“Running”:true,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:14526,“ExitCode”:0,“Error”:"",“StartedAt”:“2020-04-01T18:31:19.297919915Z”,“FinishedAt”:“0001-01-01T00:00:00Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{“com.docker.compose.config-hash”:“83e8280eea53847f563df1540e8701824cda55ec8686551086d509a9642baf4f”,“com.docker.compose.container-number”:“1”,“com.docker.compose.oneoff”:“False”,“com.docker.compose.project”:“infrastructure”,“com.docker.compose.service”:“phptraderstocht”,“com.docker.compose.version”:“1.24.0”}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“5263e9e3c2fb1159ab44ad1ea3e24ae544a4b74f61abf4b449c54bbff84f7125”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{“9000/tcp”:null},“SandboxKey”:"/var/run/docker/netns/5263e9e3c2fb",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:“dfd4bef0c615f786af597ba4ba1644926d4062d8b16585fbf7939a8e0a84bcac”,“Gateway”:“172.17.0.1”,“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:“172.17.0.2”,“IPPrefixLen”:16,“IPv6Gateway”:"",“MacAddress”:“02:42:ac:13:00:09”,“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:“dfd4bef0c615f786af597ba4ba1644926d4062d8b16585fbf7939a8e0a84bcac”,“Gateway”:“172.17.0.1”,“IPAddress”:“172.17.0.2”,“IPPrefixLen”:16,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:“02:42:ac:13:00:09”,“DriverOpts”:null}}}

<<<<>>>>
<<<>>>
<<<docker_container_status>>>
{“Status”:“running”,“Running”:true,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:7356,“ExitCode”:0,“Error”:"",“StartedAt”:“2020-03-26T19:21:41.470192965Z”,“FinishedAt”:“0001-01-01T00:00:00Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{“com.docker.compose.config-hash”:“eab11decfc410380338cc9cb9cf62f77abd6fbe1ebc39d3cd7206f52a797fd45”,“com.docker.compose.container-number”:“1”,“com.docker.compose.oneoff”:“False”,“com.docker.compose.project”:“infrastructure”,“com.docker.compose.service”:“nginx”,“com.docker.compose.version”:“1.24.0”,“maintainer”:“NGINX Docker Maintainers docker-maint@nginx.com”}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“b6d230ac01f9d697abb7bffbf916a34ce7c6296114884a2e1ee00aecc28f493f”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{“443/tcp”:[{“HostIp”:“0.0.0.0”,“HostPort”:“443”}],“80/tcp”:[{“HostIp”:“0.0.0.0”,“HostPort”:“80”}]},“SandboxKey”:"/var/run/docker/netns/b6d230ac01f9",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:“41bfbe343aa9ddd1d765b921c25b651f0566c95c6af0059a4d3a4d0a00ce4933”,“Gateway”:“172.17.0.1”,“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:“172.17.0.8”,“IPPrefixLen”:16,“IPv6Gateway”:"",“MacAddress”:“02:42:ac:41:00:08”,“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:“41bfbe343aa9ddd1d765b921c25b651f0566c95c6af0059a4d3a4d0a00ce4933”,“Gateway”:“172.17.0.1”,“IPAddress”:“172.17.0.8”,“IPPrefixLen”:16,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:“02:42:ac:10:00:18”,“DriverOpts”:null}}}

<<<<>>>>
<<<<74e8205281fb>>>>
<<<docker_container_status>>>
{“Status”:“running”,“Running”:true,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:4148,“ExitCode”:0,“Error”:"",“StartedAt”:“2020-04-07T07:22:11.219960112Z”,“FinishedAt”:“2020-04-07T07:17:25.737920707Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{“com.docker.compose.config-hash”:“a7341bf489ff31e3d5964b5e758c5b654245a060938c9391e32ee0381f8bf65e”,“com.docker.compose.container-number”:“1”,“com.docker.compose.oneoff”:“False”,“com.docker.compose.project”:“infrastructure”,“com.docker.compose.service”:“resque”,“com.docker.compose.version”:“1.24.0”}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“b943f600f8e7f3506d0cf13c7ba9c810084ba0c03e7dea6c819112c2c33527a1”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{},“SandboxKey”:"/var/run/docker/netns/b943f600f8e7",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:“a6159f7663b9ff09b8a22d0b93f73fb6f9cd42ff23452c232941af2dd1e815d3”,“Gateway”:“172.17.0.1”,“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:“172.17.0.6”,“IPPrefixLen”:16,“IPv6Gateway”:"",“MacAddress”:“02:42:ac:11:00:06”,“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:“a6159f7663b9ff09b8a22d0b93f73fb6f9cd42ff23452c232941af2dd1e815d3”,“Gateway”:“172.17.0.1”,“IPAddress”:“172.17.0.6”,“IPPrefixLen”:16,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:“02:42:ac:12:00:01”,“DriverOpts”:null}}}

<<<<>>>>
<<<>>>
<<<docker_container_status>>>
{“Status”:“running”,“Running”:true,“Paused”:false,“Restarting”:false,“OOMKilled”:false,“Dead”:false,“Pid”:4081,“ExitCode”:0,“Error”:"",“StartedAt”:“2020-02-14T17:06:26.68357224Z”,“FinishedAt”:“0001-01-01T00:00:00Z”}
<<<docker_container_node_name>>>
host4
<<<docker_container_labels>>>
{“com.docker.compose.config-hash”:“3801ea90dbf3677f7c4389e4bec5dec32e0f339a6b1621cc7490a93fb3561937”,“com.docker.compose.container-number”:“1”,“com.docker.compose.oneoff”:“False”,“com.docker.compose.project”:“infrastructure”,“com.docker.compose.service”:“redis”,“com.docker.compose.version”:“1.24.0”}
<<<docker_container_network>>>
{“Bridge”:"",“SandboxID”:“dcef7f9e18bd54843aa576a1e2787f3fb226a9eccb8fca37c81902687df2955d”,“HairpinMode”:false,“LinkLocalIPv6Address”:"",“LinkLocalIPv6PrefixLen”:0,“Ports”:{“6379/tcp”:[{“HostIp”:“0.0.0.0”,“HostPort”:“6379”}]},“SandboxKey”:"/var/run/docker/netns/dcef7f9e18bd",“SecondaryIPAddresses”:null,“SecondaryIPv6Addresses”:null,“EndpointID”:“e220aaab9db4180a24498af8ca0837bc3597e275cdc491248554820124ea0a39”,“Gateway”:“172.17.0.1”,“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“IPAddress”:“172.17.0.3”,“IPPrefixLen”:16,“IPv6Gateway”:"",“MacAddress”:“02:42:ac:12:00:09”,“Networks”:{“bridge”:{“IPAMConfig”:null,“Links”:null,“Aliases”:null,“NetworkID”:“1d4bb1458a654b1344d4b9216db065aae73a463743fba38a721deead216bed7f”,“EndpointID”:“e220aaab9db4180a24498af8ca0837bc3597e275cdc491248554820124ea0a39”,“Gateway”:“172.17.0.1”,“IPAddress”:“172.17.0.3”,“IPPrefixLen”:16,“IPv6Gateway”:"",“GlobalIPv6Address”:"",“GlobalIPv6PrefixLen”:0,“MacAddress”:“02:42:ac:12:00:13”,“DriverOpts”:null}}}

6

any idea what can be wrong?

7

The first lines show this:

<<<docker_node_info:sep(124)>>>
@docker_version_info|{}
{“Critical”: “Error: mk_docker requires the docker library. Please install it on the monitored system (pip install docker).”}
<<<>>>

Looks like that your python environment is not set corretly.

8

Neben dem was @ChristianM schon geschrieben hat wie sehen die Zeilen
<<<<>>>> wirklich in deinem Output aus?
Zwischen den vier Spitzen Klammern steht normal der Name das Containers.
Für diesen Namen muss dann ein Host im
Monitoring vorhanden sein. Diesem sollten auch Checks zugeordnet sein.

9

Thanks @ChristianM,
apparently i had to run the ‘pip install docker’ from root user.

10

@andreas-doehler from what i understand in your reply the name inside <<<<>>>> need to be the containers name and not the monitored host?
maybe that explain why i only limited amount of docker monitors and error 500(see image)?

Screenshot from 2020-09-18 11-49-14
Screenshot from 2020-09-18 11-49-141399×254 56.3 KB

if so where can i change that ?
obviously container name is different then monitored host name on most deployment.

11

Sorry for the German answer :wink:
The docker node info UNKN is result of the error message @ChristianM mentioned.
You need to create host objects inside you CMK instance with the name of the container. They can be created without IP. You can also set agent and SNMP to none. Then only the piggyback data collected from your docker host is used for the services on this object.

12

Thanks @andreas-doehler i got the idea,
one issue is when the docker plugin pickup data from docker compose
it aggregate all monitored host in one folder name
then when creating docker host for container in cmk server
it says host already exist or it pickup only one container
any way around that?

13

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.