Docker "ApiVersion" is missing

CMK version: 2.2.0p27.cre
OS version: Debian 11

Error message: Plugin exception in MKDockerClient.init: Invalid response from docker daemon: key “ApiVersion” is missing.

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

Checkmk version 2.2.0p27
Updating IPv4 DNS cache for dehya: 172.16.0.66
Trying to acquire lock on /omd/sites/Home/var/check_mk/ipaddresses.cache
Got lock on /omd/sites/Home/var/check_mk/ipaddresses.cache
Releasing lock on /omd/sites/Home/var/check_mk/ipaddresses.cache
Released lock on /omd/sites/Home/var/check_mk/ipaddresses.cache
+ FETCHING DATA
  Source: SourceInfo(hostname='dehya', ipaddress='172.16.0.66', ident='piggyback', fetcher_type=<FetcherType.PIGGYBACK: 4>, source_type=<SourceType.HOST: 1>)
[cpu_tracking] Start [7f183d4111d0]
Read from cache: NoCache(dehya, path_template=/dev/null, max_age=MaxAge(checking=0.0, discovery=0.0, inventory=0.0), simulation=False, use_only_cache=False, file_cache_mode=1)
[PiggybackFetcher] Execute data source
No piggyback files for 'dehya'. Skip processing.
No piggyback files for '172.16.0.66'. Skip processing.
[cpu_tracking] Stop [7f183d4111d0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
+ PARSE FETCHER RESULTS
  HostKey(hostname='dehya', source_type=<SourceType.HOST: 1>)  -> Add sections: []
Received no piggyback data
[cpu_tracking] Start [7f183c958590]
value store: synchronizing
Trying to acquire lock on /omd/sites/Home/tmp/check_mk/counters/dehya
Got lock on /omd/sites/Home/tmp/check_mk/counters/dehya
value store: loading from disk
Releasing lock on /omd/sites/Home/tmp/check_mk/counters/dehya
Released lock on /omd/sites/Home/tmp/check_mk/counters/dehya
No piggyback files for 'dehya'. Skip processing.
No piggyback files for '172.16.0.66'. Skip processing.
[cpu_tracking] Stop [7f183c958590 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
[piggyback] Success (but no data found for this host), execution time 0.0 sec | execution_time=0.000 user_time=0.000 system_time=0.000 children_user_time=0.000 children_system_time=0.000 cmk_time_agent=0.000

This started happening last Friday. It used to work fine until then. If I run curl -s --unix-socket /var/run/docker.sock http://localhost/version on the client I get this:

{"Platform":{"Name":""},"Components":[{"Name":"Engine","Version":"24.0.7-ce","Details":{"ApiVersion":"1.43","Arch":"amd64","BuildTime":"2024-03-08T12:00:00.000000000+00:00","Experimental":"false","GitCommit":"311b9ff0aa93","GoVersion":"go1.20.14","KernelVersion":"5.14.21-150500.55.65-default","MinAPIVersion":"1.12","Os":"linux"}},{"Name":"containerd","Version":"v1.7.10","Details":{"GitCommit":"4e1fe7492b9df85914c389d1f15a3ceedbb280ac"}},{"Name":"runc","Version":"1.1.12","Details":{"GitCommit":"v1.1.12-0-g51d5e94601ce"}},{"Name":"docker-init","Version":"0.2.0_catatonit","Details":{"GitCommit":""}}],"Version":"24.0.7-ce","ApiVersion":"1.43","MinAPIVersion":"1.12","GitCommit":"311b9ff0aa93","GoVersion":"go1.20.14","Os":"linux","Arch":"amd64","KernelVersion":"5.14.21-150500.55.65-default","BuildTime":"2024-03-08T12:00:00.000000000+00:00"}

This clearly shows that the ApiVersion is in there, so I have no idea why this is happening. If anyone has an idea how to fix this it would be great.

Seems the same issue as i noticed on my box : CRE 2.1.0p43 (normal-)agent response on Docker container breaks when upgrading SLE-15-SP5 python3-requests to latest avaiable

Try reverting python3-requests to a previous version (or similar package-name on Debian (as i am on SLE15)

  • Glowsome
1 Like

Thanks. That fixed my problem for now. The client I’m running docker on is actually an openSUSE 15.5 box, the CheckMK server runs on Debian (still haven’t had the time to migrate it to SUSE).

I installed the previous version of python3-request and now everything’s back to normal.