Recently set up CheckMK in a container, at first it worked fine and I could access the web GUI without issue, got some hosts set up etc. I did at one point recieve a gateway timeout from it but restarting the container fixed this. I really don’t know what happened since. It worked fine yesterday, wanted to continue learning it today and now I get gateway timeouts all the time. Tried to both restart the container several times as well as recreating it completely (data is in a volume, so this of course didn’t remove any of that).
This is running on Ubuntu Server 20.04 with Docker version 20.10.7.
The container is behind traefik to handle routing and TLS.
The container is running the check-mk-raw:2.0.0-latest image.
Any help is greatly appreciated, if you require any logs etc just let me know. I have however looked through the docker logs and the container reports everything as a-ok.
What says your Apache log inside the container at the time of the error messages?
Try to expose the container ports directly to minimize the impact of other software components. In my dev environment the containers are working without problem.
Funnily enough when I now tried to access the web gui to make sure I got fresh error logs it worked fine.
However restarting the container did re-introduce the problem, this is my apache error_log from restart, I did try to access the site twice however this did not seem to generate more errors in here, so I am unsure if these errors are actually relevant to the gateway timeouts.
I did see similiar errors from earlier too, but not necessarily during the times I tried to reconnect, reinforcing the fact that these might not be related. I’m including them because these seem to be the only errors I get.
[Fri Jun 25 09:00:31.734107 2021] [mpm_prefork:notice] [pid 258] AH00169: caught SIGTERM, shutting down
[Fri Jun 25 09:00:39.029274 2021] [mpm_prefork:notice] [pid 238] AH00163: Apache/2.4.38 (Debian) mod_wsgi/4.7.0 Python/3.8 mod_fcgid
/2.3.9 configured -- resuming normal operations
[Fri Jun 25 09:00:39.029328 2021] [core:notice] [pid 238] AH00094: Command line: '/usr/sbin/apache2 -f /omd/sites/cmk/etc/apache/apa
che.conf'
[Fri Jun 25 09:01:02.607260 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field id has no description.
[Fri Jun 25 09:01:02.607286 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.607306 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field nodes has no description.
[Fri Jun 25 09:01:02.607309 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.607322 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field params has no description.
[Fri Jun 25 09:01:02.607325 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.607336 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field node_visualization has no description.
[Fri Jun 25 09:01:02.607339 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.607358 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field properties has no description.
[Fri Jun 25 09:01:02.607361 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.607372 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field aggregation_function has no description.
[Fri Jun 25 09:01:02.607374 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.607384 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field computation_options has no description.
[Fri Jun 25 09:01:02.607387 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.607397 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIRuleEndpointSchema: field pack_id has no description.
[Fri Jun 25 09:01:02.607400 2021] [wsgi:error] [pid 242] warnings.warn(f"{name}: field {key} has no description.")
[Fri Jun 25 09:01:02.609519 2021] [wsgi:error] [pid 242] /omd/sites/cmk/lib/python3/cmk/gui/plugins/openapi/restful_objects/params.p
y:199: UserWarning: BIAggregationEndpointSchema: field id has no description.
[Fri Jun 25 09:01:02.609526 2021] [wsgi:error] [pid 242] warning
If there’s another place I should check for logs please do let me know, these are from _data/cmk/var/log/apache/error_log in the volume that gets mounted into /omd/sites on the container.This text will be hidden
I have now found the issue, it was indeed with Traefik and not CheckMK.
The CMK container is on more than one docker network, the one that everything traefik uses and a separate one allowing it to talk to my postfix container. Traefik seeminly randomly chooses which of the two virtual addresses to contact the CMK container on and sometimes chose the one that CMK had in the postfix network, meaning it ultimately was denied access and therefore gave a timeout.
The fix was to specify the traefik.docker.network label so that traefik always sends data on the address in the network traefik and the CMK container shares.
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.
