100 % cpuload after upgrading to 2.0

After upgrading our enterprise edition from 1.6 → 2.0, we have had a constant 100 % cpu load on our check_mk virt1 installation.

Also our filesystem was full on /aufs, but deleted the logs there to free up space.

Same here! 100% CPU usage after upgrade.

Which processes cause the CPU load?

The question is here what logs and what was there inside. I think you have old extensions installed and they are the cause for crashes and following this very high amount of log information.
The same can be the reason for very high CPU load.

Where there any notifications on you system at the update process about check what cannot be migrated?

Please take a look at the “cmc.log” and “web.log” for some more information. Also the directory with the crash logs can help"

1 Like

update.log would also be helpfull

1 Like

Apache2 according top TOP.

Then the “web.log” would be a good start as it looks like some extensions to the web interface running not correct. What says your Apache log?

Thanks, it seemed that the entire issue was caused by a single client which had the dashboard open?

[Mon Mar 15 08:45:37.281349 2021] [proxy:error] [pid 764] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:5000 (127.0.0.1) failed
[Mon Mar 15 08:45:37.281366 2021] [proxy_http:error] [pid 764] [client 1.1.1.1:54696] AH01114: HTTP: failed to make connection to backend: 127.0.0.1, referer: http://2.2.2.2/SITE/check_mk/dashboard.py

After closing the dashboard on that specific client, the CPU usage dropped from 100 % to 6 %.

Hi all,
same issue here with 2.0p5 raw running in a docker container, monitoring around 100 hosts and 2000 services. CPU Load on the CheckMK-Server has grown over the last 3 days until it reached critical levels. Reason was one client where the dashboard was continously opened (and it was the only client logged in the web interface), finally the server had to answer dozens of request every second, and the CPU load was caused by lots of apache processes. But no proxy errors or so, only many, many requests. Closed the browser (Firefox), and CPU load went down immediately. But… I would like to keep the Dashboard open permanently on the big screen, without restarting the browser every day… Any ideas?

Thanks, Brgds

Michael

Can you take a look at your “web.log” inside the site?
There you should find some hints if anything has a problem inside the complete website generation.

I kept the client running over the weekend, and again CPU is heavily loaded.
Not much in the web.log, only “[cmk.web.background-job 27654] Found no abandoned profile.” once an hour.
But dozens of requests in the access_log every second like "xx.xx.xx.xx - - [14/Jun/2021:06:13:09 +0000] “POST /cmk/check_mk/ajax_figure_dashlet_data.py HTTP/1.1” 200 1286 “https://cmk.xxx.xxx.de/cmk/check_mk/dashboard.py” “Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0"”
Restarted the browser, all back to normal.
brgds

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.