High CPU load on Checkmk server

[degan]

CMK version: 2.3.0p35
OS version: Debian 11

Since 9.7.2025 we have extremly high CPU load on our monitoring server:

After Server reboot the load gets down to 4-5, but every day at 6 pm it goes up to 30-40. At the same time i get various “service check timed out”.
It looks like, that all checks run’s at the same time. This is in idle:

And then all checks run simultaneously:

[andreas-doehler (Andreas)]

Is this a Raw edition?
If yes please take a look at this post.

Checks running in bursts causing OOM

There is an option but only on the command line → ~/etc/nagios/nagios.d/timing.cfg Adjust the option “max_concurrent_checks” in maximum to the double number of your CPU cores or lower. Per default this value is set to 0 what means start all checks you want

In your case i would test with values between 20 and 30 here.

But if the problem is exactly starting at 6 pm every day, i would also take a look at the cron jobs around this time.