CheckMK Server running out of space

Recently I noticed my CheckMK Server is slowly running out of space.

I’m using a rented cloud server for it, so I’d very much like to avoid making the partition bigger, since it would mean I’d have to rent a higher tier server.

I tried looking through some local files, but the logs and old kernels, didn’t amount to much, so I’d like to know some other options of clearing some space.

One idea I had was shortening the span, over which hardware info is stored on the server. For example the trend CheckMK calculates in the warning above is based on 2 years of history. If every service has 2 years of history stored on the server, that could amount to a lot.
I read somewhere that it’s possible to change that to 1 year, but I didn’t find out how so far.

Does someone here maybe know?
Or do you have other ideas on how I could resolve this issue?

Any help would be greatly appreciated.
Cheers
Paul

Take a look at the service OMD * disk usage

That will tell you what is consuming your disk space and then you can tackle that specific piece.

Appearantly the PNP4Nagios is taking up over 11GB. That doesnt seem right.
I’m not entirely sure what that service is for, but 11GB seems like a lot to me.

Is there a way to free up some of that space?

11GB is not much. How many services do you monitor?
What is more important are the 4GB you cannot see in your graph.
Difference between “Total size of site” and the single values.

1 Like

The size will likely also not change strongly over time, if you don’t add more services for the PNP4Nagios stuff.
The storage is reserved on new service creation and then static, like forever.

I’m monitoring 6132 Services.
Now that you mention it, that does seem odd. Any idea, where those 4gb could be found?

You should look inside your site with “du -sh *” for the big folders or files.

There is a ~4gb “nagios” Folder.
It doesnt look like there is anything in there I can just delete.

Is it possible to create a rule to have checkmk save less history of the services? I dont need anything older than a year and over >6000 Services, it probably adds up to a lot of wasted space.

Short answer - no. This is not much space, every value saved as performance data requires 300kB of storage.

This should not be so much. I think you have a problem in your instance if this folder is so huge.

I find sudo ncdu -rrx / (or whatever your mount point) very useful for interactively looking for the biggest disk hogs.

2 Likes