Stale status appeared some time

CMK version: 2.2.
OS version: rocky linux9

Error message:
image

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

just wish to know i sometime see stale status , and it disappears immediately after 2 minutes time .
what can be reason for this stale status ? how we can fix this issue?

Hi @manojr2k,

stale services can have multiple causes.

One reason could be a performance issue of the Checkmk Server. Maybe your Checkmk server can’t handle the amount of work to check everything in time. I assume you a using the default 1 minute check interval.

Can you please check in your sidebar the Service Speed-O-Meter if your scheduled rate is fulfilled or not?

Example:

Another helpful indicator would be the Monitoring Core statistics snap-in:
image

The size of your monitoring and the used hardware specs of your Checkmk Server are also important.

Based on another post, I assume you changed the checker amount to 2?

Is the problem happening since then?

Thanks in advance.
Norm

where to look exactly for Service Speed-O-Meter and Monitoring Core statistics snap-in?

i have not yet changed the checker amount to 2?

Please make sure to toggle your sidebar. I have it on the left side, but the default is the right side, I think. To add snap-ins you have to add them via the plus icon at the bottom.

Looking forward to your reply!

Regards
Norm

thanks Norm for help
below is my status , what i need to look at exactly ?

this is at the moment

Hi @manojr2k,

how often does it stay below the 80% threshold?

And how often can you see that a host is stale? Are there any correlations?

If your Checkmk Server can’t keep up with the scheduled amount of services/hosts, it will result in stale services.

Can you please share the amount of hosts and services you are trying to monitor and the hardware resources that are available?

Thanks in advance!
Norm

hi Norm

currently i m testing it with approx 100 servers.
4 vcp , and 16 GB RAM.
i m using customized shells script as well for monitoring linux .
can you please suggest ,how we can handle this stale service part ?
will it cause the performance ? or email alerts ?
today i monitored , stale is coming quite many times .
but it disappear quickly.
also when we reboot machine , that time stale shows lot of services inititally and then disappears .

hi Norm

can you please suggest anything here ?

hi Norm
waiting for your reply.

Hi @manojr2k

Just to manage expectations: This is not an official support channel.

This is a community of people who like to help other users, often in their free time. Here you get help from real experts - for free!

But this also means that you should not escalate and ‘bump’ requests. Sometimes you just have to be a little patient. And sometimes it also means that the community can’t help you.

@Norm, just like everyone else here, also has a day job, he may be busy with other things, on holiday, or sick, or there may be a million other reasons why he doesn’t answer. And that is perfectly fine.

If you would like to have professional tech support with guaranteed response times etc., please contact one of the Checkmk partners or us - but that won’t be free.

Thanks
Elias

2 Likes

Hi @manojr2k,

a deep dive into Checkmk performance issues can be very time-consuming, but I still would like to help you out.

The necessary resources depend heavily on the amount of things you do with your Checkmk (e.g. SNMP Checks). One of the most important things is to have SSD performance in your storage layer. Checkmk needs SSD’s to work nice and smooth. If that’s not the case, yet, you should change that in the first place. Other than that, I would suggest doing some test. If it’s possible, change your vCPUs to 6 and see if it runs better.
Getting professional help is also a good option. A good starting point would be to test changes and provide more detailed data of the issues and your system.

Hope this helps you and if you are planning a production deployment in a professional environment I would always suggest to consult experts on your project. Either Checkmk itself or an official Partner as @elias.voelker already suggested. :slightly_smiling_face:

Doing the Checkmk trainings helps a ton to understand Checkmk better. →

Regards
Norm

Hi Norm

Thanks for the reply .

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.