Cron Job monitoring

[Steven1 (Steven)]

2.1p25 RAW
Ubuntu 18.04 LTS

Hi

I am monitoring CRON jobs, 40 or 50 different jobs all running on 1 server.
I set up the mk-job like described here: Monitoring time-based processes (Cronjobs) (users have rights to write in the respective directories under /var/lib/check_mk_agent/job/
If I run a cmk -d on the server, I see the logs of all the different cron jobs are being sent to the checkMK server.

But if I check the web interface for the same job, I see different results

the exit code is actually ‘0’ and the job has ran much more recent, the rest of the values seem to be okay.

Does anyone have an explanation for this behavior?

[r.sander (Robert Sander)]

The job name has to be unique per host.

[Steven1 (Steven)]

They are all called “archiving_”+ a three letter extension, they are all unique.
F.I.:

[gstolz (Gerd Stolz)]

Hi Steven,

can you share the output of
“grep -A 11 archiving_BM4 ~/tmp/check_mk/cache/<hostname_in_question>”

and just to be sure:

“grep -A 11 archiving_BM4 ~/tmp/check_mk/piggyback/<hostname_in_question>/*”

maybe there are multiple files for the same job. There really isn’t much caching going on that could otherwise lead to false data, when “cmk -d” already shows the current and correct data.

Gerd

[Steven1 (Steven)]

“grep -A 11 archiving_BM4 ~/tmp/check_mk/cache/<hostname_in_question>”
That is weird… I can see archiving _BM4 two times.

How do I remove the double entries?

“grep -A 11 archiving_BM4 ~/tmp/check_mk/piggyback/<hostname_in_question>/*”

Doesn’t give ANY output.

[r.sander (Robert Sander)]

Are there multiple users with the same job name?
What are the contents of /var/lib/check_mk_agent/jobs on the host?

[Steven1 (Steven)]

Every user has 1 job, uniquely named.
The content of the files on the host seems to have the correct output from the mk-job command:
These are the directories

This is the folder content of one of the problematic jobs that we were discussing before.

File content seems to be ok as well:

[gstolz (Gerd Stolz)]

I haven’t used mk-job in a long time, but what do you get with:

find /var/lib/check_mk_agent/ -name "archiving_BM4"

or maybe if things get weird, we can grep for the content:

grep -r "start_time 1679547901" /var/lib/check_mk_agent/

[Steven1 (Steven)]

I executed your command and saw that when I was testing with the cron jobs under the root user, they all created a log file in that “/var/lib/check_mk_agent/job/root/” folder.
I deleted all those logs and now my cron jobs get detected perfectly!

What a relief, thanks for the support and thinking along!