I would like to monitor the current logfile only, we have a logfile created every day with the format /var/log/app/app-20210830-all.log. My current configuration is:
# logwatch.cfg
# This file configures mk_logwatch. Define your logfiles
# and patterns to be looked for here.
/var/log/app/app-*-all.log
C public key not found
This monitors all previous logfiles as well, is it possible to restrict it to the current logfile only?
The logfiles are rotated monthly, so I still have like 30 logfiles monitored with this rule.
It does indeed monitor all previous logfiles as well. But is that a problem? CheckMK only checks for new entries in these files. Assuming they no longer receive updates no new entries should be found in those ever.
I don’t think it’s possible to write a regex that matches the current date, so the symlink option mentioned by @rprengel may be you’re only option.
I will just change the whole approach by removing the date from the logfiles and have only /var/log/app/app-all.log instead with a daily logrotate. I think this will be a much cleaner option.
Please note that if you use log rotation you have a blind spot in the time span between the last check and the log file is rotated. AFAIK there is no solution in checkmk like we had in the nagios check_logfiles to cover this situation.
you are absolutely right. I had such a case few weeks ago on a different host, luckily I was investigating the log file as it happened. I believe this is something we have to live with until a proper solution is provided from checkmk.
I already worked on this with external company and we didnt found a good solution. Its a quite complex situation, also because so many rules for log rotation.
Hm,
if logs are so important shouldn t a centralized logfile storage be a must have to save a copy off all informations and managing long time archiving?
Will ask „ is this a job that checkmk can an should solve?“
Ralf
As long as its managed by syslog, yes you can forward to event console and manage from there, but what if you have an application which manage there own log files with own rotating mechanism?
I guess this is more the domain of tools like Splunk.
I don’t think this is such a big issue.
You can have check_mk monitor all old logfiles without any significant performance impact, as long as the old files don’t change, checkmk doesn’t care and will ignore them.
In regards to the blind spot mentioned by @mike1098 as far as I understand, that is what the option “process preexisting log messages” is for.
you just need to make sure that you run logwatch once without this option, otherwise all alerts of the last 30 days (in your case) are pulled, so configure all your lofiles and patterns, run logwatch at least once, and then configure the plugin to inlcude the preexisting messages which will only be relevant for the new logfiles, hence cover the daily blindspot
I am using ELK stack for this, and check_mk for monitoring and notifications. The solution provided by check_mk is simple and straight forward, but doesn’t allow too much tweak.
@3c2c2ff5 As the topic is closed I want to ask you if its OK that we hijack your thread and discuss the logfile blind spot issue in his thread. I can also open a new topic for that. Just let me know.
As far as I understand Process preexistent log files doesn’t fix the blind spot problem with log files.
That’s written in the help text:
If a new logfile is found we usually skip to its end to avoid processing ancient log messages. You can enable this flag to start processing the file from the beginning and see all messages that may already be present.
Let me first explain the issue:
Lets presume the check is scheduled every 5 minutes and log rotation happens at midnight.
Lets assume the last check ran 11:56pm and at 11:57pm you have a new entry in the logfile you are interested in. At 00:00 the logfile is rotated and a new empty log file is created. The next check run then at 00.01am and it will find nothing in the new log file. So the new entry is lost and could not be detected.
Now you may add all log files to the config even the rotated ones and use the option “Process preexistent log messages”
As far as I understand from the above help text if you process the new rotated log file(s) the first time it would alert on all entries because it is different the the former one. Even worse, depending in your log rotation rules, you may have this situation for all the following files in the sequence. Thsi would then happen every midnight
Also questionable if logwatch could process zipped logfiles which is commonly used in log rotation.
If one convince me that it works I am open to test it
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.
