We have 6 domain controllers which are monitored by check MK, however all of the Domain controllers are very busy and therefor the logs fill up very quickly. After clearing the log from checkMK we I get the ‘Unacknowledged messages have exceeded max size, new messages are dropped (limit 500,000 B)’ message within a few hours.
There is a stipulation that all logs should be picked up by checkMK, so turning off the logwatch for that server is not an option.
Is there a way to increase the 500kb size or a way to ‘rotate’ the logs so we don’t hit the 500kb limit?
i have done alot of research around this but i cannot find anything that doesn’t suggest deleting the logs every day, we need to keep them.
CheckMK is not a log sink where you can store all logs, it should be used more like - this event is import i will keep it. For this you have two different approaches. The normal logwatch check has a negativ matching (you define what should be thrown away) and the Event console has a positive matching (you define what you want to keep).
If you need a real log sink please use a second tool like greylog, ELK stack, Logpoint or Splunk beside CheckMK.
Yes, agreed that CheckMK is not a log sink, we are currently in the process of setting up Splunk.
Is there a way, within logwatch to only log critical and warnings?
I’ve set the windows logging level to ‘write to the log file only most important events’ and changed the finetune windows event logging to ‘all eventlogs - WARN/CRIT - without context’ under agent rules, but this hasn’t really changed anything.
You need to check if your client has also this settings. Is the automatic agent updater in use in your system?
Please check the yaml files on the machine if there are these settings active.
Hi,
if you have too much logs from your DCs you should forward them to the Event Console. There you will be able to handle the logs by using regex for Event ID and you can drop the unwanted entries automatically based on time.
Greets,
Christian
I have managed to drastically reduce the amount of logs being brought into check MK. As we have the updater enabled but the checkmk agents were not registered with the update agent, so i spent time registering all the servers which allowed the ‘write to the log file only most important events’ and 'all eventlogs - WARN/CRIT - without context’ to filter down to the agents, however we are still hitting the 500kb limit within about 5 days.
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.