Using logwatch to monitor domain controller security log, not getting event console notifications

Troubleshooting
1

I am using CMK EE 1.6.0p8 and wanting to use logwatch to monitor our Windows domain controller security log, forward and alert on any user account lockout messages.

This is not working as expected, I fake a user lockout and I can see the event ID in Windows Event Log, but nothing in event console or /omd/sites/nagios/var/mkeventd/history.

Here is my agent configuration…

1
1947×899 33.1 KB

Here is my logwatch event console forwarding rule…

2
21093×810 30.5 KB

Finally here is the actual rule setup in EC…

3
3864×688 17.9 KB

I have tested the event log message in the event console and I know that part works.

The issue is, it does not seem like the event ID is getting forwarded to my check_mk server. I don’t understand why this is since I have specified all Security event logs to be forwarded.

I was tailing /omd/sites/nagios/var/mkeventd/history and I don’t see anything in there related to the account lockout events.

Back on the domain controller I reviewed the check_mk.log and the only thing that stands out is this…

2020-02-27 10:12:02.296 [Warn ] Logwatch size have exceeded limit [500000]
2020-02-27 10:12:02.297 [Trace] Skipping logwatch pos from [206217871] to [206218822]
2020-02-27 10:12:02.418 [Err ] failed to open log ‘*’
2020-02-27 10:12:02.419 [Trace] Sending data ‘logwatch’ id is [744020502708642] length [501863]
2020-02-27 10:12:02.421 perf: Section ‘logwatch’ took [914] milliseconds
2020-02-27 10:12:02.463 Received [501991] bytes from ‘logwatch’

I tried removing the logwatch state file and no luck.

The only thing I am left with is the logwatch size exceeding limit and maybe that is preventing all the event logs from the security log to be forwarded and that somehow it is skipping.

Can I control the logwatch size limit for the windows event log? I know you can do some granular things for monitoring log files but NOT the windows event log.

Anyone able to help me out? I have hours trying to troubleshoot this without any success.

2

Hi,
try this:

  • Disable the forwarding rule to the EC.
  • Rediscover your host. Do you get new undecided services for the system, security and application log?
  • Clear the logs in CMK. This should remove the “Logwatch size have exceeded limit”
  • After that: Fake a user lockout. Do you see the message in the log services in CMK?

Karl

3

No luck, for some reason lockouts are not getting registered in EC and a notification is not being sent out.

I gave up on this, there is too many must-have features missing when it comes down to Windows Event log monitoring.

At a bare minimum we should be able to at the source exclude all noise and only forward event IDs we really care about and that does not seem to be the case. You need to forward everything and filter server side. I think this is a design flaw in logwatch.

I ended up using NXLog and granularity defining the event IDs and conditions I want to forward to check_mk’s event console via syslog protocol.

It works great, the only downside is now in addition to check_mk I also have to have NXLog installed on the domain controllers.

4

Did you apply the 4 points I suggested?

5

No I gave up and went a completely different route, installing NXLOG and using that to forward events to CMK EC and this works for our needs.

1 Like