I’m readying help documents for checkmk and I see there is this concept of the event console. Everything I read here : The Event Console - Processing logs and SNMP traps
looks like checkmk can be used as a syslog server.
Our environment isn’t huge, maybe 40 servers with 10-15 pieces of network equipment and firewalls.
When I search out other info for using checkmk as a syslog server I see this thread : Check_mk as syslog server - #6 by aeckstein
A user posts in there that checkmk is specifically NOT a syslog server. Granted the post was from 18 months ago so I’m just curious if checkmk is official supported as being a syslog server?
Given our environment isn’t super large, we’d like to use checkmk both for monitoring and acting as our syslog server. We don’t want to have to set up a totally separate product (graylog, syslog-ng, splunk, etc.) just for syslogging if we can just use checkmk to check that box.
Technically, yes, checkmk can be a syslog target. What people mean when they say Checkmk shouldn’t be used as a syslog target is:
- the event console is optimized for throughput, i.e. throw thousands of events at it, and filter what you care about, discard all the rest
→ the backend of the event console for archiving events is not optimized for holding lots of data. It’s just plain text files, with little to no indexing.
So keeping the logs of 40 servers could work, but don’t be surprised if searching for “event xy” on all servers over the last 90 days runs into a timeout.
(checkmk/history.py at 266b81a4c08a9b19c3c0ad0291531386a3d4a5f0 · Checkmk/checkmk · GitHub mentions a mongodb backend for the event console, but I haven’t found the time to play around with it, and there is no documentation about it that I’m aware of. @mschlenker master of documentation is there any :)?)
Old forum posts look like the setup is not straight forward Add Mongodb as an archive mode in the EC
TIL about this MongoDB connection… I also cannot find recent (read: last three years) werks on it. But as I understand it, this only provides a storage backend for after filtering by the Event Console, so it might be useless to analyze unforeseen things.
Checkmk will probably never be a log server while proper setup of the Event Console might dramatically reduce the need for one. How does this go together? First, the differences: Checkmk works with states and changes of states, log analysis cares for events. Now the event console tries to translate singular events or sequences of events to changes of state.
So if properly configured, for example treating a certain number of 404s per time unit as critical for a web server, the Event Console will provide you with all needed information to identify the cause (given you do not mangle log lines), thus reducing the need of a log server.
In this case of 40 servers the pragmatic approach might be storing locally and sending to the EC in parallel. Then you can learn from irregularities that make local log analysis necessary and create new EC rules until the locally stored logs are only very seldomly needed anymore as reference.