Hello Matthias,
We operate 300 sites and 15.5k hosts globally. From the beginning it was clear to us, that the local IT will be not able to deal with the rule sets and configure their own monitoring. The problem is the fact that most of the Local Admins dont deal on a daily basis with monitoring configuration. Even if training would provided the Admins will not work for a long time on monitoring configuration, forget what they studied and flood central teams with questions.
Our approach is to follow internal hard and software standards and standardize the monitoring accordingly. We provide only the necessary options like the type of host or application to the local administrators. Local IT has no access to the ruleset and just select some tags in the host object to do the configuration.
Beside that we offer specific monitoring for files, services and processes. Local IT need to send a form for this and a central team create the rules for them. Also this process is standardized and managed by change requests.
For any monitoring needed for non standardized, local specific applications the Local IT can write their own local checks. All that needs of course a good but simple internal documentation to help the local Admins to setup their Hosts.
Thanks to checkmk team who made that happen with their tool and with the flexibility they provide to implement customization’s to adapt the tool to our internal, specific needs.
I tried to explain the project in my presentation at checkmk conference in 2018:
Currently we run a project to replace our Central Monitoring, based on Nagios 3 with checkmk.
Let me know in case of questions.
Michael FRANK