Alerthandler (ROOT login not permitted)

Dear Check_MK Community,

I’m trying to deploy alerthandlers to restart systemd-services if they’re down (works well on my test server with Ubuntu 20.04 and root-login allowed).
Now I’m faced with the situation that root login via SSH is not permitted on the target host.

Check_MK Enterprise Edition 1.6.0.p15 (will be upgraded shortly to 1.6.0p17).

I have the following Rules:
(not showing the Systemd Service rules because they work and have been tested)

Remote alert handlers (Linux)
Bildschirmfoto 2020-09-16 um 10.19.00

Alerthandler configuration

For this to work though, I needed to set permissions on different

Now I have a big question mark above my head:
Why do i need to set 2 Users in 2 different places?

How do others deploy alerthandlers without root access?
Do you really need to manually change permissions to different files on all your target hosts just to get this running without root access?

alert.log (on checkmk server) output
Rule 'restart if service is down after 2nd check attempt (apache2)'...
Event Type is rc
-> matches!
Executing alert handler linux_remote for HOSTNAMEREDACTED;Systemd Service apache2
Spawned event handler with PID 25924
Handler [25924] linux_remote HOSTNAMEREDACTED;Systemd Service apache2 stopped: bash: /usr/bin/mk-remote-alert-handler: Permission denied

alert handler execution view
Bildschirmfoto 2020-09-16 um 10.49.28

permissions on /usr/bin/mk-remote-alert-handler on target server

P.S.:
Yes, the screenshots say elasticsearch (i made a screenshot of the wrong rule at first), but as you see we also want to restart apache2 via alerthandler.
Also: Just for testing, I got rid of the “only start alerthandlers on 2nd check attempt” portion of the rule.
Everything else is the same and I will definitely bring that portion back once the alerthandlers work.

If you need any more information, pls tell me.
I’m looking forward to your answers.

Hi,

Never used it myself, but still I may be able to help. In general, on a Linux system, to restart a systemd service you need root access. So, if it’s not possible to do this as root, you’ll have to configure sudo. Allow the user pixelpoint sudo rights, without password on /usr/bin/mk-remote-alert-handler. Then, instead of calling it directly, call sudo /usr/bin/mk-remote-alert-handler.

1 Like

Hello louis and thank you for your response.

I tested this approach today.
One needs to give permissions to the following directories / files:
/usr/lib/check_mk_agent (the folder itself)
/usr/lib/check_mk_agent/alert_handlers (again, folder)
/usr/lib/check_mk_agent/alert_handlers/* (alerthandlers inside the directory)
/usr/bin/mk-remote-alert-handler (this script seems to start the alerthandlers)

I did it with sudo chmod o+rx

Problem being:
As soon as you update the check mk agent on the target server, the folders keep the access permissions, but all the executables have their access permissions reset.

This includes mk-remote-alert-handler + all the alerthandlers inside the alert_handler folder, even the ones which have had their permissions modified.

Is there a way to automatically execute a post-agent-update script which gets called everytime the agent updates itself?

The only other solution would be to add the user to the root-group.
I tested this and it works flawlessly, but I don’t know what this would mean regarding security.

Hi,

Hmmm, actually I was suggesting to run your /usr/bin/mk-remote-alert-handler command using sudo so
sudo /usr/bin/mk-remote-alert-handler instead of just
/usr/bin/mk-remote-alert-handler

You can then need to give the user pixelpoint sudo rights to run /usr/bin/mk-remote-alert-handler as root without being prompted for a password.

So in your sudo configuration add something like this:

pixelpoint ALL = NOPASSWD: /usr/bin/mk-remote-alert-handler

Adding the user to the root group would basically grant this user the same privileges as the root user. From a security point of view I would highly recommend not to do this.

Hello louis,

thank you for your time and knowledge.

We solved this problem by allowing root login restricted to the CheckMK Server IP and NO PASSWORD (only public key authentication).
Now none of us can log into root (we don’t need to because we have our pixelpoint admin user), but CheckMK can do it because the agent transfers the SSH keys.
I didn’t know one could restrict the root login this way.

I thought the user who would execute the check_mk alerthandlers would also need permissions to the alerthandler scripts themselves.
Now I’m interested.
I will try this as soon as I find the time to do so on my ubuntu test vm.

Thank you for bringing that to my mind.

Hello,

I’m glad that you at least found a working solution. But from a security point of view you may want to disable remote root logins completely. Even when you only allow public key authentication.

With the sudo option I provided you don’t restrict the root login. You allow the user pixelpoint to execute this specific command with root rights. I assumed the mk-remote-alert-handler script then starts the other scripts. Since mk-remote-alert-handler runs with root rights all scripts that fork from this process will also run with root rights.

If this is not the case, you may need to add them to the sudo list.

pixelpoint ALL = NOPASSWD: /usr/bin/mk-remote-alert-handler, some-other-script, more-scripts

Good luck with the testing.

Louis

1 Like