HomeAssistant monitoring plugin

Imperial7693 · February 7, 2023, 8:09am

Hi,
I would like to suggest a development of HomeAssistant x86 plugin to be able to monitor the resources in it.
Thanks

louis · February 7, 2023, 2:11pm

Hi,

Nobody’s stopping you

And once you develop it, I’d be happy to try it out. For now I’m just satisfied with monitoring HA via SSH.

Imperial7693 · February 7, 2023, 8:11pm

Thanks! How do you do the monitoring via SSH?

mschlenker · February 7, 2023, 8:27pm

Monitoring via SSH? It’s done this way. If something is unclear, ping me. I am the one at tribe29 currently curating this article.

louis · February 8, 2023, 7:01am

Also, and that’s the annoying part, if you’re using HAOS and you install the agent inside the container, you’ll have to copy it there again on every restart of HA. Still need to try and put it somewhere in a directory that is mounted outside the container. You need to install the “Terminal & SSH” add-on in HA to be able to do this.

If you want to access the underlying OS, you’ll have to put it in debug mode acording to the instructions here:

You can then access the HAOS via SSH on port 22222.

Imperial7693 · February 9, 2023, 8:11pm

Hi, well I almost got there with the legacy mode but I cant make the ssh connection from Checkmk server. When I console in to the checkmk docker container I can ssh to the home assistant target with the key.
Check_MK fails with [agent] Agent exited with code 255: Host key verification failed.. However I can ssh in from the container as I said, so Executing the Linux agent over ssh - Checkmk Knowledge Base - Checkmk Knowledge Base is not helping much…

louis · February 10, 2023, 7:51am

Now I don’t know much about the CheckMK docker version, but are you sure you’re using the ssh key for the site user and not root for instance? That may be causing the key verification to fail.

Imperial7693 · February 10, 2023, 4:26pm

I moved to an Ubuntu VM and did a fresh installation. I am however still unable to set the ssh connection…

Agent exited with code 255: Permission denied, please try again.
Permission denied, please try again.

When I use the ssh -T -oStrictHostKeyChecking=no [inuxrootuser]@192.168.1.100 the connection works just fine.

@louis you tell to use the “site user”, do you mean “cmkadmin”? That is only the checkmk user and has no “account” for SSH in the underlying ubuntu system.

martin.schwarz · February 10, 2023, 4:39pm

No, “site user” is the Linux user account which “owns” the site’s files/directories and runs the processes. The user name is identical to the site name.

Imperial7693 · February 10, 2023, 4:50pm

Hi,
Well as I am in my home environment, I did not create a new user after installing the ubuntu server, so for me the default ubuntu user owns everything. And it does not work, sadly.

In addition I am having hard time even to install the check_mk_agent on the home assistant side.

saving to: 'check_mk_agent.linux'

check_mk_agent.linux              100%[==========================================================>]  67.09K  --.-KB/s    in 0.002s  

2023-02-10 18:37:00 (39.1 MB/s) - 'check_mk_agent.linux' saved [68705/68705]

➜  bin mv check_mk_agent.linux check_mk_agent                                   
➜  bin chmod 755 check_mk_agent                                                 
➜  bin check_mk_agent | head                                                    
<<<check_mk>>>
Version: 2.1.0p20
AgentOS: linux
Hostname: a0d7b954-ssh
AgentDirectory: /etc/check_mk
DataDirectory: /var/lib/check_mk_agent
SpoolDirectory: /var/lib/check_mk_agent/spool
PluginsDirectory: /usr/lib/check_mk_agent/plugins
LocalDirectory: /usr/lib/check_mk_agent/local
FailedPythonReason:

martin.schwarz · February 10, 2023, 4:54pm

I doubt this or we misunderstand each other. When you create a monitoring site using “omd create sitename”, a Linux user with the same name is created automatically. This is the “site user”.

Imperial7693 · February 10, 2023, 5:08pm

Well, all right. I stand corrected. There is indeed a site owner. named “monitoring” in my case.

However I still cant get the agent to run on home assitant side:

<<<check_mk>>>
Version: 2.1.0p20
AgentOS: linux
Hostname: a0d7b954-ssh
AgentDirectory: /etc/check_mk
DataDirectory: /var/lib/check_mk_agent
SpoolDirectory: /var/lib/check_mk_agent/spool
PluginsDirectory: /usr/lib/check_mk_agent/plugins
LocalDirectory: /usr/lib/check_mk_agent/local
FailedPythonReason:

Imperial7693 · February 10, 2023, 5:28pm

I am one more step in.

This is in spite of the FailedPythonReason:

However the agent does not find ANY services…

martin.schwarz · February 11, 2023, 2:46am

Sorry, I just chimed in to explain about the site user. I have no idea what “home assistant” even is

The checkmk agent is essentially just a shell script. You can take a look inside. And you can run it with “–debug”.

louis · February 13, 2023, 8:58am

Hi,

Sorry for the late reply. Something happened that’s called weekend. Anyway, it seems that with the help of Martin you at least got the agent to work.

Now, for my information, what installation type of HA are you running? Are you using HAOS or did you install your own Linux and HA on top of that? I’m running HAOS as a VM inside ProxMox. So in order to get it working here’s basically what I did.

My CheckMK server is called “Hestia” and the site is “Home”. My HA server is called “Ganyu”. So on Ganyu I first installed the add-on “Terminal & ssh”. Then on Hestia, as the Home user, I ran ‘ssh-keygen’ and just hit enter a few times (so no password). This generates 2 files, in the ~/.ssh directory, id_rsa and id_rsa.pub.

Copy the contents of id_rsa.pub and add them in HA to the configuration of “Terminal & ssh”. It should look something like this then:

You can now test this. In my case as OS user Home I should be able to ssh to Ganyu without having to specify a password (note that the first time you may be asked to accept the host key; just do this):

OMD[Home]:~$ ssh root@ganyu

| |  | |                          /\           (_)   | |            | |
| |__| | ___  _ __ ___   ___     /  \   ___ ___ _ ___| |_ __ _ _ __ | |_
|  __  |/ _ \| '_ \ _ \ / _ \   / /\ \ / __/ __| / __| __/ _\ | '_ \| __|
| |  | | (_) | | | | | |  __/  / ____ \\__ \__ \ \__ \ || (_| | | | | |_
|_|  |_|\___/|_| |_| |_|\___| /_/    \_\___/___/_|___/\__\__,_|_| |_|\__|

Welcome to the Home Assistant command line.

System information
  IPv4 addresses for enp0s18: 172.16.0.61/24
  IPv6 addresses for enp0s18: 2a02:a467:2c6:1:3128:eef1:f519:3457/64, fe80::15ef:be49:a3c0:1d75/64

  OS Version:               Home Assistant OS 9.5
  Home Assistant Core:      2023.2.3

  Home Assistant URL:       http://Ganyu.local:8123
  Observer URL:             http://Ganyu.local:4357
[core-ssh ~]$

So, now I’ve established that I can connect password-less, as the site user to my Home Assistant. Just logout and copy the linux agent to HA:

OMD[Home]:~$ scp version/share/check_mk/agents/check_mk_agent.linux root@ganyu:/usr/bin/check_mk_agent

Now, in CheckMK go to “Setup → Agents → Other integrations” . Create a new rule under “Individual program call instead of agent access”. Give your rule a name and use this command:

And tell it to only run on the hosts you want. I my case I use this also on my Router.

Save and activate your rule and you should be ready to go. Here’s a sample of what I get:

Like I said before, since my version of HA is running as a docker container, every time I restart HA I have to copy the agent to it again. Looking for a fixed location to put it. Probably should put is somewhere in the config directory of HA, since that one’s persistant.

Imperial7693 · February 13, 2023, 8:01pm

@louis, this was a great comment thanks. I think my issue was not adding /usr/bin/check_mk_agent to the command line to execute.
I have also used the actual IP address instead of $HOSTNAME$ , not sure if that made any difference.

By the way what do you get when executing check_mk_agent | head on the HA side?

<<<check_mk>>>
Version: 2.1.0p20
AgentOS: linux
Hostname: a0d7b954-ssh
AgentDirectory: /etc/check_mk
DataDirectory: /var/lib/check_mk_agent
SpoolDirectory: /var/lib/check_mk_agent/spool
PluginsDirectory: /usr/lib/check_mk_agent/plugins
LocalDirectory: /usr/lib/check_mk_agent/local
FailedPythonReason:

I am a bit concerned about that FailedPythonReason:.

louis · February 14, 2023, 6:44am

Hi,

You’re welcome. Does that mean that you got it working now? Doesn’t matter if you use the actual IP address or $HOSTNAME$. The latter is just a bit more universal, so I can use the same rule on more hosts.

Don’t worry about the python message. On my side I got this:

[core-ssh ~]$ /usr/bin/check_mk_agent |head
<<<check_mk>>>
Version: 2.1.0p20
AgentOS: linux
Hostname: core-ssh
AgentDirectory: /etc/check_mk
DataDirectory: /var/lib/check_mk_agent
SpoolDirectory: /var/lib/check_mk_agent/spool
PluginsDirectory: /usr/lib/check_mk_agent/plugins
LocalDirectory: /usr/lib/check_mk_agent/local
FailedPythonReason: No suitable python installation found.

But since the agent is a shell script in bash, it doesn’t need python. Far as I know that’s only used for some plugins.

Imperial7693 · February 14, 2023, 7:46am

Does that mean that you got it working now?
Yep, all seems to be working.
My only remaining issue is here "mktime argument out of range" in problem notifications - #3 by Imperial7693

checkitall · March 5, 2023, 2:23pm

I followed this thread with interest. I am unable to get the mk_docker plugin to work in the HAOS instance. Something hangs the check and it causes the checks to back up and the host runs out of memory. The basic agent runs fine.

From a monitoring strategy, I think it’s better to monitor services and entities through the front end HA API in addition to the basic host checks rather than watching the containers at the VM level. I’m going to start on some Python for that.

Also, I installed the check_mk_agent script into /data so that it is persistent across reboots.

louis · March 6, 2023, 7:31am

Interesting. Based on your idea of installing the docker plugin on the HAOS instance, I first tried to determine which version of Python is running on that. Seems there’s none. That explains why it wouldn’t work, but should not cause check_mk agent to hang in general. But I do recall from another host on which I run docker, that you’d also need to install the python-docker libs. Don’t know if that’s an option with HAOS. And even if you manage to get it working, I believe you’ll have to reinstall them after every HAOS upgrade.

And I totally agree, it would be better to monitor through the HA API, so you can also monitor the state of your entities and devices and all through CheckMK. But I miss the knowledge of the HA API to code something like that myself. (More, I’m missing the time to dive into it). So if you can come up with a more native way that’d be great.

And yes, after writing in this topic, I did move my agent in /config to have it persistent across reboots. Should have done that a lot sooner.