ProxMox backup extension incorrectly reports backups too old

Checkmk Exchange
1

Hi,

I installed the plugin that monitors the backup age of VMs on ProxMox. However, for some reason I find that the backup of two of the VMs is reported incorrectly:

image
image1496×166 64 KB

Looking at the backup age in ProxMox itself shows that the backups are less then 7 days old:

image
image849×174 15 KB

Looking at the agent output it’s reported correctly as well:

cmk -d digprox01
...
<<<proxmox_qemu_backup>>>
QEMU-MACHINE;;;;;/etc/pve/qemu-server/109.conf;;;;;digsvl007
QEMU-MACHINE;;;;;/etc/pve/qemu-server/102.conf;;;;;digsvl003
QEMU-MACHINE;;;;;/etc/pve/qemu-server/108.conf;;;;;digsvl006
QEMU-MACHINE;;;;;/etc/pve/qemu-server/110.conf;;;;;digsvw001
QEMU-MACHINE;;;;;/etc/pve/qemu-server/115.conf;;;;;tmp001
QEMU-MACHINE;;;;;/etc/pve/qemu-server/105.conf;;;;;digsvl218
QEMU-MACHINE;;;;;/etc/pve/qemu-server/114.conf;;;;;digsvl012
QEMU-MACHINE;;;;;/etc/pve/qemu-server/113.conf;;;;;digsvl011
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:02 INFO: Starting Backup of VM 102 (qemu)
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:02 INFO: status = running
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:02 INFO: VM Name: digsvl003
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:02 INFO: include disk 'scsi0' 'VM-Storage:102/vm-102-disk-0.qcow2' 32G
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:16 INFO: backup mode: snapshot
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:16 INFO: ionice priority: 7
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:16 INFO: creating vzdump archive '/mnt/pve/Remote_Backup/dump/vzdump-qemu-102-2021_12_04-00_00_02.vma.zst'
/var/log/vzdump/qemu-102.log 2021-12-04 00:00:16 INFO: issuing guest-agent 'fs-freeze' command
/var/log/vzdump/qemu-102.log 2021-12-04 00:02:37 INFO: issuing guest-agent 'fs-thaw' command
/var/log/vzdump/qemu-102.log 2021-12-04 00:02:38 INFO: started backup task 'a59a5854-81df-47a0-9358-74c8fef6fb4b'
/var/log/vzdump/qemu-102.log 2021-12-04 00:02:38 INFO: resuming VM again
/var/log/vzdump/qemu-102.log 2021-12-04 00:02:41 INFO:   0% (13.5 MiB of 32.0 GiB) in 3s, read: 4.5 MiB/s, write: 3.5 MiB/s
/var/log/vzdump/qemu-102.log 2021-12-04 00:03:28 INFO:   1% (328.8 MiB of 32.0 GiB) in 50s, read: 6.7 MiB/s, write: 6.1 MiB/s
/var/log/vzdump/qemu-102.log 2021-12-04 00:04:35 INFO:   2% (668.9 MiB of 32.0 GiB) in 1m 57s, read: 5.1 MiB/s, write: 4.6 MiB/s
/var/log/vzdump/qemu-102.log 2021-12-04 00:06:08 INFO:   3% (986.6 MiB of 32.0 GiB) in 3m 30s, read: 3.4 MiB/s, write: 3.2 MiB/s
...

Anyone else experiencing this issue? I only have this for these two VMs. The rest is reported correctly.

Kind regards,
Louis

2

Hi,

which Proxmox plugin is it? It doesn’t seem to be the Proxmox special agent that is build in Checkmk 2.0:

1 Like
3

Indeed it isn’t. That’s why I posted in the CheckMK Exchange group. It’s the one that checks the backup:
https://exchange.checkmk.com/p/proxmox-qemu-backup-1

4

I see. Is it better than our special agent?

5

Ah good question. I followed the instructions in the link you provided earlier. That allowed me to setup monitoring of the clustered services properly. But I didn’t see the option of monitoring the status and age of the last backup in the special agent. Did I miss something there? If so, I’d be happy to use the special agent for monitoring of the backups as well.

6

You will not see it in the special agent/at the node level. Instead, it generates piggybacked data for the monitored VM.

So if the VM is called for example vm1.domain.com in Proxmox and has the same name in Checkmk, then the data is attached to it automatically. (case sensitive!)
If the names do not match, you will have to write a rule of the type “Hostname translation for piggybacked hosts” - but I prefer to have simply the same names - makes life much easier.

7

Well, my hostnames do match, so that shouldn’t be an issue. (I just verified to make sure). However, the backup is not showing without the agent from the CheckMK exchange.

In Proxmox I have for instance the VM called digsvl003:
image

Which matches with what it’s called in in CheckMK:

image
image1042×213 24.2 KB

But to be honest, I don’t see any services generated by the piggybacked hosts. Looking at the documentation I should see more than just the backup, but there’s none.

I do have the rule for the proxmox hosts:

image
image1559×1289 130 KB

And setup the hosts to use both API integrations and the CheckMK agent:

image
image980×487 73.9 KB

I verified that I can login to the Proxmox cluster using the checkmk user and the password specified in the rule I created.

So obviously I’m missing something, but what?

Kind regards,
Louis

8

Look into /omd/sites/SITENAME/tmp/check_mk/piggyback/, there should be folders named like your VMs/LXCs. If these names are matching to the VM hostname in Checkmk, then do a rediscovery of such a VM/LXC host and it should find new services.

9

I checked that directory. It’s empty. Which explains why I don’t have any of the services. Just to be sure I checked piggyback_sources as well; that’s also empty.

10

run a cmk -D ProxmoxHostname . It will output the configuration and close to the top you will see configured the special agent call.
You can copy that and run it manually and check if you run into problems.

11

Well, that explains why it’s not working I guess:

OMD[digio]:~$ /omd/sites/digio/share/check_mk/agents/special/agent_proxmox_ve '-u' 'checkmk' '-p' '<password> '--no-cert-check' 'digprox01'
Unhandled exception: RuntimeError("Couldn't authenticate 'checkmk' @ 'https://digprox01:8006/api2/json/access/ticket'")

Copy / paste username from the command let’s me login to the ProxMox web interface, so that’s working; no typo’s there at least.

12

Try checkmk@pve as username. @pve is the authentication realm. It can be @pam (Linux) or @pve (Proxmox user)

1 Like
13

Thanks, that nailed it. Something so stupid to overlook.

1 Like