HostSectionParser: Ignoring invalid data b'<<<app-proxmox>>>'

Troubleshooting
,
1

CMK version:

2.4.0.p19
OS version:

Debian13

Proxmox version 9.1.4

Error message:

Hi, I started using the raw version to monitor my homelab proxmox cluster of two nodes, named node-1 and node-2.

Now I get on both nodes the warning:

HostSectionParser: Ignoring invalid data b’<<>>’

Relevant part of cmk debug output below. No idea what’s going wrong.

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

 HostKey(hostname='node-2', source_type=<SourceType.HOST: 1>)  -> Add sections: ['proxmox_ve_mem_usage', 'proxmox_ve_node_info', 'uptime']
  HostKey(hostname='node-2', source_type=<SourceType.HOST: 1>)  -> Add sections: ['labels', 'piggyback_source_summary', 'proxmox_ve_mem_usage', 'proxmox_ve_node_info', 'uptime']
Received piggyback data for 16 hosts
Storing piggyback data for: 'node-1'
Storing piggyback data for: 'homeassistant'
Storing piggyback data for: 'mqtt'
Storing piggyback data for: 'zigbee2mqtt'
Storing piggyback data for: 'AdGuard-1'
Storing piggyback data for: 'proxmox-datacenter-manager'
Storing piggyback data for: 'mattermost'
Storing piggyback data for: 'npm'
Storing piggyback data for: 'AdGuard-2'
Storing piggyback data for: 'myspeed'
Storing piggyback data for: 'watchyourlan'
Storing piggyback data for: 'librenms'
Storing piggyback data for: 'bind9'
Storing piggyback data for: 'pulse'
Storing piggyback data for: 'homepage'
Storing piggyback data for: 'checkmk'
CPU load             PEND Check plug-in received no monitoring data
CPU utilization      PEND Check plug-in received no monitoring data
Check_MK Agent       PEND Check plug-in received no monitoring data
Disk IO SUMMARY      PEND Check plug-in received no monitoring data
Filesystem /         PEND Check plug-in received no monitoring data
Interface 3          PEND Check plug-in received no monitoring data
Interface 4          PEND Check plug-in received no monitoring data
Interface 5          PEND Check plug-in received no monitoring data
Kernel Performance   PEND Check plug-in received no monitoring data
Memory               PEND Check plug-in received no monitoring data
Mount options of /etc/pve PEND Check plug-in received no monitoring data
NTP Time             PEND Check plug-in received no monitoring data
Number of threads    PEND Check plug-in received no monitoring data
PVE Cluster State    PEND Check plug-in received no monitoring data
Proxmox VE Memory Usage Usage: 58.30% - 8.98 GiB of 15.4 GiB
Proxmox VE Node Info Status: online, Subscription: notfound, Version: 9.1.4, Hosted VMs: 14x LXC, 1x Qemu
Systemd Service Summary PEND Check plug-in received no monitoring data
Systemd Socket Summary PEND Check plug-in received no monitoring data
TCP Connections      PEND Check plug-in received no monitoring data
Temperature Zone 0   PEND Check plug-in received no monitoring data
Temperature Zone 1   PEND Check plug-in received no monitoring data
Uptime               Up since 2026-01-01 11:02:57, Uptime: 18 days 12 hours
zpool status         PEND Check plug-in received no monitoring data
1 piggyback files for 'node-2'.
Read piggyback file '/omd/sites/monitoring/tmp/check_mk/piggyback/node-2/node-1'
[cpu_tracking] Stop [7eff1d06dd60 - Snapshot(process=posix.times_result(user=0.020000000000000018, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.03000000026077032))]
[agent] Invalid characters in 'app-proxmox' for SectionName: '-'(?), [special_proxmox_ve] Success, [piggyback] Successfully processed from source 'node-1', Missing monitoring data for plugins, checkmk_agent(!), chrony(!), cpu_loads(!), cpu_threads(!), diskstat(!), kernel_performance(!), kernel_util(!), lnx_if(!), lnx_thermal(!), mem_linux(!), mounts(!), pvecm_status(!), systemd_units_services_summary(!), systemd_units_sockets_summary(!), tcp_conn_stats(!), zfsget(!), zpool_status(!), execution time 4.0 sec | execution_time=4.000 user_time=0.050 system_time=0.000 children_user_time=0.970 children_system_time=0.080 cmk_time_agent=2.130 cmk_time_ds=0.760
2

Hi and welcome to the forum!

The plugin name (and hence the section name) must not contain a dash/minus sign (‘-’) as in app-proxmox.

The last line of your output says so:

...
[agent] Invalid characters in 'app-proxmox' for SectionName: '-'(?), ...

The plugin name is checked by the code in the file ~/lib/python3/cmk/utils/validatedstr.py and there it says:

# A plug-in name must be a non-empty string consisting only
# of letters A-z, digits and the underscore.
VALID_CHARACTERS: Final = string.ascii_letters + "_" + string.digits

If possible, rename your section and plugin to app_proxmox.

1 Like
3

Thanks for your reaction, but I’m not aware of any plugin or section with this name, do you have any guidelines or documentation links where I should start searching?

4

Oh, I’m sorry, I was under the impression that app-proxmox was a plugin that you wrote.

Anyway, there must be some line in the agent output that looks like this:

<<<app-proxmox>>>

This would indicate the begin of a new section named app-proxmox to checkmk. The text that comes after this <<<app-proxmox>>> (= lines up to the next <<<section header>>>) would then be fed to a check plugin with the name app-proxmox (if such a plugin exists).

Maybe the agent output contains a line <<<app-proxmox>>> that is actually NOT a section header but just part of the regular output and checkmk misinterprets that?

I would download the agent output and check in which context the term app-proxmox appears.

5

Okay, I found it, but does this mean I can’t name by cluster Home-Lab or the nodes node-1 and node-2 or is this a bug in the agent?

<<<local:sep(0)>>>
<<<app-proxmox>>>
Home-Lab
103/net0/2266932800/1514950518/AdGuard-1
106/net0/1655538461/174646968/mattermost
107/net0/39146982160/37946256357/npm
109/net0/451474554618/35778902568/myspeed
110/net0/1000900216/171975140/watchyourlan
113/net0/749068524/1575814003/pulse
114/net0/5344970574/928928887/homepage
116/net0/2348410/1055034/checkmk
<<<<node-2>>>>
<<<proxmox_ve_node_info:sep(0)>>>
{"lxc": ["101", "102", "103", "105", "106", "107", "108", "109", "110", "111", "112", "113", "114", "116"], "proxmox_ve_version": {"release>
<<<proxmox_ve_mem_usage:sep(0)>>>
{"max_mem": 16532213760, "mem": 9219907584}
<<<uptime>>>
1652933
<<<<>>>>
6

What files do you have in /usr/lib/check_mk_agent on this host?

2 Likes
7

No, no. Home-Lab is fine and node-1 and node-2 are also fine.

The problem is the line <<<app-proxmox>>>. The minus sign in app-proxmox is not allowed. Section names (the characters between <<< and >>>) must only contain letters (A…Z, a…z), numbers (0…9) and an underscore (_).

The <<<app-proxmox>>> comes from the agent and that is the source of the problem, I would guess.

8

That nailed it!

That directory has two subdirectories, ./plugin is empty, but ./local contains a file “proxmox”.

This proxmox file was installed when I was looking at LibreNMS. LibreNMS also has the option to monitor proxmox guests and for that they are using the (a fork?) of check_mk_agent.

Copying the proxmox file to this directory is part of the installation guide. This proxmox files contains the <<<app-proxmox>>> tag.

Removing the file solved the problem!

2 Likes
9

Glad to hear. Robert had the right idea. I’m not 100% sure but I think section names with dashes were allowed in previous checkmk versions (before 2.4 or 2.3?). That might explain why the installation guide refers to that file.

It’s still strange because the scripts in the directory /usr/lib/check_mk_agent/local should usually not output any <<<section_headers>>> because all their output is part of the <<<local>>> section. :man_shrugging:

1 Like
10

Yes, anything that outputs anything else than lines in the Local Check format are not Local Check scripts and must not be put in that directory.

If a script outputs a section header it is an agent plugin and has to be put in the plugins directory.

2 Likes