Parsing of section mtr failed

**CMK version:Checkmk Raw Edition 2.1.0p17
**OS version:Ubunt 20.04 server
Error message: vanished: mtr: Mtr to hocchudong [.] com Parsing of section mtr failed - please submit a crash report!

Dear !

I use mtr plugin to monitor icmp, install environment checkcmk as above.
I copied the plugin to the plugin directory on the host and configured the mtr file

I get error like in this post and below result. Please help me to solve this problem!
Thanks

File /etc/check_mk/mtr.cfg

root@congtt-ipam:~# cat /etc/check_mk/mtr.cfg
# Copyright (C) 2019 tribe29 GmbH - License: GNU General Public License v2
# This file is part of Checkmk (https://checkmk.com). It is subject to the terms and
# conditions defined in the file COPYING, which is part of this source code package.

# Mtr Checkmk configuration

# NOTE: your MTR report shouldn't take longer than 15 minutes

# [DEFAULTS]
# type=icmp    # icmp, tcp or udp
# count=10     # number of pings per mtr report
# force_ipv4=0 # force ipv4, exclusive with force_ipv6
# force_ipv6=0 # force ipv6, exclusive with force_ipv4
# size=64      # packet size
# time=0       # minimum time between runs, 0 / default means run if mtr doesn't run anymore
# port=80      # UDP/TCP port to connect to
# dns=0        # Use DNS resolution to lookup addresses
# address=     # Bind to source address
# interval=    # time MTR waits between sending pings
# timeout=     # ping Timeout, see mtr man page

[hocchudong.com]
type = icmp
force_ipv4 = true
dns = 0

[8.8.8.8]
type = icmp
force_ipv4 = true
dns = 0

#[ipv6.google.com]
#type = icmp
#force_ipv6 = true

#[mathias-kettner.de]
#type = tcp
#force_ipv4 = true

**Output of “cmk --debug -vvn hostname”

Checkmk version 2.1.0p17
Try license usage history update.
Trying to acquire lock on /omd/sites/mon/var/check_mk/license_usage/next_run
Got lock on /omd/sites/mon/var/check_mk/license_usage/next_run
Trying to acquire lock on /omd/sites/mon/var/check_mk/license_usage/history.json
Got lock on /omd/sites/mon/var/check_mk/license_usage/history.json
Next run time has not been reached yet. Abort.
Releasing lock on /omd/sites/mon/var/check_mk/license_usage/history.json
Released lock on /omd/sites/mon/var/check_mk/license_usage/history.json
Releasing lock on /omd/sites/mon/var/check_mk/license_usage/next_run
Released lock on /omd/sites/mon/var/check_mk/license_usage/next_run
+ FETCHING DATA
  Source: SourceType.HOST/FetcherType.TCP
[cpu_tracking] Start [7f21a4ec3310]
[TCPFetcher] Fetch with cache settings: DefaultAgentFileCache(ipam-20.6, base_path=/omd/sites/mon/tmp/check_mk/cache, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Not using cache (Too old. Age is 56 sec, allowed is 0 sec)
[TCPFetcher] Execute data source
Connecting via TCP to 192.168.20.6:6556 (5.0s timeout)
Detected transport protocol: TransportProtocol.TLS (b'16')
Reading data from agent via TLS socket
Reading data from agent
Detected transport protocol: TransportProtocol.PLAIN (b'<<')
Write data to cache file /omd/sites/mon/tmp/check_mk/cache/ipam-20.6
Trying to acquire lock on /omd/sites/mon/tmp/check_mk/cache/ipam-20.6
Got lock on /omd/sites/mon/tmp/check_mk/cache/ipam-20.6
Releasing lock on /omd/sites/mon/tmp/check_mk/cache/ipam-20.6
Released lock on /omd/sites/mon/tmp/check_mk/cache/ipam-20.6
Closing TCP connection to 192.168.20.6:6556
[cpu_tracking] Stop [7f21a4ec3310 - Snapshot(process=posix.times_result(user=0.009999999999999787, system=0.0, children_user=0.0, children_system=0.0, elapsed=6.469999998807907))]
  Source: SourceType.HOST/FetcherType.PIGGYBACK
[cpu_tracking] Start [7f21a4e1c220]
[PiggybackFetcher] Fetch with cache settings: NoCache(ipam-20.6, base_path=/omd/sites/mon/tmp/check_mk/data_source_cache/piggyback, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=True, use_outdated=False, simulation=False)
Not using cache (Cache usage disabled)
[PiggybackFetcher] Execute data source
No piggyback files for 'ipam-20.6'. Skip processing.
No piggyback files for '192.168.20.6'. Skip processing.
Not using cache (Cache usage disabled)
[cpu_tracking] Stop [7f21a4e1c220 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
+ PARSE FETCHER RESULTS
  Source: SourceType.HOST/FetcherType.TCP
<<<check_mk>>> / Transition NOOPParser -> HostSectionParser
<<<cmk_agent_ctl_status:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<checkmk_agent_plugins_lnx:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<systemd_units>>> / Transition HostSectionParser -> HostSectionParser
<<<nfsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<cifsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<mounts>>> / Transition HostSectionParser -> HostSectionParser
<<<ps_lnx>>> / Transition HostSectionParser -> HostSectionParser
<<<mem>>> / Transition HostSectionParser -> HostSectionParser
<<<cpu>>> / Transition HostSectionParser -> HostSectionParser
<<<uptime>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if:sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<tcp_conn_stats>>> / Transition HostSectionParser -> HostSectionParser
<<<diskstat>>> / Transition HostSectionParser -> HostSectionParser
<<<kernel>>> / Transition HostSectionParser -> HostSectionParser
<<<md>>> / Transition HostSectionParser -> HostSectionParser
<<<vbox_guest>>> / Transition HostSectionParser -> HostSectionParser
<<<job>>> / Transition HostSectionParser -> HostSectionParser
<<<chrony:cached(1670395527,30)>>> / Transition HostSectionParser -> HostSectionParser
<<<local:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<apt:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<mtr:sep(124)>>> / Transition HostSectionParser -> HostSectionParser
No persisted sections
  -> Add sections: ['apt', 'check_mk', 'checkmk_agent_plugins_lnx', 'chrony', 'cifsmounts', 'cmk_agent_ctl_status', 'cpu', 'df', 'diskstat', 'job', 'kernel', 'labels', 'lnx_if', 'local', 'md', 'mem', 'mounts', 'mtr', 'nfsmounts', 'ps_lnx', 'systemd_units', 'tcp_conn_stats', 'uptime', 'vbox_guest']
  Source: SourceType.HOST/FetcherType.PIGGYBACK
No persisted sections
  -> Add sections: []
Received no piggyback data
Received no piggyback data
[cpu_tracking] Start [7f21a4dd5310]
value store: synchronizing
Trying to acquire lock on /omd/sites/mon/tmp/check_mk/counters/ipam-20.6
Got lock on /omd/sites/mon/tmp/check_mk/counters/ipam-20.6
value store: loading from disk
Releasing lock on /omd/sites/mon/tmp/check_mk/counters/ipam-20.6
Released lock on /omd/sites/mon/tmp/check_mk/counters/ipam-20.6
APT Updates          64 normal updates(!), 7 security updates (login, passwd, libfdisk1, fdisk, ca-certificates, libxml2, libtiff5)(!!)
CPU load             15 min load: 0.16, 15 min load per core: 0.08 (2 cores)
CPU utilization      Total CPU: 77.03%
Check_MK Agent       Version: 2.1.0p17, OS: linux, Agent plugins: 2, Local checks: 1
Disk IO SUMMARY      Read: 0.00 B/s, Write: 86.0 kB/s, Latency: 8 milliseconds
Filesystem /         9.95% used (4.72 of 47.48 GB), trend: -59.91 MB / 24 hours
Filesystem /boot/efi 0.0008% used (4.00 kB of 510.98 MB), trend: 0.00 B / 24 hours
Hostname_IPs         localhost // 192.168.20.6
Hypervisor           Hypervisor(s): vmware (guessed by virt-what)
Interface 2          [eth0], (up), MAC: 00:50:56:84:A8:03, Speed: 10 GBit/s, In: 61.5 kB/s (<0.01%), Out: 2.67 kB/s (<0.01%)
Kernel Performance   Process Creations: 96.25/s, Context Switches: 3119.25/s, Major Page Faults: 0.00/s, Page Swap in: 0.00/s, Page Swap Out: 0.00/s
Kernel_Cmdline       BOOT_IMAGE=/boot/vmlinuz-5.4.0-132-generic root=/dev/mapper/vgubuntu20--04-root ro ksdevice=bootif lang= biosdevname=0 interface=eth0 text net.ifnames=0 ipv6.disable=1 ipv6.disable=1
LSB_Release          Ubuntu 20.04.1 LTS
Memory               Total virtual memory: 6.23% - 306.18 MB of 4.80 GB, 9 additional details available
Mount options of /   Mount options exactly as expected
Mount options of /boot/efi Mount options exactly as expected
[cpu_tracking] Stop [7f21a4dd5310 - Snapshot(process=posix.times_result(user=0.020000000000000018, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.010000001639127731))]
Trying to acquire lock on /omd/sites/mon/var/check_mk/crashes/base/d62eef24-75fa-11ed-b7fb-fbe487b01be6/crash.info
Got lock on /omd/sites/mon/var/check_mk/crashes/base/d62eef24-75fa-11ed-b7fb-fbe487b01be6/crash.info
Releasing lock on /omd/sites/mon/var/check_mk/crashes/base/d62eef24-75fa-11ed-b7fb-fbe487b01be6/crash.info
Released lock on /omd/sites/mon/var/check_mk/crashes/base/d62eef24-75fa-11ed-b7fb-fbe487b01be6/crash.info
Traceback (most recent call last):
  File "/omd/sites/mon/bin/cmk", line 98, in <module>
    exit_status = modes.call("--check", None, opts, args)
  File "/omd/sites/mon/lib/python3/cmk/base/modes/__init__.py", line 69, in call
    return handler(*handler_args)
  File "/omd/sites/mon/lib/python3/cmk/base/modes/check_mk.py", line 1804, in mode_check
    checking.commandline_checking(
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/decorator.py", line 43, in wrapped_check_func
    status, output_text = _combine_texts(check_func(hostname, *args, **kwargs))
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/checking/__init__.py", line 121, in commandline_checking
    return _execute_checkmk_checks(
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/checking/__init__.py", line 174, in _execute_checkmk_checks
    num_success, plugins_missing_data = check_host_services(
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/checking/__init__.py", line 322, in check_host_services
    success = _execute_check(
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/checking/__init__.py", line 382, in _execute_check
    submittable = get_aggregated_result(
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/checking/__init__.py", line 443, in get_aggregated_result
    section_kws, error_result = _get_monitoring_data_kwargs(
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/checking/__init__.py", line 543, in _get_monitoring_data_kwargs
    get_section_kwargs(
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/utils.py", line 35, in get_section_kwargs
    kwargs = {
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/utils.py", line 36, in <dictcomp>
    key: parsed_sections_broker.get_parsed_section(host_key, parsed_section_name)
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/data_provider.py", line 268, in get_parsed_section
    if (resolved := resolver.resolve(parser, parsed_section_name)) is None
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/data_provider.py", line 188, in resolve
    if (parsing_result := parser.parse(producer)) is not None:
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/data_provider.py", line 104, in parse
    if (parsed := self._parse_raw_data(section)) is None
  File "/omd/sites/mon/lib/python3/cmk/base/agent_based/data_provider.py", line 122, in _parse_raw_data
    return section.parse_function(list(raw_data))
  File "/omd/sites/mon/lib/python3/cmk/base/plugins/agent_based/mtr.py", line 28, in parse_mtr
    return {
  File "/omd/sites/mon/lib/python3/cmk/base/plugins/agent_based/mtr.py", line 42, in <dictcomp>
    for hostname, hopcount, rest in [(line[0], int(float(line[2])), line[3:])]
IndexError: list index out of range

** Output cmk -d hostname:

<<<mtr:sep(124)>>>
**ERROR** Report file /var/lib/check_mk_agent/mtr.report.hocchudong-com has less than 3 lines, expecting at least 1 hop! Throwing away invalid report
hocchudong.com|1670386576|10|192.168.20.1|0.0%|10|0.7|0.8|0.7|0.9|0.1|103.74.123.1|0.0%|10|1.3|1.5|1.3|1.7|0.1|169.254.4.2|0.0%|10|1.2|3.4|1.1|12.9|3.8|42.112.4.25|0.0%|10|1.5|3.2|1.5|15.6|4.4|1.53.123.85|0.0%|10|1.8|1.7|1.5|1.9|0.1|118.69.253.73|0.0%|10|21.5|21.7|21.3|22.7|0.4|118.69.184.73|0.0%|10|22.0|25.1|21.9|51.9|9.4|103.22.203.118|40.0%|10|22.8|25.6|22.6|33.5|4.2|172.71.216.2|0.0%|10|21.0|23.9|21.0|42.5|6.6|172.67.135.248|0.0%|10|22.5|22.1|21.9|22.5|0.2
**ERROR** Report file /var/lib/check_mk_agent/mtr.report.8-8-8-8 has less than 3 lines, expecting at least 1 hop! Throwing away invalid report
8.8.8.8|0|0
1 Like

Hi, we are having same issue. Any updates on this?

1 Like

same here with the latest version.

However, when running check_mk_agent manually on the cli the mtr check is working for one single interval.
For scheduled mtr checks i’m getting the same message “has less than 3 lines, expecting at least 1 hop! Throwing away invalid report”

As manual runs of mtr.py are working as expected I implemented a nasty workaround.

I added this to crontab - mtr is running every minute and results and graphs are shown in check_mk

* * * * * python3 /usr/lib/check_mk_agent/plugins/mtr.py

BTW, I was able to reproduce the problem on Rocky LInux 9.2 when I configure an IP which is not reachable. I have opened an internal ticket on this problem.

2 Likes

Its fixed as part of this werk mtr: Fix section parsing error

2 Likes