RabbitMQ Agent dont use given IP

CMK version:
2.1.0p2 / 2.0.25
OS version:
Docker

Error message:

I did try to use the RabbitMQ agent from Checkmk.
It works :slight_smile: , but not when the host is only reachable via the configured IPv4 address, not the hostname.
It seams that the RabbitMQ agent try to connect to the hostname, but not via the given IPv4 address.
The native checkmk checks e.g. ping and agent use the given IP and work.

[agent] Success, [special_rabbitmq] Agent exited with code 1: Error connecting to RabbitMQ server: HTTPConnectionPool(host='myrabbit', port=15672): Max retries exceeded with url: /api/overview?columns=cluster_name,object_totals.connections,object_totals.channels,object_totals.queues,object_totals.consumers,queue_totals.messages,queue_totals.messages_ready,queue_totals.messages_unacknowledged,message_stats.publish,message_stats.publish_details.rate,message_stats.deliver_get,message_stats.deliver_get.rate,rabbitmq_version,erlang_version, (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f407fbce520>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))**CRIT** , execution time 0.9 sec

To reproduce this i started CMK latest with docker, added a host “myrabbit” with an valid IP or FQDN in the host config IPv4 address field. Configured the Agent to allow API and CMK Agent. In the last step i configured the RabbitMQ agent. If i change the hostname into a valid hostname, all works. But if the hostname is e.g. not a FQDN name but the IPv4 Address contains a valid FQDN or IP. RabbitMQ Agent fails to resolve the host.

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

OMD[cmk]:~$ cmk --debug -vvn myrabbit
Checkmk version 2.1.0p2
Try license usage history update.
Trying to acquire lock on /omd/sites/cmk/var/check_mk/license_usage/next_run
Got lock on /omd/sites/cmk/var/check_mk/license_usage/next_run
Trying to acquire lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Got lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Next run time has not been reached yet. Abort.
Releasing lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Released lock on /omd/sites/cmk/var/check_mk/license_usage/history.json
Releasing lock on /omd/sites/cmk/var/check_mk/license_usage/next_run
Released lock on /omd/sites/cmk/var/check_mk/license_usage/next_run
+ FETCHING DATA
  Source: SourceType.HOST/FetcherType.TCP
[cpu_tracking] Start [7f39123b2490]
[TCPFetcher] Fetch with cache settings: DefaultAgentFileCache(myrabbit, base_path=/omd/sites/cmk/tmp/check_mk/cache, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Not using cache (Too old. Age is 14 sec, allowed is 0 sec)
[TCPFetcher] Execute data source
Connecting via TCP to 10.10.4.197:6556 (5.0s timeout)
Detected transport protocol: TransportProtocol.PLAIN (b'<<')
Reading data from agent
Write data to cache file /omd/sites/cmk/tmp/check_mk/cache/myrabbit
Trying to acquire lock on /omd/sites/cmk/tmp/check_mk/cache/myrabbit
Got lock on /omd/sites/cmk/tmp/check_mk/cache/myrabbit
Releasing lock on /omd/sites/cmk/tmp/check_mk/cache/myrabbit
Released lock on /omd/sites/cmk/tmp/check_mk/cache/myrabbit
Closing TCP connection to 10.10.4.197:6556
[cpu_tracking] Stop [7f39123b2490 - Snapshot(process=posix.times_result(user=0.010000000000000009, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.5))]
  Source: SourceType.HOST/FetcherType.PROGRAM
[cpu_tracking] Start [7f39123672b0]
[ProgramFetcher] Fetch with cache settings: DefaultAgentFileCache(myrabbit, base_path=/omd/sites/cmk/tmp/check_mk/data_source_cache/special_rabbitmq, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Not using cache (Does not exist)
[ProgramFetcher] Execute data source
Calling: /omd/sites/cmk/share/check_mk/agents/special/agent_rabbitmq '-P' 'http' '-m' 'cluster,nodes,vhosts,queues' '-u' 'myuser' '-s' 'mypasswd' '--hostname' 'myrabbit'
[cpu_tracking] Stop [7f39123672b0 - Snapshot(process=posix.times_result(user=0.0, system=0.009999999999999981, children_user=0.23, children_system=0.04, elapsed=0.2900000000372529))]
  Source: SourceType.HOST/FetcherType.PIGGYBACK
[cpu_tracking] Start [7f39123674f0]
[PiggybackFetcher] Fetch with cache settings: NoCache(myrabbit, base_path=/omd/sites/cmk/tmp/check_mk/data_source_cache/piggyback, max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=True, use_outdated=False, simulation=False)
Not using cache (Cache usage disabled)
[PiggybackFetcher] Execute data source
No piggyback files for 'myrabbit'. Skip processing.
No piggyback files for '10.10.4.197'. Skip processing.
Not using cache (Cache usage disabled)
[cpu_tracking] Stop [7f39123674f0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
+ PARSE FETCHER RESULTS
  Source: SourceType.HOST/FetcherType.TCP
<<<check_mk>>> / Transition NOOPParser -> HostSectionParser
<<<labels:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<df>>> / Transition HostSectionParser -> HostSectionParser
<<<systemd_units>>> / Transition HostSectionParser -> HostSectionParser
<<<nfsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<cifsmounts>>> / Transition HostSectionParser -> HostSectionParser
<<<mounts>>> / Transition HostSectionParser -> HostSectionParser
<<<ps_lnx>>> / Transition HostSectionParser -> HostSectionParser
<<<mem>>> / Transition HostSectionParser -> HostSectionParser
<<<cpu>>> / Transition HostSectionParser -> HostSectionParser
<<<uptime>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_if:sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<tcp_conn_stats>>> / Transition HostSectionParser -> HostSectionParser
<<<diskstat>>> / Transition HostSectionParser -> HostSectionParser
<<<kernel>>> / Transition HostSectionParser -> HostSectionParser
<<<md>>> / Transition HostSectionParser -> HostSectionParser
<<<vbox_guest>>> / Transition HostSectionParser -> HostSectionParser
<<<job>>> / Transition HostSectionParser -> HostSectionParser
<<<local:sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<logwatch>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_packages:cached(1654767245,43200):persist(1654782242):sep(9)>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_distro:cached(1654767245,43200):persist(1654782242):sep(124)>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_cpuinfo:cached(1654767245,43200):persist(1654782242):sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<dmidecode:cached(1654767245,43200):persist(1654782242):sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_uname:cached(1654767245,43200):persist(1654782242)>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_video:cached(1654767245,43200):persist(1654782242):sep(58)>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_ip_r:cached(1654767245,43200):persist(1654782242)>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_sysctl:cached(1654767245,43200):persist(1654782242)>>> / Transition HostSectionParser -> HostSectionParser
<<<lnx_block_devices:cached(1654767245,43200):persist(1654782242):sep(0)>>> / Transition HostSectionParser -> HostSectionParser
<<<yum:cached(1654767250,43200)>>> / Transition HostSectionParser -> HostSectionParser
No persisted sections
  -> Add sections: ['check_mk', 'cifsmounts', 'cpu', 'df', 'diskstat', 'dmidecode', 'job', 'kernel', 'labels', 'lnx_block_devices', 'lnx_cpuinfo', 'lnx_distro', 'lnx_if', 'lnx_ip_r', 'lnx_packages', 'lnx_sysctl', 'lnx_uname', 'lnx_video', 'local', 'logwatch', 'md', 'mem', 'mounts', 'nfsmounts', 'ps_lnx', 'systemd_units', 'tcp_conn_stats', 'uptime', 'vbox_guest', 'yum']
  Source: SourceType.HOST/FetcherType.PROGRAM
  -> Not adding sections: Agent exited with code 1: Error connecting to RabbitMQ server: HTTPConnectionPool(host='myrabbit', port=15672): Max retries exceeded with url: /api/overview?columns=cluster_name,object_totals.connections,object_totals.channels,object_totals.queues,object_totals.consumers,queue_totals.messages,queue_totals.messages_ready,queue_totals.messages_unacknowledged,message_stats.publish,message_stats.publish_details.rate,message_stats.deliver_get,message_stats.deliver_get.rate,rabbitmq_version,erlang_version, (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe1390312b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
  Source: SourceType.HOST/FetcherType.PIGGYBACK
No persisted sections
  -> Add sections: []
Received no piggyback data
Received no piggyback data
Received no piggyback data
[cpu_tracking] Start [7f3912367e50]
value store: synchronizing
Trying to acquire lock on /omd/sites/cmk/tmp/check_mk/counters/myrabbit
Got lock on /omd/sites/cmk/tmp/check_mk/counters/myrabbit
value store: loading from disk
Releasing lock on /omd/sites/cmk/tmp/check_mk/counters/myrabbit
Released lock on /omd/sites/cmk/tmp/check_mk/counters/myrabbit
CPU load             15 min load: 0.19, 15 min load per core: 0.19 (1 cores)
CPU utilization      Total CPU: 4.46%
Check_MK Agent       Version: 2.0.0p17, OS: linux
Disk IO SUMMARY      Read: 0.00 B/s, Write: 28.5 kB/s, Latency: 1 millisecond
Filesystem /         25.87% used (3.36 of 12.99 GB), trend: -4.09 MB / 24 hours
Filesystem /boot/efi 0.57% used (5.80 of 1021.98 MB), trend: 0.00 B / 24 hours
Interface 2          [ens192], (up), MAC: XXX, Speed: 10 GBit/s, In: 54.9 kB/s (<0.01%), Out: 60.7 kB/s (<0.01%)
Kernel Performance   Process Creations: 7.56/s, Context Switches: 518.62/s, Major Page Faults: 0.00/s, Page Swap in: 0.00/s, Page Swap Out: 0.00/s
Log /var/log/messages No error messages
Memory               Total virtual memory: 6.35% - 367.25 MB of 5.64 GB, 9 additional details available
Mount options of /   Mount options exactly as expected
Mount options of /boot/efi Mount options exactly as expected
Number of threads    258, Usage: 0.87%
SystemInfo           OS: Red Hat Enterprise Linux 8.6 Ootpa, 4.18.0-372.9.1.el8.x86_64 (2022)
Systemd Service Summary Total: 102, Disabled: 8, Failed: 0
TCP Connections      Established: 19
Uptime               Up since Jun 01 2022 16:25:17, Uptime: 8 days 0 hours
No piggyback files for 'myrabbit'. Skip processing.
No piggyback files for '10.10.4.197'. Skip processing.
[cpu_tracking] Stop [7f3912367e50 - Snapshot(process=posix.times_result(user=0.010000000000000009, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.009999999776482582))]
[agent] Success, [special_rabbitmq] Agent exited with code 1: Error connecting to RabbitMQ server: HTTPConnectionPool(host='myrabbit', port=15672): Max retries exceeded with url: /api/overview?columns=cluster_name,object_totals.connections,object_totals.channels,object_totals.queues,object_totals.consumers,queue_totals.messages,queue_totals.messages_ready,queue_totals.messages_unacknowledged,message_stats.publish,message_stats.publish_details.rate,message_stats.deliver_get,message_stats.deliver_get.rate,rabbitmq_version,erlang_version, (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe1390312b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))(!!), execution time 0.8 sec | execution_time=0.800 user_time=0.020 system_time=0.010 children_user_time=0.230 children_system_time=0.040 cmk_time_agent=0.490 cmk_time_ds=0.010

If you configure your special agent this way.


It should work with the used IP.
Instance can be a completely different hostname or also an IP.