Found no services yet

Troubleshooting
1

I have one Linux server that just won’t work on check_mk. All the others can be added fine and are working fine.

When trying to scan this problem server for services it says:

Full scan finished after 1.96 s at 2022-02-11 15:46:15. Found no services yet. To retry please execute a full scan.

On the target machine I see /var/log/messages says:

xinetd[2150]: FAIL: check_mk address from=::ffff:1.2.3.4 (IP hidden)

Any help welcome :slight_smile:

CMK version:
Checkmk 1.6.0p14

OS version:
CentOS 7

Error message:
On agent /var/log/messages => xinetd[2150]: FAIL: check_mk address from=::ffff:1.2.3.4

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

OMD[core]:~$ cmk --debug -vvn example.com (hostname hidded)
[cpu_tracking] Start with phase 'busy'
Check_MK version 1.6.0p14
Try aquire lock on /omd/sites/core/tmp/check_mk/counters/example.com (hostname hidded)
Got lock on /omd/sites/core/tmp/check_mk/counters/example.com (hostname hidded)
Releasing lock on /omd/sites/core/tmp/check_mk/counters/example.com (hostname hidded)
Released lock on /omd/sites/core/tmp/check_mk/counters/example.com (hostname hidded)
+ FETCHING DATA
[cpu_tracking] Push phase 'agent' (Stack: ['busy'])
 [agent] No persisted sections loaded
 [agent] Not using cache (Does not exist)
 [agent] Execute data source
 [agent] Connecting via TCP to 1.1.1.1:6556 (60.0s timeout) (IP Hidden)
 [agent] Reading data from agent
 [agent] ERROR: Empty output from agent at TCP port 6556
[cpu_tracking] Pop phase 'agent' (Stack: ['busy', 'agent'])
CRIT - Empty output from agent at TCP port 6556
2

You can only check your xinetd config on the target system. Is the check_mk_agent working fine if executed manually on this system?

3

Thanks for your help!

If I run /usr/bin/check_mk_agent on the target system, it outputs LOTS of stuff. Seem’s its working when run manually.

4

And now the xinetd config for the agent service is there inside some configuration for only allowed from address?

5

No, it’s completely standard from install. I can add the OMD server IP to be allowed as a test. I’ll try that.

I added only_from = 1.2.3.4 to /etc/xinetd.conf and systemctl restart xinetd.service but same issue. I’m stumped.

7

You can also do a “telnet targetip 6556” from your monitoring server for tests. The agent command is also correct inside the xinetd service i think or? Your error message says that it is blocked because of the wrong IP.

8

The telnet does connect, but that is all:

[root@monitor ~]# telnet <target ip> 6556
Trying 1.2.3.4...
Connected to 1.2.3.4.
Escape character is '^]'.
Connection closed by foreign host.

Can you please clarify? > The agent command is also correct inside the xinetd service

I’m not sure what to check for this.

I have compaired /etc/systemd/system/multi-user.target.wants/xinetd.service and /etc/systemd/system/check_mk@.service to other working agents and it is the same,

Thanks!

9

Did you add this “IP hidden” yourself or is this really part of the log entry?

Looks like the “::ffff:1.2.3.4” is the full IPv6 address you need to allow in xinetd (and perhaps also in hosts.allow).
Note the “::ffff:” prefix, not just the “1.2.3.4” part.
The whole thing is an IPv6 address which maps the IPv4 address “1.2.3.4”.

10

First i would check again what @martin.schwarz wrote. Then please don’t mix the xinetd and systemd services. If you use xinetd the check_mk@.service should be disabled. You can only have one of booth.

11

Thanks, Martin!

Yes, I did add the (IP hidden). It was the correct OMD IPv4 address but starting with ::ffff:. On reading, I found that xinetd binds to IPv6 socket, but also check IPv4 at the same time, so should be fine?

[root@example ~]# netstat -an | grep 6556
tcp6       0      0 :::6556                 :::*                    LISTEN

I did also try only_from = ::ffff:1.2.3.4 to /etc/xinetd.conf but it did not help.

I’ve never had a server not work on checkmk before. It is strange.

12

I may have gotten confused, there is no systemctl for check_mk, sorry!

[root@example ~]# systemctl status check_mk
Unit check_mk.service could not be found.

[root@example ~]# systemctl status xinetd
● xinetd.service - Xinetd A Powerful Replacement For Inetd
   Loaded: loaded (/usr/lib/systemd/system/xinetd.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2022-02-11 18:39:33 GMT; 1s ago
  Process: 10527 ExecStart=/usr/sbin/xinetd -stayalive -pidfile /var/run/xinetd.pid $EXTRAOPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 10528 (xinetd)
   CGroup: /system.slice/xinetd.service
           └─10528 /usr/sbin/xinetd -stayalive -pidfile /var/run/xinetd.pid
13

I have uninstalled xinetd and now see systemd on port 6556;

[root@example ~]# netstat -lpn | grep 6556
tcp6       0      0 :::6556                 :::*                    LISTEN      1/systemd

Its now working.

Weird.

Thanks all for the helps :slight_smile:

14

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.