Cached plugins not working (cache not written)

I’m using the latest version, 2.0.0p12 both on checkmk sever and on agent.

I used to have some cached plugins, as recommended, sshd_config and apt.
I noticed that the apt plugin wasn’t updating, it reported for many days that some updates were required but there were none, so I deleted the cache with rm /var/lib/check_mk_agent/cache/*

After that, those files were never recreated again, and I get the service problem “[agent] Version: 2.0.0p12, OS: linux, Missing monitoring data for check plugins: apt, sshd_configWARN, execution time 2.2 sec”

If I run, as root, check_mk_agent | grep -A 5 sshd_config the files are created (it doesn’t output them the first time, but if I run that again they are in the output)

So I thought it was a permission issue, I did a chmod 777 /var/lib/check_mk_agent/cache/ since I don’t know what user will write them, but nothing changes, the files aren’t created automatically…

Same issue happening in multiple servers, xinetd is running


To replicate the issue, you can download the mk_apt plugin inside /usr/lib/check_mk_agent/plugins/3600/
This is how cached plugins used to work, now I don’t see much documentation about this feature

Is the agent started by xinetd or via systemd socket unit?

I have no idea, I have the /etc/xinetd.d/check_mk created by checkmk, I only changed the only_from line:

service check_mk
{
        type           = UNLISTED
        port           = 6556
        socket_type    = stream
        protocol       = tcp
        wait           = no
        user           = root
        server         = /usr/bin/check_mk_agent

        # To avoid intentional or unintentional overload due to too many parallel
        # queries from one source we set this parameter. It limits the number of
        # concurrent connections per source address. If you need more requests
        # per source system, you can of course increase or remove this value
        # (https://github.com/tribe29/checkmk/pull/157)
        per_source     = 3

        # listen on IPv4 AND IPv6 when available on this host
        #flags          = IPv6

        # If you use fully redundant monitoring and poll the client
        # from more then one monitoring servers in parallel you might
        # want to use the agent cache wrapper:
        #server         = /usr/bin/check_mk_caching_agent

        # configure the IP address(es) of your Nagios server here:
        only_from      =  xxx.xxx.xxx.xxx 127.0.0.1

        # Don't be too verbose. Don't log every check. This might be
        # commented out for debugging. If this option is commented out
        # the default options will be used for this service.
        log_on_success =

        disable        = no
}

I never did anything else and they always worked, is there something else to do?

I actually see some errors running service xinetd status:

Oct 19 07:47:02 web08 xinetd[4889]: bind retry attempt 9
Oct 19 07:47:02 web08 xinetd[4889]: bind failed (Address already in use (errno = 98)). service = check_mk
Oct 19 07:47:02 web08 xinetd[4889]: bind retry attempt 10
Oct 19 07:47:02 web08 xinetd[4889]: bind failed (Address already in use (errno = 98)). service = check_mk

EDIT: It looks there is systemd listening on 6556

tcp6 0 0 :::6556 :::* LISTEN 1/systemd

systemctl list-sockets:

[::]:6556 check_mk.socket check_mk@188662-37.187.135.204:6556-51.15.138.178:33766.service

Please check the status of check-mk-agent-async.service with

systemctl status check-mk-agent-async.service

and start it if it’s not running with

systemctl start check-mk-agent-async.service
1 Like

In the default configuration the check_mk_agent should be managed by systemd (as the socket suggests). Not sure why the xinetd config has not been removed. Try removing the xinetd service manually.
You should have three Checkmk related systemd unit files: “check-mk-agent-async.service” (enabled and active), “check-mk-agent.socket” (enabled and active), and the template “check-mk-agent@.service”.
I think they’re called “check_mk” instead of “check-mk-agent” in the RawEdition.

1 Like

I think this is a very old file from before 1.4 which remained on the system. Today this file is called /etc/xinetd.d/check_mk_agent. I hae ssen similar occasions where this file was left over from an old agent installation.

2 Likes

Thank you for all the answers, I have removed xinetd.
I have only these check_mk services:

All the monitoring is working, except the plugins that should have the output cached…

image

Updating the agent from 2.0.0p12 to 2.0.0p17 (after having removed xinetd) fixed the issue, I think this is what did the fix:

# dpkg -i check-mk-agent_2.0.0p17-1_all.deb
(Reading database … 80631 files and directories currently installed.)
Preparing to unpack check-mk-agent_2.0.0p17-1_all.deb …
Unpacking check-mk-agent (2.0.0p17-1) over (2.0.0p12-1) …
Setting up check-mk-agent (2.0.0p17-1) …
Enable Checkmk Agent in systemd…
Created symlink /etc/systemd/system/multi-user.target.wants/check_mk-async.service → /etc/systemd /system/check_mk-async.service.

The service was still disabled but I did a systemctl enable + systemctl start and now everything works :champagne:

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.