CMK version:
CMK 2.4.0p11 and newer OS version:
not relevant Error message:
no direct error but the following problem
At the moment there are existing around 800 checks inside ~/share/check_mk/checks/
Some of these checks have problems or do completely wrong things
Until now it was easy to fix this with copying the files to “~/local/share/check_mk/checks/” and make the needed modifications.
Now i see the problem that all the files there inside the “~/local/” folder are ignored and the problems cannot be fixed anymore → WHY?
Someone will say now this folder is deprecated and should not be used anymore. True but it is also not possible to fix this problem if the old check is moved to “~/local/lib/python3/cmk(_addons)/collection/agent_based/” and migrated to the new API syntax as you will now get the error message “duplicated section definition”.
That means at the end - it is not possible to fix old old checks inside ~/share/check_mk/checks/.
We also struggle with these kinds of changes every year when testing updates. After working with Checkmk for nine years, we’ve gotten used to it and learned to adapt.
In the end, there’s really no escaping this if we want to move forward and support better APIs for development.
We’ve been aware of this for quite some time, and it has been official since April 2024:
That’s the part I don’t really understand.
Why isn’t it possible to transfer the logic of a legacy plugin into the new API? Of course, it can be quite a bit of work, but at the same time, it also offers opportunities to improve both the code and its functionality.
The handling of clustered checks, for example, is already much better in the new API compared to the legacy checks.
Maybe I’m missing something here, since I’m not that deep into development anymore—I mainly coordinate things during updates—but I’d really like to understand the challenges behind this.
I remember years ago when it was discussed to remove the legacy API I asked a checkmk developer when this may happen and he said: not so soon because we also having to migrate thousands of plugins which will last a while.
From a developer point of view I would agree but I guess the decision is related to support. Support and development are two independent departments and the situation that a specific functionality isnt supported officially but still in use internal is quite usual.
Why legacy API is forcibly removed from local will have its reason but technically I dont understand the reason as long as the oob legacy plugins are not migrated.
Maybe one from checkmk team can explain.
It is not a problem with the migration to the new API. The problem is that if a included check (not my own checks) has a problem or bug you fixed this with copy to the ~/local/ structure and thats fine.
This works not anymore for the over 800 old checks inside ~/share/check_mk/checks/ as you cannot shadow the files with your own file inside ~/local/share/…
And as i wrote if you migrate this check with the bug/problem directly to the new API then you get the duplicate section error. This was also possible to fix before with an empty file inside ~/local/share/check_mk/checks/ as a shadow file. At the moment this is also not working anymore.
That means at the moment the shadow mechanic is broken for all the files inside ~/share/check_mk/checks/.
One really ugly workaround for users of self-managed, Debian-based OS might be a two-step approach:
Use path-exclude=… in /etc/dpkg/dpkg.conf.d to prevent the installation of the affected files in the first place. With a path-exclude for dpkg (see man 1dpkg, search for --path-exclude=) you can use wildcards, meaning something like this might work: path-exclude=/opt/omd/versions/*/share/check_mk/checks/bad_check might work. You might be able to use dpkg-divert to similar effect, but it doesn’t support wildcards AFAIK & will habe to be done for each installed version. If you automate your deployment, that won’t be a problem, though.
At the same time convert the bad_check to the new API.
Thank you very much for this explanation, now I better understand the topic. We have a couple of such situation’s but until now I dont have a feedback from developers that it wont work anymore. I will follow up on this.
That would be a bug. This was specifically meant to work. I can see that having to migrate to the new API to fix a bug is a pain (*), but for any plugin there must be a way to fix it locally (because we also have to be able to fix it via MKP in a support case).
I just tested it with 3ware_disks, and I didn’t have any issues. Could it be that you are dealing with legacy check plugins for which the sections are in fact already migrated?
Which plugin(s) are you dealing with, specifically?
*) It also is a pain for us. Not loading the local checks folder is a measure to force the new API on ourselves, not on you. It is a side-effect, of course.
I must say that i think the complete shadow mechanic is broken at the moment.
Today again a system where i wanted to fix check already existing in the new format under “~/lib/python3/cmk/plugins/<name>/agent_based/” copied to “~/local/lib/python3/cmk/plugins/<name>/agent_based/” made the modifications and the files is completely ignored by the system. If i move the same file into another folder i get the expected error message that the same agent section is already defined in file XY.
The current situation is very frustrating as it is nearly impossible to debug any problems. You don’t know what the systems is doing - you have no proper error messages
Addition: it looks like a complet random behavior - plugins inside the collection folder can be modified without problem and as example all files inside prism cannot be modified and are ignored
No if it is an included check then you need to place it inside “…/python3/cmk/plugins/…”.
Only our own extensions need to be placed inside “…/cmk_addons/…”
I must say that i think the complete shadow mechanic is broken at the moment.
That is very unlikely. (But it seems we need more extensive documentation.)
Maybe I can clear a few things up here. TL;DR: Look for __init__.py files. Folders with them are the smalles entities you can shadow.
We no longer implement a custom shadowing logic here. The shadowing logic is determined exclusively by the standard python import mechanics. What is very useful here is the python namespace concept. I am afraid in this case it is also the cause for confusion. In a nutshell: Python imports work similar to finding commands on a console. We have the sys.path, which determines in which order we look where. However there is one complication: If there is an __init__.py somewhere, that is the smallest unit that can be shadowed. And the first folder where the __init__.py is found wins.
Example: cmk/plugins/fritzbox. This contains an __init__. As a result, it is “atomic” in the sense that if you want to shadow it, you need to put an __init__ file in your local version as well, and then all shipped files from that folder will be shadowed. If you don’t put an __init__, your files will be completely ingnored. The point here: Everything stays consistent. You can shadow it, but then you must take care of the whole “family”.
This makes sense wherever we have coherent plugin families.
Counter example: On the other hand: For the collection family (read: “everything we haven’t gotten round to sort properly yet”) that does not make sense of course: We really want to be able to shadow the files in there individually.
Hm . Maybe you should open a ticket so we can look at this. All I can say from here is that we stopped loading the legacy checks from the local directory in 2.4.0b2.
Hello Andreas, do you have a solution for the ignored checks in ~local/share/check_mk/checks/ ? We need to be able to modify the behavior of a few of the built-in checks, and I can’t get it done anymore!
Yes after studying the comments from @moritz a little bit more i found that you need to move these old style checks from “~/share/check_mk/checks/” to “~/local/lib/python3/cmk/plugins/collection/agent_based/” or “../cmk_addons/..”. I prefer the “../cmk/…” for the modified internal files that it is clear from where these changes are comming.
Then you have to migrate the copied checks to the new API style and you can modify it.
If the section & check names are identical these overwrite the old sections and checks from the share folder.
. Maybe you should open a ticket so we can look at this. All I can say from here is that we stopped loading the legacy checks from the local directory in