Mkp list command no longer machine readable; please add parsable output

With the latest 2.1.0p21 the output of mkp list has changed. It used to look like what the git guys call a plumbing command (concise and machine readable output) but is now porcellain at its best. The output is now a table with no delimiters. Werk 15115 shows an example:

Until 2.1.0p20

OMD[mySite]$ mkp list
my_package

Since 2.1.0p21

OMD[mySite]:~$ mkp list
Name            Version Title                 Author       Req. Version Until Version Files State
--------------- ------- --------------------- ------------ ------------ ------------- ----- -----------------------------
my_package      1.0     A test package        tribe29 GmbH 1.6.0        None          1     Enabled (active on this site)
another-package 1.0     A nother test package tribe29 GmbH 2.0.0        2.1.0         42    Enabled (inactive on this site)

This looks great for humans but there is no reasonable way to get that parsed.

My suggestion is to add an option to the mkp command so it returns that table either in a machine readable format (proper delimiter, e.g. a tab character) or – even better – to return a JSON structure with the data.

Btw, this is not the only change with the mkp command: in 1.6 it was possible to install the very same MKP (same name, same version) over and over again without any problem. Since 2.x the command complains (“XY already installed in version Z”) and stops with a bad return code. This also breaks scripted installations.

5 Likes

It should be not to hard, to add a details switch to the mkp list command so it will not break anything.

1 Like

I think there is a bug introduced.
in p20

cmk -P list

was different to

cmk -vP list

now it is the same.

2 Likes

So we already had a verbose, human readable output with mkp -v list that is now the default?

Is that a case of new developers in the team not knowing what the software already is capable of?

1 Like

More changes incoming :slight_smile:

1 Like

This is the responsible commit.

I think, if this is in preparation for 2.2 it would also be helpful to have the machine readable JSON output before 2.2 :wink:

2 Likes

Yeah, that’s the purpose of the changes. As MKPs have always been a source of upgrade issues, the first steps are to increase transparency.
As the commands purpose is to increase visibility over existing MKPs during a manual upgrade process, what would be the purpose to have that machine-readable?

1 Like

The purpose of having machine readable output is being able to handle the installation and removal of MKPs with Ansible, for example.

We deploy checkmk servers with Ansible and also install MKPs that way. Previously, we could just mkp install our_package.mkp no matter the version or whether our_package.mkp was already installed. Now this breaks if the same package and version is already installed. However, we found a solution for that. It’s pretty cumbersome, but it works.

We also use the mkp list command to get a list of installed packages. This output has now changed. I know how to pick just the first column of that new output, but I would appreciate if we could just rely on the output or at least have an option to force machine readable output.

2 Likes

I just did some major overhaul on the mkps.html article in our master branch (sorry, German only for the next days). Feel free to check after midnight to find out what’s new in 2.2.0.

And yes: we have to better communicate breaking changes, and we have to better communicate which command output is intended for parsing by scripts and which is only intended to be human-readable.

OMD[heute]:~$ mkp list --help
usage: mkp list [-h] [--json]

Show a table of all known files, including the deployment state

options:
  -h, --help  show this help message and exit
  --json      format output as json

4 Likes

Awesome! Thank you very much, @martin.hirschvogel
I’ll mark this as the solution as soon as we can download a version containing this improvement.

1 Like

Hi Everybody!
I’m sorry to see that this causes you trouble (although a certain degree of trouble was expected).
Some dramatic changes to the packaging manager had to be introduced to solve upgrade blockers for a few customers, namely this: Extension packages: Support different versions in distributed setups

This led to a lot of follow up fixes and changes in behavior, far more than is good for a stable version.
For instance: The concept that you might want to compare versions and decide which one is newer was completely unknown to the system.

It’s always tricky to change the commands if you don’t know how they are used (or meant to be used).
I’ve decided to add the --json flag to make it clear that we have one output format for machines, and one for humans. Bringing this to 2.1 would have required even more changes to the code, so I decided against it in favor of stability. I feel that in this case unfortunately we have no winning move left :frowning:

For what it’s worth, you can parse the output:
mkp list | grep "(active on this site)$" | cut -f1 -d' '
Whether that is a “reasonable way” is a matter of opinion, I’ll give you that. But I don’t think it’ll break.

If you are a power user of the package managers command line interface, I recommend you test the 2.2 version when it is in beta – that’s when changes are still fairly easy to make, in particular if something changed anyway.

The use case of scripted installations certainly should not be made more complicated than necessary.
In particular making the commands add and enable idempotent would probably help; but we’d have to make sure that we’re actually installing the identical package – not only one with the same name and version. I think we should fail if the content changed, but neither name nor version.

6 Likes

Moritz, thank you for your detailed explanation. I’m a developer myself and know exactly what you’re talking about. For now, I can help myself with some simple parsing similar to your code example. I’ll try the behaviour in 2.2 as soon as it becomes available. The --json switch is always a good idea, because it’s so popular and perfectly machine readable. TY.