How to download latest version of MK programatically

Troubleshooting
, , ,
1

Hi there,
I have a playbook that updates my Checkmk installation to the latest version and deploys the latest agents to all my hosts. I download and install the latest server version like this:

---
- name: update check_mk server
  hosts: localhost
  gather_facts: no
  tasks:
    - name: download latest check_mk server package
      ansible.builtin.get_url:
        url: https://download.checkmk.com/checkmk/2.1.0b7/check-mk-raw-2.1.0b7-sles15sp3-38.x86_64.rpm
        dest: /tmp/check-mk-raw-latest.x86_64.rpm
      register: new_server
    - name: install latest check_mk server package
      ansible.builtin.package:
        name: /tmp/check-mk-raw-latest.x86_64.rpm
        state: present
        disable_gpg_check: yes
      when: new_server.changed
    - name: stop site
      ansible.builtin.command: omd stop home
      when: new_server.changed
    - name: update site
      ansible.builtin.command: omd -f -V 2.1.0b7.cre update --conflict=install home
      when: new_server.changed
    - name: start site
      ansible.builtin.command: omd start home
      when: new_server.changed

I’m looking for a way to automatically grab the latest version (beta if available, stable otherwise) rather than having to edit that URL each time I want to run my playbook. I look at parsing the code behind Download archive of outdated Checkmk versions or Download Checkmk for free | Checkmk but this is incredibly complicated due to how the download link is dynamically generated. There must be a better way?

1 Like
2

Well, you pull the open source version, so the link is easy to construct in general, but I see your struggle.
@baris.leenders do you see an easy implementable approach to enable users to download the latest (beta) version through a generic link?

1 Like
3

This is something I have wanted to implement before. Only the maintainability stopped me from doing it. Since we are working on the download page and related topics, maybe it is worth revisiting the topic.
I will let you know here, in a few weeks, what the outcome will be.

2 Likes
4

You sir, are a very brave man.

(sorry for assuming your gender)

5 Likes
5

I would rather stay with official (whether stable or beta) releases than pulling directly from the git (which is something I did consider and yes that would be easier to automate as things currently stand). From what I can see on the git there is a separate branch for major releases but not for every patch level.

There is a json on the download page (downloads='{...}') but Ansible doesn’t make it super easy to parse through the whole webpage, find the json and then work with the json and there is no guarantee that this won’t change in the future. If that json was loaded from a dedicated json file it would be much easier to parse the json file directly with Ansible (or any other automation tool) and get the info we need from here to build the download link correctly.

No problem. We started automating installations with Ansible at my workplace, so I decided to automate with Ansible updates to my own Checkmk installation in my homelab to learn a little bit more about Ansible (and Checkmk). While at work we very rarely update Checkmk (once a year at best), I like to stay up to date with the latest features in my homelab, so having a playbook run once every day and automatically update when an update is available is always something I wanted to do. Currently I just change the rpm URL path once I get notified about a new update but removing any manual intervention would be even cooler.

6

@dnLL

there is a JSON blob in the source of the download URL, there you can “grep” all the data from you need. I use this in my CMK update check, so have a look if you like :slight_smile:
Checkmk Updates

2 Likes
7

I do a similar thing with the agent installation. My solution includes an internally available HTTP folder containing the agent files, and a symlink called “agent-latest.rpm” which I point at the new files when I generate them. You could do a similar thing - download it once, put it in a shared location called “checkmk-server-latest.rpm” and reference that in your playbook.

Not totally automatic like you asked, but probably much safer. Checkmk generally dislikes having different versions running within a distributed setup, if a new version were to be released in between you doing updates/installations of different hosts you might find yourself in a pickle.

You could also solve this in the opposite way. Given that you need the version number several times in the playbook, look it up yourself and supply it as a variable.

8

We are rebuilding the download page and plan to launch it this month. The JSON blob will be split into several ones as far as I know as the one just got too large now. The structure stays mostly the same I think, but instead of one super big JSON, we will have several smaller ones.
Thus, be aware as it will likely impact you.

9

@martin.hirschvogel thanks for the heads up. Sounds after some work for me :wink:

10

Very interesting, parsing with Python seems to make it easier than strictly with Ansible in this specific case.

Thanks for the heads up, I will wait a little bit in this case. If the JSONs could be in separate files rather than directly within the download page, it could almost be called like a REST API rather than having to parse the webpage for the JSONs.

Alternatively if the raw/free versions were in a different directory that is not password protected and allows listing, that would work as well.

11

For our Linux infrastucture running CentOS 7 and Alma Linux 8, we use Foreman/Katello to manage our packages for all Linux servers.
I have created a local repository that I maintain with all CheckMK packages, I will then manually download the RPM from Tribe29 and upload it to our CheckMK repo in Katello, then all servers have access to it via normal installation method. We run Managed Enterprice Edition and use the bakery to update our agents. But the CheckMK servers are updated using scripts as well.

Works really well.

12

Hey guys,
I saw there are several posts here asking for a way to check if there is a new version of Checkmk available. @thl-cmk already created a plugin to do this, but that relies on the page checkmk.com/download. We can’t promise that the structure of that page never changes, in fact it did change this week when we released Checkmk 2.1.0.

Because of this I have created a JSON file that contains the latest version of each of the major versions that we currently support. This file is being created every 5 minutes and is publicly available on https://download.checkmk.com/stable_downloads.json

This is just the first version, I have consulted with @thl-cmk on content of the file, but if you would like to see changes in the information available in the file, send me a message.

The current structure is as follows:

{
    "version": 1,
    "checkmk": {
        "2.1.0": {
            "version": "2.1.0",
            "release_date": 1653379969,
            "class": "stable",
            "editions": {
                "cme": {
                    ...
                },
                "cre": {
                    ...
                },
                "cfe": {
                    ...
                },
                "cee": {
                    ...
                }
            }
        },
        "2.0.0": {
            "version": "2.0.0p25",
            "release_date": 1653503541,
            "class": "oldstable",
            "editions": {
                ...
            }
        },
        "1.6.0": {
            "version": "1.6.0p28",
            "release_date": 1646148925,
            "class": "oldstable",
            "editions": {
                ...
            }
        }
    },
    "appliance": {
        "1.5.1": {
            ...
        },
        "1.4.19": {
            ...
        }
    }
}

For each support major version you can see what the latest version (patch release) is and if it is the latest stable or an “oldstable”.
For the appliance I have added the latest version of the last 2 major version (1.5.x & 1.4.x).
If you are looking for the CMA “cma-2” is meant for appliance 1.4.x and cma-3 is meant for 1.5.x

Edit: we noticed there are some hashes missing for the docker images of 2.0.0p25, we’re on it.

5 Likes
13

This is great. My only suggestion would be to add beta releases as well in the future (starting with 2.2.0b1).

14

@dnLL this will appear in there as soon as they are released.
All stable, oldstable, beta and innovation releases should be added automatically.

1 Like
15

Hint: The customer wants a real yum repo with server and agent rpms.

But since tribe29 does not provide any yum repos we do do it currently on our own.
This is for our very special use case of downloading CEE for el7 but perhaps still helpful for some users. We run this as cron job twice a day.

#!/bin/bash

for version in $(curl --silent https://download.checkmk.com/stable_downloads.json | jq -r '.checkmk[].version' | grep -v ^1\.6 )
do

  basedir="/var/www/html/checkmk"
  mkdir -p $basedir
  cd $basedir

  # wget with auth options
  wget \
    --no-check-certificate \                                                                                                                                                                  
    --user=<yourusername> \
    --password=<yourpass> \
    --continue \
    https://download.checkmk.com/checkmk/${version}/check-mk-enterprise-${version}-el7-38.x86_64.rpm

  # extract the checkmk agent rpm from the checkmk server rpm
  rpm2cpio check-mk-enterprise-${version}-el7-38.x86_64.rpm \
  | cpio --extract --verbose  --to-stdout ./opt/omd/versions/${version}.cee/share/check_mk/agents/check-mk-agent-${version}-1.noarch.rpm > check-mk-agent-${version}-1.noarch.rpm

  createrepo .

done
1 Like
16

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.