Howto: Ansible Playbook for updating (linux) Agent and plugins(stock only) CRE only

This is a How-To regarding Updating a/the Check-MK agents and plugins default available after an update of your monitoring Server via Ansible.

It will update both a/the base agent (rpm and deb) and available/installed plugins on a/the host.
The playbook will NOT install/add plugins to hosts, it will only update existing plugins.

This Playbook is still a “work in progress”, however i am at a point where i am sharing it as a proof-of-concept.

General warning
With all my work, this is a proof-of-concept as to how to monitor or deploy specific products, and unless you are fully aware of what you are getting into while i am trying to get full deployment control on something via Ansible it should under no condition NOT be used in a production environment when you are not familiar with what is being done here.

Intro
Without having an Enterprise Licence, this Ansible script gives a solution as to deploying both a new agent version, as well as updating (default) plugins on a/the monitored system.
All information is getting pulled from the CMK host, assuming it is also part of your Ansible-setup.

The works
This is a single-file Ansible Playbook, which will handle all, as long as you run it globally ( no --limit parameter)

The development
At current, it might be a bit bloated as to tasks, but it does its job as a proof of concept.
I will refactor in the future.

Test Environment
I have tested this playbook on my end with:

  • 68 hosts ( a combined set of rpm- and deb-based hosts)
  • CMK RAW edition 2.1.0p37

The code

---
- hosts: all
  vars:
    monitor_host: 'monitor.mydomain.tld'
    monitor_site: 'MyMonitor_Sitename'
    download_path: '/usr/local/install/packages'
    plugin_path: '/usr/lib/check_mk_agent/plugins'

  tasks:
    - name: Query monitoring host for latest CheckMK version
      ansible.builtin.shell: omd versions
      register: cmk_versions
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Set latest version of CheckMK package as fact
      ansible.builtin.set_fact:
        cmk_latest: "{{ cmk_versions.stdout_lines | community.general.version_sort | last | regex_search('[0-9]+.[0-9]+.[0-9]+[a-z][0-9]+') }}"
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Find latest rpm agents on monitoring host
      ansible.builtin.find:
        path: "/opt/omd/versions/{{ cmk_latest }}.cre/share/check_mk/agents"
        patterns: '*.rpm'
      register: rpm_agent_result
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Find latest deb agents on monitoring host
      ansible.builtin.find:
        path: "/opt/omd/versions/{{ cmk_latest }}.cre/share/check_mk/agents"
        patterns: '*.deb'
      register: deb_agent_result
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Set rpm agent filename as fact
      ansible.builtin.set_fact:
        rpm_latest: "{{ rpm_agent_result.files[0].path | basename }}"
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Propogate rpm fact to applicable hosts
      ansible.builtin.set_fact:
        rpm_basename: "{{ hostvars[monitor_host]['rpm_latest'] }}"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Set fact for rpm agent
      ansible.builtin.set_fact:
        rpm_download: "http://{{ hostvars[monitor_host]['ansible_fqdn'] }}/{{ monitor_site }}/check_mk/agents/{{ hostvars[monitor_host]['rpm_latest'] }}"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Set deb agent filename as fact
      ansible.builtin.set_fact:
        deb_latest: "{{ deb_agent_result.files[0].path | basename }}"
      delegate_to: "{{ monitor_host }}"
      run_once: true
        #      when: inventory_hostname == monitor_host

    - name: Propogate deb fact to applicable hosts
      ansible.builtin.set_fact:
        deb_basename: "{{ hostvars[monitor_host]['deb_latest'] }}"
      when:
        - ansible_pkg_mgr == "apt"

    - name: Set fact for deb agent
      ansible.builtin.set_fact:
        deb_download: "http://{{ hostvars[monitor_host]['ansible_fqdn'] }}/{{ monitor_site }}/check_mk/agents/{{ hostvars[monitor_host]['deb_latest'] }}"
      when:
        - ansible_pkg_mgr == "apt"

    - name: Cleanup packages directory before installing (if it exists)
      ansible.builtin.file:
        path: "{{ download_path }}"
        state: absent

    - name: Create default directories for installing
      ansible.builtin.file:
        path: "{{ download_path }}"
        state: directory
        owner: root
        group: root
        mode: 0775

    - name: Download found rpm agent with version from monitoring host
      ansible.builtin.get_url:
        url: "{{ rpm_download }}"
        dest: "{{ download_path }}/{{ rpm_basename }}"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Handle Agent install of found rpm packages
      block:
        - name: Install rpm agent via YUM
          ansible.builtin.yum:
            name: "{{ download_path }}/{{ rpm_basename }}"
            state: present
            disable_gpg_check: yes
          when:
            - ansible_pkg_mgr == "yum"

        - name: Install rpm agent via ZYPPER
          community.general.zypper:
            name: "{{ download_path }}/{{ rpm_basename }}"
            state: present
            disable_gpg_check: yes
          environment:
            ZYPP_LOCK_TIMEOUT: 30
          when:
            - ansible_pkg_mgr == "zypper"

        - name: Install rpm agent via DNF
          ansible.builtin.dnf:
            name: "{{ download_path }}/{{ rpm_basename }}"
            state: present
            disable_gpg_check: yes
          when:
            - ansible_pkg_mgr == "dnf"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Download found deb agent with version from monitoring host
      ansible.builtin.get_url:
        url: "{{ deb_download }}"
        dest: "{{ download_path }}/{{ deb_basename }}"
      when:
        - ansible_pkg_mgr == "apt"

    - name: Install deb Agent package
      ansible.builtin.apt:
        deb: "{{ download_path }}/{{ deb_basename }}"
      become: yes
      when:
        - ansible_pkg_mgr == "apt"

    - name: find installed plugins in main plugins directory
      ansible.builtin.find:
        paths: "{{ plugin_path }}"
        file_type: file
        recurse: yes 
      register: plugins_main

    - name: Set paths of found plugins as fact
      ansible.builtin.set_fact:
        plugins_main_found: "{{ plugins_main | json_query('files[*].path') }}"

    - name: Update found plugins with versions from monitoring host
      ansible.builtin.get_url:
        url: "http://{{ hostvars[monitor_host]['ansible_fqdn'] }}/{{ monitor_site }}/check_mk/agents/plugins/{{ item | basename }}"
        dest: "{{ item }}"
      with_items:
           - "{{ plugins_main_found }}"
  • Glowsome
5 Likes

Looks interesting and simple enough. :+1:

Let me just point to the official Checkmk Ansible Collection. While it might be too complex for the very specific use-case, it supports CRE as well as the commercial editions and might be good inspiration. We also do appreciate contributions there, so if something is missing, we look forward to contributions!

1 Like

Hello @robin.gierse ,
A late reply, but your suggestion is appreciated.

It was my goal to achieve just the update of both agents as well as plugins without having to introduce a new collection.

The above playbook will work for the use-case i have defined, but after having attempted an upgrade from 2.1.x → 2.2.x which gave some rather weird issues on my end ( posted elsewhere in the forum) i had to revert back.

This meant i was now having higher version agents then the actual CMK server version.
So i had to adapt my playbook into a better form to remove an existing agent ( whichever version it is/was) and then install the correct agent corresponding to the CMK-server version being operated.

This has resulted in the changes to the playbook as posted below:

---
- hosts: all
  vars:
    monitor_host: 'monitor.MyMonitorHost.tld'
    monitor_site: 'MySiteName'
    download_path: '/usr/local/install/packages'
    plugin_path: '/usr/lib/check_mk_agent/plugins'

  tasks:
    - name: Query monitoring host for latest CheckMK version
      ansible.builtin.shell: omd versions
      register: cmk_versions
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Set latest version of CheckMK package as fact
      ansible.builtin.set_fact:
        cmk_latest: "{{ cmk_versions.stdout_lines | community.general.version_sort | last | regex_search('[0-9]+.[0-9]+.[0-9]+[a-z][0-9]+') }}"
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Find latest rpm agents on monitoring host
      ansible.builtin.find:
        path: "/opt/omd/versions/{{ cmk_latest }}.cre/share/check_mk/agents"
        patterns: '*.rpm'
      register: rpm_agent_result
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Find latest deb agents on monitoring host
      ansible.builtin.find:
        path: "/opt/omd/versions/{{ cmk_latest }}.cre/share/check_mk/agents"
        patterns: '*.deb'
      register: deb_agent_result
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Set rpm agent filename as fact
      ansible.builtin.set_fact:
        rpm_latest: "{{ rpm_agent_result.files[0].path | basename }}"
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Propogate rpm fact to applicable hosts
      ansible.builtin.set_fact:
        rpm_basename: "{{ hostvars[monitor_host]['rpm_latest'] }}"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Set fact for rpm agent
      ansible.builtin.set_fact:
        rpm_download: "http://{{ hostvars[monitor_host]['ansible_fqdn'] }}/{{ monitor_site }}/check_mk/agents/{{ hostvars[monitor_host]['rpm_latest'] }}"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Set deb agent filename as fact
      ansible.builtin.set_fact:
        deb_latest: "{{ deb_agent_result.files[0].path | basename }}"
      delegate_to: "{{ monitor_host }}"
      run_once: true

    - name: Propogate deb fact to applicable hosts
      ansible.builtin.set_fact:
        deb_basename: "{{ hostvars[monitor_host]['deb_latest'] }}"
      when:
        - ansible_pkg_mgr == "apt"

    - name: Set fact for deb agent
      ansible.builtin.set_fact:
        deb_download: "http://{{ hostvars[monitor_host]['ansible_fqdn'] }}/{{ monitor_site }}/check_mk/agents/{{ hostvars[monitor_host]['deb_latest'] }}"
      when:
        - ansible_pkg_mgr == "apt"

    - name: Cleanup packages directory before installing (if it exists)
      ansible.builtin.file:
        path: "{{ download_path }}"
        state: absent

    - name: Create default directories for installing
      ansible.builtin.file:
        path: "{{ download_path }}"
        state: directory
        owner: root
        group: root
        mode: 0775

    - name: Download found rpm agent with version from monitoring host
      ansible.builtin.get_url:
        url: "{{ rpm_download }}"
        dest: "{{ download_path }}/{{ rpm_basename }}"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Handle Agent install of found rpm packages
      block:
        - name: Remove rpm agent via YUM
          ansible.builtin.yum:
            name: check-mk-agent
            state: absent
              #disable_gpg_check: yes
          when:
            - ansible_pkg_mgr == "yum"

        - name: Install rpm agent via YUM
          ansible.builtin.yum:
            name: "{{ download_path }}/{{ rpm_basename }}"
            state: present
            disable_gpg_check: yes
          when:
            - ansible_pkg_mgr == "yum"

        - name: Remove rpm agent via ZYPPER
          community.general.zypper:
            name: check-mk-agent
            state: absent
              #disable_gpg_check: yes
          environment:
            ZYPP_LOCK_TIMEOUT: 30
          when:
            - ansible_pkg_mgr == "zypper"

        - name: Install rpm agent via ZYPPER
          community.general.zypper:
            name: "{{ download_path }}/{{ rpm_basename }}"
            state: present
            disable_gpg_check: yes
          environment:
            ZYPP_LOCK_TIMEOUT: 30
          when:
            - ansible_pkg_mgr == "zypper"

        - name: Remove rpm agent via DNF
          ansible.builtin.dnf:
            name: check-mk-agent
            state: absent
              #disable_gpg_check: yes
          when:
            - ansible_pkg_mgr == "dnf"

        - name: Install rpm agent via DNF
          ansible.builtin.dnf:
            name: "{{ download_path }}/{{ rpm_basename }}"
            state: present
            disable_gpg_check: yes
          when:
            - ansible_pkg_mgr == "dnf"
      when:
        - ansible_pkg_mgr == "yum" or ansible_pkg_mgr == "dnf" or ansible_pkg_mgr == "zypper"

    - name: Download found deb agent with version from monitoring host
      ansible.builtin.get_url:
        url: "{{ deb_download }}"
        dest: "{{ download_path }}/{{ deb_basename }}"
      when:
        - ansible_pkg_mgr == "apt"

    - name: Remove deb Agent package
      ansible.builtin.apt:
        name: check-mk-agent
        state: absent
      become: yes
      when:
        - ansible_pkg_mgr == "apt"

    - name: Install deb Agent package
      ansible.builtin.apt:
        deb: "{{ download_path }}/{{ deb_basename }}"
      become: yes
      when:
        - ansible_pkg_mgr == "apt"

    - name: find installed plugins in main plugins directory
      ansible.builtin.find:
        paths: "{{ plugin_path }}"
        file_type: file
        recurse: yes 
      register: plugins_main

    - name: Set paths of found plugins as fact
      ansible.builtin.set_fact:
        plugins_main_found: "{{ plugins_main | json_query('files[*].path') }}"

    - name: Update found plugins with versions from monitoring host
      ansible.builtin.get_url:
        url: "http://{{ hostvars[monitor_host]['ansible_fqdn'] }}/{{ monitor_site }}/check_mk/agents/plugins/{{ item | basename }}"
        dest: "{{ item }}"
      with_items:
           - "{{ plugins_main_found }}"

    - name: Disable Unwanted CheckMK Services
      ansible.builtin.systemd:
        state: stopped
        enabled: no
        name: cmk-agent-ctl-daemon

Best regards,

  • Glowsome
2 Likes

Just a little update/status thingy…

I just ran an update on my CMK host (from 2.1.0p38 → 2.1.0p39) on the CMK-server.
After having run the update, the one thing i did was symlink a package i have installed for getting information on packages via yum.
As this package is not part of a/the default CMK plugins its installed in a separate path.
This means that if i were to run the above ansible-playbook it will complain about this plugin.

To solve this i created a symlink:

ln -s /opt/omd/sites/mysitename/local/share/check_mk/agents/plugins/yum /opt/omd/versions/2.1.0p39.cre/share/check_mk/agents/plugins/yum

Then i ran the playbook, and it ran flawless on all 69 hosts i have in CMK.
Do mind that when running it will produce quite a few notifications while its running.

Total runtime of the playbook on the hosts (both local and remote hosts) :

Saturday 24 February 2024  00:32:24 +0100 (0:00:27.562)       0:06:17.123 *****
===============================================================================
Update found plugins with versions from monitoring host ---------------------------------- 55.55s
Install rpm agent via ZYPPER ------------------------------------------------------------- 42.91s
Gathering Facts -------------------------------------------------------------------------- 41.49s
Disable Unwanted CheckMK Services -------------------------------------------------------- 27.56s
Install rpm agent via DNF ---------------------------------------------------------------- 24.83s
Remove rpm agent via ZYPPER -------------------------------------------------------------- 23.80s
Install deb Agent package ---------------------------------------------------------------- 20.82s
Remove rpm agent via DNF ----------------------------------------------------------------- 19.96s
find installed plugins in main plugins directory ----------------------------------------- 17.66s
Remove deb Agent package ----------------------------------------------------------------- 17.46s
Download found rpm agent with version from monitoring host ------------------------------- 17.24s
Create default directories for installing ------------------------------------------------ 14.78s
Cleanup packages directory before installing (if it exists) ------------------------------ 14.48s
Install rpm agent via YUM ----------------------------------------------------------------- 7.11s
Download found deb agent with version from monitoring host -------------------------------- 6.98s
Remove rpm agent via YUM ------------------------------------------------------------------ 6.44s
Set paths of found plugins as fact -------------------------------------------------------- 4.34s
Set fact for rpm agent -------------------------------------------------------------------- 2.93s
Propogate rpm fact to applicable hosts ---------------------------------------------------- 2.50s
Set fact for deb agent -------------------------------------------------------------------- 1.91s
  • Glowsome