RAW: CMK agent + host updates via Ansible

This How-To will let you update all your hosts with a new agent,and updates their OS-packages in one go based on an ansible setup.

Disclaimer: This is a work-in-progress, both in results as in performance.
=> i am running this playbook over 62 hosts to test ( with different OS’es and package-managers to ensure compatibillity)
Untill i am 100% sure this should not be run on production-environments - or if you do at your own risk.

Also suggestions will be very welcome to fine-tune this playbook.

Prerequisites:

  • Your hosts needs to be under control of your Ansible Setup (this should be obvious)
  • An SSH- trust on your host with the Asible controller should exist ( this should also be obvious)
  • A filepath on your Ansible hosts ( where your agents/plugins are placed) of /etc/ansible/common/files
    => This can ofcourse be adapted to your own setup.
    ==> Expected structure of CMK-files will be :
  • your agent files ( both the *.deb and the *.rpm ones)
  • your plugins ( like mk_apt, mk_zypper etc.)
    ==> Be sure to only have a/the latest/recent agent in the source directory on your Ansible setup copies down from your CMK host.
  • Have the Yum extension yum extension in place if you are using RHEL-based hosts( not available by default in CMK)

Added : for RHEL-based distrubutions you need to have yum-utils package installed for this playbook to work

The Ansible playbook :

---
- name: talk to all hosts to setup general stuff
  hosts: all
  tasks:
    # General tasks for all hosts
    # Ensuring there are no files which potentially can interfere from previous runs
    - name: cleanup directories before installing (if they exist)
      ansible.builtin.file:
        path: "{{ item }}"
        state: absent
      loop:
        - /usr/local/install/compile
        - /usr/local/install/rpm
        - /usr/local/install/packages
        - /usr/local/install/iso

    - name: create default directories for installing
      ansible.builtin.file:
        path: "{{ item }}"
        state: directory
        owner: root
        group: root
        mode: 0775
      loop:
        - /usr/local/install/source
        - /usr/local/install/compile
        - /usr/local/install/packages
        - /usr/local/install/iso

    - name: Get the package facts from the hosts
      ansible.builtin.package_facts:
        manager: auto

    - name: Group hosts based on Package Manager
      group_by:
        key: system_{{ ansible_pkg_mgr }}

- hosts: system_yum, system_dnf, system_zypper
  gather_facts: False
  tasks:
    # Tasks for multiple groups combined go in this play
    - name: Copy packages to rpm-based systems (yum- zypper- and dnf-based systems)
      ansible.builtin.copy:
        src: "{{ item }}"
        dest: "/usr/local/install/packages/"
        owner: "root"
        mode: 0750
      with_fileglob:
        - "/etc/ansible/roles/common/files/*.rpm"

    - name: Find placed .rpm files on target system
      ansible.builtin.find:
        paths: /usr/local/install/packages
        file_type: file
        patterns: '*.rpm'
      register: rpm_result

    - name: Setting rpm_list
      ansible.builtin.set_fact:
        rpm_list: "{{ rpm_result.files | map(attribute='path') | list}}"
       
- hosts: system_yum
  gather_facts: False
  tasks:
    # Tasks for yum-based hosts only go in this play.
    - name: Update yum hosts 
      ansible.builtin.yum:
        name: '*'
        state: latest

    - name: Install rpms placed in /usr/local/install/packages (via yum)
      ansible.builtin.yum:
        name: "{{rpm_list}}"
        state: present
        disable_gpg_check: yes

    - name: Create CheckMK agent subdir 3600 if not present
      ansible.builtin.file:
        path: /usr/lib/check_mk_agent/plugins/3600
        state: directory
        owner: root
        group: root
        mode: 0775

    - name: Install CheckMK yum extension for monitoring
      ansible.builtin.copy:
        src: "/etc/ansible/roles/common/files/yum"
        dest: "/usr/lib/check_mk_agent/plugins/3600"
        owner: "root"
        mode: 0750

    - name: Check yum-based hosts to see if a reboot is required
      ansible.builtin.command: needs-restarting -r
      register: needs_restarting
      changed_when: "needs_restarting.rc != 0"
      ignore_errors: yes
 
- hosts: system_zypper
  gather_facts: False
  tasks:
    # Tasks for zypper-based hosts only go in this play.
    - name: Update zypper hosts
      community.general.zypper:
        name: '*'
        state: latest
      environment:
        ZYPP_LOCK_TIMEOUT: 30

    - name: Install found packages via ZYPPER when manager is found
      community.general.zypper:
        name: "{{rpm_list}}"
        state: present
        disable_gpg_check: yes
      environment:
        ZYPP_LOCK_TIMEOUT: 30

    - name: Create CheckMK agent subdir 3600 if not present
      ansible.builtin.file:
        path: /usr/lib/check_mk_agent/plugins/3600
        state: directory
        owner: root
        group: root
        mode: 0775

    - name: Install CheckMK zypper extension for monitoring
      ansible.builtin.copy:
        src: "/etc/ansible/roles/common/files/mk_zypper"
        dest: "/usr/lib/check_mk_agent/plugins/3600"
        owner: "root"
        mode: 0750

    - name: Check zypper-based System to see if a reboot is required
      ansible.builtin.command: zypper -q ps
      register: needs_restarting
      changed_when: "needs_restarting.rc != 0"
      ignore_errors: yes

- hosts: system_dnf
  gather_facts: False
  tasks:
    # Tasks for dnf-based hosts only go in this play.
    - name: Update dnf-based hosts
      ansible.builtin.dnf:
        name: '*'
        state: latest
        update_cache: true

    - name: Install rpms placed in /usr/local/install/packages (via dnf)
      ansible.builtin.dnf:
        name: "{{rpm_list}}"
        state: present
        disable_gpg_check: yes
        
    - name: Create CheckMK agent subdir 3600 if not present
      ansible.builtin.file:
        path: /usr/lib/check_mk_agent/plugins/3600
        state: directory
        owner: root
        group: root
        mode: 0775

    - name: Check dnf-based hosts to see if a reboot is required
      ansible.builtin.command: needs-restarting -r
      register: needs_restarting
      changed_when: "needs_restarting.rc != 0"
      ignore_errors: yes
     
- hosts: system_apt
  gather_facts: False
  tasks:
    - name: Update apt package cache if the last refresh is more than 3600 seconds ago
      ansible.builtin.apt:
        update_cache: yes
        cache_valid_time: 3600

    - name: Update Packages via apt package-manager
      ansible.builtin.apt:
        upgrade: dist

    - name: Copy packages to apt-based systems
      ansible.builtin.copy:
        src: "{{ item }}"
        dest: "/usr/local/install/packages/"
        owner: "root"
        mode: 0750
      with_fileglob:
        - "/etc/ansible/roles/common/files/*.deb"

    - name: Find placed package files on apt-based system
      ansible.builtin.find:
        paths: /usr/local/install/packages
        file_type: file
        patterns: '*.deb'
      register: apt_result

    - name: Setting apt_list
      ansible.builtin.set_fact:
        apt_list: "{{ apt_result.files | map(attribute='path') | list}}"

    - name: Install found packages placed in /usr/local/install/packages (apt)
      ansible.builtin.apt:
        deb: "{{ item }}"
      loop: "{{ apt_list }}"
      become: yes

    - name: Create CheckMK agent subdir 3600 if not present
      ansible.builtin.file:
        path: "/usr/lib/check_mk_agent/plugins/3600"
        state: directory
        owner: root
        group: root
        mode: 0775

    - name: Install CheckMK apt extension for monitoring
      ansible.builtin.copy:
        src: "/etc/ansible/roles/common/files/mk_apt"
        dest: "/usr/lib/check_mk_agent/plugins/3600"
        owner: "root"
        mode: 0750

    - name: Check if reboot is required on apt-based hosts
      stat:
        path: /var/run/reboot-required
      register: reboot_required

- hosts: all
  gather_facts: False
  tasks:
    # Tasks for all hosts go in this play.
    - name: Install CheckMK extension for inventory
      ansible.builtin.copy:
        src: "/etc/ansible/roles/common/files/mk_inventory.linux"
        dest: "/usr/lib/check_mk_agent/plugins"
        owner: "root"
        mode: 0750

    - name: Stop unwanted CheckMK services
    # drop the agent communications other then over SSH
      ansible.builtin.systemd:
        state: stopped
        enabled: no
        name: cmk-agent-ctl-daemon

- hosts: system_yum, system_zypper,system_dnf
  gather_facts: false
  tasks:
   - name: reboot rpm-based systems
     ansible.builtin.reboot:
       reboot_timeout: 3600
     when:
       - needs_restarting.rc > 0

- hosts: system_apt
  gather_facts: false
  tasks:
  # reboot hosts automatically aslong they are not part of the Cluster - hence the condition of node in inventory_hostname
    - name: Reboot if required
      ansible.builtin.reboot:
        reboot_timeout: 3600
      when: 
        - reboot_required.stat.exists == true
        - "'node' not in inventory_hostname"

  • Updated 24-03-2023 - corrected some missing tasks, and text labels on tasks

May I suggest GitHub - tribe29/ansible-collection-tribe29.checkmk: The official Checkmk Ansible collection - brought to you by tribe29 - the Checkmk company.?
I haven’t gone through all steps, but maybe you find functionality in there that serves your use case.

3 Likes

And let me point towards this for OS upgrades on Linux: GitHub - thorian93/ansible-role-upgrade: This role performs upgrades on RHEL/CentOS, Debian/Ubuntu and Fedora servers.

1 Like

Missing SuSE Linux in the mix unfortunately.

Where exactly do you miss it?

@robin.gierse the reply was directed towards @thorian93 … but apparently you and him are the same ¿?
image

And i only took a glance at the playbook(s) must have overlooked the section of Zypper.
So i must correct myself on that.

However, it was my intent to make an ‘all-in-1’ playbook for updating hosts via dynamic grouping plus assurance of the latest CMK-agent / plugins in one go.

IMHO: The complexity of introducing yet another role is way beyond the scope i had in mind/wanted to create.
This ofc. is a personal opinion from my end.

  • Glowsome

Correct, private and work account. Hence, the two distinct posts.

All good, I just wanted to throw options into the ring. If they do not suit your use case, that is perfectly fine. :slight_smile:

If you can reach the CHECK_MK Server than i use statements like this…

- hosts: all
  gather_facts: false
  serial: 15

  tasks:
    - name: get check_mk_agent from OMD
      apt:
        force: true
        deb: '{{ omdsrv }}/agents/check-mk-agent_{{ cmkversion }}_all.deb'
      when: "'debian' in group_names or 'extern' in group_names"

    - name: get mk_logins from OMD
      get_url:
        url: '{{ omdsrv }}/agents/plugins/mk_logins'
        dest: '/usr/lib/check_mk_agent/plugins/'
        force: yes
        owner: root
        group: root
        mode: u=rwx,g=rx,o=rx

So i can use always the “latest” version of the checks and do not copy files from check_mk to ansible server to client_server if i understood your way right.

The case is i cannot reach my monitoring server directly, so yes your suggestion looks great, however it will not fly for me unfortunately :frowning: