Make rule_id optional in create rule rest API call

Hi,

We are trying to automate all our rules using the checkmk ansible collection and Rest API.

After reading the following “warnings” we opted for tracking a rule_id in our codebase as having a unique key was useful anyway and it ensured we would alway edit/delete the correct rule

suboptions:
    rule_id:
        description:
            - If provided, update/delete an existing rule.
            - If omitted, we try to find an equal rule based on C(properties),
              C(conditions), C(folder) and C(value_raw).
            - Please mind the additional notes below.
        type: str

notes:
    - If rule_id is omitted, due to the internal processing of the C(value_raw), finding the
      matching rule is not reliable, when C(rule_id) is omitted. This sometimes leads to the
      module not being idempotent or to rules being created over and over again.
    - If rule_id is provided, for the same reason, it might happen, that tasks changing a rule
      again and again, even if it already meets the expectations.

But now we have got to the stage where we have automated all our hosts/rules etc and we would like to recreate the site from scratch. The problem is that the rule api does not allow rule_id to be an input parameter when creating the rule. It insteads always creates it on the server when the request arrives.

Is it possible to make this optional without breaking anything? As it sits I could create all the rules by omitting the rule_id but then I would have to extract all the new ids and backport into our codebase. It would be nice to be able to pregenerate a valid id before creating rule with API

1 Like

I would say - i don’t think so. What should happen if you provide a rule id that already is existing inside the system? This can be the case as a new site already has some internally created rules with id.

Normal workflow looks like - get all rules from a ruleset (here you get the existing IDs) - make something with this result and update/delete the rule you want specified by ID.

I do understand there are system generated rules which may have assigned id’s not under “my control”. But as long as I use uuidgen to create new id’s on the client side of the rest call will the chance of duplicates not be the same? Does it really matter if the uuidgen call is made client or server side? I think there is a roughly 1 in a billion chance of creating a duplicate with uuidgen. Of course if I or someone on my team does something silly and copy/paste rule ids then that would be our problem. A snall disclaimer in the doc that just states use at own risk - ensure rule_id created with a suitable technique? Maybe it also could be fairly simple to just throw an error in API if you supply an already existing uuid?

The reason we went with tracking the id was that we were having idempotency issues etc like the ansible collection documentation warned us about. Creating a rule without an id and then using lookups etc to find it again for certain would require a lot more effort than just inserting a uniqe id at start and keep using that.

But reading between the lines then it doesn’t seem to be anything which would break apart from if we supply duplicates? Maybe we can make a fork and patch it ourselves.

Regards,
Henning

In a normal environment you cannot generate the id’s for the rules. This is done by CheckMK itself and i don’t know if it checks in the background in some way for double id’s.

But this is the only real way supported inside the CheckMK API.

Then you need to patch the CheckMK API every time you do an update on the system. I don’t think this is a good way.

I would argue from an ansible point of view it would be entirely normal to either call external functions like uuidgen on the fly while running the ansible code or even more common tracking ids and or other data which would guarantee a reproducable and identical result time after time.

  1. create vm
  2. yum install checkmk
  3. ansible-playbook checkmk.yml
  4. done
  5. repeat 3 to infinity with guaranteed no duplicates generated and if no changes made the ansible run would stay green (no changes)

I understand that today, but does it have to be? What I’m proposing does not break anything existing it just expands functionality to allow the client to provide the id. If none is provided it’s business as usual. And I guess since this is the only way today then that is also why the checkmk ansible collection has to make these warnings about idempotancy and duplicates as there is no real way of guaranteeing this using current API.

I also agree monkeypatching this every time we do updates is not very desireable. Thats why I was hoping to do it proper. Reading between the lines it seems this is not something that would be considered even as a pull request?

To give you some further context we have used checkmk for years from back in the day when there was no usable API. We used ansible to template the wato files directly. Full freedom but super easy to break when upgrading. But we as a client had full control and could do as we liked. We now have gone full licensed enterprise and wanted to get away from the messy templating and over to the cleaner Rest API. We are nearly there apart from this issue.

Thanks for responding and if its still a non starter at least it saves us the effort of a pull req. :slight_smile:

Hello Henning,

I manage a reference ruleset, export it to YAML for Ansible and update all assigned customers with the reference ruleset. But I wrote own playbooks which use the Checkmk ansible-collection.
The list of “broken” rulesets has become shorter the last times (e.g. DNS active checks still produced duplicate rules), but most of the rulesets work and update like desired.

- - rule:
      conditions:
        host_labels: []
        host_tags: []
        service_labels: []
      location:
        folder: /
      properties:
        description: State of NTP time synchronisation
      value_raw: '{''ntp_levels'': (10, 300.0, 500.0), ''alert_delay'': (43200, 86400)}'
    ruleset: checkgroup_parameters:ntp_time
- - rule:
      conditions:
        host_labels: []
        host_tags:
        - key: networking
          operator: is
          value: wan
        service_labels: []
      location:
        folder: /
      properties:
        description: Allow longer round trip times when pinging WAN hosts
      value_raw: '{''rta'': (1500.0, 3000.0), ''loss'': (80.0, 100.0), ''packets'':
        6, ''timeout'': 20}'
    ruleset: ping_levels

I’m thinking about our own rule-ids placed in the description for better reference, but the lookup logic of the ansible module will break this when other parts (options, conditions) have higher priority. An optional my-rule-id field could be the solution these problem. Then the API/Checkmk can work with the internal rule-id and customers can use their ids.

Thanks for this. I had been so focused on the “broken” internal id logic that I forgot to think of alternatives. Using the description field as a custom id should work fine as far as I can tell. The whole process of create/modify/delete rule will become a bit more chatty with the server under the hood but as long as it works better than now I’m happy.

What do you mean when you say “lookup logic of the ansible module will break this”? Now I’m thinking I will always use the rules lookup plugin to query for ruleset lookup(‘checkmk.general.rules’, ruleset=‘active_checks:http’, description_regex=‘my_unique_key’). If I get a hit then I take the id returned and feed it into the rule module call (update/delete). If there is no hit then I simply omit the id parameter of the rule module (create). All other details about the rule I already track in my own codebase

I didn’t know that the lookup supports such precise calls now:

lookup(‘checkmk.general.rules’, ruleset=‘active_checks:http’, description_regex=‘my_unique_key’).

Just forget my worries then.

Thank again. I have tested and verified that this works fine today. Inline lookups in vars section of tasks is pretty powerful and cool. The extra query to look for existence of rule with my_id becomes part of the same task to create/update the rule. For inspiration this is how I did it

First I have an ansible rule definition variable which stays neutral with regards to conditions and specific content of value_raw. Just vars that get populated later. I just focus on expanding the value_raw functionality as we need them. If value_raw needs curly or square brackets around this must be added later as otherwise ansible insists on parsing the var and the format gets messed up.

cmk_rule_definition_threads:
  properties:
    description: Configure monitoring of number of threads
  ruleset: checkgroup_parameters:threads
  value_raw: >-
    {% if cmk_rule.value.levels is defined %}
    'levels': (
      'levels', (
        {{ cmk_rule.value.levels.warn }},
        {{ cmk_rule.value.levels.crit }}
        )
      ),
    {% endif %}
    {% if cmk_rule.value.levels_percent is defined %}
    'levels_percent': (
      'levels', (
        {{ cmk_rule.value.levels_percent.warn }},
        {{ cmk_rule.value.levels_percent.crit }}
        )
      ),
    {% endif %}

Then I have a list of vars which I use to populate rule defs like above. here I can adjust levels and conditions etc so I can reuse the above def. One of these for each actual rule I want

my_rules_list:
  - id: df8a6007-6146-46d2-b438-6bedd38d8692    # I also use uuid as my_id
    type: cmk_rule_definition_threads
    value:
      levels:
        warn: 40000
        crit: 50000
    conditions:
      hosts:
        - myhost

In the loop of my_rules_list i build up the condition section first

- name: create rule condition
  ansible.builtin.set_fact:
    rule_conditions: >
      {
        {% if cmk_rule.conditions.services is defined %}
        'service_description': {
          'match_on': [
        {% for service in cmk_rule.conditions.services %}
          '{{ service }}',
        {% endfor %}
          ],
          'operator': 'one_of',
          },
        {% endif %}
        {% if cmk_rule.conditions.hosts is defined %}
        'host_name': {
          'match_on': [
        {% for host in cmk_rule.conditions.hosts %}
          '{{ host }}',
        {% endfor %}
          ],
          'operator': 'one_of',
          },
        {% endif %}
        {% if cmk_rule.conditions.host_label_groups is defined %}
        'host_label_groups': [
        {% for label_group in cmk_rule.conditions.host_label_groups %}
          {'label_group': [
        {% for label in label_group.label_group %}
               {'label': '{{ label.label }}',
                'operator': '{{ label.operator }}' },
        {% endfor %}
            ],
            'operator': '{{ label_group.operator }}' },
        {% endfor %}
        ],
        {% endif %}
       }

Then finally I put it all together. Look up the rule def, create the comment with my_id and finally lookup any existing rules in checkmk server with that id which determines if I pass the rule_id (update )or not (insert) to the rule module. So now I handle updates/inserts with guaranteed idempotency and no dups. Deletes and ordering etc is not implemented yet

- name: "configure rule {{ cmk_rule.type }} {{ cmk_rule.id | default('') }}"
  checkmk.general.rule:
    rule:
      rule_id: "{{ cmk_rule.id if existing_rule_id is defined else omit }}"
      conditions: "{{ rule_conditions }}"
      properties: "{{ rule.properties | add_key_value({'comment': comment }) }}"
      value_raw: "{{ ('{' ~ rule.value_raw ~ '}' if rule.wrap_value | default(true) else rule.value_raw) | string  }}"
      location:
        folder: "{{ host_group.dir }}"
    ruleset: "{{ rule.ruleset }}"
  vars:
    rule: "{{ lookup('ansible.builtin.vars', cmk_rule.type) }}"
    comment: "{{ 'Ansible managed - client_rule_id=' ~ cmk_rule.id if cmk_rule.id is defined else 'Ansible managed' }}"
    existing_rule_id: "{{ lookup('checkmk.general.rules',ruleset=rule.ruleset,comment_regex=cmk_rule.id) | map(attribute='id') | first }}"
  notify: activate checkmk

–Henning

Hello Henning,

Thanks for sharing your concept.

In our case it is even easier to implement. Our whole ruleset is tag based. We only maintain host, ip and associated tags in our CMDB. I just need the reference_id in my reference ruleset (pulled with API, written as yaml) and a “when” condition with the lookup of the reference_id in the description in the playbook. The reference site is only used for designing and arranging the rules. I have a test environment where I can test how the rules work. If everything looks good, I start to deploy it to all instances.

Greetings
Stefan

Hello!

If one of the answers helped you solve your question, please mark it as the solution. This way, you thank the person who helped you and also indicate that the question has been resolved. This, in turn, helps others who come across the same question.

Solution

Thank you!

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.