[BUG] Create host using REST API with custom tags

Hello,

I’m using CFE 2.0.0p11 and would like to create hosts with custom tags using the REST API but it is failing. Here is my request and response where “tag_host_type” is the custom tag. Note that the built-in tag “tag_address_family” is fine:

curl -X 'POST' \
  'http://192.168.50.2/site/check_mk/api/1.0/domain-types/host_config/collections/all' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "folder": "/core",
  "host_name": "tc001",
  "attributes": {
    "ipaddress": "192.168.50.1",
    "tag_address_family": "ip-v4-only",
    "tag_host_type": "test_client"
  }
}'


{
  "title": "Internal Server Error",
  "status": 500,
  "detail": "'NoneType' object has no attribute 'get'",
  "crash_id": "e615ac6a-4ba4-11ec-aabe-fa163e5c0a06",
  "crash_report": {
    "href": "http://10.20.161.190/site/check_mk/crash.py?crash_id=e615ac6a-4ba4-11ec-aabe-fa163e5c0a06&site=site",
    "method": "get",
    "rel": "cmk/crash-report",
    "type": "text/html"
  },
  "stack_trace": [
    "Traceback (most recent call last):",
    "  File \"/omd/sites/site/lib/python3/cmk/gui/wsgi/applications/rest_api.py\", line 403, in _wsgi_app",
    "    return wsgi_app(environ, start_response)",
    "  File \"/omd/sites/site/lib/python3/cmk/gui/wsgi/applications/rest_api.py\", line 207, in __call__",
    "    wsgi_app = self.func(ParameterDict(path_args))",
    "  File \"/omd/sites/site/lib/python3/cmk/gui/plugins/openapi/restful_objects/decorators.py\", line 533, in _validating_wrapper",
    "    response = self.func(param)",
    "  File \"/omd/sites/site/lib/python3/cmk/gui/plugins/openapi/endpoints/host_config.py\", line 79, in create_host",
    "    body['folder'].create_hosts([(host_name, body['attributes'], None)])",
    "  File \"/omd/sites/site/lib/python3/cmk/gui/watolib/hosts_and_folders.py\", line 1867, in create_hosts",
    "    self.verify_host_details(host_name, attributes)",
    "  File \"/omd/sites/site/lib/python3/cmk/gui/watolib/hosts_and_folders.py\", line 1890, in verify_host_details",
    "    _must_be_in_contactgroups(_get_cgconf_from_attributes(attributes)[\"groups\"])",
    "  File \"/omd/sites/site/lib/python3/cmk/gui/watolib/hosts_and_folders.py\", line 2134, in _get_cgconf_from_attributes",
    "    v = attributes.get(\"contactgroups\", (False, []))",
    "AttributeError: 'NoneType' object has no attribute 'get'",
    ""
  ]
}

Am I doing something wrong here or could it be a bug?

Thanks!

1 Like

I am just stumbling upon exact the same problem on 2.0.0p15.cee !
I already wasted 2h before I realized that this must be a bug.

1 Like

Hello,

I commented out the must_be_in_contactgroups call from verify_host_details (see stack trace in op) and my REST request succeeded so something not quite right with contactgroups and custom tags.

@staticmethod
def verify_host_details(name, attributes):
    # MKAuthException, MKUserError
    # _must_be_in_contactgroups(_get_cgconf_from_attributes(attributes)["groups"])
    validate_host_uniqueness("host", name)
    _attributes = update_metadata(attributes, created_by=config.user.id)

I’m sure must_be_in_contactgroups is there for a reason and removing it will have undesired side effects elsewhere but I am not familiar enough with the system atm to understand that and implement a proper fix.

Please advise.

Thanks!

Hi all

Here is my understanding thus far:

_must_be_in_contactgroups (lien 2715) is used to determine whether the current (API) user has permission to edit the host (i.e.: they are in the correct contact groups, if any).

The actual issue stems from _get_cgconf_from_attributes (line 2133) which tries getting the key contactgroups from its parameter attributes. The latter seems to be None for some reason.

The attributes themselves are given to the function verify_host_details by cmk.gui.plugins.openapi.endpoints.host_config.create_host by extracting them from the POST data body["attributes"].

Now for the body: This gets parsed in cmk.gui.plugins.openapi.restful_objects.decorators.Endpoint.wrap_with_validation by calling request_schema().load(json_data) (around line 517).

The request_schema is set in the Endpoint decorator of create_host and resolves to cmk.gui.plugins.openapi.restful_objects.request_schemas.CreateHost (this class inherits from marshmallow.Schema).

The relevant part of this schema would probably be the line attributes = fields.attributes_fields..., which resolves to cmk.gui.fields.definitions.attributes_field. This returns an object of type Nested, if I am not mistaken.

This is about where my reverse engineering ends, though. Currently, I am at a loss what happens inside the returnal of this Nested object…
Maybe someone else is better able to assist.

1 Like

Same here.

For others:
When using the cmk docker distribution, keep in mind you’ll need to docker cp the patched file into the container and then also you need to stop/start the docker container to restart the webserver instance.

once that was done, things worked as expected (maybe not as intended).

my first guess would be that the issue only affects automation users - the api doc examples seem to be using full user accounts that would have a normal, valid, contactgroup backing.

I was gonna set up webhooks from Netbox, but TBH it seems better to wait a few months than to maintain patches of the base files. :confused:

I am wondering if I just have to put the automation user to Contact Group Everything to make it work ? I’ll try later and let you know if this helps.

It would be great if anybody could try to reproduce this error in the recently released 2.0.0p16. It looks like a bug, but there have been some changes (especially regarding the custom tags) to the system since the p11, so I can’t say for sure if it is still a problem or not.

Sure thing :slight_smile:

I tried creating a host using custom tags and using custom attributes. Both tests were run on the interactive API GUI. Once as automation user with bearer authentication and another time using my personal account with cookie auth.

The following results:

  • Using custom tags seems to work as expected
    curl -X 'POST' \
      'https://HOST/SITE/check_mk/api/1.0/domain-types/host_config/collections/all' \
      -H 'accept: application/json' \
      -H 'Authorization: Bearer automation SECRET' \
      -H 'Content-Type: application/json' \
      -d '{
        "folder": "/",
        "host_name": "example.com",
        "attributes": {
          "ipaddress": "192.168.0.123",
          "tag_os_version": "deb"
        }
      }'
    
    Results in the host example.com being created in the main directory with the tag “OS version” set to “Debian”.
  • Using custom attributes, however, does not seem to work as of now. A call
    curl -X 'POST' \
      'https://HOST/SITE/check_mk/api/1.0/domain-types/host_config/collections/all' \
      -H 'accept: application/json' \
      -H 'Authorization: Bearer automation SECRET' \
      -H 'Content-Type: application/json' \
      -d '{
        "folder": "/",
        "host_name": "example.com",
        "attributes": {
          "dummy": "test"
        }
      }'
    
    where dummy would be the ID of a custom attribute, results in the same HTTP 500 as stated above:
    {
      "title": "Internal Server Error",
      "status": 500,
      "detail": "'NoneType' object has no attribute 'get'",
      "crash_id": "11d9175e-4ea4-11ec-ac6c-0242ac120008",
      "crash_report": {
        "href": "https://HOST/SITE/check_mk/crash.py?crash_id=11d9175e-4ea4-11ec-ac6c-0242ac120008&site=SITE",
        "method": "get",
        "rel": "cmk/crash-report",
        "type": "text/html"
      },
      "stack_trace": [
        "Traceback (most recent call last):",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/wsgi/applications/rest_api.py\", line 403, in _wsgi_app",
        "    return wsgi_app(environ, start_response)",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/wsgi/applications/rest_api.py\", line 207, in __call__",
        "    wsgi_app = self.func(ParameterDict(path_args))",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/plugins/openapi/restful_objects/decorators.py\", line 610, in _wrapper",
        "    response = func(param)",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/plugins/openapi/restful_objects/decorators.py\", line 541, in _validating_wrapper",
        "    response = self.func(param)",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/plugins/openapi/endpoints/host_config.py\", line 79, in create_host",
        "    body['folder'].create_hosts([(host_name, body['attributes'], None)])",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/watolib/hosts_and_folders.py\", line 1873, in create_hosts",
        "    self.verify_host_details(host_name, attributes)",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/watolib/hosts_and_folders.py\", line 1896, in verify_host_details",
        "    _must_be_in_contactgroups(_get_cgconf_from_attributes(attributes)[\"groups\"])",
        "  File \"/omd/sites/SITE/lib/python3/cmk/gui/watolib/hosts_and_folders.py\", line 2140, in _get_cgconf_from_attributes",
        "    v = attributes.get(\"contactgroups\", (False, []))",
        "AttributeError: 'NoneType' object has no attribute 'get'",
        ""
      ]
    }
    
    I submitted the report above, in case this helps.

As an FYI: When I run the non-working call while the host example.com still exists, the request instead fails with HTTP 400, stating that the host is already present.

I hope this helps.

Hello,

I retested with 2.0.0p16.cfe and couldn’t reproduce the issue. It appears to have been fixed.

Thanks for the input everyone!

I must reopen this - with p17 and automation users you cannot create hosts with custom attributes.
You get the same error message as shown here

@ChristophRauch it is also not working with the hack to remove the contactgroup check.

4 Likes

Is this bug still exist in p19?

Hi bartilla

I just tested this with p19 CEE and the issue seems to still exist, unfortunately.

I think i need to change the title. That someone will answer :smiley:
Or create a new topic as i cannot remove the solved status.

We are looking into this. :eyes:

Using the ‘Bulk create hosts’ endpoint instead of the ‘Create host’ endpoint seems to be a workaround for now.
Just passing a single host in the list, at least it’s working for me right now. Can anyone confirm?

Using the bulk endpoint does not make a difference for me :slightly_frowning_face:
As soon as I try to create a host with a custom attribute, I get the same error as if using the single host endpoint.

Sample payload that fails with HTTP 500 for me (test_attr being a custom attribute):

{
  "entries": [
    {
      "attributes": { "test_attr": "test" },
      "folder": "/",
      "host_name": "example.com"
    }
  ]
}

Edit: 2.0.0p20 CEE

Ok, the bulk create seemed to work sometimes for me.
I found a better work around eventhough it’s still pretty anoying.
You can set the contactgroups attribute to it’s default value manually by passing: 'attributes'{'contactgroups':{'groups':['all]}} in your request, alongside the other attributes including the custom tags. This works for me, I will post my explanation soon maybe the devs can check that out.

So my explanation for the bug is this:

Somewhere along the journey of creating a host, the function verify_host_details() is called.
Within this function _get_cgconf_from_attributes() is called:

/opt/omd/versions/2.0.0p19.cfe/lib/python3/cmk/gui/watolib/hosts_and_folders.py

def _get_cgconf_from_attributes(attributes: HostAttributes) -> HostContactGroupSpec:
    v = attributes.get("contactgroups", (False, []))
    return convert_cgroups_from_tuple(v

The get returns a default value False.

When the function is called from verify_host_details() Line 1984 in hosts_and_folders.py it’s return value, which is False, is used as an argument for _must_be_in_contactgroups() <Line 2728>.
In there, if no contact group is specified None is returned:

def _must_be_in_contactgroups(cgs: Iterable[ContactgroupName]) -> None:
    """Make sure that the user is in all of cgs contact groups

    This is needed when the user assigns contact groups to
    objects. He may only assign such groups he is member himself.
    """
    if config.user.may("wato.all_folders"):
        return

    if not cgs:
        return  # No contact groups specified

Which results in some other function involved in the host creation process trying to use get() on the NoneType Object which raises the error.

Thank you.
I think I misunderstood your previous post. The issue I am facing only occurs when using custom attributes, not tags. When only using tags, everything is working as expected. E.g., the following payload works flawlessly (still on p20):

{
  "folder": "/",
  "host_name": "example.com",
  "attributes": {
    "tag_os_version": "deb"
  }
}

Where os_version is a simple custom tag to set the operating system.

Your reverse engineering still holds up, though. (And nicely extends my pervious analysis in [BUG] Create host using REST API with custom tags - #4 by ttrafelet :slight_smile: )

This problem will be tackled starting this week. I will try to keep you posted here.
Look out for new releases though. :slight_smile:

Thanks everyone for the analysis!

1 Like