Rest-API: api/1.0/domain-types/service/collections/all looses comments info temporarily

General
,
1

For our use-case to notify chat when a downtime/ack has run out we currently run the following API Call every 2 minutes:
/api/1.0/domain-types/service/collections/all?columns=comments_with_extra_info&query={"op":"and","expr":[{"op":">","left":"acknowledgement_type","right":"0"}]}

The output we keep as a reference and do a diff to the current version to detect comments/downtimes that have been removed.

It works in principle and also sends out messages to chat.
But sometimes we observe that all ack infos vanisch for specific satellite sites (in distributed monitoring) for one iteration and 2 minutes later come back again.

As if check_mk is forgetting about the ACks because the livestatus link is slow or similar.
Obviously this produces a lot of unnecessary noise on our chat channel when this happens…

Is this a known issue with CMK enterprise 2.1? Is it maybe already fixed in 2.2 or later?

2

OK, so I did test things further and the problem is not that the api looses comments, but the API sometime does not report some hosts (which may or may not have comments).

Essentially what i am doing is this every 2 minutes:

curl -ksG "$CHECK_MK_URL/api/1.0/domain-types/service/collections/all" \
     --data-urlencode 'columns=host_name' \
     --data-urlencode 'columns=description' \
     --data-urlencode 'columns=comments_with_extra_info' \
     --data-urlencode 'columns=acknowledgement_type' \
     --header "Authorization: $CHECK_MK_AUTH" \
     --header "Accept: application/json" \
| jq . > "$(date +%Y%m%d-%H%M%S).json"

this queries ALL services.

And when I analyse this just counting I get:

for x in 20240416-16[45]*.json.gz 20240416-17*json.gz; do echo -n "$x "; zcat $x | jq -r '.value[] | .id' | wc -l; done
20240416-164001.json.gz 70451
20240416-164201.json.gz 70451
20240416-164401.json.gz 70451
20240416-164601.json.gz 70451
20240416-164801.json.gz 70451
20240416-165001.json.gz 70451
20240416-165201.json.gz 70451
20240416-165401.json.gz 26206
20240416-165601.json.gz 70451
20240416-165801.json.gz 70451
20240416-170001.json.gz 70451
20240416-170201.json.gz 70451
20240416-170401.json.gz 70451
20240416-170601.json.gz 70451
20240416-170801.json.gz 70451
20240416-171001.json.gz 70451

so for some reason at arround 16:54:01 the nuber of services has dropped from 70451 down to 26206 and then 2 minutes later it goes back up to 70451.

As far as I can see this is not really expected behaviour - worsted case it should be stale but nut vanish as a whole…