Special agent/plugin development, problem with discovery

Hi everyone,

I’ve currently developed more or less two special agents (iobroker homematic and deconz)(not completly finished yet)

Now I did a third one which should talk to the pihole api etc.
special agent plugin works as desired also the configuration via wato.

now I’m on the agent_based plugin part.
I registered the agent section and the data is processable in the parse function.
So far so good.

But it seems that the discovery function is not triggered and as I think I did nothing different then before I have no clue why this is not working

So, the special agent output looks like this:

<<<pihole_summaryRaw:sep(0)>>>
{"domains_being_blocked": 81590, "dns_queries_today": 213399, "ads_blocked_today": 34986, "ads_percentage_today": 16.394642, "unique_domains": 7447, "queries_forwarded": 76777, "queries_cached": 99316, "clients_ever_seen": 30, "unique_clients": 28, "dns_queries_all_types": 213246, "reply_NODATA": 27044, "reply_NXDOMAIN": 14808, "reply_CNAME": 52306, "reply_IP": 113740, "privacy_level": 0, "status": "enabled", "gravity_last_updated": {"file_exists": true, "absolute": 1621774377, "relative": {"days": 226, "hours": 0, "minutes": 8}}, "check_source_query": "summaryRaw"}

Just the section header and a one-liner json output

the part of the agent_based plugin looks like this

def parse__pihole_stats(string_table):
    data = {}
    for line, in string_table:
        str2obj = json.loads(line)
        data[str2obj["check_source_query"]] = str2obj

    pprint(data)
    return data


register.agent_section(
    name = "pihole_summaryRaw",
    parse_function = parse__pihole_stats
)

def discover__pihole_stats(section):
    pprint(section)
    for item in section:
        yield Service(item=item)


#def check__pihole_stats(item, params, section):
def check__pihole_stats(item, section):
    yield Result(
        state = State.OK,
        summary = f"item: {section[item]}"
    )

register.check_plugin(
    name = "pihole_stats",
    service_name = "Pihole %s",
    discovery_function = discover__pihole_stats,
    #check_default_parameters={},
    check_function = check__pihole_stats,
    #check_ruleset_name="pihole_stats",
)

If I run "cmk --debug -vII --plugins pihole_stats " I get only this, no result, ouput or even a execution of the discovery part.

Discovering services and host labels on: <HOSTNAME>
<HOSTNAME>:
+ FETCHING DATA
Using data from cache file /omd/sites/cmk/tmp/check_mk/cache/<HOSTNAME>
[TCPFetcher] Use cached data
Using data from cache file /omd/sites/cmk/tmp/check_mk/data_source_cache/special_pihole/<HOSTNAME>
[ProgramFetcher] Use cached data
No piggyback files for '<HOSTNAME>'. Skip processing.
No piggyback files for '<IP>'. Skip processing.
[PiggybackFetcher] Execute data source
+ PARSE FETCHER RESULTS
Storing piggyback data for: 7182957b8703
Storing piggyback data for: 43e9c319f2f1
Storing piggyback data for: 47b733547e92
Storing piggyback data for: 428e360bd489
Storing piggyback data for: 649ca77ebd98
Storing piggyback data for: 17cdcc629f22
Storing piggyback data for: b79474b1ed53
Storing piggyback data for: d31d1261104e
Storing piggyback data for: 070215af6014
Storing piggyback data for: 649db183b542
Received piggyback data for 10 hosts
+ EXECUTING HOST LABEL DISCOVERY
{'summaryRaw': {'ads_blocked_today': 35060,
                'ads_percentage_today': 16.382183,
                'check_source_query': 'summaryRaw',
                'clients_ever_seen': 30,
                'dns_queries_all_types': 213860,
                'dns_queries_today': 214013,
                'domains_being_blocked': 81590,
                'gravity_last_updated': {'absolute': 1621774377,
                                         'file_exists': True,
                                         'relative': {'days': 226,
                                                      'hours': 0,
                                                      'minutes': 12}},
                'privacy_level': 0,
                'queries_cached': 99642,
                'queries_forwarded': 76987,
                'reply_CNAME': 52443,
                'reply_IP': 114054,
                'reply_NODATA': 27140,
                'reply_NXDOMAIN': 14856,
                'status': 'enabled',
                'unique_clients': 28,
                'unique_domains': 7447}}
+ PERFORM HOST LABEL DISCOVERY
+ EXECUTING DISCOVERY PLUGINS (0)
SUCCESS - Found no services, 2 host labels

I’ve got the feeling that I’m missing something.
I appriciate every hint :smiley:

1 Like

Hi @eFDe and welcome to the checkmk community.

I am not sure if your separator

at the agent section is the problem. Can you try to remove it and run you inventory again?

Hi @tosch,

thanks for the idea, but thats not it.
My other two agents use the same for the json output.
As far as I understood the separator is in general the ascii code which separates data.
In case of json you can use 0 as separator. Otherwise (I’ve tried with removed sep) you will get this error:

Traceback (most recent call last):
  File "/omd/sites/cmk/bin/cmk", line 92, in <module>
    exit_status = modes.call(mode_name, mode_args, opts, args)
  File "/omd/sites/cmk/lib/python3/cmk/base/modes/__init__.py", line 69, in call
    return handler(*handler_args)
  File "/omd/sites/cmk/lib/python3/cmk/base/modes/check_mk.py", line 1542, in mode_discover
    discovery.do_discovery(
  File "/omd/sites/cmk/lib/python3/cmk/base/discovery.py", line 378, in do_discovery
    _do_discovery_for(
  File "/omd/sites/cmk/lib/python3/cmk/base/discovery.py", line 435, in _do_discovery_for
    discovered_services, host_label_discovery_result = _discover_host_labels_and_services(
  File "/omd/sites/cmk/lib/python3/cmk/base/discovery.py", line 1407, in _discover_host_labels_and_services
    discovered_host_labels = _discover_host_labels(
  File "/omd/sites/cmk/lib/python3/cmk/base/discovery.py", line 1226, in _discover_host_labels
    discovered_host_labels = _discover_host_labels_for_source_type(
  File "/omd/sites/cmk/lib/python3/cmk/base/discovery.py", line 1259, in _discover_host_labels_for_source_type
    applicable_sections = multi_host_sections.determine_applicable_sections(
  File "/omd/sites/cmk/lib/python3/cmk/base/checkers/host_sections.py", line 359, in determine_applicable_sections
    parsed = self._get_parsing_result(host_key, section)
  File "/omd/sites/cmk/lib/python3/cmk/base/checkers/host_sections.py", line 392, in _get_parsing_result
    parsed_result = section.parse_function(data)
  File "/omd/sites/cmk/local/lib/python3/cmk/base/plugins/agent_based/pihole_stats.py", line 188, in parse__pihole_stats
    for line, in string_table:
ValueError: too many values to unpack (expected 1)

Hi,

I think you are doing something with the separator that is not quite your plan.
It looks to me like you’re splitting your JSON output at the whitespace and then doing a json.loads line by line on what’s left of it.
It might be easier to just load the full json output as dict and then get the info you need from the dict.

And the comma should not go there I think:

Hi,

this part was actually from a checkmk tutorial. And for the moment the parse function does not seem to be the problem as it is. Like I said, thats the same thing I use with my other two agents.
And just checked it again, all of the parse functions return a dictionary

{
  'item1':{'stuff':'content'}
  'item2':{'stuff':'content'}
}

sligthly more complex :wink:
there are probably there is more than one way in doing things and probably even better once but for now this at least for me does not explain the strange behaviour.

Just for the understanding, I’m right that the discovery function processes with “section” the parsed data or does it ingest the raw section output?

@eFDe can it be that the name in the register.check_plugin dosen’t match?

<<<pihole_summaryRaw:sep(0)>>>

register.agent_section(
    name = "pihole_summaryRaw",

register.check_plugin(
    name = "pihole_stats",

try it like this

register.check_plugin(
    name = "pihole_summaryRaw",

or add sections to the register.check_plugin

register.check_plugin(
    name = "pihole_stats",
    sections = ["pihole_summaryRaw"]
4 Likes

That’s correct. I also would think that the problem shown by @thl-cmk is the reason for not showing anything.
You parsed dictionary output looks good.

Hi @thl-cmk ,

you made my day. Thats the part I was missing and (althought now it seems somehow pretty obvious) I wasn’t aware of having to declare the section in register.check_plugin if the section has a different name than the check plugin itself.

I was probably lost in “what has to have which name of something else somewhere else and what on the other hand to be ‘mapped’ by your self” :laughing:

So I think I can continue my work for now, much appreciate your help!

3 Likes

Happy to help :wink:

If you think your plugin is ok, upload it to the exchange please. Need to monitor a pihole my self.

dont forget to mark this as resolved, if it resolves your issue :slight_smile:

2 Likes

@eFDe

You should consider to upload your MKPs.

1 Like

It all started with the following Local Check.

pihole_stats-1.0.mkp (1.6 KB)

Then we (@eFDe and I decided to learn how to build datasources with checkmk).

1 Like

@eFDe uploaded the mkp to the checkmk exchange. @all: Take a look :wink:

https://exchange.checkmk.com/p/pihole-special-agent

2 Likes