Grafana Does Not See 0s (zeros) as Values from CheckMK

[osksto]

Hi,

I use the mk_mysql check-mk-agent plugin to monitor a MySQL server. The MySQL server is a slave replicating data from master. The plugin correctly created a check_mk-mysql_slave check and provides two performance metrics:

relay_log_space=76598137;;;; sync_latency=0;300;600;;

I am interested in sync_latency and I need to display it in Grafana.

I use CheckMK Enterprise and I already installed grafana-checkmk-datasource-1.1.0 to Grafana.

The problem is CheckMK displays the sync_latency values correctly - most of the time the values are zero. But Grafana does not. The problem is Grafana does not “see” the zero values.

Is this a bug in CheckMK, or in Grafana, or in my configuration?

Thank you,
Regards,
Oskar

[ChristianM (Christian Michaelski)]

Hi,
you need to figure out if data are collected beween 8:28 and 10:02. Looks like that no data collected at
the agent or plugin.

Cheers,
Christian

[andreas-doehler (Andreas)]

I think it is only a problem inside Grafana how zero values are interpreted.
Normally you can configure this.
The raw output from CMK to Grafana has the zero values i think.

[osksto]

Hi Christian,

the data were certainly collected by CheckMK.

Grafana allows to inspect the CheckMK query. I extracted the query URL and put it in a script:

$ cat mysql_slave_sync_latency.sh 
#!/bin/bash

# SERVER, USERNAME, SECRET
source SECURE.sh

DATA='request={"specification":["template",{"site":"Monitoring","host_name":"S21006","service_description":"MySQL DB Slave mysql","graph_index":1}],"data_range":{"time_range":[1605499383,1605520983]}}'
URL="https://$SERVER/Monitoring/check_mk/webapi.py?_username=$USERNAME&_secret=$SECRET&output_format=json&action=get_graph"

curl -X POST -d "$DATA" "$URL"

The output from the script looks like this

$ ./mysql_slave_sync_latency.sh 

{“result”: {“step”: 60, “start_time”: 1605499380, “end_time”: 1605521040, “curves”: [{“color”: “#ffb080”, “title”: “Sync latency”, “line_type”: “area”, “rrddata”: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.283333, 0.733333, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.9, 0.116667, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}]}, “result_code”: 0}

Notice the long sequence of zeros and occasianally different values, like in this snippet:

0, 0, 0, 0, 0, 0, 0.283333, 0.733333, 0, 0, 0, 0, 0

So the zeros are definitely returned by CheckMK. I need to figure out how to make Grafana accept them. Any tips?

Thanks,
Oskar

[osksto]

I know this is not a Grafana forum, but does anybody know how to configure Grafana to interpret the zeros from CheckMK correctly?

Thanks!

[ChristianM (Christian Michaelski)]

There is something like fill in group by function. To figure that out, please look in Grafana at this point.