Grafana Does Not See 0s (zeros) as Values from CheckMK

Hi,

I use the mk_mysql check-mk-agent plugin to monitor a MySQL server. The MySQL server is a slave replicating data from master. The plugin correctly created a check_mk-mysql_slave check and provides two performance metrics:

relay_log_space=76598137;;;; sync_latency=0;300;600;;

I am interested in sync_latency and I need to display it in Grafana.

I use CheckMK Enterprise and I already installed grafana-checkmk-datasource-1.1.0 to Grafana.

The problem is CheckMK displays the sync_latency values correctly - most of the time the values are zero. But Grafana does not. The problem is Grafana does not “see” the zero values.

Is this a bug in CheckMK, or in Grafana, or in my configuration?

Thank you,
Regards,
Oskar

Hi,
you need to figure out if data are collected beween 8:28 and 10:02. Looks like that no data collected at
the agent or plugin.

Cheers,
Christian

I think it is only a problem inside Grafana how zero values are interpreted.
Normally you can configure this.
The raw output from CMK to Grafana has the zero values i think.

Hi Christian,

the data were certainly collected by CheckMK.

Grafana allows to inspect the CheckMK query. I extracted the query URL and put it in a script:

$ cat mysql_slave_sync_latency.sh 
#!/bin/bash

# SERVER, USERNAME, SECRET
source SECURE.sh

DATA='request={"specification":["template",{"site":"Monitoring","host_name":"S21006","service_description":"MySQL DB Slave mysql","graph_index":1}],"data_range":{"time_range":[1605499383,1605520983]}}'
URL="https://$SERVER/Monitoring/check_mk/webapi.py?_username=$USERNAME&_secret=$SECRET&output_format=json&action=get_graph"

curl -X POST -d "$DATA" "$URL"

The output from the script looks like this

$ ./mysql_slave_sync_latency.sh 

{“result”: {“step”: 60, “start_time”: 1605499380, “end_time”: 1605521040, “curves”: [{“color”: “#ffb080”, “title”: “Sync latency”, “line_type”: “area”, “rrddata”: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.283333, 0.733333, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.9, 0.116667, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}]}, “result_code”: 0}

Notice the long sequence of zeros and occasianally different values, like in this snippet:

0, 0, 0, 0, 0, 0, 0.283333, 0.733333, 0, 0, 0, 0, 0

So the zeros are definitely returned by CheckMK. I need to figure out how to make Grafana accept them. Any tips?

Thanks,
Oskar

I know this is not a Grafana forum, but does anybody know how to configure Grafana to interpret the zeros from CheckMK correctly?

Thanks!

There is something like fill in group by function. To figure that out, please look in Grafana at this point.