Problem with values from checkmk-cloud-datasource

Checkmk Dev&Exchange
, , , ,
1

CMK version: 2.4.0p8 enterprise
OS version: virtual appliance, on site
Plugin: checkmk-cloud-datasource (on Grafana v12.2.0-16895479002)

in CheckMK I collect some data via a custom python script. Part of the data is also the current last ticketnumber “error-id” with the value of 28782579.

If I view the (raw) data in grafana (table view) it shows 28782600. So somewhere in between CheckMK and Grafana the the number is rounded. I did not find a way to avoid this.

Could it be because the number already has an unfavorable variable type in CheckMK? How can I test or change this?
I also tried to retrieve the value directly via the API, but failed because of the format of the API call.

Any ideas to get the exact value of the field into Grafana?

EDIT: Link to bug report in GitHub

2

Can you please check if this happens between Grafana and Checkmk or in Checkmk?

You can view the raw data of a graph via “Export as JSON” in the Graph menu. Please zoom into the new part of the graph, so no raw values getting merged.

image
image1622×1072 168 KB

I also asked some colleges, and they seem to agree that our metric backend (RRD) is not the place to store integers. There seem to be multiple obstacles that are problematic with that. One of them is the data point merging mentioned above: if you zoom out on the graph, multiple points are merged and their average is calculated. Another problem seems to be that RRD interpolates values to a fixed time steps.

I think you are working around the fact that there is currently no way to show the service summary in Grafana? Make the summary field available to display · Issue #200 · Checkmk/grafana-checkmk-datasource · GitHub

3

That’s a simple RRD problem i think.
Here you can find a good explanation what rrdtool does with your stored number.
http://rrdtool.vandenbogaerdt.nl/process.php

But in the end you can only avoid this behavior with two different approaches.

  • configure your RRD in a way that it does no normalization and consolidation
    • i think this is difficult to achieve inside a CMK environment
  • sent your performance metrics direct to a metric storage like InfluxDB where you can define way easier how the consolidation is done. Normally there is no automatic consolidation like it is done inside the CMK rrd’s.
4

Thank you for your answer.

This is the original Data

image
image1335×40 8.21 KB

and the
image

But in the JSON it is already rounded to

…“rrddata”: [28831100.0,…

So I think I am beginning to understand the problem now: The data is correcly reported to CheckMK, but will be saved as a rounded number because of rrd. Thats also why Grafana can only get the rounded values. So there seems not to be a way to just pass a value from the Service Metrics to Grafana. (Service Summery is just “OK” in my case)

Historical we are comming from Icinga and someone implemented it in this way, but I understand that CheckMK is just not to be the right tool for this job.

Thank you for pointing me in the right direction. I will search a for a different approach.

Actually we don’t need history, but just the last value from the python script, I will look into the live-data via wrapper PLUTEX/grafana-livestatus: CheckMK Livestatus for Grafana JSON datasource

5

The mentioned livestatus wrapper seems not work with the current version of CheckMK and apache. Any idea how to get livedata into Grafana without an extra storage like InfluxDB?