I found some time to play around with the new prometheus special agents and the prometheus_custom check. I had to configure a “dummy” host for this and use the piggyback option to connect the service with other hosts. This dummy host was configured with “no agent but all configured special agents”.
The connection to prometheus works and writes the piggyback data file.
After a while I recognize that the values are everytime the same. The problem was that the piggyback file was not updated regulary. If I run cmk -vpn PROMETHEUS_HOST it gets updated but not every 60s.
After changing the host to “with agent and all configured special agents” it works now.
I am pretty sure that other special agents works without any host agent. Has anyone else tried this? If you have any suggestions what I did wrong please help me If this looks like a bug / point to improve let me know how to help with this.
Hey Marcus,
this is a known issue and the team knows about it and has been discussing solutions around it. Not sure, when we will get to it as it is an architectural topic.
What you did is a viable work-around and probably the best solution in the meantime.
Sorry, Martin
Thank you for your open words. For my special problem it would be ok to have a field to define the IP of the prometheus host in the agent section and not only the port. Maybe this could help you and the team.
The IP comes via the hosts to which the rule are assigned. That’s the standard mechanism for most special agents.
How would specifying the IP help you in your case?
I have a special configuration here. My prometheus is listening only on one of my interfaces (e.g. 192.168.1.1) and the agent is configured for the main interface (e.g. 10.10.10.10). Thats why I needed to create the dummy host for my secondary interface and have now 2 hosts which queries the same agent. This would be no problem if the prometheus agent would not need the normal agent. But if I could specify a query IP for the special Agent or configure that it should use an additional configured IP from my host I would not have this double agent problem. I have never tried to configure the additional IP, would this work?
If this looks like a bug / point to improve let me know how to help with this.