Systemtime is wrong because of the agent execution time (false-positive)

Hello everyone,
I came across a strange behavior of the systemtime check. That I want to adress to you guys. Hope this is the right category for this.

Quick overview of the situation:
I have a V-Center appliance with a checkmk agent installed.
I’m monitoring the V-Center with the vphere api (special agent) and the checkmk agent.
The total execution time is around 30 sec… On the V-Center is a systemtime check deployed that gathers the unix time and format it to windows time format (systemtime check). The reason for this is to register a time drift on the vcenter.

Because of the agent execution time the time drift of the systemtime check is around +30 sec…
The system time of the V-Center ist correct and the ntp is working.

The agent execution time is the reason for a false-positive because checkmk is not considering the total execution time of a agent in the calculation for the right system time.

I hope this is clear enough and you get what I mean.
Maybe we can discuss this topic here and get a solution for this.

Thanks in advance :wink:
Norm

I think this happens only on vCenter servers with extra agent installed.
Is your vsphere special agent configured like it was in the old versions (vSphere host + agent)?
I would try to use the configuration the newer way that i setup the normal agent and the vsphere special agent as normal datasource.
In every agent the system time is the last output to prevent the problem you described.
In this old setting i think you get first the agent output generated and then it fetches the special agent output and there comes the delay.

Hi @andreas-doehler
I double checked the problem.

I’m monitoring the vcenter using the special vsphere datasource and a installed checkmk agent. I think that’s the newer way. I guess. :thinking:
This is the rule:


The vsphere datasource outputs the system time. The time section is not coming from the agent. I used another vcenter host for comparison.
The execution time is still around 30 sec. The system time section is from the vsphere datasource. Because of the execution time, the reported system time from the vsphere datasource is outdated and i get a false positive in the system time check.
To workaround this I installed a plugin on the agent that takes the unix system time and converts it to the system time output like a windows system would output it to the checkmk. With this in place, I overwrite the wrong time from the vsphere datasource. But it’s not consistent and a pretty dirty workaround.
The optimal solution would be that the execution time is considered in the server side check for the system time.

Thanks for the help.

That’s the old way :slight_smile:
An actual datasource program should run beside every other datasource program. The ESX special agent was the first special agent with also a normal agent on the same system.
You can try one thing - create two hosts one with special agent only and one with normal CMK agent.
Is the system time then also wrong?
If this is working i would try to create a host with the configuration for normal agent and all enabled datasource programs. The ESX datasource program should then be configured as vSphere without agent.

Hi @andreas-doehler
The customer upgraded the vCenter to the latest version (6.7).
The execution time is now ~1sec. So there is no real problem at the moment and I can’t test this behaviour anymore.
Thank you anyways for your support. :slightly_smiling_face: