Windows Agent - LOCAL section not found

Troubleshooting
#1

CMK version: 1.6.0p27
OS version: Windows 2008 R2 / Windows 2012 R2

Hello everyone,

Im having trouble find out where to look to fix this:
We have some windows servers (doesnt seems to be related to specific O.S. version) that the agent output is not showing the LOCAL scripts.

When we run the test locally (PS C:\Program Files (x86)\checkmk\service> .\check_mk_agent.exe test) , everything is fine.We can see the local scripts. Ex;

  TCP    [fe80::5efe:10.233.202.14%13]:45000  [::]:0                 LISTENING
<<<>>>
<<<local>>>
0 Check_Dns - OK - velocity.vcamp.io is resolving correctly
0 check_ec2scheduled - No scheduled events
0 check_mcollective - mcollective is health
0 PuppetCheck - OK -  puppet agent service is running
<<<systemtime>>>
1655919143
PS C:\Program Files (x86)\checkmk\service>

Now if we run the command from the cmk server or any other server: (telnet servername 6556)
The LOCAL section is gone Ex:

  TCP    [fe80::5efe:10.233.202.14%13]:19101  [::]:0                 LISTENING
  TCP    [fe80::5efe:10.233.202.14%13]:28080  [::]:0                 LISTENING
  TCP    [fe80::5efe:10.233.202.14%13]:45000  [::]:0                 LISTENING
<<<>>>
<<<>>>
<<<systemtime>>>
1655919843
Connection closed by foreign host.
[root@cmkcore0005 ~]# telnet c24m0051 6556

On cmk Dashboard, we see this output:

WARN - [agent] Version: 1.6.0p20, OS: windows, Missing agent sections: local WARN, execution time 1.3 sec

The discovery also doesnt find it:

OMD[default]:~$ cmk -vII c24m0051.aws.vtscloud.io
WARNING: Found rules for deprecated ruleset 'inventory_services'. These rules are not applied anymore. In case you still need them, you need to migrate them by hand. Otherwise you can remove them from your configuration.
Discovering services on: c24m0051.aws.vtscloud.io
c24m0051.aws.vtscloud.io:
+ FETCHING DATA
 [agent] Execute data source
 [piggyback] Execute data source
No piggyback files for 'c24m0051.aws.vtscloud.io'. Skip processing.
No piggyback files for '10.233.202.14'. Skip processing.
+ EXECUTING DISCOVERY PLUGINS (1756)
ps.perf does not support discovery. Skipping it.
win_netstat does not support discovery. Skipping it.
  2 df
  1 mem.win
  1 services.summary
  1 systemtime
  1 uptime
  2 winperf_if
  2 winperf_phydisk
  1 winperf_processor.util
  1 wmi_cpuload
SUCCESS - Found 12 services, no host labels
OMD[default]:~$

We have checked the logs and nothing seems off.
We also tried to compare from any other server that is working fine but was not able to find anything so far.

Is worth mention that we started to see this when we upgraded the agents from 1.5.0 version.
He keeps the old version disabled but on some servers we have uninstalled the older version and got the same behavior.
So that makes us believe is not related but we are not sure.

Any help would be appreciated.

Thank you

#2

Hi.
You have too many dashes in your output. The parser looks at the dashe to find metric data. The second dash is cause of the problem.
Try this output instead:

0 Check_Dns - velocity.vcamp.io is resolving correctly
0 check_ec2scheduled - No scheduled events
0 check_mcollective - mcollective is health
0 PuppetCheck - puppet agent service is running

RG, Christian

#3

That’s not the problem. The parser only looks for spaces and counts them.
All text after the third space is agent output.

Keep in mind that the agent runs with the rights of your locally logged in user and not with the user agent runs.
With default settings the agent runs as local system and should have no problems to execute your local scripts. But if you changed from local system to a normal user account it would be normal that not all scripts are visible for this user.

Inside your agent log you should see that the agent starts your local check also if triggered from network and not locally.

For a local test it would be better if you can check with a “telnet localhost 6556”.

#4

i opened a prompt window as LOCAL SYSTEM and runned check_mk_agent test. No issues
Tried the telnet localhost 6556 but get screw up results.

eckmk\agent\check_mk.user.yml
                             AgentDirectory: C:\Program Files (x86)\checkmk\service
                                                                                   PluginsDirectory: C:\ProgramData\chec
kmk\agent\plugins
                 StateDirectory: C:\ProgramData\checkmk\agent\state
                                                                   ConfigDirectory: C:\ProgramData\checkmk\agent\config
                                                                                                                       T
empDirectory: C:\ProgramData\checkmk\agent\tmp
                                              LogDirectory: C:\ProgramData\checkmk\agent\log
                                                                                            SpoolDirectory: C:\ProgramDa
ta\checkmk\agent\spool
                      LocalDirectory: C:\ProgramData\checkmk\agent\local
                                                                        OnlyFrom:
                                                                                  <<<wmi_cpuload:sep(124)>>>
                                                                                                            [system_perf
]
 AlignmentFixupsPersec|Caption|ContextSwitchesPersec|Description|ExceptionDispatchesPersec|FileControlBytesPersec|FileCo
ntrolOperationsPersec|FileDataOperationsPersec|FileReadBytesPersec|FileReadOperationsPersec|FileWriteBytesPersec|FileWri
teOperationsPersec|FloatingEmulationsPersec|Frequency_Object|Frequency_PerfTime|Frequency_Sys100NS|Name|PercentRegistryQ
uotaInUse|PercentRegistryQuotaInUse_Base|Processes|ProcessorQueueLength|SystemCallsPersec|SystemUpTime|Threads|Timestamp
_Object|Timestamp_PerfTime|Timestamp_Sys100NS|WMIStatus
                                                       0||682109142||1232888|91115887828|1270469858|255165907|3619945786
20|181466669|44375973573|73699238|0|10000000|2441427|10000000||114625628|4294967295|153|0|795889746|133000866004991166|4
156|133008179467028727|1785524483187|133008035467020000|OK

Doesnt seem to show everything. Im trying to find a way do use a diferent “console” to test the telnet locally.

By the way, i edit the script to remove the extra dashes just in case but no luck. We have over 5000 servers, only like 10 or so have this kind of issue. That indicates is something locally indeed.
Not able to find out anything in common on them yet. Still investigating.

Worth mention that i tested all scripts, removing one by one from the folder. Same behavior.
We have 1 EXE, 1 python and 2 powershell.
So it seems is not a issue with an specific file type.