Local check on windows not being picked up

I am trying to implement a local check that monitors my backups on SQL Server. The script is in C:\check_mk\local and all it does is run a stored procedure in the local SQL instance that generates output such as:

P _sqlserver_backups_SVWCHI6PSQL1_DBA_full minOld=5480.05;10080;11520 Full backup for database DBA last occurred 3.80559 days (91.3342 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_master_full minOld=515.4;10080;11520 Full backup for database master last occurred 0.357917 days (8.59 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_model_full minOld=515.4;10080;11520 Full backup for database model last occurred 0.357917 days (8.59 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_msdb_full minOld=515.383;10080;11520 Full backup for database msdb last occurred 0.357905 days (8.58972 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_ReportServer_full minOld=5479.87;10080;11520 Full backup for database ReportServer last occurred 3.80546 days (91.3311 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_ReportServer_log minOld=5.4;=120;300 Log backup for database ReportServer last occurred 0.09 hours ago.
P _sqlserver_backups_SVWCHI6PSQL1_SentryOne_full minOld=5476.65;10080;11520 Full backup for database SentryOne last occurred 3.80323 days (91.2775 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_SQLdmRepository_full minOld=5470.42;10080;11520 Full backup for database SQLdmRepository last occurred 3.7989 days (91.1736 hours) ago.

we have a similar script that outputs the same info for our postgres backups on Linux. The only difference is that it prints <<<local>>> first - I had removed it from the windows version after reading some other stuff online but it doesn’t work with it, either. I also tried writing the output to a log in C:\check_mk\temp as was suggested somewhere online as well.

I’ve done many tabula rasa refreshes, restarted the local cmk agent, etc. but nothing seems to make the cmk server see this new check. I also see it when I run .\check_mk_agent.exe test > test.out locally.

Any ideas what might be needed to get this check recognized ?

You probably need to tell us what version number and raw or CEE, etc…

This path is no general path for local checks. If you run 1.6 agent then the path should be C:\ProgramData\checkmk\agent\local\

This line is produced by the agent itself. You only need it if your output is not generated from a script in the folder i mentioned above.

1 Like

We are running 1.5. there is no C:\ProgramData\checkmk\agent\local\ folder.

PS C:\check_mk> .\check_mk_agent.exe version
Check_MK_Agent version 1.5.0p11

Ok for 1.5 it should be - C:\program files(x86)\check_mk\local
But the paths tested by the agent you can see in the first lines of your agent output.

There is no C:\program files(x86)\check_mk\local folder. I have run .\check_mk_agent.exe test and it is seeing the local check.

Now you go to your monitoring web gui and use the option “download agent output” to inspect if the local check output is also there.

the local dir is LocalDirectory: C:\check_mk\local but the checks aren’t showing up in the output for some reason even though they do show up if I run .\check_mk_agent.exe test

<<<check_mk>>>
Version: 1.5.0p11
BuildDate: Jan  8 2019
AgentOS: windows
Hostname: svwchi6psql1
Architecture: 64bit
WorkingDirectory: C:\WINDOWS\system32
ConfigFile: C:\check_mk\check_mk.ini
LocalConfigFile: C:\check_mk\check_mk_local.ini
AgentDirectory: C:\check_mk
PluginsDirectory: C:\check_mk\plugins
StateDirectory: C:\check_mk\state
ConfigDirectory: C:\check_mk\config
TempDirectory: C:\check_mk\temp
LogDirectory: C:\check_mk\log
SpoolDirectory: C:\check_mk\spool
LocalDirectory: C:\check_mk\local

here is what shows up when doing check_mk_agent.exe test. It is not in the agent output from the gui. I have restarted the local agent multiple times.

Platform,SVWCHI6PSQL1,,1,4,4,,<array>,1,3932100000,1,1,,,0,0,3,,Peak6,1,65535,65535,<array>,OK,,,,,,,x64-based PC,3,17179398144,,6," ParameterBinding(Out-File): name="InputObject"; value="<<<local>>>" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_DBA_full minOld=6537.07;10080;11520 Full backup for database DBA last occurred 4.53963 days (108.951 hours) ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_master_full minOld=132.433;10080;11520 Full backup for database master last occurred 0.0919676 days (2.20722 hours) ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_model_full minOld=132.417;10080;11520 Full backup for database model last occurred 0.091956 days (2.20694 hours) ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_msdb_full minOld=132.417;10080;11520 Full backup for database msdb last occurred 0.091956 days (2.20694 hours) ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_ReportServer_full minOld=6536.88;10080;11520 Full backup for database ReportServer last occurred 4.5395 days (108.948 hours) ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_ReportServer_log minOld=42.4167;=120;300 Log backup for database ReportServer last occurred 0.706944 hours ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_SentryOne_full minOld=6533.67;10080;11520 Full backup for database SentryOne last occurred 4.53727 days (108.894 hours) ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="P _sqlserver_backups_SVWCHI6PSQL1_SQLdmRepository_full minOld=6527.43;10080;11520 Full backup for database SQLdmRepository last occurred 4.53294 days (108.791 hours) ago." ParameterBinding(Out-File): name="InputObject"; value="" ParameterBinding(Out-File): name="InputObject"; value="<<<>>>" ParameterBinding(Out-File): name="InputObject";

This output will not work.
Why there are many Powershell comments like the “ParameterBinding(Out-File)”?

I don’t know. it’s a simple one line script. This is the entire thing:

sqlcmd -S . -E -Q "exec DBA.dbo.prcDBAcheckMK_BackupCheck"

the procedure returns multiple lines, which is in the exact same output as a bash script we use to check our Postgres backups. If I change it to a .bat file it still doesn’t work, either.

But you see in your output that there is some text around this lines.
This has something to do with - how you wrote your powershell script.
Such a simple command inside a powershell script is not possible normally.

it is, actually. All it’s doing is running sqlcmd as shown above. Like I said, changing it to a .bat file doesn’t work, either. This is the output if I run the script manually. It’s the same whether it’s a .ps1 or .bat file

PS C:\check_mk\local> .\checkmk_backup_alert.ps1
P _sqlserver_backups_SVWCHI6PSQL1_DBA_full minOld=6598.87;10080;11520 Full backup for database DBA last occurred 4.58255 days (109.981 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_master_full minOld=194.233;10080;11520 Full backup for database master last occurred 0.134884 days (3.23722 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_model_full minOld=194.217;10080;11520 Full backup for database model last occurred 0.134873 days (3.23694 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_msdb_full minOld=194.217;10080;11520 Full backup for database msdb last occurred 0.134873 days (3.23694 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_ReportServer_full minOld=6598.68;10080;11520 Full backup for database ReportServer last occurred 4.58242 days (109.978 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_ReportServer_log minOld=44.2167;=120;300 Log backup for database ReportServer last occurred 0.736944 hours ago.
P _sqlserver_backups_SVWCHI6PSQL1_SentryOne_full minOld=6595.47;10080;11520 Full backup for database SentryOne last occurred 4.58019 days (109.924 hours) ago.
P _sqlserver_backups_SVWCHI6PSQL1_SQLdmRepository_full minOld=6589.23;10080;11520 Full backup for database SQLdmRepository last occurred 4.57586 days (109.821 hours) ago.

The output outside of the agent context is not relevant.
Remember that the script when it runs has no environment - it runs as local system.
You check it every time as a local user.
The only test what you should do is a “cmk -d hostname” from your monitoring server to get the real output directly.

which is the same as running the ‘download agent output’ from above, right ? When I change it to a .bat file, which has the one simple windows command, it does not appear in the agent output. Forget about it being a Powershell script for now. it is a .bat file.

Then i would think a little bit more about the environment.
It looks more like that this command you use is not possible to run.
Inspect the agent log. This can be switched on with the “crash_debug = yes” inside your “check_mk.ini”

there are lots of other .bat (and powershell) files in the plugins directory. Not sure why it wouldn’t run a simple one line bat file

The command you call is the problem not that it is a bat file. Do you had a look at the log file?

there is nothing in the log file indicating an issue w/ the script/bat file.

Then you need to add a little bit debug output to your script file to see what the problem is.

If your script works correctly you have only two possible outcomes.

  • Inside the agent output you see your expected output
  • Inside the agent output you get an error message with some useful information

I think the problem is that you executable called from the script is not found if it runs from the agent as local system. This would be the only explanation i can think off.

it’s probably the LocalSystem issue, yes. I solved the problem by just running the script on a linux box and it works fine.

Edit: that’s not the issue, actually. I used psexec.exe to run a cmd.exe as the SYSTEM account and then ran the bat file and it works just fine.