Monitoring queues in Exchange 2016

Troubleshooting
1

Dear fellow monitoring-enthusiasts;

I am trying to configure Checkmk Enterprise 1.6.0p6 to monitor MS Exchange 2016 queue lengths. The following two old mailing list posts were my guide for now:

This is what I’ve got so far:

  1. Added a rule in “Monitoring Agents > Agent Plugins > Windows Performance-Counter objects” to feed the counters from winperf base object “MSExchangeTransport Queues” into the check “winperf_msx_queues”. My “check_mk.bakery.yml” on the monitored host now contains these additional lines:
winperf:
  enabled: true
  counters:
    - MSExchangeTransport Queues: msx_queues
  1. My agent output now shows all counters I want to monitor (The first number on each line is the offset from the winperf base object 140960 a.k.a. MSExchangeTransport Queues):
<<<winperf_msx_queues>>>
1580217862.88 140960 1948612
6 instances: total_excluding_priority_none none_priority low_priority normal_priority high_priority _total
2 0 0 0 0 0 0 rawcount
4 0 0 0 0 0 0 rawcount
6 0 0 0 0 0 0 rawcount
8 0 0 0 0 0 0 rawcount
10 0 0 0 0 0 0 rawcount
58 0 0 0 0 0 0 rawcount

This is where I am stuck now: The old mailing list posts now add the following to a file named “main.mk” (The first argument is the counter name, the second is its offset to the base):

#Define Exchange queues to be monitored
winperf_msx_queues = {
“External Active Remote Delivery” : “2”,
“Internal Active Remote Delivery” : “4”,
“External Retry Remote Delivery” : “6”,
“Internal Retry Remote Delivery” : “8”,
“Active Mailbox Delivery” : “10”,
“Poison” : “58”,
}

But my current “main.mk” clearly states:

# Main configuration file of Check_MK
#We highly recommend to use WATO to configure Check_MK these days.

So my question now is: Where in WATO do I configure the “Exchange queues to be monitored”?

I am happy about every answer, thank you in advance for your help!

2

I found the rule “Parameters for discovered services > Discovery - automatic service detection > MS Exchange Message Queues Discovery” but this does not seem to change anything.

e.g. the check “Queue Poison Queue Length” still reports the following:

Invalid check parameter : Offset: The value ‘44’ has the wrong type str, but must be of type int
Variable: checkgroup_parameters:msx_queues
Parameters: {‘levels’: (500, 2000), ‘offset’: ‘44’}

The relating check is “/opt/omd/versions/1.6.0p6.cee/share/check_mk/checks/winperf_msx_queues”, wich defines the following:

...
# Queues to be inventorized (number are relative to counter base)
winperf_msx_queues = {
    "Active Remote Delivery": "2",
    "Retry Remote Delivery": "4",
    "Active Mailbox Delivery": "6",
    "Poison Queue Length": "44",
}
...

I’d guess this means my “MS Exchange Message Queues Discovery”-Rule does not overwrite the default parameters. I defined my offset to be 58 (Currently the default in Exchange 2016), not 44 but the check still reports that.

Thanks again for any help or some pointers in the right direction.

3

I checked with your data an created the following rule inside MS Exchange Message Queues Discovery

image
image1318×732 39 KB
^
Then the discovery shows all the defined queues without error.
image
image1169×313 40.4 KB

In your case the problem is inside the rule MS Exchange Message Queues. There is some offset defined in your system. Remove the offset and check if it is working then.

1 Like
4

Hello Andreas;

Thank you for your quick reply, it helped a lot in troubleshooting this issue.

I found a legacy file (my other servers which I set up after the 1.5 era don’t have that file) on my system called /opt/omd/sites/OER/var/check_mk/core/config.mk which contains the following:

...
winperf_msx_queues = {'Retry Remote Delivery': '4', 'Active Remote Delivery': '2', 'Poison Queue Length': '44', 'Active Mailbox Delivery': '6'}
...

I moved the file so it does not mess with my checks. I also created the same rule as you did, there are no other rules of the type “MS Exchange Message Queues Discovery” defined:

image
image802×410 18.4 KB

After doing a full discovery the checks stay the same, but the parameters no longer show the “Invalid check parameter”-error (I mentioned this in my first post). But the offset stays the same and other no other queues are found:

image
image832×319 19.1 KB

Funny thing is, this happens only on one of my systems. I have other Checkmk Servers in other locations where the configuration works just fine.

Are there maybe some other files messing with my checks? Any guesses?

5

Found it:

Since the site I’m working with has been around for so long, the check “winperf_msx_queues” had to added manually as a local check back then… Even though it’s now a builtin check, it was being overwritten/replaced by a local check.

Here is what I did to fix the issue in my case:

Run: find /opt/omd/sites/OER -name "\*winperf_msx\*"

This should show you, where the checks are being run from. The default locations are as follows:

/opt/omd/sites/OER/var/check_mk/precompiled_checks/builtin/winperf_msx_queues
/opt/omd/sites/OER/tmp/check_mk/check_includes/builtin/winperf_msx_queues

If any of those locations show local instead of builtin, you are probably running a local check.

In my case this came up in the output:

/opt/omd/sites/OER/var/check_mk/precompiled_checks/local/winperf_msx_queues

After I removed the check from local and restored the builtin one, everything worked out as wanted.

Disclaimer: I’m no Checkmk- or Linux-Expert. This is what worked in my case, it might not work in yours. Please consult the Official Documentation before applying any of my described changes.