ActiveMQ error since 2.0

Hello,

since the update to CheckMK 2.0.0 I get the following error with ActiveMQ:

[agent] Version: 2.0.0p3, OS: linux, [special_activemq] Agent exited with code 1: option --protocol not recognizedCRIT , Missing monitoring data for check plugins: mq_queuesWARN , execution time 1.3 sec

This is configured via the Rule:
Setup → Agents → Other integrations → Apache ActiveMQ queues

In this Rule I can/have specified:

  • Server Name
  • Port Number
  • Protocol (HTTP/HTTPS)
  • Use Piggyback
  • Basicauth settings

The check worked before the update and nothing other was changed.
I tried to configure the check all possible ways, but the result is always the same.
I have no glue where this “protocol error” comes from.

Would be fine if someone have an hint for me.

Thank you

Hello @sho,

can you run the following on the commandline as the site-user (replace HOSTNAME accordingly) and post the output:
cmk --debug -vv HOSTNAME
There you should see what special agent checkmk is trying to execute. Maybe there is an older version of the agent inside the ~/local hierarchy that is being executed instead of the one provided with checkmk 2.0.
Up until 2.0 the activemq-special-agent could only communicate via http, the option to select the protocol was not available in older versions (without a backport).

Cheers,
Lorenz

Hello @lkoenig,

Thank you for your reply.
I tried the command you mentioned, with the following result:
cmk --debug -vv
Checkmk version 2.0.0p3
Try license usage history update.
Try aquire lock on /omd/sites//var/check_mk/license_usage/next_run
Got lock on /omd/sites//var/check_mk/license_usage/next_run
Try aquire lock on /omd/sites//var/check_mk/license_usage/history.json
Got lock on /omd/sites//var/check_mk/license_usage/history.json
Next run time has not been reached yet. Abort.
Releasing lock on /omd/sites//var/check_mk/license_usage/history.json
Released lock on /omd/sites//var/check_mk/license_usage/history.json
Releasing lock on /omd/sites//var/check_mk/license_usage/next_run
Released lock on /omd/sites//var/check_mk/license_usage/next_run
Updating IPv4 DNS cache for : XXX.XXX.XXX.XXX
Try aquire lock on /omd/sites//var/check_mk/ipaddresses.cache
Got lock on /omd/sites//var/check_mk/ipaddresses.cache
Releasing lock on /omd/sites//var/check_mk/ipaddresses.cache
Released lock on /omd/sites//var/check_mk/ipaddresses.cache
+ FETCHING DATA
Source: SourceType.HOST/FetcherType.PIGGYBACK
[cpu_tracking] Start [7fe0c10fd9d0]
Piggyback file ‘/omd/sites//tmp/check_mk/piggyback//’:
No piggyback files for ‘XXX.XXX.XXX.XXX’. Skip processing.
[PiggybackFetcher] Fetch with cache settings: NoCache(path=PosixPath(’/omd/sites//tmp/check_mk/data_source_cache/piggyback/’), max_age=0, disabled=False, use_outdated=False, simulation=False), Cache enabled: False
[PiggybackFetcher] Execute data source
[cpu_tracking] Stop [7fe0c10fd9d0 - Snapshot(process=posix.times_result(user=0.010000000000000009, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.0))]
[cpu_tracking] Start [7fe0c10fd0a0]
+ PARSE FETCHER RESULTS
Source: SourceType.HOST/FetcherType.PIGGYBACK
No persisted sections loaded
→ Add sections: [‘esx_vsphere_vm’, ‘labels’]
Received no piggyback data
Loading item states
Try aquire lock on /omd/sites//tmp/check_mk/counters/
Got lock on /omd/sites//tmp/check_mk/counters/
Releasing lock on /omd/sites//tmp/check_mk/counters/
Released lock on /omd/sites//tmp/check_mk/counters/
Saving item states
Piggyback file ‘/omd/sites//tmp/check_mk/piggyback//’:
No piggyback files for ‘XXX.XXX.XXX.XXX’. Skip processing.
[cpu_tracking] Stop [7fe0c10fd0a0 - Snapshot(process=posix.times_result(user=0.010000000000000009, system=0.0, children_user=0.0, children_system=0.0, elapsed=0.009999999776482582))]
[piggyback] Valid sources: , execution time 0.0 sec | execution_time=0.010 user_time=0.020 system_time=0.000 children_user_time=0.000 children_system_time=0.000 cmk_time_agent=0.000

Sadly nothing related to “activemq” or any “protocol” issue…

Thank you

It looks like checkmk is only fetching the piggyback-data for this host and the checkmk-agent as well as the activemq-agent are not being executed.
Did you run this command for the same host where there error was shown in you first post? And does it still show the same error on the webinterface for that host?
It looks like the host for which you ran the command is not configured for any active agent queries.

Hello,
sorry for my late answer.
I run the command you mentioned direct on the cmk host, with the host to be checked as argument.
Yes, this was the same host. For me a little bit confusing is, that here a agent is needed. Because from the settings in my first post, i thought that’s a kind of “active check”, so cmk can connect direct to the activemq?

Hi,

I’m running into the same problem:

no unmonitored services found, no vanished services found, no new host labels, [special_activemq] Agent exited with code 1: option --protocol not recognizedCRIT

In the debug output I see that it tries to execute this command:

Calling: /omd/sites/TWO/share/check_mk/agents/special/agent_activemq ‘–servername’ ‘145.20.183.111’ ‘–port’ ‘8161’ ‘–protocol’ ‘http’ ‘–username’ ‘admin’ ‘–password’ ‘xxxx’

If I run this on the command line I get:

/agent_activemq --servername 145.20.183.111 --port 8161 --protocol http --username admin --password xxxxx
Usage:
agent_activemq --servername {servername} --port {port} [–piggyback] [–username {username} --password {password}]
.
option --protocol not recognized

If however I run without the --protocol option as suggested, I get this:

agent_activemq --servername 145.20.183.111 --port 8161 --username admin --password xxxxx
Unable to connect. Credentials might be incorrect: mismatched tag: line 117, column 2

And in the web configuration of the check I cannot choose to not have a protocol. Any suggestions on how to solve this issue are more than welcome.

Hello Louis,

thank you for your reply.
Because it seeems that I’m not alone with this case, I reported a bug by Check_MK.
I’ll keep you informed.

Hello @sho, hello @louis,

it seems that the commandline-option “protocol” was not correctly implemented in the current activemq-agent.
As seen in the source-code for the active_mq-agent (~/lib/check_mk/special_agents/agent_activemq.py):

29     short_options = ""
30     long_options = ["piggyback", "servername=", "port=", "username=", "password="]

However, a few lines down, the protocol-parameter is correctly parsed:

57         elif o in ['--protocol']:
58             opt_protocol = a

And if you don’t specify that parameter it defaults to http. But right now you don’t even reach that far in the code, as the agent aborts because it doesn’t recognize the protocol parameter.
As @louis stated, the rule automatically adds the protocol parameter, so you can’t configure the command line in a way, that it doesn’t send the protocol parameter.

A quick fix would be to add the protocol parameter to the /opt/omd/versions/XXXX/lib/check_mk/special_agents/agent_activemq.py on line 30 as such:

30     long_options = ["piggyback", "servername=", "port=", "username=", "password=", "protocol="]

In version 1.6 and below i would just say copy the agent to the ~/local hierarchy, make the changes there and checkmk should use the corrected agent.
But in 2.0 they changed the folder-structure and internals quite heavily, so in my quick testing that did not work.

If you however need a quick fix, you could do the following but be aware, that this is not update safe and would apply to all sites on your monitoring server!

  1. As root make a backup of the special-agent (change the version accordingly):
cp /opt/omd/versions/2.0.0p3.cre/lib/check_mk/special_agents/agent_activemq.py  /opt/omd/versions/2.0.0p3.cre/lib/check_mk/special_agents/agent_activemq.py.bak
  1. And then (also as root) add the above mentioned parameter in line 30 to /opt/omd/versions/2.0.0p3.cre/lib/check_mk/special_agents/agent_activemq.py

After a cmk -R the agent should now recognize the protocol parameter.
Again, this would be a quick and dirty fix, please make a backup of the original file if you try this, but i hope this helps.

If anyone knows how the ~/local hierarchy works with 2.0 for modifying the built-in files, please let me know!

Cheers,
Lorenz

1 Like

Hi,

Thank you for your extensive answer. For me that solves the problem for now. I no longer get the error on the discovery and ActiveMQ monitoring works again.

Regards,
Louis.

Hello lkoenig,

thanks for your detailed explanation and bug fix proposal.
I’ll try that, but because it’s working for louis, I’m sure it’s wworking for me too.
I’m looking forward that this get fixed upstream soon.

Thank you
sho

@louis glad to hear it worked!
@sho I opened a Pull Request for these changes in the offical checkmk-repository yesterday. I hope that it’ll get merged soon so the fix can be in the next release.

Cheers,
Lorenz

Just FYI: There is now a Werk regarding this issue, so this should be fixed in 2.0.0p4: Apache ActiveMQ queues special agent: Parse protocol argument

2 Likes