Timeout when activating changes on distributed node

[RichardUppheim]

I’m running CheckMK Enterprise 1.6.0p17, and recently I’ve had a couple of issues when activating changes on one distributed node:

State: Failed. Started at: 17:48:12. Finished at: 17:50:03.
Failed: Got invalid data:
Internal automation error: Your request timed out after 110 seconds. This issue may be related to a local configuration problem or a request which works with a too large number of objects. But if you think this issue is a bug, please send a crash report.
Traceback (most recent call last):
File “/omd/sites/ger0/lib/python/cmk/gui/wato/pages/automation.py”, line 186, in _execute_automation_command
html.write(repr(automation.execute(automation.get_request())))
File “/omd/sites/ger0/lib/python/cmk/gui/wato/pages/activate_changes.py”, line 519, in execute
return cmk.gui.watolib.activate_changes.execute_activate_changes(request.domains)
File “/omd/sites/ger0/lib/python/cmk/gui/watolib/activate_changes.py”, line 1285, in execute_activate_changes
warnings = domain_class().activate()
File “/omd/sites/ger0/lib/python/cmk/gui/watolib/config_domains.py”, line 69, in activate
return check_mk_local_automation(config.wato_activation_method)
File “/omd/sites/ger0/lib/python/cmk/gui/watolib/automations.py”, line 112, in check_mk_local_automation
outdata = p.stdout.read()
File “/omd/sites/ger0/lib/python/cmk/gui/htmllib.py”, line 790, in handle_request_timeout
“issue is a bug, please send a crash report.”) % duration)
RequestTimeout: Your request timed out after 110 seconds. This issue may be related to a local configuration problem or a request which works with a too large number of objects. But if you think this issue is a bug, please send a crash report.

I’ve looked at similar topics, for this issue, and one suggestion mentions running this command:
cmk --debug -vvR
This completes successfully with no errors. I did note this line, perhaps unrelated:
Time needed [cmc_all_hosts]: 153.53 sec

This node is busier than other distributed nodes, but I don’t appear to be constrained on cpu or memory. Any ideas?

[ChristianM (Christian Michaelski)]

Hi,
did you check your configuration with cmk --debug -vvU if there are errors?
Cheers,
Christian

[RichardUppheim]

HI Christian,

thanks for the suggestion, I hadn’t tried with that particular parameter. It completed without any errors today when I ran this.
Edit for clarification: the command ran without issues. Activating changes still has issues as per the original post.

[RichardUppheim]

Just to close off the topic and for future reference, I couldn’t find any issues with the configuration, but WATO was still showing the error. I activated the changes on the command line on the distributed site, then cleared the contents of the replication_changes file for the distributed site under “var/check_mk/wato/” on the master.
Subsequent activations in WATO have gone through without issue.