Debug AWS connection

Troubleshooting
#1

Hi,

I would like to monitor AWS via checkmk.
I have the correct permisions on AWS and service ‘Amazon Web Services (AWS)’ on checkmk.

For any reason I didn’t see any discovered services.

OMD[monitor]:~$ cmk -vI AWS
Discovering services and host labels on: AWS
AWS:
+ FETCHING DATA
[ProgramFetcher] Execute data source
No piggyback files for 'AWS'. Skip processing.
[PiggybackFetcher] Execute data source
+ PARSE FETCHER RESULTS
Received no piggyback data
+ EXECUTING HOST LABEL DISCOVERY
+ PERFORM HOST LABEL DISCOVERY
+ EXECUTING DISCOVERY PLUGINS (0)
SUCCESS - Found no new services, no new host labels

And service “Check_MK Discovery” show this error:

no unmonitored services found, no vanished services found, no new host labels, [special_aws] Agent exited with code 1: **CRIT**

There’s any way to troubleshoot this problem? any log? any command?

Thanks in advance.

#2

You could try cmk -vvI --debug AWS

#3

this is the output:

OMD[monitor]:~$ cmk -vvI --debug AWS
Discovering services and host labels on: AWS
AWS:
+ FETCHING DATA
  Source: SourceType.HOST/FetcherType.PROGRAM
[cpu_tracking] Start [7f4bcac35dc0]
Calling: /omd/sites/monitor/share/check_mk/agents/special/agent_aws '--regions' 'eu-west-1' '--global-services' 'ce' '--services' 'cloudwatch_alarms' 'dynamodb' 'ebs' 'ec2' 'elb' 'elbv2' 'glacier' 'rds' 's3' 'wafv2' '--ec2-limits' '--ebs-limits' '--s3-limits' '--glacier-limits' '--elb-limits' '--elbv2-limits' '--rds-limits' '--cloudwatch_alarms-limits' '--dynamodb-limits' '--wafv2-limits' '--s3-requests' '--cloudwatch-alarms' '--wafv2-cloudfront' '--hostname' 'AWS'
STDIN (first 30 bytes): {"access_key_id": "ASIATK2XCBK... (total 106 bytes)
[ProgramFetcher] Fetch with cache settings: DefaultAgentFileCache(base_path=PosixPath('/omd/sites/monitor/tmp/check_mk/data_source_cache/special_aws/AWS'), max_age=MaxAge(checking=0, discovery=120, inventory=120), disabled=False, use_outdated=False, simulation=False)
Not using cache (Does not exist)
[ProgramFetcher] Execute data source
[cpu_tracking] Stop [7f4bcac35dc0 - Snapshot(process=posix.times_result(user=0.0, system=0.0, children_user=0.68, children_system=0.1, elapsed=1.290000000037253))]
Trying to acquire lock on /omd/sites/monitor/var/check_mk/crashes/base/878d5330-01a5-11ec-9c5e-506b8df62905/crash.info
Got lock on /omd/sites/monitor/var/check_mk/crashes/base/878d5330-01a5-11ec-9c5e-506b8df62905/crash.info
Releasing lock on /omd/sites/monitor/var/check_mk/crashes/base/878d5330-01a5-11ec-9c5e-506b8df62905/crash.info
Released lock on /omd/sites/monitor/var/check_mk/crashes/base/878d5330-01a5-11ec-9c5e-506b8df62905/crash.info
Traceback (most recent call last):
  File "/omd/sites/monitor/bin/cmk", line 92, in <module>
    exit_status = modes.call(mode_name, mode_args, opts, args)
  File "/omd/sites/monitor/lib/python3/cmk/base/modes/__init__.py", line 69, in call
    return handler(*handler_args)
  File "/omd/sites/monitor/lib/python3/cmk/base/modes/check_mk.py", line 1542, in mode_discover
    discovery.do_discovery(
  File "/omd/sites/monitor/lib/python3/cmk/base/discovery.py", line 371, in do_discovery
    fetcher_messages=list(
  File "/omd/sites/monitor/lib/python3/cmk/base/checkers/_checkers.py", line 247, in fetch_all
    raw_data = source.fetch()
  File "/omd/sites/monitor/lib/python3/cmk/base/checkers/_abstract.py", line 163, in fetch
    return fetcher.fetch(self.mode)
  File "/omd/sites/monitor/lib/python3/cmk/fetchers/_base.py", line 263, in fetch
    return result.OK(self._fetch(mode))
  File "/omd/sites/monitor/lib/python3/cmk/fetchers/_base.py", line 281, in _fetch
    raw_data = self._fetch_from_io(mode)
  File "/omd/sites/monitor/lib/python3/cmk/fetchers/program.py", line 133, in _fetch_from_io
    raise MKFetcherError("Agent exited with code %d: %s" %
cmk.fetchers._base.MKFetcherError: Agent exited with code 1:
#4

This is the actual call to the special agent. You can add --debug and --verbose to its command line arguments and call it directly. You also need to add --access-key-id and --secret-access-key as this was omitted from the debug output above.

#5

Hello,

Did you happen to get any traction on this? Same thing I am experiencing with two of my AWS regions. Set up three other regions and they work just fine with separate IAM users.

Scott