Post upgrade to 2.4.0, check_mail_loop breaks

CMK version: 2.4.0 (no patch) CCE
OS version: Debian 12

Error message: Unhandled exception: RuntimeError('Either Username/Passwort or ClientID/ClientSecret/TenantID have to be set')

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

cmk -D --debug -vvn <REDACTED>

<REDACTED>
Addresses:              <REDACTED>
Tags:                   [address_family:ip-v4-only], [agent:cmk-agent], [checkmk-agent:checkmk-agent], [criticality:prod], [ip-v4:ip-v4], [is_unifi:unset], [networking:lan], [piggyback:no-piggyback], [site:<SITE>], [snmp_ds:no-snmp], [tcp:tcp]
Labels:                 [cmk/device_type:container], [cmk/os_family:linux], [cmk/os_name:Debian GNU/Linux], [cmk/os_platform:debian], [cmk/os_type:linux], [cmk/os_version:12], [cmk/site:<SITE>]
Host groups:            check_mk
Contact groups:         all
Agent mode:             Normal Checkmk agent, or special agent if configured
Type of agent:          TCP: <REDACTED>:6556
Services:
  checktype                      item                  params                                                                                                                                                                                                                                                                                                                                                                                                                                                     description               groups
  ------------------------------ --------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------- ------
  apt                            None                  {'normal': 1, 'removals': 1, 'security': 2}                                                                                                                                                                                                                                                                                                                                                                                                                APT Updates
  cpu_utilization_os             None                  {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         CPU utilization
  checkmk_agent                  None                  {'agent_version': ('ignore', {}), 'agent_version_missmatch': 1, 'restricted_address_mismatch': 1, 'legacy_pull_mode': 1, 'only_from': ('cmk_postprocessed', 'only_from', None)}                                                                                                                                                                                                                                                                            Check_MK Agent
  diskstat                       SUMMARY               {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         Disk IO SUMMARY
  df                             /                     {'levels': (80.0, 90.0), 'magic_normsize': 20, 'levels_low': (50.0, 60.0), 'show_levels': 'onmagic', 'inodes_levels': (10.0, 5.0), 'show_inodes': 'onlow', 'show_reserved': False, 'trend_range': 24, 'trend_perfdata': True, 'item_appearance': 'mountpoint', 'mountpoint_for_block_devices': 'volume_name'}                                                                                                                                              Filesystem /
  lnx_if                         2                     {'errors': {'both': ('perc', (0.1, 0.5))}, 'discovered_oper_status': ['1'], 'discovered_speed': 10000000000, 'item_appearance': 'index'}                                                                                                                                                                                                                                                                                                                   Interface 2
  mem_linux                      None                  {'levels_virtual': ('perc_used', (98.0, 98.0)), 'levels_total': ('perc_used', (120.0, 150.0)), 'levels_shm': 'ignore', 'levels_pagetables': 'ignore', 'levels_committed': ('perc_used', (125.0, 150.0)), 'levels_commitlimit': ('perc_free', (20.0, 10.0)), 'levels_vmalloc': ('abs_free', (52428800, 31457280)), 'levels_hardwarecorrupted': ('abs_used', (1, 1)), 'levels_ram': ('perc_used', (98.0, 98.0)), 'levels_swap': ('perc_used', (98.0, 98.0))} Memory
  postfix_mailq                  default               {'deferred': (2, 10), 'active': (5, 10), 'failed': (1, 1)}                                                                                                                                                                                                                                                                                                                                                                                                 Postfix Queue default
  postfix_mailq_status           default               {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         Postfix status default
  local                          Reboot required       {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         Reboot required
  systemd_units_services_summary None                  {'states': {'active': 0, 'inactive': 0, 'failed': 2}, 'states_default': 2, 'activating_levels': None, 'deactivating_levels': (30, 60), 'reloading_levels': (30, 60), 'ignored': []}                                                                                                                                                                                                                                                                        Systemd Service Summary
  systemd_units_services         postfix               {'states': {'active': 0, 'inactive': 1, 'failed': 2}, 'states_default': 1, 'else': 1}                                                                                                                                                                                                                                                                                                                                                                      Systemd Service postfix
  systemd_units_services         postfix@-             {'states': {'active': 0, 'inactive': 1, 'failed': 2}, 'states_default': 1, 'else': 1}                                                                                                                                                                                                                                                                                                                                                                      Systemd Service postfix@-
  systemd_units_sockets_summary  None                  {'states': {'active': 0, 'inactive': 0, 'failed': 2}, 'states_default': 2, 'activating_levels': None, 'deactivating_levels': (30, 60), 'reloading_levels': (30, 60), 'ignored': ['systemd-journald-audit$']}                                                                                                                                                                                                                                               Systemd Socket Summary
  tcp_conn_stats                 None                  {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         TCP Connections
  uptime                         None                  {'min': (1800, 0)}                                                                                                                                                                                                                                                                                                                                                                                                                                         Uptime
  local                          sshd currently banned {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         sshd currently banned
  local                          sshd currently failed {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         sshd currently failed
  local                          sshd total banned     {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         sshd total banned
  local                          sshd total failed     {}                                                                                                                                                                                                                                                                                                                                                                                                                                                         sshd total failed

Being an active check, it does not show up in cmk -D. My postfix relay is no-authentication required, but limited by firewall and postfix config to only certain IPv4. The CheckMK container is allowed and notifications flow as expected, so it’s not that.

Here is the rule. It was working fine on 2.3. It broke with the 2.4.0 upgrade:


In text form (line breaks added for readability):

check_mk_active-mail_loop!--fetch-protocol=IMAP --fetch-server=imap.gmail.com 
--fetch-tls --fetch-username <REDACTED> --fetch-password-reference imap_loop_gmail_pass:/omd/sites/<SITE>/var/check_mk/core/helper_config/latest/stored_passwords 
--connect-timeout=60 --send-protocol=SMTP --send-server=<REDACTED> --send-port=25 
--mail-from=<REDACTED>@gmail.com --mail-to=<REDACTED>@gmail.com 
--delete-messages '--status-suffix=<REDACTED>-Local Postfix' 
--warning=3660 --critical=3720

I don’t see where I miss user or ‘passwort’ (sic). I have tried an explicit password and pulling from the credential store.

I am having a hard time executing the check directly from the CLI. I have yet to locate the script that “check_mk_active-mail_loop” resolves to.

Just found the binary:
/opt/omd/versions/2.4.0.cce/lib/python3/cmk/plugins/emailchecks/libexec/check_mail_loop
I verified all required args are present plus additional ones.


Traceback from --debug:

Traceback (most recent call last):
  File "/opt/omd/versions/2.4.0.cce/lib/python3/cmk/plugins/emailchecks/libexec/check_mail_loop", line 10, in <module>
    main()
  File "/omd/sites/SITE/lib/python3/cmk/plugins/emailchecks/lib/check_mail_loop.py", line 325, in main
    active_check_main(create_argument_parser(), check_mail_roundtrip)
  File "/omd/sites/SITE/lib/python3/cmk/plugins/emailchecks/lib/utils.py", line 184, in active_check_main
    exitcode, status, perfdata = _active_check_main_core(argument_parser, check_fn, sys.argv[1:])
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/SITE/lib/python3/cmk/plugins/emailchecks/lib/utils.py", line 132, in _active_check_main_core
    return check_fn(args)
           ^^^^^^^^^^^^^^
  File "/omd/sites/SITE/lib/python3/cmk/plugins/emailchecks/lib/check_mail_loop.py", line 223, in check_mail_roundtrip
    send = parse_trx_arguments(args, Scope.SEND)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/SITE/lib/python3/cmk/plugins/emailchecks/lib/ac_args.py", line 209, in parse_trx_arguments
    auth=_parse_auth(raw),
         ^^^^^^^^^^^^^^^^
  File "/omd/sites/SITE/lib/python3/cmk/plugins/emailchecks/lib/ac_args.py", line 168, in _parse_auth
    raise RuntimeError(
RuntimeError: Either Username/Passwort or ClientID/ClientSecret/TenantID have to be set

I added a debug print to the _parse_auth to see what it is getting in:

DEBUG:
{'server': 'REDACTED', 'email_address': None, 'username': None, 'password': None, 'password_reference': None, 'client_id': None, 'client_secret': None, 'client_secret_reference': None, 'tenant_id': None, 'protocol': 'SMTP', 'port': 25, 'tls': False, 'disable_cert_validation': False}

For the sending/local SMTP relay, that is correct. For testing I added bogus SMTP auth. Now it gives a new error:
Could not send email (SMTPNotSupportedError('SMTP AUTH extension not supported by server.'))

So I’m caught in a catch-22 here. I can’t make the plugin run without SMTP authentication. My server doesn’t use authentication though (instead uses firewall and mynetworks to filter).

UPDATE:
I found Werk #15472, but it references adding POP3 back as an option, not changing the auth to allow no authentication for SMTP.

Thanks for reporting the problem. I can reproduce it and opened an internal ticket.

1 Like

If it’s not too much trouble to ask, can you post the werk when a public werk number is available? I want to watch the “release notes” (if you will) for this fix. Thanks!

Sure, I’ll keep you posted.

1 Like

Its fixed as part of 2.4.0p3 Werk #18103: check_mail_loop: Allow no authentication for SMTP

1 Like

I upgraded to 2.4.0p3, and I’m still seeing the same error. I checked the werk list within my site, and 18103 was not listed.

Sorry. You are right. It still persists.

It seems the werk was initially incorrectly tagged, so the fix was not actually part of 2.4.0p3.
The version in the werk is now adjusted and the fix should be in the upcoming release.

Thanks to you both! I look forward to its release

Just installed p4 and the unauthenticated check is working again. Thank you!

1 Like