CheckMK 2.2 can't log in as cmkadmin, password reset not working, user not locked

[coofercat (Ralph Bolton)]

CMK version: 2.2.0p22.cre
OS version: Ubuntu 22.04

Error message: Incorrect username or password. Please try again.

I’ve tried to log in as cmkadmin with the existing password (from a Checkmk 2.1.0p31.cre install), and it failed. Pretty much all the instructions on the subject say to run:

htpasswd -m etc/htpasswd cmkadmin

…which I did, and set a new password. I still can’t log in though. Since the -m means “MD5”, which is disabled in cmk 2.2, I figured maybe it was supposed to be bcrypt:

htpasswd -B etc/htpasswd cmkadmin

…but still no luck. Likewise cmk-passwd cmkadmin. I’ve tried it a few times, and yes, I have locked the user. I’ve unlocked it again by resetting the password, this appears to work, as it no longer says “user is locked”, but instead says “Incorrect username or password. Please try again.”.

In all cases above, I’ve been able to verify the password in htpasswd as correct with:

htpasswd -v etc/htpasswd cmkadmin

I’ve also tried changing the password to different values (in case there’s a character which doesn’t work). I’m running out of ideas - what else can I do here?

(the back story here is that after the upgrade, it seems to have changed the LDAP rules so that no LDAP user is an admin any longer - as such, I need the cmkadmin user to be able to go in to fix it!)

[coofercat (Ralph Bolton)]

Oh my… solved my own problem, but goodness me…

I found var/log/web.log, which shows this if you try to log in with an MD5 password (ie. htpasswd -m ...:

2024-02-19 12:03:11,911 [30] [cmk.utils.crypto.password_hashing 1399756] Invalid hash. Only bcrypt is supported.
Traceback (most recent call last):
  File "/omd/sites/checkmk/lib/python3/cmk/utils/crypto/password_hashing.py", line 103, in verify
    valid = _context.verify(password.raw, password_hash)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk/lib/python3.11/site-packages/passlib/context.py", line 2343, in verify
    record = self._get_or_identify_record(hash, scheme, category)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk/lib/python3.11/site-packages/passlib/context.py", line 2031, in _get_or_identify_record
    return self._identify_record(hash, category)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/omd/sites/checkmk/lib/python3.11/site-packages/passlib/context.py", line 1132, in identify_record
    raise exc.UnknownHashError("hash could not be identified")
passlib.exc.UnknownHashError: hash could not be identified
2024-02-19 12:03:11,965 [30] [cmk.web.auth 1399756] Login failed for username: cmkadmin (existing: Yes, locked: Yes, failed logins until locked: -20), client: 10.0.16.240

What’s more, the user gets locked in two places (which feels like a bug to me). The htpasswd file gets locked (so resetting the password there fixes it) - this stops the UI saying the user is locked. However, you ALSO have to make sure it’s not locked in var/check_mk/web/cmkadmin/cached_profile.mk. This stops the log file saying you’re locked (and allows you to log in).

That file looks like this:

{'user_scheme_serial': 1, 'alias': 'cmkadmin', 'roles': ['admin'], 'locked': False, 'connector': 'htpasswd'}

If it says 'locked': True then change to False and run omd restart to have it take effect. I think I saw some weirdness on restart, solved by resetting the htpasswd password again. YMMV

I’m not 100% sure about this, but I think the entire procedure is:

1. Edit var/check_mk/web/cmkadmin/cached_profile.mk
2. run omd restart
3. Reset the password:cmk-passwd cmkadmin (or htpasswd: htpasswd -B etc/htpasswd cmkadmin`)
4. Log in again

(of anyone’s wondering about the back story… the solution is to re-enter the LDAP bind password and re-save/test)

[Dirk]

Maybe also try cmk-passwd instead of htpasswd?

Cmkadmin - User locked

Since checkmk 2.x the deliver a separate tool to reset passwords: cmk-passwd. Try cmk-passwd -h: cmkpasswd is a utility to add and change Checkmk user accounts in a similar fashion to htpasswd. cmkpasswd will select a secure hashing algorithm that is compatible with this version of Checkmk to protect passwords. Note that the main purpose of this program is setting and resetting the password for the cmkadmin user. For other tasks, such as deleting or deactivating users, use the web interfac…

[coofercat (Ralph Bolton)]

Thanks - yes, I did try that.

It turns out that cmk-passwd does the right things in htpasswd, but does not properly unlock a locked user - you still need to edit var/check_mk/web/cmkadmin/cached_profile.mk and restart

[Dirk]

Ah, ok. I didn’t know that it doesn’t unlock a user. I thought it does but I have no experience with locked users yet

[coofercat (Ralph Bolton)]

Yeah, seems like an oversight/bug that it wouldn’t fully unlock a user when you reset the password. It also seems crazy you need to restart the whole site to fully unlock a user - there may be a less invasive way to do it, but I couldn’t find it.

[Dirk]

@coofercat

restart the whole site to fully unlock a user - there may be a less invasive way to do it

I think just omd restart apache is sufficient. This will make it re-read the ~/etc/htpasswd.