I failed to set up Kerberos Single sign-on (SSO) for Checkmk and Active Directory (AD) in the first place so I decided to write this How-to.
http://files.stippmilch.de/checkmk/checkmk_kerberos_howto.html
Unfortunately I am not allowed to attach html or pdf files, only sh, vbs, and ps1 are allowed 
A simple text dump of the document for the search engines.
Kerberos SSO & Active Directory in 2023 v1.0
Grepmeister MiMiMi
github@stippmilch.de
version v1.0, 2023-07-27 10:00:00
Table of Contents
1. Intro
2. What needs to be done ?
3. Prerequisite
3.1. Set up /etc/krb5.conf
3.2. On Windows
4. Create the keytab file
4.1. Linux (msktutil)
4.2. Windows (ktpass.exe)
4.3. Verify the keytab file and SP in AD
4.4. Install the keytab file on the checkmk server
5. Apache Config
6. Browser Settings
6.1. curl
6.2. Firefox
6.3. Google Chrome
6.4. Microsoft EDGE
6.5. Safari
7. Troubleshooting
8. Resources
9. Known problems
1. Intro
I failed to set up Kerberos Single sign-on (SSO) for Checkmk and Active
Directory (AD) using the Single sign-on with Kerberos document (which is
not officially supported by Checkmk btw.).
It
s not complete as of 2023-07-18.
I guess this is why it
s not officially supported.
I noticed that not only me but also other Checkmk users failed to set up
Kerberos.
So I decided to write this How-to.
It
s not easy and it
s not trivial, since many pieces have to fit
together.
The source of this document is located here.
Please open github issues or pull request to improve the document.
The SAML document introduced for checkmk 2.2. helped a lot to get the
apache config settings right.
I am using Windows Server 2019 and Checkmk 2.1 while writing this
document.
Table 1. Example Environment used in this document:
Item Example in this document Note
Kerberos Realm IT.MYCOMPANY.TLD Always upper
case.
ServicePrincipal HTTP/monitoring.it.mycompany.tld
s HTTP not
(SP) HTTPS.
Monitoring https://monitoring.it.mycompany.tld/prod
Server URL
In Active
Directory (AD)
you have to map
AD Service User svc_checkmk_http_prod a
ServicePrincipal
(SP) to a user
object.
Domain acts as Key
Controller (DC) ad.mycompany.tld Distribution
Center (KDC)
Checkmk Site prod the checkmk site
2. What needs to be done ?
We need to set up a trust relationship between the Checkmk Servers
site apache and the Active Directory (AD) by exchanging a kerberos
keytab file once.
A keytab file contains a kind of shared secret.
To create the keytab file we will either use msktutil on Linux or
ktpass.exe on Windows .
We will need a domain user to map the ServicePrincipal (SP) to.
On Windows you have to create the user manually first.
msktutil on linux will create the user on its own for you.
Finally we have to adjust the checkmk site apache configuration to use
the kerberos in the file auth.conf .
3. Prerequisite
Your checkmk is reachable by a FQDN and uses https (see Securing the
web interface with HTTPS).
Forward and reverse DNS entries are working.
Your Web Browsers trust your checkmk server
s FQDN. (see Browser
Settings below)
You have got a Kerberos Ticket Granting Tickets (TGT) (kinit
<AD-User>, klist, on Windows you get it automatically on Domain login)
Keep in mind that usually your Active Directory (AD) Servers first
have to sync. It can take up to 15 min until all servers know about
the new user and the SPN mapping.
3.1. Set up /etc/krb5.conf
On Linux most kerberos related tools and libraries honor the
/etc/krb5.conf config file.
Therefore I use this minimal /etc/krb5.conf:
/etc/krb5.conf
1 [libdefaults]
2 default_realm = IT.MYCOMPANY.TLD (1)
4 dns_lookup_kdc = false
5 dns_uri_lookup = false
7 [realms]
8 IT.MYCOMPANY.TLD = {
9 kdc = ad.mycompany.tld (2)
10 }
12 [domain_realm]
13 .mycompany.tld = IT.MYCOMPANY.TLD (3)
14 mycompany.tld = IT.MYCOMPANY.TLD (4)
1 replace with your Kerberos Realm
2 replace with your Active Directory (AD)
3 map your DNS domain to your Kerberos Realm
4 map your DNS domain to your Kerberos Realm
3.2. On Windows
Windows Servers typically already know everything about their kerberos
domain.
Usually there is no need to configure anything.
4. Create the keytab file
I prefer to use msktutil to create the keytab containing the
ServicePrincipal (SP), because it is easy and just works.
But since msktutil is not available e.g. on the Checkmk Appliance I
will also explain how to get a keytab file with the Windows tool
ktpass.exe (which has got some pitfalls).
4.1. Linux (msktutil)
You have to adjust some values to match your environment.
as checkmk site user obtain a Kerberos ticket-granting ticket:
kinit <ad-admin-user>
e.g.
$ kinit Administrator@IT.MYCOMPANY.TLD
Password for Administrator@IT.MYCOMPANY.TLD
now it should look like this
$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: Administrator@IT.MYCOMPANY.TLD
Valid starting Expires Service principal
24.07.2023 19:56:34 25.07.2023 05:56:34 krbtgt/IT.MYCOMPANY.TLD@IT.MYCOMPANY.TLD
renew until 25.07.2023 19:56:17
With this TGT we can run kerberized commands against the AD.
This script will create the keytab file for you using your TGT:
msktutil.sh
1 #!/bin/bash
3 msktutil \
4 create \
5 --server ad.mycompany.tld \
6 --description "Created by $USER on $(date +%F)" \
7 --dont-expire-password \
8 --no-pac \
9 --no-reverse-lookups \
10 --user-creds-only \
11 --use-service-account \
12 --keytab svc_checkmk_http_prod.keytab \
13 --account-name svc_checkmk_http_prod \
14 --realm IT.MYCOMPANY.TLD \
15 --enctypes 0x10 \
16 --service HTTP/monitoring.mycompany.tld
Line 05: --server use one of your active directory servers
Line 12: --keytab the name of your keytab file
Line 14: --realm your kerberos realm
Line 15: --enctypes 0x10 stands for aes256-cts-hmac-sha1
Line 16: --service your ServicePrincipal (SP) in the form HTTP/<FQDN>
list the content of the keytab file
$ klist -kte svc_checkmk_http_prod.keytab
Keytab name: FILE:svc_checkmk_http_prod.keytab
KVNO Timestamp Principal
---- ------------------- ------------------------------------------------------
1 24.07.2023 20:13:14 svc_checkmk_http_prod@IT.MYCOMPANY.TLD (aes256-cts-hmac-sha1-96)
1 24.07.2023 20:13:14 HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD (aes256-cts-hmac-sha1-96)
test if you can authenticate using the the keytab
$ kinit -kt svc_checkmk_http_prod.keytab 'svc_checkmk_http_prod@IT.MYCOMPANY.TLD'
$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: svc_checkmk_http_prod@IT.MYCOMPANY.TLD (1)
Valid starting Expires Service principal
24.07.2023 20:16:23 25.07.2023 06:16:23 krbtgt/IT.MYCOMPANY.TLD@IT.MYCOMPANY.TLD
renew until 25.07.2023 20:16:23
1 your kerberos identity
try to get a kerberos service ticket
$ kvno HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD
HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD: kvno = 1
Now you can continue with Install the keytab file on the checkmk server.
4.2. Windows (ktpass.exe)
i.
if you can not use msktutil on linux:
Create a User
Open Active Directory Users and Computers
New > User
User logon name: svc_checkmk_http_prod
Next
Password: Choose a random password, we will set it to random later
anyhow.
User must change password at next login (not selected)
User cannot change password
Passwod never expires
Account is disabled (not selected)
Finish
Properties > Account > Account Options:
This account supports Kerberos AES 256 bit encryption.
Apply
Export Keytab file
open an cmd.exe as Administrator
I use AES256-CTS-HMAC-SHA1-96 because I believe that this is state of
the art.
Create the keytab file
ktpass ^
-princ HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD ^(1)
-mapuser svc_checkmk_http_prod@IT.MYCOMPANY.TLD ^(2)
-out c:\svc_checkmk_http_prod.keytab ^(3)
-ptype KRB5_NT_PRINCIPAL ^
-crypto AES256-SHA1 ^
+rndPass
1 adjust for your ServicePrincipal (SP)
2 adjust for your service user
keytab file, delete it after it got moved to the checkmk site
3
Result should look like this:
C:\Users\Administrator>ktpass ^
-princ HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD ^
-mapuser svc_checkmk_http_prod@IT.MYCOMPANY.TLD ^
-out c:\svc_checkmk_http_prod.keytab ^
-ptype KRB5_NT_PRINCIPAL ^
-crypto AES256-SHA1 ^
+rndPass
Targeting domain controller: ad.mycompany.tld
Successfully mapped HTTP/monitoring.mycompany.tld to svc_checkmk_http_pro.
Password successfully set!
Key created.
Output keytab to c:\svc_checkmk_http_prod.keytab:
Keytab version: 0x502
keysize 95 HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD ptype 1 (KRB5_NT_PRINCIPAL) vno 3 etype 0x12 (AES256-SHA1) keylength 32 (0xd44915d771c04f7b12b74b4caa2f018d51c6007642962bbe4821aa83be395af1)
once you copied the keytab file into your checkmk site you should
remove the file as it contains a valid domain password.
it can take up to n minutes that this gets replicated to your other
domain controllers.
Check the Service Principal to User mapping
C:\Users\Administrator>setspn -L svc_checkmk_http_prod
Registered ServicePrincipalNames for CN=svc_checkmk_http_prod,CN=Users,DC=mycompany,DC=tld:
HTTP/monitoring.mycompany.tld
4.2.1. install the keytab
the quickest way is to pull it from the DC to the omd site
OMD[prod]:~$ kinit <ad-adminitstrator>
OMD[prod]:~$ smbclient -k //ad.mycompany.tld/C$
smb: \> get svc_checkmk_http_prod.keytab
getting file \svc_checkmk_http_prod.keytab of size 101 as svc_checkmk_http_prod.keytab (98.6 KiloBytes/sec) (average 98.6 KiloBytes/sec)
Or use putty/scp or whatever to copy the svc_checkmk_http_prod.keytab
to your checkmk site.
4.3. Verify the keytab file and SP in AD
since we Set up /etc/krb5.conf we can now use tools like kinit, klist,
kvno
list the keytab file content
OMD[prod]:~$ klist -kt svc_checkmk_http_prod.keytab
Keytab name: FILE:svc_checkmk_http_prod.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
3 01/01/70 01:00:00 HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD
get a TGT
OMD[prod]:~$ kdestroy
OMD[prod]:~$ kinit Administrator@IT.MYCOMPANY.TLD
Password for Administrator@IT.MYCOMPANY.TLD:
OMD[prod]:~$ klist
Ticket cache: FILE:/tmp/krb5cc_999
Default principal: Administrator@IT.MYCOMPANY.TLD
Valid starting Expires Service principal
07/26/23 16:39:07 07/27/23 02:39:07 krbtgt/IT.MYCOMPANY.TLD@IT.MYCOMPANY.TLD
renew until 07/27/23 16:39:07
try to get the service ticket
OMD[prod]:~$ kvno HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD
HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD: kvno = 3
list your TGT and service tickets
OMD[prod]:~$ klist -e
Ticket cache: FILE:/tmp/krb5cc_999
Default principal: Administrator@IT.MYCOMPANY.TLD
Valid starting Expires Service principal
07/26/23 16:39:07 07/27/23 02:39:07 krbtgt/IT.MYCOMPANY.TLD@IT.MYCOMPANY.TLD
renew until 07/27/23 16:39:07, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96
07/26/23 16:39:20 07/27/23 02:39:07 HTTP/monitoring.mycompany.tld@IT.MYCOMPANY.TLD
renew until 07/27/23 16:39:07, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96
4.4. Install the keytab file on the checkmk server
copy the svc_checkmk_http_prod.keytab to the prod site
/omd/sites/prod/etc/apache/svc_checkmk_http_prod.keytab
make sure it is only readable by the prod site user:
chown prod:prod svc_checkmk_http_prod.keytab
chmod 400 svc_checkmk_http_prod.keytab
the result should look like this
OMD[prod]:~$ ls -l ~/etc/apache/svc_checkmk_http_prod.keytab
-r-------- 1 prod prod 198 Jul 24 20:13 /omd/sites/prod/etc/apache/svc_checkmk_http_prod.keytab
5. Apache Config
As site user prod:
move away cookie_auth.conf, we do not need it
mv -v ~/etc/apache/conf.d/cookie_auth.conf ~/cookie_auth.conf.bak
New Apache Config ~/etc/apache/conf.d/auth.conf
1 Define SITE prod
2 # ^^^^ (1)
4 Define REALM IT.MYCOMPANY.TLD
5 # ^^^^^^^^^^^^^^^^ (2)
7 <IfModule !mod_auth_kerb.c>
8 LoadModule auth_kerb_module /usr/lib/apache2/modules/mod_auth_kerb.so
9 # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (3)
10 </IfModule>
12 <Location /${SITE}>
14 # Use Kerberos auth only in case there is no Checkmk authentication
15 # cookie provided by the user and whitelist also some other required URLs
17 <If "! %{HTTP_COOKIE} =~ /^(.*;)?auth_${SITE}/ && \
18 ! %{REQUEST_URI} = '/${SITE}/check_mk/register_agent.py' && \
19 ! %{REQUEST_URI} = '/${SITE}/check_mk/deploy_agent.py' && \
20 ! %{REQUEST_URI} = '/${SITE}/check_mk/run_cron.py' && \
21 ! %{REQUEST_URI} = '/${SITE}/check_mk/restapi.py' && \
22 ! %{REQUEST_URI} = '/${SITE}/check_mk/automation.py' && \
23 ! %{REQUEST_URI} -strmatch '/${SITE}/check_mk/api/*' && \
24 ! %{REQUEST_URI} = '/${SITE}check_mk/ajax_graph_images.py' && \
25 ! %{QUERY_STRING} =~ /(_secret=|auth_|register_agent)/ && \
26 ! %{REQUEST_URI} =~ m#^/${SITE}/(omd/|check_mk/((images|themes)/.*\.(png|svg)|login\.py|.*\.(css|js)))# ">
28 Order allow,deny
29 Allow from all
31 Require valid-user
33 AuthType Kerberos
34 AuthName "Checkmk AD Kerberos Login"
35 KrbMethodNegotiate on
36 KrbMethodK5Passwd off
37 KrbLocalUserMapping on
38 KrbSaveCredentials off
40 # Environment specific: Path to the keytab, REALM and ServicePrincipal
41 Krb5Keytab /omd/sites/${SITE}/etc/apache/svc_checkmk_http_prod.keytab
42 # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (4)
44 KrbServiceName HTTP/monitoring.it.mycompany.tld@IT.MYCOMPANY.TLD
45 # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (5)
46 KrbAuthRealm ${REALM}
48 ErrorDocument 401 '<html> \
49 <head> \
50 <meta http-equiv="refresh" content="1; URL=/${SITE}/check_mk/login.py"> \
51 </head> \
52 <body> \
53 Kerberos authentication failed, redirecting to login page. \
54 <a href="/${SITE}/check_mk/login.py">Click here</a>. \
55 </body> \
56 </html>'
58 </If>
60 </Location>
62 # These files are accessible unauthenticated (login page and needed ressources)
63 <LocationMatch /${SITE}/(omd/|check_mk/(images/.*\.png|login\.py|.*\.(css|js)))>
64 Order allow,deny
65 Allow from all
66 Satisfy any
67 </LocationMatch>
1 add your checkmk site name (instance)
2 add your Kerberos Realm
3 on redhat based systems the path is different FIXME
4 add your ServicePrincipal (SP) HTTP/monitoring.it.mycompany.tld
restart site apache
omd restart apache
I noticed, that it is crucial that forward and reverse DNS match.
Now, with a valid TGT you can access your site.
6. Browser Settings
6.1. curl
handy for troubleshooting and quick testing your kerberos setup.
make sure you have a valid tgt ticket (kinit <ad-user>, klist )
curl can do kerberos via negotate autentication
curl --negotiate --user : https://monitoring.it.company.tld/prod/
check for WWW-Authenticate and Authorization headers:
curl \
--verbose \
--silent \
--negotiate -u : \
"https://monitoring.mycomany.tld/prod/check_mk/user_profile.py" 2>&1 \
| grep -P '^[<>]+ (WWW-Authenticate|Authorization): +'
You should see somehting like this
> Authorization: Negotiate YIIFzwYGKwYBBQUCoIIF... (1)
(multiline base64 output
< WWW-Authenticate: Negotiate oYG3MIG0oAMKAQChY... (2)
1 request header
2 response header
6.2. Firefox
Configuring Firefox for Negotiate Authentication
Enter you DNS Domain for which you want to use kerberos:
about: config
network.negotiate-auth.trusted-uris: .mycomany.tld
So it will match monitoring.mycomany.tld
Firefox will then send a HTTP Header to the Checkmk Server that
signals the apache, that it can do Kerberos.
This sould bring you to your user profile without a login screen:
firefox-esr "https://monitoring.mycomany.tld/prod/check_mk/user_profile.py"
6.3. Google Chrome
They recently changed the kerberos relevant parameter for
google-chrome and its clones from --auth-server-whitelist to
--auth-server-allowlist
This works now for me now:
google-chrome --auth-server-allowlist="*.mycompany.tld" https://monitoring.mycomany.tld/prod/check_mk/
s now
-allowlist not
-whitelist anymore.
Source:
https://support.google.com/chrome/thread/201738899/kerberos-sso-stopped-working-under-linux-after-updating-chrome-to-110?hl=en
6.4. Microsoft EDGE
On Windows EDGE usually already trusts your DNS domain.
6.5. Safari
I have no clue. But Safari can do Kerberos SSO as well. Somehow.
7. Troubleshooting
If something is fishy, start from scratch. Delete the domain user,
wait until all DCs synced and create it again.
increasing the apache debug level does not help much instead run it in
the foreground with -X and LogLevel increased to trace8
omd stop apache
$ grep ^LogLevel etc/apache/apache.conf
LogLevel trace8
/usr/sbin/apache2 -f /omd/sites/<site>/etc/apache/apache.conf -X
/usr/sbin/apache2 -f /omd/sites/prod/etc/apache/apache.conf -X
If you recreated the ServicePrincipal and keytab for some reason it
might be the case that you are still working with the old Service
Ticket therefor do a kdestroy, kinit <username> again. On Windows you
may have to logout and login again.
8. Resources
https://web.mit.edu/kerberos/krb5-1.21/doc/admin/conf_files/krb5_conf.html
https://social.technet.microsoft.com/wiki/contents/articles/2751.kerberos-interoperability-step-by-step-guide-for-windows-server-2003.aspx
https://learn.microsoft.com/en-US/troubleshoot/windows-server/windows-security/kdc-event-16-27-des-encryption-disabled#cause
9. Known problems
svc_checkmk_http_prod gets truncated to svc_checkmk_http_pro
sAMAccountName: svc_checkmk_http_pro
Hi,
thanks for sharing with all! I adapted much of your input into the article.
Hey,
Maybe this will sound dumb.
But how do you login with users from a group in AD?
Im confused whether this is made still inside inside kerberos or checkmk?
I’m not versed in this technology
thanks
If you set up Kerberos like in the document, everybody who can authenticate to your Active Directory can also login to your checkmk instance.
But often this is not wanted.
There you can define a role with no access and make it the default
and for some other users you can assign roles like normal user, guest user or administrator.
Usually you set up an ldap sync in Checkmk and let it map ldap groups to Checkmk roles.
I followed your document and was able to make sense of all of this. Thanks a lot for that
I setup LDAP on the past so it should be no issue. I’m just wondering, why is it required to modify the apache auth.conf as well for the LDAP?
Won’t the LDAP configuration inside checkmk be enough? I’m thinking because with this new auth.conf file the authentication won’t be using LDAP and therefore not work if we don’t specify to use LDAP as well?
I’m just wondering, why is it required to modify the apache auth.conf as well for the LDAP?
No it’s not, it’s just an option.
Won’t the LDAP configuration inside checkmk be enough?
Yes, in combination with mapping ldap groups to checkmk roles.