Check_httpv2 cert check on own CA webservices: error trying to connect: invalid peer certificate: UnknownIssuer

gero1 · April 30, 2024, 3:53pm

CMK version: 3.2.0 raw
OS version: Ubuntu Server 22.04

Since I switched my cert checks from check_http (now deprecated in 2.3.0) to check_httpv2 I get the following error message for Web-Services that are using our internal CA:

Example URL: https://greenbone.ms.wvk.lwl.org

OMD[cmk]:/opt/omd/sites/cmk/lib/nagios/plugins$ ./check_httpv2 -vvv --url https://greenbone.ms.wvk.lwl.org
2024-04-30T15:35:34.899472Z TRACE send_request: hyper::client::pool: checkout waiting for idle connection: ("https", greenbone.ms.wvk.lwl.org)
2024-04-30T15:35:34.899513Z DEBUG send_request: reqwest::connect: starting new connection: https://greenbone.ms.wvk.lwl.org/
2024-04-30T15:35:34.899526Z TRACE send_request: hyper::client::connect::http: Http::connect; scheme=Some("https"), host=Some("greenbone.ms.wvk.lwl.org"), port=None
2024-04-30T15:35:34.899604Z DEBUG hyper::client::connect::dns: resolving host="greenbone.ms.wvk.lwl.org"
2024-04-30T15:35:34.900535Z DEBUG send_request: hyper::client::connect::http: connecting to 172.25.64.80:443
2024-04-30T15:35:34.900807Z DEBUG send_request: hyper::client::connect::http: connected to 172.25.64.80:443
2024-04-30T15:35:34.900828Z DEBUG send_request: rustls::client::hs: No cached session for DnsName("greenbone.ms.wvk.lwl.org")
2024-04-30T15:35:34.900912Z DEBUG send_request: rustls::client::hs: Not resuming any session
2024-04-30T15:35:34.900935Z TRACE send_request: rustls::client::hs: Sending ClientHello Message {
    version: TLSv1_0,
    payload: Handshake {
        parsed: HandshakeMessagePayload {
            typ: ClientHello,
            payload: ClientHello(
                ClientHelloPayload {
                    client_version: TLSv1_2,
                    random: 013df31011c981a1196ba8d62590d60f3444d0eec6c9dad3baa7345537d8baa0,
                    session_id: 8ce29d1dc140961a7cb5e21e58b0a68c818d50a6b8350a33b94a34d03a2576e3,
                    cipher_suites: [
                        TLS13_AES_256_GCM_SHA384,
                        TLS13_AES_128_GCM_SHA256,
                        TLS13_CHACHA20_POLY1305_SHA256,
                        TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
                        TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
                        TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
                        TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
                        TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
                        TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,
                        TLS_EMPTY_RENEGOTIATION_INFO_SCSV,
                    ],
                    compression_methods: [
                        Null,
                    ],
                    extensions: [
                        SupportedVersions(
                            [
                                TLSv1_3,
                                TLSv1_2,
                            ],
                        ),
                        ECPointFormats(
                            [
                                Uncompressed,
                            ],
                        ),
                        NamedGroups(
                            [
                                X25519,
                                secp256r1,
                                secp384r1,
                            ],
                        ),
                        SignatureAlgorithms(
                            [
                                ECDSA_NISTP384_SHA384,
                                ECDSA_NISTP256_SHA256,
                                ED25519,
                                RSA_PSS_SHA512,
                                RSA_PSS_SHA384,
                                RSA_PSS_SHA256,
                                RSA_PKCS1_SHA512,
                                RSA_PKCS1_SHA384,
                                RSA_PKCS1_SHA256,
                            ],
                        ),
                        ExtendedMasterSecretRequest,
                        CertificateStatusRequest(
                            OCSP(
                                OCSPCertificateStatusRequest {
                                    responder_ids: [],
                                    extensions: ,
                                },
                            ),
                        ),
                        ServerName(
                            [
                                ServerName {
                                    typ: HostName,
                                    payload: HostName(
                                        DnsName(
                                            "greenbone.ms.wvk.lwl.org",
                                        ),
                                    ),
                                },
                            ],
                        ),
                        SignedCertificateTimestampRequest,
                        KeyShare(
                            [
                                KeyShareEntry {
                                    group: X25519,
                                    payload: 3485ecad23a2c74b15ad0efc11509a9f03cd6c3289bce63f78d821e227f1c972,
                                },
                            ],
                        ),
                        PresharedKeyModes(
                            [
                                PSK_DHE_KE,
                            ],
                        ),
                        Protocols(
                            [
                                ProtocolName(
                                    6832,
                                ),
                                ProtocolName(
                                    687474702f312e31,
                                ),
                            ],
                        ),
                        SessionTicket(
                            Request,
                        ),
                    ],
                },
            ),
        },
        encoded: 010001080303013df31011c981a1196ba8d62590d60f3444d0eec6c9dad3baa7345537d8baa0208ce29d1dc140961a7cb5e21e58b0a68c818d50a6b8350a33b94a34d03a2576e30014130213011303c02cc02bcca9c030c02fcca800ff010000ab002b00050403040303000b00020100000a00080006001d00170018000d00140012050304030807080608050804060105010401001700000005000501000000000000001d001b000018677265656e626f6e652e6d732e77766b2e6c776c2e6f726700120000003300260024001d00203485ecad23a2c74b15ad0efc11509a9f03cd6c3289bce63f78d821e227f1c972002d000201010010000e000c02683208687474702f312e3100230000,
    },
}
2024-04-30T15:35:34.905774Z TRACE send_request: rustls::client::hs: We got ServerHello ServerHelloPayload {
    legacy_version: TLSv1_2,
    random: e16d915e524176bffb90a9ac155355da40977bde0da7f132db59a3e297734b30,
    session_id: 8ce29d1dc140961a7cb5e21e58b0a68c818d50a6b8350a33b94a34d03a2576e3,
    cipher_suite: TLS13_AES_256_GCM_SHA384,
    compression_method: Null,
    extensions: [
        SupportedVersions(
            TLSv1_3,
        ),
        KeyShare(
            KeyShareEntry {
                group: X25519,
                payload: eb76cf3822a0002b9468cb4751c3657a7176c79a9a1e1dc86791caf1500adf02,
            },
        ),
    ],
}
2024-04-30T15:35:34.905802Z DEBUG send_request: rustls::client::hs: Using ciphersuite TLS13_AES_256_GCM_SHA384
2024-04-30T15:35:34.905816Z DEBUG send_request: rustls::client::tls13: Not resuming
2024-04-30T15:35:34.905820Z TRACE send_request: rustls::client::client_conn: EarlyData rejected
2024-04-30T15:35:34.905887Z TRACE send_request: rustls::conn: Dropping CCS
2024-04-30T15:35:34.905899Z DEBUG send_request: rustls::client::tls13: TLS1.3 encrypted extensions: [ServerNameAck, Protocols([ProtocolName(6832)])]
2024-04-30T15:35:34.905905Z DEBUG send_request: rustls::client::hs: ALPN protocol is Some(b"h2")
2024-04-30T15:35:34.905922Z TRACE send_request: rustls::client::tls13: Server cert is [Certificate(b"0\x82\x06\xff0\x82\x04\xe7\xa0\x03\x02\x01\x02\x02\x13\x15\0\0\x01\xeb\xe1\x06\r\xe2}>]\0\0\x02\0\0\x01\xeb0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\00e1\x0b0\t\x06\x03U\x04\x06\x13\x02DE1\x0c0\n\x06\x03U\x04\x08\x13\x03NRW1\x110\x0f\x06\x03U\x04\x07\x13\x08Muenster1\x0c0\n\x06\x03U\x04\n\x13\x03KVW1\x0f0\r\x06\x03U\x04\x0b\x13\x06KVW-IT1\x160\x14\x06\x03U\x04\x03\x13\rKVW-MachineCA0\x1e\x17\r231117105846Z\x17\r251116105846Z0\x81\x801\x0b0\t\x06\x03U\x04\x06\x13\x02DE1\x1c0\x1a\x06\x03U\x04\x08\x13\x13Nordrhein-Westfalen1\x110\x0f\x06\x03U\x04\x07\x13\x08Muenster1\x0c0\n\x06\x03U\x04\n\x13\x03kvw1\x0f0\r\x06\x03U\x04\x0b\x13\x06kvw-IT1!0\x1f\x06\x03U\x04\x03\x13\x18greenbone.ms.wvk.lwl.org0\x82\x02\"0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x01\x05\0\x03\x82\x02\x0f\00\x82\x02\n\x02\x82\x02\x01\0\x9e\xa3,+\x95\xe1&\xc4n\0h\xd99\x89\xe2t\xdcG\xac\xaa\xad5\xa7;\x82\xa1\x8c\xe5\x8cu\xfd\xe8\x01O\xd0/\x83\r\x08&F\xa8\x0eG{Q(\xb0\x159\x08\x80\xe2\x04n\xcc\xb7\xc2\xdfLX%\x85\x1b\xc3\xa7\x15\x83\x8c\x8c_\xc3\xd0f\r\x18\xbb\"\xba\xf6zT\xf0\x07\xccTe\xe5\xe8\x18\x7fF`G\x94UqR\x19Y<\xb6[\xccx\x04\xe2a\x983\xd1\xcaIG\xf5\xe0\x07\xe4\xca\xf7\xd8\xb4\x87\xcay@\xce\xb5\xd5\xf1e\xf5k$0\xc8\xa0P(\xa8\xe6\xb1\x88\x91U%\xca\x94\x8e@\xff\xf6\xfe_:,\x0bIO@\xe0\x86\x0f\x8e/,&s\xe8\x9a\xf9}\xcd\xb7=6\xbc\x93\xa4\xd8\x03.\x96M\xae\x8f\xd3\xea\x06$\xf6\xea\x8f\xd6sM\x8b\xb4|MnY\x01\x90\0\x0c\n&\x86@e>\xf3\xd7`\x84\x16\x19\xfa\x8f\xaf\xb4A\xa4\xb16\x01\xc5\xfc\xb8#5\x98\xf8\xb8\xda\xb6e\xc0s\x07x*\xe7@\x96\xed5<\xe9\xf2\xe8\xc5M\xb5\xec\xbc\xba/\xfd\xf5\x82\xa2\xba\xdd\xa2\x84\xccB\xe5%\xe5\x0fb\x12\xe9ijf\xf8\xe0tW[\xa6\x8bH\x94\xbf\x1c\x96\xba\xd5\xeb\xbeB0\x8a\x89\xa8\xb6\xf9\xact\xcd\xc71W\xee\xdfoYE7\xc8\x12\x07\xcc\xb2\xe1ep\xd6\xb0G\xf4\xaaJ\xb8\xcfZ\x99\x04\xcd\xe2+r\xa6\x06\xc2\x94\xebA\xfe\x97\xfbp9u\x9a0^\xdd\xc8\xd5Ub& \xad#80\xfe\xf8\xee\xe2\x1a\xfcEv\xe8\x1d\x13g\x8c\x1c_\xf2\x8eZ\xc3\xb0\xcf\xb5\\\xf82k\xd5?a\0\xd7V\xfa\xf1\xb8\xb7\xd8{85\xc9U\x90\xedu8x\xf5fDMH\xe9\x88)\xe0\xdc\xf8\xb0\x8d:\x91\xabc\x93Qd\x87\xeaw:\xba.$\x04\xde\xc4\xc3t\xed\xe8\xf0\x0c\x0f\xcd\xd8\xdb\x93\x0e\xa6T\xd3\xa0wv\xaa\x86\xcf\xbc\xe0bk\xd0\x07\x99.\xcbE\xd6\x1d\xe71_\xaf\xc1\x8a{\xb1\xfb6lq\xea\xae\xd9\x80u\x07V\xcc\xbe}\x97\xaeN\xd3\x02\xe9\x84\xcf3r.\xc8\x7f8\x8d\x02!\x02\x03\x01\0\x01\xa3\x82\x01\x8a0\x82\x01\x860\x0e\x06\x03U\x1d\x0f\x01\x01\xff\x04\x04\x03\x02\x05\xa00\x13\x06\x03U\x1d%\x04\x0c0\n\x06\x08+\x06\x01\x05\x05\x07\x03\x010.\x06\x03U\x1d\x11\x04'0%\x82\x18greenbone.ms.wvk.lwl.org\x82\tgreenbone0\x1d\x06\x03U\x1d\x0e\x04\x16\x04\x14\xea\xf6\xa9h\x9cgu\xd4\x8a\xa7l\x02E\xbcB\xc4\x1f\x8d\xe3.0\x1f\x06\x03U\x1d#\x04\x180\x16\x80\x14\xa3sGo\xa3Ak&\x17DD\xb5\x86\x1dvG\xab\xa5\xfdx0D\x06\x03U\x1d\x1f\x04=0;09\xa07\xa05\x863http://pki.kvw-muenster.de/pki/KVW-MachineCA(2).crl0O\x06\x08+\x06\x01\x05\x05\x07\x01\x01\x04C0A0?\x06\x08+\x06\x01\x05\x05\x070\x02\x863http://pki.kvw-muenster.de/pki/KVW-MachineCA(2).crt0;\x06\t+\x06\x01\x04\x01\x827\x15\x07\x04.0,\x06$+\x06\x01\x04\x01\x827\x15\x08\x83\x9f\xc9m\x82\xc8\xc2\x17\x86\xa9\x8d\x17\xc2\xc8\x03\xdc\xee\x16M\x81\x93\xaau\x82\x83\xafz\x02\x01d\x02\x01\x080\x1b\x06\t+\x06\x01\x04\x01\x827\x15\n\x04\x0e0\x0c0\n\x06\x08+\x06\x01\x05\x05\x07\x03\x010\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\0\x03\x82\x02\x01\01\x1a\xb46#\xd71\xb3\xaa\x99]B\x9f\xcd\xa0,\xec\xa5MF\xd6\xa3J\x0f\xf5\x86\xd9\x02vXJ\xe5\xd1\xc1\x07\x1b_\x81\x1c\xcbS\x89\xbe\xe5Z\x14k\xe2\xce1C\x9d$x\xaf\xfb\xf3X`\"~\xde`\xd9xv\xfe\xf3\x91O\xaed{4\xdb\xc9\xbf\xeef\\\x1b\xda\xd4\x03\x0e>\xbd\x83\xb0n\x7f(:\xc2\xf6^\xe8n\x04w]\xe3\x1d\x1a,\xe7\xc0\xe7\xc7\x95\xfd\xadnr\x94\x90\x0697\x8b\xbc\xde\xcb\x90\xd1x?\x90\x18x/\x18\xb1\xfc<\x1b\x90p\xc0\xac\xcf\xadm\x06Y\xf5@0\x06\xf9t\x0f[3\xb6\x9ct\x8c\xafM\xaff\x02\x8b\xb1@\xf7\xe4\xc7J\xb5^\x92sAQ\xc1J\xd8\xe5\xd4\x91\x05\x966\xf3!\x97\xa5\xe9\x15\x8d\x96f(\xcf\x80\x1cD\x02\x83\xe5*\x04L\xf2\x90/\xdc\x07\xc4\xc6\xca\xa2}\x02\xd0'\xe01\xb0\x91DJ\xd1\xe5\xba1D\xa7\xae\x928*\x8b{)\xf5Y\xda\x11S\x90\x83\xa9\x05\xc7\xd2\xdb\xc1Lpq\xbe\xa3[=\xa5\x7f\xb9}$Sb\x82\x02w:K\xf0\xbb\xc57\x8c\xf0\x16[\xf1g\xc6bQ\xfeQ;\0\xfc\x9c_\xed\xb7\x8dtKC0c2\xd78L\xae\xf7\x82\x17J\x88t\x04\x1ee\xd4\xb2/\xb2\x8eO\xb0\x9fS\xefu<\xb0\x1bP\xb4\x8b\xf0\xc4\xcb0\xff4z\xd5uN|\xc0\x9e\xe3\xe4~\x13p\xa8\x1f\x0c\xbc\x93\xb2\xb6\xccH\x0b$\xb0\xd4X8\xaa\x17|lA\xe4\xf2\x93mI`\x18\t\x02\xe0\xc2kA\xd6\xcf\xda(\xc2}Il\xf9\xf1\x99,\x92:Gy\xa8x\xf82\n\xe7\x98\xeb\x1au\x13\xc3+\xda\xe2\xfa\xa5`\xadd\xbb^\x17\xe9_\x1e\xdd\xfe\xd1e\x98Q\xd2\x9f\xa8o:2\xdeU{(\xf6\xc3R1\xc0\xe3}x\x04Cn;5v>\xbfB\x02\x94\xe7\x92mN\xb5\xd7\n36\xce\xcd\xcc\x14\xe9\0\xbc\xc6\xdb\x19a\xdc/\xc0\xcajw\xee?\xab\xe5\x02\xdc\xaf+A\x98Xw\x15\x12\xa7\xb3b\xa6\xc6Oh\xc2+\x8c\x12\x1b4*kt")]
2024-04-30T15:35:34.906050Z TRACE send_request: hyper::client::pool: checkout dropped for ("https", greenbone.ms.wvk.lwl.org)
error sending request for url (https://greenbone.ms.wvk.lwl.org/): error trying to connect: invalid peer certificate: UnknownIssuer (!!)
error sending request for url (https://greenbone.ms.wvk.lwl.org/): error trying to connect: invalid peer certificate: UnknownIssuer (!!)

How can I fix this? So that I am not getting an error?

chauhan_sudhir · April 30, 2024, 5:38pm

Have you also checked check_cert?

 ~/lib/nagios/plugins/check_cert --url greenbone.ms.wvk.lwl.org --port 443
OK - Certificate obtained in 51 ms, Verification: OK, Certificate expires in 54 day(s) (Jun 24 06:36:22 2024 +00:00)
response_time=51ms;60000;90000;;
Certificate chain verification OK
validity=54;30;0;;

andreas-doehler · April 30, 2024, 6:06pm

Is it possible that the ca certificate is missing in the chain send by your webserver? Or better said no chain was send?

gero1 · May 1, 2024, 11:05am

Thanks, that was the problem

r.sander · May 1, 2024, 7:25pm

The webserver does not need to send the CA root certificate, only the intermediates.

The client (in this case check_httpv2) needs to have the CA root certificate locally to verify from the end of the certificate chain.

wregz8q2r6 · May 22, 2024, 10:21am

Having a similar problem on a LAN scope with the same basic error message:

OMD[mysite]:~/lib/nagios/plugins$ ./check_httpv2 --url https://10.10.10.10:12345
error sending request for url (https://10.10.10.10:12345/): error trying to connect: invalid peer certificate: UnknownIssuer (!!)
error sending request for url (https://10.10.10.10:12345/): error trying to connect: invalid peer certificate: UnknownIssuer (!!)

OMD[mysite]:~/lib/nagios/plugins$ ./check_http --ssl -L --sni -p 12345 -I 10.10.10.10
<A HREF="https://10.10.10.10:12345/" target="_blank">HTTP OK: HTTP/1.1 200 OK - 4907 bytes in 0.050 second response time </A>|time=0.050296s;;;0.000000;10.000000 size=4907B;;;0;

What makes it difficult is that a) it’s some stupid auto-signed cert issued to localhost yet forces https over http, and b) I don’t have access to where I suspect the .crt to be in order to import it to checkmk and (hopefully) force-accept it.
Browsers say it’s invalid too, I’m not sure why it worked with check_http v1 to begin with but it did.
Simplest way (short of just leaving it on v1 until that feature gets kicked out, and then… concocting something, I guess) would be to just have something akin to curl -l / wget --no-check-certificate but the new plugin doesn’t feature that option.

: Well, I found the cert eventually and managed to import the right one, alas…

OMD[mysite]:~$ lib/nagios/plugins/check_httpv2 --url https://10.10.10.10:12345
error sending request for url (https://10.10.10.10:12345/): error trying to connect: invalid peer certificate: NotValidForName (!!)
error sending request for url (https://10.10.10.10:12345/): error trying to connect: invalid peer certificate: NotValidForName (!!)

Not sure how to fix that one, since the CN this thing is issued to is “localhost” and there doesn’t seem to be an ignore option.

chauhan_sudhir · May 22, 2024, 3:40pm

Maybe you can put this strange certificate in the site trust store and try the check again ?
Setup >> General >> Global settings >> Trusted certificate authorities for SSL

wregz8q2r6 · May 22, 2024, 3:46pm

Thanks for the suggestion -
I tried that already once I managed to locate and extract it, it’s what merely changed my error from “UnknownIssuer” to “NotValidForName” because of the awkward “localhost” CN (and no SAN) in it.