Register not working

Troubleshooting
1

CMK version: 2.1 p12
OS version: Windows Server 2022

Error message:
Do you want to establish this connection? [Y/n]
Y

thread 'reqwest-internal-sync-runtime' panicked at 'capacity overflow', library\alloc\src\raw_vec.rs:518:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'event loop thread panicked', C:\Users\sk\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.11.11\src\blocking\client.rs:1118:5

Output of “cmk --debug -vvn hostname”: (If it is a problem with checks or plugins)

I have registered over 100 hosts successfull but something is wrong with this one when I use that command:

& 'C:\Program Files (x86)\checkmk\service\cmk-agent-ctl.exe' register -s checkmk.XXX.net -i STAR -P 'XXXXX' -U automation -H sys-vbr02
2

Please try to run the commad with admin privileges.

3

I’m admin. tried also with open powershell with admin mode
Strange Today after Windows Updates I tried again and it worked.

Why does it don’t drop a specific error?

4

We are running into exactly the same issue on several hosts and those are up-to-date. Powershell is executed as admin…I’m not into Rust and lacking the sourcecode, so I can’t give much advice regarding to that, but it seems, there may be some timeout issue when sending the webrequest.

‘reqwest-internal-sync-runtime’ panicked at ‘capacity overflow’

Concerning the error, the only hit in searching the reqwest github is at:

Line 971 in Implemenation ClientHandle

the event_loop_panicked function (leading to “event loop thread panicked” error) is executed only when timeouts fail, as far as I understand the code.

It’s likely, the Devs have to look into this. I would really love to see that issue fixed. It’s not threatening but annoying.

1 Like
5

This looks like something, @SergejKipnis might want to take a look at.

6

Cannot let problems lie around for too long so I dug into it a few inches deeper and compared it to the output of one of our linux clients, which btw. work flawlessly.
The Windows request seems to die when trying the Certificate Request. Here is the debug output of both clients:
Linux:

[root@ClientHostNameHere ~]# cmk-agent-ctl register -H ClientHostNameHere -s CMKHostNameHere -i CMKInstance  -U AdminAcc -P AdminPW -vv --trust-cert
INFO [cmk_agent_ctl] starting
INFO [cmk_agent_ctl] Loaded config from '"/var/lib/cmk-agent/cmk-agent-ctl.toml"', legacy pull 'LegacyPullMarker("/var/lib/cmk-agent/allow-legacy-pull")' absent
DEBUG [reqwest::connect] starting new connection: https://CMKHostNameHere/
DEBUG [reqwest::async_impl::client] response '200 OK' for https://CMKHostNameHere/CMKInstance/check_mk/api/1.0/domain-types/internal/actions/discover-receiver/invoke
DEBUG [reqwest::connect] starting new connection: https://CMKHostNameHere:8000/
DEBUG [reqwest::async_impl::client] response '200 OK' for https://CMKHostNameHere:8000/CMKInstance/agent-receiver/pairing
!!! => Somewhere here the windows agent dies !!!
DEBUG [reqwest::connect] starting new connection: https://CMKHostNameHere:8000/
DEBUG [rustls::client::hs] No cached session for DnsName(DnsName(DnsName("CMKHostNameHere")))
DEBUG [rustls::client::hs] Not resuming any session
DEBUG [rustls::client::hs] Using ciphersuite TLS13_AES_256_GCM_SHA384
DEBUG [rustls::client::tls13] Not resuming
DEBUG [rustls::client::tls13] TLS1.3 encrypted extensions: []
DEBUG [rustls::client::hs] ALPN protocol is None
DEBUG [rustls::client::tls13] Got CertificateRequest CertificateRequestPayloadTLS13 { context: , extensions: [SignatureAlgorithms([ECDSA_NISTP256_SHA256, ECDSA_NISTP384_SHA384, ECDSA_NISTP521_SHA512, ED25519, ED448, Unknown(2057), Unknown(2058), Unknown(2059), RSA_PSS_SHA256, RSA_PSS_SHA384, RSA_PSS_SHA512, RSA_PKCS1_SHA256, RSA_PKCS1_SHA384, RSA_PKCS1_SHA512, Unknown(771), ECDSA_SHA1_Legacy, Unknown(769), RSA_PKCS1_SHA1])] }
DEBUG [rustls::client::common] Client auth requested but no cert/sigscheme available
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [reqwest::async_impl::client] response '204 No Content' for https://CMKHostNameHere:8000/CMKInstance/agent-receiver/register_with_hostname
DEBUG [reqwest::connect] starting new connection: https://CMKHostNameHere:8000/
DEBUG [rustls::client::hs] No cached session for DnsName(DnsName(DnsName("CMKHostNameHere")))
DEBUG [rustls::client::hs] Not resuming any session
DEBUG [rustls::client::hs] Using ciphersuite TLS13_AES_256_GCM_SHA384
DEBUG [rustls::client::tls13] Not resuming
DEBUG [rustls::client::tls13] TLS1.3 encrypted extensions: []
DEBUG [rustls::client::hs] ALPN protocol is None
DEBUG [rustls::client::tls13] Got CertificateRequest CertificateRequestPayloadTLS13 { context: , extensions: [SignatureAlgorithms([ECDSA_NISTP256_SHA256, ECDSA_NISTP384_SHA384, ECDSA_NISTP521_SHA512, ED25519, ED448, Unknown(2057), Unknown(2058), Unknown(2059), RSA_PSS_SHA256, RSA_PSS_SHA384, RSA_PSS_SHA512, RSA_PKCS1_SHA256, RSA_PKCS1_SHA384, RSA_PKCS1_SHA512, Unknown(771), ECDSA_SHA1_Legacy, Unknown(769), RSA_PKCS1_SHA1])] }
DEBUG [rustls::client::common] Attempting client auth
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [reqwest::async_impl::client] response '200 OK' for https://CMKHostNameHere:8000/CMKInstance/agent-receiver/registration_status/SomeNiceIDHere

Windows:

PS C:\ProgramData\checkmk\agent\bin> .\cmk-agent-ctl.exe register --trust-cert -H ClientHostNameHere -s CMKHostNameHere -i CMKInstance  -U AdminAcc -P AdminPW -vv
[2022-11-04 13:20:49.407595 +01:00] INFO [cmk_agent_ctl] src\main.rs:14: starting
[2022-11-04 13:20:49.408645 +01:00] INFO [cmk_agent_ctl] src\lib.rs:41: Loaded config from '"C:\\ProgramData\\checkmk\\agent\\cmk-agent-ctl.toml"', legacy pull 'LegacyPullMarker("C:\\ProgramData\\checkmk\\agent\\allow-legacy-pull")' exists
[2022-11-04 13:20:49.417123 +01:00] DEBUG [reqwest::connect] C:\Users\sk\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.11.11\src\connect.rs:560: starting new connection: https://CMKHostNameHere/
[2022-11-04 13:20:49.471035 +01:00] DEBUG [reqwest::async_impl::client] C:\Users\sk\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.11.11\src\async_impl\client.rs:2017: response '200 OK' for https://CMKHostNameHere/CMKInstance/check_mk/api/1.0/domain-types/internal/actions/discover-receiver/invoke
[2022-11-04 13:20:50.009310 +01:00] DEBUG [reqwest::connect] C:\Users\sk\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.11.11\src\connect.rs:560: starting new connection: https://CMKHostNameHere:8000/
thread 'reqwest-internal-sync-runtime' panicked at 'capacity overflow', library\alloc\src\raw_vec.rs:518:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'event loop thread panicked', C:\Users\sk\.cargo\registry\src\github.com-1ecc6299db9ec823\reqwest-0.11.11\src\blocking\client.rs:1118:5
7

Could it come maybe from the TLS problems which were in the news which where coming from Microsoft Updates? There was a hotfix released yesterday?

4 Likes
8

Well done! That indeed solved the issue. Thanks, man.

9

Hi Maik, could you please check the answer that helped you as Solution (it’s in the menu under 3 dots)? I think it would be a nice way to show the helping person some appreciation and also show other users that the issue has a solution :slight_smile: Thank you!

10

Thanks @righter and @Maik for tracking this down so far!

However, if you’re right and it’s a regression caused by an Oct '22 update, it will only affect systems that are kept up-to-date, so it might be fixed without action from our side.

Please keep us updated whether this comes up again!
If it does, we’ll have a look at this and evaluate whether we can/should apply a workaround for this problem.

11

We faced the same problem and were able to fix it with your solution. Thanks a lot! :slight_smile:

12

Thanks, that helped!

13

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.