Thanks Karl.
The key pair was in place for root and the user I used for installation but not the site user indeed.
The bad news for me is SSH monitoring only seem to work with id_ed25519 and id_ed25519.pub key pair.
Many of my clients refuse it by default with:
userauth_pubkey: unsupported public key algorithm: ssh-ed25519 [preauth]
Do you know if and where I could switch to id_rsa and id_rsa.pub (RSA format)?