Ok, I found it, after posting here, so I’ll share the solution.
The webserver was set to receive only requests coming in on the specific name of that website (which is good practise)
So there was a difference between entering hostname and entering the IP address in the form in CheckMK’s webserver.
It turns out, I need to use the ‘virtual host’ option and leave the ‘hostname’ blank (I didn’t know that was an option)