Ok, I got it down to 5 seconds execution time.
The issue was that I didn’t know there were 2 config files actively being used!
# system: 'C:\Program Files (x86)\checkmk\service\check_mk.yml'
# user : 'C:\ProgramData\checkmk\agent\check_mk.user.yml'
And it seems I have to change it in both files otherwise the agent will continue using the same config again and again.
I don’t know why there isn’t put more effort in into making that fact more obvious.
Thanks for the help again, I can now go fix other servers as well.