Well, this is embarrassing. But I want to share it anyway, in case someone else meets this issue:
A colleague of mine hat his own version of the main dashboard.
I am not sure why but that overwrote the main dashboard for all users. I guess we need to read some more docs on that one. 