Are Contact Groups appropriate to use as a "permissions system"?

I mean the opposite. Notifying all contacts makes sense, but I would also expect to NOT notify non-contacts (unless there’s specifically been a global rule set for it, because that’s an admin-level scope, not user-level).

Well as long as the notification system is the only hole here, I can work with that.