That will depend on the way your IDP works.
I use risk-based policies to deny access to a resource configured on mine.
In essence what i do is i check after authentication on my IDP if a/the user is member of a group.
If the user is not part of the group he/she will get an ‘access denied’ from the IDP for the resource.