for the individual users,roles and domain you can control policy using the policy option in the admin console ..
for custom privilges,
you can have an attribute associated with that privilege and control access based on the value you set for the attributes.
If you want use the isAllowed method within a custom provider, then you get the user profile information and read the attribute value to get his privilege ..
For adding custom attributes you need to use the ipsadmin command line tool ..
HTH ..