You can remove a permission from a federation access group in Horizon Console.
If a role that is applicable only to federation access groups comes to have administrator permissions on access groups, before you can remove the administrator permissions on the federation access groups, you must remove the corresponding permissions for the assigned administrators on the access groups.