Assign the invoice role to another user first. Then remove the invoice role from the user that should not have the role.