When using the RoleSync utility to purge app users that do not currently access CRM, not all such users will be deleted.
The reason why not all non-accessing CRM users can be removed is because if a user has ever created a record in the application that is tied to the database, the user must remain in the system so their "paper trail" is available for future reporting.
I.E. User X added a payment to a constituent record. User x leaves the organization. After User X has left, it is discovered that the payment amount was incorrect. System Administrators may want to see who initially added the payment in question. The System Administrator sees that User X initially added the payment because his account was not purged after he left.