Add a new system role for the specific smart field.  Manually add it to the user who should have access to this field.