This happens when you have the box checked next to 'Include blanks' when choosing your operator.  Uncheck this box to only include records that have a value associated with a field.