You will need to delete the address on the source constituent's record prior to merging the two duplicate records.
  1. From Constituents > Constituent Search, search for and open the constituent record
  2. Select the Contact tab
  3. Click the drop-down arrow next to the address and Delete
  4. Repeat this for each address on the record
Now you should be able to complete the merge.
  1. Go to Constituents > Duplicates > Merge two constituents
  2. Search for and enter the Source and Target constituents
  3. In the Merge Configuration drop-down, select Default Configuration
  4. Click Merge