1. Edit the contact or relationship record
2. Change the address to the correct one for the contact.
3. Click OK and choose No when prompted to update all references. This will only change the contact's address, not every reference to the organization.