"No one will ever buy my house with him living next door."