"You're putting up walls and you're avoiding him"