"I didn't kick him out of our lives."