"It was hard. His kids liked me better than him."