"Your father cared more about educating freedmen's children"