"You thought Fenians, communists, low people..."