"They had to live in secret, and when Oppenheimer objected"