"You know, I don't even think that she'd mind if I had an affair."