"because I don't think it's fair to punish a man for making small talk,"