"And I just couldn't tell you what I wanted to tell you to your face,"