"I'm so disappointed it didn't work with him and Penny."