"some of your caucus members are ready to flee to me."