"It's starting to add up against him."