The main point was that all politicians get forced into voting for policies they are against, because often they're bundled with some other policy they are for and they have to vote yes or no to the whole package. So the later someone runs for president, the more the opposition can claim that he's hypocritical, and/or has a track record of voting for ridiculous policies.
Plus, of course, it's harder to keep a clean record the older you get, since you have that many more years to slip up, and the opposition is always waiting for their chance.