On November 02 2012 01:32 TheTenthDoc wrote:
Why should pollsters adjust their results to fit some arbitrary assumption of turnout when their standard methodology is showing a different result? They don't set out to sample X% democrats, X% republicans, and X% independents-that's silly and means your poll is totally useless. If 40% of the respondents that are likely voters are Democrats you should report that. Obtaining data about partisan identification of the country is a goal of polling, not something you should assume before hand.
Public policy polling, for example, doesn't weight by party ID. Most of the groups that weight by party ID are not very "powerful" in Silver's model because-surprise, surprise-they tend to give poorer/outlier results. Unless you're saying weighting by demographics to make them reflect the makeup of America is weighting by party ID, which is hogwash.
Edit: Of the "major" organizations that are polling this cycle, all I can find that weights by party ID is Rasmussen, unless PPP and SurveyUSA are outright lying.
Show nested quote +
On November 02 2012 00:43 xDaunt wrote:
This is all bullshit and a misrepresentation of what I have been saying over the past couple months. I have made it very clear what my objection to a majority of the polls has been: most polls are clearly oversampling democrats and reflect a voter turnout akin to 2008 (a +9 democrat advantage) as opposed to what is likely to happen this year (a +1 republican advantage or so). I've seen all of the arguments about why party ID does not matter, and quite frankly, I am not convinced. There's nearly a 1:1 correlation between party ID and one's choice for president. While there are some problems with the party ID metric and its malleability, I find it impossible that polls showing unwarranted +5 to +10 democrat party ID advantages are accurate.
This has been my analysis and my argument, and I have not deviated from it. Could I be wrong? Sure, but I am not expecting it. We'll find it out in 5 days.
On November 01 2012 19:12 paralleluniverse wrote:
+ Show Spoiler +
+ Show Spoiler +
The sheer amount of hackery and hypocrisy coming from xDaunt about polls is absolutely staggering. Before the Denver debate, when Obama was losing, he was relentless in denouncing the polls as wrong and biased. See for example here.
And since then, Romney has gained big, and suddenly he's cherry picking polls as if it proves the doom of Obama, for example here. So there was a liberal conspiracy to make Obama's poll numbers better than they really were, and once Romney started gaining after Denver, suddenly, inexplicably, the conspiracy stopped, despite there being no change in polling methodology?
More like, anything showing Obama winning is biased, and anything showing Romney winning must be the truth. Because, like the rest of the right-wing media, anything contrary with their worldview must be bias. Like Nate Silver giving Obama an 80% chance of winning, climate science, evolution. It's all bias. These cries of bias, from pundits and forum posters who don't know a damn thing about statistics, just underscores the continual and ceaseless anti-intellectualism of the right.
Take for example the attacks on Nate Silver from The National Review, which I responded to earlier by pointing out that the author is clueless about statistics. He hails Real Clear Politics's unweighted average of polls as somehow superior to Silver's. He doesn't know that it's a fact of statistics that weighting by the sample size of polls reduces the standard error, and that Silver does even better because he weights by sample size and the past reliability of the poll. And there's nothing at all "subjective" about this weighting method, as the author claims. Silver isn't weighting anything, his model is, and he takes what his computer spits out. It's the model, not the man.
Then there's Politico quoting Joe Scarborough with an article from another know-nothing, who says that:
This guy doesn't understand probability. There is absolutely nothing paradoxical about a close election race and one candidate having a high chance of winning. Suppose that in a population of 1000, the true state of the race is 510 people voting for Obama, 490 people voting for Romney and that these preferences have held steady for a very long time. Then polls of this population will show a very tight race, but Obama would have a very high chance of winning, because the preference of the population doesn't change much. Closeness does not necessarily imply that the probability of Obama winning is 50.1%. This extreme example isn't even too far from the real world, Obama has a small, but consistent and stubborn lead in the battleground states that matter.
And here's an absolutely moronic tweet from Politico again:
This is the pinnacle of stupidity. No shit Nate Silver is "simply averaging public polls". Nate Silver has been completely transparent in explaining his model. You can read all about it on Wikipedia and the links within. We don't want secret sauce, we want rigorous and sound statistical methodology, and that's exactly what Nate's Silver does. And as Krugman argues, this "secret sauce" statement is possibly motivated by the fact that Nate Silver, and statisticians like him, makes the job of the innumerate pundit obsolete.
If not by analyzing polls, how else would you predict elections? By reading pundits, like the ones who prove to the world that they know absolutely nothing about statistics when they write articles like the ones linked above? Gut feeling, which is pretty much what xDaunt does? And to prefer relying on that, instead of textbook statistical analysis, because the latter shows Obama winning, is not surprising given the anti-intellectualism of the right. What are the chances a right-winger will trust in evidence and math, when they reject climate science and evolution?
What we don't see is right-wing commentators making any sensible criticism of Silver's statistical methodology. Obviously, because as the above article writers have proved to the world, they don't know a damn thing about statistics. They just call him bias because he shows that Obama is winning. In fact, the only valid criticism I've seen in the media is the article from David Brooks who says that Silver's model can't predict events like the leaking of the 47% video, an awful debate performance from Obama, etc. And this is true. That's why Silver has a nowcast and a forecast, and why the forecast isn't a flat horizontal line, because the information up to the current time increases as time goes on.
Of course, it's not just pundits who don't know anything about statistics. There's a lot of posters here too. For example, xDaunt, again, claims that:
But that is not at all surprising. Polls have margins of error. The fact that there's a lot of inconsistency between polls showing Obama winning and Romney winning in Florida just shows that there's a tight race. If the true vote for each candidate is almost 50%, then we would expect that about half the polls show Obama winning and the other half show Romney winning. And the fact that this is what we see is merely indicative of a very close race in Florida. There is nothing amusing, unexpected, or wrong about it.
There's this guy who thinks a poll of 1000 people is OK for a small state, but too small for the country.
This guy demonstrates failure to understand some of the most basic facts of statistics: if the population size is large, a poll of 1000 people is virtually just as accurate for a population of 5 million as it is for a population of 500 million as I've explained here.
And then there's people just making shit up:
And with no supporting evidence.
The fact is that according to Nate Silver, Obama has almost an 80% chance of winning. And the prediction markets put it in the high 60s. To deny this by cherry picking polls (national polls, not even state polls) that show Romney winning, as xDaunt does, is completely dishonest. It's not even valid because an aggregate of polls is a lower variance estimator than picking a few polls where Romney is winning. It's also absolutely hypocritical for xDaunt because he was criticizing polls for exhibiting liberal bias before the race tightened after Denver.
But that doesn't mean that the race is over. A 20% chance of winning is not bad at all, a 20% chance is 1 in 5, it would really be over if it were 1 in 20 (5%) or 1 in 100 (1%). 20% events happen all the time. A 20% chance is equal to the chance that a randomly selected bronze player is zerg (according to SC2Ranks). And if it turns out that Romney does win, it does not in itself prove that Silver was wrong or that I was wrong in believing him, simply because 20% chance events happen *all the time*. To claim otherwise, would be to not understand probability.
Nate Silver publishes the vote share by state along with a margin of error (95% confidence interval). Therefore, theory suggests that we would expect that about 1 in 20 of his predictions are wrong in the sense that they lie outside of his margins of error. If it turns out that he called somewhat more than 1 in 20 states incorrectly, then it would show that Nate Silver is wrong, and that I'm wrong for believing him.
Another reason why Nate Silver could be wrong is if the polls are wrong. But as Drew Linzer explains, there is good evidence to believe that the polls are accurate. Not that xDaunt can use this argument anyway without being a hypocrite, since he is selectively pointing to polls where Romney is winning.
If there's one single reason why I didn't become a right-winger, it would unmistakably be because I hate anti-intellectualism, and the dumb attacks from the right on Silver, on this forum and in the punditry, which only prove that they know nothing about statistics, is exactly why I hate the right.
And since then, Romney has gained big, and suddenly he's cherry picking polls as if it proves the doom of Obama, for example here. So there was a liberal conspiracy to make Obama's poll numbers better than they really were, and once Romney started gaining after Denver, suddenly, inexplicably, the conspiracy stopped, despite there being no change in polling methodology?
More like, anything showing Obama winning is biased, and anything showing Romney winning must be the truth. Because, like the rest of the right-wing media, anything contrary with their worldview must be bias. Like Nate Silver giving Obama an 80% chance of winning, climate science, evolution. It's all bias. These cries of bias, from pundits and forum posters who don't know a damn thing about statistics, just underscores the continual and ceaseless anti-intellectualism of the right.
Take for example the attacks on Nate Silver from The National Review, which I responded to earlier by pointing out that the author is clueless about statistics. He hails Real Clear Politics's unweighted average of polls as somehow superior to Silver's. He doesn't know that it's a fact of statistics that weighting by the sample size of polls reduces the standard error, and that Silver does even better because he weights by sample size and the past reliability of the poll. And there's nothing at all "subjective" about this weighting method, as the author claims. Silver isn't weighting anything, his model is, and he takes what his computer spits out. It's the model, not the man.
Then there's Politico quoting Joe Scarborough with an article from another know-nothing, who says that:
"Nate Silver says this is a 73.6 percent chance that the president is going to win? Nobody in that campaign thinks they have a 73 percent chance — they think they have a 50.1 percent chance of winning. And you talk to the Romney people, it's the same thing," Scarborough said. "Both sides understand that it is close, and it could go either way. And anybody that thinks that this race is anything but a tossup right now is such an ideologue, they should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they're jokes."
This guy doesn't understand probability. There is absolutely nothing paradoxical about a close election race and one candidate having a high chance of winning. Suppose that in a population of 1000, the true state of the race is 510 people voting for Obama, 490 people voting for Romney and that these preferences have held steady for a very long time. Then polls of this population will show a very tight race, but Obama would have a very high chance of winning, because the preference of the population doesn't change much. Closeness does not necessarily imply that the probability of Obama winning is 50.1%. This extreme example isn't even too far from the real world, Obama has a small, but consistent and stubborn lead in the battleground states that matter.
And here's an absolutely moronic tweet from Politico again:
Avert your gaze, liberals: Nate Silver admits he's simply averaging public polls and there is no secret sauce
This is the pinnacle of stupidity. No shit Nate Silver is "simply averaging public polls". Nate Silver has been completely transparent in explaining his model. You can read all about it on Wikipedia and the links within. We don't want secret sauce, we want rigorous and sound statistical methodology, and that's exactly what Nate's Silver does. And as Krugman argues, this "secret sauce" statement is possibly motivated by the fact that Nate Silver, and statisticians like him, makes the job of the innumerate pundit obsolete.
If not by analyzing polls, how else would you predict elections? By reading pundits, like the ones who prove to the world that they know absolutely nothing about statistics when they write articles like the ones linked above? Gut feeling, which is pretty much what xDaunt does? And to prefer relying on that, instead of textbook statistical analysis, because the latter shows Obama winning, is not surprising given the anti-intellectualism of the right. What are the chances a right-winger will trust in evidence and math, when they reject climate science and evolution?
What we don't see is right-wing commentators making any sensible criticism of Silver's statistical methodology. Obviously, because as the above article writers have proved to the world, they don't know a damn thing about statistics. They just call him bias because he shows that Obama is winning. In fact, the only valid criticism I've seen in the media is the article from David Brooks who says that Silver's model can't predict events like the leaking of the 47% video, an awful debate performance from Obama, etc. And this is true. That's why Silver has a nowcast and a forecast, and why the forecast isn't a flat horizontal line, because the information up to the current time increases as time goes on.
Of course, it's not just pundits who don't know anything about statistics. There's a lot of posters here too. For example, xDaunt, again, claims that:
On October 31 2012 23:56 xDaunt wrote:
The disconnect and inconsistency between many of the polls is very amusing. Someone's going to write a book on this when it's all done.
Show nested quote +
On October 31 2012 23:54 Risen wrote:
Ehh, I'm pretty sure Florida is going to Romney lol.
On October 31 2012 22:38 nevermindthebollocks wrote:
I admit it is always hard for me to image Romney getting more than 40% of the national vote (or even 20%) but I think this shows the key big swing states are Obama's"
http://www.cbsnews.com/8301-250_162-57542715/poll-obama-holds-small-ohio-edge-fla-va-tight/?tag=categoryDoorLead;catDoorHero
Mr. Obama now leads Romney 50 percent to 45 percent among likely voters in Ohio - exactly where the race stood on Oct. 22. His lead in Florida, however, has shrunk from nine points in September to just one point in the new survey, which shows Mr. Obama with 48 percent support and Romney with 47 percent. The president's lead in Virginia has shrunk from five points in early October to two points in the new survey, which shows him with a 49 percent to 47 percent advantage.
I have a feeling there's still a chance for North Carolina too and the election will be all but over before the polls even close in Ohio.
I admit it is always hard for me to image Romney getting more than 40% of the national vote (or even 20%) but I think this shows the key big swing states are Obama's"
http://www.cbsnews.com/8301-250_162-57542715/poll-obama-holds-small-ohio-edge-fla-va-tight/?tag=categoryDoorLead;catDoorHero
Mr. Obama now leads Romney 50 percent to 45 percent among likely voters in Ohio - exactly where the race stood on Oct. 22. His lead in Florida, however, has shrunk from nine points in September to just one point in the new survey, which shows Mr. Obama with 48 percent support and Romney with 47 percent. The president's lead in Virginia has shrunk from five points in early October to two points in the new survey, which shows him with a 49 percent to 47 percent advantage.
I have a feeling there's still a chance for North Carolina too and the election will be all but over before the polls even close in Ohio.
Ehh, I'm pretty sure Florida is going to Romney lol.
The disconnect and inconsistency between many of the polls is very amusing. Someone's going to write a book on this when it's all done.
But that is not at all surprising. Polls have margins of error. The fact that there's a lot of inconsistency between polls showing Obama winning and Romney winning in Florida just shows that there's a tight race. If the true vote for each candidate is almost 50%, then we would expect that about half the polls show Obama winning and the other half show Romney winning. And the fact that this is what we see is merely indicative of a very close race in Florida. There is nothing amusing, unexpected, or wrong about it.
There's this guy who thinks a poll of 1000 people is OK for a small state, but too small for the country.
On September 12 2012 01:53 radiatoren wrote:
However, ~1000 people are too small a sample to carry any significance in itself for a country with 315 million inhabitants or even only counting swing states of about 76 millions.
[...]
In other words: The poll is invalid from the get go due to too few participants. Had it been for a single state, like North Carolina, 1000 would be a decent poll, but that is not the case here.
However, ~1000 people are too small a sample to carry any significance in itself for a country with 315 million inhabitants or even only counting swing states of about 76 millions.
[...]
In other words: The poll is invalid from the get go due to too few participants. Had it been for a single state, like North Carolina, 1000 would be a decent poll, but that is not the case here.
This guy demonstrates failure to understand some of the most basic facts of statistics: if the population size is large, a poll of 1000 people is virtually just as accurate for a population of 5 million as it is for a population of 500 million as I've explained here.
And then there's people just making shit up:
On November 01 2012 00:35 Recognizable wrote:
It's the same every election. I believe some mathmatician once proved that polls didn't do any better than random chance.
It's the same every election. I believe some mathmatician once proved that polls didn't do any better than random chance.
And with no supporting evidence.
The fact is that according to Nate Silver, Obama has almost an 80% chance of winning. And the prediction markets put it in the high 60s. To deny this by cherry picking polls (national polls, not even state polls) that show Romney winning, as xDaunt does, is completely dishonest. It's not even valid because an aggregate of polls is a lower variance estimator than picking a few polls where Romney is winning. It's also absolutely hypocritical for xDaunt because he was criticizing polls for exhibiting liberal bias before the race tightened after Denver.
But that doesn't mean that the race is over. A 20% chance of winning is not bad at all, a 20% chance is 1 in 5, it would really be over if it were 1 in 20 (5%) or 1 in 100 (1%). 20% events happen all the time. A 20% chance is equal to the chance that a randomly selected bronze player is zerg (according to SC2Ranks). And if it turns out that Romney does win, it does not in itself prove that Silver was wrong or that I was wrong in believing him, simply because 20% chance events happen *all the time*. To claim otherwise, would be to not understand probability.
Nate Silver publishes the vote share by state along with a margin of error (95% confidence interval). Therefore, theory suggests that we would expect that about 1 in 20 of his predictions are wrong in the sense that they lie outside of his margins of error. If it turns out that he called somewhat more than 1 in 20 states incorrectly, then it would show that Nate Silver is wrong, and that I'm wrong for believing him.
Another reason why Nate Silver could be wrong is if the polls are wrong. But as Drew Linzer explains, there is good evidence to believe that the polls are accurate. Not that xDaunt can use this argument anyway without being a hypocrite, since he is selectively pointing to polls where Romney is winning.
If there's one single reason why I didn't become a right-winger, it would unmistakably be because I hate anti-intellectualism, and the dumb attacks from the right on Silver, on this forum and in the punditry, which only prove that they know nothing about statistics, is exactly why I hate the right.
This is all bullshit and a misrepresentation of what I have been saying over the past couple months. I have made it very clear what my objection to a majority of the polls has been: most polls are clearly oversampling democrats and reflect a voter turnout akin to 2008 (a +9 democrat advantage) as opposed to what is likely to happen this year (a +1 republican advantage or so). I've seen all of the arguments about why party ID does not matter, and quite frankly, I am not convinced. There's nearly a 1:1 correlation between party ID and one's choice for president. While there are some problems with the party ID metric and its malleability, I find it impossible that polls showing unwarranted +5 to +10 democrat party ID advantages are accurate.
This has been my analysis and my argument, and I have not deviated from it. Could I be wrong? Sure, but I am not expecting it. We'll find it out in 5 days.
Why should pollsters adjust their results to fit some arbitrary assumption of turnout when their standard methodology is showing a different result? They don't set out to sample X% democrats, X% republicans, and X% independents-that's silly and means your poll is totally useless. If 40% of the respondents that are likely voters are Democrats you should report that. Obtaining data about partisan identification of the country is a goal of polling, not something you should assume before hand.
Public policy polling, for example, doesn't weight by party ID. Most of the groups that weight by party ID are not very "powerful" in Silver's model because-surprise, surprise-they tend to give poorer/outlier results. Unless you're saying weighting by demographics to make them reflect the makeup of America is weighting by party ID, which is hogwash.
Edit: Of the "major" organizations that are polling this cycle, all I can find that weights by party ID is Rasmussen, unless PPP and SurveyUSA are outright lying.
I don't have the answers for how to fix the polling assuming that there is a problem. You can't just arbitrarily re-weight them by party-ID.
As far as I know, only Rasmussen accounts for party-ID in his models, and he has probably been the most accurate pollster since 2004.
Again, we'll know in 5 days who is right and who is wrong.