Thu 23 Oct 2008
I find Sam Wang's polling analysis at PEC to be smart and informed, and his commentary to be pretty interesting. But his confidence scares me. Wang seems to be doing all the right things. He's making his data and his models freely available. He's keeping his models purposely simple in order to avoid the chance of a mistake. And he's not fiddling with his model as time goes on. All good things.
But… Wang thought his model was right in 2004 until it turned out it wasn't. And people probably didn't think much of the problems in polling methodology until the Dewey / Truman fiasco, and then they did. What don't we know now? I think the data problem is the more dangerous one. As a commenter on PEC points out, polling is a 'black art', and though we gain a lot by doing a meta-analysis, we're still at the mercy of the underlying numbers. Another commenter points out that for Obama, a win is not a win. We want to crush McCain. Doing so sets a completely different political landscape than a narrow victory would.
Anyway, I don't suggest that Wang should be doing anything differently on the statistics. But, and I'm sure he hates the constant comparisons to Nate Silver at 538, where Nate just says that McCain is in trouble, Wang advises that Presidential race is in the bag, and we should forget about it and start focusing on Senate races. A little more temperance would make the commentary much more palatable for me.