Why I Despise Public Polling
I haven't posted much in the past year, but recent developments have gotten my attention.
I have long regarded public polling with more than a measure of disdain. My distaste is primarily with the flawed sampling and methodology utilized by the polling organizations – antiquated screens, RDD, self-identified partisanship and likelihood of voting with little or no voter file validation - and the trumpeting of facile analysis without fully releasing the underlying data.
Perhaps the worst part of public polling is the complete inability of the talking heads from the polling world to avoid hyperbole or to speak in metaphor-free English, compounding the weakness of their data and diluting the quality of the public discourse.
The Pace University polling efforts struck me as a potentially bright light, with greater disclosure of data, a better sampling methodology and comprehensive and voluminous analysis. But alas, they are gone too soon.
So what triggers my comments today? The release of yet more crappy data that media types will naturally regurgitate for weeks to come.
In the past ten days there have been two public polls (Siena in NY and Quinnipiac in NJ) that have shown Giuliani with a lead over Obama in both states and competitive with Clinton in NY and leading her in NJ.
The results are not surprising, given the relatively high name ID and voter familiarity with the Giuliani and Clinton names and brands, but what do they mean in practical terms for the various candidates?
Well, practically speaking they shouldn't mean anything, but the results could have significant impact on the GOP nominating fight. The findings echo and validate the strategic foundation of the Giuliani candidacy: that he can place New York and New Jersey in play. Part of his donor pitch is that he claims he is the ONLY Republican candidate who can credibly engage in Democratic territory.
If a Republican were to win NY and NJ, the Democrats would be hard pressed to replace those 46 Electoral Votes. The Democrat would have to win Colorado, Florida AND Ohio, and at least two out of three among Iowa, Nevada and New Mexico (while holding New Hampshire and Wisconsin) - an extremely tall order.
Many of those of us who know Giuliani (and yes, we know what familiarity breeds) cannot fathom how the GOP could nominate him, but we also know that they want to win, and know how to win. Electability is a major factor for their activists.
It's a major factor for Democrats as well (perhaps even more so). Other recent public polling (notably the recent Gallup poll) has shown Clinton's favorables sliding (broad-based, they called it), and Obama and Edwards generally frozen in place. The only real movement or curiousity on the Democratic side seems to be with Vice President Gore, who has continued to express a non-Shermanesque disinterest in running.
If Clinton continues to slide, she can’t win. If Obama can't win in the northeast in November, he can’t win, and Edwards remains frozen in a distant third (because someone with his name ID should be doing better) he can’t win the nomination, much less in November. So who emerges? Dodd? Gore?
But Gore isn't running. Why are public pollsters even including a candidate who is not running? How is that even remotely scientific? Are they doing it out of prurience or some other form of curiosity?
The the flawed and potentially false question still gets raised by this "research" - how can a Democrat win if they lose the northeast?
Now I don't think for a second that will end up being the case, but that is potentially the way it will be covered - and sometimes that is all that matters. Maybe that is what the pollsters want - a competitive northeast so that THEY can be relevant.
Most pollsters would say that they only measure opinion. My point is that they should base that measurement on votes or more scientific methods. Since they don't and the media will slavishly cover this so-called public data, all that these polls do is skew and eventually cement public opinion on false data points. The self-fulfilling nature of these polls undercuts the public discourse and public interests the polling institutes purport to represent and serve.
The proof is found in the real source of my pique, the responses to Q2 of the Q-Poll - which show that a majority of respondents are not firm in their support and "might change their minds" about who they are supporting for the Democratic nomination. If that’s the case, then why release the numbers at all? And why no mention of this softness or uncertainty in the press release of the poll?
Pollsters defend this kind of early/non-predictive polling and their craft generally by saying "it is only a snapshot of public opinion at this particular time." The problem with that “defense” is that their data show (at least for the Q-poll) it is a "snapshot" of uncertainty. All it tells us is something that we already know - the race is unformed and volatile, but the conclusions they push are entirely different.
We don't need a poll to tell us the race remains undecided, and we certainly don’t need crappy conclusions, particularly if the spin on this poll doesn’t match the reality of the mood of the electorate.
We live in a rapidly changing and increasingly segmented world. Unless and until public polling institutions reform and refine their methods and motivations, they really aren't worth much. The average voter will probably get more substantive value out of a conversation with their barber or bartender than from this so-called "data." I think I am going to get a haircut.
And that's Why I Despise Public Polling.
Post new comment