AE Blog

Traditional public opinion polling has evolved over the years, but since the latter part of the 20th Century how those surveys are conducted hasn't changed all that much. To completely oversimplify, the process is pretty straightforward: get a representative sample of respondents on the telephone, design an instrument that asks good questions, weight the sample when appropriate and run the crosstabs.

While all of those steps are important to ensure accuracy, pollsters tightly hold their secret formulas for weighting samples. It's the industry's version of KFC's 13 original spices. Virtually all pollsters, however, when thinking about weighting do so by weighting a combination of demographic information or partisanship.

In the last few years changes in technology have complicated matters a bit for pollsters. The rise of the use of cell phones and caller ID have skewed the kinds of people that answer survey calls and agree to participate in the research. There has been a tremendous amount of debate in the pollster world, for example, about cell phones. For a long time, cell phones were excluded from most polling research. While many (if not most) pollsters now try to incorporate cell phone numbers into their research, the response rate is often different between landline and cell phone respondents.

Generally, pollsters know that most "cell phone only" voters (that is, voters without a traditional landline) are younger, and the assumption is that by weighting the responses of young voters reached by landline, one can model the responses of those (largely young people) without a traditional land line telephone.

A new study by the Pew Research Center for the People and the Press points out that simply weighting for demographics does not, however, provide an accurate representation of cell phone only voters' attitudes and opinions.

From the Pew Report:

"...cell-only young people are considerably less likely than young people reached by landline to identify with or lean to the Republican Party, and even less likely to say they support John McCain. Among landline respondents under age 30, there is an 18-point gap in party identification - 54% identify or lean Democratic while 36% are Republican. Among the cell-only respondents under age 30, there is a 34-point gap - 62% are Democrats, 28% Republican. The difference among registered voters on the horserace is similar: 39% of registered voters under 30 reached by landline favor McCain, compared with just 27% of cell-only respondents. Obama is backed by 52% of landline respondents under 30, compared with 62% of the cell-only."

What is missed by substituting landline young voters with cell phone only young voters is that there are distinct socioeconomic and social values that drive the decision to live "landline free." These are lifestyle choices, choices about where young people allocate their personal financial resources, and those choices are made by processing a set of information through a set of social values. Said another way, one's worldview (and one's economic situation) influences the decision as to whether or not to have a landline telephone.

The same kinds of social values that determine whether or not one subscribes to landline telephone service - social values our company tracks like Aversion to Complexity, Technology Anxiety, Tried and True - impact their decisions as voters. It makes sense that younger voters that prefer traditional and established technologies like land line telephones would prefer a traditional establishment candidate like John McCain. Importantly for 2008 (and beyond), pollsters that treat landline and cell phone only voters the same are making less than accurate predictions.

This is far from a deep analysis as to why younger people choose landline telephones, but it should serve as a reminder for those of us working to understand voter behavior - the decisions we make as to the kinds of technologies we adopt are processed through a set of social values that also process information about candidates, voting and issues. As technology continues to change the way pollsters communicate with voters, we'll need to think carefully about the tactics we use to reach voters and the kinds of weighting we use to make accurate estimates. The original 13 herbs and spices just might not cut it anymore.