I read with great interest the recent Canadian Press story A voter's guide to political polling in this 2015 federal election year, republished in the Huffington Post on March 14, 2015. I also read it with great concern. While it outlined why some individuals think telephone polling is the so-called "gold-standard" of survey research, I think the piece missed some important elements, and doesn't completely deal with the methodological challenges of traditional phone polling and new opportunities associated with online polling.
The piece attributes a move away from phone polling because of rising costs. But regardless of cost, while this method for studying public attitudes and opinions might have once been the "gold standard," it is no longer, because most people on the other end of the phone line don't want to participate. Based on my conversations with colleagues in the industry, rejection and non-completion on phone polls run to the 95 per cent range. This means, of the 10 people a polling firm might contact, more than nine aren't inclined to do the survey.
What's left is a questionable mix of available respondents who often don't properly reflect the diversity and demographic differences of Canadian society. So-called "robo-polling," where a computer makes the call and asks the questions while a respondent punches keys on their phone, is further eroding whatever small group of willing people is left. Based on this, is it fair to say a "gold standard" exists? In the context of today's norms about anonymous communications, telephone polling -- whether via live operator or computer voice is basically spam.
I've been a pollster for 40 years, and I can tell you that with the exception of governments with their enormous resources for projects (think the monthly labour market study), few studies carried out by commercial research firms today come close to achieving true random samples. That said, all respectable firms work very hard to achieve samples that are representative of the many segments of Canadian society -- all the way to the micro-regional level.
Many statisticians don't like so-called "non-random" polls but I'd say equally as many don't understand how reputable firms today must build large panels of people who have agreed in advance to participate in survey research. Like all polling, whether on the phone or online, survey work requires some form of "opt-in." The reality of polling in the 21st century is that people who don't want to be involved can and will say "no." The investment in polling has moved from placing interviewers in call centers to spending money to attract members of the public to participate in the survey program of one or more polling firms.
Sometimes a technique of "sample questions" is used to bring potential members into a panel for the purpose of future research. Many of the 130,000 members of the Angus Reid Forum online panel were first made aware of the panel via sample recruiting questions such as the "do you like Justin Trudeau?" query referred to in the Canadian Press story. At any given time hundreds of these sample questions are placed throughout the web. They are used as recruitment vehicles -- not as measures of the popularity of Trudeau. When we occasionally release polling results about federal leaders and parties, that's not the data we're using!
The story also neglects to address the significant and notable quality differences that exist in the online polling world. Organizations such as U.K.-based YouGov have done very well projecting election results. They have routinely performed better than the telephone polls that have reported during the same election. According to a fivethirtyeight column in the New York Times, in the last U.S. election, four of the seven most accurate pollsters at the time (including my former commercial polling firm) used online polling methods. This pattern has been repeated in dozens of elections in Canada, the U.S. and the U.K.
Not all polling organizations are involved in publicly released surveys for "marketing" purposes. In the US, the highly regarded Pew Research Center is carrying out an extensive program of releasing no-cost, publically accessible research to provide policy makers, researchers, reporters, and interested individuals with an unbiased reading of the public mood. Here in Canada, the non-for-profit institute that bears my name (the Angus Reid Institute) is trying to do the same without support from commercial interests.
A more important element is the innovation that Internet polling enables to explore subjects once very difficult, if not impossible, to cover via telephone surveys. Recently, among other topics, the Angus Reid Institute looked at the experiences of Canadians in such sensitive areas such as sexual harassment at work, bullying in school, and the treatment of loved ones in palliative care. These are subjects respondents might not have felt comfortable talking about in a phone poll. The online experience can provide a greater sense of anonymity and safety.
Further, online polling provides opportunities to put video and photos in front of people for truer reactions. Polling that verbally seeks opinions about the wearing of the hijab or niqab in the Canadian workplace are arguably less effective than those that present respondents with images of exactly what they're being asked about.
Finally, polling today is about much more than calling election horse races. A huge array of issues from hockey violence to experiences with the health care system cries out to be understood through the lens of the kind of public opinion research that isn't limited to an impersonal voice on the other end of a phone, and isn't dependent on a respondent's desire or willingness to answer questions when an uninvited phone call comes, but on their own time.