THE BLOG

Can We Still Trust Polls?

07/08/2013 11:27 EDT | Updated 09/07/2013 05:12 EDT

We have all heard of that Jurassic era when gigantic and fierce vertebrate creatures were ruling the world. The laws of evolution being what they are, no living soul has ever seen dinosaurs. They nonetheless remain a scientific curiosity, creatures of our imagination, starring in the children's cartoons airing on Saturday mornings.

A look at the recent history of polling suggest that margins of error are about to share the fate of those Brachiosauridae and their contemporaries. It is becoming more and more likely that the next generation of Canadians will have to refer to History books to learn about polls with representative samples. Or perhaps will they encounter such a notion through some vintage "Mad-Men like" television show depicting the beginnings of the 21st century...

In a few decades, the phrase " This survey carries a margin of error of X%, 19 times out of 20", will probably sound as anachronistic as ancient latin. As a matter of fact, we, ourselves, have seen or heard less and less of those words lately, even though the whole country has been the theatre of an impressive succession of provincial and federal electoral campaigns and partisan leadership races, meaning very busy times for pollsters.

Originally, polls were designed to give us a picture, a snapshot, of a people's opinions, of its views on specific topics, of its aspirations and concerns. In an era where confidence intervals, margins of error and statistical power -- as boring as these terms may sound -- seem to be a luxury and not a moral requirement for someone who advances numbers, we are entitled to wonder how accurate these pictures, weekly delivered to the public and abundantly discussed by political commentators, are.

That shift is not incomprehensible. Continuous time media and the liquidity with which information circulates certainly puts a lot of pressure on pollster to provide surveys at an unprecedented frequency. This transformation is costly in terms of resources and time and leaves less scope for methodological scrutiny. Moreover, as the use of landlines and of other traditional means of communication with respondents are steadily declining, the challenges of proceeding to rigorous polling are expanding (as the recent electoral campaigns in Alberta, in Quebec and in British Columbia have taught us).

As technologies expand and unveil statistical, mathematical and communicational prowess that we would not have dreamed of a decade ago, that we stand by lower statistical standards is a relatively surprising outcome.

Or is it?

With the decline of the margin of error, we have witnessed the burial of the long form census and seen the public funding for scientific and academic research been reduced at all levels of government -- for convincing evidence, one only has to take a look at the metamorphose of National Research Council or at the cuts to the major public federal and provincial research funds. Could the golden days of reliable publicly disclosed data seem be coming to an end, despite the progress of science itself.?

Polls, representative ones, play an important role in reinforcing our democracies by informing both politicians and voters of the ebb and flow of important issues. For this to remain, it is time to ask if the shift in concerns from accuracy to rapidity is serving us best. And -- as pollsters are highly sensitive to the public's demands -- if our own thirst for such frequent investigation of the public opinion is making us better informed citizens.

In a world where "more" has become a substitute for "best", what future is there for numbers, reliable ones?