[...]
As the British journal
Nature so aptly argued
in an October 2016 article, telephone polling is so flawed that many governments have stopped using it. Consider the following:
- Fewer than 10 per cent of respondents will agree to participate in a telephone survey. This creates what scientists call ‘opt-in bias’.
- Low opt-in rates force pollsters to over-sample minority demographics. Since opt-in bias tells us they are likely outliers to begin with, this leads to a very distorted view of public opinion.
- Merely posing a question to respondents creates question bias.
- Small sample sizes are not representative. The government’s rolling survey will sample 500 people a week. That is not enough to tell us what people across the country — from every region, language and socio-economic group — think. Not even close.
- Rolling or ‘windowed’ surveys are controversial because they are trying to mimic longitudinal studies which survey the same people over a longer period of time. With new technology, there is no need for this mimicry. Real longitudinal studies are already possible, for a fraction of the cost the government paid for this substitute method.
The 2015 Canadian federal election provides a good illustration of how telephone polls can lead to wrong insights, and cause politicians and policymakers to make the wrong decisions.
Polls released a week after the viral release of a photo of the body of three-year-old refugee Alan Kurdi washed ashore in Turkey showed a drop in Conservative support. The media immediately assumed that the drop in support was due to Kurdi’s death; “social media listening” on Twitter seemed to confirm this. Tweets about Kurdish and Syrian refugees increased 300-fold, as did criticism of the Harper government, which had been slow to let refugees into the country.
What Harper, the media and just about everyone else failed to recognize was that the fall in Conservative support had
nothing to do with Alan Kurdi’s death. The day before Kurdi drowned, Statistics Canada published a report saying that Canada had entered a technical recession. It was this report — not the drowning — that caused the fall in Conservative support. People who voted Harper did so because they believed him to be a good steward of the economy. They were not concerned about his policies on refugees.[...]
