With Leicester City winning the Premier League, The Chicago Cubs winning the World Series and now Donald Trump winning the US election, it’s been a year of unpredictable results.
Perhaps they should have taken guidance from the animals who tapped, ate and sniffed Trump more than they did Clinton. A Siberian polar bear named Felix, an Indian fish named Chanakya, a Chinese monkey named Geda and a group of puppies all proved a better informant than the polls when predicting the win. So where did it go wrong for the polls?
While it was the media reporting pushing the results of the polls, Datamine founder Paul O’Connor says it was the media itself that potentially influenced the poll results by telling the US population that voting for Trump is non-PC—a statement a quick Google search confirms as headlines pairing Trump with the non-PC vote are listed down the screen, with one simply reading “The Anti-PC Vote”.
One such story, on The Atlantic, discusses Trump’s fight against political correctness with the dialogue from a 22-year-old Trump supporter, who agreed to have his views shared on the condition of anonymity. According to the story, he was concerned the views he expressed might be used to deny him future opportunities.
When The Guardian did a similar story in March, its Trump supporters also chose to remain anonymous, with one referring to themselves as “a closet Trump voter” and another admitted their wife didn’t even know.
Anonymity is a common theme among those who have chosen to express their vote for Trump, a behaviour in line with O’Connor’s belief that poll respondents may not have felt comfortable expressing how they truly felt. He says they would instead tell the poll one thing, and vote for another.
He does, however, add that that bias could yield different results depending on the polling method, giving the example of a face-to-face survey potentially being different to an online or phone survey.
Though the phenomenon is not limited to polls conducted for this year’s presidential election, Research Association New Zealand chief executive Rob Bree says the phrase ‘Trump shy’ was coined this year in reference to a Trump voter who refused to discuss their political beliefs or lied about them.
While this may suggest Trump voters lack confidence in their opinion, it could be seen to show the strength of their support for Trump as it was able to withstand the stream of criticisms towards him in the media.
In 2014, Forbes reported a study in the Journal of Consumer Research called “Social Defaults: Observed Choices Become Choice Defaults”. It noted the conclusion that when people didn’t have a strong opinion about the choices presented to them, they simply mimicked the people around them. It was called the “social default”.
This being the case, and had their opinion not have been strong, it is likely many would have been swayed to vote Clinton, but instead their ticking of the Trump box on the day was a confident one.
TRA head of strategy Colleen Ryan agrees the negative media coverage of Trump would stop people admitting their support for him. “It’s like a dirty secret,” she says. “And the ballot booth gives them anonymity whereas answering polling questions makes them feel exposed.”
However, it’s not a deliberate lie, she says; instead, they answer questions in a way that makes them seem rational and logical even though it’s not a good predictor of future behavior.
She says the polls measure what people think they should say about the rational argument being put forward—“they measure what people think about the candidates, not what they feel”.
Ryan gives the example of the Trump campaign, which understood how to market brand ‘Trump’ to Americans, something polls don’t measure.
It’s this consideration of emotion as a driver of behavior that enabled what she calls “enlightened insight companies” to still be able to predict the ice cream flavour someone will chose, but why polls are unable to correctly choose a winner between two parties. Ryan says enlightened insight companies use methods that reflect how people actually behave, rather than what they say they will do.
And when establishing these methods, O’Connor pointed out that businesses have the luxury to run more experiments and learn from changes in the market, whereas the US presidential election only runs once every four years.
So what are the best prediction methods?
Ryan says none of the methods are working, and instead suggests the use of predictive market techniques because “people are much better at predicting other people’s behaviour than they are at predicting their own”.
In fact, Ryan says polls are “not at all” a successful way of making a prediction.
Bree calls polls one way of making predictions, but at best it’s a science of estimation. He too acknowledges the way people behave differently to how they said they would.
“The reality is a person might state a belief on Monday morning and act out of line with that belief on Monday afternoon.”
Despite this, he says polling in New Zealand is traditionally around 97 to 98 percent accurate, but again adds “it’s not a perfect science”.
O’Connor believes the age of asking people’s thoughts has gone, and says the use of behavioural data driven insights is the key to predicting future outcomes. And while he says survey methods are still valid, those using them need to know how to weight them to get a reliable answer.
All surveys have some biases in them, which he says can come from factors ranging from the survey design, to individual preference to answer a survey. For example, “with a sample size as small as a couple of thousand people or where response rates are as low as 10 percent, it can be difficult to truly represent the population you are trying to predict,” he says.
“The difference between a good and a bad survey comes from the ability to negate the inherent biases.”
O’Connor notes the LA Times poll was the only major survey consistently predicting a Trump win because it was able to adjust for sample errors better than others by using a more complex weighting applied to the results.
He also points out that for a more accurate prediction, the polls for the election should have put a greater focus on key swing states, such as Pennsylvania, because surveying California was less useful as it was likely to vote democrat.
And while these suggestions may help the polls for the next election, O’Connor says that without actually being able to see into the future, there are always going to be instances of incorrect prediction.
“It is not about predicting a future outcome correctly 100 percent of the time, but using data to produce correct predictions far more often than just guess,” he says. “Predictions will sometimes be wrong (think about the weather), but they are typically a lot better than a just a finger in the air.”