They say numbers never lie, but as a Guardian article stated earlier in the year, it’s becoming apparent that the ability of statistics to accurately represent the world is declining. After all, in the last two years, the world has been left reeling from a series of high-profile political shocks that have left many pollsters and analysts in deep disrepute.
Political polling was, and still is, at a crisis point. But as they say, one man’s mistake is another man’s opportunity, and with New Zealand set to go to the polls in September, pollsters here are hoping they won’t make the same mistakes as their overseas counterparts. At least, that’s the case over at Newshub, which has announced its new and improved polling method in the run-up to the 2017 general election.
For the first time ever, Newshub and Reid Research will conduct 25 percent of its polling via the internet. The remaining 75 percent of polling will continue to be collected via landline phone calls, with its sampling size of 1000 respondents and its margin of error of 3.1 percent remaining unchanged. The addition of internet polling—aided by Trace Research and its director Andrew Zhu—will aim to enhance access to 18-35-year-olds, as well as better reflect the declining use of landlines in New Zealand.
“We just thought that polling was too important to stand still and keep doing things the way we were always doing it,” says Newshub political editor Patrick Gower, who was heavily involved in the shaping of the new polling system. “We’re now using the best of the old with our accurate landline polling, but we’ve been willing to move and take the jump and use the internet to supplement that.”
“We were really satisfied with the way 1000 landlines were going, but the people who actually do the phone calls were saying they were having difficulty getting to 18-35s. They were having trouble finding them, and it would take them a long time just ringing and ringing to get the statistics. As part of that, they can now concentrate on finding other people and we can get the 18-35s over the internet, which is the way to reach them,” he says.
Mixing old and new
Gower says the decision to allow 25 percent of the results to come from internet polling was based on information that showed that around the same percentage of households no longer owned a landline. And while in today’s web-dominant age, 25 percent seems a whole lot smaller than one would expect, Gower acknowledges that the continuing decrease in landline usage means the number will remain up for review.
“[The number of households without landlines] will be higher by the time the year is out, maybe 33-36 percent, but we decided that since 25 percent of households don’t have landlines, that was a pretty good number for us to go on,” says Gower.
“Over time, if we want to increase the number of people we get over the internet we can … So we’re not ruling anything out in terms of using it more and more. That’s one of the exciting bits of it. Unlike other political polling groups in New Zealand, mainly TVNZ and the political parties themselves who poll privately, we’re prepared to take the jump and modernise our polling and I think that will make us more accurate than all of them. And in the end, that’s our job because this information is so important and so influential that you can’t just sit on your hands and keep doing it the same old way.”
“Polling was too important to stand still and keep doing things the way we were always doing it.”
When asked if other polling methods were considered, such as calling people via mobile phones, Gower says that while the option was considered, the method turned out to be too costly and time-consuming to execute.
“You don’t have a White Pages or register of mobile phone numbers. So to call a mobile, you have to guess the number,” he says. “It’s flying blind going through to mobile phone numbers and there’s a huge cost to that as well. I wanted to do it and we investigated it, but it was actually just too hard to do.”
“But the other way to reach people through their phones is through the internet. You’re not ringing it, but you’re getting to their phones, right? So that’s what we decided to do. But until calling mobiles is cheaper or we’re able to get some idea of who’s got what number or what numbers are actually in use, it just remains too difficult.”
The power rating
In another first for Newshub polling, the media outlet has also introduced a feature called the Power Rating which will show how voters believe New Zealand’s two potential leaders, Bill English and Andrew Little, are performing head-to-head.
While ‘Preferred Prime Minister’ is the main measure used in New Zealand (and continues to be used by Newshub), the poll is based on an unprompted question, whereas the Power Rating asks respondents to rank how the two main leaders are performing overall.
“With preferred PM, people are asked, ‘Who would you like to see as PM?’ And they can name Bill English, Andrew Little, Jacinda Ardern, Mickey Mouse, Mike McRoberts, whoever they want. But the Power Rating is a different question altogether,” explains Gower.
“We ask people how Bill English is performing, and we ask people how Andrew Little is performing. We give them a scale from very poor to very well, or don’t know, or neither, and people answer on that scale. So that gives us an idea of how people actually think the leaders are performing. It’s a very different kind of question and gives us a very different sort of data set.”
“Effectively, it’s a net rating. We take who thinks they’re performing good and then we take away who thinks they’re performing bad. That gives us a number which is actually really simple to understand. If it’s positive, they’re doing well. If it’s negative, they’re doing poorly. So in this latest one, Bill English is +35. He’s well and truly positive. People think he’s doing a good job. Andrew Little is -1. He’s just below 0. Basically, half the people think he’s doing a good job, half the people think he’s doing a bad job.”
A two-horse race?
While the Power Rating adds another dimension to the political discourse running up to election day, the danger remains that such polling sets up a two-horse race, a dynamic that may make sense in a presidential system, but perhaps less so in New Zealand’s multi-party framework.
“The reality is, even though it is a mixed system, only two people can be PM and it’s either going to be Little or English. So in that sense, it is a two-horse race,” says Gower.
“But you’re quite right that the main thing is to look at how all the other parties are going. And we do that, we probably do that better than anyone in terms of giving the picture of them altogether. [For example], we have the Labour and Greens rankings, but also the Labour and Greens vote put together.”
“That’s the responsibility of me to use the data to show that it’s more than just the leaders [and]to explain the full data set. That’s a pretty big job when you’ve got eight to ten different political movements ranging on this poll from 47 percent of the population to 0.4 percent for some of the minor parties, or even 0.0 percent in the case of Hone Harawira’s Mana Party. So that’s what MMP’s dealt us with and we’ve got a duty to reflect that.”
The point of polling
Polling, in all its statistically interesting and newsworthy ways, also strikes the danger of shifting discussion away from the ‘real issues’ with its undue influence on opinion and media. In fact, National and New Zealand First argued back in 1999 that opinion polls had negatively influenced their results in the general election. The following year, New Zealand First leader Winston Peters introduced a bill to ban opinion polls within 28 days of an election. He referred to international research which showed that polls had a substantial ‘bandwagon effect’ and criticised the media’s tendency to use polls to create ‘horse-race’ coverage and obscure party policy and debate. National also supported the Bill due to its fear that centrist voters would simply cast their party vote on the basis of whoever of the two major parties was ahead in the polls, and the concern that Labour’s significant margin in the polls may have lead voters to see the election as a foregone conclusion (a phenomenon that, ironically, has been reversed in recent years).
So, in a modern political climate dominated by celebritisation, personalisation, gossip and conjecture, all of which supposedly obscure the politics at heart, the question remains: what’s the point of knowing how our parties and leaders do anyway?
“[People] need to know everything. They need to know how their parties are going, how their leaders are going, and how the main policies are going, and we ask about those [policies]as well,” says Gower. “We asked questions around National’s Superannuation decision, we asked about how they think the government is controlling the housing market and we asked about immigration this time around as well.”
When we do political reporting, it’s a patchwork quilt, and polling is just one square of that.
After all, while horse race coverage has serious implications for how citizens think and act, and governments are becoming more apprehensive about the effects of publicising pre-election poll results (in most countries, exit polls are only able to be published after all official polls have closed), people who are exposed to information on levels of political support are also encouraged to think about the reasons for and against the position of the mass collective.  Polls are also a major way in which public opinion makes itself felt, and acts as a counterweight to other areas of influence, such as the mass media and politicians themselves. 
“In the end, my belief is that as a journalist, we’re here to inform the public,” says Gower. “Now, that’s not just through polling. When we do political reporting, it’s a patchwork quilt, and polling is just one square of that. Policy is another square of that. Following the politics within the parties is another square of that. Seeing what the public thinks and trying to reflect their concerns is another square. So polls are just one way of informing the public how politics is working. It’s not the only way, it shouldn’t be the only way, and it needs to be done in a way that doesn’t take away from all those other concerns.”
“Everyone raises these concerns and I’ve got the same ones. If politics is just a horse race and it’s just polling, I’d be out of it. I’d rather go work for the TAB or Trackside. But thankfully, politics is about a whole lot more interesting things and polling adds to that. It shouldn’t take away from it.”
A matter of trust
But with blow after blow for the polling community, an increasing number of people have come to view the practice with a hefty air of both indifference and scepticism.
In 2015, David Cameron’s Conservatives swept to an unexpected victory in the UK general election after weeks of polling groomed the public to believe a nail-bitingly close outcome was in the works. The following year saw the British public served with another shock blow to their expectations when Leave trumped Remain in a referendum to decide whether Britain should leave the EU, a result not even those that campaigned for the outcome had come to expect. Then in November, the US (and the world), were treated to the biggest blow yet when a Trump defied the odds to win the presidency over Hillary Clinton.
“If people don’t have confidence in what you’re doing, there’s no point. So if people don’t have confidence in polling, what’s the point in doing it? They’re not going to listen,” says Gower. “That’s why we wanted to go back and look at the methodology and now we have something we’re really satisfied with that we can stand behind when people do have questions.”
Gower, who was in the US last year covering the election for Newshub, also notes that much of the distrust also stems from “reckless” analysis by those in the media. After all, Nate Silver, renowned numbers geek and founder of FiveThirtyEight, wrote in the aftermath of the election result that, “There was a modest polling error … but even a modest error was enough to provide for plenty of paths to victory for Trump. We think people should have been better prepared for it. There was widespread complacency about Clinton’s chances in a way that wasn’t justified by a careful analysis of the data and the uncertainties surrounding it.”
Gower sees Trump’s victory as the final wake-up call for those in the fourth estate, saying that “It’s not so much the data that they were getting, but some of the analysis the media were doing off the data.”
“That is a cause of professional reflection. They’ve made these mistakes, and one of the lessons of that is we get the best data and then we interpret the data. We don’t go beyond it, extrapolate it, take it further than it needs to go because that’s when you get a US situation,” he says.
“The interpretation of data by the media is crucial. You can have this methodology that we’re using and have it be bang on and perfect, but I could be too loose with the way I analyse and present that data, and all that hard work can be undone by that. So in the end, it comes down to me and the other people who present it.”
Luckily, in New Zealand, things are a little less dire, and with the country’s more straightforward MMP system (no states, no electoral colleges, no long-winded process), polling can be done in a much simpler way. And with the first Newshub poll released earlier in the month, Gower says the results are telling.
“Despite all the noise around the superannuation rise, National is on the right side of that argument. People are, by a majority, pretty happy with it. I think the National Party probably knew that because they would’ve been doing their own polling ahead of it. So they were always confident they could come out and win it,” he says.
“We also see Jacinda Ardern, for instance (who’s polling higher than Little). The polls have helped us identify a potential political phenomenon.”
“We’re not guessing anymore about how it’s going down, we’re using data with our analysis. It’s not just me saying ‘I think’, I’m able to use the data to actually add that to my analysis.”
“But like any poll in election year, they’re going to go up and down and we’re going to see a lot more different results before September.”
 Moy, P., & Rinke, E. M. (2012). Attitudinal and Behavioural Consequences of Published Opinion Polls. In C. Holtz-Bacha & J. Stroembaeck (Eds.), Opinion Polls and the Media: Reflecting and Shaping Public Opinion (pp. 225-245). Basingstoke: Palgrave Macmillan.
 Bradburn, N. M., & Sudman, S. (1988). Polls & surveys: Understanding what they tell us. San Francisco: Jossey-Bass.