The stock imagery on the release might show people laughing with magazines, but there probably aren't too many smiles in the print industry after several unexpected fieldwork issues affected the quality of readership data for Nielsen's newly pimped out Consumer and Media Insights readership survey.
Nielsen's managing director Stuart Jamieson couldn't be reached for comment, but the main issues impacting on the Q1 2011 survey, according to a letter sent to publishers and agencies (see below for full text), were the suspension of fieldwork in Christchurch and surrounding areas following the earthquake that resulted in a shortfall of interviews, best practice methods not being followed by all interviewers that led to an exclusion of some data and staff shortages in Auckland.
"Nielsen and the survey auditor, Professor Peter Danaher believe that the unprecedented effect of these fieldwork issues has had a negative impact on readership results for the quarter," Jamieson said in the letter. "Not all issues with Q1 fieldwork can be resolved immediately. The survey auditor has requested that missing interviews in both Christchurch and other Regions must be replaced. We will be working with the Auditor with a managed programme to replace surveys over the remaining survey periods of 2011."
After the Print Media Industry Research Review Group signed up Nielsen as its preferred research partner after a big pitch, the new survey was released with great fanfare a few months back and offered plenty of promise for the print sector. Fusing Statistics New Zealand's Household Expenditure data to the survey meant publishers would be able to approach advertisers with, for example, specific data about how much readers or demographic groups spent in certain categories. It's very valuable and helpful information, and while this data is thought to be robust and unaffected by the fieldwork issues, it's disappointing for all concerned that its accuracy may be called into question.
Some might wonder why this data would even be released when doubts will inevitably surface about its accuracy. But, given the excitement over the new fusion, it needed to get that important piece of information out to the market and releasing the weighted data was seen as a slightly better option than not releasing any data at all; the lesser of two evils.
The usual flurry of press releases from the magazine publishers trumpeting their various gains hasn't been forthcoming this year, due to a self-imposed moratorium on self-congratulation because of the fieldwork issues. But the biggest winners when comparing Q1 2010 with Q1 2011 are, once again, niche, special interest magazine titles, although some of the mainstream titles recorded what Nielsen refers to as 'statistically significant readership gains'.
Download the magazine comparatives here. Magazine Comparatives Q2 2010 - Q1 2011
In the weekly market, Lucky Break, The Listener and Rugby News all upped their numbers, while the big ones, Woman's Day and Women's Weekly remained fairly static with similar audience share from last year.
Among the monthlys, Australian Women's Weekly, Kia Ora, North & South, Metro, Avenues and Boating NZ all went up, as did Taste, NZ Fishing News, NZ Classic Car and Healthy Food Guide. Skywatch, however, dropped significantly.
The big bi-monthly movers were NZ Geographic, NZ Fishing World, Rip It Up and NZ Life and Leisure.
Food titles had a very good run this time round. Cuisine held firm with 398,000 readers, Healthy Food Guide is the next biggest with 346,000 readers (a 12 percent increase) and Foodtown magazine went from 274,000 to 332,000. Taste went from 165,000 to 217,000 and Dish increased from 108,000 to 134,000.
Homestyle and men's mag M2 were new additions this time round, but English Woman's Weekly, Groove Guide, Better Homes & Gardens, National Geographic and Jasons Motels weren't included.
Not too much front page news for the newspaper biz, however, with only Dairy News and the NZ Herald's entertainment supplement TimeOut heading in the right direction.
The NBR, Dom Post and the Sunday News all had significant drops, but, given the state of the newspaper market overseas, all the papers that didn't have a significant decrease can probably pat themselves on the back.
Download the newspaper comparatives here Newspaper Comparatives Q2 2010 - Q1 2011
I am writing to you with some important information regarding the Nielsen Consumer and Media Insights Q2 2010 to Q1 2011 (April 2010 – March 2011) survey release. This information covers 4 key areas:
1. Major fieldwork issues in Q1, 2011 (Jan-March 2011)
2. The impact on readership estimates
3. Resolution of fieldwork issues
4. Use of Q2 2010 to Q1 2011 CMI data
During the first quarter of 2011 Nielsen has experienced several unexpected fieldwork issues that have affected the quality of readership data collected during that period.
• The Christchurch earthquake on February 22nd resulted in the suspension of fieldwork in Christchurch and surrounding areas. This has resulted in a shortfall of n=136 interviews and shortfalls for some demographic groups.
• Quality control analysis of Q1 fieldwork in other regions identified that best practice methods may not have been followed by all interviewers. We have identified that this was the case with one interviewer and their interviews have also been excluded from the Q1, 2011 sample.
• Auckland fieldwork practices where significantly affected by staff shortages which were compounded by interviewer illness. As a consequence interviewers worked on non-standard days in order to complete the fieldwork. This is also outside best practice guidelines.
• The impact of these issues was amplified by a change in the annual sample structure ( from 10 interviews to 12 interviews per area unit).
Impact on Readership Estimates
• Nielsen and the survey auditor, Professor Peter Danaher believe that the unprecedented effect of these fieldwork issues has had a negative impact on readership results for the quarter
• Work undertaken to reduce the impact of these fieldwork issues has resulted in a delayed release of the Q2 2010 to Q1 2011 CMI survey data
Resolution of Fieldwork Issues
• Not all issues with Q1 fieldwork can be resolved immediately. The survey auditor has requested that missing interviews in both Christchurch and other Regions must be replaced. We will be working with the Auditor with a managed program to replace surveys over the remaining survey periods of 2011.
• We have immediately reverted back to the previous sample structure.
• We have introduced new weighting criteria for Q1, 2011 data to reduce the impact of fieldwork issues.
• We have addressed staff shortages and are continuing with a recruitment programme.
• We have introduced new field work reporting and quality control systems.
Use of Q2 2010 – Q1 2011 CMI data
• As the quarter only contributes 12 weeks interviewing or 25% of the full annual sample the PMIRRG has agreed to the release of this data with a view that readership issues will be resolved over time.
• Users of data should be aware that total readership estimates for Q1 2011 have been negatively impacted, however the revised weighting system has minimised the impact on the Full Year results, on reader profiles & lifestyle data.
• Fieldwork issues have not affected the fusion of new Household Expenditure data that will be available for the first time in the Q2 2010 – Q1 2011 CMI database.
Please be assured that the ongoing quality of the CMI survey is of the highest priority to both myself and my team. If you have any questions or require any further information please do not hesitate to contact me.