The purpose of this book is to summarize the current state of knowledge about online panels. It will bring together in one volume the best empirical-based research on what has become a very important yet controversial method of collecting data online. Online panels are used in many fields including: survey research, opinion polling, market research, medical research, psychology, political science and economics, to name a few.
Over about the last decade, there has been a major global shift in survey and market research toward survey data collection using online panels. In the market research sector, for example, ESOMAR estimates that in 2009, 28% of global spending was on online research--mostly in online panels. This compares to only 13% in 2004. In the US alone online revenues amounted to around two billion dollars in 2010. Yet despite their widespread use remarkably little is known with confidence about the quality of the data online panels generate.
Advocates of using online panels for population description, mostly in the commercial sector, argue that with proper selection of panel members and post survey adjustment techniques they can produce accurate estimates of almost any population of interest. And they claim that they can do so much more quickly and at a lower cost than with other methods. Other researchers, mostly in the academic and government sectors, have reservations, noting that the vast majority of online panels are used in ways that violate many of the basic assumptions of probability theory, thereby calling into question the representativeness of the samples produced from these sources. They also worry about the incentive-driven culture of online panels and about the cumulative effects of heavy survey taking. Other than using online panels for population description, online panels might be advantageously employed for non-population descriptive research.
Widespread professional interest in these issues has existed for over a decade, but relatively little high quality research has been subjected to peer review and been published. On one side of the debate, advocates of online panels have mostly made their case in conference settings and corporate white papers and have produced little by way of well-documented studies in peer-reviewed journals. They have tended to eschew theoretical arguments in favor of straightforward empirical results to show that the method works. In recent years, only a small number of peer-reviewed publications have appeared reporting empirical research evaluating online panels and comparing their results to data collected by other methods.
Given the above, we invite public and commercial sector researchers to submit proposals for chapters in an edited volume. The editors encourage comparative empirical studies that demonstrate differences or similarities in results across methods. The primary purpose of the book is to provide new empirical evidence evaluating all sources of potential error in online panel measurements, to compare such errors to errors in parallel data collected by other methods, and to identify methods for minimizing error.
A subset of illustrative possible topics include:
Sampling & Coverage Errors: Probability and Nonprobability methods, including sample representativeness and other aspects of external validity
Online Panel recruitment methods and Nonresponse
Panel Maintenance and Attrition
Online Sample Selection and Adjustment Techniques
Measurement Errors in Online Research, including satisficing, missing data, imputation, respondent burden, and other measurement error topics
Interactions among various sources of Survey Errors in online panel research
Use of online panels where representativeness is not a goal
Rare/low incidence population studies
A goal of the book is to be as interdisciplinary as possible, so a wide array of other topics is within the scope of the book.
The editors invite potential authors to submit a 4-6 page double-spaced chapter proposal with the following:
Detailed description of the chapter content
Tentative outline of chapter
Description of the dataset(s) that will provide the foundation of what is reported in the chapter
Up to five key references
The proposal should be sent via email to online.panels.wiley@gmail.com
2012
January 31th Detailed editors' feedback to chapter authors of first round of submission
March 15 Detailed editors' feedback to chapter authors of second round of submission
September 1st First chapter draft due
November 1st First draft editors' feedback due to authors
2013
February 15th Second chapter draft due
April 1st Second draft editors' feedback due to authors
June 30 Final draft delivered
Author Requirements for Dataset Access
A requirement of participating authors reporting quantitative results is to provide an electronic copy of all original data analyzed for a chapter, in addition to analysis syntax and analysis output showing the results reported in the chapter. Detailed instruction on how to prepare the dataset, metadata, and technical requirements will be communicated to authors who are invited to contribute to the book.
The datasets will be placed on a book companion website. This will allow any user to re-analyze the data and potentially provide more insights.