Since fourth quarter 2007, Sample Source Auditors®, a division of Mktg. Inc., has been conducting an extensive study of global online panels by compiling and analyzing data for the Grand Mean Project®. This study has collected over 200 waves of 500 interviews, representing panel companies in 35 countries. It is currently the largest and most inclusive online sample assessment to date. (The Grand Mean combines available data across sources for a country or region. For example, a Grand Mean can measure respondent tenure across panels within a country).

The Grand Mean Project® is a unique tracking study designed to examine respondent behaviors within individual panel sources and also across different panel providers. It is a streamlined instrument containing several batteries of attitudinal questions that enable us to draw conclusions about the panelists and the panel sources involved. This survey was translated into the local languages required by 35 international markets. The mix of questions covers: Technology and the media, Participation in market research, Buyer Behavior, Interests, Values & Lifestyle and Demographics.

To insure that sampling frames exhibit appropriate levels of continuity, predictability and reliability they must be measured over time. We use segmentation analysis as a new metric that will allow us to anchor online data in a new non-probabilistic sampling frame. We use the Grand Mean of segmentations (Buyer Behavior - 37 variables; Sociographic - 31 variables; Media - 31 variables are the three core segmentations as well as seven market segmentations: appliance, banking and insurance, automobile, clothing, consumer electronics, entertainment, grocery shopping) within each country to establish this new metric.

We continue to collect data on an unprecedented basis; to create a baseline to track, clarify and project the panel universe as it changes. By measuring variability around the Grand Mean, we can stabilize market research data in the future. Figures 1&2 demonstrate the variability between sample sources in the US. Our focus in the American markets was inter-panel and sourcing mode differences in buying behavior (Gittelman and Trimarchi, CASRO 2009). We found that within sourcing modes, relative homogeneity existed with variance most predominant between modes. Our analysis shows that heterogeneity is widespread globally due to differences in: methods in recruiting sample, aging of respondents, and the vast diversity of sources that are being used to create the panels.