The Grand Mean Project®
Since fourth quarter 2007, Sample Source Auditors®, a division of Mktg. Inc., has been conducting an
extensive study of global online panels by compiling and analyzing data for the Grand Mean Project®.
This study has collected over 200 waves of 500 interviews, representing panel companies in 35
countries. It is currently the largest and most inclusive online sample assessment to date. (The Grand
Mean combines available data across sources for a country or region. For example, a Grand Mean can
measure respondent tenure across panels within a country).
The Grand Mean Project® is a unique tracking study designed to examine respondent behaviors within
individual panel sources and also across different panel providers. It is a streamlined instrument
containing several batteries of attitudinal questions that enable us to draw conclusions about the
panelists and the panel sources involved. This survey was translated into the local languages required
by 35 international markets. The mix of questions covers: Technology and the media, Participation in
market research, Buyer Behavior, Interests, Values & Lifestyle and Demographics.
To insure that sampling frames exhibit appropriate levels of continuity, predictability and reliability
they must be measured over time. We use segmentation analysis as a new metric that will allow us to
anchor online data in a new non-probabilistic sampling frame. We use the Grand Mean of
segmentations (Buyer Behavior - 37 variables; Sociographic - 31 variables; Media - 31 variables are
the three core segmentations as well as seven market segmentations: appliance, banking and insurance,
automobile, clothing, consumer electronics, entertainment, grocery shopping) within each country to
establish this new metric.
We continue to collect data on an unprecedented basis; to create a baseline to track, clarify and project
the panel universe as it changes. By measuring variability around the Grand Mean, we can stabilize
market research data in the future. Figures 1&2 demonstrate the variability between sample sources in
the US. Our focus in the American markets was inter-panel and sourcing mode differences in buying
behavior (Gittelman and Trimarchi, CASRO 2009). We found that within sourcing modes, relative
homogeneity existed with variance most predominant between modes. Our analysis shows that
heterogeneity is widespread globally due to differences in: methods in recruiting sample, aging of
respondents, and the vast diversity of sources that are being used to create the panels.
Sourcing is one of the greatest influences driving behavioral variability between sample sources. The
chart below examines buyer behavior variability by the type of data source. These sources include
established panels, River sample, phone and panel/river mix.
To provide meaningful quality audits, we depend on reliable metrics. We maintain an independent
auditing process in the development of our measures. Our objective is to measure sample differences
that appear not only across panels, but within the same panel over time. Auditing sample consistency
is critical to quality research and the future success of online panels.
|