Online Panels: A New Tool For Maximizing Customer Insight

Vision Critical
2006
Download PDF

A NEW ERA FOR RESEARCH AND ENGAGEMENT

The $22 billion (US) global market research and polling industry is undergoing rapid change in response to the emergence of the Internet as a tool for the collection of information on the tastes, preferences and behaviors of consumers and citizens. Part of this change involves a revolutionary shift in a fundamental element of market research: increasingly “sample size” is no longer the sole determinant of survey costs, and new software holds the promise of allowing an unprecedented level of direct client engagement in ongoing research.

Today, the market research industry finds itself in “the best of times and the worst of times”. The telephone, which has been the dominant data collection medium for several generations, is starting to yield questionable results. Fewer and fewer consumers and citizens want to talk with strangers on the telephone. Furthermore, new screening technology and the spread of cell phones make it increasingly difficult to contact the small group of willing survey respondents that remain. Recent estimates place the response rates for telephone surveys in North America at under 30 percent.

At the same time, the Internet has emerged to take the place of the telephone. The growth of the Internet is one of the most exciting and promising developments in the history of market research. The Internet places visually engaging stimuli into the research process. It empowers respondents to choose when and how to share their opinions and attitudes. Even better, it changes the cost equation in research since the “distribution” charges of telephone interviews are largely eliminated.

However, the fundamental challenge with Internet research is in some respects the same that confront conventional telephone research: where to find “willing” respondents. To solve this challenge is the key to unlocking the promise of Internet research. Increasingly, the solution is found through applying the principles of a research tool that has been around for a long time: the panel. With panels, Internet based research is transformed from “spam” to a critical tool for timely and cost effective research.

Panels progressively bridge the gap between a population that is reluctant to cooperate with telephone surveys and the organizations that need to know their perceptions and attitudes more than ever before.

WHAT IS AN ACCESS PANEL?

Access panels are owned by research companies that rent “access” to their databases to end-clients. Organizations can purchase a list of panelists consisting of a group of individuals who have agreed to participate in ongoing research on a particular topic, or company.

Participants “enter” a panel by completing a profiling questionnaire that contains basic information such as: demographics, lifestyle characteristics and media habits, which can then be used to guide future survey participation.

Panels are a growing phenomenon in the market research industry. Tony Cowling, Chairman and CEO of TNS, the third largest research agency in the world-with revenues of $1.4 billion-summarized the future of the market research industry with three words: ‘Panels, panels, panels”. According to Cowling, demand for proprietary (custom) panel software and hosting services is expected to soar over the next decade, as traditional market research buyers and companies new to market research come to realize the huge gains in both quality and cost these research platforms achieve.

The corollary of Cowlings upbeat assessment is nicely summarized by Jared Carleton, IT Consultant & Analyst for Frost & Sullivan, a strategic IT consulting firm, who warns that “those that try to avoid online research will be left behind by their competitors.”

ONLINE VS.OFFLINE PANELS

With Internet penetration levels in North America exceeding 75 percent, most organizations can now conduct solid, representative online studies of their customers and target markets. The visual dynamic of the Internet creates new opportunities to more accurately measure new concepts, advertising material and retail environments. Concerns about the accuracy of online research have largely abated in the wake of a flood of studies comparing telephone and Internet based approaches.

One of the most comprehensive projects assessing traditional and emerging methodologies was carried out by General Mills, a US based manufacturer of packaged foods. When the results of over 80 case studies were compared, their researchers found an almost perfect correlation between telephone and Internet based approaches. These findings were replicated in a recent Stanford University study which found “striking similarity of distributions across nearly all questions” and “remarkable comparability of
results”. In the UK, YouGov, an Internet-based polling organization, consistently produces election polls that have proved more accurate at predicting election outcomes than telephone surveys.

A panel can replace, or supplement, many traditional forms of research such as: telephone and mail surveys, focus groups and on-site interviews. Online panel research yields results within 72 hours, whereas telephone survey results can take 3 weeks. Depending on the nature of the sample source, online research savings are between 50 and 90 percent of the cost of telephone based approaches.

Today, online panels are being used to explore the full range of market research issues:

Profile all types of customers
Customer satisfaction surveys
Issues management
Advertising research
Communications testing
Market segmentation
Usage and attitude studies
Brand research
Product feedback
Positioning research
New product/service development
New product concept tests
Qualitative exploration
Conjoint or other advanced quantitative
methods
Tracking studies

Online research provides geographical advantages to recruit true market segments by virtually connecting participants. Previously, geography has been the primary default to assemble focus groups, where companies are forced to assume travel costs to attain the desired focus group target. Based on the profiling survey results, researchers can refine their target segmentation for little, or no additional cost.

Web conferencing technology (web cameras) allows researchers to monitor the facial expressions and body language, while participants take part in a more relaxed setting, such as their homes.

CUSTOMER ADVISORY PANELS

The Customer Advisory Panel (CAP) is rapidly becoming the principle tool by which corporations, associations and institutions assess the views, attitudes and perceptions of their potential and existing customers, members, stakeholders and employees. Applying the CAP concept online opens the door to a whole new world of ongoing communication and research, which includes: newsletters, discussion forums, focus groups, longitudinal studies and analysis.

The CAP eliminates the middleman by providing clients the ability to grow and full control of very large panels; thereby, permitting more timely and effective research than ever before.

Customer Advisory Panel Advantages:

Achieve depth to respondent profiles with every study, as participation in a panel develops each panelist’s connection with brands. Companies will discover new market and consumertrends as panelists’ insights grow and evolve overtime.
With instantaneous results, companies can receive digestible, accelerated and timely data.Researchers can send shorter surveys more frequently for no additional cost. Companies receive a favorable return on investment by conducting more research for less money.
A CAP allows companies to make more confident, strategic and tactical decisions with highercustomer validation than traditional methods.

PARTICIPATION

Online surveys are more convenient for the respondent, and are less disruptive and intrusive than traditional research methods. Participants can choose the place and time for completing surveys. Response rates range from 45 to 65 percent and higher, depending on the desired target group. Higher cooperation rates are the direct result of respondent participation in custom online panels.

Once a panelist’s commitment is confirmed through a double opt-in process, a profiling questionnaire containing demographic and behavioral data is deployed to assist in the definition of future survey target segmentation. Profiling information is essential to build a balanced panel and ensure clients reach their desired market segments. The profiling questionnaire is customized for each client. It should not contain more than 35 questions, and only take 10 minutes to complete.

ACCESS NICHE MARKETS

New technology and sophisticated forms of measurement enable companies to gain valuable insight into industry trends and market niches. A longitudinal approach connects companies to their customers by providing insight into their daily lives. It addresses questions such as: what is important to them, how do they feel about a particular policy, or approach; how do their attitudes and behavior change over
time; and do they have a connection with the product offerings.

The CAP enables an organization to source and target a specific audience that is relevant to the study. Target segmentation is achieved by screening panelists to gain access to low incidence customers, to explore issues, and to determine perceptions among that group. For example, people who have rockclimbed in the last 6 months.

By monitoring panel responses, an organization may discover a group of panelists with strong opinions that oppose the company’s marketing messages. They can dig deeper through the instant deployment of a survey, or the launch of a discussion forum to a specific group of panelists to uncover the issues that concern them.

SNIFF TRENDS

Eric von Hippel, Professor at MIT's Sloan School of Management, finds that forward-thinking businesses are setting up online forums to identify 'lead customers'. Lead customers can be defined as those who are early adopters, passionate users of a company’s product, and willing work with a company to drive innovation. Insight from 'lead customers' is an excellent indication of where the customer base will be in six months.

GAIN CONTROL

Organizations can own and control their research by adopting CAP technology, while decreasing their internal R&D turn around time and costs. End-clients can directly access their CAP to test new concepts with visual prompts, quickly evaluate customer service, or instantly answer tactical and strategic questions.

The company controls all aspects of the panel including: tiered permission access across various departments, who authors a study and who can access the data; and rules of the study content, the length and the number of studies deployed each month/year. This type of control can ensure uniform quality data, and reduce the likelihood of respondents being over surveyed.

End-clients determine the burden rules for CAP members. They create policies for purging and refreshing panelist samples to avoid an increasing problem with access panels; the professional respondent. A recent report by comScore Networks, an Internet research company, found that 30 percent of all Internet surveys are completed by only 0.25 percent of the Internet population.

FAST TURNAROUND

Easy access to online sample and specific subgroups enable organizations to quickly develop and launch surveys that facilitate a shorter research timeframe. With real time data collection, organizations begin analyzing results instantaneously.

A quick turnaround opens doors to conduct research that otherwise would not be possible: last minute research, effective crisis management and disaster checks, extreme tactical issues, and the management of a greater number of surveys at one time. Organizations are conducting twice as much research more easily and quickly than ever before.

LOWER COSTS

End-clients enjoy significant cost advantages by recruiting CAP across an existing customer database. The hard costs associated to recruit one CAP panelist can be amortized across several surveys, to lower the overall cost of the research. Simply stated, organizations get more bang for their research buck.

Recruit once and survey often with multiple studies per year, without all the incremental costs. This is especially important for niche markets where the cost of reaching a low incidence, and the target audience is prohibitively high.
Higher response rates decrease the recruiting costs incurred to achieve target sample sizes. As well, shorter questionnaires reduce panelist drop-out rates.
Immediate online data collection further decreases costs by the elimination of postage, long distance, interviewer and data entry charges.

David Hamilton, Director of Marketing for Microsoft.com, claims to have saved over $600,000 within 9 months by testing marketing campaigns, probing panelists for insight, and correcting problem areas before launching national and international marketing campaigns.

FACILITATE COMMUNICATION

Each interaction with panelists is a form of public relations. With multiple modes of communication and panelist connection points, a wide range of educational activities can be conducted using a CAP. The information collected from panelists can then be used to develop and distribute educational materials to a wider population.

Surveys are information gathering tools, forms of public relations, and educational processes for panelists. Surveys give the ability to fully examine any number of issues pertinent to an organization through the assessment of issue awareness.

Information gathered from a survey can be used to develop and publish effective communication materials on prominent aspects of an issue to educate and inform a wider population of customers, or stakeholders.

Emails are used to invite participants to participate in a survey and share relevant study information. Engagement of panelists on a regular basis is the key to a successful panel and emails are an excellent and efficient method. Using information gathered from the profiling questionnaire, emails can be customized to a panelist’s attitudes, interests, behaviors, or demographics.

Newsletters are instrumental in keeping panelists connected and engaged by providing information on new programs, initiatives, information updates and new or modified services. Incorporate the results of surveys into newsletters for a greater connection to panelists, which will provide increased credibility to the message.

Newsletters are particularly important for panel segments that are infrequently contacted for research purposes. For example, rural customers in your sample that have not been invited to a survey may not see the value in being a panelist, and eventually drop out. Next year, a rural store location has a crisis, but the panel’s rural target segment has faded away.

Forums, or bulletin boards bridge quantitative and qualitative research by interacting and engaging in 3-way dialogue with panelists. Not only does this technique connect clients with customers, it also connects panelists to each other.

A sub-section of the panel is invited to join a discussion forum to provide their opinion regarding a specific topic, or issue. The online discussion allows panelists to view the opinions and perceptions of other panelists and respond to those postings. By monitoring these discussions, organizations can assess interest in different topics, hot buttons for an issue, and audience perceptions and knowledge.

Similar to forums and bulletins boards, online focus groups are another qualitative research tool facilitated through panels. Qualified participants are invited to enter an interactive chat room that is controlled by a moderator.

CONSIDERATIONS IN PLANNING A CUSTOMER ADVISORY PANEL

Vision Critical currently manages over 100 panels for clients drawn from a wide range of categories. In our experience a number of factors affect both the cost and ease of panel construction. CAP’s are an ongoing commitment that takes careful thought, energy and time to successfully manage and maintain.

A successful panel requires a champion to oversee, manage and promote the tool internally. Without a champion, CAP’s have the potential to be underutilized and lose their power. When planning a CAP, take into consideration the following:

INTERNAL RESOURCES

Organizations must allocate resources to manage, maintain and effectively utilize its CAP, or its full potential, as a research tool will fade as panelists become disengaged. To maintain a healthy panel requires time and energy to develop communication plans, analyze results and generate reports. Organizations have multiple options to allocate panel management responsibilities:

A full-time, or part-time internal resource to oversee all aspects of panel management
Panel management responsibilities are divided among internal and external resources
An external resource manages all panel responsibilities under the supervision of an internal consultant

Panel solutions currently available in the market give an organization the option of full-service, selfservice or assisted self-service alternatives for conducting research. The full-service option involves outsourcing research to a market research supplier under the guidance of the client. The research supplier develops the questionnaires, codes and launches the surveys. Then they monitor the study progress, analyzes the data and generates the reports, and oversees all aspects of panel management.

Self-service provides clients with the ability to internally perform all of above mentioned full service activities for a fraction of the price. Self-serve clients control the entire panel management process and can easily accommodate last minute changes without delay, or additional costs. If considering the selfservice option, organizations must evaluate their internal capacity and allocate sufficient resources to successfully manage a healthy panel.

Assisted self-serve offers a perfect blend of support services from the panel provider. The client is responsible for survey design and analytics. While the panel provider, allocates an account manager to assist in every facet of the client’s research needs from developing a panel management and retention plan to survey programming and incentive execution.

Each organization chooses a different service option depending on their internal expertise, resource availability, timing and budgetary issues.

RESEARCH PLANNING

An organization’s research plan should be developed to include short-term and long-term research strategies. A properly devised CAP study plan will manage internal expectations and maintain internal interest in the CAP research results. Organizations should schedule studies well in advance to guarantee the CAP will utilize panelists effectively. To prevent panelists from dropping out, balance the length of time between studies to keep panelists engaged and to avoid over-exposure.

Establishing panel rules for frequency of contact, length of survey, response rates, and purging will improve panel activity. It is important to note that numerous survey invitations may contribute to respondent fatigue and systematic non-response among certain segments of the panel. On the other hand, dummy surveys can be deployed to keep under-used panel segments engaged and reduce panelist deserters. A successful panel will allocate resources to nurture relationships by viewing the CAP as a group of individual members with different needs, motivations and expectations.

BRANDED OR BLIND

Early in the CAP planning phase, a decision must be made if a client’s brand is to be used. A branded panel is recommended to examine specific customer opinions, or when a company’s touch points will be used as recruitment sources. Branded panels are ideal platforms to establish loyalty with current customers, and assess how customer attachment changes over time.

If the panel is to investigate entire market trends, a blind panel is created without disclosing the client’s identity to ensure unbiased results. A blind panel will generate high response rates when surveying a high recognition topic, product, or service. Recruitment will be easier if the topic, product, or service has high advertising frequency. If the blind panel topic is a specialized, or niche market, lowering the sample quota will keep the response rate high, as the panel will comprise of ‘enthusiasts’.

Although the cost of building a blind market panel is often significantly higher than a branded panel, this is offset by the reward of gaining invaluable market insight into non-customers and competitive knowledge.

RECRUITMENT CAMPAIGN

Once the target audience and desired panel size has been selected, panel recruitment begins by analyzing client’s assets, leveraging databases, placing invitations on websites and creating incentive plans.

Incentives are a powerful recruitment tool to include in the panel planning process, in order to secure respondent participation. Several incentive levels may be required to reach various market segments using numerous recruitment methods. Careful consideration is required regarding the length of the questionnaire, study topic and frequency of contact. This ensures the incentive is perceived as sufficient compensation for the respondent’s participation.

It takes more than just research to build a successful recruitment campaign, it involves marketing and media planning to find the correct campaign messaging to interest and engage prospective panelists. Companies should consider what they hope to achieve from their panel: will they turn panelists into experts overtime to provide educated responses, or will the company tightly control the panelist groups to gut check ideas with fresh panel responses?

Recruitment incentives involve a closer look at intrinsic and extrinsic motivation - not just offering cash and prizes. A panelist’s motivation to join a panel varies depending on the individual. Panelists may be motivated by retribution, while others want to give their opinions and know how the information is used to develop new products and services. Some panelists wish to belong to a broader community and connect with others who have similar interests regardless of incentives.

While intrinsic and extrinsic motivations vary by person, panelist responses provide invaluable information. Panelists often agree to participate because they share a common interest with an organization, or genuinely love specific products. A healthy panel thrives on allowing panelists to provide input, participate in the exchange of ideas, and ultimately learn how their feedback and study results have helped the organization.

Statistics show that motivation equalizes over time as panelists who express their opinions and improve products increase from 41 to 50 percent, while panelists interest in retribution decreases from 56 to 47 percent.

To create a level of trust with panelists, companies can provide the details of the overall research objectives and relay how the data will be used in the future. Panels that clearly define research topics or issues are easier to recruit than sending a generic survey invitation. For example, young mothers are more likely to join a panel that examines children’s health and education issues, while business travelers have a strong motivation to join air-travel-related panels. Neither group may care to participate in a
panel investigating attitudes toward an electricity company.

Determining current and potential customer touch points is an effective and efficient way to recruit and build a CAP. Touch points include company websites, points of purchase, warranty registration and loyalty programs.

In our experience, the size of the incentive is directly related to the trust points of a panel. If a panel has high interest, only a smaller incentive is required for panelists to respond. Similarly, if a study involves general interest questions, or is a blind panel, a larger incentive is required to entice panelists to join. Furthermore, panelists respond favorably to 1 in 10 chance to win $100, as opposed to a one-time draw of $1000.

RETENTION STRATEGY & PANEL HEALTH

A well developed incentive plan will go beyond the recruitment process and include incentives for respondents participating in ongoing research. The cost of fulfilling incentives can be fairly significant so incentive offerings must be carefully planned. The power of information is a great incentive to maintaining panelists. Share survey results and explain how panelist’s feedback will be used to develop and/or improve products and services.

Whereas a recruitment campaign focuses on gaining the prospective panelists’ attention, the retention strategy focuses on maintaining and engaging panelists by implementing the following key elements:

Avoid long surveys with tedious, poorly designed questions.
Engage and stimulate respondents using surveys infused with rich media or discussion forums to promote panelist interconnectivity.
Ensure the tone is appropriate to your target.
Consistent and professional branding promotes familiarity and trust.
Provide ‘member only’ reports and feedback study results to panelists to create a circle of dialogue that positively affects retention and panelist motivation.

Retention is unique to each panel. The symbolic value of incentive systems to reward, or thank those who participate is a powerful retention tool. Incentives can vary greatly, some examples are: cash, member-only reports, charitable donations, surprise draws, or panelist driven content. For example, a Vision Critical client in the retail food industry increased panel recruitment fourfold by offering a free muffin voucher to thank survey participants for their time.

Panel statistics should be monitored on a monthly basis to determine the health of a panel. To fully understand the panel’s health, clients must compare the total number of survey invitations sent against the number of responses received. Panelist turnover can occur through panelists dropping out, or the panel administrator purging panelists. To determine a respondent’s willingness to participate, compare the overall number of study invitations to the number of surveys completed. If a panelist’s overall response rate is low, it is recommended to remove the panelist.

Panel Health considerations include:

Frequency of Contact/Burden Rules: Are panelists “experts’”, or tightly controlled groups to ensure fresh responses? If the panel is primarily used to test concepts, tighter burden rules should be applied to ensure an adequate amount of time has passed between concept testing. Loose burden rules suffice, if the panel is testing a variety of topics. Keep in mind to under burden is more dangerous than to over burden.
Internal decisions must be made to set tiered permission access among panel administrators and study authors.
Limit the questionnaire length to a maximum of 10 minutes. Incentives should be appropriate to the time it takes to complete the survey.
Define what constitutes a non-responder, and set appropriate actions taken to purge panelists. If a panelist’s response rate is zero out of eight studies sent, quietly remove the panelist from a branded panel without notification. Due to the cost of recruiting a blind, low incidence panel, send a gentle reminder when the panelist is reaching the predetermined purge rate.
Send direct panel invitations to specific target groups. Avoid sending a mass, open arms style invitation that attracts professional survey respondents.

TIMING

A CAP can be designed and fully operational within 6 to 9 weeks, it all depends on the complexity of the recruitment plan. A research schedule should include 2 to 3 weeks to develop the recruitment plan and profiling questionnaire; 2 to 3 weeks to install the software and customize the creative look of the product to your organization; and, 2 to 3 weeks to recruit the panelists.

The survey’s complexity, along with the urgency and depth of the information required, influences whether individual studies will take a few days, or several weeks to collect sufficient data.

ANALYSIS AND REPORTING

Analyzing results and generating reports is made easy with a panel solution that offers robust analysis and reporting capabilities. Analysis functions should allow users to review data in various formats including graphs, tables and charts. Panel research software should provide the ability to weight data, perform significance testing, real time reporting, generate cross-tabs and filter respondents.

Depending on the main analysis and reporting capabilities of a panel software solution, additional resources may be required for more complex needs such as regression, or segmentation. The panel solution should permit exports to a statistical package such as SPSS, or SAS.

CONCLUSION

Today, the penetration of the Internet has brought forth rapid changes in market research, which leaves organizations to wonder: what to do with all this new technology. The traditional tools and methodologies that have worked in the past, will not keep up to the changes in market research. Don E. Schultz, Professor of Integrated Marketing Communication at NorthWestern University comments, “What does all this add up to? It’s consumer access, not consumer exposure. It’s when consumers want to hear and learn, not when marketers want to talk that really counts.”

Vision Critical has made it possible for organizations to gain a competitive edge through instantaneous market intelligence. Market researchers are embracing customer advisory panels to eliminate the guess work and make knowledgeable decisions with the backing of dedicated panelists.

ABOUT VISION CRITICAL

Vision Critical delivers innovative tools and dedicated account management to recruit, engage and learn more with a customer advisory panel. At a moments notice you can investigate the attitudes, behavior and opinions of existing, or potential customers with the use of interactive surveys, discussion forums and 3D environments. See instant results and track responses over time to harvest deep insight.

Three core principles have helped us become leaders in the field of online research.

We Simplify the research process by unifying design and technology.
Empower end-clients with the tools to conduct research themselves.
Enrich market research with the latest advancements in panel management technology and rich media applications.

The Vision Critical executive team has over 65 years of combined experience in the real world of market research and polling, including 15 software start-ups to their credit. The company originated in 2000 and is led by the North American research authority with over 30 years in the industry, Dr. Angus Reid. Organizations that use our technology include Nestle, Time Inc., Virgin Mobile, Fast Company and the NHL. For more information, please visit us at www.visioncritical.com.

APPENDIX A: CHOOSING A PANEL VENDOR

When the time comes to decide on panel software, or a research supplier, compare each solution to determine product performance, flexibility, vendor-provided services, and all the limitations first. During the review and evaluation process, ask the following:

What are the capabilities of the panel software to access and author questionnaires, filter samples, launch surveys, and manage multiple studies?
Will the software accommodate questionnaire design elements: skip patterns, rotation, randomization, blocks, multiple questions per page, auto next button, calculated questions andmandatory, or optional responses?
Are length limitations placed on the questionnaire?
Can the software incorporate visuals such as 2D and 3D images, audio and video streaming?
Is there the ability to incorporate, evaluate and discuss issues, such as product packaging and pricing, in a virtually simulated environment?
What type of on-going support services does the vendor supply? Does it include: a dedicated account manager to provide panelist support, help desk functions, monthly panel health reports and user training?
How is the recruitment plan developed?
Is guidance offered on incentive planning for panelist recruitment and ongoing studies?
What are the qualitative capabilities of the research software?
What will my panel interface look like? Will it be customized? How is the creative designed?
What does the panel portal look like? Can it be customized? What are the portal limitations?
How does the software communicate with panelists: emails, newsletters, study invitations, discussion board and websites?
What panel management tools are used to assess the health of the panel, sample size and effective panel utilization?
What are the reporting capabilities? What types of reports are generated, and in what formats? Is there the ability to weight data, perform significance testing and real time reporting?
What are the exporting and importing capabilities for both for data and reports? What file formats can be accommodated? Can you easily import data into more advanced statistical packages? Is data exporting a simple and easy process?