Tips For Reading Market Research Proposals

(part of Chapter 7: Market Research Agency Selection)

Reading proposals can be tricky. Obviously, a market research agency is trying to sell itself to you, so they’re going to represent their abilities in the best way possible. So what can you do? Learn how to read proposals carefully and look out for details that are too good to be true.

Be realistic when considering your needs versus an agency’s abilities, and trust your gut. Read on to learn some practical proposal-reading tips.

So what should you look for as you read through these proposals? Here are some tips that will help you focus on the most important content.

1. Avoid comparing solely on price. When comparing proposals, the temptation is to immediately flip to the last page and compare fee estimates. Is Company A proposing $50,000 and Company B proposing $100,000? Don’t dismiss Company B right away—it might simply be using more realistic pricing assumptions.

As a general rule, if the proposed prices vary by more than 20 percent, you can be certain that the agencies used different assumptions. Pricing will vary if they used different assumptions about the difficulty of finding qualified respondents (also referred to as incidence rate), required deliverables, or the types of data analysis required. Perhaps one agency built in some analyses that would be overkill for you. Perhaps another assumed you were providing the sample and as a result proposed a markedly low fee.

There’s certainly nothing wrong with calling agencies back and redirecting any incorrect assumptions, but why waste the time? If your RFP is precise, you should get fee estimates in the same ballpark. If your RFP is precise, and fees are still really divergent, ask the high and low bidders to comment.

2. Look at the biographies carefully. It’s always a good idea to compare proposals based on the descriptions of the team bios submitted. A reputable MR firm will tell you something about who’s going to be assigned to your project, should it be selected. Read the language carefully: Is it sending you the bios of representative team members or those of the professionals it actually intends to assign?

Some large MR firms have reputations for bringing in their big guns (senior consultants and VPs) to sell a project, but then relegating actual day-to-day project management to a junior-level person. If your project is simple, you might be better off with more of a junior- to mid-level person because their daily rates are going to be lower, so the research will cost you less. But if you are doing something that’s very significant, then you want senior-level project management.

As an aside, the bait-and-switch, when it happens, is often unintentional. If you send out an RFP in January, but don’t select a vendor until April, there is a good chance that the initially assigned consultants will be booked with other clients.

Here are a few other things to look for as you review bios:

• Are the bios from full-time employees or subcontractors? Using subcontractors might not be a bad thing, but it does say something about a firm’s in-house expertise and ability to handle project loads.


• Do the bios indicate what other clients the team members have worked with? Have they worked with clients in your space? Do they work with your competitors?


• If they do work with your competitors, don’t dismiss them right away. After all, this may mean they have a lot of relevant contextual knowledge from which you can benefit. Of course, you should ask about conflicts of interest and how they would protect your confidential information. One option: simply ask them to not engage with your competitor while they are engaged with you.


• Do the bios show relevant expertise? How many of the proposed team members have worked on similar projects in the past? For example, if you are doing a brand awareness project, what relevant experience is highlighted in the bios?


• How long have team members been with the agency? Have the key members been with the agency for at least three years? That shows that they have a track record with that agency’s processes and best practices. After all, if you buy into a sales pitch that includes this firm’s approach to, say, project management, you will want someone managing the project that has been applying those project skills for more than a year.

3. Reality check the proposed timeline. If one agency is proposing a timeline that is markedly shorter than anybody else’s, it’s a red flag. Of course, markedly longer is a red flag as well. But in reality, I’ve seen more clients get burned by agencies that proposed an unrealistically short timeline.

Typically, if one agency’s timeline is significantly shorter than everybody else’s, it’s making overly aggressive promises to win your business. Of course you want the MR data tomorrow, so the promise of a short timeline is enticing. But the truth of the matter is that even in the world of online research (which has quicker data collection than other options), a high-quality primary MR project still takes at least four to six weeks.

Think about it. At minimum, you’re going to need two weeks for questionnaire design or moderator guide development alone. And if you have 2 or more internal clients who need to be in on approval cycles or a legal department that has to okay the content, it can easily be more. Rushing through that process is going to cause heartache for everyone at the end of the project. So if a bidding agency proposes a timeline that seems just too good to be true, it probably is. You really need to ask them what their assumptions are for each phase of the project.

Do exceptions exist? Yes. Online or phone surveys can be done in a day or two, provided most of the following conditions are met: if it’s a consumer study; if you have a standard questionnaire that you are fielding; if you literally have five or six questions; if you have a great sample source that can execute in 24 hours; if you are buying into an ongoing survey program (referred to as omnibus surveys); and if you don’t need any real reporting—just the data.

4. Review the project management approach. Compare proposals based on how the firms talk about project management.

Do they really seem to understand the importance of managing the project on a day-to-day and week-to-week basis? How do they describe the kickoff process? Do they advocate for in-person kickoffs, or do they seem to prefer a more casual approach? Do they state that they’re going to provide weekly written status memos? How do they say they’re going to track key milestones? Are they planning to have weekly status conferences with you, the client? Although the level of project management required varies by project complexity, it’s always good to ensure that a potential supplier’s approach will align with your expectations.

5. Look for mentions of QA. Do they mention QA in their proposals? What kind of process have they built in for doing quality checks throughout the process, not just at the end once the data are collected? What information or best practices do they cite in the proposal that addresses quality at key junctures?

If you ask, “What steps do you take to make sure the questionnaire is programmed correctly?” how do they respond? This is a completely valid question. Imagine if you were just done with data collection, only to find that due to a programming error, a key question was displayed with the wrong answer options? Ouch!

6. Highlight any content that seems odd or wrong. MR proposals should pass a basic gut check: Does the proposal seem honest and believable? Is the price too good to be true? Are any statements unclear? You should ask about any content you are unsure of.

7. Beyond boilerplate. I always prefer a proposal that shows some original thought as opposed to just regurgitating what was in the RFP and adding in some boilerplate. Some MR firms overuse boilerplate. Using boilerplate doesn’t mean a proposal is bad, and it doesn’t mean the firm is not a good company. But won’t you feel better reading a proposal that was clearly written for you, addressing your needs?

8. Value add (if you are looking for a full service agency). Look for any evidence that suggests that the MR firm is going to add real value to the project, as opposed to just moving through a process in a robotic fashion. For example, has it identified in the proposal what it thinks the greatest risks are in doing this project successfully and identified what it thinks it can do to mitigate those risks? Has it made it clear that it understands the critical success factors that will make this project successful for you and the rest of your team upon completion? Has it shown that it understands the topic by identifying some preliminary hypotheses that could be relevant to the study?

These things show that (a) it cares enough about the work to really put some thought into the proposal, (b) that it is going to be a partner that is going to add value to the process, and (c) that it really understands your business and will be able to speak credibly with your internal colleagues.

This is an excerpt from the book, "How to Hire & Manage Market Research Agencies," which is available on Amazon. Published by Research Rockstar LLC. Copyright © by Kathryn Korostoff. All rights reserved.

Tweet

Browse "How to Hire + Manage Market Research Agencies" by section

Intro: Maximize Success: Know The Risks
     Understanding Market Research Challenges

Chapter 1: Know Your Goals
     Planning Your Market Research Project
     Defining Appropriate Goals For Your Market Research Project
     Practical Market Research Planning Considerations
     Questionnaire Design Risks: The Challenge of Staying Focused
     Defining Project Goals Through Questionnaire Real Estate

Chapter 2: Defining Your Project's Scope
     Project Scope: Product Concept Testing and Message Testing
     Project Scope: Market Segmentation
     Project Scope: Customer Loyalty and Win-Loss Research
     Project Scope: Brand Awareness and Needs Discovery
     Revealing Sponsorship of Your Market Research Project
     Planning For Your Target Population: B2B or B2C?

Chapter 3: Considering Internal Audience
     Understanding Your Internal Audience: Research Receptiveness
     Dealing With Hard-Core Market Research Skeptics

Chapter 4: Leveraging Existing Research
     Leverage Existing Research

Chapter 5: Planning Your Agency Relationship
     Building a Relationship With Your Agency
     Hiring a Market Research Agency: Managing Your Budget

Chapter 6: RFIs and RFPs
     The Benefits of RFIs
     The RFP: Finding Great Research Partners
     A Few Helpful Reminders on RFPs

Chapter 7: Market Research Agency Selection
     Tips For Reading Market Research Proposals
     Checking Agency References and Follow-Up
     How to Check Market Research Agency Credentials
     Choosing the Right Market Research Agency

Chapter 8: Managing the Agency
     Project Management Planning
     Keeping Your Market Research Project On Track

Chapter 9: A Successful Deliverables Strategy
     Planning Your Market Research Deliverables
     Planning a Great Presentation of Research Results
     Presentation Tactics and Options

Understanding Market Research Challenges

by Kathryn Korostoff, Research Rockstar LLC

Understanding the risks associated with MR is the key to project success. By familiarizing yourself with the risks associated with MR before beginning any projects, you will know what to avoid, resulting in higher success rates and more accurate research results. Read on to learn about the most common market research challenges. Read Article »

Planning Your Market Research Project

by Kathryn Korostoff, Research Rockstar LLC

While planning a market research project may appear to be a fairly straightforward task, in reality, it takes some real time and effort. Agency selection is one of the key factors to project success, so don’t rush the process. Read on to learn about the timeline logistics of hiring the perfect firm and setting things in the right direction from the start. Read Article »

Defining Appropriate Goals For Your Market Research Project

by Kathryn Korostoff, Research Rockstar LLC

When it comes to defining appropriate project goals, it is best to think in terms of questions or hypotheses. Your golden ticket lies in formulating fewer, “high-level” goals rather than multiple ones in need of conquering. Too many goals dilute a project and increase the risk of failure. Read on to learn how to formulate your goals. Read Article »

Practical Market Research Planning Considerations

by Kathryn Korostoff, Research Rockstar LLC

Prior to conducting any project, it’s necessary to decide if custom market research is truly the best choice. If custom MR turns out to be the best option for your project, the next step is to think precisely about defining project goals. Market research is truly a garbage in/garbage out process; start with clear goals, and your chances for market research success are excellent. Read Article »

Questionnaire Design Risks: The Challenge of Staying Focused

by Kathryn Korostoff, Research Rockstar LLC

One of the main reasons MR projects fail is due to lack of an agenda, or the inability to stay focused on an originally projected design. As more individuals become involved with a project, they bring with them the risk of transforming a once precise questionnaire into an excessively long one with illogical flow. The most important aspects to keep in mind when designing a questionnaire are fit, flow and length. Without these, the questionnaire becomes a burden on the respondents, which directly affects project success. Read on to find out how to avoid the pitfalls of questionnaire design. Read Article »

Defining Project Goals Through Questionnaire Real Estate

by Kathryn Korostoff, Research Rockstar LLC

It is important that everyone on a team understand the primary goals of a market research project. While getting a group of individuals to agree on priorities may be a daunting task, certain exercises such as Chartstorming™ may help you reach a consensus. Read on to learn how to manage your questionnaire real estate through better group collaboration. Read Article »

Project Scope: Product Concept Testing and Message Testing

by Kathryn Korostoff, Research Rockstar LLC

While defining a project scope may seem like a daunting task, it really comes down to two things; knowing you want to get out of your project, and deciding how you’re going to get it. It also helps to know some of the common language used to define market research studies. Read on to learn about product concept testing and message testing. Read Article »

Project Scope: Market Segmentation

by Kathryn Korostoff, Research Rockstar LLC

While defining a project scope may seem like a daunting task, it really comes down to two things; knowing you want to get out of your project, and deciding how you’re going to get it. It also helps to know some of the common language used to define market research studies. Read on to learn about a common project type: market segmentation. Read Article »

Project Scope: Customer Loyalty and Win-Loss Research

by Kathryn Korostoff, Research Rockstar LLC

While defining a project scope may seem like a daunting task, it really comes down to two things; knowing you want to get out of your project, and deciding how you’re going to get it. It also helps to know some of the common language used to define market research studies. Read on to learn about customer loyalty and win-loss research. Read Article »

Project Scope: Brand Awareness and Needs Discovery

by Kathryn Korostoff, Research Rockstar LLC

While defining a project scope may seem like a daunting task, it really comes down to two things; knowing you want to get out of your project, and deciding how you’re going to get it. It also helps to know some of the common language used to define market research studies. Read on to learn about brand awareness and needs discovery. Read Article »

Revealing Sponsorship of Your Market Research Project

by By Kathryn Korostoff, Research Rockstar LLC

Defining your project scope also includes whether or not to reveal the sponsor of your market research study. Read these guidelines for help making this decision based on best practices and common exceptions. Read Article »

Planning For Your Target Population: B2B or B2C?

by By Kathryn Korostoff, Research Rockstar LLC

B2B and B2C research are two very different animals. Which one are you doing? The answer has implications for partner selection, questionnaire design, and sample source options. Read Article »

Understanding Your Internal Audience: Research Receptiveness

by By Kathryn Korostoff, Research Rockstar LLC

There are multiple facets to consider about an internal audience’s readiness for market research. These items range from an audience’s receptiveness to specific techniques, all the way to presentation of data. It is important to take note of the your internal audience’s abilities and preferences to make sure they will be able to receive and apply research results successfully. Read on to learn about aspects to consider when assessing your internal audience’s readiness for MR. Read Article »

Dealing With Hard-Core Market Research Skeptics

by By Kathryn Korostoff, Research Rockstar LLC

While we all hope for an accepting reception from colleagues when it comes to research findings, it’s a rare occurrence for everyone to be on board. Dealing with internal skeptics is no easy task, but a few precautions taken prior to delivering your research findings can greatly reduce disruption risk. Read on to see what you can do to get skeptics on your side. Read Article »

Leverage Existing Research

by Kathryn Korostoff, Research Rockstar LLC

Before you jump into your primary market research head first, it’s important that you first seek out existing, relevant research. This not only saves your organization time, but also prevents wasted funding on a project that would have duplicated existing results. If finding relevant research seems like a daunting task, don’t worry, there are multiple sources you can turn to. Read on to learn how to get started. Read Article »

Building a Relationship With Your Agency

by By Kathryn Korostoff, Research Rockstar LLC

Before you select a market research agency, think carefully about the type of relationship you want. It’s important that the agency you choose can work with your team in a way that will truly meet your needs. There are hundreds of agencies out there; they do not all have the same styles or collaboration approaches. Read on to learn how to build a great relationship with your chosen agency. Read Article »

Hiring a Market Research Agency: Managing Your Budget

by By Kathryn Korostoff, Research Rockstar LLC

Before you select a market research agency, think carefully about the type of relationship you want, especially when planning your budget. It’s important that the agency you choose can work with your team in a way that will truly meet your needs. There are hundreds of agencies out there; they do not all have the same styles or collaboration approaches. Read on to learn how to build a great relationship with your chosen agency. Read Article »

The Benefits of RFIs

by By Kathryn Korostoff, Research Rockstar LLC

Taking time to write either an RFI or RFP may seem like a burden, but in the long run as a form of damage control that saves you both time and money. For market research, it is more common to go straight to an RFP. But in some cases starting with an RFI can “filter” a long list of possible companies down to a more manageable set. Read on to learn how to write RFIs and RFPs for market research. Read Article »

The RFP: Finding Great Research Partners

by By Kathryn Korostoff, Research Rockstar LLC

In many cases, skipping an RFI and going straight to an RFP is an appropriate choice. If you’ve already narrowed your choices to a manageable number of potential research partners (no more than 5), it’s time to issue an RFP. A well-written RFP will result in proposals that will help you choose the best research partner for your needs. Read on to learn about the elements of a successful RFP. Read Article »

A Few Helpful Reminders on RFPs

by By Kathryn Korostoff, Research Rockstar LLC

Documenting your market research in an RFP saves time, energy and money. Some aspects to consider include legalese and deliverables. When looking at the specifics of your market research needs, it is important to consider your internal audience, too. Read on for some practial RFP tips. Read Article »

Checking Agency References and Follow-Up

by By Kathryn Korostoff, Research Rockstar LLC

While you should always check a research agency’s provided references, it’s also smart to look for more objective sources who can attest to a firm’s capabilities. In addition, the way in which a market research firm follows-up their proposal submission says a lot about their communication skills and style. Read on to learn about checking market research agency references, and other steps you can take in choosing an agency. Read Article »

How to Check Market Research Agency Credentials

by By Kathryn Korostoff, Research Rockstar LLC

There are literally hundreds of market research agencies from which you can choose. Of course, finding ones that have the right qualifications can be tricky. There is no professional certification required in the field, so you need to do a little work. Read on for some tips on checking potential research partners for appropriate credentials. Read Article »

Choosing the Right Market Research Agency

by By Kathryn Korostoff, Research Rockstar LLC

Once you’ve narrowed your agency choices down to a short-list of finalists, choosing between them can be a daunting task. While it may be tempting to go with the agency offering the lowest price, it’s important to consider other factors, too. Read on to learn some different strategies your organization can use to make the right agency choice. Read Article »

Project Management Planning

by By Kathryn Korostoff, Research Rockstar LLC

Once you’re hired an agency, then what? Well, if you’re looking to reap the benefits of a successful project, your active involvement is necessary. By working closely with your agency, you’ll be able to represent your internal clients’ needs while providing the agency with important direction. Indeed, your involvement can help the agency be more successful. Read on to learn what to expect from working with an agency once you start the project. Read Article »

Keeping Your Market Research Project On Track

by By Kathryn Korostoff, Research Rockstar LLC

Though it is easy to jump to conclusions at any sign of trouble, the important thing to remember is that both you and your agency are working towards the ultimate success of your project. That said, there are legitimate issues that may arise. Luckily, most issues have reasonable solutions—and an experienced researcher will help you navigate them. Read on to learn about legitimate red flags versus project paranoia. Read Article »

Planning Your Market Research Deliverables

by By Kathryn Korostoff, Research Rockstar LLC

When delivering market research project findings, the most effective approach is one that combines multiple forms of delivery. After all, this information is the kind you want retained by your colleagues so they can really apply it. And we all know that delivering a 100-page slide deck just puts them to sleep. Read on to learn more. Read Article »

Planning a Great Presentation of Research Results

by By Kathryn Korostoff, Research Rockstar LLC

Think of your project’s final presentation as more of a performance. Assigning roles and specific duties to individuals keeps the presentation organized and engaging. You want the audience to be attentive, so it’s important to include them. Don’t be afraid to ask questions or start dialogue. Read on to learn about the basics of a great presentation. Read Article »

Presentation Tactics and Options

by By Kathryn Korostoff, Research Rockstar LLC

Imagine it is the end of a market research project, and you are getting ready to present the final results. Are you ready? With a little planning, you will be. The following Q & A answers key questions about presentation tactics to make sure your presentation goes smoothly. Also offered are some alternatives to traditional presentation tactics. Read Article »