The Five Eras of Online Sampling: An Industry Perspective

I’ve had the privilege of working for some of the most innovative and disruptive companies in the market research industry over the past twenty years. During that time, I observed the industry evolving and transitioning through several distinct eras marked by meaningful shifts in strategy, execution, and/or KPIs.

The Birth of the Online Era

My first job was with Greenfield Online, which was regarded as a pioneer in online sampling. I vividly remember the lively debates with methodologists at leading research agencies about online penetration, representation, and how online panels couldn’t compare to the gold standard of random digit dial interviewing. I was typically unable to convince the PhD across the table from me of the methodological soundness of online sampling; however, the time and cost savings afforded by online sampling were undeniable and compelling. Over the next decade, the industry transitioned from offline to online with reckless abandon.

The Consolidation Era

In contrast to the research agencies steeped in methodological rigor and statistical modeling, the sample companies were VC-backed technology and sales-driven organizations. Competition was fierce, and the winners in the space scaled up rapidly, eventually entering a phase of consolidation. In fact, many of the companies you work with today are made up of as many as fifteen acquisitions and mergers. With all the emphasis on growth and expansion, we have lost sight of the lifeblood of our industry, the survey respondents.

The Programmatic Era

The next chapter of the story is perhaps yet more important as the era of programmatic sampling emerged. Prior to this era, the large incumbent suppliers had essentially outgrown their ability to build and maintain large enough panels to fulfill demand. As a solution to this, they built sophisticated routing technology that enabled them to aggregate respondents from sources beyond their own panels.

Often, these sources were marketed under various names, including “River” or “Integrated Partners,” among others. The aggregation era was a massive technological advancement, fueling the continued scale and growth of established sample companies. However, unlike the programmatic exchanges that were coming to market, there was little to no transparency into the underlying sources of respondents.

The marketplaces and exchanges sent shockwaves through the industry. For the first time ever, buyers of online sample were given complete-level details on which source a respondent came from. Not only could they reduce costs with total transparency, but they could also utilize technology to do it more efficiently.

On the other side of the equation, suppliers who had for years delivered millions of completed surveys to the established provider’s routers were now selling directly via marketplaces to agency buyers and earning higher prices than ever before. This created an entirely new set of skills and metrics for the supply chain, known as yield management, as more survey inventory became available to sell through APIs.

The Algorithmic Yield Era

The basics of yield management are pretty simple. Yield management is maximizing revenue by analyzing historical data on pricing, conversion, and time. It uses a predictive model to determine the optimal survey match for a given respondent factoring in their history, observed behavioral preferences, and other market factors. The suppliers’ main goal is to achieve maximum revenue generation while balancing their respondents’ experience.

A session, or click, occurs when a respondent is live on a website or in an app and wants to participate in a survey. A supplier has a limited number of active sessions on any given day and has thousands of potential survey opportunities to offer respondents. Since the supplier is only paid if the respondent starts, qualifies, and completes a survey, suppliers have to make a split-second decision on which survey(s) to offer and promote to the respondent to maximize their earnings potential.

They are in the business of acquiring members to join their panels, retaining them, and earning enough money over a member's lifetime to make a profit. Therefore, yield management is critical for their business to succeed. Superior yield management leads to higher conversion rates, which in turn result in more earnings per click (EPC), ultimately leading to increased overall revenue that can be invested in member retention and new member acquisition, thereby driving scale.

The Goldfish Effect Era

The last component of yield management is time, and over the past few years, it’s become increasingly important. Time is money, and respondents are only able/willing to spend a specific finite amount of time participating in surveys. Even the very best yield management systems leave the final decision of what to click on to the respondent.

They have their own innate, maybe even subconscious, yield management algorithms in their brains that rapidly assess the surveys being presented to them by the supplier and make their final decision of what order they will click/attempt the surveys available to them (often optimizing their own incentive earning potential, i.e. maximum incentive for minimal time).

At PureSpectrum, we’ve observed a drastic shift in average survey length over the past two years, to the extent that it made more sense to measure pricing by LOI segments instead of overall price trends.

Accordingly, sophisticated programmatic suppliers are increasingly prioritizing shorter surveys in their yield management algorithms by adding a length of interview (LOI) variable to their yield management, thereby creating the Earnings Per Minute (EPM) metric. By combining these factors, we get a new metric, Earnings Per Click Per Minute (EPCM), which includes Price, Conversion, and LOI. Suppliers now evaluate active sessions/clicks in terms of how much can be earned per minute per session, and how survey minutes can be optimized to drive retention and increase the lifetime value of the respondent.

As more buying transitions to programmatic, buyers who consider price, conversion, and survey length will garner the best delivery at the expense of those who do not. Suppliers will prioritize the best surveys to ensure they can successfully grow and scale their businesses as the balance in the ecosystem is restored. Looking ahead, the path forward is increasingly clear.

Supply dynamics have shifted, and the industry must work to rebuild an ecosystem that is sustainable for all parties involved, including respondents, suppliers, and buyers. Each stakeholder has a role in shaping that future. At PureSpectrum, we continue to invest in technology that strengthens yield management and creates better alignment across the value chain. When studies are designed with these market realities in mind, feasibility improves, and data quality follows.

Learn more about how PureSpectrum enables smarter sampling here.