Synthetic Research: When Are Findings Decision-Ready?

Editor’s Note: Following our recent IIEX.AI event, I’m thrilled to share Ashley's synthesis of one of the conference’s most vital themes: how researchers move synthetic data from rapid generation to real-world decision confidence. A special thank you to Nancy Brigham (Dynata), Jim Cummings (Viewpoints AI / Boston University), Dr. Jannik Meyners (quantilope), Daniela Lipkin (VERVE), and Feranmi Muraina (The Magnum Ice Cream Company) for sharing the practical framework and validation guardrails featured in this piece.

---

A synthetic respondent can produce an answer quickly. Deciding what that answer is worth takes more work.

For market researchers, that distinction is becoming central to the practical use of synthetic research. In this context, the term encompasses AI-generated responses, simulated consumers, and category twins. Because each application serves a distinct purpose, each requires a different approach to validation.

A plausible response may help a team explore an idea, challenge an assumption, or prepare better questions. Whether it can support a business decision depends on the evidence behind it, the way it has been tested, and the consequences of being wrong.

At IIEX.AI on September 23–24, practitioners described approaches that move beyond the promise of faster, cheaper research. Their focus was on validation, purposeful iteration, and boundaries: what synthetic findings can support, where human research remains essential, and when a result should be set aside.

The useful question is becoming more specific: What has this finding earned the right to inform?

What Makes a Synthetic Finding Useful?

Generating an answer is only the beginning. Researchers still need to establish whether the output is suitable for the intended application.

As Nancy Brigham, Vice President, Head of Research Science at Dynata, explained:

“Quality validation is critical with synthetic data. Synthetic data will be generated by a model. Whether you can use the output for reliable business insights is another question entirely. So, validation is actually far more important than generation.”

That distinction changes how teams should evaluate synthetic research. An articulate response, a convincing persona, or a familiar-looking pattern does not establish that the result reflects the people or market a business needs to understand.

An exploratory conversation with a category twin serves a different purpose from a synthetic dataset intended for analysis. Evidence that supports one use does not automatically establish suitability for another.

Before evaluating the output, teams need to define its job. Is it helping generate hypotheses? Improve concepts? Explore possible reactions? Support a consequential investment? Each calls for a different level and type of evidence.

What Should Researchers Test After the First Result?

A first synthetic readout can reveal an interesting pattern. The next step is to investigate how easily that pattern breaks.

Jim Cummings, Head of Product at Viewpoints AI and Associate Professor at Boston University, described iteration as a way to build confidence, provided each additional study addresses a specific weakness:

“What’s moving you up the ladder isn’t simply the idea of, I did more studies, and they all agreed with what I found. Great. It’s rather that each study is designed in a way to target a specific threat to validity, a specific threat to your confidence in that initial finding where you could have been wrong.”

For researchers, this means giving every rerun a purpose. Could the finding depend on how the question was phrased? On the information supplied to the simulation? On an assumption about the audience?

Those questions turn iteration into a research design task. Before running another study, identify what could undermine the finding and what result would change the interpretation.

Repeated agreement can otherwise create confidence without resolving the original uncertainty. Several outputs may share the same underlying limitation.

Conflicting results require attention, too. Cummings described an unexplained reversal as a reason to step back, revisit the simulation’s inputs, or conduct additional human research. Continuing to rerun a study until a preferred answer returns would leave the underlying problem unresolved.

The practical value of iteration lies in learning where a finding holds, where it changes, and whether the team can explain the difference.

What Human Evidence Sits Underneath the Simulation?

The relationship between synthetic research and human research is more specific than a replacement debate suggests.

In discussing category twins, Dr. Jannik Meyners, Vice President of Data Science & AI at quantilope, emphasized the quality of the underlying evidence:

“The twins are only as good as the human data underneath it. And it’s always needed for any application of synthetic data to be built on good and also fresh and deep human data.”

That makes the source material part of the assessment. Researchers need to understand whose experiences are represented, what the original research covered, and whether the evidence remains relevant to the current question.

A twin built to explore a category cannot automatically answer every question about that category. Its usefulness has boundaries shaped by its inputs and intended purpose.

For teams using these tools, making those boundaries visible is part of responsible delivery. Stakeholders should understand what the simulation draws on, what has been tested, and where additional human evidence is needed.

Does Validation Continue After Launch?

Validation also needs to account for change. Consumer behavior evolves, new research becomes available, and simulations themselves may be updated.

Describing work with simulated consumers, Daniela Lipkin, Director at VERVE, explained:

“And importantly, validation, it isn’t something that we do once and then forget about. We continue to test them and monitor the simulations as they evolve.”

Lipkin also described testing against real consumer research and examining responses against real verbatims for characteristics such as tone and language.

These checks address different questions. A response may sound like a consumer without accurately anticipating what consumers would say or do. Natural language is useful, but its realism cannot carry the entire validation burden.

For research teams, ongoing validation creates an operational responsibility: deciding who monitors performance, what prompts a review, and when new human research is needed to refresh the evidence.

A successful initial test gives teams a starting point. Continued use requires continued scrutiny.

When is Directional Evidence Enough?

Directional findings can have substantial value. They can help teams identify tensions, refine language, improve an early concept, or decide what deserves further investigation.

A finding may be useful for choosing which concepts to investigate next without being sufficient to justify a launch. Readiness depends on the decision, the validation performed, and the uncertainty that remains.

Feranmi Muraina, Global Manager, Digital & Communities Insights (AI Transformation) at the Magnum Ice Cream Company, described a guardrail used within the organization:

“And then we decided that go or no-go calls, binary decisions, in essence, would always have a human input at this point in time.”

In the surrounding discussion, Muraina explained that the company uses synthetic panels for augmentation and optimization, then checks final optimized ideas with real consumers. Human involvement therefore includes gathering consumer evidence before committing to significant decisions.

Muraina presented this as an organizational guardrail for the current application, rather than a universal judgment about synthetic capability.

That offers a useful model for other teams: define the boundary around the decision being made. Early exploration and a major launch commitment carry different consequences. Their evidence requirements should reflect that difference.

“Decision-ready” should describe readiness for a specified decision, with clear limits on how the finding can be used.

Can the Validation Process Tell You to Stop?

A credible validation process must leave room for rejection.

Brigham described grading synthetic output to establish what analyses it can support, including withholding data when validation is insufficient:

“We advance only when the evidence supports it, and pause or stop when the validation is weak.”

That willingness to stop is an important test of the process itself. If every finding eventually receives approval, validation risks becoming a justification exercise.

The practical standard emerging from these discussions is demanding but familiar to researchers: define the intended use, test the weaknesses, document the limitations, and match the evidence to the decision.

Synthetic research can make additional exploration and testing more accessible. Its value depends on how teams use that opportunity to investigate uncertainty.

Before a synthetic finding enters a recommendation, researchers should be able to explain what it is grounded in, how it was tested, which decision it can support, and what would require further human research.