The Constraint AI Was Supposed to Remove

The case for synthetic research usually arrives as an escape story. Audience feedback has always been the slow, expensive step: recruiting participants, scheduling sessions, moderating, analyzing, with weeks of elapsed time for one round of reactions. An AI persona promises a version of that input in minutes, and somewhere inside the promise sits an assumption that rarely gets said out loud: the research budget this replaces can now shrink. I run the AI side of this trade, as the technical owner of a synthetic persona evaluation framework built inside a global agency, and operating the system has taught me the opposite lesson. The framework has raised the value of the human research underneath it, because the dependency on that research shows up at every layer you inspect.

What a persona is made of

A synthetic persona that produces useful judgment is built long before the model runs. In our framework, a persona profile is synthesized from real research: jobs-to-be-done studies that establish what the audience is trying to accomplish, ranked decision-criteria data such as MaxDiff and conjoint results and win/loss analysis that establish what they trade off, and interview transcripts that carry their actual language. The profile is then scored for completeness and confidence, and the top rating means one specific thing: fully research-grounded, with no significant gaps that force the system to make assumptions on the client’s behalf.

The revealing part is what happens below that bar. A thin profile does not fail. It runs, it produces fluent output, and the output is worse in two predictable ways: the feedback gets more generic, and the system flags more uncertainty. That graceful degradation is the honest behavior, and it is also the trap, because the difference between a persona built on a real segmentation study and a persona built on a creative director’s hunch is invisible in the formatting. Both return confident prose, and only one of them is telling you something about your customers. The quality ceiling of the output is set before the model runs at all, which is why I have come to describe the input side bluntly: garbage in produces confident garbage out, and the confidence is the dangerous part.

This is where the published evidence and the operating experience agree. In the Behavioral Economics Guide 2026, Paola Schietekat, María López, Juan de Rus, and Dario Krpan published a study of synthetic participants, “Synthetic Participants: What Are They for in Business and How Can We Make Them Better?”, which concludes that synthetic participants “are increasingly useful tools for specific, well-defined tasks when their construction is grounded in empirical data, explicit psychological theory and transparent calibration procedures.” The same study calls periodic recalibration against human reference data “not an optional refinement but a structural requirement.” Read those conditions as a procurement officer would. Grounding is consumption of human research at build time. Recalibration is consumption of human research on a schedule, forever. The credibility conditions for the technology that was supposed to reduce your research dependence are written in units of research.

Even the audit consumes research

I have argued earlier in this series that an honest system claims a narrow scope, one persona evaluating one piece of work to inform human judgment, and that its judgments must carry an evidence trail a reviewer can reconstruct. The calibration layer is where those commitments get tested against reality, and it is also where the research dependency becomes unavoidable, because the only way to know whether a synthetic persona resembles the audience it simulates is to compare it against real humans.

The layer that does exactly this began as a manual pilot: a calibration tool, first run by hand against real concept-test transcripts from a live client study, that scores how closely the synthetic persona’s reactions align with what human participants said about the same work, across meaning, sentiment, salience, stance, and distribution. We are now scaling it into a repeatable production capability. The tool’s own documentation carries the limitation that matters for this argument. A high similarity score means the synthetic persona mirrors the human data it was checked against, and it validates nothing about whether that human data was representative in the first place. Transcript sample size, participant selection, and moderation approach all bound the reference baseline, and thin transcripts explicitly reduce the confidence of the score. Follow the chain and the conclusion is hard to avoid: the persona depends on research, the check on the persona depends on more research, and the quality of both is inherited rather than computed. There is no layer of the system where the human data stops mattering.

The economics of this are hybrid, and the literature is explicit about it. In “Synthetic Audiences as Proxies for Human Users: A Systematic Literature Review” (IEEE Access, 2026), Theodoros Lappas and Apostolos Filippas review one hundred studies of LLM-based synthetic audiences and find that “synthetic responses are most useful when they are explicitly anchored to human ground truth.” They report one study in which combining predictions from multiple models with modest human samples increased precision while cutting required sample sizes by 20 to 30 percent. The review is careful to present that figure as an illustrative estimate from a narrow experimental context. The number deserves a careful reading, because both halves of it carry a message. Research goes further than it used to: a small, well-designed human study can now calibrate a synthetic capability that keeps evaluating work long after the study itself would have been shelved. And none of it becomes optional: the reduction is in sample size, and the requirement that humans anchor the system does not go away at any percentage. The loop is redesigned rather than exited.

What changes for leaders

If the dependency runs this direction, several standard moves are backwards, and the corrections are practical.

Research budgets should be treated as infrastructure investment rather than a cost that AI adoption offsets. The instinct to fund the AI capability by trimming the research line cuts the input that determines the capability’s quality, and the cut shows up later as generic personas and unverifiable calibration, where it is expensive to diagnose. The stronger posture is to hold or grow the research line and change what it buys: fewer, sharper studies designed to work harder.

Existing research should be inventoried as an asset, because most organizations are richer than they think. Past concept tests, surveys, and A/B results are calibration baselines waiting to be used; historical focus-group transcripts are reference data a similarity check can run against. An archive that was gathered for one decision and filed away can now anchor a standing capability, which changes what the archive is worth and argues for knowing exactly what is in it.

The next study you commission should be designed for dual use. A study designed only for the insight it delivers this quarter and a study designed also to ground and calibrate synthetic personas are different artifacts, and the difference is mostly discipline that is cheap at design time: ranked rather than unranked preference data, transcripts captured and retained in usable form, segments labeled clearly enough that a profile built from them can show its provenance. The second study keeps paying after the first one has been presented and shelved.

And any vendor selling synthetic research should be asked one question early: show the research these personas stand on. A persona whose provenance cannot be produced is an opinion generator with a demographic label, and no amount of output polish changes that. The follow-up question is the calibration path: what human reference data the system has been checked against, and what the plan is for checking it against yours.

I will name the objection directly, because it is fair: an argument that research becomes more valuable, made by someone whose system consumes research, can sound like a vendor defending a budget. My incentive actually points the other way. Operating the AI side would be simpler and cheaper if client research were optional, and the persona-building pipeline would be shorter if a demographic sketch were enough. The dependency is simply what the system’s own quality scoring, the published evidence, and the calibration tool’s documented limits all report, and pretending otherwise would relocate the failure to the place it costs most to find.

The durable consequence is worth stating plainly. Synthetic evaluation capability is going to commoditize, because the models underneath it are available to everyone and improving for everyone at once. What does not commoditize is the research feeding it: your customers’ jobs, trade-offs, objections, and language, captured well enough to ground a persona and audit it afterwards. That corpus is proprietary by nature; it compounds as studies accumulate, and every improvement in the models makes it more usable rather than less necessary. The market is currently pricing primary research as the cost that AI removes. The organizations that read the dependency correctly will spend the next few years quietly buying the asset everyone else is discounting.