Optimizing Use of AI for Insights

Diverse business team in a modern office during a meeting, discussing artificial intelligence strategy. Concept of AI innovation

“We have an AI mandate—now where should we use it?

There’s no debate that the adoption of AI across the corporate world has accelerated and will continue to do so, including for Insights and Marketing Strategy leaders. In fact, many of HawkPartners’ clients are increasingly facing “AI mandates”: internal pushes to leverage AI as much as possible across all functions and business units. This tendency – often driven by senior corporate leaders – is understandable given that AI seemingly offers the promise of better, faster, and cheaper…for everything!

Our Insights and Strategy clients no doubt are feeling this pressure to utilize AI too. But as they attempt to navigate the fast-growing (and at times confusing) lineup of AI tools becoming available for insights work, it’s frequently unclear how and when these AI tools should be adopted. HawkPartners has always been methodology-agnostic, and now – more than ever – we are compelled to offer our experience and judgment to help clients sort through these dizzying choices. For what it’s worth, we view these myriad AI-driven techniques as valuable options that, when applied intentionally and appropriately, can maximize the value that clients get from their research. In some cases they offer significant reductions in turnaround time, and other times they can help make results more actionable. Sometimes they simply provide a less expensive alternative, and in a few cases they can potentially offer all of the above! But just like any research tool or methodology, when applied in the wrong context or for the wrong reasons, AI-driven tools can compromise the quality and rigor of the desired insights, putting the reliability of resulting business decisions at serious risk. Choosing and applying them correctly is essential.

This article focuses specifically on three specific AI-enabled research and insights methodologies whose use is growing quickly and that our clients ask us about most: AI Personas, Synthetic Survey Respondents/Data, and AI-Moderation. We discuss what they are, how they should best be used, and some meaningful limitations clients should be aware of before adopting them. But these are only a representative handful of the options available, and many of the same principles can be applied broadly. It’s also worth mentioning that we are taking for granted the broader operational use of AI, such as synthesizing foundational knowledge on a topic, drafting reports and creating project documents, as well as improved efficiency of general business activities.

Let’s take a look at each of these three methods.

AI Personas

AI personas are an interactive representation of a specific customer or audience segment, providing conversational experiences to help bring learnings about the subgroup of customers to life. In particular, clients can query the Personas about a wide array of topics using natural language conversations (unlike the static personas of yesteryear which often were little more than descriptive PDF or PPT documents). Conversational AI personas allow brand, insights, and strategy teams to interact directly with them, giving client teams the ability to generate hypotheses before primary research, pressure-test creative ideas, and better understand and internalize the needs and sentiments of target customer segments.

For example, we have seen clients build healthcare professional (HCP) personas for their own internal exploration. We have also helped clients develop even more in-depth personas by combining prior research, internal data (e.g., CRM data), publicly available data, and new proprietary primary research tailored to fill important gaps in the existing datasets.

While HawkPartners views AI personas as a valuable addition to the portfolio of insights methodologies, we also recognize their limitations. Most important, they are only as good as the data they are built on. When AI personas are developed based merely on publicly available data (typically through LLMs), they can be helpful for more basic tasks like generating hypotheses before conducting novel primary research, but in most cases shouldn’t be relied on for critical business decisions. AI personas are far more powerful when built using proprietary client-owned data, including qualitative and/or quantitative research, CRM inputs, and other first-party data. However, even then, when data quality or availability is low – clients risk creating personas that can create plausible-sounding answers but aren’t sufficiently grounded in real data. And because personas are often trained on historical data, they should not be treated as reliable predictors of future behavior, nor as definitive evidence of how customers will respond to novel products or unique marketing messages. They can, however, still be useful for exploring reactions, pressure-testing early ideas, and identifying key questions to validate through subsequent primary research.

Synthetic Respondents

Synthetic respondents, also known as synthetic sample, are AI agents built to mimic human respondents in quantitative primary research. While synthetic respondents may be drawing the most media buzz among the three methodologies in this article, in our view they warrant the most caution given that significant limitations of this approach remain. As a result, we currently don’t recommend them as a pure substitute for primary research. There are several use cases where they can be appropriate: as a fast directional read, a low-stakes pre-test, or a gut check before committing budget to large-scale primary research. Used these ways, and with clear eyes that further primary research with humans is probably still needed, they can certainly have a place. But, as of today, their value is probably more exploratory than confirmatory.

In parallel studies that we’ve run with both human and synthetic respondents, we’ve seen a mixed bag of results: in some cases, the major themes surfaced by the research are consistent across methods, but the magnitude of differences by subgroups can diverge significantly (with synthetic respondents typically resulting in less differentiation); in other cases, reported attitudes and behaviors varied wildly across methods. This often occurs because synthetic respondents are less likely to use the extremes of scales and thus results tend to converge on the mean, as opposed to representing the full spectrum of views of a diverse swath of respondents. The independent literature on synthetic research paints a similar picture, with academic studies finding the gains concentrated in a single strong signal (i.e. general attitudes) while more specific attributes flatten out (e.g., brand assessments). Even some of our partners who offer synthetic data concede that critical business decisions should be confirmed with human research. Though the consensus on synthetic data is still forming, and will undoubtedly shift over time, at the moment we don’t feel that synthetic respondents are quite ready to replace humans for the majority of primary research.

AI Moderation

AI moderation is another methodology that is generating significant buzz. It spans a spectrum, ranging from follow-up probes in open-ended survey questions, to conversational surveys, to stand-alone interviews that an AI agent conducts on its own. Overall, the methodology promises scale, speed and cost effectiveness, and we feel it has a valid place in research. However, like the other emerging methods, it should be employed with caution, for the right use cases and with the appropriate respondent populations.

As an “add-on” or augment to an online survey, open-ended probes and follow up questions administered by AI can add depth and context, offering efficient access to the “why’s” behind the numbers. Furthermore, it can be done just-in-time and at the convenience of respondents. This method is a great option for enhancing the value of quantitative surveys, especially for clients interested in a low-risk way to introduce AI.

When considering AI moderation for standalone interviews, we feel it can be useful when topics are very well-structured and don’t require flexibility/adaptation in moderation – respondent pre-work/homework is a good example – especially for customer journeys, diary studies, or rapid iteration concept tests. However, when conducting deeper emotional insight research or complex decision journeys, or discussing sensitive topics that benefit from human empathy, we’ve found that AI moderation platforms frequently fall short. In these conversations, where probes must be nimbler than variations of “tell me more about that,” deep understanding of the category is a value add that only an experienced human moderator can bring: knowing which thread to pull, recognizing when an offhand comment is the real signal, and how to “hear” what a respondent is not saying.

The respondent audience should also be taken into consideration when considering AI moderation. It’s best used with easy-to-recruit, gen pop audiences with well-documented needs and behaviors. However, when a study involves niche audiences such as rare patient types or stakeholders who are expensive and hard to recruit (think Interventional Cardiologists or C-suite executives you’re paying a large incentive to reach) the reliability and experience of a human moderator are critical. We find that in these interviews respondents are more likely to provide deeper and more insightful comments (i.e., beyond the expected) when the questions feel more conversational/less formulaic and the moderator can demonstrate knowledge of and engagement in the topic at hand.

Conclusion

At HawkPartners, we view AI as a powerful technology to be applied deliberately where it clearly adds value – rather than by default. As shown, the value of AI comes from matching the technologies’ capabilities and strengths to the right use cases. That’s where we, HawkPartners, can help. As you weigh where AI fits in with your next project, we’re glad to think it through with you: to help evaluate a use case, test it against your objectives, and find the places it adds real speed and value without compromising the integrity of the work. We look forward to the conversation!

Get timely articles, perspectives and trend updates

"*" indicates required fields

This field is for validation purposes and should be left unchanged.
Name*