Context
Microsoft has been driving brand love by creating products and experiences that foster usage, loyalty, adoption, and advocacy. Microsoft’s research has shown that creating positive moments is the pathway to creating love.
The challenge Microsoft has is that, until now, obtaining a deep understanding of these emotional experiences has required qualitative approaches. However, qualitative understanding has, by its very nature, been based on limited samples of participants. And insights obtained from a small number of people, even when they are deep and rich, are a risk for businesses building strategies that require significant investment and that impact business performance.
Microsoft and its research partner, Nexxt Intelligence, believe that the advent of generative AI changes this dynamic. Large language models that underpin Conversational AI, when trained for a market research context, facilitate rich conversational interactions with participants that elicit deep understanding at scale and speed. Microsoft teamed with Nexxt Intelligence| inca to leverage Conversational AI to enable and encourage participants to share their stories and emotions on a large scale, which would allow for these narratives to be quantified and analyzed against behavioral metrics and business KPIs, ultimately driving brand strategy.
Objectives & Hypotheses
The team had two objectives in mind as they conceptualized the project:
Insight Needs: To bridge the emotional narratives and business KPIs, and provide actional insights to help Xbox continue to drive brand love
Research on Research: To understand and quantify the effectiveness and impact of using Conversational AI for market research
Methodology
With those objectives in mind, the team set up an A/B test: two parallel studies, one using the Nexxt Intelligence| inca Conversational AI survey platform, and the other using a traditional quantitative survey platform, among Xbox customers who had engaged with Xbox in the past 12 months. The surveys were conducted in the US in the spring of 2024.
inca Conversational AI Survey: n=1,600; LOI=5 minutes
Traditional Quantitative Survey: n= 4,200; LOI=10 minutes (this survey had other objectives beyond what’s covered in the inca survey)
Nexxt Intelligence’s inca platform blends qualitative capabilities with quantitative functionality. The chatbot-led survey solution uses AI to probe participants’ answers to open-ended questions in real-time facilitating a two-way conversation, much like a human moderator. Beyond the AI-driven inca SmartProbe and back-end analytics, the survey platform features an interactive chat interface, digitized projective techniques, specialized tools, and various quantitative question types with corresponding analytics.