Let’s Get Conversational!
There’s a dirty little secret that lots of people working in market research, specifically quantitative market research, don’t want to admit to. That secret? A lot of market research surveys – the very surveys used to find insights used to make decisions – are, to be blunt, a bit rubbish.
They’re rubbish because the respondent experience is poor – dull, unimaginative surveys don’t lead to insightful, thoughtful responses from respondents. Even when the survey experience is better, quantitative research is best at revealing facts – the what and not so much the why.
But what if there was quantitative research that provided respondents an enjoyable, interesting experience and could uncover the why and the what? Wouldn’t that be great?! Well, the good news is, there is – and it’s called conversational surveys.
What are conversational surveys?
Conversational surveys, also known as conversational AI, use a chatbot to lead participants through the survey. The experience can be more engaging for survey participants compared to traditional online surveys. Good conversational surveys establish a rapport with participants, make them feel listened to, and mimic the experience of having a conversation!
The experience for participants is more akin to having a WhatsApp conversation with someone, rather than completing a traditional online questionnaire. This results in more thoughtful answers and better data. Participants can be asked a range of closed, quantitative questions and open-ended, qualitative questions. The conversational survey format is device neutral but, unlike many online questionnaires, works well on mobile devices.
A conversational survey becomes conversational AI when it uses AI to probe participants’ survey answers, and to cluster and theme the verbatim responses. Conversational AI leverages the latest advances in machine learning and natural language processing to do this.
Let’s look at how conversational surveys work!
Conversational surveys also use images to enhance the participant experience. For example, in the illustration below, the survey (1) uses emojis rather than words to understand how people feel about a topic and (2) embeds photographs of famous monuments in a question about countries people would like to visit.
Quantitative: The what
The above examples are quantitative questions and can provide quant data from hundreds or even thousands of participants. Most types of quant questions – single choice, multiple choice, scales, ratings, image/attitudinal batteries – can be asked in conversational surveys. These typical quant questions are made visual for participants and are designed to deliver an appropriate experience for all devices: mobile, desktop/laptop, or tablet. This can be seen in the examples below.
Qualitative: The why
The biggest difference between a conversational survey and a traditional online questionnaire is the much greater use of open-ended questions in the former. We find the chat-based format of a conversational survey allows for improved responses to open-ended questions over a traditional online questionnaire, both in the quality and quantity of verbatim. This is further enhanced by the good use of probing, via follow-up questions, to go beyond the first response and gain deeper insights from participants.
Whilst we would never go as far as to say that conversational surveys can be a replacement for the deep qualitative insights that can be obtained from qual depth interviews or focus groups, the insights from conversational surveys have a strong qualitative dimension through the rich, open-ended responses that can be obtained.
Researchers working with conversational surveys soon get used to structuring open-ended questions in more projective ways. For example, rather than asking…
“What, if anything, could have been better about your stay at Hilton Hotels?”
…a projective such as the following will lead to richer, more insightful verbatim:
“If you were the CEO of Hilton Hotels, what are the key changes you would put in place to improve the guest experience?”
We consider conversational surveys to be Qual x Quant. They use qualitative principles of building rapport and engaging participants in order to obtain a greater depth of understanding at a quantitative scale.
Where the AI comes in: Probing
One of the ways that conversational AI is more than just a conversational survey is how the probing works. Not all conversational AI approaches probing in the same way, and the degree of ‘intelligence’ in the AI differs. However, the AI will work to ‘read; what the participant says so that the chatbot delivers an appropriate probe in response.
Depending on the nature and depth of the participant’s initial response, the probe used will be different. To give examples, AI probing might work in the following four different ways:
Uninformative responses – if a participant gives a one-word or brief answer, as is often the case with online research verbatim questions, the chatbot probe asks the participant to be more specific in order to elicit more feedback.
Gibberish – if a participant enters gibberish, another common issue with online questionnaire verbatims, the chatbot pulls them up on it and seeks a better response.
Smart probing – when a participant has provided a good first response, the chatbot will ask for more information. Here, the AI will often playback a word or phrase from the initial verbatim, and the chatbot will ask the participant why they said that.
Targeted probing – here we set a pre-defined instruction for the AI to probe on a particular word or phrase we are interested in if a participant mentions it in their initial open-ended answer. For example, we may seek to probe further on mentions of a brand name or a product attribute.
Where the AI comes in: Analysis and speed
Whilst a large volume of open-ended, unstructured data brings the potential for deeper insights, it also brings the challenge of how to analyse it all, particularly when project timelines are tight. This is the second area where the AI comes in – to cluster and theme verbatim feedback. Natural language processing (NLP) is the key ingredient conversational AI utilises. You can think of it as using machines to understand human language – which in this context means automatically clustering text into meaningful themes.
With the best conversational AI, this process happens in real-time. This means that conversational AI can offer all the benefits of the latest automated market research systems, such as live reporting of quantitative data to dashboards, but it also does this for the open-ended, qualitative data.
So, having learned more about conversational AI, I hope that, like me, you’ll become a convert. No more dull, rubbish surveys – conversational AI is the future of online research. A better participant experience, better insights, and better business decisions. Hurrah!