On a summer day in 2007, I visited MoMA (the Museum of Modern Art) in New York and soon found myself, amongst many others, admiring Romanian artist Dan Perjovschi’s sketches all over the walls in the atrium, ceilings to floors filled with his witty reflections on “WHAT HAPPENED TO US?”. It felt like a scene in Shusaku Endo’s novel “Deep River”, where on the banks of the Ganges River, everyone was finding their own discoveries.

I was drawn to a drawing that illustrated tragedy vs. statistics. It hit home instantly. 

With only a few crosses, “tragedy” seemed to bring me up close and personal with what might have happened and unpacked downright heavy emotions: the unfulfilled dreams, the unthinkable struggles, the unreachable helplessness…No matter what the case might have been, I could feel their loss and pain. 

In the image on the bottom, “tragedy” became “statistics” as crosses multiplied, which made me focus my attention on the impact and the scope of what might have happened. In the meantime, I felt I was removed from those acute feelings and became a rational and analytical observer.  

The juxtaposition of those two words resonated with me immediately and showed vividly to me the difference between qualitative research and quantitative research, especially at a time when I had very few opportunities to practice qualitative research after moving to Canada. It reminded me of what I was missing so badly about qualitative research — the stories, the emotions, and the one-on-one personal connection that brought those to life.  

Fast forward to thirteen years later, that drawing still has not faded in my memory. In fact, it emerges every time when concerns on data quality and respondent engagement are raised for quantitative research. 

It is an issue that our entire industry faces; people don’t like to take surveys anymore, assuming that they did once upon a time.  Respondent engagement is important because it determines data quality. Therefore, there have been many innovations focused on “quality checks” that help kick out “bad” respondents.

Take a look at  this question that I came across as I was diligently answering pages after pages of heavy-duty rating scale questions. After pondering on this shockingly simple but strange question for a minute, I figured that it was  a measure to make sure that I was paying attention.  

But what kind of message are we sending through a quality check question such as that? Is it really going to help solve the respondent engagement problem?

Unlikely. It’s the respondents that we should blame. If we don’t make it easy, motivating, or even delightful for respondents to participate in a survey, why should we expect that they would  pay attention? Let’s take a look at a few examples.

Example 1:

I was planning to go to Los Angeles but had to cancel the trip because of the Coronavirus outbreak. I got this survey after calling Expedia. I would think it’s not uncommon to have two flights involved in a trip, and in this case, I booked the outbound flight  with WestJet and the inbound one with Air Canada. How am I supposed to decide which one is “the primary airline carrier your contact related to”?

Example 2:

Am I taking a survey or am I being tested on my mental math?

Example 3:

Could there be a more difficult design for a rank-order question? Plus, who would think “1” means the least important and “4” the most important? If we believe that our respondents don’t read instructions, can we imagine the kind of data we end up receiving? 

If we keep throwing questions that are impossible to answer or hard to navigate to our research participants, we can’t really blame them for not being enchanted, can we? 

In qualitative research, ethnography in particular, we always go the extra mile to make things convenient for our respondents. We make sure that respondents are participating in an environment that’s natural to them. We check in from time to time to make sure that respondents know their opinions are heard and valuable. 

WHAT HAPPENED TO MARKET RESEARCH? As we focus on getting the statistics, we are so removed from the one-on-one interaction that we seem to forget about the importance of building rapport, which is fundamental to the success of any qualitative research. 

To obtain reliable statistics, what we really need to fix is the lack of rapport, the human connection. We need to put our ethnographer hat on. We must become respondent-centric. It’s time for survey research to use some design thinking. 

In a separate blog article, “Are Chatbots the Silver Bullet for Market Research?”, I explored the use of chatbots for market research as an alternative to traditional surveys. Chatbots can be an effective agent for market research and feedback collection only if we fully and deeply leverage its ability to provide one-on-one attention to our research participants. With that rapport, chatbots have the potential to gather, as Perjovschi has, both Statistics and Tragedy. 

We are passionate about building technologies and methodologies for better human understanding. If you also find this topic exciting, we invite you to sign up for our monthly newsletter, “the Nexxt Iteration”, for the latest development in insight-technology and our fresh perspectives.