In Their Eyes: How Understanding Respondent Perceptions Can Help Improve the Research Process

Have you ever wondered what participants think of the qualitative research process? I did. When curiosity overwhelmed me, I undertook primary research to uncover what individuals who had participated in any type of qualitative research in the past year thought of the process, from the initial recruitment contact to payment of honoraria. We partnered with M3 USA Corporation on this research effort. The goal was to see what could be done differently to improve research quality and ensure that participants have a good experience.

What Was Done?

Two phases of research were executed. The first phase was a qualitative bulletin board with 15 physicians and 13 patients from the US, UK and France. It was a single board conducted in English. (Thank you very much to my French participants!) The second phase was a 20-minute quantitative, native-language survey that was designed based on additional qualitative research conducted in Brazil, France, UK and the US with physicians and patients as well as with physicians in China.

To participate in the research, respondents needed to have participated in at least two qualitative research studies in the past year. These studies could have been IDIs (in-depth interviews), focus groups, TDIs (telephone depth interviews), mobile and/or bulletin board-based research.

To acknowledge, I focus on health care research, so the participants are physicians and patients. However, in talking to colleagues that work in B2B and health care, they have told me that the physicians essentially behave much like B2B respondents. And, at the end of the day, patients are consumers. So, I firmly believe that these learnings extend well beyond health care research.

What Do Respondents Want?

This research demonstrated that respondents want four key things from researchers:

1. To know what the heck is going on
2. To be respected
3. To be comfortable
4. To be rewarded

1. To know what the heck is going on. Participants want to understand the “what” at every stage of the process. 

   During recruitment, beyond knowing the type of research it will be, participants also want to know where it will take place, at what time and what the honorarium is. Plus, they want to know: What is the research all about? This is where we often fall short of expectations.QRCs tend to avoid providing too many details about the “what” because we are concerned about individuals cheating the screener or biasing invited participants. However, the “what” can play a significant role in respondents’ decision to respond to the invitation. They want to know that they are going to be useful and valuable contributors to the research. Therefore, respondents who could potentially be valuable contributors may not respond to the invitation without knowing more about the “what”.

   It is also perceived that researchers are being vague on purpose as they know that there are certain types of research that individuals do not like to participate in.

   “Occasionally the information regarding the research topic is vague. A bit more detail would allow us to decide if we are able to provide a good, informed opinion.” – Physician

   While providing more details in the invitation may be challenging, we could try to be a little more creative and a little less vague. But, more importantly, once the participant has been screened in, why can we not satisfy their curiosity and provide more robust information about the research in the final invitation? There are ways to not bias the respondent for the fieldwork while making them excited to participate.

   During fieldwork, they want to understand the objectives of the research and what decisions will be made based on the research. They want to understand this because they want to be helpful. There may be times we cannot provide the details at the start of the interview as we do not want to bias the respondent, but we could provide them more insight into what we are trying to achieve as the interview progresses. I have incorporated more robust introductions or release of information during the interview, and I have found that respondents seem more engaged and the insights are richer for it.

   They also want to understand the objectives to help them understand why we are asking the questions we are asking. While it is a moderating technique to ask questions from a variety of angles, respondents pick up on this, and they do not always know what else we are looking for. This leaves them frustrated and feeling that they are not giving us what we want. I have become even more conscious of this during my interviews and try to avoid it.

   “Talking about the same thing for too long gets boring and you lose interest. Plus, after what you say, if they keep asking you to extend and what else, what else, you start making things up just to get them to stop asking you.” – Consumer

   “Sometimes there is only so much you can say about a particular question, but often the interviewer wants more…it becomes very repetitive and we seem to be going round in circles. No agenda had been set out and, if it had, maybe I would have been able to elaborate.” – Physician

   They also are confused by projective techniques; they do not understand what we could possibly be getting out of them, particularly the professional audiences. But, I know I have found them valuable, even with physicians. To address this, I have begun offering a quick analytic synopsis of the exercise. It takes me no more than 30 seconds to quickly summarize what I have learned from the exercise, but it also gets the respondent’s buy-in to the exercise that was just done—we really did learn something from it.

   At the end of the research, they want to know what happens next and, specifically, when they will receive their honoraria. I admit that I did not always know that answer as my fieldwork partners pay the respondents on my behalf. I now make more of an effort to know this and to pre-emptively tell them at the end of the interview. They find it irritating and embarrassing when they must track down their honoraria.

2. To be Respected. Key to this is that they want their time respected. This is a multi-fold issue:

• Screeners are getting longer. As such, the screening process takes longer, particularly since respondents go through the process multiple times for verification purposes.
• Technology may result in additional time spent by the respondent (e.g., needing to do a technology check in advance of the interview or to log on in advance for a webcam interview).
• The interview itself may take longer than originally scheduled.The onus is on researchers to manage time expectations, which can be done by strictly enforcing the idea that screeners only include questions that terminate unqualified respondents or that place respondents into quotas. Any questions that are information only should be captured in a different manner. There may come a time when we need to compensate participants to complete screeners above a certain length. In fact, some fieldwork agencies are already implementing this strategy.

  We can provide better time estimates for the total commitment to the project, including time for any prep or technology checks.

  Finally, continuing to push back on clients when the discussion guide is becoming unmanageable for the planned interview length is something we need to do. Some in the insights departments of pharmaceutical and biotech firms whom I have spoken with acknowledge that their marketing teams try to accomplish too much in a single study. They recognize the need to educate their teams on what can reasonably be done in a qualitative research study.

  “[I don’t like it when] my time is not respected and the group or interview runs over.  Having to excuse yourself is embarrassing and it should always start and end on time.” – Consumer

  “What I don’t like is that someone calls 10-15 minutes [beforehand] to test the system and to connect me as I have already tested the system and I know how to connect.” – Physician

  Respondents, particularly consumers, also want to be listened to and engaged with. As the screening process becomes more automated, while efficient, it is dehumanizing the process. And, during the interviews, individuals, particularly consumers, spoke about instances where they felt that the interview was too “formulaic” or that the moderator was “patronizing”.

3. To Feel Comfortable. Respondents spoke of being prompted to answer screening questions in a specific way, which made them uncomfortable. They even noted that it seemed “unethical” to them. We know that fieldwork agencies are already aware of this issue and work to avoid it, but there is still more to be done.As we use more and more technology as part of the research process, we need to make sure that the respondents are comfortable with the technology and receive the instructions needed to successfully complete the research.

   “My worst experience was being escorted out of Home Depot because as part of my video I included a Home Depot clerk who wasn’t at all happy. They wanted to take my video card and I told them this wasn’t Russia.” – Consumer

   “The technical side of things did not go well…I emailed, called and sent a message via the Google link to say I was there but having trouble. It took them so long to reply that I felt forgotten and unsupported and almost walked away.” – Consumer

   The moderator must be cautious that he or she is not causing the respondent discomfort by:

• Seeming disappointed in the respondent’s answers.
• Asking what appears to be leading questions.
• Not managing group discussion/letting a respondent dominate.
• Not knowing the lingo for a technical category.“It was a study about chocolate mousse yogurts.  The container was made of glass and looked more sophisticated than usual ones.  I said that because it didn’t look like a cheap quality I could easily serve those to guests as deserts and he said really?  Not sure he wanted to be invited over.”- Consumer

4. To be Rewarded. An honorarium is the biggest driver to participation for both professionals and consumers, according to the research we conducted. However, participants find that getting paid, when the research is not in-person, can take up to three weeks for consumers or up to five weeks for professionals. Both groups would prefer to receive payment, on average, within two weeks.When assessing whether the honorarium is sufficient, in-person participants factor more than the time in the front room. They also add in ancillary factors such as transportation costs, babysitters, travel time, preparation time and waiting room time.

Final Thoughts

What respondents want from researchers may seem obvious; they want to be viewed as valuable members of the research team and to be treated as such. And, they are valuable. It is important for qualitative researchers to remember the total participant experience as we juggle the various pressures put on us when designing research, including timelines and budget. Be an advocate for respondents, because without them there is no research.