Overwhelming curiosity drove me to understand the experience of qualitative research participants. The results, discussed in the article In Their Eyes: How Understanding Respondent Perceptions Can Help Improve the Research Process, published in the Winter 2018 edition of QRCA VIEWS, made it apparent that, as an industry, there is room to improve the participant experience.
While I had my opinions, I asked research buyers and providers about what was uncovered and what it means to the market research industry. Based on those conversations, this article outlines six areas where qualitative researchers can improve the research process to make it a better experience for our participants.
1: Be More Forthcoming about the Topic in Our Research Invitations
I know this is a tricky topic. There are several reasons why researchers do not want to reveal too much too soon: biasing the research, negatively impacting recruiting when we know the topic may have less appeal (e.g., testing advertising with physicians) or enabling cheaters/repeaters.
Among physicians and consumers, 25% and 22% respectively, rank the research topic as the most important piece of information to include in a research invitation. To provide some additional context: Among consumers, the research topic is chosen as the most important piece of information by more individuals than other key pieces of information (e.g., honoraria, time commitment, type of qualitative research). Among physicians, this is selected as frequently as honoraria as the most important piece of information to include.
Researchers at fieldwork agencies are not surprised as they have found being able to provide details, even a little, can help garner participants’ attention and increase their completion success rate. It also helps keep respondents from requesting to be removed from their panel when the research does not live up to expectations.
Researchers note that creativity in our invitation is the key to addressing this. We need to allow participants to have a better understanding of the research topic without giving away too much information. For example, we might include more detail in the research invitation like this:
Great! We’d like to invite you to participate in a marketing research study that’s all about VITAMINS. It will take place June 9 & 10, and you can take part whenever and however it’s convenient for you. During the discussion, you’ll be getting a sneak peek at all kinds of new ideas about vitamins, and you’ll be interacting with other fellow vitamin enthusiasts.
Along with a fun or interesting sounding invitation, one researcher suggested “acknowledge the elephant in the room” when a research topic might be off-putting. Doing so would allow participants who truly do not like a type or topic of research to self-select out and save everyone painful and unproductive interviews.
2: Streamline the Screening Process
Respondents complain about the time they spend qualifying to participate. They find the process repetitive (e.g., screened online, rescreened when scheduled, and rescreened during confirmation or at the facility). Of course, there are reasons for this process, but respondents do not know them—they simply know they are being asked the same questions again and again. Researchers need to consider if repeated screening of the respondents is truly necessary.
The screener length is another complaint. Physicians and consumers feel that, on average, at most seven and nine questions, respectively, should be asked to determine whether they qualify. This is not always feasible, but field agencies (who see a lot of screeners) have a few suggestions:
- Avoid redundancy (e.g., instead of asking the type of cancers seen, then ask how many patients they see with each of the selected cancers—just ask how many patients they see with each cancer type. If they say 0, then they do not see that cancer.).
- Do not use screeners as a means of data collection; do not ask questions that do not qualify or determine a quota group. Any other questions should be included in the fieldwork/interview.
- See if the agency records any standard demographic information as part of their panel. If so, rather than ask those questions, just confirm that nothing has changed (e.g., panels may have the individual’s age or a physician’s number of years in practice stored).
3: Respect the Time Requirements that Were Quoted in the Invitation
Twenty-five percent of physicians and 43% of consumers agree that they often participate in research that exceeds the amount of time that they were told it would take. While they may seem obvious, suggestions to manage time include:
- Make an accurate assumption at the outset on how much time will be needed to meet the objectives.
- Moderators should take responsibility for ensuring that the research concludes within the time quoted.
- Experienced moderators know when a discussion guide is too long; guideline lengths need to be managed during the development process.
- Research buyers acknowledge that they also need to do a better job of managing team expectations on what can realistically be accomplished in a single-market research study.
- Always ask when it looks like you’ll need to go over on time, and pay additional honoraria to compensate when more time is needed.
- Ensure that the invitation outlines any additional time needed (e.g., to test technology) and compensate the respondent for that time.
- Pay additional honoraria for travel time (not just a token increase in incentive).
4: Be More Forthcoming about the Research Objectives
Like providing more details in the invitation to the research, this is a balancing act. Once they are participating in the research, respondents want to know what we are looking to understand with our research. Why are they there? What decisions will be made as a result? They want to know this so they can be as useful as they possibly can. As one physician put it, “Sometimes there is only so much you can say about a particular question, but often the interviewer wants more…it becomes very repetitive and we seem to be going round in circles. No agenda had been set out and, if it had, maybe I would have been able to elaborate.”
It is a balance of what to share and when about the goals and objectives. We want to maximize unaided and organic responses…but…a more detailed background/context could allow for a more in-depth and detailed discussion, so the participants are “not playing a guessing game.”
Researchers struggle to provide definitive suggestions, as this is a project-by-project decision. To avoid biasing the research, information could be provided in stages throughout the research rather than in the beginning. Also, employing methods that get at ideas from a different angle rather than continuous probing may get the depth without appearing repetitive.
5: Pay in a Timely Manner
Honoraria is the biggest driver to participation. However, there is a gap between payment expectations and reality. Physicians report receiving payment for non-in-person research, on average, five weeks later, while consumers report it takes three weeks. However, both groups think it should take only two weeks. More interestingly, 29% of physicians and 31% of patients agree that there are likely qualitative market research studies for which they have never received the honoraria because they lost track of the fact that it was owed to them.
Researchers raise two key issues as driving payment delays. First, longer research buyer payment terms (e.g., 120+ days in some cases) have a trickledown effect, with market research firms being slower to pay the recruiting agencies and then the recruiting agencies being slower to pay honoraria. As one field agency said, “We are not banks!” The other is an internal accounting practice where everything is paid at one time each month. These policies are set by departments outside of market research that do not understand these intricacies.
However, we could better communicate the timing of honoraria payment, so participants know when they can expect to be paid by both the recruiters and the moderators. Consider using alternative methods (e.g., incentive payment companies) to self-pay respondents. However, this can pose challenges given the increasing regulations regarding PII (personally identifiable information).
6: Increase Participant Confidence in the Process
Only 44% of physicians and 43% of consumers agree with the statement:
“I feel safe participating in market research and that my identity is protected, and my information is treated in confidence.”
Researchers agree that this is low and generally feel this percentage should be in the 80% range, if not higher. As one research service provider said, “Obviously there is work to be done. Not only does it have implications in terms of our image, etc., but it is getting harder and harder to get good respondents to participate in research. And without good respondents we can’t get good insights. Without good insights we can’t have strong strategic recommendations. Basically, we are killing our industry over time.”
The research was unclear as to what was driving this lack of confidence. One idea for improving confidence in anonymity is to provide additional explanation of our codes of conduct and/or data protection laws and how we comply with them.
Final Thought
Allow me to reiterate what I stated in my previous article—it is important for us, as qualitative researchers, to be advocates for our respondents because without them there is no research.
Be the first to comment