By Katrin Scheibert, Head of Design UK, Kadence International, London, England, kscheibert@kadence.com; Faye Soares, Customer Insights Manager, Dow Jones Customer Insights, London, England, faye.soares@dowjones.com; and Rupert Sinclair, Head of Insight UK, Kadence International, London, England, rsinclair@kadence.com;
This article shares selected learnings from a paper cowritten by Kadence International and Dow Jones, which was awarded Best Paper at ESOMAR Congress 2023. Here, we examine a range of learnings from our pioneering exploration and assessment of the metaverse as an opportunity for qualitative research, including the potential we believe it offers at the current time and in the future.
The Emergence of the Metaverse
Virtual reality (VR) occupies a rather eccentric place in the collective mind: simultaneously futuristic and woefully dated. In the 1990s, major brands (including Nintendo and Sega) attempted to bring VR headsets into the world’s homes. The experiences provided by these devices fell short of public expectations, failing to meet the hype. Sales were modest, and the generation of headsets was largely considered unsuccessful. Following these extravagant and unsuccessful forays into virtual reality, its application was restricted to professional use for many years, finding use in diverse but well-funded fields from NASA’s training of astronauts to medical students simulating operations.
Since 2021, there has been a renewed push into this area. Major investment has been poured into virtual reality from the likes of Meta, Microsoft, and Apple. The result has been the emergence of a range of high-quality VR headsets at accessible price points, bringing these devices into millions of homes and workplaces across the globe. Alongside the hardware, resources are flowing into software for virtual environments with use cases including gaming, socializing, and workplace collaboration.
Collectively, virtual environments have been named the metaverse. The metaverse is generally defined as a vision of the internet connected by virtual or augmented reality. So, rather than clicking and scrolling through the modern web, the metaverse provides a more immersive experience, allowing us to walk around and interact with virtual environments.
In October 2021, Facebook CEO Mark Zuckerberg presented his vision for the future of the social media giant, formalizing the company’s focus on the metaverse. Addressing the company’s annual conference, Zuckerberg announced the company was rebranding as Meta and detailed how his company aimed to build a new version of the internet: “We believe the metaverse will be the successor to the mobile internet; we’ll be able to feel present—like we’re right there with people no matter how far apart we actually are.”
Soon after, investment appeared to be pouring in from all angles. In June 2022, McKinsey1 reported that “more than $120 billion [had been invested] in the metaverse in the first five months of 2022.”
The energetic activity in this space attracted widespread attention and prompted animated debate around how the metaverse could be expected to disrupt and revolutionize our lives.
Within the research industry, however, there was little talk of how recent investments in the metaverse would impact us and any potential opportunities. With a shared curiosity about the impact of emerging technology on our industry, Kadence International and the Dow Jones’ Customer Insights team took this opportunity to explore the topic for ourselves.
What were we hoping to learn?
Ultimately, our goal was to determine whether the metaverse could be an effective tool for market research, now or in the future.
In this article, we will walk through some of the learnings related to the opportunities, challenges, and limitations of the metaverse in the specific context of market research.
We consider the current time but also the medium-term future as consumer access to hardware widens, and investment continues to be poured into the space from tech giants such as Facebook, Google, and Apple.
Examining learnings around opportunities, challenges, and limitations helps determine whether the metaverse is the future of our industry or simply a passing distraction.
Building a Program of Research in the Metaverse
Starting from a point of minimal technical knowledge, we were aware that we had a lot to learn before we could run an effective program of research that was not plagued by technical issues. Based on previous experience, we were keenly aware that the whole pilot could easily fall apart if we didn’t select the most appropriate technology and become proficient in using it.
To address whether the metaverse will shape the future of research, we were interested in road-testing a range of research exercises and identifying where strengths and weaknesses lay for research.
We prepared the following research tasks to carry out in the metaverse. Research sessions included all research tasks.
- Group discussion based on screenshots submitted by respondents prior to the session (and annotation of these).
- Navigation of a “live” news website as a group, including using menus and searching for content.
- Co-creation task to create an “ideal” news website landing page with the moderator building a prototype page in the session based on the group suggestions.
We used the VR environment called Horizon Workrooms, which we felt offered functionality most suited to our requirements. A range of stimulus types were used in the groups—with the goal of understanding which stimulus types worked well or better in virtual reality.
We were unable to provide each respondent with a £300 headset (around $400 in the U.S.). For this reason, it was necessary that our sample be composed of tech-forward participants who own Oculus Quest 2 devices.
Within this narrow universe, we attempted to draw a sample of respondents that was as diverse as possible. Our achieved sample ranged in age from 23 to 55, with a good gender mix and a range of professional backgrounds.
Respondents were invited to participate in one of three research sessions. The three 90-minute sessions were scheduled over a two-week period. Respondents attended the sessions using their own avatars.
We varied the number of respondents per session to learn about the impact of group size on group dynamics in a metaverse environment. Groups were comprised of four, three, and two participants.
Given the unfamiliar nature of the task for respondents, we knew the sessions could easily be plagued by technical issues and distract us from assessing the methodology. Prior to the sessions, to limit technical difficulties, we shared an engaging onboarding document that included step-by-step instructions on how to prepare for the session and pretask.
What did we learn?
In many ways, the experience outperformed our expectations and offered compelling benefits over heritage techniques, both traditional and digital.
We also discovered important limitations that would act as barriers to using the metaverse for qualitative research.
Below, we share a handful of critical learnings from both perspectives.
Important Benefits
Quality of Interactions: The rapport and richness of interactions between participants surprised us and far outperformed our expectations.
The relatively realistic gesticulations, 3-D sounds, and expressive avatars all contributed in their own way.
Eye contact felt convincing to the point of being uncomfortable when maintained for too long. On many occasions, respondents effortlessly and naturally transitioned to discussing nonrelevant topics of mutual interest. The research team ended the sessions feeling far more familiar and acquainted with respondents than in a more traditional digital qualitative session.
In many ways, the quality and depth of interactions felt closer to face-to-face groups than digital equivalents. The immersive nature of the sessions gave a genuine sense of occupying the same physical space. Clearly, this is a benefit when seeking to understand individuals and their behaviors.
Undivided Attention: Conducting digital depths, moderators regularly compete with interruptions from family members and pets, as well as distractions from email and social media. Due to the immersive nature of the VR headset, respondents typically appeared to provide their full attention in the session, with few taking the headset off to attend to other tasks—another strong benefit of the metaverse.
This also helps make metaverse research more akin to face-to-face focus groups than digital equivalents.
Active Optimization of Room Layout: The ability to effortlessly switch room layout between a collaborative roundtable set-up and lecture-style auditorium allowed us to optimize how participants interacted with each other and the stimulus, depending on the task at hand.
For example, co-creation tasks worked well at a roundtable. When trialing a live website, it made more sense to project the website onto a large screen at the front of a single row of respondents, with all facing toward the front.
Occasional changes in the virtual room’s layout also seemed to revive more lethargic respondents.
Sharing Stimulus: Sharing stimulus in a face-to-face focus group can sometimes pose a challenge to participants.
For example, in person, it can be difficult to see stimuli from the opposite end of the room. On other occasions, participants may have to wait for a stimulus to be passed between respondents before having a moment or two to consider it.
In the metaverse research sessions, respondents were able to clearly view a stimulus projected directly in front of them, enabling them to consider it fully and offer feedback. They could also easily annotate the stimulus without the need to leave their seat. We believe this functionality is a particular benefit in the context of UX and co-creation research.
Key Limitations and Challenges
In many ways, the experience of conducting research in the metaverse exceeded our expectations. However, we also encountered a few factors that made the experience frustrating and challenging.
Sample Heavily Skewed toward Early Adopters: As you might expect, the profile of those who currently own a VR device is heavily skewed toward early adopters. Today, this poses a major challenge in drawing a sample that reflects a target audience.
Technical Challenges: Despite our efforts to supply detailed instructions to participants, we encountered technical issues with respondents gaining access to sessions. When the pilot was conducted, gaining access was far more complex and convoluted than typical video meetings. The issues associated with this mundane element of hosting a session were frustrating and time-consuming.
No Viewing Facility for Clients: For the purposes of the pilot, we used Horizon Workrooms, which is primarily positioned as a tool for workplace collaboration. Unsurprisingly, this did not include functionality for a viewing facility or “back room” where clients can observe unobtrusively in real time. For this reason, the Dow Jones Customer Insights team watched video recordings afterward, which meant they were unable to discuss insights with the moderator or alter the direction of discussions. This is a notable disadvantage over traditional face-to-face and digital methods.
Implications for OUR Industry?
While we were impressed with the overall experience, a number of factors (some mentioned above) combined to make executing our program of research in the metaverse challenging and, at times, pretty frustrating.
Individually, no single element posed an insurmountable challenge, and none alone would be sufficiently off-putting, but in aggregate, they emphasised the need for extensive planning, trouble-shooting skills, and technical know-how.
Right now, you would need to be exceptionally motivated and determined to select a metaverse-based qualitative approach over other alternatives. Simply put, the benefits do not currently outweigh the inconvenience.
Looking to the future, we can expect many of the technical challenges to be remedied by the manufacturers. Meanwhile, from an insight industry perspective, it would probably require a fairly modest investment to build a platform that includes the functionality we need or add this to an existing platform. For example, adding a viewing facility option.
The arrival of Apple’s legendarily loyal customer base, coupled with increased activity from Meta, is likely to lead to a dramatic increase in headset owners as the category matures. This, in turn, will make recruiting a nonearly adopter sample feasible, opening up many opportunities for our industry.
Based on our experience during the pilot, we believe considerable potential exists for metaverse research once the category has time to mature.
The foundations are already in place, but the devices and software must reach a point where these can fairly effortlessly be deployed in research. When we arrive at this point, straightforward exercises like the pilot sessions we undertook will become routine and unremarkable. Then, we believe VR and the metaverse will become a core element of the researcher tool kit.
What the Future Market Research Metaverse Platform Should Look Like
For the industry to harness opportunities presented by the metaverse, skill sets and approaches will need to adapt and evolve.
Some responsibility will land with pioneering agencies, and other elements will be down to individual researchers. There may also be a role for the industry more broadly to collaborate in developing platforms that can be deployed in research.
An essential aspect involves creating a platform that incorporates key components necessary for hosting research sessions. At the very least, these key components would include a “backroom,” a waiting room for respondents, and mechanisms to prevent the display of respondents’ full names.
A metaverse platform for market research might also feature a library of room configurations suitable for research exercises and functionality for paying respondent incentives. Development of this kind of platform could be spearheaded by one agency or potentially a collaborative effort by one or multiple industry groups.
Individual researchers will also need to adapt to thrive in this space.
Proficiency with hardware and research platforms is crucial, but a diverse set of additional skills will be necessary. New techniques will be essential for effectively moderating qualitative sessions in these environments, including methods to nurture natural conversation between participants and ensure diverse voices are heard.
Meanwhile, new types of research exercises will need to be built using virtual co-creation tools that are not available in real-life settings. New skills will also be required to understand and interpret virtual tools of personal expression, such as avatars.
Above all, researchers will require curiosity and adaptability to thrive in this new space, enabling them to blend existing skill sets with new ones and make the most of opportunities for evolving qualitative techniques in the metaverse.
Reference
- McKinsey & Company, “Value Creation in the Metaverse: The real business of the virtual world,” June 2022, https://bit.ly/42Dzd1y
