The Importance of Measuring ROI of Insights

Billions of dollars are spent around the world on insights and market research, which leads some to question the value of research. In previous decades, conducting research was usually a matter of faith. People felt that having evidence meant they made better decisions. In particular, research was seen as a method of reducing risk.

However, these days, this “trust and hope” approach to spending money is hard to justify. People (especially the C-suite, and in particular the CFO) expect to see proof that their money was well spent. They want to see what return they got on their investment, and they want to measure the return on investment (ROI) of insights and of their insight processes.

Twenty years ago, the notion of measuring the ROI of insights seemed quaint, the domain of strange people who thought everything could be reduced to numbers. Guess what? Those people now run the show, and more and more companies are demanding that research (along with almost every aspect of the organization) shows the value of its contribution. This article shows that measuring the ROI is possible, a lot of organizations are doing it, and it is a useful thing to do.

Yes, It Is Possible to Measure ROI

One thing that has held back the measurement of the ROI of insights was the widespread belief that it was not possible. Most of the reasons for this belief relate to the mindset of the typical insight professional. Researchers felt that any measurement of their contribution would lack precision, completeness, and be premature. For example, if insight was one of several inputs into a business decision, how can we accurately ascribe the exact contribution of research to it? Second, because the result of the business decision may be known only in the longer term, how can we assess, now, the value of an input to the decision? The value from an insight may be more than just the specific result being requested; it may generate new learnings, ideas, and opportunities, either immediately or at some later time. How will these wider ideas be accounted for?

But as the old saying goes, “Those who say it can’t be done should get out of the way of the people doing it.” Measuring the ROI of insights is now commonplace. Its use has become widespread because it has adopted many of the same approaches that are used to measure the ROI of other factors, such as the ROI of design, marketing, and automation.

Three Ways to Estimate ROI

Whether we are measuring the ROI of pencils, roads, banking services, or insights, there is no perfect method, so we adopt one or more of the following three approaches:

1. Total value contributed. For example, if a project generates $1 million of extra revenue and if insights were deemed to have contributed, then this is recorded in the insights column. Of course, this $1 million will also be recorded in other columns, such as the columns for marketing, design, and engineering. Yes, this implies double counting, but if research does not engage in this process, it does not stop the double counting. Marketing, design, and engineering will still double count it, but insights will be left looking ineffectual.
2. The cost of the alternative. There are at least two aspects to the cost of the alternative. First, what might have happened if the research had not been conducted? For example, instead of testing the concept in focus groups, and finding out that it needed to have X in order to be successful, maybe an MVP (minimum viable product) would have been developed and pilot tested. Pilots, demos, and MVPs are often much more expensive and imply delays. Second, there is the relative savings of one research method over another. If switching from face-to-face groups to an online method saved money and provided a sufficiently good answer, then there is a ROI that should be recorded and mentioned.
3. The perceived value by the recipient of the insights. The most common way to estimate this form of ROI is to simply ask the project owner what percentage of the success of the project was due to the insights, in combination with the overall return on the whole project. For example, if a project generated $1 million in additional revenue and the team estimated that insights contributed 25 percent to the success of the project, then the return for insights is $250,000.

Truth, Not Accuracy

One of the most telling criticisms of research is that it prioritizes accuracy over truth. Too often, researchers have failed to make a clear recommendation because of the search for ever-greater accuracy. When a rock is falling from the cliffs, the truth is that there is danger and you need to move. The difference between ten and fifteen rocks falling from the cliff is a matter of accuracy that does not alter the truth of the danger nor the need to move. In terms of the ROI of insights, the truth is that insights drive better decisions, and this should not be obscured by a passion for ever-greater accuracy. If the message about the usefulness of research and insights does not reach the wider business because we are waiting for more accuracy, the truth of the need for insights will be lost.

One of the best illustrations of the value of insights to business, and the need to highlight this truth, comes from Steve Jobs and Apple. For many years, Jobs made statements such as, “We do no market research,” and “People don’t know what they want until you show it to them. That’s why I never rely on market research.” But in the 2012 court case with Samsung, Apple disclosed that it conducted a vast range of research and that these insights underpinned much of its success.

Five Key Steps to Measuring the ROI of Insights

1. Defining the project and its goals properly before you start

What are the goals of a new project? A new ad may be targeted at increasing revenue or market share. The intended outcome must be defined, and the value of the research should be assessed in terms of improving these outcomes. In The Lean Startup, Eric Ries talks about the need to “gain validated learning.” This means conducting experiments with defined goals that can illustrate the failure or success of an idea in ways that allow progress to be made.

2. Baking the measurements into the process

You should not wait until a project concludes before deciding which elements to measure; the evaluation should be baked into the process. A great example was given by Simon Harrington when he was a client of mine. Simon was the chief marketing officer at CBS Outdoors (now Exterion Media), and I was the managing director of Vision Critical U.K. CBS Outdoors operated a highly successful community called work.shop.play, and Simon was determined to show the value of the community to the rest of the business. To measure the ROI of the community, Simon ensured that for every project that involved insights from work.shop.play, an ID flag was created in Salesforce. At the end of each quarter and year, it was a simple matter to calculate the total value of new business that had been won with projects that utilizing insights from the work.shop.play community.

3. Learning from the project team

The insights team does not need to define what the uplift caused by research was, and indeed they shouldn’t. As James Wycherley has pointed out in his recent book, Transforming Insight: The 42 Secrets of Successful Corporate Insight Teams, insight teams often estimate the value too low. Wycherley advocates asking people from the project team to evaluate the impact of the research. Talking about brands such as eBay and Barclays Bank, Wycherley reports, “It wasn’t unusual for the other departments to say that 25 percent or 30 percent of the value uplift was down to insight.”

4. Reporting promptly

Old news is an oxymoron. The initial impact of insights needs to be reported promptly. By baking the measurements into the project, and by asking immediately how valuable the insights were, the research team can get a good first estimate.

5. Revisiting later

Projects should be reviewed three months and twelve months later, to update the learnings from the project, including the ROI of the insights. This will give a clearer view of the impact of the project (against its goals), allowing a clearer statement of the impact of the insights to be generated.

The Benefits of ROI Measurement

A 2018 study by the Global Research Business Network identified six key benefits from measuring the ROI of insights and demonstrating business impact:

1. Increased stakeholder satisfaction
2. A seat at the strategic table
3. Greater budget
4. More budget control
5. More freedom to innovate and initiate strategic work
6. More resources

These are six really good reasons to measure the ROI. Between them, they expand the relevance and importance of insights.

ROI Is for Qual Just as Much as Quant

Because ROI is best measured in numbers, usually in monetary terms, it may seem more appropriate to apply to quantitative projects. However, to believe that is to misunderstand what ROI measures. The ROI of insights is not related to what was done, and it does not relate to the volume of outputs. It is defined by the actions that the recipients of the insights were able to take. For example, usability research is a great example of an area where observational qualitative research can provide advice that can be readily turned into changes that increase use, satisfaction, and sales. Similarly, qualitative research with lapsed users can provide information that reduces churn, something easily measured in terms of ROI.

Key Reading

If you want to jump-start your measurement of ROI, here are five useful resources:

1. ROI of Insights, a 2017 joint report by BCG (Boston Consulting Group) and GRBN (Global Research Business Network). Available for free download: http://bit.ly/VIEWS-ROI-of-Insight
2. The Expert Guide to Measuring Not Counting, a report published by the U.K.’s IPA (Institute of Practitioners in Advertising). Available for free download: http://bit.ly/VIEWS-IPA-Guide-Measuring
3. Transforming Insight: The 42 Secrets of Successful Corporate Insight Teams, a book written by James Wycherley, CEO of the Insight Management Academy. Available for purchase from Amazon. http://bit.ly/VIEWS-Transforming-Insight
4. Capturing the ROI of CX, a report from Forrester, focusing on customer experience. Available for free download: http://bit.ly/VIEWS-Forrester-ROI-of-CX
5. “Quantifying the Value of UX,” a post by Meghan Wenzel on UX Collective, showing how to demonstrate ROI and defend design decisions. http://bit.ly/VIEWS-Quantifying-UX

Summary

• Insights are too expensive and too important not to calculate the ROI.
• The ROI of insights can be calculated and is being calculated by many organizations.
• Focus on prioritizing truth rather than accuracy.
• Start by defining the objectives of the project, and select metrics and targets before the project starts—bake the measurement into the process.
• Focus on the value to the business, and ask the users of the insights to assess the contribution to the outcomes.
• Report the results, and revisit the findings later to see if further information is available.
• See measuring the ROI of insights as a way to show why more research should be conducted.