By Mike Carlon, Moderator and Host, Uncorking a Story, Stamford, Connecticut, michael.carlon@uncorkingastory.com
Aside from a five-year stint working in consumer packaged goods, I’ve spent the majority of my 25-plus-year career working with financial services firms in both marketing research and user experience research capacities. Over that time span, I’ve seen seismic shifts in how we approach product development, user experience, and customer experience research. To highlight this, let’s get in our DeLorean, set the time back to 1997, and get our miles per hour up to 88 to get a glimpse into what our research realities were in the early days of the Web as a consumer medium.
Product Development—Then
In 1997 I was working for a company called Modem Media, which billed itself as the world’s first interactive agency. We were credited with having designed and placed the first banner ad and attracted blue-chip clients eager to win in the “new media.” One of them was Citibank (now Citi), which hired Modem to redesign its website. The project went well and led to more work, including the opportunity to help the bank develop a digital wallet. There was only one problem: no one knew what the heck a digital wallet was because such things didn’t exist. Enter Modem’s small but mighty research department.
At the time, online research was in its infancy, and we still approached marketing research in a very traditional manner. My team put together a three-phased plan consisting of ideation research, concept evaluation, and usability testing. The total investment was well into the six figures and took approximately six months from start to finish.
Ideation Research
As mentioned, a digital wallet didn’t exist in 1997—heck, it would be 10 years until the iPhone was even launched, when such a thing would have been more plausible. In order to inform what this idea could be, we held a number of focus groups with digital-forward consumers to discuss what they’d expect to see in something called a digital wallet. We also came armed with some ideas that our clients and creative team had generated to get an understanding of where we might be able to take this product. Coming out of that research—which was conducted across New York, Chicago, and San Francisco—we had a list of features that were then taken into quantitative research to get a read on how much prospective users liked each feature as a way of prioritizing what we’d offer in a more formal concept. Timing: two months of work from start to finish. Budget: approximately $150,000.
Concept Evaluation
Armed with consumer insights, our creative team went off to create some product concepts that would bring this idea to life a bit more. In addition to expanding on the features, the goal of the next round of research was to get some direction on design ideas for this conceptual product. Here again, we employed a round of nine focus groups with tech-forward consumers who were also Citibank customers, rationalizing that only they would be able to use this product, and we wanted to get a sense of what impact, if any, it might have on customer loyalty. Timing: six weeks of work and a budget of $75,000.
Usability Testing
Our clients inside Citibank were very excited about the product we were helping them develop, and a lot of that had to do with the fact that we had supported feature and design decisions with customer insights. As such, they greenlit the development of an interactive prototype that we brought to consumers in one-on-one usability testing, where we sat with customers and gave them tasks to complete using this tool to identify ways in which it could be made more intuitive/easier to use. Timing: The development of this prototype took two months, and testing required four weeks from start to finish at a budget of $30,000.
No Green Light
After half a year of work and about half a million dollars invested between research and development costs (creative resources and engineers aren’t free), the client team at Citibank decided to shelve the product. Needless to say, the agency team was let down given all of the time and effort spent to research this idea, which, admittedly, was way ahead of its time. That said, this outcome was common in those days when clients had experimentation budgets for developing interactive product ideas and were encouraged (and rewarded) for spending it.
Product Development—Now
Fast-forward over a quarter century, and this project would have unfolded quite differently. Not only have the tools for conducting this type of research changed, but the entire mindset around product development in financial services has shifted from being solution-led to problem-led.
I have a saying that I impart to all my product-focused clients—fall in love with the problem, not the solution. In the Citibank example from 1997, we led with a solution (i.e., a digital wallet) rather than a problem. I don’t want to sound too harsh here because, in those days, it was unclear what anyone wanted in digital experiences from their bank beyond more convenient ways to access account information, transfer money between accounts, and pay bills. In essence, back then, we had a lot of ideas searching for problems to solve. The world is much different now.
Discovery Today—Start by Identifying the Problem
If we were to conduct this research today, we’d start by exploring the various problems that our customers face around payments. Digital payment cards are one solution that might address a payments problem, but they would be one of potentially many we’d explore. Instead of going into a round of qualitative and quantitative exploratory research by asking what customers might want in a particular solution (i.e., a digital wallet), we’d invest time and resources in identifying points of friction that our customers face and ideating a number of solutions to address those problems. We’d bring these solutions to some discovery research sessions over the course of a two-week sprint.
Instead of running nine focus groups in a small number of markets, we’d opt to conduct remote research on a one-on-one basis and eliminate geography as a constraint. Instead of hiring out to an external research vendor, this research would be conducted by staff UX researchers, and instead of going in with a written concept, we’d use low-fidelity prototypes to bring each solution to life. Whereas this first phase took two months and over six figures in 1997, it would now take two weeks at an investment of internal design costs and recruiting fees. Two weeks and $5,500 out-of-pocket sounds about right. Depending on the feedback, we’d conduct a number of follow-up sprints using a similar method until we were reasonably confident that the product or feature was solving a core problem that our customers face.
Build It Right
Another mantra I use with my financial services clients is to build the right thing, then build it right. Where our discovery phase is all about identifying what the right thing is, building it right is all about making sure our solution is easy and intuitive for customers to use. To evaluate this, we have tools at our disposal that we didn’t have in the late 1990s. One tool we frequently use is unmoderated user testing.
In an unmoderated test, participants are recruited to evaluate a prototype and given minimal, if any, instructions as to how it works. Using either their desktop computer or mobile device, testers access a platform such as UserTesting, UserZoom, or DScout and are given certain tasks to complete using a clickable prototype. Importantly, there is no direct interaction between participant and researcher—aside from answering some open- or closed-ended questions, most insights are gained by observation. This serves to reduce researcher bias and observer bias and allows us to be more efficient in how we collect and analyze feedback. Whereas 12 usability tests used to take four to six weeks to complete at a cost of approximately $30,000 in the 1990s, we can launch and fill an unmoderated study with the same sample size using an unmoderated testing approach literally overnight for less than a tenth of the cost.
A Change in Thinking about Customers
Changes in the financial services sector, though, are not just a story of old versus new. I’ve seen a shift in how clients in this industry think of and talk about customers. When I was first coming up, I heard customers spoken of as “deposit accounts.” Talk about feeling like a number rather than a person. Additionally, the research I used to do for these clients was very functional—we’d test out targeted direct mail offers in an attempt to win customers through better offers, such as increased interest rates on savings accounts or loyalty pricing on loans and lines of credits if customers met certain deposit thresholds. Of course, all of this was an attempt to increase money on deposit, which is certainly important for a bank, but those offers eventually expire, and it’s back to business as usual.
Today, though, I see much more focus on customer experience (CX). While I can’t speak to the motivations for all institutions in this space, I’ve seen three reasons among my client base for the increased focus on CX:
- The increased ability to manage accounts digitally means less foot traffic in brick-and-mortar branches. Maintaining loyalty with fewer personal interactions means having a great digital experience.
- The ability to use digital tools for self-service (i.e., disputing a transaction) means fewer calls to a customer service center, which translates to savings for the bank.
- Last, and important, FinTech firms have been popping up and disrupting not only the deposit game but lending as well. Most of them are taking a tech-first approach to customer acquisition and management, often with slick user experiences. Legacy institutions are now investing more and more in CX and UX to achieve parity with these new players.
To highlight the changes I’ve seen with my traditional banking clients, I’ll share an example of how projects shifted from being product and offer-focused to experience-focused. I led one project for a large national bank a few years back that involved interviewing former business banking customers to explore the reasons why they left the bank. As part of this, I asked each participant to draft a Dear John letter to this bank as a way of uncovering both the functional and emotional ways in which the relationship soured.
These letters explored:
- The Love Story: the elements of attraction, what the former customer appreciated about the bank, and a description of “the good times.”
- The Tensions: when and how chinks in the armor began to appear and any steps taken on behalf of the customer and bank to resolve issues.
- The Last Straw: what ultimately led to the end of the relationship, who they left the bank for, and what their “new partner” is doing better than the former bank.
- Reconciliation: How does the bank need to change to have any hope for reconciliation?
In addition to this exercise, I had a number of other questions in my discussion guide; however, we rarely got to them. In most cases, walking through these letters at the start of each interview filled up an entire hour, but my client didn’t mind. It was exactly the feedback they needed to hear. Much of the feedback had to do with something involving customer experience—from inconvenient hours of operation for small-business owners to poor communication from relationship managers about changes to deposit requirements to avoid monthly fees, this bank started to realize the importance customer experience plays in retention.
Also coming into focus was the superior experiences of FinTechs, including SoFi, Stripe, and Square. This bank repeated the study with consumers, and the letters generated from both went directly to the C-suite. The bank has since stopped thinking of customers as “deposits,” invested design resources in mapping out both the current and ideal customer experience journey, and has fully embraced a customer-first mindset.
Forging into Financial Services
Obviously, it is unfair to compare how the financial services industry would approach product development in 1997 versus 2024. Not only has the technology for conducting research changed, but the mindset for how to develop products has shifted from a solutions-first approach to a problem-first approach. Ultimately, these trends have enabled those of us who serve the financial services industry to gather insights faster and less expensively.
If you are a researcher trying to break into the financial services product development industry, consider the following:
- Demonstrate how you have used research to build empathy for customers with senior leaders. This will speak volumes to those institutions that have embraced a customer-forward strategy.
- Similarly, showcase how you have helped clients identify problems for customers and how you’ve worked in collaboration with clients on drafting (and hopefully testing) potential solutions. This is in line with taking a problem-first approach versus a solutions-first approach, which is at the heart of modern-day product development.
- Showcase how you have embraced emerging research technologies, such as remote interviewing, mobile ethnography, and unmoderated testing, to bring speed and agility to the research process. While there is still a place for traditional methods, most clients in this space, particularly those focused on user experience, will want to see how you have embraced technology to shorten timelines, reduce costs, and deliver actionable results.
- Demonstrate that you have a test and iterate mindset. The ideas you bring to customers for review don’t have to be “fully baked.” Often, they are low-fidelity concepts or prototypes used as a starting point for discovery. Show how you can take a “test and learn” approach so that valuable time isn’t spent on trying to get something perfect before it goes into testing.
Break on Through
While the advice above will help you stand apart from competition, the elephant in the room is that financial services is a regulated industry—which poses challenges for smaller qualitative shops and independent moderators to break into. Compliance requirements, technology audits, insurance requirements, and risk audits are just some of the things smaller companies will find as challenges when trying to get a master services agreement (MSA) with any of the large, national financial services institutions. My advice here is twofold:
- Aim Local or Regional. The smaller, regional, and local banks will still require some hoops to jump through to get onboarded, but given their smaller size, you can get championed much quicker.
- Freelance through agencies that have an existing relationship with a bank or institution. Many research companies, tech companies, and advertising agencies have already established MSAs with the big financial services companies. One strategy I’ve used is to focus business development outreach on these companies, who sometimes look to deepen their bench of researchers with financial services expertise—particularly those adept in emerging technologies and methodologies.
When I started my career as a qualitative consultant, I didn’t set out to specialize in any one industry; the financial services sector was something I just fell into. If there is one thing that I’ve learned over time, it’s that people are very passionate when it comes to their money, and this has led to some lively (and sometimes heated) discussions when talking about things that some might consider mundane, such as the usability of a mobile app feature to deposit or transfer money or how to best structure a digital account opening experience for a business checking account. In addition, I have found that participants truly appreciate the opportunity to provide input into things that will make their financial lives easier. While I know some of my colleagues who specialize in CPG believe they get more opportunities to do more creative work, I can honestly say my time spent in financial services has truly made a difference in the lives of the people who use the tools I’ve helped to research, and people are never at a loss for words when it comes to anything that touches their finances. I suppose the adage is true—money talks!
