Reviewed by Mark Wheeler, Principal, Wheeler Research LLC, Bryn Mawr, Pennsylvania, mark@wheelerresearch.com
The authors of Noise: A Flaw in Human Judgment, particularly lead author and Nobel Prize winner Daniel Kahneman, are so well-known in behavioral science that together they resemble an all-star team in sports or a “super group” in classic rock. This makes their new book feel like a must-read.
The concept of noise is important for any of us who rely on the perceptions, opinions, or choices of others. Noise is defined as the unwanted variability of judgment, that is usually random, or at least arbitrary and unplanned. In qualitative consulting, noise isn’t what you have when two respondents disagree. Noise occurs when the decisions themselves become unreliable. As researchers and consultants, we would do well to better understand noise, in order to minimize its impact.
Most relevant is the idea of occasion noise—the same person can be confronted with the identical decision twice, yet make a different choice each time. Examples of occasion noise range from the amusing to the concerning. The authors describe cases in which wine experts never score the same wine identically in blind taste tests. Similarly, physicians can be presented with the same cases on different days and reach different diagnoses.
Aside from the personal inconsistency (troubling enough in medical cases), there appear to be a few prime reasons that explain why our judgments can’t be perfectly reliable. Even slight variations in mood, stress, or fatigue turn out to have surprisingly strong effects. This part of the book brought to mind a classic psychology experiment that proved how easy it is for us to be influenced by random or trivial things. The psychologist Norman Schwarz and his colleagues placed a dime where people could find it. Shortly after making their small discovery, they were surveyed and asked to rate their overall satisfaction with life as a whole. When compared to others who didn’t acquire the dime, the finders reported significantly higher satisfaction and happiness with their entire lives.
Our mood doesn’t just change what we think—it changes how we think. Positive experiences make us happier, but also less reflective, and more gullible to whatever we see and hear. The authors describe a fascinating experiment in which people were shown pseudo-profound nonsense (e.g., “Wholeness quiets infinite phenomena”). When the experimenters were able to induce a good mood in readers—by watching a happy video—the readers became receptive to the silly messages and impressed by their deep meaning (even though there was no meaning).
It works the other way, too—people in bad moods can become better at realistically assessing situations and better at rejecting dubious claims. But we don’t want key decision-makers to be too unhappy—research has found judicial sentences to be more severe when the weather is uncomfortably hot, and stock market performance can be affected (for better and worse) by the amount of sunshine.
Getting back to noise—the point is that we often think we are giving thoughtful unbiased opinions, but we are not. Many of our qualitative respondents probably aren’t either. It is humbling to think that our respondents might respond positively to our client’s draft messages because they have enjoyed making small talk with the moderator. We don’t want our respondents to dislike the target stimuli because they were recently aggravated by the experience of logging into the web platform.
It might surprise you that I am not going to recommend that qualitative consultants rush out to buy this book. Much of the book centers around types of noise that aren’t immediately relevant to many of us, such as noise within larger systems like judicial or occupational settings.
Instead, I ask readers to keep a further eye out for other advances in behavioral science generally, and with respect to noise more generally. Just as the decision-making biases identified by Daniel Kahneman and others (see his classic, Thinking, Fast and Slow) are now being frequently applied into our research and consulting work, the concepts of noise and noise reduction are sure to become a bigger part of our work.
