Irrationally Held Truths Take a Toll
The idea of making informed decisions based on data is neither new nor controversial. Any organization building self-service applications for customers would agree with the notion of collecting and using data to design these applications. Too often, however, organizations make decisions that seem to fly in the face of a data-driven design process, and I’ve been trying to understand why that happens.
The start of a poor decision is often a deeply held truth espoused by the organization: “Our customers don’t care about feature X” or “Transaction Y is too hard to automate” or “We can’t recognize ‘representative’ in the IVR or everyone will transfer out.”
Those are not ideas that organizations think are probably true; those are core ideas about automated self-service that they know are true. Such convictions are a by-product of long-term, in-depth experience in a domain. Our brains are hardwired to evaluate the information available and draw conclusions, so I don’t fault anyone for holding this kind of belief.
The problem is when people are unwilling to question why they hold such beliefs or how the beliefs were formed. Many times, those convictions are very much like old wives’ tales: They’re based on incomplete information or a misinterpretation of the information, and they’re almost never the result of the believer’s direct experience. Believing that it takes seven years to digest gum that you have swallowed is relatively benign and unlikely to cause you any significant harm, although it’s untrue (see http://www.snopes.com/oldwives/chewgum.asp), but holding fast to unsubstantiated beliefs about self-service can have a dramatic impact on your business.
I am reminded of a quote by 19th-century British biologist Thomas Huxley: “Irrationally held truths may be more harmful than reasoned errors.” Irrationally held truths are more dangerous because they seem like facts, but they often fail to hold up to close scrutiny. When I asked a client to explain the claim that “customers aren’t interested in feature X,” the explanation was that since no one chose feature X in the IVR, it must be unimportant.
This is one possible interpretation of the data, but not the only one. Perhaps customers don’t choose feature X in the IVR because they can’t find it, or they are uncomfortable using the feature in an automated system and choose to complete the task in a different way. Or, perhaps customers don’t choose feature X because they don’t understand the menu option advertising feature X in the IVR. In the absence of any other data, any one of these theories is equally plausible based on the call statistic that no one selects feature X.
Part of my job as a consultant is to point out situations exactly like this one, to uncover an organization’s deeply held truths and help them see that there may be alternative interpretations of the data. For that to work, the organization must be willing to question those beliefs and commit time and resources to a data-driven decision-making process.
The most unpopular recommendation I make to clients is that it’s impossible to know the best way to proceed based on current information. Often, there are multiple plausible interpretations of a set of data, and the only way to distinguish among them is to collect additional data.
The most common form of missing data concerns customers’ motivations, attitudes, and opinions—exactly the sort of information that is uncovered by user research, usability testing, and other user-centered design methods. Organizations committed to providing excellent automated experiences for customers understand that user data is vital to making good business decisions, but I still encounter a great deal of skepticism about its value.
Information on user motivation and opinion seems too squishy and subjective to count as data, so some organizations undervalue its relevance in decision-making. In lieu of collecting user data, those organizations often fall back on their deeply held beliefs, which may be even less rigorous than the user data that they disdain.
The moral of the story is: Take the time to question your assumptions and figure out why you believe what you do about your self-service systems. The exercise of raising alternate interpretations can be uncomfortable, but testing your hypotheses can deliver real business benefits. On the other hand, reluctance to question deeply held truths means organizations risk making poor decisions about how to build self-service systems that customers use willingly.
Susan Hura, Ph.D., is principal and founder of SpeechUsability, a VUI design consulting firm. She can be reached at susan@speechusability.com.