Revisiting the ROI of Speech
A good voice user interface (VUI) is central to any successful speech application. Although VUI's are made up of many components, if the persona is very memorable, users' perceptions of it can dominate their opinions about the entire system, overwhelming all other aspects of the system in the users' minds. As such, a good or bad persona can have major consequences for the success of a system.
I've posed some questions about personas to a panel of five distinguished VUI designers — Bruce Balentine of EIG; Melissa Dougherty and Wally Brill of Voice Partners; Blade Kotelly of Edify; and Walter Rolandi of The Voice User Interface Company. Their responses give a good perspective on current thinking about personas.
About Personas
How do you define "persona" as used in speech applications?
Bruce Balentine: I see the word being used in two ways. First, as a generic term for those human-like characteristics that emerge spontaneously from application behaviors. The second use is more specific, and refers to a design philosophy. In this usage, some designers explicitly consider personality characteristics and biography as a formal part of the specification and design of dialogues.
Melissa Dougherty/Wally Brill: Persona is the consistent character of the voice interaction—and every system has one, whether it was pro-actively designed, or an accidental result. Extensive academic and commercial usability research has shown that callers take away a perception of a person behind the voice, every time they hear one.
Blade Kotelly: A speech application's persona includes personality traits a caller might attribute to that system after interacting with it. A persona heavily influences a caller's emotional reaction. Some designers who have trouble envisioning how those few elements can be used to create a multitude of personalities will go so far as to ascribe the speech persona a job, children, and hobbies.
Walter Rolandi: In the context of a voice user interface, a persona is the personality of the application: it is the epiphenomenon of voice talent variables such as gender and pitch; speaking variables such as intonation, pace and warmth; and prompt variables such as content and length. Unfortunately, in the speech industry, the term has come to connote entertainment-oriented and excessively animated voice user interfaces.
Do all speech applications have personas?
Balentine: All speech applications have personas according to my first generic usage. As soon as a machine presents voice cues to a user — and the user in turn gives voice responses in reply — an anthropomorphic model appears. The second more specific usage of persona does not apply to all applications. An explicit attempt to specify the anthropomorphic character — sort of "designing a person from the outside in" — is only one of several ways that designers can approach user interface design.
Kotelly: Yes, whether it's carefully crafted or not, speech applications always have a persona - there is always some human quality you can (and will) ascribe to it. It's the same for real people — no one is devoid of personality, rather they might be deemed "dull" or "apathetic."
Rolandi: In a sense, yes. A persona will emerge whether by accident or by design. For example, even if it were the expressed intent of a designer to create an application devoid of a persona, one would still emerge. Just as people attribute personality characteristics to pets, users would eventually attribute personality characteristics to the application.
How have personas evolved in the past five years?
Dougherty/ Brill: In our business, personas have evolved considerably. We often test alternative personas to identify the one that best "fits" the brand, caller group and task. We have understood the value of multiple personas—not as a gimmick, but to create a sense that the speech system isn't just impersonal automation, or to convey different aspects of the brand value to different customer segments. We're deploying Hispanic, Spanish-speaking personas, designed and tested among that audience—not just translated from an English language deployment. As mobile commerce takes off, we're also seeing personas that are more a part of a richer, entertainment experience in applications selling ringtones or music, for example.
Kotelly: The fact that designers focus on and sell the art of creating personas is an evolution in itself. Five years ago, only a few designers spent time and effort on creating a persona as part of their application design. Now, it's expected that any respected design organization will focus on extending a company's brand by crafting a suitable persona for their speech application.
The Value of Personas
Do all speech applications need personas?
Balentine: No. Personas in my second usage make no contribution to usability (i.e. task completion or task efficiency) and therefore have no value in applications where usability is the overriding goal. Personas are useful only when there are marketing goals that go beyond usability.
Kotelly: Since we assume that all systems have them, the question becomes "what's the advantage a persona can yield?"
A person's capacity to enjoy using an automated system is influenced not only by functionality, but of course by the way it handles those functions. Do you want to motivate someone to complete an arduous task? Motivate them, engage them. This doesn't mean the systems should take up more of the caller's time by adding long phrases or casting syrupy voices that have the effect of being sincere and caring.
Rolandi: While entertainment-oriented personas may well play a valuable role in such diverse applications as video games or retail telephone sales, they are invariably inappropriate in the customer service domain. The reason is simple: people call customer service lines only to solve a problem or to get something done. Callers are typically annoyed that they have to call customer service in the first place because, in a perfect world, there would never be a problem with a bill, an unpaid claim, a newly purchased product, etc. Having to deal with a cheery, enthused machine is yet another nuisance.
Give a quantifiable example of the positive impact of a persona on an application.
Balentine: Amtrak's Julie has, by definition, succeeded at the "branding" goals of the persona. She has been featured on Saturday Night Live, National Public Radio, and in many newspaper articles. The impact is on public relations (e.g. name recognition) - a value that benefits from negative as well as positive press ("There's no such thing as bad publicity"). I have not seen any data that suggests a contribution by Julie to usability, revenue, or customer satisfaction.
Dougherty/ Brill: When we compare, in research, the brand communication of a pro-actively designed persona versus an accidental persona, customer satisfaction and brand ratings improve. In our Cellular One case history, we defined personas that matched the brand, and redesigned an existing speech system to accommodate that persona and our high level design. The result? Bill payment in the speech system increased by an additional 10 percent and dollars collected by the system have increased by five percent. Bill payment with the representative has declined by 21 percent. According to Western Wireless, the cost of the persona redesign effort was covered by increased revenues generated solely by bill payment in the speech system alone, during the first months of its operation.
Kotelly: Publicity is hardly quantifiable, but look at the personas of United Airlines Flight Information and Julie from Amtrak; in addition to being featured on NPR and in numerous articles, they were recently mocked on Saturday Night Live. And, of course, users prefer a good persona to a poorly crafted one...but they aren't likely to give out the customer satisfaction numbers when the bad one performs poorly.
Rolandi: I cannot name a single successful speech application where the success of the application can be attributed to the application's persona. On the contrary, I have seen a number of failed speech applications where failure was either due to, or compounded by, the application's persona.
What was the best persona you've ever encountered?
Dougherty/ Brill: I'm really partial to two of the four personas in the Cellular One application. But at Voice Partners, we're naturally modest so we don't like to mention it too much.
Kotelly: Hand's down it was Wildfire. Wildfire was a system that was basically voicemail on steroids. The amazing thing about this personality is that even though you worked with her every day, several times a day, the persona never got old or tiring. The prompts were so well crafted from both a text and a recording perspective that they never sounded too long or dragging.
Rolandi: For me, the best "personas" are like the best wallpapers: if you don't notice them, then they are probably appropriate. In practice, this usually means the application uses matter-of-fact, theatrically unpresumptuous and business-like prompts.
What was the worst persona you've ever encountered?
Dougherty/ Brill: There's a major mobile provider in the U.S. that was one of the first to deploy speech, and depended heavily on a named persona. I was a customer at the time, and she was distant and disconnected from me as a customer. She didn't want to help me and didn't know how. On top of that, she introduced herself by name, and then was fairly chirpy about our interaction, despite the fact that she didn't seem interested in really helping.
Also for the UK readers, not to name names but it's a certain cinema chain's film line! It often has been quoted by nay-sayers of speech as a reason to hang on to DTMF. The persona is not only overly chirpy, but she is horribly hampered by asking the wrong questions of the caller thereby making the interaction longer than necessary.
Rolandi: This is a difficult question in that every time I think I've seen the worst, something new comes along, goes even further over the top and sets a new standard of persona excess.
Persona Design
How much work is needed to design a good persona?
Balentine: I prefer the design philosophy called "emergent." That is, I like to design for usability, establish simple principles for managing turn taking, grounding, confirmation, and progressive disclosure, and then let the persona "emerge" spontaneously from the interaction of those parts. Several days to a few weeks are required for this design process.
Kotelly: Developing a good persona can be as easy as selecting the voice that simply sounds right to the designer, writing prompts that match the task and audience calling, and coaching that voice appropriately in the studio. The process doesn't have to add much effort to the normal level of effort required for a project. For companies wishing to strengthen or re-define their brand, a designer may spend many additional hours during the initial research phases of a project.
Rolandi: Less than you might think.
What is the most important consideration when designing a persona?
Balentine: Focusing on the user is very difficult when the designer is focusing on the persona. There is a tendency to let persona decisions cloud usability decisions. So keeping the user's task in mind is the most important consideration — as with all productivity user interfaces.
One major aspect of persona design is developing an extensive biography for the character. We hear that "she attended Andersen University" and "his hobbies are slalom skiing and dog breeding." The process derives from theater and film acting. But it is highly overdone. The reason is that an IVR has such a limited repertoire of behaviors that the subtleties of the design are not observable in the application. So, more often than not, it becomes an internal and self-indulgent exercise for the stakeholders. It costs lots of money, delivers little payback.
Dougherty/ Brill: Personas must "fit" with the caller group itself, their expectations of the brand they've called, the task they're about to complete. Therefore, personas must be based on data, and evaluated by actual callers to determine (1) how well they fit with ongoing expectations and (2) the usability of the experience that they create.
Kotelly: Above all else, a designer needs to understand who the target users are for a system, or the persona has no basis. One could create a very successful persona for a video game company that uses words like "cool" and "right on" and prompts a caller to say, "where's my stuff" instead of "track my shipment." But that persona, while effective for late-20-somethings calling a video game company, would fail if implemented for a large bank.
Rolandi: The user is always the most important thing to consider when designing an application. Again, when considering customer self-service applications, there is no reason to believe that an animated persona can be of assistance to a user.
How are personas tested?
Balentine: Personas are rarely objectively tested. Personas are generally accepted on faith by internal stakeholders (since that's the group toward whom they are mainly targeted). External tests (focus groups, questionnaires, customer satisfaction measures) are often bogus, with leading questions and false assumptions.
The industry could do much better in this regard. Sometimes the enterprise hosting the persona is actually the last — not the first — to discover that the persona is a laughing stock or an irritant.
Dougherty/ Brill: We have callers interact with a WOZ (Wizard of Oz) system to complete specific tasks, in one-on-one interviews. If more than one persona is being evaluated, the caller actually compares different "systems"— each representing a persona — to accomplish certain tasks. It is critically important to have callers interact with the system to get something done, because many of their reactions relate to how well she/he helps them.
Kotelly: The only true way to measure the effects of the persona on an application is to conduct an apples-to-apples comparison, by measuring the percentage of successful calls before and after a new persona is introduced. For this to work, nothing other than the personality of the system (prompt text, voice, coaching style) can be changed. However, most companies are motivated to increase automation rates in any way possible, and will rarely, if ever, change the persona of an automated system without adding new functionality, or re-designing the ordering of questions within an application. Therefore, we often rely on focus groups to identify whether customers respond favorably to a new persona.
Rolandi: I experimented with frustration and sarcasm indices where I divided the number of "frustration behaviors" or sarcastic user responses observed during a recorded user interaction by the total number of user responses made during the session. It seemed to provide a reliable, empirically derived insight into the user experience, but, in the end, the method was probably overkill.
Should an application ever have more than one persona?
Balentine: Less than one is better than more than one. But once you've bought the assumptions that pull your focus away from your customers and toward your persona, then I say the more the merrier. The additional ones rarely do more harm than the first persona does, and can be more fun for the stakeholders and designers who seem to find them so fascinating.
Dougherty/ Brill: People attach meaning to each new voice they talk to, so we encourage customers not to use multiple voices in a single interaction unless the second voice has meaning: like receiving a new kind of assistance, for example, premium versus standard service levels for segmented customers.
That said, we do see value in rotating personas, just as callers would reach a different person in the call center each time they call. Each of these personas handles a whole call, and research suggests that this approach creates a sense that the system is more personal — less "automated."
Kotelly: In general, it's much more useful for a company to develop one consistent persona that forms an extension of its brand identity. However, I've heard voice portals for example, (automated systems that provide weather, traffic, sports scores, etc.) that use multiple personas very effectively. Each persona is the "expert" on one particular type of information, and the personas hand-off to each other as the caller navigates the system.
Rolandi: In some applications, the use of two or more voice talents may be an effective way to convey different types of information to the user. However, this should not be taken to mean that multiple, invariably bubbly personas would ever be good. If one animated persona is bad, how can more than one be better?
Future Personas
What will future personas be like?
Balentine: I think it's a fad that will disappear by itself as more money is diverted to usability.
Dougherty/ Brill: We see much more personalization where appropriate. For instance, in telematics: since a car is a very personal thing to the owner, it's possible that the vehicle system persona may be quite familiar with the driver. "Wally do you want me to put your seat back to your usual position?"
Taken together, the intelligence of the system and the consistency of the persona with the brand and the task will serve to make personas more invisible: matching so well to the task, getting things done seamlessly and accommodating the caller in the most natural ways. In the end, personas won't be the most noticeable thing about the application - they're just the right tool for the job.
Kotelly: Personas will evolve as society becomes used to speech systems. Comparing usability tests I ran in 1997 compared to 1999, the 1997 participants said they had never spoken to a speech system, while the 1999 ones said they did it all the time. So instead of long introductions that explained that, "in a minute we'll have a short conversation" and tell them how to get "help" if they get stuck, we could be more streamlined.
Rolandi: Let's hope that future personas will be less oriented toward entertaining the user and more oriented towards solving his problem or serving his needs.
Summary
The personality that a speech system projects can have strong effects, positive or negative, on the success of an application. Designers need to be aware of the factors that influence users' evaluations of a persona. While the panelists differ as to the amount of overt attention that should be given to persona design, in general they agree on several important points.
- Every panelist emphasized the importance of focus on the user and the usability of the system.
- We saw a specific example of a case where a well-designed persona improved the bottom line of an application.
- Personas can have negative effects on application success. Several panelists pointed out the potential for extreme personas to be offensive. For example, inappropriate personas (the panelists mentioned "chirpy," "bubbly," "syrupy," "cheery," "animated," and "enthused") can be irritating, can interfere with users accomplishing their tasks, and can lead to negative publicity.
- The persona has to be consistent with the application, for example, overly cheerful personas aren't appropriate where the users are calling about problems with a product. Extremely informal personas aren't appropriate in serious contexts like banking.