Air Cover in the Trenches
When I read the hot topics on the VUI design message boards, I’m always struck by what isn’t there, what isn’t being lamented or debated, what captures our attention and what doesn’t. Recently, I asked VUI designers to identify the biggest issue facing our field. What’s striking is that only 8 percent of respondents cited a lack of research or data, compared to 43 percent who mourn our perceived lack of professionalism and 21 percent who point to the absence of best practices. What’s going on here? Don’t we get that we won’t garner much professional respect, or our practices can’t be the best, without research?
In the state of the art today, there’s a yawning gulf between VUI research and practice. This might be because:
• Social/communicative behavior is complicated. In fact, the mere study of it in the wild can change behavior.
• Most existing research is difficult to apply or tangentially related to the specific problems of VUI design.
• Existing knowledge on communication is, at best, diffusedly spread among academic disciplines (English, linguistics, interpersonal and mass communications, psychology, speech-language pathology, marketing). It takes a concerted effort and lots of free time to access relevant findings.
• Pertinent disciplines often have very limited cross-fertilization and can be rigidly defined.
• Numerous qualitative and quantitative methodologies exist for studying communication.
• Even though the VUI field boasts a large number of advanced degrees—in my survey, roughly two-thirds of respondents held a master’s or doctorate—they generally focus on design only.
• Designers have wildly divergent backgrounds, and many lack the necessary basis in applied behavioral science methodologies, variable selection, or data collection and analysis techniques for user research or usability testing.
• A reluctance to share data or research for fear of selling company secrets in our competitive climate is apparent.
• And, being the new kids on the block, nobody’s ever really heard of VUI design.
There are probably more reasons, but you get the idea.
In spite of this, the bottom line is that a designer without data-based rationale saying "I think we should say the prompt this way" is just another person with another opinion. And why does that make the VUI designer any better than the next guy who wants it worded his way? Doing design by intuition is no way to build a credible professional discipline.
Let’s take a cue from traditional human factors and human-computer interaction. In Human Factors in Engineering Design, Mark Sanders and Ernest McCormick asserted, "The central approach of human factors is the application of relevant information about human capabilities and behavior…largely based on experimentation and observation. Research plays a central role in this regard."
More recent writings on the subject also point to the symbiotic relationship between research and practice: Basic research provides a basis for the design of new products, and practical design stimulates the questions that will advance our knowledge base. Thus, prerequisite knowledge for any VUI practitioner is behavioral research methodologies and findings related to our social-communicative design domain.
As a field, we need to start talking about how to advance data-driven design and stop complaining that no one respects us. Designers should demand the air cover that comes from research when they’re in the trenches. If you don’t have this background, you need to be persistent about getting it from your company, professional organizations, and industry conferences. Don’t be afraid to tell mentors, peers, and managers what you need to design effectively. They should praise your self-insight and either educate you or hire someone with the complementary skills to facilitate your design work.
As professionals, we need to insist that user feedback obtained with broadly accepted methods be part of every project; in fact, it’s more important than the VUI design itself. In addition, no company can reasonably claim to have a state-of-the-art VUI practice without a rigorous user research and usability program. In academia, we need to start addressing the needs of our applied science by breaking the arbitrary walls between departments and facilitating multidisciplinary research. To garner professional respect and have the only best practice that matters, we need to make research a focal component of our user-centric VUI design process. Let’s start the advocacy today.
Melanie Polkosky, Ph.D., is a social-cognitive psychologist and speech-language pathologist who has researched and designed speech, graphic, and multimedia user experiences for more than 12 years. She is currently a human factors psychologist and senior consultant at IBM. She can be reached at polkosky@comcast.net.