A Change Is Gonna Come
In honor of the new year, let’s discuss one of the most overlooked issues in designing a quality speech-based user experience: change management. In my last few projects, implementing a rigorous change management process seemed more important than the user redesign experience itself. In each case, I was redesigning a system that had either languished for years without attention or had been subjected to almost continuous change based on the whim of virtually anyone who decided a new word was needed. In both cases, creating a rigorous change management process was essential to a disciplined, responsive user experience and, in turn, realization of business benefit.
What goes into effective change management? In most types of user experience design, including speech-based interaction, there are several important stages:
1. Initiating a change request. The first step is to raise an issue that would require a change and communicate it to the user experience owner. In some organizations, failure occurs as early as this initial step: There is no centralized control and thus no one person has oversight of the user experience from a holistic, strategic point of view. The individual responsible must be able to refuse change requests that will damage usability. Such empowerment is especially critical when a change request has been put forward without articulation of the underlying problem. For example, “Add X to the menu” or “The prompt needs to say Y” are not usability problems, but rather design modifications that do not provide adequate rationale for change.
2. Validating the request. The need for change must be established not only from a business or technical perspective, but also from the user perspective. This is the step that is most often omitted, even when a change management process is in place. Lately, I’ve seen situations where organizations have the most advanced reporting packages available but a haphazard approach to analysis. In other situations, there is simply no data available; the automated system is a black box. To validate the need for an experience change, data is required, such as that provided by tuning, call data analysis, or usability testing, which is then subjected to skilled analysis. Many times, such data will confirm that no change is necessary or highlight an unexpected problem other than the one that was initially identified.
3. Determining the design remediation. Depending on the data used for validation, multiple design changes may be necessary. For example, a tuning might most obviously point to an omitted grammar item but may identify one or more prompt changes as well. In usability testing, participants will often experience delayed or compounding confusion as they progress through a flow. Therefore, it’s important to fully assess all causal factors and not get sidetracked by those that are easiest to implement.
4. Obtain a sign-off from stakeholders. Once the redesign effort has been identified, stakeholders should be presented with the change request, evidence of need, and design remediation. This step not only ensures their agreement with the change but educates stakeholders that there is not always a straight line between a perception that a change is required and a specific design.
5. Implementing the change. This stage may involve rewriting menus or prompts, reorganizing options, rerecording, and coding. The most common change request I’ve experienced is that a function “needs to be added to the Main Menu,” often without any consideration of volume associated with the function, whether there is a so-called Main Menu, or the extent to which the particular function even fits into the highest level of an information architecture. The Main Menu, or first prompt, is valuable real estate; correspondingly, unequivocal evidence should be required to make a change there.
6. Monitoring the outcome of change. Here we return to call data monitoring, usability testing, tuning, or other objective means of gathering information to determine whether the change has had the intended impact. Whether a change has or hasn’t produced a desired outcome, the cycle of change management should start again. Human language is a living, dynamic phenomenon. Monitoring and reassessment of any language-based experience must be ongoing as well.
A process like this ensures changes are necessary, effective, and based on an objectively validated need, as opposed to conjecture and assumption. In the realm of change management, it behooves most of us to think like a psychologist: Be highly skeptical of your own perception of user behavior and assume no usability problem exists (the null hypothesis) until objective, high-quality evidence demonstrates otherwise.
For this year’s resolution, go ahead and re-evaluate your own change management process. After all, a change is gonna come: How will you respond?
Melanie Polkosky, Ph.D., is a social-cognitive psychologist and speech language pathologist who has researched and designed speech, graphics, and multimedia user experiences for almost 15 years. She is currently a human factors psychologist and senior consultant at IBM. She can be reached at polkosky@comcast.net.