Proof is in the Applications

The gauntlet was thrown down in late July when TuVox accepted entry into the inaugural Speech Solutions CHALLENGE. Shortly thereafter, six other teams followed suit and one of the most exciting demonstrations utilizing speech technologies was underway. The anticipation reached a crescendo on the morning of September 29 when CHALLENGE committee chair Ken Rehor stepped up to the podium inside New York City’s Marriott Marquis and delivered details of the application. Needless to say, rarely have the hearts of speech developers raced, or their fingers moved, so quickly upon receiving an assignment. In addition to TuVox, teams from Edify, Intervoice, VoiceObjects, Vocalocity/ Newfound Communications/ScanSoft, Angel.com/Comverse/Voice Partners and Convergys/Audium worked under the watchful eye of SpeechTEK attendees and CHALLENGE organizers. Each squad approached the CHALLENGE with a different strategy. “To prepare for the Speech CHALLENGE we held a couple of practice sessions,” said Ron Owens, director of professional services at Intervoice. “In order to get used to working under time pressure, we tried to use contest conditions. We didn’t give the team the applications in advance and everything had to be done in six hours. It gave the team a chance to learn how to best work as a team, dividing up labor and setting up interim milestones.” THE APPLICATION Following months of deliberation, the CHALLENGE Committee zeroed in on a situation that would require callers to make appointments at a car repair shop. The application was chosen because it is typical of thousands of businesses that have callers schedule appointments. To meet the CHALLENGE callers needed to use the application to perform the following tasks: Collect user and vehicle information (10digit telephone number; make, model and year of vehicle); Schedule an appointment on the desired day and time for routine maintenance, repair a specific problem or record a description of the problem for the service manager (diagnostic consultation); Specify time for making appointments where some time slots were unavailable. Alternatives needed to be given. The caller had to be told the date and time to drop off the vehicle and when pickup was expected; and A unique six-digit confirmation code needed be provided upon successfully scheduling an appointment. Early on in the CHALLENGE, most teams realized they were in a real race against the clock. This was due to some rather time consuming tasks, such as voice prompts needing to be recorded (TuVox recorded 376 voice prompts for its application) and many times re-recorded. So, in order to have a deployed application by the deadline, for the most part, the teams developed a get-itdone mentality. Without the intense time constraints, other industry best practices could have been utilized to implement more confirms and corrections in to the application. However, the applications deployed by all seven teams were verified by SpeechTEK Certified Speech Application Testers VocaLabs and Voice Architects and met the criteria. “In light of the short development time the challenge applications demonstrated strong performance characteristics,” said John Moffly of Voice Architects. “The average errors per state at 0.35 for the applications was what one would expect to see for a pre-production effort, but the raw recognition rate at 65 percent was only 18 percent below what Voice Architects typically sees in production applications. All in all a strong validation of how far the industry has come in making speech effective and affordable.” “We were impressed with the performance of all seven teams,” said Peter Leppik of VocaLabs. “Every team managed to create a functioning VUI from scratch. As expected, not everything was polished and smooth, but this proves that with the right tools and expertise, quality speech applications can be deployed even on a tight schedule.” CHALLENGING, BUT A BLAST Team Edify was decked out in racing attire and exemplified true team camaraderie. The conglomerate of Angel.com/Comverse/ VoicePartners constructed its application wearing hardhats and incorporated tons of personality into it’s deployment. Dawn, a voice that screams Chicago, fields calls for Sam’s Garage. And depending on the problem, Sam himself jumps on the line to assist the call. “Speech technology doesn’t really exist until you give it a voice,” added Melissa Dougherty, principal at Voice Partners. “Call ‘Sam’s Car Repair’ and talk to Dawn Brevik, she’s persona personified, and if your car is making a strange noise, just ask for Sam.” While time limited what some teams could do, others did implement intriguing bells and whistles. Voice Objects, for example, answered its call with good morning, afternoon or evening depending on the time of day. It offered SMS service in order for the confirmation number to be text messaged to the callers mobile phone and the choice of male or female personas to interact with. Amazingly, the Voice Objects application server was in Germany, the VoiceXML server was in London and application development was completed in New York City via web browsers. The calls were connected from the United States to London over VoIP. “We were thrilled with the results of SpeechTEK 2003 and the associated Solutions CHALLENGE,” said Tiemo Winterkamp, head of product management & strategy of VoiceObjects. “Especially the Solutions CHALLENGE showed that younger companies in the market, focused on open standards based solutions, can keep pace if not overhaul the older legacy systems architectures. VoiceObjects demon strated its lightweight Web-browser based architecture for development and deployment of voice services at the CHALLENGE and on the exhibition booth with tremendous success.” Perhaps the most fully-featured application was created by the troika of Vocalocity, Newfound Communications and ScanSoft. This demonstration offered multiple languages, the option to leave a message and when the recognition worked flawlessly was able to complete a call in two-and-a-half minutes. This team completed its first demo within one hour. “The Speech Solutions CHALLENGE gives conference attendees a perfect opportunity to compare the traditional proprietary approach to IVR with standards-based voice application technology,” said Michael L. Bergelson, president and CEO of Audium. “We feel that this is a great forum to demonstrate the depth and maturity of today’s standards-based speech platforms and showcase the advantages of VoiceXML solutions in today's marketplace." THE PEOPLE'S CHOICE The Speech Solutions CHALLENGE was spotlighted throughout SpeechTEK. Registered SpeechTEK attendees were given the opportunity to call and demo each application via a toll-free line. The calls were hosted by Intel and West Corporation and a calling Pavilion was sponsored by Voice Genie. Each registered attendee had the opportunity to cast a vote for his or her favorite application, and the team that received the most votes was dubbed the People’s Choice Award winner. The efficiency of the Convergys/Audium team paid off as it was declared the People’s Choice. This team mapped out its entire day, from analysis and call center flow to deployment and production testing in 15 minute increments. It completed the coding of its application by 2:30 PM and tested for the final two hours. “We are delighted to receive this ‘People’s Choice’ endorsement of our solution,” said Ted Cwiok, general manager of speech solutions for Convergys. While Convergys/Audium was selected as the attendees favorite application, by the closing of SpeechTEK it was perfectly clear that the Speech Solutions CHALLENGE was a success for everyone involved. “We’re so pleased with how the CHALLENGE went,” said Matt Tosiello, marketing manager for Angel.com. “Despite the obvious time constraint, we feel our team was able to step up to the challenge and provide not only a functional speech application, but one that would resonate with the individual users.” CONCLUSION Rehor put it best when he compared a speech application to a puppy. Both have start up costs and ongoing costs. There is the initial cost of purchasing the puppy, and the ongoing cost of feeding and caring for the puppy. Speech user interfaces have both initial development costs and ongoing costs including tuning and maintenance. The CHALLENGE demonstrated the initial development costs can be minimized. But don’t forget the ongoing tuning and maintenance costs which were not addressed by the CHALLENGE. “Giving seven teams the same situation to develop a speech application for and having seven individual solutions deployed underscored that developing a speech application is very much an art, and not a science,” said Bryan Mekechuk, partner at Pacific Crest Consulting Group. “Although tools and methods can reduce the time and effort required, designing the application is unique and important to callers’ perceptions of the solution.” “One of the most vivid learnings from this exercise was the central role of professional services to making a speech app live,” said Mark Plakias of Zelos Group, a coorganizer of the event. “We thought tools would be the big story, but smart people in coordinated teams turned out to be the bigger story.” If you have comments about the Speech Solutions CHALLENGE, please e-mail Ken Rehor at ken@rehor.com.