2018 SPEECH TECHNOLOGY IMPLEMENTATION AWARD: SAYIN’ IT SAM
Our three winning client companies used speech and voice applications to solve a business problem or seize a business opportunity
A Game Platform Helps Nonverbal Children Find Their Voice
Supporting a child affected by severe autism and speech delays takes commitment and creativity. Nea Hanscomb’s son, Palmer, now 17, was labeled “nonverbal” at a very early age and also recently diagnosed with apraxia. As Hanscomb—who is CEO and founder of San Francisco-based Appropo Software—investigated tools and learning strategies to help Palmer, she noticed about five years ago that he was very motivated by his iPad. “I was amazed, considering his lack of interest in a lot of other activities,” she recalls. The revelation led her to begin searching for speech recognition applications. When she couldn’t find anything suitable, she decided to leverage her years in technology and her experience as a graphic designer to create a solution herself.
If You Want It Done Right…
Those early efforts began with a prototype built on Nuance’s Dragon Mobile. Hanscomb says it worked fairly well, though the graphics weren’t at the level she wanted. “I felt it had potential, but I needed to make a more polished application that was more appealing to the market,” she says. Recent statistics released by the CDC show that autism spectrum disorder (ASD) affects one in 59 children. That’s a dramatic increase from 2012, when the CDC estimated that ASD affected one in 88 children.
Taking the lessons she learned from the prototype, Hanscomb found another group of specialty app developers who worked with her to create better animations. “We also added the character ‘Sam’ to my original idea, which is more engaging for kids,” she explains, and the Sayin’ It Sam platform was born. Hanscomb not only saw an opportunity to assist children with autism, she also wanted to target early language learners. “My market is pretty niche, and I felt it could apply to typically developing kids as well,” she says.
But the second iteration wasn’t quite fast enough to provide the experience Hanscomb wanted. Dragon’s cloud connections proved to be a weak spot. “There was a delay in the responses, and this market couldn’t be patient enough for that,” Hanscomb says. She then came across an application built by Zachary Rubin, who was working in the University of California Santa Cruz’s Assistive Sociotechnical Solutions for Individuals with Special needs using Technology (ASSIST) Lab at that time. Hanscomb tracked down Rubin, who is now head engineer at Appropo Software and a professor at Renton Technical College in Washington State, and together they worked to recode Hanscomb’s game to use OpenEars (Politepix), the speech recognition software behind the ASSIST Lab’s earlier efforts.
The Sayin’ It Sam platform works locally on iPad and iPhone devices, providing users with fast interactions. Two free and four paid games are available, each featuring the engaging Sam character, who escorts players through the various game screens as they cover shapes, colors, and similar concepts. Popping balloons and fluttering birds keep children entertained while the game encourages them to interact through speaking. “We try to include approximations for kids that make common speech errors, such as eliminating the first or last consonant of a word,” Hanscomb explains. A number of other known speech variances have also been designed into the platform.
The Future’s Bright for Sayin’ It Sam
Obtaining more comprehensive voice data for Sayin’ It Sam remains a challenge, as privacy issues around children’s speech limit access to much of the information. “They’re not made available due to COPPA [Children’s Online Privacy Protection Act] regulations, which we are in compliance with,” Hanscomb explains. She reaches out to volunteers through social media to assist with recordings of 3- to 5-year-olds, and by obtaining parental permission, she’s able to make recordings herself at local preschools.
Translating the application into additional languages is also a key priority. Hanscomb is already looking to translate Sayin’ It Sam into Spanish to assist ESL learners. A number of other language options, Arabic and Mandarin among them, have also generated interest. Hanscomb sees no limits on what the technology can do. “As soon as we improve the code and increase our market share, I’m looking for a technical partner to take Sayin’ It Sam to the next level, with different translations, more games, and product development.”
Related Articles
At a time when the pace of change in speech technology evolution and adoption seems to be on hyperdrive, a few key trends are pointing where the industry will move in the coming months.
17 Sep 2018
Analyzing, and Improving, Agents' Performance
10 Sep 2018
As part of this year's Speech Industry Awards issue, we wanted to give some attention to the many innovative uses of speech technology—and, more broadly, artificial intelligence (AI)—we've seen throughout the year in almost every industry vertical. Read on to find out more about just a few of the implementations that caught our eye in 2018.
05 Sep 2018
Letting Customers Send Their Money Where Their Mouth Is.
27 Aug 2018
We present the thinkers and innovators who are creating new tools and approaches for speech technology—and fostering the next generation of talent.
06 Aug 2018
These cutting-edge vendors are leading the way in AI, analytics, natural language, smart speakers, and more.
01 Aug 2018