Speech Technology Magazine

 

2018 SPEECH TECHNOLOGY IMPLEMENTATION AWARD: SAYIN’ IT SAM

A Game Platform Helps Nonverbal Children Find Their Voice.
By Julie Knudson - Posted Aug 13, 2018
Page1 of 1
Bookmark and Share

Our three winning client companies used speech and voice applications to solve a business problem or seize a business opportunity

A Game Platform Helps Nonverbal Children Find Their Voice

Supporting a child affected by severe autism and speech delays takes commitment and creativity. Nea Hanscomb’s son, Palmer, now 17, was labeled “nonverbal” at a very early age and also recently diagnosed with apraxia. As Hanscomb—who is CEO and founder of San Francisco-based Appropo Software—investigated tools and learning strategies to help Palmer, she noticed about five years ago that he was very motivated by his iPad. “I was amazed, considering his lack of interest in a lot of other activities,” she recalls. The revelation led her to begin searching for speech recognition applications. When she couldn’t find anything suitable, she decided to leverage her years in technology and her experience as a graphic designer to create a solution herself.

If You Want It Done Right…

Those early efforts began with a prototype built on Nuance’s Dragon Mobile. Hanscomb says it worked fairly well, though the graphics weren’t at the level she wanted. “I felt it had potential, but I needed to make a more polished application that was more appealing to the market,” she says. Recent statistics released by the CDC show that autism spectrum disorder (ASD) affects one in 59 children. That’s a dramatic increase from 2012, when the CDC estimated that ASD affected one in 88 children.

Taking the lessons she learned from the prototype, Hanscomb found another group of specialty app developers who worked with her to create better animations. “We also added the character ‘Sam’ to my original idea, which is more engaging for kids,” she explains, and the Sayin’ It Sam platform was born. Hanscomb not only saw an opportunity to assist children with autism, she also wanted to target early language learners. “My market is pretty niche, and I felt it could apply to typically developing kids as well,” she says.

But the second iteration wasn’t quite fast enough to provide the experience Hanscomb wanted. Dragon’s cloud connections proved to be a weak spot. “There was a delay in the responses, and this market couldn’t be patient enough for that,” Hanscomb says. She then came across an application built by Zachary Rubin, who was working in the University of California Santa Cruz’s Assistive Sociotechnical Solutions for Individuals with Special needs using Technology (ASSIST) Lab at that time. Hanscomb tracked down Rubin, who is now head engineer at Appropo Software and a professor at Renton Technical College in Washington State, and together they worked to recode Hanscomb’s game to use OpenEars (Politepix), the speech recognition software behind the ASSIST Lab’s earlier efforts. 

The Sayin’ It Sam platform works locally on iPad and iPhone devices, providing users with fast interactions. Two free and four paid games are available, each featuring the engaging Sam character, who escorts players through the various game screens as they cover shapes, colors, and similar concepts. Popping balloons and fluttering birds keep children entertained while the game encourages them to interact through speaking. “We try to include approximations for kids that make common speech errors, such as eliminating the first or last consonant of a word,” Hanscomb explains. A number of other known speech variances have also been designed into the platform.

The Future’s Bright for Sayin’ It Sam

Obtaining more comprehensive voice data for Sayin’ It Sam remains a challenge, as privacy issues around children’s speech limit access to much of the information. “They’re not made available due to COPPA [Children’s Online Privacy Protection Act] regulations, which we are in compliance with,” Hanscomb explains. She reaches out to volunteers through social media to assist with recordings of 3- to 5-year-olds, and by obtaining parental permission, she’s able to make recordings herself at local preschools.

Translating the application into additional languages is also a key priority. Hanscomb is already looking to translate Sayin’ It Sam into Spanish to assist ESL learners. A number of other language options, Arabic and Mandarin among them, have also generated interest. Hanscomb sees no limits on what the technology can do. “As soon as we improve the code and increase our market share, I’m looking for a technical partner to take Sayin’ It Sam to the next level, with different translations, more games, and product development.”

Page1 of 1