Replacing Text with Context
Like most sites with a heavy video component, YouTube’s search capability relies heavily on tagging—assigning video files with keywords so that search engines can identify them.
While textual search works perfectly well when a user wants to find textual information, determining video content by tagging is very much like comprehending a movie by reading the blurbs on its poster.
If a media company wants to ensure users are getting the content they’re searching for, it must leverage multimedia search, which incorporates both image and speech recognition to scour a video’s actual content. At last month’s DemoFest, Microsoft’s adCenter Labs showcased Contextual Ads for Video, which uses speech recognition to transcribe conversations. Using "lexical analysis," and by listening to pauses in the speech, the application segments each conversation by topic, giving it context and making it searchable. However, the tools displayed at DemoFest are years away from live deployments.
By contrast blinkx, a multimedia search engine, has been particularly active for the past year, providing search capabilities for media content providers like Britain’s ITN Source and French news station FRANCE 24.
"Once people are aware of what can be done, they tend to go very quickly from concept to application and from application to ROI," says blinkx CEO Suranga Chandratillake. Most of blinkx’s customers have revenue models tied directly to video views, either because users pay to watch them or because of advertising that appears alongside the video.
"And you can do a lot more than [direct search]," Chandratillake adds. For instance, enterprises can suggest videos to customers based on past viewing tendencies. "The point is, don’t just think search. Think beyond that," Chandratillake says.
But because multimedia search is a relatively nascent technology, enterprises that want it don’t always know how to incorporate it. "People aren’t aware of what they can do and how they can do it," Chandratillake says. "As a result, you’re getting a lot of Frankenstein solutions—people trying to build things internally or use technology that wasn’t designed for video."
For instance, media companies sometimes try reconfiguring an existing text-based search engine to accommodate video or audio files. Doing so leads to unreliable searches and fewer views—particularly detrimental for a media company that relies on users watching its content.
However, the core technology to enable multimedia search, such as natural language processing output or speech-to-text (STT), still isn’t fully optimized to satisfy casual users.
EveryZing CEO Tom Wilde, whose search engine uses STT technology from BBN, acknowledges that users expect the intelligence of multimedia search to equal that of textual search. "And the actual intelligence [for multimedia search] lags behind that expectation, and that’s what we’re working to bring up," Wilde says.
The problem, according to Wilde, is search engine optimization for video content, or the way engines properly categorize the relevance of video or audio files based on the searcher’s needs. Because the speech recognition component isn’t as reliable as the Web crawlers used to scour text, multimedia search isn’t yet as efficient as text search.
This hinders the potential of universal search, which, according to Google’s definition, "will blend listings from its news, video, images, local, and book search engines among those it gathers from crawling Web pages." A Google search for "Barack Obama" returns both book and news results, as well as the presidential candidate’s official site and Wikipedia entry. However, Google doesn’t bring back video entries featuring Obama unless the user specifically requests them, and almost all of those videos are culled from Google-owned YouTube. Ultimately, Google’s universal search doesn’t actually search universally.
"Users are format-agnostic," Wilde says. "The problem with looking for information, I don’t care if it’s in a video or document, I just want it to be relevant." But it’s unlikely that Google’s culture will change. "Yahoo and Google and Ask are not in business to help [multimedia content] publishers primarily," Wilde says. "Aside from delivering ads to them, the publisher plays a minor role in terms of having a seat at the table. If the content publisher isn’t an active participant, you will always fall short."