ElevenLabs Launches AI Dubbing

ElevenLabs, a provider of voice artificial intelligence technology, has launched AI Dubbing, a tool that enables users to automatically translate any speech into up to 20 other languages while maintaining the original speaker's voice and speech patterns.

AI Dubbing combines the company's research on multilingual speech synthesis, voice cloning, text and audio processing, into a single tool The AI Dubbing feature supports voice translation across more than 20 languages currently supported by the Eleven Multilingual v2 model, including Hindi, Portuguese, Spanish, Japanese, Ukrainian, Polish or Arabic.

This move further extends ElevenLab's research across the audio AI space; enabling emotions and intonation from original audio to be transformed to dubbed audio. The company has also developed its own method to detect the voices of different speakers and methods to remove background noise, differentiating music and noise from dialogue. This means the tool can handle audio or video containing multiple speakers, ensuring each voice remains distinct and recognizable across languages.

"Growing up in Poland, we watched English language movies that had been dubbed by a single narrator. It means every actor has the same voice. It takes so much magic out of the experience. The release of AI Dubbing is our biggest step yet toward eliminating these linguistic barriers of content. It will help audiences enjoy any content they want, regardless of the language they speak. And it will mean content creators can easily and authentically access a far bigger audience across the world. Having experienced these language barriers first hand, my co-founder Piotr and I are so excited to finally bring AI dubbing to life," ElevenLabs CEO and Co-Founder Mati Staniszewski said in a statement.