Captions Unveils Lipdub App

Captions, providers of an artificial intelligence-powered creative studio, has launched Lipdub, an app that automatically translates and dubs video content into 28 languages.

Lipdub not only translates the subject’s voice but also syncs lip movements to match the translated audio. Users can record or upload a video and the app will change the subject's voice and lip movements to any language, including French, Hindi, Spanish, Italian, Portuguese, and more. Lipdub not only translates languages, but also incorporates dialects and even vocabulary, with options like Gen Z and Texas slang.

At the core of Lipdub's innovative technology is generative AI that facilitates the app's voice translations and synchronization of voice to lip movements. The algorithm can pick up on subtleties of dialects and tones of voice. All videos produced using Lipdub are marked with a Lipdub AI-generated label to ensure transparency.

"Captions entered the translation space in the fall of 2022 with the release of translated captioning. We then launched our voice translation feature, AI Dubbing, in early 2023 and were quickly met with enthusiastic adoption from users around the world," said Gaurav Misra, co-founder and CEO of Captions, in a statement. "AI Dubbing's success inspired us to push this technology to the next level and introduce synced lip movement to the mix. Since then we've been focused on making the technology widely available, leading us to create and launch Lipdub as its own separate app."