HeyGen offers AI-powered video translation with impressive lip-syncing capabilities


AI startup HeyGen has unveiled a tool that can translate videos up to five minutes long into different languages. The software also clones the voice of the person in the video and adjusts lip movements accordingly.

In a test by Twitter user Jon Finger, the translation worked flawlessly despite the background noise of a busy street. In the edited video, the distracting sounds are filtered out.

The synthetic voice in a different language is very reminiscent of the original, although it still sounds slightly robotic or mechanical. It is also remarkable how faithfully the intonation is reproduced.

The fact that the lip movements have been changed by the AI is barely visible to the naked eye. HeyGen makes another adjustment to the video track, though, because the face appears much brighter after the translation than in the original clip.



As a native speaker, I can confirm that the German translation is somewhat monotonous, but definitely authentic. If I didn’t know it was an AI translation, I might find the intonation unusual, but not unnatural.

The official demo video features popular tech YouTuber Marques Brownlee with a Spanish voice and Apple CEO Tim Cook with an Indian voice.

Video: HeyGen

Beta: Ten languages input, eight languages output

The “Video Translate” tool within the experimental offerings of HeyGen Labs is currently in an open beta phase. At launch, it supports English, Spanish, French, Chinese, German, Italian, Portuguese, Dutch, Hindi, and Japanese as input languages. However, it can only translate into English, Spanish, French, Hindi, Italian, German, Polish and Portuguese. “Many more” languages will be added in the coming weeks.


In addition to the technical requirements, such as a minimum length of 30 seconds, MP4, Quicktime and Webm file types, and a resolution between 480 x 480 pixels and 1920 x 1920 pixels, HeyGen provides other tips to get the best possible result.


Israeli start-up made headlines with such a service. However, no major production has yet been released with AI dubbing. However, “Top Gun: Maverick” used AI to bring back Val Kilmer’s voice, so it’s fair to say that the technology is making its way into mainstream movies.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top