Google’s VideoPoet can turn your text into videos; here’s what you need to know

Google announced its PaLM 2 and Gemini language models last year in mid-2023 and went on to declare how its AI is, in fact, multimodal. This implied how it could generate just about everything — beginning from text or pictures to audio and even videos.

And while ChatGPT’s GPT4 has only recently begun to venture into reproducing photos based on text prompts, Google seems to have gone the extra mile to challenge the notion. Titled ‘VideoPoet’, Google is now the first organisation that has harnessed AI capabilities to produce videos based on text. If you’re excited about its possibilities, here’s what you need to know.

Google VideoPoet: What is it, and what can it do?

Google VideoPoet is essentially a large language model that can produce videos based on text prompts. Whether you’re describing something real or fictional, it can whip up a video on your laptop within seconds. Of course, the trick to having accurate results is to be as descriptive as possible. This could mean you type “a pink cat eating a burger” or simply mention a simple prompt describing the scene.

ALSO READ: How to summarise online articles using this AI tool from Google

Like image generators, VideoPoet can also perform edits based on your feedback. For instance, you can ask for it to crop the frame or trim it according to your needs.

How does VideoPoet work?

A technical paper published by Google goes on to mention how VideoPoet is different when compared to conventional text-to-image and text-to-video. However, Google VideoPoet is a large language model that works on a pre-training process which translates images and video frames into a common language – known as ‘tokens’.

Google additionally claims that it used one billion image-text pairs and 270 million public video samples to train VideoPoet. Eventually, VideoPoet evolved to be capable of predicting these ‘video tokens’, similar to that of an LLM model that predicts text tokens. While the model right now can produce two second videos, VideoPoet is capable of creating longer videos of up to eight to ten seconds.

Who can use VideoPoet?

Google has rolled out a bunch of example videos on its official website, but it has stopped short of announcing a launch date for VideoPoet. While AI image and text generators are gradually rolling out for the masses, video generation is certainly computationally expensive, which means it may not be free for usage. However, we might just have to wait and watch.