These Make-A-Video images are thought-provoking but somehow a bit off-putting according to users and the researchers who designed the algorithm.

Meta Creates New Reality with Text-Prompted Image Generation

The engineers and scientists at Meta have produced a strange and fascinating text-to-video that will set a new bar in the recent popularity of this AI technology. The surge in this tech has been nothing short of impressive. From Dall-E to Midjourny, the images these algorithms have produced have won art prizes and it has become all the rage for some home-computer folks. The Make-A-Video program is a giant improvement according to Devin Coldaway who writes for techcrunch.com about Zuckerburg’s new gadget.

Make-A-Video doesn’t actually change the game that much on the back end — as the researchers note in the paper describing it, “a model that has only seen text describing images is surprisingly effective at generating short videos.”

The AI uses the existing and effective diffusion technique for creating images, which essentially works backward from pure visual static, “denoising,” toward the target prompt. What’s added here is that the model was also given unsupervised training (i.e., it examined the data itself with no strong guidance from humans) on a bunch of unlabeled video content.

What it knows from the first is how to make a realistic image; what it knows from the second is what sequential frames of a video look like. Amazingly, it is able to put these together very effectively with no particular training on how they should be combined.

“In all aspects, spatial and temporal resolution, faithfulness to text, and quality, Make-A-Video sets the new state-of-the-art in text-to-video generation, as determined by both qualitative and quantitative measures,” write the researchers.

So the new system has been receiving a mixed bag of reviews. Yes, say the users it is amazing and fun. However, when you watch what is essentially a text-prompted ‘GIF’ maker, you see what has been called the creepy side of Make A Video. Some movements seem almost otherworldly and downright odd. Some areas are really a little creepy when characters melt into each other.

Make-A-Video created these animated, multi-colored bear figures. (Source: Meta)

People report they think it’s fascinating that however realistic these videos are in one sense, they’re all so bizarre and off-putting in others. That they can be generated quickly and arbitrarily is incredible — and it will only get better. But even the best image generators still have that surreal quality that’s hard to put your finger on.

And if you sign up for this opening version of Make A Video you can get on the list for free updates when they come out.

Make-A-Video also allows for transforming still images and other videos into variants or extensions thereof, much like how image generators can also be prompted with images themselves. The results are slightly less disturbing. Take a look at these images in action at the link below and see what all the excitement is about.

read more at techcrunch.com