Neural Network Generates ‘Talk’ from Single Image, Video from only 8 Frames

It’s been possible to create entire scenes using an actor’s image, but that’s only because of existing film with a breadth of expressions that filmmakers have mined using computer-generated images. Now, with a single image a neural network is able to animate a face to “talk” like a living human.

So far, the talking head models include Mona Lisa, Albert Einstein and Marilyn Monroe.

What’s interesting, however, is that historical figures like Einstein and Monroe have been captured in many films, so when you compare their expressions in films with those created by the convolutional neural network, it’s easy to pick out the fakes.

According to a story in LiveScience.com by Mindy Weisberger, “researchers trained the algorithm to understand facial features’ general shapes and how they behave relative to each other, and then to apply that information to still images.”

Egor Zakharov, an engineer with the Skolkovo Institute of Science and Technology, and the Samsung AI Center, both in Moscow, narrated the YouTube video showing how the animated images work. The science behind it was published in the journal arxiv.org.

The “creepy” factor of CGI-generated humans has been explained by Karl MacDorman, a professor in the Computer-Human Interaction Program at Indiana University, as the “Uncanny Valley Effect,” in which the image is close, but not quite human in its imperfections.

MacDorman described the uncanny valley this way: “an eerie feeling elicited by a human character that is highly realistic in some aspects but not others,” he told Life’s Little Mysteries, a sister site to LiveScience. For example, as MacDorman discovered in a recent study, “this feeling can be elicited by a mechanical-looking robot that sounds human or a human looking robot that sounds mechanical — or moves in a mechanical way.”

The potential for abuse exists, but won’t be a threat until the technology advances enough to eliminate the Uncanny Valley Effect. Deepfake videos that don’t give themselves away are yet to come. A video of Salvador Dali, however, used multiple images, an actor and a recorded series of statements to create a film and allow the avatar of the famous painter to realistically interact with visitors in the artist’s namesake museum.

read more at livescience.com