A demo of the GANpaint program allows users to manipulate photos of churches or upload their own.

MIT/IBM Program Helps Designers with Changes, IDs Fake Photos

GANpaint, a program led by MIT professor Antonio Torralba as part of the MIT-IBM Watson AI Lab he directs, allows computer scientists to “paint with the neurons” of a neural network — specifically, a popular type of network called a generative adversarial network (GAN).

GANpaint Studio, available online as a demo, shows how a user can upload an image and modify its appearance, from changing the size of objects to adding completely new items, like trees and buildings. Adding video clips would enable computer-graphics editors to quickly compose specific arrangements of objects needed for a particular shot.

“Right now, machine learning systems are these black boxes that we don’t always know how to improve, kind of like those old TV sets that you have to fix by hitting them on the side,” said David Bau, lead author on a related paper about the system with a team overseen by Torralba. “This research suggests that, while it might be scary to open up the TV and take a look at all the wires, there’s going to be a lot of meaningful information in there.”

The researchers hope to gain more control over GAN networks, while also curbing abuse of AI technologies that allow people to alter photos and make fakes look real. Co-author Jun-Yan Zhu says that he believes that better understanding GANs and their errors will help researchers better eliminate fakery.

“You need to know your opponent before you can defend against it,” said Zhu, a post-doctoral researcher at CSAIL. “This understanding may potentially help us detect fake images more easily.”

The MIT researchers who created GANpaint will present a paper on its utility and how it works at the SIGGRAPH conference in Los Angeles this August.