Nvidia has released a new Artificial Intelligence (AI) algorithm capable of producing stunning photos from only a few lines and shapes. Not only are the results strikingly realistic and quick to produce, but the AI also exhibits impressive adaptive capabilities, generating realistic reflections on water or modifying the sky if snow is present.
Called GauGAN, the software is just a demonstration of what’s Nvidia’s neural network platforms have to offer. Its essential purpose is to produce an image as if it were painted by a human, and it seems to work as intended.
The AI uses generative adversarial networks (GAN) — a system in which two neural networks contest with each other, most often to produce different types of images. It’s an unsupervised learning technique: you feed a database into it and “teach” the AI how to progress. Of course, there’s much more to it in the actual process. Nvidia used 1 million images, mostly from Flickr’s Creative Commons.
“It’s like a coloring book picture that describes where a tree is, where the sun is, where the sky is,” said Bryan Catanzaro, vice president of applied deep learning research at NVIDIA. “And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colors, based on what it has learned about real images.”
It has a simplistic user interface and only three tools: a paint bucket, pen, and pencil. But even with these limited means, the algorithm shows impressive performance. You simply select a tool, and then you click on the type of object you want to produce. Want to draw a tree? Just draw a few lines or a rough shape. Rocks or mountains? A rough shape will do. Waterfall? Yeah, that’s just a line.
To make things even more stunning, it does all this in a matter of seconds — of course, while running on highly performant hardware. But Nvidia developers say that with small tweaks, the algorithm can run on any platform.
The AI can generate thousands of different objects, learning from the real world. It also has a randomness figure embedded into it, so if you draw the same shape several times, the result will come out different each time.
It’s easy to see how this could be used for nefarious purposes like producing fake videos — and we’ve already seen how realistic fake videos can be — but Nvidia says it wants to use this AI to make the world a better place, enabling anyone to become an artist and improving the workflow of people working in fields like architecture and urban planning; only time will tell.
If only this was around in the Paint days of yore, eh?
Here’s a quick demo of the AI’s capabilities: