All these hyper-realistic faces were generated using NVidia’s new algorithm and it’s awesome — and a bit scary.
Credits: NVidia.
Women, children, different skin tones and complexities — it doesn’t matter: NVidia’s algorithm generates them all equally well. The algorithm separates coarse features (such as pose and identity) from finer details, producing faces in different positions and lighting. It can even throw in some random details like blemishes or freckles.
To better illustrate this ability, the computer scientists who created this exhibit a face generated with different amounts of noise. The results are truly impressive.
Effect of noise inputs at different layers of our generator. (a) Noise is applied to all layers. (b) No noise. (c) Noise in fine layers only. Noise in coarse layers only. We can see that the artificial omission of noise leads to featureless “painterly” look. Coarse noise causes large-scale curling of hair and appearance of larger background features, while
the fine noise brings out the finer curls of hair, finer background detail, and skin pores.
“We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature,” a paper published on arXiv reads. “The new architecture leads to an automatically learned, unsupervised separation of high-level
attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis.”
Neural style transfer, the technique they used here, is one that has been used before to generate synthetic images — think of those algorithms that let you transform a photo into a particular style. Imagine a landscape image as if it were painted by Van Gogh, for instance. Neural style transfer typically requires three images: a content image, a style reference image, and the input image you want to style. In this case, NVidia taught its generative adversarial network (GAN) to generate a number of styles: faces with glasses, different hairstyles, different ages, etc. As far as we can tell, there’s not a particular weak point in the algorithm’s outputs.
However, the network also tried generating cat faces — and while results are still impressive they’re not as good. Some are indeed indistinguishable from the real thing, but a few are quite bizarre. Can you spot them?
Image credits: NVidia.
“CATS continues to be a difficult dataset due to the high intrinsic variation in poses, zoom levels, and backgrounds,” the team explain.
However, they did much better on different types of datasets — with cars and bedrooms.
AI-generated cars and bedrooms. Image credits: NVidia.
So what does this mean? Well, for starters, we might never be able to trust anything we see on the internet. It’s a remarkable achievement for NVidia’s engineers and for AI progress in general, but it’s hard to envision all the ways in which this will be used. It’s amazingly realistic — maybe even too realistic.
Here’s a video detailing the face generation process:
