A novel algorithm can turn blurry, unrecognizable pictures of people's faces into eerily convincing portraits. The AI creates realistic-looking faces with up to 64 times the resolution of the blurred images.
While the AI does not produce an exact replica of the initial portrait and it sometimes reimagines features that weren't there in the first place, the end result is remarkably close to reality.
"Never have super-resolution images been created at this resolution before with this much detail," said Duke computer scientist Cynthia Rudin, who led the team.
It's not the first time researchers have worked to recreate the faces behind blurred images. Traditional algorithms, however, are 'trained'. They take a low-resolution image and try to add extra pixels by guessing what would fit. The end result is a plausible average, but relatively low-quality. The pixels filled in by the algorithm tend to be a decent, but not perfect fit, typically producing fuzzy and indistinct images.
But this novel AI opts for a different approach.
Instead of adding detail to a low-resolution image, the system scours a database of artificial, AI-generated faces, looking for the ones that look the most like the blurred image. The AI is then confronted by another AI, in what is called a "generative adversarial network" (or GAN).
This second AI confronts the first one, trying to find errors in the produced face. The first network gets better and better, and when the second one can't tell the difference between the blurred image and the produced face. Even given pixelated photos where the eyes and mouth are barely recognizable, "our algorithm still manages to do something with it, which is something that traditional approaches can't do," said co-author Alex Damian '20, a Duke math major.
Remarkably, the resulting faces aren't actually real -- which, researchers say, means that it can't be used to identify people. It can't turn an out-of-focus security camera photo into an image of a real person. Instead, what it does is generate a new face that doesn't exist, but looks like the image.
Instead, researchers say the same technique could be used on something other than faces, creating sharp, realistic images from a number of images, including (but not limited to) satellite imagery, microscopy, astronomy, and medical imaging.
See the results and upload images for yourself at http://pulse.cs.duke.edu/. Journal Reference: PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models, arXiv:2003.03808 [cs.CV] arxiv.org/abs/2003.03808
