You know that stereotypic scene from the CSI movies when they zoom in on a car they can barely see and then read the license plate clearly? Well, that bit of fiction might turn into reality, as computer scientists from the Max Planck Institute for Intelligent Systems in Tübingen have used artificial intelligence to create high-definition images from low-resolution photos.
It’s not the first time researchers have looked at something like this. The technology is called single-image super-resolution (SISR). SISR has been researched for decades, but without much success. No matter how you look at it, the problem was that they just didn’t have enough pixels to generate a sharp image.
Now, researchers developed a tool called EnhanceNet-PAT, which uses AI to generate new pixels and “fill” the image up.
“The task of super-resolution has been studied for decades,” Mehdi M.S. Sajjadi, one of the researchers on the project, told Digital Trends. “Before this work, even the state of the art has been producing very blurry images, especially at textured regions. The reason for this is that they asked their neural networks the impossible — to reconstruct the original image with pixel-perfect accuracy. Since this is impossible, the neural networks produce blurry results. We take a different approach [by instead asking] the neural network to produce realistic textures. To do this, the neural network takes a look at the whole image, detects regions, and uses this semantic information to produce realistic textures and sharper images.”
First, the neural network was fed a large data set of different images. It learned different textures and colors. Then, it was given downscaled images which it had to improve. The upscaled results were then compared to the initial photo, with the algorithm analyzing and learning from these differences. After a while, it did a good enough job without any human input.
Of course, this isn’t a magic fix and not all photos can be fixed (at least not yet), but results are exciting. As for the applications, there’s no shortage of those, Sajjadi says. The algorithm could be used to restore old family photos or give them a good enough resolution for larger prints; on a more pragmatic level, the technology could greatly help in object recognition, which has potential in detecting pedestrians and other objects in self-driving cars.
Journal Reference: Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch. EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis.
