homehome Home chatchat Notifications


Text AI can produce images -- and it's very good at it

AI is already nearing sci-fi territory.

Mihai Andrei
July 31, 2020 @ 1:26 pm

share Share

This AI was designed to work with text. Now, researchers have tweaked it to work with images, predicting pixels and filling out incomplete images.

GPT-2 is a text-generating algorithm. Trained on billions and billions of pages of words, it’s capable of absorbing the structure of the text and then writing texts of its own, starting from simple prompts. The algorithm also uses unsupervised learning, which makes it much easier for researchers to train it without taking a lot of their time. The AI system was presented in February and proved capable of writing convincing passages of English.

Now, researchers have put GPT-2 up to a different task: working with images.

The algorithm itself is not well-suited to working with images, at least not in a conventional sense. It was designed to work with one-dimensional data (strings of letters), not 2D images.

To bypass this shortcoming, researchers unfurled images into a single string of pixels, essentially treating pixels as if they were letters. After the algorithm was trained thusly, the new version of the algorithm was called iGPT.

They then fed halves of images and asked the AI to complete the picture. Here are some examples:

Image credits: OpenAI.

The results are already impressive. If you look at the lower half of the photos above, they’re all generated by the AI, pixel by pixel, and they look eerily realistic. The three birds, for instance, are shown standing on different surfaces, all of them believable. The droplets of water too show different veridic possibilities, and all in all, it’s an amazing accomplishment from iGPT.

This also hints at one of the holy grails of machine learning: generalizable algorithms. Nowadays, AIs can be very good at a single task (whether it’s chess, text, or images), but it’s still only one task. Using one algorithm for multiple tasks is an encouraging sign for generalizable approaches.

The results are even more exciting when you consider that GPT-2 is already last year’s AI. Recently, the next generation, GPT-3, was presented by researchers and it’s already putting its predecessor to shame, by generating some stunningly realistic texts.

There’s no telling what GPT-3 will be capable of, both in terms of text generation and image generation. It’s exciting — and a little bit scary — to imagine the results.

The original paper can be read here.

share Share

China Resurrected an Abandoned Soviet 'Sea Monster' That's Part Airplane, Part Hovercraft

The Soviet Union's wildest aircraft just got a second life in China.

A Rocket Carried Cannabis Seeds and 166 Human Remains into Space But Their Capsule Never Made It Back

The spacecraft crashed into the Pacific Ocean after a parachute failure, ending a bold experiment in space biology and memorial spaceflight.

Ancient ‘Zombie’ Fungus Trapped in Amber Shows Mind Control Began in the Age of the Dinosaurs

The zombie fungus from the age of the dinosaurs.

Your browser lets websites track you even without cookies

Most users don't even know this type of surveillance exists.

What's Seasonal Body Image Dissatisfaction and How Not to Fall into Its Trap

This season doesn’t have to be about comparison or self-criticism.

Why a 20-Minute Nap Could Be Key to Unlocking 'Eureka!' Moments Like Salvador Dalí

A 20-minute nap can boost your chances of a creative breakthrough, according to new research.

The world's oldest boomerang is even older than we thought, but it's not Australian

The story of the boomerang goes back in time even more.

Swarms of tiny robots could go up your nose, melt the mucus and clean your sinuses

The "search-and-destroy” microrobot system can chemically shred the resident bacterial biofilm.

What if Every Roadkill Had a Memorial?

Road ecology, the scientific study of how road networks impact ecosystems, presents a perfect opportunity for community science projects.

Fireball Passes Over Southeastern United States

It’s a bird! It’s a plane! It’s… a bolide!