homehome Home chatchat Notifications


Text AI can produce images -- and it's very good at it

AI is already nearing sci-fi territory.

Mihai Andrei
July 31, 2020 @ 1:26 pm

share Share

This AI was designed to work with text. Now, researchers have tweaked it to work with images, predicting pixels and filling out incomplete images.

GPT-2 is a text-generating algorithm. Trained on billions and billions of pages of words, it’s capable of absorbing the structure of the text and then writing texts of its own, starting from simple prompts. The algorithm also uses unsupervised learning, which makes it much easier for researchers to train it without taking a lot of their time. The AI system was presented in February and proved capable of writing convincing passages of English.

Now, researchers have put GPT-2 up to a different task: working with images.

The algorithm itself is not well-suited to working with images, at least not in a conventional sense. It was designed to work with one-dimensional data (strings of letters), not 2D images.

To bypass this shortcoming, researchers unfurled images into a single string of pixels, essentially treating pixels as if they were letters. After the algorithm was trained thusly, the new version of the algorithm was called iGPT.

They then fed halves of images and asked the AI to complete the picture. Here are some examples:

Image credits: OpenAI.

The results are already impressive. If you look at the lower half of the photos above, they’re all generated by the AI, pixel by pixel, and they look eerily realistic. The three birds, for instance, are shown standing on different surfaces, all of them believable. The droplets of water too show different veridic possibilities, and all in all, it’s an amazing accomplishment from iGPT.

This also hints at one of the holy grails of machine learning: generalizable algorithms. Nowadays, AIs can be very good at a single task (whether it’s chess, text, or images), but it’s still only one task. Using one algorithm for multiple tasks is an encouraging sign for generalizable approaches.

The results are even more exciting when you consider that GPT-2 is already last year’s AI. Recently, the next generation, GPT-3, was presented by researchers and it’s already putting its predecessor to shame, by generating some stunningly realistic texts.

There’s no telling what GPT-3 will be capable of, both in terms of text generation and image generation. It’s exciting — and a little bit scary — to imagine the results.

The original paper can be read here.

share Share

This Plastic Dissolves in Seawater and Leaves Behind Zero Microplastics

Japanese scientists unveil a material that dissolves in hours in contact with salt, leaving no trace behind.

Women Rate Women’s Looks Higher Than Even Men

Across cultures, both sexes find female faces more attractive—especially women.

AI-Based Method Restores Priceless Renaissance Art in Under 4 Hours Rather Than Months

A digital mask restores a 15th-century painting in just hours — not centuries.

Meet the Dragon Prince: The Closest Known Ancestor to T-Rex

This nimble dinosaur may have sparked the evolution of one of the deadliest predators on Earth.

Your Breathing Is Unique and Can Be Used to ID You Like a Fingerprint

Your breath can tell a lot more about you that you thought.

In the UK, robotic surgery will become the default for small surgeries

In a decade, the country expects 90% of all keyhole surgeries to include robots.

Bioengineered tooth "grows" in the gum and fuses with existing nerves to mimic the real thing

Implants have come a long way. But we can do even better.

The Real Singularity: AI Memes Are Now Funnier, On Average, Than Human Ones

People still make the funniest memes but AI is catching up fast.

Scientists Turn Timber Into SuperWood: 50% Stronger Than Steel and 90% More Environmentally Friendly

This isn’t your average timber.

A Massive Particle Blasted Through Earth and Scientists Think It Might Be The First Detection of Dark Matter

A deep-sea telescope may have just caught dark matter in action for the first time.