Credit: Cao et al.

Artificial intelligence (AI) can handle not only mundane and repetitive tasks, but can also work very well for some creative endeavors. We’ve seen AIs capable of writing poems, novels, movie scripts, and even classical music. Visual arts are a particularly interesting area to apply AIs, with researchers demonstrating algorithms that create paintings in all sorts of styles and, more recently, caricatures that look remarkably similar to what a human artist would draw.

The caricature algorithms were developed by computer scientists at Microsoft and City University of Hong Kong. The engineers made two separate AI systems that form a type of neural net called a Generative Adversarial Network (GAN). One of the algorithms tries to make a realistic version of an input (i.e. someone’s portrait), while the other compares the output to real-world examples in order to evaluate the work.

For this particular study, one of the GANs was built to analyze and exaggerate certain facial features from uploaded photos. The other GAN added pen strokes and artistic styling common among caricatures. “In this way, a difficult cross-domain translation problem is decoupled into two easier tasks,” the authors wrote, referring to their custom neural networks which they called “CariGANs”.

Schematic of CariGANs.

Interestingly, the algorithms can also work in reverse: converting caricatures into photo-like realistic renditions.

Caricatures to photos. Credit: Cao et al.

There are some limitations as to what the CariGANs ca achieve. The geometric exaggeration is more obviously observed in the face shape than other facial features. What’s more, some geometric exaggerations on ears, hairs, wrinkles and such, cannot be covered because the algorithms can only read 33 out of 63 landmarks lying on the face contour. This limitation can be solved by adding more landmarks, the researchers said.

It’s also possible to create video caricatures. The clip below shows a caricature of Donald Trump giving a speech, whose facial features have been exaggerated by the AI frame by frame.

The caricature algorithms will be officially presented at presented at SIGGRAPH Asia 2018, which will be held in Tokyo in December. Meanwhile, the work has been published in the pre-print server Arxiv.