For the first time, researchers implemented a type of AI called a deep neural network into a photonic (light-based) device. In doing so, they’ve come closer to making a machine that processes what it “sees” like humans do, very quickly and efficiently.

For instance, the photonic deep neural network can classify a single image in less than 570 picoseconds, or nearly 2 billion images per second. To put things into perspective, the frame rate for fluid footage sits between 23 and 120 frames per second.

“Direct, clock-less processing of optical data eliminates analog-to-digital conversion and the requirement for a large memory module, allowing faster and more energy-efficient neural networks for the next generations of deep learning systems,” wrote the authors from the University of Pennsylvania.

Classifying images almost instantly

Computer vision, the AI field that trains computers to capture and interpret information from images and videos, has come a mighty long way. If you’ve ever unlocked your smartphone by staring into its camera, you’ve seen this at play. Behind the scenes, a machine learning model allows the phone’s chip to classify objects and respond accordingly.

Combined with other data and different machine learning techniques, computer vision has a bright future ahead of it by enabling sophisticated processes like autonomous driving, automated manufacturing, disease diagnosis, security surveillance, and much more.

But although computers seem like they perform the tasks instantly, in reality, they’re quite slow at it — at least compared to one of the best image recognition devices in the world: the human brain.

The problem stems from the hardware required to support computer vision, including the graphics processing units (GPUS), whose computation speed is limited by their clocks’ frequency, memories, and processing units. Information is shuttled constantly back and forth between all these components, at a great cost in both time and energy.

However, neurons and the connections between them, known as synapses, can store and compute data in the same biological units, allowing the brain to process visual information lightning-fast and precisely.

In a new study, electrical engineers at the University of Pennsylvania attempted to mimic the way neurons process visual information by developing a deep neural network that uses a photonic chip to directly classify images without the need for the usual bells and whistles, such as clocks, sensors, or memory modules.

The tiny chip, consisting of nine “neurons”, measures just about 9.3 square millimeters in size. To record and process visual information, an image is projected onto a 5-by-6 array, with each pixel routed to the neurons via optical channels.

Light signals are constantly adjusted as they pass through the microchip’s layers of neurons, which is how it can read results — and it does so very fast.

“Computation-by-propagation, where the computation takes place as the wave propagates through a medium, can perform computation at the speed of light,” Firooz Aflatouni, a researcher at the University of Pennsylvania and one of the authors of the new study, told IEEE Spectrum.

To test the photonic chip, the researchers classified a set of 216 letters as either p or d, as well as another set of 432 letters as p, d, a, or t. The chip and deep neural network had an accuracy of 93.8% and 89.8%, respectively.

This is just a proof of concept, which is why the researchers used only a couple of artificial neurons. However, the setup is entirely scalable and could be used to classify videos and even 3d objects. In fact, it’s not limited to images. Audio and speech can be converted to the optical domain and classified using the same technology.

The findings appeared in the journal Nature.