Neuroscientists at Georgetown University Medical Center have invented a novel device that translates images into sounds, enabling individuals who are blind to recognize simple faces. This study marks a significant stride in understanding and harnessing the brain’s adaptability.

The Science of Seeing with Sound

Not all visual information is processed in the same way in the human brain. While the brain processes most of the objects you see, like cars or houses, with the lateral occipital complex (LOC), human faces are processed in an entirely different region of the brain. This is called the fusiform face area (FFA). There’s in fact a whole network of brain areas involved in face perception, but the FFA is the most important.

How the FFA and its related network components develop is still poorly understood. This is where the new research comes in. Led by Dr. Josef Rauschecker, the team developed a sensory substitution device that encodes visual patterns into auditory signals. The goal was to investigate how the FFA reacts in people who were born blind and have never seen a face.

“It’s been known for some time that people who are blind can compensate for their loss of vision, to a certain extent, by using their other senses,” says Rauschecker of the Department of Neuroscience at Georgetown University.

“Our study tested the extent to which this plasticity, or compensation, between seeing and hearing exists by encoding basic visual patterns into auditory patterns with the aid of a technical device we refer to as a sensory substitution device. With the use of functional magnetic resonance imaging (fMRI), we can determine where in the brain this compensatory plasticity is taking place.”

Currently, the device enables recognition of simple, emoji-like faces through sound. Starting with basic shapes, participants gradually learn to interpret more complex images, such as a simplified drawing of a house or human face. We aim to refine this technology to recognize actual faces, which could revolutionize how visually impaired individuals interact with the world,” shares Dr. Rauschecker.

In a comprehensive experiment, six blind and ten sighted individuals underwent functional MRI scans while experiencing the image-to-sound translations. The researchers observed that while blind individuals showed brain activity primarily in the left fusiform face area, sighted participants exhibited activity in the right fusiform face area. This distinction could provide insight into refining the sensory substitution device.

But the most important discovery is that blind people can develop their fusiform face area without requiring experience with actual visual faces. Instead, the geometry of face shape can be conveyed and, indeed, processed by other senses, such as hearing.

The ultimate goal of this research is to improve the resolution of the device thereby allowing the visually impaired to receive more complex information about the faces of people they interact with or the objects they interact with.

“We would love to be able to find out whether it is possible for people who are blind to learn to recognize individuals from their pictures. This may need a lot more practice with our device, but now that we’ve pinpointed the region of the brain where the translation is taking place, we may have a better handle on how to fine-tune our processes,” Rauschecker concludes.

The findings appeared in the journal PLOS ONE.