Two neuroscientists demonstrated a powerful technology which starting from neural activity alone can reconstruct people’s faces.
Hongmi Lee and Brice A. Kuhl used a fMRI scanner image the brains of participants as they were presented with photos one after another from a set comprising 1,000 portraits. Meanwhile, each photo was stripped into its basic constituents, namely 300 eigenfaces. These eigenfaces act like statistical patterns that can be used as anchors to match a request. Finally, using machine learning the researchers trained a computer to associate a neural response for each.
This exact process, only in reverse, was used for the decoding or “mind-reading” part. Participants were shown test face or portraits which they never saw before. The neural responses were analyzed to generate eigenfaces, then these were stitched to form a final digital portrait. Sort of like how the mind’s eye sees a person’s face.
Though far from perfect, you can see the kind of accuracy this system has in the before and after below.
A second, maybe even more interesting experiment was made in which participants were asked to remember the face of a person. It turns out that the recalled faces could also be reconstructed, based on information glean from a memory-centered area of the brain called the angular gyrus.
The reconstructions were objectively worse than those made using neural patterns fired by the brains of people looking at a picture of the original face, but still pretty powerful. The scatter-plot charts show a correlation between the properties of the original and reconstructed faces.
There are some similarities, but overall the stitched images are still lacking as you can clearly see. Now, these reconstructions might not look impressive, but researchers are still working in rough, uncharted terrain. By analyzing more neural patterns, say from thousands of participants, and fine tweaking their reconstruction algorithms, it’s foreseeable that these reconstructions could become a lot better.
Before you imagine someone will read your mind soon, take comfort in the fact that your brain, as any other, is unique. Our experiences and genetics have shaped unique neural patterns, and a machine would require a sort of brain map for each of our brains before it can start reading it. That’s if it can ever filter the constant rattle and noise that goes inside our heads. Oh, and if you though this is creepy, wait until you see its motion picture version (made by another team).