By the time your brain decides whether a face is familiar or forgettable, it’s already sorted it — often unfairly. Within just 600 milliseconds, your neurons make snap judgments. And if the face doesn’t belong to your own racial group, that judgment may be warped in subtle but significant ways.

A new pair of studies from researchers at the University of Toronto Scarborough peels back the curtain on what’s known as the Other-Race Effect (ORE) — a psychological phenomenon where people find it harder to recognize faces of races different from their own. While the effect is well documented, the new research offers the clearest look yet at the neural and perceptual mechanisms behind it, blending artificial intelligence with brainwave recordings to reconstruct how we actually “see” other-race faces in our minds.

“What we found was striking — people are so much better at seeing the facial details of people from their own race,” says Adrian Nestor, associate professor in the Department of Psychology and senior author of both studies.

Reconstructing Perception, Literally

In the first of the two studies, researchers used a kind of deep learning system known as a generative adversarial network (GAN) to visualize how East Asian and white participants mentally perceived a series of unfamiliar faces. After viewing and rating the similarity of face images, participants’ responses were used to reconstruct what they “saw” using the AI system.

The findings were quietly disconcerting. Across the board, participants generated more accurate mental images of same-race faces than other-race faces. When they visualized faces from another race, the reconstructions were not only fuzzier, they also skewed systematically.

People tended to see other-race faces as more average-looking, more expressive, and younger than they really were.

The study’s design made it clear that these weren’t just differences in familiarity or memory. The images were carefully controlled for factors like brightness, gaze, and facial expression. The distortions came not from the faces, but from the brains interpreting them.

Decoding the Racial Bias in Real Time

In the second study, the researchers took things one step further. They recorded brain activity using EEG, which captures rapid electrical signals from the scalp, as participants viewed a similar set of East Asian and white faces.

Again, they reconstructed what participants saw — not from what they said, but from the pattern of their brainwaves.

It sounds a bit like mind-reading — and it kind of is. The algorithms scientists use today can infer what the brain sees in surprising detail.

The researchers found that within the first 600 milliseconds after seeing a face, distinct patterns of neural activity emerged. These depended on whether the face was of the same race or a different one. Same-race faces triggered more differentiated brain responses, suggesting that people perceive them in finer detail.

“When it comes to other-race faces, the brain responses were less distinct, indicating that these faces are processed more generally and with less detail,” explains Moaz Shoura, a Ph.D. student in Nestor’s lab and co-author of the studies. “This suggests that our brains tend to group other-race faces together, leading to less accurate recognition and reinforcing ORE.”

To make sense of these differences, the team mapped what’s called a “face space” — a kind of psychological map where similar faces cluster together. In this neural face space, same-race faces were more spread out. Other-race faces were packed tightly, collapsing into a kind of average that blurred distinctions.

So, what can we do with this?

This supports an idea that’s been circulating in cognitive psychology since the 1990s. It suggests that our brains build up a dense, high-resolution map of familiar faces (often from our own racial group), but compress unfamiliar ones into generic, blurry templates.

The biases uncovered by this work may help explain real-world issues, from errors in eyewitness testimony to biases in hiring.

“This could explain why people often have difficulty recognizing faces from other races. The brain isn’t processing facial appearance as distinctly and accurately,” says Nestor.

This has echoes in everyday experience. Stereotypes about age and emotionality could be subtly reinforced by these distorted perceptions. “It’s important to know exactly how people experience distortions in their emotional perception,” says Nestor. “Seeing what’s going on in a person’s mind who misinterprets positive emotions as negative ones, for instance, can help with diagnosing mental health disorders and developing treatments.”

The Other-Race Effect suggests our brains is wired — through experience, exposure, and unconscious bias — to misrepresent some faces more than others. And in doing so, this phenomenon offers a scientific explanation for something many people know intuitively: sometimes, the face in front of us isn’t the one our brain is seeing.

“If we can better understand how the brain processes faces,” says Shoura, “we can develop strategies to reduce the impact bias can have when we first meet face-to-face with someone from another race.”