Imagine controlling a virtual quadcopter with just your thoughts and finger movements. For patient T5, this is no longer a dream. Researchers have developed a high-performance brain-computer interface (BCI) that enables individuals with paralysis, like patient T5, to control virtual environments with unprecedented ability.

The participant, a 69-year-old man, saw controlling the quadcopter as a metaphor for “rising up” from his physical limitations.

A new world for people suffering from paralysis

More than five million Americans live with severe motor impairments, many yearning for activities that foster social connection, leisure, and competition. Even something that we take for granted, like video games, could make a world of a difference for many patients. However, it’s not so easy to get people suffering from paralysis to play video games. Current assistive technologies often fail to provide the intricate control needed for gaming or complex virtual interactions.

T5 is part of the BrainGate2 Neural Interface System clinical trial, which launched in 2009. The goal of the trial is to develop brain-computer interfaces to restore movement and communication for people with paralysis.

The first step was developing an interface to help paralyzed people control a computer cursor by decoding electrical activity in the brain. This isn’t just for games — it can enable people to use the internet, write emails, and communicate with the world. Suddenly, with this ability, computers can become a part of their lives again.

But cursors only get you so far.

To do things like pilot drones or play some games, you need to operate in more than two dimensions.

Unlike prior systems that focused on simpler two-dimensional controls, this interface allows three groups of fingers to move independently, with the thumb capable of two-dimensional motion. The result? A total of four degrees of freedom, doubling the functional capabilities of earlier models.

How it works

The study employed a cutting-edge BCI implanted in the brain’s motor cortex. Using 96-channel microelectrode arrays, the device records neural activity associated with finger movements. This neural data is decoded using advanced machine learning algorithms, which translate it into commands for controlling virtual fingers.

Current approaches often take a general view of the entire hand. This new study separated the fingers into three groups: thumb (1), pointer and middle finger (2), and ring finger and pinky (3). It took a bit of training, but in the end, T5 could move each finger group independently with remarkable finesse. He would signal intentions to move faster, curl, or change direction in three dimensions.

The test was then to map finger movements to a virtual drone. The participant navigated through rings in obstacle courses with remarkable ability and speed.

The patient described the experience as freeing and empowering, like playing a musical instrument or riding a bicycle.

The implications are far-reaching

The implications of this technology extend far beyond gaming. Fine motor control via BCIs could revolutionize robotic prosthetics and virtual environments. Applications range from typing on keyboards and operating digital tools to remote work and even artistic expression.

“Implantable brain–computer interfaces (iBCIs) are devices that translate intracranially recorded brain signals to instructions for computer software or electronic devices. They are regarded as promising solutions for people with motor impairments, offering a means to achieve tasks in daily life and maintain autonomy,” write Nick Ramsey and Mariska Vansteensel at the University Medical Center Utrecht, in an accompanying article. The two were not involved in the study but were commenting on the research.

Researchers also noted the potential for social inclusion. Video gaming, for instance, is a globally popular activity that fosters community and competition. By enabling people with paralysis to compete on an equal playing field, this technology addresses their unmet needs for leisure and social engagement.

Linking brains to computers has gone from science fiction to a complex reality in just 20 years. However, despite this encouraging progress and the fact that the patient greatly enjoyed the process, the technology isn’t without hurdles.

“Such developments are likely to encourage the development of materially and functionally stable devices. Whether the recreational features developed here will find their way as an add-on to medical use of iBCIs, however, will depend on robust demonstration of benefit to patients of iBCIs focused on more functional tasks, such as restoring communication by speech or restoring mobility,” conclude Ramsey and Vansteensel.

The study has been published in the journal Nature Medicine DOI: 10.1038/s41591-024-03341-8