Man paralyzed from neck down uses AI brain implants to write out text messages

Read my blips

Video A combination of brain implants and a neural network helped a 65-year-old man paralyzed from the neck down type out text messages on a computer at 90 characters per minute, faster than any other known brain-machine interface.

The patient, referred to as T5 in a research paper published [preprint] in Nature on Wednesday, is the first person to test the technology, which was developed by a team of researchers led by America's Stanford University.

Two widgets were attached to the surface of T5’s brain; the devices featured hundreds of fine electrodes that penetrated about a millimetre into the patient’s gray matter. The test subject was then asked to imagine writing out 572 sentences over the course of three days. These text passages contained all the letters of the alphabet as well as punctuation marks. T5 was asked to represent spaces in between words using the greater than symbol, >.

Signals from the electrodes were then given to a recurrent neural network as input. The model was trained to map each specific reading from T5's brain to the corresponding character as output. The brain wave patterns recorded from thinking about handwriting the letter ‘a’, for example, were distinct from the ones produced when imagining writing the letter ‘b’. Thus, the software could be trained to associate the signals for 'a' with the letter 'a', and so on, so that as the patient thought about writing each character in a sentence, the neural net would decode the train of brain signals into the desired characters.

With a data set of 31,472 characters, the machine learning algorithm was able to learn how to decode T5’s brain signals to each character he was trying to write correctly about 94 per cent of the time. The characters were then displayed so he was able to communicate.

Here's a gentle video explaining the experiment.

Youtube Video

Unfortunately, there’s no delete button in this system; T5 had to push on even if he had made a mistake, such as imagining transcribing the wrong letter or punctuation mark. The character error rate was reduced from six per cent to 3.4 per cent by implementing an auto-correct feature. It’s about as accurate as today’s state-of-the-art speech-to-text systems, the researchers claimed.

It should be noted that the character error rate for free typing, when T5 was not transcribing text given by the researchers, was higher at 8.54 per cent and reduced to 2.25 per cent when an auto-correcting language model was used.

“Together, these results suggest that, even years after paralysis, the neural representation of handwriting in the motor cortex is probably strong enough to be useful for a BCI,” the team wrote, referring to a brain-computer interface. T5 was paralyzed due to a spinal cord injury, but the part of his brain that controls movement is still intact.

John Ngai, director of the US National Institutes of Health's BRAIN Initiative, who was not directly involved in the research, called the study “an important milestone” for BCIs and machine learning algorithms. “This knowledge is providing a critical foundation for improving the lives of others with neurological injuries and disorders," he said in a statement. The NIH, a government organization, helped fund the research.

Not a fit for all

Although the study seems promising, the team admitted there are a lot of challenges to overcome before this kind of technology can be commercialized or otherwise used by many more people. First of all, it has only been demonstrated on one person so far. The team will have to, as the tech stands today, retrain their model for each individual's brain signals, and the performance may not be consistent from patient to patient.

“Why performance varies from person to person is still an unknown question,” Frank Willett, lead author of the study and a research scientist at Stanford’s Neural Prosthetics Translational Laboratory, told The Register.

“One cause is likely that the sensors sometimes record from different numbers of neurons – so sometimes when the sensor is placed into a person’s brain, it is particularly 'hot' and records a lot of neurons, while other times it does not. This is an open question in the field, and designing sensors that can always record many neurons is an important goal that others are working on.”

The academics also continuously retrained the system on T5’s brain signals to calibrate the software before they conducted experiments. Willett said that a system used in the real-world would have to work on minimal training data and that users shouldn’t have to retrain the machines every day.

“To translate the technology into a real product, it needs to be streamlined – the user should be able to use the BCI without needing to take too much time to train it," he said.

"So we need to improve the algorithms so that they can work well with only a little bit of training data. In addition, it should be smart enough to automatically track how neural activity changes over time, so that the user does not have to pause to retrain the system each day.”

To translate the technology into a real product, it needs to be streamlined

The invasive nature of the electrodes is also an issue; they have to stay implanted in a patient’s brain and will have to be made out of a material that is durable and safe. “Finally, the microelectrode device should be wireless and fully implanted,” Willett added. The software must also be able to run on a desktop computer or smartphone: it’s no good having to lug around heavy custom equipment.

“It is important to recognize that the current system is a proof of concept that a high-performance handwriting BCI is possible (in a single participant); it is not yet a complete, clinically viable system,” the paper concluded.

“More work is needed to demonstrate high performance in additional people, expand the character set (for example, capital letters), enable text editing and deletion, and maintain robustness to changes in neural activity without interrupting the user for decoder retraining. More broadly, intracortical microelectrode array technology is still maturing, and requires further demonstrations of longevity, safety and efficacy before widespread clinical adoption.” ®

Broader topics

Other stories you might like

  • Verizon: Ransomware sees biggest jump in five years
    We're only here for DBIRs

    The cybersecurity landscape continues to expand and evolve rapidly, fueled in large part by the cat-and-mouse game between miscreants trying to get into corporate IT environments and those hired by enterprises and security vendors to keep them out.

    Despite all that, Verizon's annual security breach report is again showing that there are constants in the field, including that ransomware continues to be a fast-growing threat and that the "human element" still plays a central role in most security breaches, whether it's through social engineering, bad decisions, or similar.

    According to the US carrier's 2022 Data Breach Investigations Report (DBIR) released this week [PDF], ransomware accounted for 25 percent of the observed security incidents that occurred between November 1, 2020, and October 31, 2021, and was present in 70 percent of all malware infections. Ransomware outbreaks increased 13 percent year-over-year, a larger increase than the previous five years combined.

    Continue reading
  • Slack-for-engineers Mattermost on open source and data sovereignty
    Control and access are becoming a hot button for orgs

    Interview "It's our data, it's our intellectual property. Being able to migrate it out those systems is near impossible... It was a real frustration for us."

    These were the words of communication and collaboration platform Mattermost's founder and CTO, Corey Hulen, speaking to The Register about open source, sovereignty and audio bridges.

    "Some of the history of Mattermost is exactly that problem," says Hulen of the issue of closed source software. "We were using proprietary tools – we were not a collaboration platform before, we were a games company before – [and] we were extremely frustrated because we couldn't get our intellectual property out of those systems..."

    Continue reading
  • UK government having hard time complying with its own IR35 tax rules
    This shouldn't come as much of a surprise if you've been reading the headlines at all

    Government departments are guilty of high levels of non-compliance with the UK's off-payroll tax regime, according to a report by MPs.

    Difficulties meeting the IR35 rules, which apply to many IT contractors, in central government reflect poor implementation by Her Majesty's Revenue & Customs (HMRC) and other government bodies, the Public Accounts Committee (PAC) said.

    "Central government is spending hundreds of millions of pounds to cover tax owed for individuals wrongly assessed as self-employed. Government departments and agencies owed, or expected to owe, HMRC £263 million in 2020–21 due to incorrect administration of the rules," the report said.

    Continue reading
  • Internet went offline in Pakistan as protestors marched for ousted prime minister
    Two hour outage 'consistent with an intentional disruption to service' said NetBlocks

    Internet interruption-watcher NetBlocks has reported internet outages across Pakistan on Wednesday, perhaps timed to coincide with large public protests over the ousting of Prime Minister Imran Khan.

    The watchdog organisation asserted that outages started after 5:00PM and lasted for about two hours. NetBlocks referred to them as “consistent with an intentional disruption to service.”

    Continue reading
  • Suspected phishing email crime boss cuffed in Nigeria
    Interpol, cops swoop with intel from cybersecurity bods

    Interpol and cops in Africa have arrested a Nigerian man suspected of running a multi-continent cybercrime ring that specialized in phishing emails targeting businesses.

    His alleged operation was responsible for so-called business email compromise (BEC), a mix of fraud and social engineering in which staff at targeted companies are hoodwinked into, for example, wiring funds to scammers or sending out sensitive information. This can be done by sending messages that impersonate executives or suppliers, with instructions on where to send payments or data, sometimes by breaking into an employee's work email account to do so.

    The 37-year-old's detention is part of a year-long, counter-BEC initiative code-named Operation Delilah that involved international law enforcement, and started with intelligence from cybersecurity companies Group-IB, Palo Alto Networks Unit 42, and Trend Micro.

    Continue reading

Biting the hand that feeds IT © 1998–2022