Hosts: Benjamin Thompson &
Shamini Bundell
May 12, 2021
Source: Nature
Photo Source: Unsplash, Amy Hirschi
The brain implant that turns thoughts into text A new neural interface lets people type with their mind.
In this episode: 00:45 A brain interface to type out thoughts Researchers have developed a brain-computer interface that is able to read brain signals from people thinking about handwriting, and translate them into on-screen text. The team hope this technology could be used to help people with paralysis to communicate quicker than before.
TRANSCRIPT Listen to the latest science news, with Benjamin Thompson and Shamini Bundell. Host: Shamini Bundell Welcome back to the Nature Podcast. This week, the implant that turns thoughts into text. Host: Benjamin Thompson And what craftspeople can teach scientists about materials. I'm Benjamin Thompson. Host: Shamini Bundell And I'm Shamini Bundell. Interviewer: Shamini Bundell Brain-computer interfaces or BCIs are, as the name suggests, interfaces between the brain and a computer. They can read electrical signals in the brain and translate those signals via an algorithm into computer actions like moving a cursor or typing. This week, researcher Frank Willett from Stanford University and his team have demonstrated a faster way to use BCIs for writing, which could allow people who've been paralysed to communicate more efficiently. They recruited a volunteer who couldn't move his hand due to paralysis and then asked him to imagine writing with a pen. The algorithm was trained on his brain patterns while he thought about writing. And after a while, he was able to type text on a computer screen with impressive speed. I called Frank and started by asking him how brain-computer interfaces work. Interviewee: Frank Willett So BCI works by recording the neural activity in some way, so here it was recorded with an implanted array of electrodes. Then there's some algorithm that finds the relevant patterns in that activity and translates it. Interviewer: Shamini Bundell And what kind of routes have people tried to develop so far in trying to translate brain signals into written text and how did your approach differ? Interviewee: Frank Willett Yeah, I guess there's a lot of different approaches. I mean, the one that was most on our minds was this point-and-click method, which we think was the prior best, where you're moving this computer cursor on a screen to each key that you want to hit, and you click on that key and it types that letter. So, this is different from that. So, instead of moving a single cursor from key to key, instead you're trying to handwrite something. You're just quickly trying to write this string of letters, and we show that on the screen. Interviewer: Shamini Bundell And how does that work? Interviewee: Frank Willett Yeah, it's, well, that's the magic, that's the secret sauce of the algorithm, and that's basically just pattern recognition. So, it's just looking at the patterns of neural activity and it's remembering. It knows what kind of pattern is associated with each letter and then when it sees that in the neural recordings it types that letter out. Interviewer: Shamini Bundell So, the neurons that are firing in my head now when I try and handwrite something, if I then subsequently become paralysed, those same neurons are still firing – the patterns are still there for you to observe. Interviewee: Frank Willett Yeah, exactly. Interviewer: Shamini Bundell It does sound incredibly complicated because it sounds like you would need to know the inner workings of the brain, this like in-depth neuroscientific understanding, but actually, you're kind of avoiding that by using pattern recognition. So, is that like a machine learning system? Interviewee: Frank Willett Yeah, exactly. It just learns based on having seen it many times before, so that's part of the calibration process where we collect data of the participant trying to write all these different letters multiple times and then the algorithm is able to then, through those many repetitions, kind of form an image of what each one looks like. Interviewer: Shamini Bundell So, you've so far tried this out on one participant, and they've basically had to do a load of training to show a computer what trying to write ‘a’ looks like and ‘b’. So, at the moment, it's personalised to their brain. Interviewee: Frank Willett Definitely in the future we want to look towards ways of making that process a lot faster. And we also hope that when we translate this to additional people, we'll be able to leverage that so that it won't take as long on those additional people, right, that there'll be some shared structure. Interviewer: Shamini Bundell And there's a video online of this actually working – so comparing your participant using your new handwriting-based BCI system compared to a previous method – and the cursor method is very impressive but it's relatively slow, so I can see the moving the cursor around to ‘y’, ‘o’, ‘u’, ‘space’, ‘m’. Whereas your participant here, it's noticeably and impressively faster. What's the comparison there? Interviewee: Frank Willett The original point-and-click typing device peaked at around 40 characters per minute, whereas this new method does 90 characters per minute. So, 90 characters per minute is about 18 words per minute, which is kind of exciting because it starts to get into the range that's maybe comparable to normal handwriting speeds or, in this case, kind of comparable to how you would type on a smartphone. Interviewer: Shamini Bundell So, the technology that you're using of implanting an electrode array into the brain and then communicating that, that's kind of the same as what's been done previously, but the novelty is then using handwriting. So, what's been the benefit of that? Interviewee: Frank Willett The reason why we think that handwriting was much more effective than the point-and-click cursor movements is because when you try to handwrite each different character, that evokes a very different pattern of neural activity for each character, which is great for BCIs because that makes things easy to distinguish in the neural activity. When you're doing like a point-and-click cursor, right, when you're going to nearby keys, that evokes very similar patterns. Interviewer: Shamini Bundell And given that at the moment it does require quite a lot of setup, is this going to be widely usable? Interviewee: Frank Willett I mean, obviously, we hope that yeah, it can one day be turned into a product that anyone who's has severe paralysis who can't speak or communicate could get something like this implanted. I think in the future, you could imagine if things really develop along really nicely maybe this could be part of a general-purpose device that lets you control a computer. So, even if you're, let's say, your spinal cord injured and you can move your head and face and still talk, well, maybe this could be part of a general device that lets you type on a computer and click things more easily. Interviewer: Shamini Bundell So, what is the next step now for you in this research? Interviewee: Frank Willett Yeah, well, with the handwriting stuff I think one big theme that emerged for future work is kind of making it much more streamlined for practical use and in particular the calibration times, making that faster, and then also, similarly, when things change across days. So, sometimes you get different patterns of neural activity on different days because the implant device maybe moves around a little bit so you record from different neurons. So, instead of having to retrain it every day, it would be great if instead you were able to seamlessly in the background kind of keep track of these changes, so basically minimising this training time and making it more streamlined. Interviewer: Shamini Bundell That was Frank Willett from Stanford University. Head over to the show notes, where you can find a link to Frank's paper and the video I described of the system in action.
Do you use such technologies How could such technologies impact your health and privacy?
If this article was helpful to you, donate to the Shidonna Raven Garden and Cook E-Magazine Today. Thank you in advance.