gf2045-2013-proceedings-berger
Lincoln Center, New York City, June 2013
Global Future 2045: Towards a New Strategy for Human Evolution, Congress Proceedings
Engineering Memories: A Cognitive Neural Prosthesis for Restoring and Enhancing Memory Function
Dr. Theodore Berger
David Packard Prof. of Engineering and Prof. of Biomedical Engineering and Neuroscience, and director, Center for Neural Engineering (CNE) at the University of Southern California (USC)
Abstract: Dr. Berger leads a multi-disciplinary collaboration with Drs. Marmarelis, Song, Granacki, Heck, and Liu at the University of Southern California, Dr. Cheung at City University of Hong Kong, Drs. Hampson and Deadwyler at Wake Forest University, and Dr. Gerhardt at the University of Kentucky, that is developing a microchip-based neural prosthesis for the hippocampus, a region of the brain responsible for long-term memory. Damage to the hippocampus is frequently associated with epilepsy, stroke, and dementia (Alzheimer’s Disease), and is considered to underlie the memory deficits characteristic of these neurological conditions. The essential goals of Dr. Berger’s multi-laboratory effort include: (1) experimental study of neuron and neural network function during memory formation -- how does the hippocampus encode information?, (2) formulation of biologically realistic models of neural system dynamics -- can that encoding process be described mathematically to realize a predictive model of how the hippocampus responds to any event?, (3) microchip implementation of neural system models -- can the mathematical model be realized as a set of electronic circuits to achieve parallel processing, rapid computational speed, and miniaturization?, and (4) creation of conformal neuron-electrode interfaces -- can cytoarchitectonic-appropriate multi-electrode arrays be created to optimize bi-directional communication with the brain?
By integrating solutions to these component problems, the team is realizing a biomimetic model of hippocampal nonlinear dynamics that can perform the same function as part of the hippocampus. Through bi-directional communication with other neural tissue that normally provides the inputs and outputs to/from a damaged hippocampal area, the biomimetic model can serve as a neural prosthesis. A proof-of-concept is presented using rats that have been chronically implanted with stimulation/recording micro-electrodes throughout multiple regions of the CA3 and CA1 hippocampus, and that have been trained using a delayed, non-match-to-sample task. Normal hippocampal functioning is required for successful delayed non-match-to-sample memory.
Memory-behavioral function of the hippocampus is blocked pharmacologically, and then in the presence of that blockade, hippocampal memory/behavioral function is restored by a multi-input, multi-output model of hippocampal nonlinear dynamics that interacts bi-directionally with the in vivo hippocampus. The model is used to predict output of the CA1 hippocampus in the form of spatio-temporal patterns of neural activity – hippocampal memory codes; electrical stimulation of CA1 cells is used to “drive” the output of hippocampus to the desired (predicted) state. Using the same procedures in implanted animals with an intact, normally functioning hippocampus substantially enhances memory strength and thus, learned behavior is improved. An analogous system has been demonstrated to interact with prefrontal cortical memory function in monkeys when transmitting between layers 2/3 and 5 of that brain structure. These results show for the first time that it is possible to create “hybrid electronic-biological” systems that mimic physiological properties, and thus, may be used as neural prostheses to restore damaged brain regions – even those regions that underlie cognitive function.
Transcript
I want to tell you about a project that, as you just heard, involves many people. In fact this project is like a lot of new science projects. The days of the one researcher working til midnight in their lab with their one graduate student? Those days are over. Science now – in particular, science for these kinds of projects – involves many people with many different disciplines operating in many different places. And so the real opening slide,
of course – moi! But the real slide looks like this. These are the senior researchers. These are not the people who actually do the work. These are people who think about what should be done and hand it out to other people to do. So these are the senior researchers. Each one of these people has a lab of anywhere from 10 to 20 people. This is a very different kind of doing science and this is the sort of project that I'll be telling you about. This project is to develop a neural prosthesis for cognition, first for memory, but later for other types of cognitive functions besides memory, or in addition to memory. The goal here, I mean the problem – the goal is to solve this problem, which is: how to develop biomimetic systems for cognition.
And there are other types of prostheses that I know that you've heard about: cochlear implants for the ear, artificial retinas for the eye, and stimulators into the muscles of the arms or the legs, and as you saw yesterday, artificial arms, artificial hands. These projects have their own difficulties, but there is a unique difficulty for trying to develop neural prostheses for cognition. The difference is the following: in these kinds of prostheses, the prostheses for sensory systems and for motor systems, we're trying to interface the nervous system, the brain, to things in the outside world. But for cognition we're not trying to do that. What we're trying to do is, when we have a suddenly have a hole in the brain – when there's been a stroke, when there's been a disease of some kind – and we need to replace circuitry in the brain, we have to record from the still-working parts of the brain; look at that data; look at those signals, the electrical signals; decide what to do about them; transform them; operate on them the same way that that part of the brain, the missing part of the brain, used to do; and then, after we've performed the operation that we want, put it back into the brain.
So it's a very different set of problems. It involves having to replace the connections and information transmission between brain areas, not between the brain and the external world. We're attempting to do this, develop one of these prosthetic systems, for a part of the brain called the hippocampus. It's a part of the brain that you've been exercising over and over again during the last two days. It's the part of the brain that you use for very rapidly forming new memories about new things. That's what the hippocampus is for. It basically takes, in very simple language and very accurate language, it takes short-term memories and changes them into long-term memories. So whatever the electrical activity is that is in the parts of the brain that are representing short-term memories, these are the inputs to the hippocampus, and these signals pass through the several layers of the hippocampus and come out the other side in a form. They're still electrical signals, but the patterns have been changed, so they're in a form that the rest of the brain can accept as long-term memories. We don't know why that's true, but we know that it is true. Later on we'll figure out why, but for right now we just know that it is true, and what we're trying to do is to develop the same kinds of transformations, except we want to do those transformations using software, using hardware, and we want to perform those outside the brain. We're trying to understand what types of transformations take place inside the hippocampal circuitry and mimic those transformations.
There are a variety of reasons why the internal circuitry of the hippocampus becomes damaged. It can become damaged because of stroke; that disrupts one part of the hippocampus. Aging disrupts basically all of the hippocampus. Blunt head trauma actually selectively damages neurons in a different part of the hippocampus; epilepsy, a different part. What's common to all of these rather large-scale diseases is that they influence the hippocampus and cause memory problems. Why it is I can't remember where my keys are is another thing. I really don't know why, but many times I've been caught (my wife will tell you this is truth) shaking my fist going “where are my keys?!” and the jingle lets me know that they're right there. But that's another problem, I'm sure.
But anyway, so we want to understand what these transformations are all about, and embed those transformations into a set of microchips, or a single microchip if we can, and then, using multi-site electrode arrays, record from the part of the hippocampus that's still functioning and transmit that up to the microchip (which can exist at the top of the head, it doesn't have to be inside the brain). The microchip then performs the transformations that the circuitry in the hippocampus used to perform, and then transmits that back down, past the damage, to the still-functioning rest of the hippocampus. So we bypass the damaged areas, and we bypass them with a set of software or a set of hardware that performs this transformation, which I'll explain in a little bit. All right? That's the basic idea.
If we take a cross-section through the hippocampus and look at it, what you see is something like on the following slide. Don't worry about any of these details. The only point is that there's a circuit, and the circuit has several layers. This is the first component of the circuit. Here's the second one, here's the third one and the fourth one. So there are four or five of these layers, and at each one of these layers what happens is that short-term memory becomes slightly more like long-term memory. So the electrical signals that are here become slightly different after coming out of the first layer, slightly different still after the second layer; and it is these kinds of transformations that we have to first understand, then we have to mathematically model, and then we have to embed into some kind of a VLSI device. Our goal, then, is to figure this out.
So how do we how we go about doing that? The issue, then, is to try to understand first what are the basic rules for how any part of the brain talks to another part of the brain? Because we want to understand what happens at this connection, and then this connection, and then this connection, so at each one of those points, the first part of the hippocampus is talking to the next part, talking to the next part, talking to the next part. So what's involved in all of that? What's involved in that is shown here: when any single neuron sends information to another neuron, it does that using electrical signals that are just like pulses. They're all-or-none events. So it's a pulse and then a series of pulses. The electrical signal, that is the membrane potential across the the membrane of a neuron, varies all over the place. It can be very different, but it only influences that neuron. But as soon as membrane potential reaches a point where the cell basically says “okay, I want to send this information on,” all of a sudden the membrane potential reaches this very high level and that all-or-none event gets sent down the axon of the neuron and gets projected from these cell bodies to these cell bodies, and from there to the next layer. These signals, these action potentials, are all the same amplitude. One doesn't look any different than the next one. Which means that the amplitude of the signal can't carry any information, because it never changes. So the only thing that can carry information is the time between these action potentials. That's the only thing that can carry information.
So what does that mean? That means that cells send information to other cells using temporal patterns. Again, your brain is just singing all the time. The problem is when it doesn't sing the same song, and different parts of your brain are singing different songs, then you're in trouble. But nonetheless it's always using temporal patterns to pass information from one neuron to another. And again, any one neuron doesn't really perform very much of a function. Populations of cells represent what I look like or what you look like. Not a single neuron, but populations. So cells are really coding information in terms of space-time patterns: space – different places – and time – the temporal code. This is the basic principle that we have to to understand if we want to go and look at these transformations: single neurons transmit information in terms of temporal patterns; populations of cells, space-time patterns. I'll show you things that look like this, and when I'm showing you these kind of colored, great-looking T-shirt graphs, this will be space-time patterns. This is kind of depressing when you realize that my daughter loved this slide when she was four and I'm still showing it. She was bored by the time she was six, but I'm still using it.
But this is how we test for memory in a rat. What we do with a rat is we train it to face a wall, and there are two levers in the wall, and one of the two levers comes out. The animal knows that if it's going to get its drink of water, what it has to do is to remember, “well, it was either on the left or it was on the right,” and then we train the animal to go to the opposite side of the wall, and it pokes its nose in this place that has a light above it. It does that until the time is up, and then it has to go back and it faces the first wall again. Both of the two levers come out and the rat has to press the opposite one of the one that it pressed before. So it has to show us that it remembered what we showed it first, and if it does that, it gets its drink of water, which you know the rat likes because it wags his tail, which only dogs really do. I don't know why we have rats wagging tails, but what the hell. So, anyway, so this is how we test for memory. There's one of these trials approximately every minute. So we're making the animal form a new memory every minute after every minute. And while while the animal's doing this we have electrodes that are implanted throughout the hippocampus and in both hemispheres.
So we're sampling over a wide part of the hippocampus, and we're recording from cells in two layers. One is the output layer of the hippocampus, that's this one; and then we're recording from the layer before that, so we get to look at one of these transformations. Not all of them at same time, but one. And the basic point is – there are many basic points, but one is that if you look at what any single cell is doing, what you find is that some cells respond to just position, so there's a memory cell for position. Are things on the left or are they on the right? And these things form gradually over time, these memories. And there are other kinds of cells that only care, “is this the sample (the first part of trial), or is it the behavior (the nonmatching part of the trial)?” and there are hierarchies of these, so there are conjunctive cells, and there are the most important cells which are these trial-type cells. These are the cells that you need to actually solve the problem.
So the the hippocampus begins to form memories for different levels of this problem, and eventually there are cells that respond to the sample, when you first present the animal model with a left or a right; and then also there are cells that respond to the opposite position during this nonmatch phase. But you can see right here how unimportant level of activity is. All of these cells at these times are active. So YOU can tell that this cell is different from that cell because this cell has a conjunctive pattern whereas this cell has a position pattern, but it's the hippocampus that needs to tell. You're not important – well, Ted, never insult the audience [audience laughs]. I didn't really mean that. But what what we need to do is to find out, how does the hippocampus know this? The next cell down can't tell by looking at “is the activity coming to me high or is it low?” because all of these cells have high activity. So the point is that the only way you can tell is if you look at the very small details of what these cells are performing when they're high or when they're low, the temporal pattern can tell you. So the issue for us is can we develop methods that will allow us to say “this temporal pattern is different than that temporal pattern is different from this temporal pattern” - can we do this? And can we do it fast enough so that we can tell what's going on when the animal's learning?
This [slide filled with equations] is what you're going to have to remember to get out of the room for lunch. So start memorizing quickly [audience laughs]. I'm going to start going through this because I forgot to take this slide out. I can't possibly take the time to go through all this, but the point of these equations is, this is one of the modeling methods that we're using, and what these equations do is keep track of “is it a single pulse and if it is, did it occur here, here, here, or here? Is it a pair of pulses and if so that they occur like this or like that or like this? Is it a tripletive pulse?” etc. etc. and so this is the way that we enhance this kind of modeling methodology to a point where we can tell very accurately what the space-time pattern is. Again, I won't go through all this because I want to get to what's really important, but the point is that we've worked on this methodology for how to identify the temporal pattern, and it's fantastic work that I didn't do, my students did this, they're really smart folks.
So here's the end result. This is just one small example of a pattern that exists, the space-time pattern that exists between these two populations of neurons: one, the output layer of the hippocampus, and the other the next output layer. So these are eight neurons and this is time going this way, and this actually has several trials that are represented in this strip of time. The level of activity is color-coded, so if the color is red or dark red, that means there were lots of action potentials, and if it's blue then there were none. So you can see that across space, across neurons, and across time, there is a very consistent pattern of activity. Some cells are very active. Some aren't active at all. Some cells are changing up and down from being lowly active to highly active. This is what we observe, and the question is what does our model predict? We want a model that predicts while we're looking at this. Again what we're trying to do is replace one of the layers of the hippocampus. Imagine that it's been damaged and we are left with this space-time pattern. This is what used to exist. It's been damaged and taken out, and we need to build a system that allows us to predict what's going to happen at this output layer of the hippocampus. How well can we predict it? Our model is very good at being able to predict what's happening here, just like what happened in the real biological system. This is where you're supposed to go “ooh, aah, ooh, ahh” [audience goes “ooh, aah, ooh, ahh”]. Thank you, thank you, that's good. That's a good audience. I'm going to take you with me everywhere I go.
So that's the prediction. Now let's look at a couple of cases. This is the left. Remember I was talking about what happens when the lever comes on the left and the animal goes up presses that left lever. Or it can come up on the right and the animal goes up and presses that right lever. So this is what the space-time codes look like, so you can see very clearly that this space-time code is very different than this space-time code. So, clearly, if you're the cell listening to this one you can clearly tell what's happening, and so this code, this memory code, is developing gradually as we train the animal and then it occurs very rapidly. It's the same on every trial, and so whatever part of the brain is listening to this can hear and can discriminate on the basis of the space-time pattern, what's happening and what the memory is. And likewise here. So there are four or five different memories the animal needs to solve this problem, and you can see that these different memories are clearly different from one another as long as we look at the space-time code. If we look at just level of activity, it would all look the same.
This is – I won't say too much about this, but I'll just say that I told you before about how important it was, these trial type cells are the big ones, because they respond to the left and then they respond to the right. And these trial type cells develop slowly over time, so you start out with a task like this, you don't have very many of these trial type cells. It's because the hierarchy has not yet been built. The low-level memories haven't been formed yet. As the low-level memories become formed, then you get conjunctive versions of this and then trial type cells. So it's really interesting to watch. You have a totally different hippocampus, and imagine that as you go to different problems – I mean, you don't sit every day and perform a hundred trials in a row like we ask these rats to do. You're doing very different things all the time, and when you move from one problem to another problem your hippocampus is wiped out and the new memories are now ready, and you're able now to move on and solve a different problem. It's a very dynamic process. It's really cool.
So like I said there are about four or five memories that need to be solved, at least four or five, to solve this problem, and what we did at first was to develop models for each one of those memories. And that's going to be a lousy prosthesis. What are you going to do when someone – how do I know that the patient wants to read in the memory, the space-time code for washing their hands, or for using a particular perfume? I don't know what they want to do. And I can't sit and wait and guess. I'll be reading in memories for all kinds of the wrong things. So what we really need is a model that has many different memories in it, not just one, and with only 20 single memories I'd have to guess which one to use. So we developed another model, and I'll just make it quick, we developed a model that had all four of the memories that were necessary to solve this problem. We just asked one model to learn them all, and it did quite easily, and that means that we don't have to solve any kind of problem for when do we read it in. It's just whenever there is an input that says “the next thing I want to do is use that perfume” then the memory for the right perfume is brought into the hippocampus and then used. I know I didn't explain that really well, I'm sorry. You can't win them all, right? But it's a very important finding.
So, how does this become a prosthesis? What we did was to record, not from the output layer, but the next-to-the-output layer, and we passed this through our model. It made a prediction about what the output layer should look like, what the space-time pattern should look like, and then we stored that pattern for a given animal at a given trial type, we stored that in a computer. And we did many different versions of these trials and let the animal give us many different examples of what “left sample” looked like. And then we applied a drug to the hippocampus, which temporarily knocked it out so that after the drug wears off the hippocampus can come back and function normally, but at least for a few hours the hippocampus doesn't function at all. And when this is happening, the only memory the animal has is short-term memory, so the animal can remember something – “it came on my left, okay” – they can remember that for ten seconds, five or ten seconds. That's as much of a short-term memory as a rat has. Yours is longer, but it's just a poor rat. So clearly the ability to form new long-term memories was knocked out and, well, there it was.
So what we did was to wait until either the left or the right event was presented to the animal, and then, because we knew the animal couldn't generate its own long-term memory, we electrically stimulated, through the same electrodes that we had been recording through, and we injected, basically, the memory in electrical terms. So there was no space-time pattern so we gave it a space-time pattern. And we stimulated the output of the hippocampus, so wherever the hippocampus normally sent this memory it went to wherever those places were. And when we did that, the animal then behaved as if it had generated the long-term memory. So even though it hadn't, we gave it a long-term memory and the animal behaved as if it had generated one.
So this is a forgetting curve for an animal, a normal animal. These are the delay times, these are five-second intervals, and as the intervals get longer and longer the animal begins to forget. It would reach chance level at about 45 seconds. That's about as long as the rat can remember this kind of event. When we block the hippocampus, this whole curve drops. And again, it stays above chance levels only out to about 10 seconds. By the time it's 11 or 12 seconds the animal's at chance levels. It really doesn't remember anymore which position we showed the animal the lever. When we gave it the code, we electrically stimulated with the codes that we had stored, now the curve comes back up again.
So it's it's not quite at control levels, but it's almost at control levels, so we've reinstated long-term memory in these rats even though they can't generate the long-term memory themselves. This is really terrific, but, again, thinking about it from the point of view of being a prosthesis, you really don't want somebody looking over your shoulder and deciding which memory they think you should have at this moment. I mean, that has a creep factor of about, on a scale from 1-10 it's about a 12. So you really want to have the patient generating their own long-term memories just like they used to.
So without going into the details about how we did this, we basically gave less of this drug, which meant that we didn't knock out both of these two layers we were looking at. We only knocked out the last layer. That meant that this next-to-last layer, this one here, that was still functioning. So the short-term memories were still being formed at the input of the hippocampus, they went through the first couple of layers, and then they stop. But still they made it through the first few layers, so we recorded from that next-to-last layer, fed the space-time pattern that we recorded there to our model, the model predicted what the last layer should look like, and then we took that prediction and we electrically stimulated the last layer to generate that last memory. And it worked. So now we no longer had to be there. Now we didn't have to watch what the rat was doing. Now the rat could, on its own, decide – in fact, it could have decided not to form a memory all if it didn't want to. So if the bar was was on the left or the bar was on the right, it didn't make any difference. The rat formed its own long-term memory and then continued on, or as we said, when it was satiated – when it had enough water for the day – it just stopped responding, stopped forming memories all. That's the kind of thing you want for a prosthesis.
So we we've put together many of the really foundational issues of what you would need for a memory prosthesis. We've discovered a lot of things about the space-time coding. We know how to put together a model that's very accurate in terms of predicting what the output layer of the hippocampus should look like. We can have models that have many different memories in them, not just one, and we can let the subject control whether or not the memories are formed. And so this is a number of different things that are quite essential. It also turns out I'm out of time, so I'm going to have to just give you sound bites for these last few. I had planned that for the the next couple of issues, but not this one, but I'll do it for this one.
So. If you take a control animal and everything's fine in their hippocampus, you use the model and the electrical stimulation so that the animal is generating a new long-term memory, and you add to that long-term memory with electrical stimulation, you can enhance memory. So we can not just replace memory, but we can enhance memory if we add our external, our exogenous memory coding to the animal's endogenous memory coding. That's what all this says. And of course we would like to move beyond. This will just show you that it is possible to make VLSI devices that can do the same thing as the software. We're about in the third or fourth generation of VLSI devices. We're not yet at the point where we can mount this on the animal's head but we're close. By next year we'll have that capability.
We'd like to move up from rats (and this is where I had planned just to give you sound bites), so we we've moved to macaque monkeys, and so here we use visual cues. It's very much like the other task. We present one cue to the animal, then we take it away, there's a delay period, then we show the animal several pictures and he has to pick the picture that he saw before. So it's the same kind of memory task, different technologies, but it works. It works very well. The same kind of strategies – the same kind of modeling strategy, the same kind of implementation strategy – works for nonhuman primates as it does for rats.
We're now working on humans. We're actually going to be trying to solve two problems at once. One is control of epilepsy, which often damages the hippocampus, and so chronically epileptic patients very commonly have the same kind of of memory problems that stroke patients do, and that people that have damage to the hippocampus do, and so we're going to be helping. There is a treatment paradigm that involves using electrical simulation. When the hippocampus starts to the fire with an epileptic pattern, it fires very synchronously and a lot of cells fire at the same time and they fire a lot. One of the ways to stop that is to treat it with drugs – treat the patient with drugs – but there are many patients that are intractable to these kind of drugs. For one reason or another they don't work with these patients. One of the ways to treat these patients is in fact to use electrical stimulation. For some reason using electrical simulation sort of knocks the hippocampus out of this rythmicity. You can suppress seizures that way. That involves putting electrodes into different parts of the hippocampus and discovering what kinds of space-time patterns to use for suppression.
So the goals are very much the same for controlling epilepsy as they are for understanding how the hippocampus works in terms of forming new long-term memories. We've started already with looking at – these are actually sections of the hippocampus from a human. One of the other solutions is to take out one of hippocampi, which creates again a series of memory problems. It solves the epilepsy problem but creates memory problems. When the neurosurgeons do that, they give those pieces of the hippocampus to us and we slice it up, put it into a dish, and we can keep a human hippocampus alive for about 24-36 hours. And so we use this as a way of looking directly at the brain structure and finding out where we should put our electrodes, what kinds of patterns to use, and this kind of thing. So the first stage of these kind of treatments is to use human slices.
But this is our first human patient where we've implanted electrodes in the hippocampus. These shaded areas that you see here that are going in this direction, those are actually electrodes that are in the hippocampus. They're in the temporal lobe of the hippocampus. And we've already recorded from this patient while they were performing the same tasks that the monkeys do, and we're beginning to get our first data from humans while they're performing this memory problem, and we will soon be able to start learning what kind of coding schemes the human uses for different long-term memories, and trying to develop the same kind of of prosthesis problems.
So like the work that Ray Kurzweil was talking about, where all of a sudden there's this burst of new findings and acceleration of results, we're finding the same thing. We thought we'd be in rats until the day I retired, and now within a few years, we've jumped from rats to humans. So we hope to to bring this to the human population for their use and for the betterment of humanity, and very shortly. Thank you very much.
