- Email: [email protected]
A Dual Role for Hippocampal Replay
Neuron
Previews Egea, J., and Klein, R. (2007). Trends Cell Biol. 17, 230–238.
Gogolla, N., Galimberti, I., Deguchi, Y., and Caroni, P. (2009). Neuron 62, 510–525.
Galimberti, I., Bednarek, E., Donato, F., and Caroni, P. (2010). Neuron 65, this issue, 627– 642.
Henze, D.A., Urban, N.N., and Barrionuevo, G. (2000). Neuroscience 98, 407–427.
Gogolla, N., Galimberti, I., and Caroni, P. (2007). Curr. Opin. Neurobiol. 17, 516–524.
Klein, R. (2009). Nat. Neurosci. 12, 15–20.
Rutishauser, U. (2008). Nat. Rev. Neurosci. 9, 26–35. Seki, T., and Rutishauser, U. (1998). J. Neurosci. 18, 3757–3766. Xu, N.J., and Henkemeyer, M. (2009). Nat. Neurosci. 12, 268–276.
A Dual Role for Hippocampal Replay Dori Derdikman1,* and May-Britt Moser1,* 1Kavli Institute for Systems Neuroscience and the Centre for the Biology of Memory, Norwegian University of Science and Technology, N-7489, Trondheim, Norway *Correspondence: [email protected] (D.D.), [email protected] (M.-B.M.) DOI 10.1016/j.neuron.2010.02.022
Hippocampal place cells are active when the rat is at certain locations. The sequence of these active place cells is known to be rapidly replayed during sharp-wave-ripple events in the EEG. In this issue of Neuron, a study by Gupta et al. challenges previous notions that the replayed trajectories are solely a passive echo of past episodes. Rather, replay might also be an active process constructing a Tolmanian cognitive map of space, which makes flexible navigation possible. Specific types of memory, such as episodic memory, are believed to be encoded in the hippocampus (Squire, 1992). When the brain tries to encode such memories, several challenges are encountered. The first is that the timescale of episodes is much longer than timescales available for encoding sequences in the brain. While a short episode, such as a rat running along a meter-long linear track, can last many seconds, the induction of long-term plasticity occurs at a timescale of tens of milliseconds (Dan and Poo, 2004), suggesting that it could be convenient for the brain to compress episodes in time in order to store them. The second challenge is that every episode occurs only once. No event occurs twice in exactly the same manner. As the ancient Greek philosopher Heraclitus said: ‘‘Everything changes and nothing remains still.’’ In order to solve this problem, the brain needs to find a way to reactivate the episode. One mechanism that might be used is to extract the relevant information from an episode and replay it again and again until the memory is consolidated (Wilson and McNaughton, 1994).
Indeed, instances of memory replay have been found in the hippocampus of rats. Replay was first shown during sleep sessions following spatial experience, where it has been demonstrated that if two place cells tend to fire one after the other when the rat is running through their place fields on a linear track, the same cells will tend to fire one after the other when the rat is in a state of slowwave sleep after the task (Skaggs and McNaughton, 1996; Wilson and McNaughton, 1994), during events of hippocampal ripples and sharp waves. Sharp waves and ripples are events recorded in the hippocampal local field potential, which are a mark of irregularly occurring synchronized bursts of cellular discharge in CA1, originating most likely from CA3 (Buzsa´ki, 1989). Later, it was shown that the replay occurs also during quiet awake states of the rat (Foster and Wilson, 2006; O’Neill et al., 2006). While to our knowledge during slow-wave sleep hippocampal replay occurs only forward in time (Skaggs and McNaughton, 1996), in the awake state replay was found to occur both forward in time and backward in time (Davidson et al., 2009; Diba and
582 Neuron 65, March 11, 2010 ª2010 Elsevier Inc.
Buzsa´ki, 2007; Foster and Wilson, 2006), such that when the rat was running from A to B to C the replay event may occur forward starting from A to B to C or backward starting from C to B to A. Replay events were also shown to occur at a remote place and time from the original episode—a rat may replay an event from environment X while being in environment Y (Davidson et al., 2009; Karlsson and Frank, 2009). Recent work has demonstrated that disrupting replay events in the hippocampus can potentially impair acquisition of spatial memory tasks (Girardeau et al., 2009). All in all, converging evidence suggests that the hippocampus may use replay in order to consolidate episodic memories. In this issue of Neuron, Gupta and colleagues (Gupta et al., 2010) question the thoughts presented above, that hippocampal replay has a role just as a cellular mechanism for memory consolidation. In order to address this question, they measured instances of replay in the hippocampus by reconstructing the path of the rat based on the activity of the place cell ensemble. Each place cell fires with a certain probability when the rat is in a
Neuron
Previews Constructed-route replay
Backward replay G
F E
Opposite-side replay
H
I A B
C D
Forward replay
F E I
J K
J K L M N
Alternating task
Single-loop task
X
G
X
Figure 1. General Description of the Main Results of Gupta et al. The rat was rewarded at the red X for performing either a single-loop task (left, yellow circular arrow) or an alternating task (right, orange N-shaped arrow). Two major forms of replay were seen: forward replay (such as A / B / C / D in bubble, left) and backward replay (such as H / G / F / E in bubble, left, which is opposite to the running direction). There was a larger tendency to replay events far away from the rat (such as I / J / K / L / M / N in bubble, middle) in the single-loop task (left) than in the alternating task (right). One rat replayed trajectories it never performed (such as G / F / E / I / J / K in bubble, right).
given position in the environment. Thus, when recording from an ensemble of place cells, it is possible to calculate the probability of each individual cell in the ensemble to fire when the rat is in a given position in the environment. Once we know that, using Bayes’ theorem, it is straightforward to answer the inverse question, namely—what is the probability of the rat to be in a certain position, given that the place cell ensemble fired in a specific sequence? This way it is possible to reconstruct where the ensemble of hippocampal place cells ‘‘think’’ the rat is at a given moment, even when the rat is not physically at that given position, such as during replay events. If replay events are only replaying previously experienced episodes, Gupta et al. suggested that the following conjectures could be deduced: 1. The more a rat has experienced a track and passed through the place fields on the track, the more replay events of that track should occur in the rat’s brain. 2. Replay events should be more related to recent memories than to remote memories. 3. Replay events occur only along trajectories experienced by the rat.
According to their results, all three conjectures are wrong: 1. More experience / More replay: Tracks experienced more often by the rat did not induce more replay events. The rats were running only in one direction along a figure ‘‘N’’ maze (Figure 1); however, replay events occurred in both forward and reverse directions, although the reverse direction was hardly ever experienced by the rat. Moreover, the number of replay events on the central stem was smaller than the number of replay events on some segments of the outer loops, although the rat ran twice as much on the central stem as on the outer loops (Gupta et al., 2010). As recently reported in another study, the replay events tended to occur more often close to the reward location (Singer and Frank, 2009), demonstrating that the frequency of replay is not a passive reflection of the rat’s running path. Forward replay events tended to be more frequent immediately after the reward location, and reverse replay events tended to be more frequent before the reward
location (Gupta et al., 2010, their Figure 3). 2. More recent / More replay: Recent trajectories were not replayed more than trajectories that happened earlier. On the contrary, when the rat was in a single loop task, there was more replay of the opposite side than when the rat was performing the alternating task (Figure 1). This is strange because in the alternating task the rat was more recently on the other side. The result is in accord with previous work that demonstrated replay of one environment while the rat was physically in another environment (Karlsson and Frank, 2009). We suggest that this counterintuitive result can have several explanations. First it could be an effect of adaptation: cells that have not fired recently may have a higher propensity to join a replay event than cells that have done so. Second, this result may question the role of replay in consolidation at all: it could be that replay events are actually ‘‘erasing’’ irrelevant paths from the task, such as the paths on the other side, in the single-loop task (Mehta, 2007). 3. Replay occurs only along experienced trajectories: Gupta et al. demonstrated that replay could occur backward, in an opposite direction to the direction of the rat’s running. Assuming that the rat rarely ran backward, this implies that those backward trajectories were rarely experienced by the rat. In previous studies of backward replay (Davidson et al., 2009; Diba and Buzsa´ki, 2007; Foster and Wilson, 2006), the rat had experienced the backward trajectories as part of the training process. Gupta et al. even demonstrated, although for one rat only, a replay trajectory that the rat never physically performed and which appeared significantly more often than expected (Figure 1; Gupta et al., 2010, their Figure 6). Based on these observations, we have to rethink what is the role of replay for memory. If never-performed trajectories are expressed during replay, these data support the alternative notion that a
Neuron 65, March 11, 2010 ª2010 Elsevier Inc. 583
Neuron
Previews complete Tolmanian cognitive map of the environment is formed during replay. The findings suggest that replay events contain a mixture of information from recent and remote events, as well as sequences corresponding to trajectories that the animal has not taken. Such activation during sharp-wave-ripple events might result in flexible routes to the goal on subsequent trials. Perhaps such flexible routes involve the activation of ensembles of grid cells during replay episodes (Hafting et al., 2005), which might facilitate the possibility of hippocampal replay of routes not yet frequented by the rat. The replay mechanisms can assist the rat to find shortcut routes, similar to the nevertraversed shortcut taken by the rat in Tolman’s sunburst maze, which led Tolman to propose his famous cognitive map theory (Tolman, 1948). We suggest that such active construction of a flexible routing system is associated with the CA3 subfield of the hippocampus. This subfield is thought to be involved in the generation of sharpwave-ripple events that are associated with replay (Buzsa´ki, 1989). Brun et al. (2002) demonstrated that rats with CA3 lesions were perfectly capable of finding a platform location in an annular water maze, where they could follow one trajectory without having to find a new route, and then know to stop where they expected the platform to be. This demonstrates that these rats possessed intact
consolidation mechanisms for spatial memory. The same rats could not solve the regular open-field Morris water maze task, because that required construction of flexible routes to reach the platform location. From these observations we conclude that CA3 might be involved in the construction of the cognitive map but not in the consolidation of a specific route. If the replay mechanisms in CA3 are indeed those involved in the construction, interfering with these mechanisms may impair flexible navigation while not necessarily impairing consolidation of episodic memory. The report by Gupta et al. raises the possibility that hippocampal replay has a dual role: memory consolidation and active construction of a Tolmanian cognitive map. As noted by the authors, attempts to explain the generation of replay from a pure sequence mechanism are bound to fail, because as we discuss here they cannot explain the active-constructive component of replay, namely, the generation of new routes and replay in the reverse direction. Rather, new models that take into account the constructive nature of replay should be sought.
Buzsa´ki, G. (1989). Neuroscience 31, 551–570. Dan, Y., and Poo, M.M. (2004). Neuron 44, 23–30. Davidson, T.J., Kloosterman, F., and Wilson, M.A. (2009). Neuron 63, 497–507. Diba, K., and Buzsa´ki, G. (2007). Nat. Neurosci. 10, 1241–1242. Foster, D.J., and Wilson, M.A. (2006). Nature 440, 680–683. Girardeau, G., Benchenane, K., Wiener, S.I., Buzsa´ki, G., and Zugaro, M.B. (2009). Nat. Neurosci. 12, 1222–1223. Gupta, A.S., van der Meer, M.A.A., Touretzky, D.S., and Redish, A.D. (2010). Neuron 65, this issue, 695–705. Hafting, T., Fyhn, M., Molden, S., Moser, M.B., and Moser, E.I. (2005). Nature 436, 801–806. Karlsson, M.P., and Frank, L.M. (2009). Nat. Neurosci. 12, 913–918. Mehta, M.R. (2007). Nat. Neurosci. 10, 13–15. O’Neill, J., Senior, T., and Csicsvari, J. (2006). Neuron 49, 143–155. Singer, A.C., and Frank, L.M. (2009). Neuron 64, 910–921. Skaggs, W.E., and McNaughton, B.L. (1996). Science 271, 1870–1873. Squire, L.R. (1992). Psychol. Rev. 99, 195–231.
REFERENCES Tolman, E.C. (1948). Psychol. Rev. 55, 189–208. Brun, V.H., Otnass, M.K., Molden, S., Steffenach, H.A., Witter, M.P., Moser, M.B., and Moser, E.I. (2002). Science 296, 2243–2246.
584 Neuron 65, March 11, 2010 ª2010 Elsevier Inc.
Wilson, M.A., and McNaughton, B.L. (1994). Science 265, 676–679.
Previews Egea, J., and Klein, R. (2007). Trends Cell Biol. 17, 230–238.
Gogolla, N., Galimberti, I., Deguchi, Y., and Caroni, P. (2009). Neuron 62, 510–525.
Galimberti, I., Bednarek, E., Donato, F., and Caroni, P. (2010). Neuron 65, this issue, 627– 642.
Henze, D.A., Urban, N.N., and Barrionuevo, G. (2000). Neuroscience 98, 407–427.
Gogolla, N., Galimberti, I., and Caroni, P. (2007). Curr. Opin. Neurobiol. 17, 516–524.
Klein, R. (2009). Nat. Neurosci. 12, 15–20.
Rutishauser, U. (2008). Nat. Rev. Neurosci. 9, 26–35. Seki, T., and Rutishauser, U. (1998). J. Neurosci. 18, 3757–3766. Xu, N.J., and Henkemeyer, M. (2009). Nat. Neurosci. 12, 268–276.
A Dual Role for Hippocampal Replay Dori Derdikman1,* and May-Britt Moser1,* 1Kavli Institute for Systems Neuroscience and the Centre for the Biology of Memory, Norwegian University of Science and Technology, N-7489, Trondheim, Norway *Correspondence: [email protected] (D.D.), [email protected] (M.-B.M.) DOI 10.1016/j.neuron.2010.02.022
Hippocampal place cells are active when the rat is at certain locations. The sequence of these active place cells is known to be rapidly replayed during sharp-wave-ripple events in the EEG. In this issue of Neuron, a study by Gupta et al. challenges previous notions that the replayed trajectories are solely a passive echo of past episodes. Rather, replay might also be an active process constructing a Tolmanian cognitive map of space, which makes flexible navigation possible. Specific types of memory, such as episodic memory, are believed to be encoded in the hippocampus (Squire, 1992). When the brain tries to encode such memories, several challenges are encountered. The first is that the timescale of episodes is much longer than timescales available for encoding sequences in the brain. While a short episode, such as a rat running along a meter-long linear track, can last many seconds, the induction of long-term plasticity occurs at a timescale of tens of milliseconds (Dan and Poo, 2004), suggesting that it could be convenient for the brain to compress episodes in time in order to store them. The second challenge is that every episode occurs only once. No event occurs twice in exactly the same manner. As the ancient Greek philosopher Heraclitus said: ‘‘Everything changes and nothing remains still.’’ In order to solve this problem, the brain needs to find a way to reactivate the episode. One mechanism that might be used is to extract the relevant information from an episode and replay it again and again until the memory is consolidated (Wilson and McNaughton, 1994).
Indeed, instances of memory replay have been found in the hippocampus of rats. Replay was first shown during sleep sessions following spatial experience, where it has been demonstrated that if two place cells tend to fire one after the other when the rat is running through their place fields on a linear track, the same cells will tend to fire one after the other when the rat is in a state of slowwave sleep after the task (Skaggs and McNaughton, 1996; Wilson and McNaughton, 1994), during events of hippocampal ripples and sharp waves. Sharp waves and ripples are events recorded in the hippocampal local field potential, which are a mark of irregularly occurring synchronized bursts of cellular discharge in CA1, originating most likely from CA3 (Buzsa´ki, 1989). Later, it was shown that the replay occurs also during quiet awake states of the rat (Foster and Wilson, 2006; O’Neill et al., 2006). While to our knowledge during slow-wave sleep hippocampal replay occurs only forward in time (Skaggs and McNaughton, 1996), in the awake state replay was found to occur both forward in time and backward in time (Davidson et al., 2009; Diba and
582 Neuron 65, March 11, 2010 ª2010 Elsevier Inc.
Buzsa´ki, 2007; Foster and Wilson, 2006), such that when the rat was running from A to B to C the replay event may occur forward starting from A to B to C or backward starting from C to B to A. Replay events were also shown to occur at a remote place and time from the original episode—a rat may replay an event from environment X while being in environment Y (Davidson et al., 2009; Karlsson and Frank, 2009). Recent work has demonstrated that disrupting replay events in the hippocampus can potentially impair acquisition of spatial memory tasks (Girardeau et al., 2009). All in all, converging evidence suggests that the hippocampus may use replay in order to consolidate episodic memories. In this issue of Neuron, Gupta and colleagues (Gupta et al., 2010) question the thoughts presented above, that hippocampal replay has a role just as a cellular mechanism for memory consolidation. In order to address this question, they measured instances of replay in the hippocampus by reconstructing the path of the rat based on the activity of the place cell ensemble. Each place cell fires with a certain probability when the rat is in a
Neuron
Previews Constructed-route replay
Backward replay G
F E
Opposite-side replay
H
I A B
C D
Forward replay
F E I
J K
J K L M N
Alternating task
Single-loop task
X
G
X
Figure 1. General Description of the Main Results of Gupta et al. The rat was rewarded at the red X for performing either a single-loop task (left, yellow circular arrow) or an alternating task (right, orange N-shaped arrow). Two major forms of replay were seen: forward replay (such as A / B / C / D in bubble, left) and backward replay (such as H / G / F / E in bubble, left, which is opposite to the running direction). There was a larger tendency to replay events far away from the rat (such as I / J / K / L / M / N in bubble, middle) in the single-loop task (left) than in the alternating task (right). One rat replayed trajectories it never performed (such as G / F / E / I / J / K in bubble, right).
given position in the environment. Thus, when recording from an ensemble of place cells, it is possible to calculate the probability of each individual cell in the ensemble to fire when the rat is in a given position in the environment. Once we know that, using Bayes’ theorem, it is straightforward to answer the inverse question, namely—what is the probability of the rat to be in a certain position, given that the place cell ensemble fired in a specific sequence? This way it is possible to reconstruct where the ensemble of hippocampal place cells ‘‘think’’ the rat is at a given moment, even when the rat is not physically at that given position, such as during replay events. If replay events are only replaying previously experienced episodes, Gupta et al. suggested that the following conjectures could be deduced: 1. The more a rat has experienced a track and passed through the place fields on the track, the more replay events of that track should occur in the rat’s brain. 2. Replay events should be more related to recent memories than to remote memories. 3. Replay events occur only along trajectories experienced by the rat.
According to their results, all three conjectures are wrong: 1. More experience / More replay: Tracks experienced more often by the rat did not induce more replay events. The rats were running only in one direction along a figure ‘‘N’’ maze (Figure 1); however, replay events occurred in both forward and reverse directions, although the reverse direction was hardly ever experienced by the rat. Moreover, the number of replay events on the central stem was smaller than the number of replay events on some segments of the outer loops, although the rat ran twice as much on the central stem as on the outer loops (Gupta et al., 2010). As recently reported in another study, the replay events tended to occur more often close to the reward location (Singer and Frank, 2009), demonstrating that the frequency of replay is not a passive reflection of the rat’s running path. Forward replay events tended to be more frequent immediately after the reward location, and reverse replay events tended to be more frequent before the reward
location (Gupta et al., 2010, their Figure 3). 2. More recent / More replay: Recent trajectories were not replayed more than trajectories that happened earlier. On the contrary, when the rat was in a single loop task, there was more replay of the opposite side than when the rat was performing the alternating task (Figure 1). This is strange because in the alternating task the rat was more recently on the other side. The result is in accord with previous work that demonstrated replay of one environment while the rat was physically in another environment (Karlsson and Frank, 2009). We suggest that this counterintuitive result can have several explanations. First it could be an effect of adaptation: cells that have not fired recently may have a higher propensity to join a replay event than cells that have done so. Second, this result may question the role of replay in consolidation at all: it could be that replay events are actually ‘‘erasing’’ irrelevant paths from the task, such as the paths on the other side, in the single-loop task (Mehta, 2007). 3. Replay occurs only along experienced trajectories: Gupta et al. demonstrated that replay could occur backward, in an opposite direction to the direction of the rat’s running. Assuming that the rat rarely ran backward, this implies that those backward trajectories were rarely experienced by the rat. In previous studies of backward replay (Davidson et al., 2009; Diba and Buzsa´ki, 2007; Foster and Wilson, 2006), the rat had experienced the backward trajectories as part of the training process. Gupta et al. even demonstrated, although for one rat only, a replay trajectory that the rat never physically performed and which appeared significantly more often than expected (Figure 1; Gupta et al., 2010, their Figure 6). Based on these observations, we have to rethink what is the role of replay for memory. If never-performed trajectories are expressed during replay, these data support the alternative notion that a
Neuron 65, March 11, 2010 ª2010 Elsevier Inc. 583
Neuron
Previews complete Tolmanian cognitive map of the environment is formed during replay. The findings suggest that replay events contain a mixture of information from recent and remote events, as well as sequences corresponding to trajectories that the animal has not taken. Such activation during sharp-wave-ripple events might result in flexible routes to the goal on subsequent trials. Perhaps such flexible routes involve the activation of ensembles of grid cells during replay episodes (Hafting et al., 2005), which might facilitate the possibility of hippocampal replay of routes not yet frequented by the rat. The replay mechanisms can assist the rat to find shortcut routes, similar to the nevertraversed shortcut taken by the rat in Tolman’s sunburst maze, which led Tolman to propose his famous cognitive map theory (Tolman, 1948). We suggest that such active construction of a flexible routing system is associated with the CA3 subfield of the hippocampus. This subfield is thought to be involved in the generation of sharpwave-ripple events that are associated with replay (Buzsa´ki, 1989). Brun et al. (2002) demonstrated that rats with CA3 lesions were perfectly capable of finding a platform location in an annular water maze, where they could follow one trajectory without having to find a new route, and then know to stop where they expected the platform to be. This demonstrates that these rats possessed intact
consolidation mechanisms for spatial memory. The same rats could not solve the regular open-field Morris water maze task, because that required construction of flexible routes to reach the platform location. From these observations we conclude that CA3 might be involved in the construction of the cognitive map but not in the consolidation of a specific route. If the replay mechanisms in CA3 are indeed those involved in the construction, interfering with these mechanisms may impair flexible navigation while not necessarily impairing consolidation of episodic memory. The report by Gupta et al. raises the possibility that hippocampal replay has a dual role: memory consolidation and active construction of a Tolmanian cognitive map. As noted by the authors, attempts to explain the generation of replay from a pure sequence mechanism are bound to fail, because as we discuss here they cannot explain the active-constructive component of replay, namely, the generation of new routes and replay in the reverse direction. Rather, new models that take into account the constructive nature of replay should be sought.
Buzsa´ki, G. (1989). Neuroscience 31, 551–570. Dan, Y., and Poo, M.M. (2004). Neuron 44, 23–30. Davidson, T.J., Kloosterman, F., and Wilson, M.A. (2009). Neuron 63, 497–507. Diba, K., and Buzsa´ki, G. (2007). Nat. Neurosci. 10, 1241–1242. Foster, D.J., and Wilson, M.A. (2006). Nature 440, 680–683. Girardeau, G., Benchenane, K., Wiener, S.I., Buzsa´ki, G., and Zugaro, M.B. (2009). Nat. Neurosci. 12, 1222–1223. Gupta, A.S., van der Meer, M.A.A., Touretzky, D.S., and Redish, A.D. (2010). Neuron 65, this issue, 695–705. Hafting, T., Fyhn, M., Molden, S., Moser, M.B., and Moser, E.I. (2005). Nature 436, 801–806. Karlsson, M.P., and Frank, L.M. (2009). Nat. Neurosci. 12, 913–918. Mehta, M.R. (2007). Nat. Neurosci. 10, 13–15. O’Neill, J., Senior, T., and Csicsvari, J. (2006). Neuron 49, 143–155. Singer, A.C., and Frank, L.M. (2009). Neuron 64, 910–921. Skaggs, W.E., and McNaughton, B.L. (1996). Science 271, 1870–1873. Squire, L.R. (1992). Psychol. Rev. 99, 195–231.
REFERENCES Tolman, E.C. (1948). Psychol. Rev. 55, 189–208. Brun, V.H., Otnass, M.K., Molden, S., Steffenach, H.A., Witter, M.P., Moser, M.B., and Moser, E.I. (2002). Science 296, 2243–2246.
584 Neuron 65, March 11, 2010 ª2010 Elsevier Inc.
Wilson, M.A., and McNaughton, B.L. (1994). Science 265, 676–679.