Sleep: off-line memory reprocessing

Review

Tononi et al. – Complexity and coherency

47 Gilbert, C.D. and Wiesel, T.N. (1989) Columnar specificity of intrinsic

large variability in evoked cortical responses Science 273, 1868–1871

horizontal and corticocortical connections in cat visual cortex

60 Roland, P.E. (1993) Brain Activation, Wiley–Liss

J. Neurosci. 9, 2432–2442

61 Hobson, J.A., Stickgold, R. and Pace-Schott, E.F. (1998) The

48 Malach, R. et al. (1993) Relationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo

neuropsychology of REM sleep dreaming NeuroReport 9, R1–14 62 Steriade, M. (1997) Synchronized activities of coupled oscillators in the

targeted biocytin injections in primate striate cortex Proc. Natl. Acad.

cerebral cortex and thalamus at different levels of vigilance Cereb.

Sci. U. S. A. 90, 10469–10473

Cortex 7, 583–604

49 Weliky, M. and Katz, L.C. (1994) Functional mapping of horizontal connections in developing ferret visual cortex: experiments and modeling J. Neurosci. 14, 7291–7305

63 Rechtschaffen, A. (1978) The single-mindedness and isolation of dreams Sleep 1, 97–109 64 Foulkes, D. (1982) Children’s Dreams: Longitudinal Studies, Wiley

50 Schmidt, K.E. et al. (1997) The perceptual grouping criterion of

65 Llinas, R.R. and Ribary, U. (1994) Perception as an oneiric-like state

colinearity is reflected by anisotropies of connections in the primary

modulated by the senses, in Large-scale Neuronal Theories of the Brain

visual cortex Eur. J. Neurosci. 9, 1083–1089

(Koch, C. and Davis, J.L., eds), pp. 111–124, MIT Press

51 Friston, K.J. et al. (1995) Characterising the complexity of neuronal interactions Hum. Brain Mapp. 3, 302–314

66 Lowel, S. and Singer, W. (1992) Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity Science

52 Marrosu, F. et al. (1995) Microdialysis measurement of cortical and hippocampal acetylcholine release during sleep–wake cycle in freely moving cats Brain Res. 671, 329–332

255, 209–212 67 Wilson,

M.A.

and

McNaughton,

B.L.

(1994)

Reactivation

of

hippocampal ensemble memories during sleep Science 265, 676–679

53 Foulkes, D. (1985) Dreaming: A Cognitive–Psychological Analysis, Lawrence Erlbaum

68 Lumer, E.D., Edelman, G.M. and Tononi, G. (1997) Neural dynamics in a model of the thalamocortical system: II. The role of neural synchrony

54 Brivanlou, I.H., Warland, D.K. and Meister, M. (1998) Mechanisms of concerted firing among retinal ganglion cells Neuron 20, 527–539 55 Penn, A.A. et al. (1998) Competition in retinogeniculate patterning driven by spontaneous activity Science 279, 2108–2112

tested through perturbations of spike timing Cereb. Cortex 7, 228–236 69 Tononi, G., Sporns, O. and Edelman, G.M. (1996) A complexity measure for selective matching of signals by the brain Proc. Natl. Acad. Sci. U. S. A. 93, 3422–3427

56 Roffwarg, H.P., Muzio, J.N. and Dement, W.C. (1966) Ontogenic development of the human sleep–dream cycle Science 152, 614–619 57 Eggermont, J.J. (1990) The Correlative Brain : Theory and Experiment in Neural Interaction, Springer-Verlag

70 Bruner, J.S. (1973) Beyond the Information Given: Studies in the Psychology of Knowing, W.W. Norton 71 Gaetz, M., Weinberg, H., Rzempoluck, E. and Jantzen, K.J. (1998) Neural network classifications and correlation analysis of EEG and

58 Burns, B.D., Stean, J.P. and Webb, A.C. (1979) The effects of sleep on neurons in isolated cerebral cortex Proc. R. Soc. London Ser. B 206, 281–291

MEG activity accompanying spontaneous reversals of the Necker cube Cognit. Brain Res. 6, 335–346 72 Sporns, O. and Tononi, G., eds (1994) Selectionism and the Brain,

59 Arieli, A. et al. (1996) Dynamics of ongoing activity: explanation of the

Academic Press

Sleep: off-line memory reprocessing Robert Stickgold Behavioral studies of memory and learning in both humans and animals support a role for sleep in the consolidation and integration of memories. Physiological studies of hippocampal and cortical activity as well as of brainstem neuromodulatory systems demonstrate the state-dependence of communication both between and within the neocortex and hippocampus. These findings are consonant with observed cognition during sleep and immediately following awakening. R. Stickgold is at the Laboratory of Neurophysiology, Department of Psychiatry, Harvard Medical School, 74 Fenwood Road, Boston, MA 02115, USA. tel: +1 617 734 1300 ext. 316 fax: +1 617 734 7851 e-mail: rstickgold@ hms.harvard.edu

484

T

he study of the relationship between sleep and memory has had a long and complex history. Two hundred years ago, David Hartley proposed that dreaming altered the strength of associative links in memory1. But the first systematic studies of sleep and memory were only carried out in 1924, when Jenkins and Dallenbach2 used sleep studies to test Ebbinghaus’ theory of memory decay3. They demonstrated that recall was diminished less by a night of sleep than an equivalent amount of wake time, and concluded that memory loss normally resulted from sensory interference rather than passive decay; no active role of sleep was considered. 1364-6613/98/$ – see front matter © 1998 Elsevier Science. All rights reserved. Trends in Cognitive Sciences – Vol. 2, No. 12,

With the discovery of rapid eye movement sleep (REM) in 1953 (Ref. 4), the question of sleep and memory reemerged. Still, twenty years later Greenberg and Pearlman5 found no general acceptance of the hypothesis that REM was involved in information processing, despite numerous reports supporting such a role. Since then possible roles of sleep in strengthening, integrating, and even erasing memories6,7 have been continually debated. In what follows I will review evidence from (1) behavioral studies of memory and learning, (2) physiological studies of hippocampal activity and brainstem neuromodulatory systems, and (3) neural network models of information storage and PII: S1364-6613(98)01258-3

December 1998

Stickgold – Sleep and memory processing

retrieval, to explain the role of sleep in memory consolidation and integration. Behavioral studies of learning in animals Since the discovery of REM, most animal studies of sleep and learning have focused on the role of REM in memory consolidation. Much of this work has been reviewed by Smith8 and by Hennevin and her colleagues9. Based on studies using a variety of protocols and test paradigms, Smith has proposed the following general rules for REMassociated learning: (1) Training of rats on both appetitive and aversive tasks, including multiple-goal maze10, operant bar press11, shuttle avoidance12, and classical conditioning13 tasks, leads to an increase in subsequent REM, as does simple exposure to an enriched environment for as little as one day14 or as long as six weeks15. These increases appear not to be due simply to the stress of the training protocol, but to active learning of new material9. (2) The increased REM often begins only several hours after training and lasts for a limited period of time8. (3) REM deprivation during these ‘REM windows’, but not during earlier or later periods, can partially or even totally block improved task performance on subsequent retesting8,16. (4) The timing of this ‘REM window’ varies with the task8, with the intensity of training17, and with the strain of rat tested18. These studies led Smith8 to conclude that memory consolidation following task training requires processes selectively active during REM and that the organism homeostatically adjusts its REM in response to memory consolidation demands. Not all tasks show this REM dependent learning, and the characterization of which tasks are REM dependent remains incomplete. Reviewing a large number of learning studies, Pearlman19 concluded that simpler tasks were not impaired by post-training REM deprivation, while more complex tasks were, although the basis for this categorization is unclear. More recently, Hennevin and her colleagues have taken a very different approach to the question of REM and learning, looking directly at cortical activity to demonstrate that learning and plasticity can occur during REM (Ref. 9). Three sets of experiments demonstrated that during REM sleep the rat brain can (1) access and respond to learned associations formed during wake, (2) form new associations and respond to them in subsequent wake, and (3) reinforce prior learning to enhance subsequent waking behavior. In the first set of studies, Hennevin, Maho, and their colleagues looked at plastic changes in the rat brain after learning a new association. They found that following the pairing of a tone with electric shock, the tone alone induced larger responses in the medial geniculate nucleus (MGm)20, the dorsal hippocampus (dHC)21, and the lateral nucleus of the amygdala (AL)9. When responses were measured during REM, the same effects were seen; tones associated during waking with foot shocks produced larger responses in the MGm, dHC, and AL than before conditioning. These increases were comparable in magnitude to those seen in wake and, in the MGm and AL, faster in REM. They concluded that learning-dependent plastic changes in neuronal activity formed during wake were accessible during REM. In the second set of studies, they showed that new associations could be formed during REM, but not during slow-

Review

wave sleep (SWS). In one study, they presented an aversive electrical stimulation of the central gray (which mediates pain perception) immediately after a neutral stimulation of the MGm (which mediates sound perception). They then measured dHC activity during presentation of the neutral stimulus. Their results clearly showed that stimulus pairing during wake or REM produced an increased dHC response, while pairing during SWS had no effect at all22. In the last studies, the ability of REM to enhance retention and consolidation of prior learning was investigated by testing the effect of mesencephalic reticular formation (MRF) stimulation during REM on memory consolidation. Such stimulation, given immediately following task training in rats enhances subsequent performance23. The MRF is located immediately rostral of the pontine reticular formation (PRF) which controls the onset of REM, and stimulation of the MRF conceivably activates components of the REM machinery. This possibility is supported by the finding that MRF stimulation following training in rats enhanced performance23, diminished the normal REM increase following training24 and eliminated the normally deleterious effect of REM deprivation25. In this context, Hennevin26 showed that when rats were trained over six days to find food in a maze, electrical stimulation of the MRF during REM, 1.5–4.0 hours after training, enhanced subsequent performance. In contrast, MRF stimulation in SWS or wake during the same post-training time period produced no improvement. The results suggest that MRF stimulation, by augmenting the normal PRF activation during REM, enhances a normal REM-dependent memory consolidation process. Taken together, the animal studies provide strong evidence that (1) the brain is capable of carrying out all of the processes required for memory consolidation during REM but not during SWS, (2) the amount of REM is homeostatically adjusted to the demands of memory consolidation, (3) REM is required during specific post-training windows for consolidation to occur on some but not all tasks, (4) MRF stimulation immediately after training can eliminate this requirement for REM sleep, and (5) MRF stimulation during REM can enhance the consolidation. All of this points towards a critical role of sleep, and particularly of REM, in memory consolidation. The basis of the dichotomy between REM-dependent and REM-independent processes remains unclear. Behavioral studies of learning in humans Most human studies of sleep and learning also have looked at REM. This work has recently been reviewed by Smith27, and, as with the animal studies, the relationship between sleep and learning is task dependent. While some studies showed clear impairment of learning with REM deprivation, others showed no effect28. For example, REM deprivation consistently produced no effects on retention of word lists or paired associates28,29. Smith has looked at the effects of REM deprivation on specific memory systems. REM deprivation after training had no effect on declarative/explicit tasks such as a word recognition task or the Rey–Osterrieth task30, but hindered subsequent performance of implicit/procedural tasks such as a

485 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998

Review

Stickgold – Sleep and memory processing

Fig. 1 Sample screens for the visual discrimination task. Each target screen consisted of a rotated ‘T’, as in (A), or ‘L’, as in (B), at the fixation point. A horizontal (in A), or vertical (in B), array of three diagonal bars appeared in the lower left quadrant of the visual field against a background of horizontal bars. Fixation points and target arrays have been darkened to aid the reader, but were not darkened in the actual test. The exact positions of both the horizontal and diagonal bars varied slightly from screen to screen. (C) A sample mask, with a combined ‘T’ and ‘L’ at the fixation point and combined horizontal and diagonal bars at all other positions. A session consisted of up to 25 blocks of 50 trials each. Each block had a fixed interstimulus interval, and this decreased monotonically throughout the course of the session.

word fragment completion task and the tower of Hanoi task27, even when retesting was a week after REM deprivation. Moving away from the sleep deprivation paradigm, Plihal and Born compared the amount of learning on a declarative (paired associates) learning task and a procedural (mirror writing) task following periods of wake or sleep31. They found that SWS-rich sleep (i.e. the first half of the night) enhanced subsequent performance on the declarative memory task – but not the procedural task – compared with an equivalent amount of either SWS-poor sleep (i.e. the second half of the night) or wake time. In contrast, REMrich sleep (during the second half of the night) enhanced performance on the procedural but not the declarative task. Their results suggested that REM plays a larger role in the consolidation of procedural memories while SWS is more critical for declarative memory consolidation. In support of this model, Karni and his colleagues32 have shown that REM deprivation prevents improvement on a procedural (visual discrimination) memory task. Subjects were shown a target screen containing a background of horizontal dashes with either a ‘T’ (as in Fig. 1A) or an ‘L’ (as in Fig. 1B) at the fixation point, and three diagonal bars in the lower left quadrant arranged either horizontally (as in Fig. 1A) or vertically (as in Fig 1B). The target screen was displayed for 16 ms, then a blank screen was presented for a variable interval (the interstimulus interval, ISI), followed by a mask (as in Fig. 1C) for 16 ms and then a blank screen until response. Subjects indicated the orientation of the array of diagonal bars as well as the fixation letter (to guarantee fixation). The discriminatory threshold was the interpolated ISI at which a subject showed 80% accuracy on vertical–horizontal discriminations. When trained in the evening and retested the following morning, the discrimination threshold decreased an average of 24%, from 97 ms to 74 ms. In contrast, following a night of REM deprivation, subjects showed no overnight improvement32. Using this same task, Stickgold et al.33 demonstrated that overnight improvement was proportional to the amount of SWS during the first quarter of the night as well as to the amount of REM in the last quarter, but was not correlated

with either sleep parameter during other portions of the night (Fig. 2A). The temporal sequence of these dependencies suggested a two-step process of memory consolidation requiring SWS followed by REM. This hypothesis was supported by their finding that improvement was even more highly correlated with the product of the amounts of early SWS and late REM (Pearson correlation coefficient, r = 0.89; Fig. 2B). Taken together, the combined animal and human studies support the concept that SWS and REM each contribute importantly to the process of memory consolidation. The human studies support the hypothesis that SWS is more critical for the consolidation of (hippocampally dependent) declarative/episodic memories, while REM is more important for (hippocampally independent) procedural/implicit memories. At the same time, the animal studies make clear the important role of REM sleep in hippocampally dependent spatial tasks, and Stickgold33 has shown that both SWS and REM contribute to improved performance on a presumably non-hippocampal procedural memory task. Thus SWS and REM may facilitate different components of the memory consolidation process and may contribute more or less to the consolidation of a given task based on the nature of the task. Neurophysiological and neurochemical studies A clearer picture of how SWS and REM sleep stages might play distinctly different roles in memory consolidation comes from related neurophysiological and neurochemical studies: Neuronal activity in the hippocampal formation When rats actively explore a new environment, pyramidal cells within the CA1 and CA3 regions of the hippocampus form a spatial (albeit non-topographic) map of the environment, wherein individual cells – referred to as ‘place cells’ – become responsive to specific locations within the environment34. The development of these spatial maps is driven by the flow of sensory information from the neocortex into the hippocampus during active exploration. Firing of place cells during exploration occurs in phase with the peaks of theta waves35, a condition known to produce a long-term potentiation of synaptic efficacy in the hippocampus36 and believed to be associated with memory formation and consolidation. Recordings from the hippocampus and entorhinal cortex of the rat have shown that communication between the neocortex and hippocampus is highly dependent on the wake–sleep state of the animal. During active exploration, theta waves (4–8 Hz) can be recorded from the hippocampus. These waves are driven by entorhinal afferents to the hippocampal granule and pyramidal cells, and enhance the sensitivity of pyramidal cells to afferent inputs37. In conjunction with the theta waves, information flows from the neocortex into the hippocampus via the perforant pathway, through the superficial layers of the entorhinal cortex37–39. During quiet rest, the theta waves stop and EEG sharp wave potentials (SWPs) appear39. Simultaneously, perforant pathway transmission into the hippocampus ceases and is replaced by transmission out of the hippocampus and

486 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998

Stickgold – Sleep and memory processing

A

B

p < 0.01 p < 0.05

0.6

Improvement (ms)

Pearson r-value

0.8

0.4 0.2 0.0 –0.2

Review

40 30 20 10 0

1

2

3

4

50

100

150

200

SWS1 ´ REM4 (% ´ %)

Quartile

Fig. 2 VDT learning. (A) Correlation of learning with SWS and REM across the night. For each quartile of the night, the Pearson correlation coefficient between SWS% and overnight improvement (filled squares) and between REM% and overnight improvement (open circles) was calculated. Heavy dashed line: correlations significant (p < 0.05) for Pearson r-values greater than 0.57; light dashed line: correlations more significant (p < 0.01) for Pearson r-values greater than 0.71. (B) Two-step model of memory consolidation. Improvement is plotted as a function of the product of the amount of SWS during the first quarter of the night and the amount of REM in the last quarter. Both amounts are quantified as percents of the entire night33. The strong correlation (r = 0.89, p < 0.0001) suggests a two-step consolidation process, including an early, SWS-dependent process and a late REM-dependent one.

back to the cortex through the deep layers of the entorhinal cortex39. This same pattern of information flow appears again during SWS. Thus, during active exploration, sensory input flows into the neocortex and from there into the hippocampus, while during quiet rest and SWS, information flows in the reverse direction39. With the onset of REM, the pattern reverses again. Sharp wave potentials and transmission out of the hippocampus cease, and transmission into the hippocampus reappears along with hippocampal theta waves39. If hippocampal activity during waking exploration is so strongly dependent on the movement of the rat through its environment, what drives this activity during other wake–sleep stages? For example, how does pyramidal cell activity during SWS, when information is flowing back from the hippocampus to the neocortex, relate to its activity during earlier exploration? The answer appears to be that they are highly correlated. Whether looking at the activity of individual place cells, responsive to specific locations within the environment40, or ensembles of cells which fire in coordinated fashions in certain locations41, hippocampal place cells show patterns of firing during SWS that mimic those seen during prior waking exploration. Thus the hippocampus appears to be playing back to the neocortex at least a close copy of the information it received during earlier exploration. This replay is, however, short-lived, decaying with a time constant of approximately 12 minutes and disappearing by 30 minutes41. During REM sleep, the situation appears more complex. Theta waves reappear in the hippocampus and information flows from the neocortex into the hippocampus39. Although place cells fire in phase with the peaks of theta waves during active exploration, their alignment with theta during REM sleep depends on the novelty of the environment. And although place cells that map to locations in novel environments fire in phase with theta waves (as they do during active exploration), the phase shifts over time so that after a week the firing is 180° out of phase with the theta waves42. Such out-of-phase firing has been shown to

reverse the long-term potentiation produced by in-phase firing43,44, presumably leading to the loss of the hippocampal map of the familiar environment. Thus, during REM sleep, the hippocampus appears to consolidate memories of novel environments while erasing those of more familiar environments. While hippocampal ‘forgetting’ of familiar environments may seem counterintuitive, the time course of hippocampal forgetting is similar to that for the development of hippocampus-independent contextual memories45, and may reflect the erasure of hippocampal memories after they have been copied to more permanent storage systems. In summary, these results demonstrate the state-dependent nature of regional information processing in the rat. Information flows from neocortex to hippocampus during active exploration and REM, and in the opposite direction during quiet rest and SWS (Ref. 39). Sleep thus provides a period of ‘dialogue’ between the cortex and hippocampus37. Studies of hippocampal place cells reinforce this concept. These cells are initially activated via the cortex during waking exploration and the information thus stored is subsequently played back during SWS (Refs 40,41). Finally, during REM, as information again flows from neocortex to hippocampus, place cells are again activated, but now either in or out of phase with hippocampal theta waves, depending on the familiarity of the spatial locations encoded42. Such phase shifts might well switch the hippocampus from a record mode to an erase mode, eliminating hippocampal memories once adequately represented in neocortical structures. Neuromodulatory studies In addition to state-dependent changes in communication between the hippocampus and cortex, the wake–NREM– REM cycle also produces massive shifts in cortical neuromodulation (Fig. 3). Of the four major neuromodulatory systems – acetylcholine (ACh), dopamine (DA), norepinephrine (NE), and serotonin (5-HT) – all but DA are under strict regulation across the sleep cycle. ACh is at minimal levels during SWS and maximal during REM (Refs 46,47). ACh levels in the hippocampus are significantly higher

487 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998

Review

Stickgold – Sleep and memory processing

Active Wake

Quiet Wake

SWS

REM

ACh

++

+

–

+++

NE 5-HT

++

+

+

–

since cholinergic stimulation is known to increase REM (Ref. 55). While Hasselmo’s model was directed towards intracortical memory processing, the tight temporal correlation between the direction of hippocampal–cortical information flow and the levels of cholinergic neuromodulation in the brain suggests that neuromodulators may also facilitate the transfer and processing of memories between hippocampus and neocortex across the wake–sleep cycle.

Fig. 3 State-dependent neuromodulation. The relative activity of the cholinergic (ACh), noradrenergic (NE) and serotonergic (5-HT) neuromodulatory systems are shown across the wake–sleep cycle46–49. Abbreviations: REM, rapid eye movement sleep; SWS, slow-wave sleep.

during REM than during wake, while neocortical levels are similar in the two states. In contrast, NE and 5-HT levels are high during SWS and near zero during REM, and are higher in waking than during either sleep state48,49. These same neuromodulators play important roles in memory consolidation50, suggesting that their cyclic variation during sleep are functionally related to the consolidation process. The role of ACh in learning and memory has been reviewed by Hasselmo and Bower51. The antimuscarinic agents atropine and scopolamine inhibit learning in both humans52 and animals53, but have little effect on recall52. Tissue slice studies of rat somatosensory cortex show that the cholinergic agonist carbachol selectively inhibits the primarily feedback synapses in layer I, while having no effect on the primarily afferent and feedforward synaptic connections in layer IV (Ref. 54). These findings led Hasselmo and Bower51 to propose that high ACh biases the system for memory encoding, while low levels bias the system towards recall. Within such a model, REM would facilitate the formation of new cortical connections and associations, while SWS would facilitate activation of existing cortical memories based on partial inputs. Thus, neuromodulatory systems may provide a switching mechanism that shifts the cortex between two modes of memory processing, and Hasselmo and Bower have suggested that the levels of cholinergic activity may be modulated by ongoing memory processing demands of the system51. Such a mechanism might explain the increase in REM seen in ‘REM windows’8 following task training,

30

ms

20

10

0 NREM

REM

Fig. 4 Semantic priming after awakening from REM and NREM. The relative efficacy of normally weak (open circles) and strong (filled circles) primes are reversed in REM and accentuated during NREM (see Ref. 57).

Sleep and cognition If different components of memory processing occur during different sleep stages, one might expect to see this reflected in cognitive processes active during sleep. REM dreams are generally thought to be more hyperassociative and bizarre than NREM dreams56,57, while NREM mentation appears more thought-like and perseverative in nature, with less of the bizarreness that characterizes REM dreams58,59. Further evidence for this difference comes from cognitive testing performed immediately following awakenings from REM and NREM sleep. Semantic priming is considered a measure of the automatic spread of activation from a ‘node’ representing one word to nodes representing semantically related words60. It can thus be used to test for global alterations in the strengths of associative links across the REM–NREM cycle. Using this paradigm, Stickgold and his colleagues57 have shown that the relative efficacy of weak and strong primes is reversed in REM and accentuated during NREM compared to normal waking values (Fig. 4). Thus, REM enhances both the activation of weak associations and the occurrence of bizarre and hyperassociative dreams. One could speculate that the cholinergically driven shifts in neocortical activity and silencing of hippocampal outflow seen during REM may contribute to these statedependent shifts in cognition. If so, then dreams themselves may be reflections of neocortically based memory processing occurring during REM, and this same argument could explain the apparent role of REM sleep in the consolidation of procedural (cortical, non-hippocampal) learning31. Sleep and memory How do these findings map onto the human sleep cycle? Adult humans normally have a single, consolidated period of sleep lasting approximately eight hours (Box 1). Across the night, the depth of sleep follows a 90-minute ultradian cycle during which sleep first deepens into SWS and then becomes progressively lighter until REM is entered. While the periodicity of this cycle remains relatively constant across the night, the distribution of sleep stages does not. Early cycles are dominated by SWS with almost no REM, while late cycles are dominated by REM with no SWS. Cortical–hippocampal communication and cholinergic neuromodulation presumably remain phase locked to this sleep cycle, as shown in Fig. 5. This pattern suggests a process of memory consolidation involving alternating high and low levels of both cortical and hippocampal ACh coupled with reversals in the direction of information flow between the neocortex and hippocampus. Several authors have suggested a role for the reversing information flow in memory consolidation. Based on these patterns of information flow, Buzsaki37 has

488 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998

Review

Stickgold – Sleep and memory processing

Box 1. The human sleep cycle In humans and cats, sleep is normally divided into five stagesa, according to the standards of Rechtschaffen and Kalesb, and defined by three parameters: patterns of brain activity measured by electroencephalography (EEG), coordinated eye movements measured by electrooculography (EOG), and tonic muscle activity measured by electromyography (EMG) (Fig. A). Quiet waking is characterized primarily by a high frequency, low amplitude EEG with moderate muscle tone. In the presleep period, alpha waves (8–12 Hz) appear. In the sleep onset period, known as Stage I sleep, the alpha waves are generally replaced with a mixture of faster beta waves (> 12 Hz) and slower theta waves (4–8 Hz), and slow, rolling eye movements (SEMs) with periods of 3–5 s. Unambiguous sleep begins with the appearance of sporadic EEG complexes known as spindles and K-complex, which define Stage II light sleep. SEMs normally disappear by the start of Stage II. After 10–30 minutes in Stage II sleep, subjects normally descend into Stage III and IV, characterized by slow (0.5–2.0 Hz) delta waves occupying progressively more of the EEG record. The differences between these two stages are primarily quantitative, and they are often jointly referred to as delta sleep, deep sleep, or slow wave sleep (SWS). Together with Stage II, they constitute nonREM (NREM) sleep. After 15–30 minutes in SWS, subjects return to Stage II sleep and then to a variant of Stage I sleep, REM sleep, characterized by a desynchronized EEG pattern not obviously different from waking (hence the names ‘desynchronized’ sleep and ‘paradoxical’ sleep), atonia of the skeletal muscles, and bursts of rapid eye movements (hence, ‘REM’ sleep). It is also referred to as ‘ascending’ Stage I sleep, as distinct from the sleep onset ‘descending’ Stage I sleep. The initial cycle – from waking, through SWS, and then into REM – normally takes 50–70 minutes, and thereafter the cycle occurs with an impressively regular 90 min periodc (Fig. B). Despite this regularity, the distribution of stages shifts across the night, with SWS predominating early in the night along with only brief REM periods, and long REM periods (shown by the thicker lines) alternating with Stage II, but no SWS, late in the night. In rats and mice, Stages II, III and IV are normally blocked together as SWS. This can be confusing, as Stage II is not considered part of SWS in humans and cats. In addition, ‘REM’ sleep is a misnomer in the rodent, as these non-visual animals do not have clear rapid eye movements. The equivalent phase (desynchronized sleep) is often characterized by the recording of theta waves from within the hippocampus. Nevertheless, the phrase ‘REM’ sleep is still used, most often by analogy to the equivalent human sleep stage. Fig. A Sample recordings of EEG, EOG, and EMG from various sleep stages. References a Dement, W.C. and Kleitman, N. (1957) Cyclic variations in EEG during sleep and

their

relation

to

eye

movements,

body

motility,

and

dreaming

Electroencephalogr. Clin. Neurophysiol. 9, 673 b Rechtschaffen, A. and Kales, A. (1968) A Manual of Standardized Terminology, Techniques and Scoring System for Sleep Stages of Human Subjects, Brain Information Service, University of California, Los Angeles c Berger, R.J. (1969) The sleep and dream cycle, in Sleep: Physiology and Pathology:

Wake I/REM II III SWS IV 11 PM

1 AM

3 AM

5 AM

7 AM

Fig. B Sleep hypnogram of a canonical night of sleep.

a Symposium (Kales, A., ed.), pp. 17–32, Lippincott

described a ‘hippocampo–neocortical dialogue’ wherein information is transferred rapidly from the neocortex to the hippocampus at the time of acquisition and then hippocampal consolidation and replay of the stored representation back to the cortex occurs during SWS. McClelland, McNaughton and O’Reilly61 proposed a similar model based on neural network considerations. They proposed that the hippocampus provides a rapidly encoded, sparse memory storage system ideal for the formation of distinct episodic memories with affect associated through the strong connections between hippocampus and

amygdala. In contrast, the neocortex offers a slowly consolidating, dense memory storage system. Formation of independent memories within the neocortex results from frequent reactivation of the memory trace. Such repetition can be driven by reenactment of a sensorimotor pattern, as in most procedural learning paradigms, or by activation of a hippocampal representation of the memory, which would reactivate the cortical pattern. By using slow, automatic replay from the hippocampus (over days, weeks, or even years), high density, overlapping storage becomes feasible. SWS would be an obvious time for such replay to occur,

489 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998

Review

Stickgold – Sleep and memory processing

Fig. 5 The ultradian cycle and information processing. Changes in cholinergic neuromodulation and hippocampo–neocortical communication is superimposed on the human REM–NREM cycle across the night. The slow shift from SWS domination to REM domination across the night is seen in amounts of SWS and REM as well as in duration of REM periods and amplitude and frequency of rapid eye movements within REM. The cholinergic neuromodulation is presumed to follow this pattern because rapid eye movements reflect the activity of brainstem cholinergic neurons.

since information is known to flow from the hippocampus to the cortex at this time and there would be no competition from external sensory inputs. The possibility of such an extended period of replay is supported by the fact that recall of specific memories becomes independent of the hippocampus only over time. For example, hippocampal strokes often result in temporally graded retrograde amnesia that extends back as far as 15 years prior to the stroke62. In animal studies, the period of retrograde amnesia is speciesvariable, ranging from 1–14 days in rats45 to 2–12 weeks in

Outstanding questions • Animal studies suggest that sleep, and REM sleep in particular, might be an absolute requirement for maintenance of some forms of learning. To what extent is this true in humans? Are sleep-dependent processes absolutely required for memory consolidation and integration, or can these same processes occur during various waking states? • What factors determine which memory and learning tasks are REM- or SWS-dependent? Is the degree of hippocampal involvement in the learning process critical? • Animal studies suggest that ‘REM windows’ exist during which these REM-dependent processes must occur. Can these ‘windows’ of time be found in humans as well? Does memory consolidation occur in a single night or across multiple nights? • The greater ease of forming hippocampal memories makes their consolidation and integration in the neocortex difficult to measure. What experimental paradigms can make these neocortical changes more apparent? • In at least one paradigm, learning appears to be dependent on SWS early in the night. Are older subjects, who normally lack most SWS, proportionately impaired in learning this task? Could this contribute to age-dependent cognitive decline in general? • Is there sleep-dependent erasure of hippocampal memory? The development of hippocampus-independent learning normally can be measured after a week. Can it be prevented by sleep deprivation?

monkeys63, although these differences could equally well reflect task- rather than species-dependencies. A possible role for the cycles of reciprocal communication across the night was proposed by Hinton64 in his ‘wake–sleep’ algorithm for unsupervised learning. Using simulated neural networks, the algorithm activates ‘bottomup recognition connections’ during the ‘wake’ phase to produce a representation of inputs in one or more hidden layers. During the ‘sleep’ phase, ‘top-down generative connections’ attempt to reinstantiate the original signal in the input layer. By alternating activity in the two directions, the hidden layer representations are modified until they produce an optimal representation of the original signal. To instantiate this model, we suggest that during waking and REM (Hinton’s ‘wake’ state), information flows from the cortex (input layers) into the hippocampus (hidden layers), where a representation of the original sensory information is created. Then, during SWS (Hinton’s ‘sleep’ state), the information is played back from the hippocampus to the cortex. Over successive 90-minute REM cycles, the unsupervised networks tune the hippocampal representation to more accurately represent the original episodic event. Hinton’s model can equally well be applied to a purely cortical memory consolidation and integration paradigm, where the ‘wake’ and ‘sleep’ states reflect high and low levels of neocortical ACh in NREM and REM, respectively, and the two ‘layers’ reflect more, or less complex and integrated, representations of the original sensory input. But in reality, both systems of consolidation and integration probably act simultaneously. While the hippocampal–neocortical component would support the transfer and integration of hippocampally based declarative and episodic memories into semantic knowledge, the neocortical ACh component would further support the consolidation and integration of neocortically based implicit and procedural memories. The highly overlapping memory storage seen in the neocortex can serve other functions besides efficient memory storage. Specifically, such a design facilitates the formation of associative links between memories. Both the neuromodulatory (high ACh) and neurophysiological (information flow initiated in cortex) characteristics of REM suggest that this sleep state would enhance the activation of and, presumably, encoding of, new associations within the cortex, whether prompted by intracortical inputs (e.g. semantic priming) or by inputs from the hippocampus. Conclusions Considerable behavioral and neurophysiological evidence now supports critical roles for both REM and SWS in the processing of memories. But rather than simply strengthening established memories, sleep appears to serve two distinct functions: first, to support the transfer of memories between the hippocampus and neocortex, and second, to foster the integration of neocortical memories into wider associative networks. The first of these, the memory transfer function, serves hippocampally dependent declarative/episodic memories, and several researchers have suggested elements of this function. McClelland and his colleagues61 have provided a model for how and why hippocampal memories are transferred to

490 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998

Stickgold – Sleep and memory processing

the cortex. But they ignored the replay of information from cortex to hippocampus and discussed the playback from hippocampus to cortex solely in terms of strengthening existing memory traces. In contrast, Hinton64 emphasized the value of alternating the direction of information flow (Buzsáki’s ‘hippocampo–neocortical dialogue’37) in allowing alternating brain states to facilitate the fine-tuning of memory traces. While both of these models acknowledge a possible role of sleep in their models, it was Buzsáki who explicitly proposed differential roles for REM and SWS (Ref. 38). The second suggested role for sleep, that of integration into associative networks, serves both declarative/episodic and procedural/implicit memories. Here the ability of neuromodulatory systems to switch memory-processing algorithms in the brain51 provides the basis for both Hinton’s alternating brain states and our proposed REM-dependent activation of wider associative networks57. The strengthening of these wider network connections during REM could also form the basis of the consolidation of procedural memories in REM. By enlarging the network activated by a specific stimulus (via associative networks), more computing power could be brought into play and greater speed and accuracy would become possible. This fits with the fMRI data showing that visual discrimination task training33,65 enlarges the cortical areas activated by target stimuli66. It remains for future studies in both humans and animals to determine both the precise characteristics of memories that result in their REM and SWS dependence, and the detailed neurochemical and neurophysiological changes during these sleep states that underlie these memory processes.

Review

reacquisition d’un apprentisage à renforcement positive Brain Res. 70, 43–54 12 Smith, C. and Young, J. (1980) Reversal of paradoxical sleep deprivation by amygdaloid stimulation during learning Physiol. Behav. 24, 1035–1039 13 Lecas, J.C. (1976) Changes in paradoxical sleep accompanying instrumental learning in the cat Neurosci. Lett. 3, 341–355 14 Krech, D., Rosenzweig, M.R. and Bennet, E.L. (1962) Relations between brain chemistry and problem solving among rats in enriched and impoverished environments J. Comp. Physiol. Psychol. 55, 801–807 15 Mirmiran, M., van den Dungen, H. and Uylings, H.B. (1982) Sleep patterns during rearing under different environmental conditions in juvenile rats Brain Res. 233, 287–298 16 Smith, C. and Butler, S. (1982) Paradoxical sleep at selective times following training is necessary for learning Physiol. Behav. 29, 469–473 17 Smith, C., Young, J. and Young, W. (1980) Prolonged increases in paradoxical sleep during and after avoidance task acquisition Sleep 3, 67–81 18 Kitahama, K., Valatx, J-L. and Jouvet, M. (1976) Apprentissage d’un labyrinthe en y chez deux souches de souris: effets de la privation instrumentale et pharmacolgique du sommeil Brain Res. 108, 75–86 19 Pearlman, C. (1979) REM sleep and information processing: evidence from animal studies Neurosci. Biobehav. Rev. 3, 57–68 20 Hennevin, E. et al. (1993) Learning-induced plasticity in the medial geniculate nucleus is expressed during paradoxical sleep Behav. Neurosci. 107, 1018–1030 21 Maho, C. et al. (1991) Evocation in paradoxical sleep of a hippocampal conditioned cellular response acquired during waking Psychobiology 19, 193–205 22 Maho, C. and Bloch, V. (1992) Responses of hippocampal cells can be conditioned during paradoxical sleep Brain Res. 581, 115–122 23 Deweer, B. (1976) Selective facilitative effect of post-trial reticular stimulation in discriminative learning in the rat Behav. Proc. 1, 243–257 24 Bloch, V., Hennevin, E. and Leconte, P. (1977) Interaction between post-trial reticular stimulation and subsequent paradoxical sleep in memory consolidation processes, in Neurobiology of Sleep and Memory (Drucker-Colin, R.R. and McGaugh, J.L., eds), pp. 255–277, Academic Press 25 Hars, B. and Hennevin, E. (1983) Reminder abolishes impairment of learning induced by paradoxical sleep retardation Physiol. Behav. 30,

Acknowledgements This article is the product of several years of close collaboration with J. Allan Hobson, to whom I am deeply indebted. I also thank Michael Hasselmo for valuable discussions. This work has been supported by grants from the National Institutes of Health (MH 48,832) and the

831–836 26 Hennevin, E., Hars, B. and Bloch, V. (1989) Improvement of learning by mesencephalic reticular stimulation during postlearning paradoxical sleep Behav. Neural Biol. 51, 291–306 27 Smith, C. (1995) Sleep states and memory processes Behav. Brain Res.

MacArthur Foundation’s Mind–Body network.

69, 137–145 28 Empson, J.A.C. and Clarke, P.R.F. (1970) Rapid eye movements and remembering Nature 227, 287–288

References 1 Hartley, D. (1801) Observations on Akan, his Frame, his Duty and his

29 McGrath, M.H. and Cohen, D.B. (1978) REM sleep facilitation of adaptive waking behavior: a review of the literature Psychiatr. Bull. 85,

Eexpectations, Johnson 2 Jenkins, J.G. and Dallenbach, K.M. (1924) LXXII. Obliviscence during

24–57 30 Conway, J. and Smith, C. (1994) REM Sleep and Learning in Humans: A

sleep and waking Am. J. Psychol. 35, 605–612 3 Ebbinghaus, H. (1885) Uber das Gedachtnis (English transl. 1913,

Sensitivity to Specific Types of Learning Tasks, 12th Congress of the European Sleep Research Society, Florence, Italy

Memory), Teachers’ College Press 4 Aserinsky, E. and Kleitman, N. (1953) Regularly occurring periods of

31 Plihal, W. and Born, J. (1997) Effects of early and late nocturnal

ocular motility and concomitant phenomena during sleep Science 118,

sleep on declarative and procedural memory J. Cogn. Neurosci. 9, 534–547

361–375 5 Greenberg, R. and Pearlman, C.A. (1974) Cutting the REM nerve: an approach to the adaptive function of REM sleep Perspect. Biol. Med.

32 Karni, A. et al. (1994) Dependence on REM sleep of overnight improvement of a perceptual skill Science 265, 679–682 33 Stickgold, R. et al. (1998) Visual discrimination learning: both NREM

17, 513–521 6 Crick, F. and Mitchison, G. (1983) The function of dream sleep Nature

and REM are required Sleep 21 (Suppl.), 256 34 O’Keefe, J. and Dostrovsky, J. (1971) The hippocampus as a spatial

304, 111–114 7 Giuditta, A., et al. (1995) The sequential hypothesis of the function of

map: preliminary evidence from unit activity in the freely moving rat Brain Res. 34, 171–175

sleep Behav. Brain Res. 69, 157–166 8 Smith, C. (1985) Sleep states and learning: a review of the animal

35 Otto, T. et al. (1991) Learning-related patterns of CA1 spike trains parallel stimulation parameters optimal for inducing hippocampal

literature Neurosci. Biobehav. Rev. 9, 157–168 9 Hennevin, E., et al. (1995) Processing of learned information in

long-term potentiation Hippocampus 1, 181–192

paradoxical sleep: relevance for memory Behav. Brain Res. 69, 125–135

36 Pavlides, C. et al. (1988) Long-term potentiation in the dentate gyrus is

10 Lucero, M. (1970) Lengthening of REM sleep duration consecutive to

induced preferentially on the positive phase of theta-rhythm Brain Res. 439, 383–387

learning in the rat Brain Res. 20, 319–322 11 Hennevin, E., Leconte, P. and Bloch, V. (1974) Augmentation du sommeil paradoxal provoquée par l’acquisition, l’extinction et la

37 Buzsáki, G. (1996) The hippocampo–neocortical dialogue Cereb. Cortex 6, 81–92

491 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998

Review

Stickgold – Sleep and memory processing

38 Buzsáki, G. (1989) Two-stage model of memory trace formation: a role for ‘noisy’ brain states Neuroscience 31, 551–570

scopolamine on storage, retrieval and organizational processes in memory Psychopharmacologia 44, 257–262

39 Chrobak, J.J. and Buzsáki, G. (1994) Selective activation of deep layer (V–VI) retrohippocampal cortical neurons during hippocampal sharp waves in the behaving rat J. Neurosci. 14, 1660–1670

53 Whitehouse, J.M. (1967) Cholinergic mechanisms in discrimination learning as a function of stimuli J. Comp. Physiol. Psychol. 63, 448–451 54 Hasselmo, M.E. and Cekic, M. (1996) Suppression of synaptic

40 Pavlides, C. and Winson, J. (1989) Influences of hippocampal place cell

transmission may allow combination of associative feedback and self-

firing in the awake state on the activity of these cells during

organizing feedforward connections in the neocortex Behav. Brain

subsequent sleep episodes J. Neurosci. 9, 2907–2918

Res. 79, 153–161

41 Wilson, M.A. and McNaughton, B.L. (1994) Reactivation of hippocampal ensemble memories during sleep Science 265, 676–679

55 Baghdoyan, H.A. et al. (1987) A neuroanatomical gradient in the pontine

42 Poe, G.R. et al. (1996) Place cell theta phase firing profile differences from maze running to REM sleep: familiar versus novel place fields Soc. Neurosci. Abstr. 22, 1871

tegmentum

for

the

cholinoceptive

induction

of

desynchronized sleep signs Brain Res. 414, 245–261 56 Rittenhouse, C.D., Stickgold, R. and Hobson, J.A. (1994) Constraints on the transformation of characters and objects in dream reports

43 Huerta, P.T. and Lisman, J.E. (1995) Bidirectional synaptic plasticity induced by a single burst during cholinergic theta oscillation in CA1 in vitro Neuron 15, 1053–1063

Conscious. Cognit. 3, 100–113 57 Stickgold, R. et al. Sleep induced changes in associative memory J. Cogn. Neurosci. (in press)

44 Holscher, C., Anwyl, R. and Rowan, M.J. (1997) Stimulation on the positive phase of hippocampal theta rhythm induces long-term potentiation that can be depotentiated by stimulation on the negative phase in area CA1 in vivo J. Neurosci. 17, 6470–6477

58 Rechtschaffen, A., Verdone, P. and Wheaton, J. (1963) Reports of mental activity during sleep Can. J. Psychiatry 8, 409–414 59 Foulkes, D. (1962) Dream reports from different stages of sleep J. Abnorm. Soc. Psychol. 65, 14–25

45 Kim, J.J. and Fanselow, M.S. (1992) Modality-specific retrograde amnesia of fear Science 256, 675–677

60 Meyer, D.E. and Schvaneveldt, R.W. (1971) Facilitation in recognizing pairs of words: evidence of a dependence between retrieval

46 Kametani, H. and Kawamura, H. (1990) Alterations in acetylcholine release in the rat hippocampus during sleep-wakefulness detected by intracerebral dialysis Life Sci. 47, 421–426

operations J. Exp. Psychol. 90, 227–234 61 McClelland, J.L., McNaughton, B.L. and O’Reilly, R.C. (1995) Why there are complementary learning systems in the hippocampus and

47 Marrosu, F. et al. (1995) Microdialysis measurement of cortical and hippocampal acetylcholine release during sleep–wake cycle in freely moving cats Brain Res. 671, 329–332

neocortex: insights from the successes and failures of connectionist models of learning and memory Psychol. Rev. 102, 419–457 62 Squire, L.R. (1992) Memory and the hippocampus: a synthesis from

48 Aston-Jones, G. and Bloom, F.E. (1981) Activity of norepinephrinecontaining locus coeruleus neurons in behaving rats anticipates fluctuations in the sleep–waking cycle J. Neurosci. 1, 876–886

findings with rats, monkeys, and humans Psychol. Rev. 99, 191–231 63 Zola-Morgan, S. and Squire, L.R. (1990) The primate hippocampal formation: evidence for a time-limited role in memory storage Science

49 Portas, C.M. et al. (1998) On-line detection of extracellular levels of serotonin in dorsal raphe nucleus and frontal cortex over the sleep/wake cycle in the freely moving rat Neuroscience 83, 807–814

250, 288–290 64 Hinton, G.E. et al. (1995) The ‘Wake–Sleep’ algorithm for unsupervised neural networks Science 268, 1158–1161

50 Hasselmo, M.E., Wyble, B.P. and Wallenstein, G.V. (1996) Encoding and

65 Karni, A. and Sagi, D. (1991) Where practice makes perfect in texture

retrieval of episodic memories: role of cholinergic and GABAergic

discrimination: evidence for primary visual cortex plasticity Proc. Natl.

modulation in the hippocampus Hippocampus 6, 693–708

Acad. Sci. U. S. A. 88, 4966–4970

51 Hasselmo, M.E. and Bower, J.M. (1993) Acetylcholine and memory Trends Neurosci. 16, 218–222

66 Karni, A. et al. (1995) Slow changes in primary and secondary visual cortex associated with perceptual skill learning: an fMRI study

52 Groneim, M.M. and Mewaldt, S.P. (1975) Effects of diazepam and

NeuroImage 3, S543

Coming next year to Trends in Cognitive Sciences •Amygdala circuitry in attentional and representational process, by P. Holland and M. Gallagher •Computational studies of the development of functionally specialized neural modules, by R.A. Jacobs •Learning to recognize objects, by G. Wallis and H. Bulthoff •Keeping perception accurate, by F. Bedford •The functional anatomy of emotion and affective style, by R.J. Davidson and W. Irwin •Ten years of the rational analysis of cognition, by N. Chater and M. Oaksford •The constructive brain: neural, cognitive and computational evidence, by S.R. Quartz •Declarative and episodic memory: what can animals remember about their past? by D. Griffiths, N. Clayton and A. Dickinson •Concepts are more than categories, by K.O. Solomon, D.L. Medin and E. Lynch •Modularity and cognition, by M. Coltheart •Neuromodulation: acetylcholine and memory consolidation, by M.E. Hasselmo •The neural correlates of consciousness: an experimental framework, by C. Frith et al.

492 Trends in Cognitive Sciences – Vol. 2, No. 12,

December 1998