- Email: [email protected]
Steven Hsiao: In Memoriam
Neuron
Obituary Steven Hsiao: In Memoriam Hopkins neuroscientist Steven Hsiao, one of the world’s leading experts on the neural basis of touch, passed away on June 16, 2014 at the age of 59 from lung cancer. Steve, the heir of an eminent line of sensory neuroscientists that included Vernon B. Mouncastle and Kenneth O. Johnson, sought to understand the neural code: how do spatio-temporal patterns of activity along the neuraxis— from the nerve through several levels of processing in cortex—encode information about the environment? What language do neurons use to communicate with each other? In addressing questions of neural coding in the sense of touch, and following the example of his illustrious mentors, Steve fearlessly tackled the hard problems that face somatosensory research. For example, as there is no off-the-shelf tactile equivalent of visual monitors or audio speakers, Steve designed and constructed sophisticated pieces of experimental apparatus to probe the sense of touch. Steve also embraced and elaborated on the cutting edge of mathematical approaches to address neural coding. Steve believed that, to achieve a true understanding of neural coding, one had not only to develop a mechanistic understanding of how stimulus properties are encoded in the neuronal activity, but also to link these patterns of neuronal activity to perception. Steve’s study of roughness perception, which he carried out with Ken Johnson and other colleagues at Hopkins, provides a nice illustration of his multifaceted approach to sensory neuroscience. For a decade, Ken had been trying to understand how textured surfaces are represented in the responses of nerve fibers (Johnson and Lamb, 1981). When Steve joined Ken’s lab in the late eighties, he became involved in this project. The story goes that he was running his fingers across one of the textured ‘‘drums’’ that were used in the neurophysiological experiments. The drum was a cylinder on which dots had been embossed in different configurations. Steve noticed that, as the dots got further apart, the surface of the drum became rougher,
then smoother again, following an inverse U-shaped function (Connor et al., 1990). There did not seem to be any objective reason why this should be the case, but he reasoned that something in the evoked pattern of activity in the nerve must underlie this perceptual phenomenon. In an elegant series of studies, Steve and his colleagues explored the neural basis of perceived roughness using a pioneering hypothesis-driven approach (Connor et al., 1990; Connor and Johnson, 1992; Blake et al., 1997; Yoshioka et al., 2001). Specifically, they developed, operationalized, and tested a series of hypotheses as to which aspect of the neuronal activity matched the psychophysical ratings of roughness. After a decade of cleverly designed experiments, they were able to narrow the set of hypotheses down to one: the roughness of the embossed dot patterns—the mysterious inverse U-shaped function—was determined by the spatial layout of the response of one population of nerve fibers. The disciplined approach of identifying and successively eliminating the various hypotheses had led to a single, satisfying conclusion. As is always the case, the story ended up being more complicated, in that this result only held for relatively coarse textures (Weber et al., 2013). However, the path to achieving it—the quantitatively rigorous, hypothesis-driven approach— would endure and, in fact, inspired later efforts. While Mountcastle’s and Johnson’s seminal somatosensory studies and the roughness story (above) focused on how tactile information is initially transduced
Steven Hsiao
458 Neuron 85, February 4, 2015 ª2015 Elsevier Inc.
by a population array of somatosensory afferents, Steve was deeply driven by the goal of understanding how that population ‘‘image’’ is transformed to new forms of neural representation as it is propagated and processed within the CNS. He, like Johnson and Mountcastle, believed that this approach was the key to understanding not only somatosensation, but sensory cortical processing more broadly (especially vision; see below). To this end, Steve and Ken Johnson developed highly innovative methods of delivering controlled patterns of skin deformation to the fingertips (e.g., a rotating stimulus drum, a 400-probe stimulator; see below). In work with John Phillips, Steve and Ken combined the first of those innovations with state-of-the-art methods in awake behaving non-human primate cortical neuronal recording to give the world the first glimpse of how the isomorphic image transmitted by the sensory afferent population is transformed to a remarkably new and mysterious form of population representation in the somatosensory cortex (Phillips et al., 1988). But as an engineer, Steve was not satisfied with neural phenomenology. Instead, he wanted quantitative, predictive models of the neuronal responses—the engineer’s ‘‘transfer function’’ for each type of cortical neuron. Working with Jim DiCarlo and Ken Johnson, using cutting-edge data analysis methods, Steve helped develop the first quantitative, predictive receptive field models of cortical sensory processing in primary somatosensory cortical area 3b (DiCarlo et al., 1998). In doing so, they showed that pseudo-linear models could explain most of the stimulus-evoked neuronal activity and that the receptive fields of neurons in cortical area 3b exhibit a wide variety of spatial-temporal structures. Notably, the spatial separation of the excitatory and inhibitory fields in the receptive fields of many of these neurons (2 mm) was almost exactly that predicted by the earlier roughness work (above). In other words, these neurons seem to compute the spatial code inferred from the roughness work, and their firing rates can account for subjects’
Neuron
Obituary perception of roughness! In later work, Steve and Ken confirmed and extended those RF results using yet another sophisticated data analysis approach (spatiotemporal receptive fields) (Sripati et al., 2006). Remarkably, the structure of receptive fields in somatosensory cortex were shown to have much in common with their counterparts in primary visual and auditory cortex, suggesting that these three cortical areas implement very similar ‘‘image’’ processing strategies (deCharms and Zador, 2000). This work also had an important impact on the field from a conceptual and methodological point of view, as this way of conceptualizing and characterizing successive cortical transformation of neural population ‘‘images’’ forms the foundation of contemporary views on both visual and auditory hierarchical processing. After receiving his PhD, Steve set off on his own, founding his lab at Hopkins alongside that of his mentors, Mountcastle and Johnson. As a Principle Investigator, Steve focused on several key open questions in somatosensory neuroscience. One of the overarching questions that became a focus of his work was how tactile and proprioceptive information are integrated to perceive the shape of an object through haptic exploration (Hsiao, 2008). Indeed, when we grasp an object, multiple fingertips come into contact with it, at minimum the thumb and one of the fingers. The interpretation of the tactile information originating from each of these contact points needs to be integrated with proprioceptive information about the relative position of the contact points to achieve a 3D percept of the object. Steve’s team began to address this question by investigating, in a series of psychophysical experiments, the haptic perception of object size (Berryman et al., 2006). In these studies, subjects (blindfolded) judged the size of objects under several conditions. In the control condition, subjects had no difficulty in identifying the size of the object. However, if the cutaneous feedback was distorted (for example by desensitizing cutaneous nerve fibers) or eliminated (by anesthetizing the fingertip), size perception was severely impaired. These experiments demonstrated that our ability to make out a very basic 3D property of an
object—namely its size—relies on both cutaneous and proprioceptive input. In parallel, Steve’s team conducted a series of neurophysiological experiments in which they recorded the responses of neurons in cortical areas S1 and S2 while manipulating both the cutaneous and proprioceptive input. Specifically, the monkey’s hand was fixed in an actuated hand-holder that allowed the experimenter to change the relative position of the digits from trial to trial by manipulating the degree of abduction/adduction. Steve also developed a device that could precisely indent shapes anywhere on the surface of the skin (Lane et al., 2010). Incidentally, the study constitutes a compelling illustration of the difficulties in addressing questions of tactile and proprioceptive integration and Steve’s fearlessness and ability to address them. In these experiments, the shapes consisted of bars that could be indented anywhere on the palmar surface of the hand at any orientation. Steve discovered subpopulations of somatosensory neurons whose responses to the tactile stimuli (the oriented bars) were modulated by the configuration of the hand, demonstrating for the first time that the integration of tactile and proprioceptive information occurred at the single-cell level in primary somatosensory cortex (Kim et al., 2008). Steve’s lab also carried out seminal experiments to investigate somatosensory representations in S2. First, his team showed that S2 comprises three different functional fields: a central one in which neurons are primarily sensitive to cutaneous stimulation, flanked by two others (anterior and posterior) that respond to both cutaneous and proprioceptive input. He hypothesized that the latter two fields are involved in integrating tactile and proprioceptive information for stereognosis. Then, having previously shown that S1 neurons are orientation selective—that is, respond preferentially to edges at a specific orientation impinging on their receptive fields (Hsiao et al., 2002; Bensmaia et al., 2008)—Steve showed that the orientation tuning of S2 neurons is consistent over large swaths of skin, spanning the surface not only of the entire fingerpad (Thakur et al., 2006), but of multiple fingerpads (Fitzgerald et al., 2006). This positional tolerance of orientation tuning in touch was analogous
to its counterpart in vision and implies that sensory information in both systems is elaborated using similar neural mechanisms. Another hallmark of hierarchical processing in vision is the increase in complexity of neuronal feature selectivity as one ascends the visual hierarchy. Steve and colleagues showed that this was also the case in touch. Indeed, Steve recorded the responses of S1 and S2 neurons to edges that varied not only in orientation, but also in curvature (Yau et al., 2009, 2013). While neurons at early stages of somatosensory processing were sensitive to orientation, neurons at later stages, including cortical area 2 and S2, exhibited tuning for stimulus curvature. In fact, the responses of these curvature- and orientation-selective neurons were quantitatively similar to their counterparts in visual area V4, a result that further bolstered the idea that the principles of shape synthesis are similar in vision and touch. The striking parallels between visual and tactile processing were maybe most compellingly illustrated in a series of studies investigating the representation of tactile motion in the somatosensory system. In the 1990s, Steve and his mentor Ken Johnson had developed the tactile equivalent of a visual monitor, consisting of 400 probes, each under independent computer control, arrayed in a 20 3 20 grid over 1 cm2 (Killebrew et al., 2007). In a series of psychophysical and neurophysiological studies, Steve and colleagues used this stimulator to deliver motion stimuli to the fingertip of human subjects and non-human primates. They identified neurons in S1 that were tuned for the direction of bars scanned across the skin (Pei et al., 2010), as have been found in primary visual cortex. However, some of these direction-tuned neurons were tuned for motion direction regardless of what object moved across the skin. In fact, the direction signal conveyed by these neurons was invariant with other stimulus properties, including not only shape, but speed and amplitude as well. Steve and colleagues then measured the neuronal responses evoked by tactile analogs of stimuli that had been used to probe visual motion processing, including gratings, barberpoles, and random dot displays (Pei et al., 2008, 2010, 2011).
Neuron 85, February 4, 2015 ª2015 Elsevier Inc. 459
Neuron
Obituary They found that the responses to tactile motion of a subpopulation of S1 neurons were highly analogous to those reported in the middle temporal area, a visual area dedicated to motion processing. Thus, touch and vision employ analogous neural codes to represent not only shape information, but motion information as well. While much of Steve’s core work focused on understanding how somatosensory information is represented and transformed along the neuraxis, he was also a pioneer in working to understand how changes in mental state (especially attention) modulate these representations. In a seminal study, Steve and colleagues combined their trademark highly controlled tactile stimulation with behaving non-human primate neurophysiology to reveal that switching between visual and somatosensory tasks results in strong changes in the firing rates of neurons in both S1 and S2 somatosensory cortices (Hsiao et al., 1993). These early results were the foundation of later work that provided some of the first neurophysiological evidence for the idea that selective gating of sensory information might also be the result of changes in temporal synchrony among somatosensory neurons, presumably facilitating the transmission of sensory information to downstream neurons (Steinmetz et al., 2000). Not satisfied with those first phenomenological results, Steve worked with Hopkins colleague Ernst Niebur to explore his experimental results in the context of a range of theoretical ideas on attention (Niebur et al., 2002; Roy et al., 2007; Ray et al., 2008a, 2008b, 2008c), and he continued to pursue this line of inquiry right up until his death. Emblematic of his great drive to look for common cortical sensory processing strategies, a study published posthumously showed that the brain employs similar temporalcorrelation-based information gating strategies in both somatosensory and visual ‘‘feature attention’’ tasks (Gomez-Ramirez et al., 2014). While much remains to be understood in the area of attention, Steve provided a distinctive perspective on this important problem. Steve was also working to leverage basic scientific insights to develop ways to restore somatosensation in individuals who had lost it, including amputees and tetraplegic patients (Hsiao et al., 2011).
The idea was to develop algorithms that convert the output of sensors on the prosthetic limb into patterns of electrical stimulation in the nerve or in the brain to elicit meaningful tactile sensations. Unfortunately, he passed away before he was able to accomplish his goals, but he set the stage for others to continue this promising line of work. Steve was not just an eminent scientist and world leader in his field; he was also a very generous, personable, and likeable individual. As co-director of the Johns Hopkins graduate program in neuroscience, he was very dedicated to educating the next generation of scientists. One of his most distinctive character attributes was his laughter, with which he was very generous. He was always composed and sociable, he was an avid fan of college basketball (Duke was his team, to several of his colleagues’ dismay), loved to sail, and had sophisticated taste in food, wine, and scotch. He was a devoted husband to his wife Jocelyne DiRuggiero, also a Hopkins professor, and father to two sons, Kevin and Andrew. He chose not to tell many of his colleagues and friends of his terminal diagnosis in his last year because he wished to be treated as a colleague and a friend, not as a dying man. He will be deeply missed, but his personal and professional legacy will live on.
recognition: the relative contributions of slowly and rapidly adapting mechanoreceptors to perceived roughness. J. Neurosci. 17, 7480–7489.
Sliman J. Bensmaia1,* and James J. DiCarlo2,*
Hsiao, S.S., Fettiplace, M., and Darbandi, B. (2011). Sensory feedback for upper limb prostheses. Prog. Brain Res. 192, 69–81.
1
Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL 60637, USA 2 Department of Brain and Cognitive Sciences and McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA *Correspondence: [email protected] (S.J.B.), [email protected] (J.J.D.) http://dx.doi.org/10.1016/j.neuron.2015.01.015 REFERENCES Bensmaia, S.J., Denchev, P.V., Dammann, J.F., 3rd, Craig, J.C., and Hsiao, S.S. (2008). The representation of stimulus orientation in the early stages of somatosensory processing. J. Neurosci. 28, 776–786.
Connor, C.E., and Johnson, K.O. (1992). Neural coding of tactile texture: comparison of spatial and temporal mechanisms for roughness perception. J. Neurosci. 12, 3414–3426. Connor, C.E., Hsiao, S.S., Phillips, J.R., and Johnson, K.O. (1990). Tactile roughness: neural codes that account for psychophysical magnitude estimates. J. Neurosci. 10, 3823–3836. deCharms, R.C., and Zador, A. (2000). Neural representation and the cortical code. Annu. Rev. Neurosci. 23, 613–647. DiCarlo, J.J., Johnson, K.O., and Hsiao, S.S. (1998). Structure of receptive fields in area 3b of primary somatosensory cortex in the alert monkey. J. Neurosci. 18, 2626–2645. Fitzgerald, P.J., Lane, J.W., Thakur, P.H., and Hsiao, S.S. (2006). Receptive field properties of the macaque second somatosensory cortex: representation of orientation on different finger pads. J. Neurosci. 26, 6473–6484. Gomez-Ramirez, M., Trzcinski, N.K., Mihalas, S., Niebur, E., and Hsiao, S.S. (2014). Temporal correlation mechanisms and their role in feature selection: a single-unit study in primate somatosensory cortex. PLoS Biol. 12, e1002004. Hsiao, S. (2008). Central mechanisms of tactile shape perception. Curr. Opin. Neurobiol. 18, 418–424. Hsiao, S.S., O’Shaughnessy, D.M., and Johnson, K.O. (1993). Effects of selective attention on spatial form processing in monkey primary and secondary somatosensory cortex. J. Neurophysiol. 70, 444–447. Hsiao, S.S., Lane, J., and Fitzgerald, P. (2002). Representation of orientation in the somatosensory system. Behav. Brain Res. 135, 93–103.
Johnson, K.O., and Lamb, G.D. (1981). Neural mechanisms of spatial tactile discrimination: neural patterns evoked by braille-like dot patterns in the monkey. J. Physiol. 310, 117–144. Killebrew, J.H., Bensmaı¨a, S.J., Dammann, J.F., Denchev, P., Hsiao, S.S., Craig, J.C., and Johnson, K.O. (2007). A dense array stimulator to generate arbitrary spatio-temporal tactile stimuli. J. Neurosci. Methods 161, 62–74. Kim, S.S., Thakur, P.H., and Hsiao, S.S. (2008). Effects of proprioception on the responses of neurons in area 3a, 3b, 1 and 2 of SI cortex of the Macaque. Society for Neuroscience, (Washington, DC). Lane, J.W., Fitzgerald, P.J., Yau, J.M., Pembeci, I., and Hsiao, S.S. (2010). A tactile stimulator for studying passive shape perception. J. Neurosci. Methods 185, 221–229.
Berryman, L.J., Yau, J.M., and Hsiao, S.S. (2006). Representation of object size in the somatosensory system. J. Neurophysiol. 96, 27–39.
Niebur, E., Hsiao, S.S., and Johnson, K.O. (2002). Synchrony: a neuronal mechanism for attentional selection? Curr. Opin. Neurobiol. 12, 190–194.
Blake, D.T., Hsiao, S.S., and Johnson, K.O. (1997). Neural coding mechanisms in tactile pattern
Pei, Y.C., Hsiao, S.S., and Bensmaia, S.J. (2008). The tactile integration of local motion cues is
460 Neuron 85, February 4, 2015 ª2015 Elsevier Inc.
Neuron
Obituary analogous to its visual counterpart. Proc. Natl. Acad. Sci. USA 105, 8130–8135. Pei, Y.C., Hsiao, S.S., Craig, J.C., and Bensmaia, S.J. (2010). Shape invariant coding of motion direction in somatosensory cortex. PLoS Biol. 8, e1000305. Pei, Y.C., Hsiao, S.S., Craig, J.C., and Bensmaia, S.J. (2011). Neural mechanisms of tactile motion integration in somatosensory cortex. Neuron 69, 536–547. Phillips, J.R., Johnson, K.O., and Hsiao, S.S. (1988). Spatial pattern representation and transformation in monkey somatosensory cortex. Proc. Natl. Acad. Sci. USA 85, 1317–1321. Ray, S., Crone, N.E., Niebur, E., Franaszczuk, P.J., and Hsiao, S.S. (2008a). Neural correlates of highgamma oscillations (60-200 Hz) in macaque local field potentials and their potential implications in electrocorticography. J. Neurosci. 28, 11526– 11536. Ray, S., Hsiao, S.S., Crone, N.E., Franaszczuk, P.J., and Niebur, E. (2008b). Effect of stimulus in-
tensity on the spike-local field potential relationship in the secondary somatosensory cortex. J. Neurosci. 28, 7334–7343. Ray, S., Niebur, E., Hsiao, S.S., Sinai, A., and Crone, N.E. (2008c). High-frequency gamma activity (80150Hz) is increased in human cortex during selective attention. Clin. Neurophysiol. 119, 116–133. Roy, A., Steinmetz, P.N., Hsiao, S.S., Johnson, K.O., and Niebur, E. (2007). Synchrony: a neural correlate of somatosensory attention. J. Neurophysiol. 98, 1645–1661. Sripati, A.P., Yoshioka, T., Denchev, P., Hsiao, S.S., and Johnson, K.O. (2006). Spatiotemporal receptive fields of peripheral afferents and cortical area 3b and 1 neurons in the primate somatosensory system. J. Neurosci. 26, 2101–2114. Steinmetz, P.N., Roy, A., Fitzgerald, P.J., Hsiao, S.S., Johnson, K.O., and Niebur, E. (2000). Attention modulates synchronized neuronal firing in primate somatosensory cortex. Nature 404, 187–190. Thakur, P.H., Fitzgerald, P.J., Lane, J.W., and Hsiao, S.S. (2006). Receptive field properties of
the macaque second somatosensory cortex: nonlinear mechanisms underlying the representation of orientation within a finger pad. J. Neurosci. 26, 13567–13575. Weber, A.I., Saal, H.P., Lieber, J.D., Cheng, J.W., Manfredi, L.R., Dammann, J.F., 3rd, and Bensmaia, S.J. (2013). Spatial and temporal codes mediate the tactile perception of natural textures. Proc. Natl. Acad. Sci. USA 110, 17107–17112. Yau, J.M., Pasupathy, A., Fitzgerald, P.J., Hsiao, S.S., and Connor, C.E. (2009). Analogous intermediate shape coding in vision and touch. Proc. Natl. Acad. Sci. USA 106, 16457–16462. Yau, J.M., Connor, C.E., and Hsiao, S.S. (2013). Representation of tactile curvature in macaque somatosensory area 2. J. Neurophysiol. 109, 2999– 3012. Yoshioka, T., Gibb, B., Dorsch, A.K., Hsiao, S.S., and Johnson, K.O. (2001). Neural coding mechanisms underlying perceived roughness of finely textured surfaces. J. Neurosci. 21, 6905– 6916.
Neuron 85, February 4, 2015 ª2015 Elsevier Inc. 461
Obituary Steven Hsiao: In Memoriam Hopkins neuroscientist Steven Hsiao, one of the world’s leading experts on the neural basis of touch, passed away on June 16, 2014 at the age of 59 from lung cancer. Steve, the heir of an eminent line of sensory neuroscientists that included Vernon B. Mouncastle and Kenneth O. Johnson, sought to understand the neural code: how do spatio-temporal patterns of activity along the neuraxis— from the nerve through several levels of processing in cortex—encode information about the environment? What language do neurons use to communicate with each other? In addressing questions of neural coding in the sense of touch, and following the example of his illustrious mentors, Steve fearlessly tackled the hard problems that face somatosensory research. For example, as there is no off-the-shelf tactile equivalent of visual monitors or audio speakers, Steve designed and constructed sophisticated pieces of experimental apparatus to probe the sense of touch. Steve also embraced and elaborated on the cutting edge of mathematical approaches to address neural coding. Steve believed that, to achieve a true understanding of neural coding, one had not only to develop a mechanistic understanding of how stimulus properties are encoded in the neuronal activity, but also to link these patterns of neuronal activity to perception. Steve’s study of roughness perception, which he carried out with Ken Johnson and other colleagues at Hopkins, provides a nice illustration of his multifaceted approach to sensory neuroscience. For a decade, Ken had been trying to understand how textured surfaces are represented in the responses of nerve fibers (Johnson and Lamb, 1981). When Steve joined Ken’s lab in the late eighties, he became involved in this project. The story goes that he was running his fingers across one of the textured ‘‘drums’’ that were used in the neurophysiological experiments. The drum was a cylinder on which dots had been embossed in different configurations. Steve noticed that, as the dots got further apart, the surface of the drum became rougher,
then smoother again, following an inverse U-shaped function (Connor et al., 1990). There did not seem to be any objective reason why this should be the case, but he reasoned that something in the evoked pattern of activity in the nerve must underlie this perceptual phenomenon. In an elegant series of studies, Steve and his colleagues explored the neural basis of perceived roughness using a pioneering hypothesis-driven approach (Connor et al., 1990; Connor and Johnson, 1992; Blake et al., 1997; Yoshioka et al., 2001). Specifically, they developed, operationalized, and tested a series of hypotheses as to which aspect of the neuronal activity matched the psychophysical ratings of roughness. After a decade of cleverly designed experiments, they were able to narrow the set of hypotheses down to one: the roughness of the embossed dot patterns—the mysterious inverse U-shaped function—was determined by the spatial layout of the response of one population of nerve fibers. The disciplined approach of identifying and successively eliminating the various hypotheses had led to a single, satisfying conclusion. As is always the case, the story ended up being more complicated, in that this result only held for relatively coarse textures (Weber et al., 2013). However, the path to achieving it—the quantitatively rigorous, hypothesis-driven approach— would endure and, in fact, inspired later efforts. While Mountcastle’s and Johnson’s seminal somatosensory studies and the roughness story (above) focused on how tactile information is initially transduced
Steven Hsiao
458 Neuron 85, February 4, 2015 ª2015 Elsevier Inc.
by a population array of somatosensory afferents, Steve was deeply driven by the goal of understanding how that population ‘‘image’’ is transformed to new forms of neural representation as it is propagated and processed within the CNS. He, like Johnson and Mountcastle, believed that this approach was the key to understanding not only somatosensation, but sensory cortical processing more broadly (especially vision; see below). To this end, Steve and Ken Johnson developed highly innovative methods of delivering controlled patterns of skin deformation to the fingertips (e.g., a rotating stimulus drum, a 400-probe stimulator; see below). In work with John Phillips, Steve and Ken combined the first of those innovations with state-of-the-art methods in awake behaving non-human primate cortical neuronal recording to give the world the first glimpse of how the isomorphic image transmitted by the sensory afferent population is transformed to a remarkably new and mysterious form of population representation in the somatosensory cortex (Phillips et al., 1988). But as an engineer, Steve was not satisfied with neural phenomenology. Instead, he wanted quantitative, predictive models of the neuronal responses—the engineer’s ‘‘transfer function’’ for each type of cortical neuron. Working with Jim DiCarlo and Ken Johnson, using cutting-edge data analysis methods, Steve helped develop the first quantitative, predictive receptive field models of cortical sensory processing in primary somatosensory cortical area 3b (DiCarlo et al., 1998). In doing so, they showed that pseudo-linear models could explain most of the stimulus-evoked neuronal activity and that the receptive fields of neurons in cortical area 3b exhibit a wide variety of spatial-temporal structures. Notably, the spatial separation of the excitatory and inhibitory fields in the receptive fields of many of these neurons (2 mm) was almost exactly that predicted by the earlier roughness work (above). In other words, these neurons seem to compute the spatial code inferred from the roughness work, and their firing rates can account for subjects’
Neuron
Obituary perception of roughness! In later work, Steve and Ken confirmed and extended those RF results using yet another sophisticated data analysis approach (spatiotemporal receptive fields) (Sripati et al., 2006). Remarkably, the structure of receptive fields in somatosensory cortex were shown to have much in common with their counterparts in primary visual and auditory cortex, suggesting that these three cortical areas implement very similar ‘‘image’’ processing strategies (deCharms and Zador, 2000). This work also had an important impact on the field from a conceptual and methodological point of view, as this way of conceptualizing and characterizing successive cortical transformation of neural population ‘‘images’’ forms the foundation of contemporary views on both visual and auditory hierarchical processing. After receiving his PhD, Steve set off on his own, founding his lab at Hopkins alongside that of his mentors, Mountcastle and Johnson. As a Principle Investigator, Steve focused on several key open questions in somatosensory neuroscience. One of the overarching questions that became a focus of his work was how tactile and proprioceptive information are integrated to perceive the shape of an object through haptic exploration (Hsiao, 2008). Indeed, when we grasp an object, multiple fingertips come into contact with it, at minimum the thumb and one of the fingers. The interpretation of the tactile information originating from each of these contact points needs to be integrated with proprioceptive information about the relative position of the contact points to achieve a 3D percept of the object. Steve’s team began to address this question by investigating, in a series of psychophysical experiments, the haptic perception of object size (Berryman et al., 2006). In these studies, subjects (blindfolded) judged the size of objects under several conditions. In the control condition, subjects had no difficulty in identifying the size of the object. However, if the cutaneous feedback was distorted (for example by desensitizing cutaneous nerve fibers) or eliminated (by anesthetizing the fingertip), size perception was severely impaired. These experiments demonstrated that our ability to make out a very basic 3D property of an
object—namely its size—relies on both cutaneous and proprioceptive input. In parallel, Steve’s team conducted a series of neurophysiological experiments in which they recorded the responses of neurons in cortical areas S1 and S2 while manipulating both the cutaneous and proprioceptive input. Specifically, the monkey’s hand was fixed in an actuated hand-holder that allowed the experimenter to change the relative position of the digits from trial to trial by manipulating the degree of abduction/adduction. Steve also developed a device that could precisely indent shapes anywhere on the surface of the skin (Lane et al., 2010). Incidentally, the study constitutes a compelling illustration of the difficulties in addressing questions of tactile and proprioceptive integration and Steve’s fearlessness and ability to address them. In these experiments, the shapes consisted of bars that could be indented anywhere on the palmar surface of the hand at any orientation. Steve discovered subpopulations of somatosensory neurons whose responses to the tactile stimuli (the oriented bars) were modulated by the configuration of the hand, demonstrating for the first time that the integration of tactile and proprioceptive information occurred at the single-cell level in primary somatosensory cortex (Kim et al., 2008). Steve’s lab also carried out seminal experiments to investigate somatosensory representations in S2. First, his team showed that S2 comprises three different functional fields: a central one in which neurons are primarily sensitive to cutaneous stimulation, flanked by two others (anterior and posterior) that respond to both cutaneous and proprioceptive input. He hypothesized that the latter two fields are involved in integrating tactile and proprioceptive information for stereognosis. Then, having previously shown that S1 neurons are orientation selective—that is, respond preferentially to edges at a specific orientation impinging on their receptive fields (Hsiao et al., 2002; Bensmaia et al., 2008)—Steve showed that the orientation tuning of S2 neurons is consistent over large swaths of skin, spanning the surface not only of the entire fingerpad (Thakur et al., 2006), but of multiple fingerpads (Fitzgerald et al., 2006). This positional tolerance of orientation tuning in touch was analogous
to its counterpart in vision and implies that sensory information in both systems is elaborated using similar neural mechanisms. Another hallmark of hierarchical processing in vision is the increase in complexity of neuronal feature selectivity as one ascends the visual hierarchy. Steve and colleagues showed that this was also the case in touch. Indeed, Steve recorded the responses of S1 and S2 neurons to edges that varied not only in orientation, but also in curvature (Yau et al., 2009, 2013). While neurons at early stages of somatosensory processing were sensitive to orientation, neurons at later stages, including cortical area 2 and S2, exhibited tuning for stimulus curvature. In fact, the responses of these curvature- and orientation-selective neurons were quantitatively similar to their counterparts in visual area V4, a result that further bolstered the idea that the principles of shape synthesis are similar in vision and touch. The striking parallels between visual and tactile processing were maybe most compellingly illustrated in a series of studies investigating the representation of tactile motion in the somatosensory system. In the 1990s, Steve and his mentor Ken Johnson had developed the tactile equivalent of a visual monitor, consisting of 400 probes, each under independent computer control, arrayed in a 20 3 20 grid over 1 cm2 (Killebrew et al., 2007). In a series of psychophysical and neurophysiological studies, Steve and colleagues used this stimulator to deliver motion stimuli to the fingertip of human subjects and non-human primates. They identified neurons in S1 that were tuned for the direction of bars scanned across the skin (Pei et al., 2010), as have been found in primary visual cortex. However, some of these direction-tuned neurons were tuned for motion direction regardless of what object moved across the skin. In fact, the direction signal conveyed by these neurons was invariant with other stimulus properties, including not only shape, but speed and amplitude as well. Steve and colleagues then measured the neuronal responses evoked by tactile analogs of stimuli that had been used to probe visual motion processing, including gratings, barberpoles, and random dot displays (Pei et al., 2008, 2010, 2011).
Neuron 85, February 4, 2015 ª2015 Elsevier Inc. 459
Neuron
Obituary They found that the responses to tactile motion of a subpopulation of S1 neurons were highly analogous to those reported in the middle temporal area, a visual area dedicated to motion processing. Thus, touch and vision employ analogous neural codes to represent not only shape information, but motion information as well. While much of Steve’s core work focused on understanding how somatosensory information is represented and transformed along the neuraxis, he was also a pioneer in working to understand how changes in mental state (especially attention) modulate these representations. In a seminal study, Steve and colleagues combined their trademark highly controlled tactile stimulation with behaving non-human primate neurophysiology to reveal that switching between visual and somatosensory tasks results in strong changes in the firing rates of neurons in both S1 and S2 somatosensory cortices (Hsiao et al., 1993). These early results were the foundation of later work that provided some of the first neurophysiological evidence for the idea that selective gating of sensory information might also be the result of changes in temporal synchrony among somatosensory neurons, presumably facilitating the transmission of sensory information to downstream neurons (Steinmetz et al., 2000). Not satisfied with those first phenomenological results, Steve worked with Hopkins colleague Ernst Niebur to explore his experimental results in the context of a range of theoretical ideas on attention (Niebur et al., 2002; Roy et al., 2007; Ray et al., 2008a, 2008b, 2008c), and he continued to pursue this line of inquiry right up until his death. Emblematic of his great drive to look for common cortical sensory processing strategies, a study published posthumously showed that the brain employs similar temporalcorrelation-based information gating strategies in both somatosensory and visual ‘‘feature attention’’ tasks (Gomez-Ramirez et al., 2014). While much remains to be understood in the area of attention, Steve provided a distinctive perspective on this important problem. Steve was also working to leverage basic scientific insights to develop ways to restore somatosensation in individuals who had lost it, including amputees and tetraplegic patients (Hsiao et al., 2011).
The idea was to develop algorithms that convert the output of sensors on the prosthetic limb into patterns of electrical stimulation in the nerve or in the brain to elicit meaningful tactile sensations. Unfortunately, he passed away before he was able to accomplish his goals, but he set the stage for others to continue this promising line of work. Steve was not just an eminent scientist and world leader in his field; he was also a very generous, personable, and likeable individual. As co-director of the Johns Hopkins graduate program in neuroscience, he was very dedicated to educating the next generation of scientists. One of his most distinctive character attributes was his laughter, with which he was very generous. He was always composed and sociable, he was an avid fan of college basketball (Duke was his team, to several of his colleagues’ dismay), loved to sail, and had sophisticated taste in food, wine, and scotch. He was a devoted husband to his wife Jocelyne DiRuggiero, also a Hopkins professor, and father to two sons, Kevin and Andrew. He chose not to tell many of his colleagues and friends of his terminal diagnosis in his last year because he wished to be treated as a colleague and a friend, not as a dying man. He will be deeply missed, but his personal and professional legacy will live on.
recognition: the relative contributions of slowly and rapidly adapting mechanoreceptors to perceived roughness. J. Neurosci. 17, 7480–7489.
Sliman J. Bensmaia1,* and James J. DiCarlo2,*
Hsiao, S.S., Fettiplace, M., and Darbandi, B. (2011). Sensory feedback for upper limb prostheses. Prog. Brain Res. 192, 69–81.
1
Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL 60637, USA 2 Department of Brain and Cognitive Sciences and McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA *Correspondence: [email protected] (S.J.B.), [email protected] (J.J.D.) http://dx.doi.org/10.1016/j.neuron.2015.01.015 REFERENCES Bensmaia, S.J., Denchev, P.V., Dammann, J.F., 3rd, Craig, J.C., and Hsiao, S.S. (2008). The representation of stimulus orientation in the early stages of somatosensory processing. J. Neurosci. 28, 776–786.
Connor, C.E., and Johnson, K.O. (1992). Neural coding of tactile texture: comparison of spatial and temporal mechanisms for roughness perception. J. Neurosci. 12, 3414–3426. Connor, C.E., Hsiao, S.S., Phillips, J.R., and Johnson, K.O. (1990). Tactile roughness: neural codes that account for psychophysical magnitude estimates. J. Neurosci. 10, 3823–3836. deCharms, R.C., and Zador, A. (2000). Neural representation and the cortical code. Annu. Rev. Neurosci. 23, 613–647. DiCarlo, J.J., Johnson, K.O., and Hsiao, S.S. (1998). Structure of receptive fields in area 3b of primary somatosensory cortex in the alert monkey. J. Neurosci. 18, 2626–2645. Fitzgerald, P.J., Lane, J.W., Thakur, P.H., and Hsiao, S.S. (2006). Receptive field properties of the macaque second somatosensory cortex: representation of orientation on different finger pads. J. Neurosci. 26, 6473–6484. Gomez-Ramirez, M., Trzcinski, N.K., Mihalas, S., Niebur, E., and Hsiao, S.S. (2014). Temporal correlation mechanisms and their role in feature selection: a single-unit study in primate somatosensory cortex. PLoS Biol. 12, e1002004. Hsiao, S. (2008). Central mechanisms of tactile shape perception. Curr. Opin. Neurobiol. 18, 418–424. Hsiao, S.S., O’Shaughnessy, D.M., and Johnson, K.O. (1993). Effects of selective attention on spatial form processing in monkey primary and secondary somatosensory cortex. J. Neurophysiol. 70, 444–447. Hsiao, S.S., Lane, J., and Fitzgerald, P. (2002). Representation of orientation in the somatosensory system. Behav. Brain Res. 135, 93–103.
Johnson, K.O., and Lamb, G.D. (1981). Neural mechanisms of spatial tactile discrimination: neural patterns evoked by braille-like dot patterns in the monkey. J. Physiol. 310, 117–144. Killebrew, J.H., Bensmaı¨a, S.J., Dammann, J.F., Denchev, P., Hsiao, S.S., Craig, J.C., and Johnson, K.O. (2007). A dense array stimulator to generate arbitrary spatio-temporal tactile stimuli. J. Neurosci. Methods 161, 62–74. Kim, S.S., Thakur, P.H., and Hsiao, S.S. (2008). Effects of proprioception on the responses of neurons in area 3a, 3b, 1 and 2 of SI cortex of the Macaque. Society for Neuroscience, (Washington, DC). Lane, J.W., Fitzgerald, P.J., Yau, J.M., Pembeci, I., and Hsiao, S.S. (2010). A tactile stimulator for studying passive shape perception. J. Neurosci. Methods 185, 221–229.
Berryman, L.J., Yau, J.M., and Hsiao, S.S. (2006). Representation of object size in the somatosensory system. J. Neurophysiol. 96, 27–39.
Niebur, E., Hsiao, S.S., and Johnson, K.O. (2002). Synchrony: a neuronal mechanism for attentional selection? Curr. Opin. Neurobiol. 12, 190–194.
Blake, D.T., Hsiao, S.S., and Johnson, K.O. (1997). Neural coding mechanisms in tactile pattern
Pei, Y.C., Hsiao, S.S., and Bensmaia, S.J. (2008). The tactile integration of local motion cues is
460 Neuron 85, February 4, 2015 ª2015 Elsevier Inc.
Neuron
Obituary analogous to its visual counterpart. Proc. Natl. Acad. Sci. USA 105, 8130–8135. Pei, Y.C., Hsiao, S.S., Craig, J.C., and Bensmaia, S.J. (2010). Shape invariant coding of motion direction in somatosensory cortex. PLoS Biol. 8, e1000305. Pei, Y.C., Hsiao, S.S., Craig, J.C., and Bensmaia, S.J. (2011). Neural mechanisms of tactile motion integration in somatosensory cortex. Neuron 69, 536–547. Phillips, J.R., Johnson, K.O., and Hsiao, S.S. (1988). Spatial pattern representation and transformation in monkey somatosensory cortex. Proc. Natl. Acad. Sci. USA 85, 1317–1321. Ray, S., Crone, N.E., Niebur, E., Franaszczuk, P.J., and Hsiao, S.S. (2008a). Neural correlates of highgamma oscillations (60-200 Hz) in macaque local field potentials and their potential implications in electrocorticography. J. Neurosci. 28, 11526– 11536. Ray, S., Hsiao, S.S., Crone, N.E., Franaszczuk, P.J., and Niebur, E. (2008b). Effect of stimulus in-
tensity on the spike-local field potential relationship in the secondary somatosensory cortex. J. Neurosci. 28, 7334–7343. Ray, S., Niebur, E., Hsiao, S.S., Sinai, A., and Crone, N.E. (2008c). High-frequency gamma activity (80150Hz) is increased in human cortex during selective attention. Clin. Neurophysiol. 119, 116–133. Roy, A., Steinmetz, P.N., Hsiao, S.S., Johnson, K.O., and Niebur, E. (2007). Synchrony: a neural correlate of somatosensory attention. J. Neurophysiol. 98, 1645–1661. Sripati, A.P., Yoshioka, T., Denchev, P., Hsiao, S.S., and Johnson, K.O. (2006). Spatiotemporal receptive fields of peripheral afferents and cortical area 3b and 1 neurons in the primate somatosensory system. J. Neurosci. 26, 2101–2114. Steinmetz, P.N., Roy, A., Fitzgerald, P.J., Hsiao, S.S., Johnson, K.O., and Niebur, E. (2000). Attention modulates synchronized neuronal firing in primate somatosensory cortex. Nature 404, 187–190. Thakur, P.H., Fitzgerald, P.J., Lane, J.W., and Hsiao, S.S. (2006). Receptive field properties of
the macaque second somatosensory cortex: nonlinear mechanisms underlying the representation of orientation within a finger pad. J. Neurosci. 26, 13567–13575. Weber, A.I., Saal, H.P., Lieber, J.D., Cheng, J.W., Manfredi, L.R., Dammann, J.F., 3rd, and Bensmaia, S.J. (2013). Spatial and temporal codes mediate the tactile perception of natural textures. Proc. Natl. Acad. Sci. USA 110, 17107–17112. Yau, J.M., Pasupathy, A., Fitzgerald, P.J., Hsiao, S.S., and Connor, C.E. (2009). Analogous intermediate shape coding in vision and touch. Proc. Natl. Acad. Sci. USA 106, 16457–16462. Yau, J.M., Connor, C.E., and Hsiao, S.S. (2013). Representation of tactile curvature in macaque somatosensory area 2. J. Neurophysiol. 109, 2999– 3012. Yoshioka, T., Gibb, B., Dorsch, A.K., Hsiao, S.S., and Johnson, K.O. (2001). Neural coding mechanisms underlying perceived roughness of finely textured surfaces. J. Neurosci. 21, 6905– 6916.
Neuron 85, February 4, 2015 ª2015 Elsevier Inc. 461