- Email: [email protected]
How free decisions originate from deterministic neural dynamics?
e330
Abstracts / Neuroscience Research 68S (2010) e223–e334
andpartially observable systems. Thanks to these advantages, the proposedcontrol framework can be applied to real systems such as humanoidrobots.
with multiple actions showed the larger success rate (93.3%) than that of the hand-coded control (51.2%).
doi:10.1016/j.neures.2010.07.1461
doi:10.1016/j.neures.2010.07.1464
P2-r13 Extraction of movement primitives without explicit labeling for imitation learning
P2-r16 How free decisions originate from deterministic neural dynamics?
Yuka Ariki 1,3 , Jun Morimoto 2,3 , Sang-Ho Hyon 2,3
Jun Namikawa , Ryunosuke Nishimoto, Jun Tani
1
Brain Science Institute, RIKEN
ATR 2 ATR-CNS Kyoto Japan 3 NAIST Nara Japan
Imitation learning has been one of the major research topics in neuroscience, especially after remarkable neural activities related to the imitation learning are observed in a monkeys brain. Some neurons in monkeys premotor cortex (area F5), called mirror neurons, activates not only when a monkey observes a specific behavior but also when the monkey generates the same behavior. However, in general, the imitated movements are not identical to the instructors behaviors since dynamic and kinematic properties of an imitator and an instructor are different. Therefore, the question is how the observed behaviors are converted to imitated actions. The concept of movement primitives is one of the key ideas to explain this conversion. Movement primitives are components that can generate action sequence to accomplish a goal directed behavior. In our study, we define a movement primitive as a linear dynamical model. The movement primitives are extracted from captured human movements by using the Gaussian mixture of linear dynamical models without explicit labeling process by an experimenter. We show that the extracted movement primitives can be used to recognize newly observed human behaviors and generate imitated movements by using a simple fourlink robot model. doi:10.1016/j.neures.2010.07.1462
P2-r14 Development of a P-E hybrid exoskeleton robot for walking and postural rehabilitation Sang-Ho Hyon 1,2 , Jun Morimoto 1 , Mitsuo Kawato 1 1
ATR-CNS, Kyoto 2 Dept of Robotics, Ritsumeikan Univ, Shiga
We propose a novel exoskeleton robot aimed at rehabilitation on biped walking or postural control for elderly persons, spinal cord injury, or stroke patients.By combining pneumatic muscles with electric motors, the robot works not only as a lightweight, torque controllable prosthesis for lower extremities and trunk, but also an autonomous robot that can compliantly balance itself.This paper shows the hardware overview and the basic experimental results.
Neuroscience studies recently reported that forthcoming actions are determined by unconscious mental processes long before we become aware of our intention to act. Libet et al. (1985) showed that conscious decision to press the button was preceded by a negative brain potential for a few hundred milliseconds, the so-called “readiness potential” that originates from the supplementary motor area (SMA).Soon et al. (2008) showed that the preceded brain activity is initiated in prefrontal cortex (PFC) and parietal cortex up to 7 s before the conscious decisions from their fMRI study.These evidences could be interpreted that the decision is initially generated unconsciously in the PFC which is regarded as the top of the hierarchy for action organization, and then it propagates downward to the SMA which is responsible for preparation of immediate motor programs. Despite these progress, we may still have a question how the initial PFC activities for the decisions originate at the first setout. For investigating the possible hypothesis of brain mechanism accounting for the free decision, we conducted model studies accompanied by neuro-robotics experiments in terms of dynamical systems. The model consists of a slow dynamics network in the higher level and the fast dynamics network in the lower level, and is a relatively simple implementation that shares cortical hierarchies like PFC and SMA in the brain. The robot controlled by the network is tutored to imitate experimenter’s free decision iterations of moving an object, which is a modified version of Soon’s task. After the learning, we show that the robot is able to mimic the experimenter’s voluntary movements by means of self-organized deterministic chaos.By observing neural activities in the network that mimics experimenter’s movements, we find that neural units in higher-level constitute the chaotic dynamics and activities of them exhibit long time correlation by which outcome of decisions can be predicted. doi:10.1016/j.neures.2010.07.1465
P2-r17 Human brain image registration for diffusion MRI data and neuro-fiber categorization Kevin C.Y. Chien 1 , Shigeyuki Oba 1,2 , Shin Ishii 1 1
doi:10.1016/j.neures.2010.07.1463
P2-r15 Reinforcement learning using multiple actions Hayato Nakama , Tsubasa Asano, Satoshi Yamada Graduate School of Engineering, Okayama University of Science, Okayama Reinforcement learning (RL) is a collection of methods for discovering nearoptimal solutions to stochastic sequential decision problems. An RL system interacts with the environment by executing actions and receiving rewards from the environment. However, many realistic control problems are too large to be solved practically by searching the state-space using primitive actions. One way to overcome this difficulty is to use “multiple action” in RL. The multiple action consists of a group of primitive actions and the termination conditions. When one of multiple actions is selected, the group of actions is executed until the termination condition is satisfied. In this study, the action space is constructed of multiple actions. Since multiple actions commit the learning agent to act in a particular purposeful way for a sustained period of time, it is expected that they accelerate the reinforcement learning. The task trained in this study is that e-puck robot approaches the target, which has the correct image on the side with a lamp, with avoiding walls and 4 dummy objects(correct image without lamp, wrong image with lamp, wrong image without lamp, and just lamp). Since several kinds of sensor values, distance sensors, optical sensors and camera, are required to achieve the task, it is difficult that the ordinary RL learned the control. The following seven multiple actions are used. GoStraight(), TurnRightToSensD(N), TurnRightToSensO(N), TurnLeftToSensD(N), TurnLeftToSensO(N), SearchTarget(), and ApproachTarget() (SensD, SensO and N denote distance sensor, optical sensor and the number of sensor, respectively). The performance of the RL with multiple actions was compared with that of the hand-coded control using multiple actions in the e-puck simulator. The control trained by the RL
System Science, Kyoto University 2 PRESTO, Japan Science and Technology Agency Diffusion magnetic resonance imaging (MRI) of human brain provides exhaustive information of fiber trajectories that reflect global neural connectivity between brain regions, which is considered as akey of connectome study to unveil detail functions of brain and its difference between individuals. In order to understand the total structural meaning of fiber trajectories, it is important to extract fiber bundles that are common between different individuals, where fiber bundles are groups of neural fibers with similar starting and/or end points that transmit information from one area of brain to another. However, the task of extracting fiber bundles is challenging because of following reasons: (1) fiber bundle is not well defined in mathematical manner, (2) computational problem because of large number of fibers, and (3) artifact alters global shape of brain even between different scans of same individual brain. Methods based on statistical information of fibers to extract fiber bundles have not been able to overcome the difficulties above. In this study, we propose employing Mean Shift algorithm to address problems (1) and (3). A CUDA version of the Mean Shift algorithm is written to illustrate the paradigm of replacing multi-CPU computer with CUDA enabled GPU especially when performing computationally expensive calculations. This attempt will address problem (2). Furthermore, we hypothesize that functional areas of brain are related to fiber bundles traversing and acting as message highways across them. Basing on this hypothesis, we aim to link biological functions of brain areas with its structure across different individuals by registering fiber bundles within brain images. A nonlinear image registration technique is chosen, warping brain images containing fiber bundles to resemble the fiber structure of a reference brain image. Taken altogether, our research will provide a novel analysis tool for neuroanatomy. doi:10.1016/j.neures.2010.07.1466
Abstracts / Neuroscience Research 68S (2010) e223–e334
andpartially observable systems. Thanks to these advantages, the proposedcontrol framework can be applied to real systems such as humanoidrobots.
with multiple actions showed the larger success rate (93.3%) than that of the hand-coded control (51.2%).
doi:10.1016/j.neures.2010.07.1461
doi:10.1016/j.neures.2010.07.1464
P2-r13 Extraction of movement primitives without explicit labeling for imitation learning
P2-r16 How free decisions originate from deterministic neural dynamics?
Yuka Ariki 1,3 , Jun Morimoto 2,3 , Sang-Ho Hyon 2,3
Jun Namikawa , Ryunosuke Nishimoto, Jun Tani
1
Brain Science Institute, RIKEN
ATR 2 ATR-CNS Kyoto Japan 3 NAIST Nara Japan
Imitation learning has been one of the major research topics in neuroscience, especially after remarkable neural activities related to the imitation learning are observed in a monkeys brain. Some neurons in monkeys premotor cortex (area F5), called mirror neurons, activates not only when a monkey observes a specific behavior but also when the monkey generates the same behavior. However, in general, the imitated movements are not identical to the instructors behaviors since dynamic and kinematic properties of an imitator and an instructor are different. Therefore, the question is how the observed behaviors are converted to imitated actions. The concept of movement primitives is one of the key ideas to explain this conversion. Movement primitives are components that can generate action sequence to accomplish a goal directed behavior. In our study, we define a movement primitive as a linear dynamical model. The movement primitives are extracted from captured human movements by using the Gaussian mixture of linear dynamical models without explicit labeling process by an experimenter. We show that the extracted movement primitives can be used to recognize newly observed human behaviors and generate imitated movements by using a simple fourlink robot model. doi:10.1016/j.neures.2010.07.1462
P2-r14 Development of a P-E hybrid exoskeleton robot for walking and postural rehabilitation Sang-Ho Hyon 1,2 , Jun Morimoto 1 , Mitsuo Kawato 1 1
ATR-CNS, Kyoto 2 Dept of Robotics, Ritsumeikan Univ, Shiga
We propose a novel exoskeleton robot aimed at rehabilitation on biped walking or postural control for elderly persons, spinal cord injury, or stroke patients.By combining pneumatic muscles with electric motors, the robot works not only as a lightweight, torque controllable prosthesis for lower extremities and trunk, but also an autonomous robot that can compliantly balance itself.This paper shows the hardware overview and the basic experimental results.
Neuroscience studies recently reported that forthcoming actions are determined by unconscious mental processes long before we become aware of our intention to act. Libet et al. (1985) showed that conscious decision to press the button was preceded by a negative brain potential for a few hundred milliseconds, the so-called “readiness potential” that originates from the supplementary motor area (SMA).Soon et al. (2008) showed that the preceded brain activity is initiated in prefrontal cortex (PFC) and parietal cortex up to 7 s before the conscious decisions from their fMRI study.These evidences could be interpreted that the decision is initially generated unconsciously in the PFC which is regarded as the top of the hierarchy for action organization, and then it propagates downward to the SMA which is responsible for preparation of immediate motor programs. Despite these progress, we may still have a question how the initial PFC activities for the decisions originate at the first setout. For investigating the possible hypothesis of brain mechanism accounting for the free decision, we conducted model studies accompanied by neuro-robotics experiments in terms of dynamical systems. The model consists of a slow dynamics network in the higher level and the fast dynamics network in the lower level, and is a relatively simple implementation that shares cortical hierarchies like PFC and SMA in the brain. The robot controlled by the network is tutored to imitate experimenter’s free decision iterations of moving an object, which is a modified version of Soon’s task. After the learning, we show that the robot is able to mimic the experimenter’s voluntary movements by means of self-organized deterministic chaos.By observing neural activities in the network that mimics experimenter’s movements, we find that neural units in higher-level constitute the chaotic dynamics and activities of them exhibit long time correlation by which outcome of decisions can be predicted. doi:10.1016/j.neures.2010.07.1465
P2-r17 Human brain image registration for diffusion MRI data and neuro-fiber categorization Kevin C.Y. Chien 1 , Shigeyuki Oba 1,2 , Shin Ishii 1 1
doi:10.1016/j.neures.2010.07.1463
P2-r15 Reinforcement learning using multiple actions Hayato Nakama , Tsubasa Asano, Satoshi Yamada Graduate School of Engineering, Okayama University of Science, Okayama Reinforcement learning (RL) is a collection of methods for discovering nearoptimal solutions to stochastic sequential decision problems. An RL system interacts with the environment by executing actions and receiving rewards from the environment. However, many realistic control problems are too large to be solved practically by searching the state-space using primitive actions. One way to overcome this difficulty is to use “multiple action” in RL. The multiple action consists of a group of primitive actions and the termination conditions. When one of multiple actions is selected, the group of actions is executed until the termination condition is satisfied. In this study, the action space is constructed of multiple actions. Since multiple actions commit the learning agent to act in a particular purposeful way for a sustained period of time, it is expected that they accelerate the reinforcement learning. The task trained in this study is that e-puck robot approaches the target, which has the correct image on the side with a lamp, with avoiding walls and 4 dummy objects(correct image without lamp, wrong image with lamp, wrong image without lamp, and just lamp). Since several kinds of sensor values, distance sensors, optical sensors and camera, are required to achieve the task, it is difficult that the ordinary RL learned the control. The following seven multiple actions are used. GoStraight(), TurnRightToSensD(N), TurnRightToSensO(N), TurnLeftToSensD(N), TurnLeftToSensO(N), SearchTarget(), and ApproachTarget() (SensD, SensO and N denote distance sensor, optical sensor and the number of sensor, respectively). The performance of the RL with multiple actions was compared with that of the hand-coded control using multiple actions in the e-puck simulator. The control trained by the RL
System Science, Kyoto University 2 PRESTO, Japan Science and Technology Agency Diffusion magnetic resonance imaging (MRI) of human brain provides exhaustive information of fiber trajectories that reflect global neural connectivity between brain regions, which is considered as akey of connectome study to unveil detail functions of brain and its difference between individuals. In order to understand the total structural meaning of fiber trajectories, it is important to extract fiber bundles that are common between different individuals, where fiber bundles are groups of neural fibers with similar starting and/or end points that transmit information from one area of brain to another. However, the task of extracting fiber bundles is challenging because of following reasons: (1) fiber bundle is not well defined in mathematical manner, (2) computational problem because of large number of fibers, and (3) artifact alters global shape of brain even between different scans of same individual brain. Methods based on statistical information of fibers to extract fiber bundles have not been able to overcome the difficulties above. In this study, we propose employing Mean Shift algorithm to address problems (1) and (3). A CUDA version of the Mean Shift algorithm is written to illustrate the paradigm of replacing multi-CPU computer with CUDA enabled GPU especially when performing computationally expensive calculations. This attempt will address problem (2). Furthermore, we hypothesize that functional areas of brain are related to fiber bundles traversing and acting as message highways across them. Basing on this hypothesis, we aim to link biological functions of brain areas with its structure across different individuals by registering fiber bundles within brain images. A nonlinear image registration technique is chosen, warping brain images containing fiber bundles to resemble the fiber structure of a reference brain image. Taken altogether, our research will provide a novel analysis tool for neuroanatomy. doi:10.1016/j.neures.2010.07.1466