- Email: [email protected]
Can the synchronization of cortical areas be evidenced by fMRI?
Neurocomputing 26}27 (1999) 1019}1024
Can the synchronization of cortical areas be evidenced by fMRI? F. Frisone *, P. Vitali, G. Ianno` , M. Marongiu , P.G. Morasso , A. Pilot, G. Rodriguez, M. Rosa, F. Sardanelli DIST } Department of Informatics, Systems and Telecommunication, University of Genoa, Via Opera Pia 13, I-16145 Genova, Italy DIMI } Department of Internal Medicine, Service of Clinic Neurophysiology, University of Genoa - S. Martino Hospital, V.le Benedetto XV, 6, 16132 Genova, Italy Department of Medical Physics, S. Martino Hospital, L.go R. Benzi 10, 16132 Genova, Italy Service of Neuroradiology, S. Martino Hospital, L.go R. Benzi 10, 16132 Genova, Italy Radiology Institut, S. Martino Hospital, University of Genoa, L.go R. Benzi 10, 16132 Genova, Italy
Abstract The goal of this study is to investigate the possibility of analyzing spatio-temporal organization of the human cortical activity during di!erent complex tasks, by means of fMRI. To evidence cortical areas synchronization we propose a computational approach based on a self-organizing neural networks (`neural gasa) that detects time-dependent alterations in the regional intensity of the functional signal. Results of the application of such approach are reported and are compared with the results obtained with a standard statistical package (SPM96). Future experimental investigations will be aimed at the analysis of spatio-temporal structures of cortical activity in pathological conditions, such as epilepsy. 1999 Published by Elsevier Science B.V. All rights reserved. Keywords: fMRI; Self-organized neural networks; Multiple cortical areas synchronization
1. Introduction In functional magnetic resonance imaging (fMRI) the key problem of data analysis is to detect the weak (2}5%) BOLD signal component `drowneda in the MR noisy
* Corresponding author. Tel.: #39-10-3532749; fax: #39-10-3532154. E-mail address: [email protected] (F. Frisone) 0925-2312/99/$ } see front matter 1999 Published by Elsevier Science B.V. All rights reserved. PII: S 0 9 2 5 - 2 3 1 2 ( 9 9 ) 0 0 0 9 9 - 5
1020
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
signal. Current approaches (like that using cross-correlation analysis or statistical parametric mapping) reduce total time-varying information in one static map of activation. Moreover, these approaches imply presumptive knowledge of expected stimulus-response pattern, that is absent in spontaneous events like hallucinations, sleep, or epileptic seizures. Although the fMRI temporal resolution is limited by the intrinsic characteristic of the BOLD signal (delayed 2}4 s with respect to neural "ring), it is theoretically possible to evidence peaks of activation with a temporal resolution of the order of 1 s (depending on the required spatial resolution). Self-organizing arti"cial neural networks are a class of unsupervised learning algorithms that "nd their application in all cases the structure of the data we have to analyze is unknown `a prioria. Moreover, they can be successfully employed to estimate the topological invariants of data [1]. Examples of this class of algorithms are self-organizing maps (SOM) [2], vector quantization (VQ) [5], neural gas (NG) [4], etc. The properties of these algorithms in the "eld of biosignal analysis have been investigated in the literature [6]. For our purposes, we chose the NG model, which has been successfully applied to vector quantization and time-series prediction [3]. This model, when applied to the task of vector quantization, reaches a distortion error lower than that resulting from K-means clustering, maximum-entropy clustering and from Kohonen's feature map. Moreover, NG obeys a gradient descent on a wellde"ned energy surface (like the maximum-entropy clustering, in contrast to Kohonen's feature map algorithm). It is also demonstrated [3] that the convergence of the NG algorithm is faster than the convergence of the three other approaches and this is important for practical applications where adaptation steps are `expensivea. 2. Methods The novelty of our study of fMRI analysis by self-organizing neural networks is the production of a time-varying maps of activation. In particular, we analyzed four di!erent experimental paradigms: the classic "nger-tapping task, two language tasks (phonetic #uency and, verbal judgment), and a word recall task. 2.1. fMRI experiments Subjects were healthy young volunteers, who gave an informed consent: a 28 yr old right-handed woman, performed the "nger-tapping and word recall tasks; four subjects (2 male and 2 female, 20}32 yr, 2 right and 2 left-handers) performed the two language tasks. In the "nger-tapping task, the subject was requested to perform sequential "nger opposition (thumb to index, thumb to middle, thumb to ring, thumb to pinkie, thumb to index, and so on), as fast and accurate as possible. In the control phases, he had to count mentally. In the phonetic #uency task, the subject during the activation phases had to think as many words as possible which begin by a letter given by the experimenter. In the control phases, the subject had to continuously repeat mentally
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
1021
the nonsense word `blaa. In the verbal judgment task, the subject had to mentally "nd words according to de"nitions given by the experimenter ("ve each phase). In the control phases, the subject thought `calma when experimenter said `you must staya. In the word recall task, the subject mentally repeated ten words given by the experimenter 10 min before; in the control phases, the subject had to count mentally. Structural T1-weighted and functional EPI images (TE 66, TR 0.96, Flip-Angle 903) were acquired on the same slice position by means of a Siemens 1,5 T scanner. During the "nger-tapping and language tasks, 40 volumes were acquired (5 activation #5 control for 4 cycles). The acquisition time for a whole volume was 2 s and the inter-acquisition time was 2.2 s. In the "nger-tapping task, the volume was composed of 10 slices covering the primary motor cortex (5 mm thickness, matrix 128 * 128). During the language tasks volumes were composed of 18 slices encompassing all brain. During the memory task 40 volumes encompassing all brain (volume matrix 128*128*18) were acquired (6 activation#4 control for 4 cycles), the acquisition time was 2 s and the inter-acquisition time was 8 s. Movement artifacts were compensated by automatic image alignment (SPM96 software). 2.2. Computational framework Let n denote the number of subsequent scans in a fMRI experiment. The dynamics of each pixel p over all scan acquisition time spots can be interpreted as a vector *3RL in the n-dimensional feature space of possible fMRI n-scans time sequences (in other words, for each pixel p, * is the time series of n scan values; n"40, in our case). We computed the amplitude of the fast Fourier transform (FFT) of * and performed a simple data compression by retaining only 18 coe$cients (from the 3rd to 18th one). In this way, we eliminated both the low- and high-frequencies components of *, producing a 18-dimensional vector * 3R" (D"18). The neural model was applied to such vectors, in order to identify image areas with similar FFT coe$cients. In the NG neural network these groups are represented by neuronal weight-vectors or vector prototypes w . The NG algorithm determines the cluster centers by applying an E iterative update rule each time a data vector * is presented *w "e ) h (kwE(* )) ) ""* !w "" g"1,2,N, E H E
(1)
where N is the number of neurons in the network; e3[0,1] is the step size (describes the overall extent of the modi"cation); h (kwE(* )) is a decay function that decreases H monotonically for increasing kwE with a characteristic decay constant j (in the * simulations we chose h (kwE(* ))"e\IwE H) and kwE(* ) is the number of vectors w for H which the relation ""* !w""(""* !w "" holds. The variation of w vectors during E E learning obeys a stochastic gradient descent on the cost function
1 , E(w, j)" d"* P(* )h (kwE(* ))""* !w "", H E 2C(j) E
(2)
with C(j)" ,h (kwE) and P(* ) a probability density function of data points * 3R". H
1022
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
Fig. 1. The features values (at the end of the training) of the weight prototype vector w relative to the neuron that classi"es the functional signal in the "nger-tapping task.
Fig. 2. Activation map in the "nger-tapping task. Top panel: NG approach; bottom panel: SPM96 approach.
We used the "nger-tapping task to train the NG network. We employed 30 neurons and 19 000 samples, obtained by means of smoothing the images with a Gaussian spatial "lter and thresholding the 40 volumes. The training set was presented to the network four times. Among the 30 neurons we found that only one could classify the functional signals. (The other neurons identi"ed anatomical features.) Fig. 1 shows the vector prototype of such neuron and Fig. 2 the activated area, computed according to a minimum distance criterion in the metric of the FFT features space.
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
1023
3. Results and discussion Our approach is able like the standard SPM96 package to identify all major cluster of activation. Furthermore, some cortical and sub-cortical areas of activation are identi"ed by our method and not by SPM96 (see Fig. 2). This is true also in cognitive experiments, where the intensity of activation is smaller compared to motor studies. The power of the neural network approach is the independence from the stimulation paradigm and the ability to detect the functional rate of the fMRI signal without presumptive knowledge of the stimulus-response model: the network trained with the "nger-tapping task is able to detect activation in other tasks. A preliminary study [7] reported a self-organizing neural networks approach able to identify the strong activation of 8 Hz checkerboard visual stimulation: this stimulation (the "rst applied in fMRI experiments) caused the maximum activation of the cortex. Moreover, this study, like the standard SPM96 elaboration, present a static map. Our approach not only highlights the static part of the cortical activation, but also the timing structure not observable with the standard tools. The spatio-temporal cortical activation patterns showed (see Fig. 3) a change of activity level in the areas involved. In other words it can be observed that the `peaks of activationa were shifted in time in the di!erent areas of the same slice. The proposed model provides a stimulus and an explanatory tool for integrating the FMRI technique, which has good spatial resolution but coarse timing resolution, with EEG recordings, which are characterized by complementary properties. Future experimental investigations will be aimed in this direction, i.e. the analysis of the spatio-temporal structure of cortical activity in normal subjects and pathological conditions, such as epilepsy.
Fig. 3. In this left-handers, the activation in right-sided Wernicke's and Broca's areas presents a peak almost 4 s before the left-sided Broca's area. The "ve images in the top panel are acquired during the activation phase; the others "ve (bottom panel) during control phase. Activation areas are white in all slices. The activation intensity is in grey scale.
1024
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
References [1] F. Frisone, F. Firenze, P.G. Morasso, L. Ricciardiello, Application of topology-representing networks to the estimation of the intrinsic dimensionality of data, in: ICANN95 } International Conference on Arti"cial Neural Networks, vol. 1, Paris, October 1995, pp. 323}327. [2] T. Kohonen, The self organizing map, Proc. IEEE 78 (1990) 1464}1480. [3] T. Martinetz, Competitive Hebbian learning rule forms perfectly topology preserving maps, in: S. Gielen, B. Kappen (Eds.), ICANN93 } International Conference on Arti"cal Neural Networks, Amsterdam, 1993. [4] T. Martinetz, K. Schulten, A &neural-gas' network learns topologies, in: T. Kohonen, K. Makisara, O. Simula, J. Kangas (Eds.), Arti"cial Neural Networks, North-Holland, Amsterdam, 1991. [5] K. Rose, E. Gurewitz, G.C. Fox, Vector quantization by deterministic annealing, IEEE Trans. Inform. Theory 38 (4) (1992) 1249}1257. [6] A. WismuK ller, D.R. Dersch, Neural network computation in biomedical research: chances for conceptual cross-fertilization, Theory Biosci. 116 (3) (1997). [7] A. WismuK ller, D.R. Dersch, B. Lipinski, K. Hahn, D. Auer, A neural network approach to functional MRI pattern analysis } clustering of time-series by hierarchical vector quantization, in: L. Niklasson, M. BodeH n, T. Ziemke (Eds.), Perspective in Neural Computing } ICANN98, vol. 2, Springer, 1998, pp. 857}862.
Can the synchronization of cortical areas be evidenced by fMRI? F. Frisone *, P. Vitali, G. Ianno` , M. Marongiu , P.G. Morasso , A. Pilot, G. Rodriguez, M. Rosa, F. Sardanelli DIST } Department of Informatics, Systems and Telecommunication, University of Genoa, Via Opera Pia 13, I-16145 Genova, Italy DIMI } Department of Internal Medicine, Service of Clinic Neurophysiology, University of Genoa - S. Martino Hospital, V.le Benedetto XV, 6, 16132 Genova, Italy Department of Medical Physics, S. Martino Hospital, L.go R. Benzi 10, 16132 Genova, Italy Service of Neuroradiology, S. Martino Hospital, L.go R. Benzi 10, 16132 Genova, Italy Radiology Institut, S. Martino Hospital, University of Genoa, L.go R. Benzi 10, 16132 Genova, Italy
Abstract The goal of this study is to investigate the possibility of analyzing spatio-temporal organization of the human cortical activity during di!erent complex tasks, by means of fMRI. To evidence cortical areas synchronization we propose a computational approach based on a self-organizing neural networks (`neural gasa) that detects time-dependent alterations in the regional intensity of the functional signal. Results of the application of such approach are reported and are compared with the results obtained with a standard statistical package (SPM96). Future experimental investigations will be aimed at the analysis of spatio-temporal structures of cortical activity in pathological conditions, such as epilepsy. 1999 Published by Elsevier Science B.V. All rights reserved. Keywords: fMRI; Self-organized neural networks; Multiple cortical areas synchronization
1. Introduction In functional magnetic resonance imaging (fMRI) the key problem of data analysis is to detect the weak (2}5%) BOLD signal component `drowneda in the MR noisy
* Corresponding author. Tel.: #39-10-3532749; fax: #39-10-3532154. E-mail address: [email protected] (F. Frisone) 0925-2312/99/$ } see front matter 1999 Published by Elsevier Science B.V. All rights reserved. PII: S 0 9 2 5 - 2 3 1 2 ( 9 9 ) 0 0 0 9 9 - 5
1020
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
signal. Current approaches (like that using cross-correlation analysis or statistical parametric mapping) reduce total time-varying information in one static map of activation. Moreover, these approaches imply presumptive knowledge of expected stimulus-response pattern, that is absent in spontaneous events like hallucinations, sleep, or epileptic seizures. Although the fMRI temporal resolution is limited by the intrinsic characteristic of the BOLD signal (delayed 2}4 s with respect to neural "ring), it is theoretically possible to evidence peaks of activation with a temporal resolution of the order of 1 s (depending on the required spatial resolution). Self-organizing arti"cial neural networks are a class of unsupervised learning algorithms that "nd their application in all cases the structure of the data we have to analyze is unknown `a prioria. Moreover, they can be successfully employed to estimate the topological invariants of data [1]. Examples of this class of algorithms are self-organizing maps (SOM) [2], vector quantization (VQ) [5], neural gas (NG) [4], etc. The properties of these algorithms in the "eld of biosignal analysis have been investigated in the literature [6]. For our purposes, we chose the NG model, which has been successfully applied to vector quantization and time-series prediction [3]. This model, when applied to the task of vector quantization, reaches a distortion error lower than that resulting from K-means clustering, maximum-entropy clustering and from Kohonen's feature map. Moreover, NG obeys a gradient descent on a wellde"ned energy surface (like the maximum-entropy clustering, in contrast to Kohonen's feature map algorithm). It is also demonstrated [3] that the convergence of the NG algorithm is faster than the convergence of the three other approaches and this is important for practical applications where adaptation steps are `expensivea. 2. Methods The novelty of our study of fMRI analysis by self-organizing neural networks is the production of a time-varying maps of activation. In particular, we analyzed four di!erent experimental paradigms: the classic "nger-tapping task, two language tasks (phonetic #uency and, verbal judgment), and a word recall task. 2.1. fMRI experiments Subjects were healthy young volunteers, who gave an informed consent: a 28 yr old right-handed woman, performed the "nger-tapping and word recall tasks; four subjects (2 male and 2 female, 20}32 yr, 2 right and 2 left-handers) performed the two language tasks. In the "nger-tapping task, the subject was requested to perform sequential "nger opposition (thumb to index, thumb to middle, thumb to ring, thumb to pinkie, thumb to index, and so on), as fast and accurate as possible. In the control phases, he had to count mentally. In the phonetic #uency task, the subject during the activation phases had to think as many words as possible which begin by a letter given by the experimenter. In the control phases, the subject had to continuously repeat mentally
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
1021
the nonsense word `blaa. In the verbal judgment task, the subject had to mentally "nd words according to de"nitions given by the experimenter ("ve each phase). In the control phases, the subject thought `calma when experimenter said `you must staya. In the word recall task, the subject mentally repeated ten words given by the experimenter 10 min before; in the control phases, the subject had to count mentally. Structural T1-weighted and functional EPI images (TE 66, TR 0.96, Flip-Angle 903) were acquired on the same slice position by means of a Siemens 1,5 T scanner. During the "nger-tapping and language tasks, 40 volumes were acquired (5 activation #5 control for 4 cycles). The acquisition time for a whole volume was 2 s and the inter-acquisition time was 2.2 s. In the "nger-tapping task, the volume was composed of 10 slices covering the primary motor cortex (5 mm thickness, matrix 128 * 128). During the language tasks volumes were composed of 18 slices encompassing all brain. During the memory task 40 volumes encompassing all brain (volume matrix 128*128*18) were acquired (6 activation#4 control for 4 cycles), the acquisition time was 2 s and the inter-acquisition time was 8 s. Movement artifacts were compensated by automatic image alignment (SPM96 software). 2.2. Computational framework Let n denote the number of subsequent scans in a fMRI experiment. The dynamics of each pixel p over all scan acquisition time spots can be interpreted as a vector *3RL in the n-dimensional feature space of possible fMRI n-scans time sequences (in other words, for each pixel p, * is the time series of n scan values; n"40, in our case). We computed the amplitude of the fast Fourier transform (FFT) of * and performed a simple data compression by retaining only 18 coe$cients (from the 3rd to 18th one). In this way, we eliminated both the low- and high-frequencies components of *, producing a 18-dimensional vector * 3R" (D"18). The neural model was applied to such vectors, in order to identify image areas with similar FFT coe$cients. In the NG neural network these groups are represented by neuronal weight-vectors or vector prototypes w . The NG algorithm determines the cluster centers by applying an E iterative update rule each time a data vector * is presented *w "e ) h (kwE(* )) ) ""* !w "" g"1,2,N, E H E
(1)
where N is the number of neurons in the network; e3[0,1] is the step size (describes the overall extent of the modi"cation); h (kwE(* )) is a decay function that decreases H monotonically for increasing kwE with a characteristic decay constant j (in the * simulations we chose h (kwE(* ))"e\IwE H) and kwE(* ) is the number of vectors w for H which the relation ""* !w""(""* !w "" holds. The variation of w vectors during E E learning obeys a stochastic gradient descent on the cost function
1 , E(w, j)" d"* P(* )h (kwE(* ))""* !w "", H E 2C(j) E
(2)
with C(j)" ,h (kwE) and P(* ) a probability density function of data points * 3R". H
1022
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
Fig. 1. The features values (at the end of the training) of the weight prototype vector w relative to the neuron that classi"es the functional signal in the "nger-tapping task.
Fig. 2. Activation map in the "nger-tapping task. Top panel: NG approach; bottom panel: SPM96 approach.
We used the "nger-tapping task to train the NG network. We employed 30 neurons and 19 000 samples, obtained by means of smoothing the images with a Gaussian spatial "lter and thresholding the 40 volumes. The training set was presented to the network four times. Among the 30 neurons we found that only one could classify the functional signals. (The other neurons identi"ed anatomical features.) Fig. 1 shows the vector prototype of such neuron and Fig. 2 the activated area, computed according to a minimum distance criterion in the metric of the FFT features space.
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
1023
3. Results and discussion Our approach is able like the standard SPM96 package to identify all major cluster of activation. Furthermore, some cortical and sub-cortical areas of activation are identi"ed by our method and not by SPM96 (see Fig. 2). This is true also in cognitive experiments, where the intensity of activation is smaller compared to motor studies. The power of the neural network approach is the independence from the stimulation paradigm and the ability to detect the functional rate of the fMRI signal without presumptive knowledge of the stimulus-response model: the network trained with the "nger-tapping task is able to detect activation in other tasks. A preliminary study [7] reported a self-organizing neural networks approach able to identify the strong activation of 8 Hz checkerboard visual stimulation: this stimulation (the "rst applied in fMRI experiments) caused the maximum activation of the cortex. Moreover, this study, like the standard SPM96 elaboration, present a static map. Our approach not only highlights the static part of the cortical activation, but also the timing structure not observable with the standard tools. The spatio-temporal cortical activation patterns showed (see Fig. 3) a change of activity level in the areas involved. In other words it can be observed that the `peaks of activationa were shifted in time in the di!erent areas of the same slice. The proposed model provides a stimulus and an explanatory tool for integrating the FMRI technique, which has good spatial resolution but coarse timing resolution, with EEG recordings, which are characterized by complementary properties. Future experimental investigations will be aimed in this direction, i.e. the analysis of the spatio-temporal structure of cortical activity in normal subjects and pathological conditions, such as epilepsy.
Fig. 3. In this left-handers, the activation in right-sided Wernicke's and Broca's areas presents a peak almost 4 s before the left-sided Broca's area. The "ve images in the top panel are acquired during the activation phase; the others "ve (bottom panel) during control phase. Activation areas are white in all slices. The activation intensity is in grey scale.
1024
F. Frisone et al. / Neurocomputing 26}27 (1999) 1019}1024
References [1] F. Frisone, F. Firenze, P.G. Morasso, L. Ricciardiello, Application of topology-representing networks to the estimation of the intrinsic dimensionality of data, in: ICANN95 } International Conference on Arti"cial Neural Networks, vol. 1, Paris, October 1995, pp. 323}327. [2] T. Kohonen, The self organizing map, Proc. IEEE 78 (1990) 1464}1480. [3] T. Martinetz, Competitive Hebbian learning rule forms perfectly topology preserving maps, in: S. Gielen, B. Kappen (Eds.), ICANN93 } International Conference on Arti"cal Neural Networks, Amsterdam, 1993. [4] T. Martinetz, K. Schulten, A &neural-gas' network learns topologies, in: T. Kohonen, K. Makisara, O. Simula, J. Kangas (Eds.), Arti"cial Neural Networks, North-Holland, Amsterdam, 1991. [5] K. Rose, E. Gurewitz, G.C. Fox, Vector quantization by deterministic annealing, IEEE Trans. Inform. Theory 38 (4) (1992) 1249}1257. [6] A. WismuK ller, D.R. Dersch, Neural network computation in biomedical research: chances for conceptual cross-fertilization, Theory Biosci. 116 (3) (1997). [7] A. WismuK ller, D.R. Dersch, B. Lipinski, K. Hahn, D. Auer, A neural network approach to functional MRI pattern analysis } clustering of time-series by hierarchical vector quantization, in: L. Niklasson, M. BodeH n, T. Ziemke (Eds.), Perspective in Neural Computing } ICANN98, vol. 2, Springer, 1998, pp. 857}862.