- Email: [email protected]
Sulcus Identification and Labeling
Sulcus Identification and Labeling J-F Mangin, M Perrot, and G Operto, CEA, Gif-sur-Yvette, France; CATI Multicenter Neuroimaging Platform, Paris, France A Cachia, Universite´ Paris Descartes, Paris, France C Fischer, CEA, Gif-sur-Yvette, France; CATI Multicenter Neuroimaging Platform, Paris, France J Lefe`vre, Aix-Marseille Universite´, Marseille, France D Rivie`re, CEA, Gif-sur-Yvette, France; CATI Multicenter Neuroimaging Platform, Paris, France ã 2015 Elsevier Inc. All rights reserved.
Before the advent of MRI, the cortical folding pattern of patients was out of sight, out of mind. The sulcus nomenclature, which was taught in the anatomy classes, had no real use except for neuroanatomists and to some extent neurosurgeons. Few people were aware of the considerable interindividual variability of the folding pattern and of the complete lack of understanding of the origin of this variability. The fantastic development of MRI could have triggered a renewal of interest, but the actual scientific drive was functional imaging, which requires morphological variability to be removed to simplify statistical group analysis. Hence, for the brain mapping community, the variability of the folding pattern is mainly an impediment to perfect spatial normalization. Today, spatial normalization technologies have reached a very efficient stage where primary sulci, which are good landmarks of primary architectural areas, are reasonably aligned across subjects (Ashburner, 2007; Fischl et al., 2008). It is tempting to consider that the variability of the folding pattern is an epiphenomenon of low interest that can be forgotten. However, the normalization paradigm has provided the opportunity to quantify the morphological variability, which has raised awareness that this variability can be a very valuable source of information. For instance, techniques like voxelbased morphometry or cortical thickness analysis have provided a myriad of insights about the impact of development, pathologies, or cognitive skills on brain structures (Toga & Thompson, 2003). Hence, the variability of the folding pattern has received more and more attention. A wave of research programs aiming at explaining the origin and the dynamics of the folding process was triggered (Lefe`vre & Mangin, 2010; Regis et al., 2005; Reillo, de Juan Romero, Garcı´a-Cabezas, & Borrell, 2011; Sun & Hevner, 2014; Taber, 2014; Toro & Burnod, 2005; Van Essen, 1997). These research programs propose various hypotheses leading to consider the folding pattern as a proxy of the underlying architecture of the cerebral cortex. Then, the sulci geometry could provide biomarkers of abnormal development or the signature of specific architectures (Cachia et al., 2014; Dubois et al., 2008; Plaze et al., 2011; Sun et al., 2012; Weiner et al., 2014).
The Need for Computational Anatomy The quantification of the folding pattern is a difficult issue. The simplest strategies rely on global or local gyrification indexes measuring the amount of cortex buried into the folds (Schaer et al., 2008; Toro et al., 2008; Zilles, Palomero-Gallagher, & Amunts, 2013). A spectral approach can provide a richer insight
Brain Mapping: An Encyclopedic Reference
of the folding global features (Germanaud et al., 2012). Following the footsteps of the neuroanatomists is more ambitious because it requires the identification of the sulci described in the literature. Very few experts can perform this difficult and tedious task for the complete cerebral cortex. With the usual radiological point of view, namely, a series of slices, even the largest sulci can be difficult to recognize, which explains the scarcity of relevant knowledge to overcome the interindividual variability (see Figure 1). 3-D rendering of the cortical surface is of great help, but it is often insufficient to deal with unusual folding patterns that require inspecting the shape of the fold depths (Regis et al., 2005). Hence, combining 3-D and 2-D views is often mandatory. Cortical inflation is an attractive alternative to exhibit the buried cortex (Van Essen, Drury, Joshi, & Miller, 1998), but the deformations from the actual folding geometry can be disturbing for sulcus identification. Whatever the visualization strategy, our ignorance with respect to the origin of the variability prevents us to decode safely configurations where the main sulci are split into pieces and reorganized into unusual patterns. A dedicated atlas describing the most usual patterns is probably the best guideline for sulcus identification, but this atlas is not comprehensive because it stems from the study of only 25 brains (Ono, Kubik, & Abarnathey, 1990). The reliable identification of secondary and tertiary folds (Petrides, 2012) is beyond reach with the current state of knowledge. The complexity of the cortical folding pattern is overwhelming for human experts. Hence, computational anatomy is now a key player to help the field to harness the folding variability. Alignment with a single-subject cortical surface atlas has the merit to provide the rough localization of primary sulci (Destrieux, Fischl, Dale, & Halgren, 2010), but is not sufficient to obtain accurate definition of the sulci for shape analysis. The idiosyncrasies of the template brain are not a good model for any other brain. The solution could lie into multisubject atlases (Heckemann, Hajnal, Aljabar, Rueckert, & Hammers, 2006), but this approach may not be flexible enough to overcome the ambiguities hampering a reliable pairwise alignment between cortical patterns. Hence, the community developed alternatives with a stronger computer vision flavor. First, bottom-up processing pipelines convert standard MR images into synthetic representations of individual folding geometries; and second, pattern recognition techniques match such representations with a model of the sulci (see Figure 2). Automatic recognition of the sulci provides a range of opportunities for morphometry (see Figure 3; Mangin, Jouvent, & Cachia, 2010) or sulcusdriven spatial normalization (Auzias et al., 2011, 2013; Joshi et al., 2010; Thompson & Toga, 1996).
http://dx.doi.org/10.1016/B978-0-12-397025-1.00307-9
365
Figure 1 Sulci are very difficult to recognize in a stack of brain slices.
S.Pe.C.median. F.C.L.r.diag. F.C.L.r.asc. F.C.L.r.ant.
‘An artificial neuroanatomis’
S.Pe.C.marginal. S.Pe.C.sup. S.Pe.C.inter. S.Pe.C.inf.
S.C. F.C.L.r.retroC.tr. F.I.P.Po.C.inf. S.Po.C.sup. S.Pa.sup. S.GSM.
S.F.sup.
F.I.P.r.int.1 S.T.s.ter.asc.ant F.I.P.r.int.2
S.F.inter.
3-D Retina
Elementary folds
S.F.inf.
F.I.P. S.T.s.ter.asc.post.
S.F.inf.ant. S.F.marginal.
Occipital
S.F.orbitaire. S.Or. F.C.L.a. F.C.L.p. Insula S.T.pol.
S.T.i.post. S.O.T.lat.post. S.O.T.lat.med. F.C.L.r.sc.post. S.T.i.ant. S.C.sylvian. S.T.s. F.C.L.r.sc.ant.
Sulcus model (a)
(d)
(e)
(b)
Bottom-up processing
Pattern recognition
Sulcus recognition (c)
(f)
Figure 2 A computer vision pipeline mimicking a human anatomist. Its 3-D retina is the standard space of the brain mapping community. After detection of the building blocks of the folding pattern from a negative mold of the brain, the sulci of the standard nomenclature are reassembled according to a model inferred from a learning database. Reproduced from Perrot, M., Rivie`re, D., and Mangin, J. F. (2011). Cortical sulci recognition and spatial normalization. Medical Image Analysis, 15(4), 529–550.
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
367
Figure 3 Once identified, a sulcus provides various morphometric features like length (red), depth (yellow), surface area (blue), or average span between its walls (right).
Bottom-Up Representations of the Folding Pattern This section describes examples of bottom-up strategies. Each approach aims at converting the implicit encoding of the cortical folding pattern embedded in the geometry of the cortical surface into a synthetic graphical representation. Hence, the pattern recognition system dedicated to sulcus recognition can deal with more abstract representations than images. Abstract representations open the door to machine learning strategies beyond reach for image registration. The first generation of techniques relied on voxel-based representations of the cortex. After a pipeline leading to a classification of gray matter and white matter, each elementary fold is detected and represented by its median surface (see Figure 2(d)). These pieces of surface stem, for instance, from the 3-D skeletonization of a negative mold of white matter (see Figure 2(c); Mangin et al., 1995; Mangin, Frouin, Bloch, Re´gis, & Lo´pez-Krahe, 1995). This skeleton is a set of connected surfaces generated from iterative homotopic erosions scalping the mold while preserving its initial topology. This process can be combined with a watershed algorithm using MR intensities as altitude in order to impose the localization of the skeleton in the ‘crevasse’ corresponding to the cerebrospinal fluid filling up the folds (Mangin et al., 2004). The skeleton is finally split into elementary pieces at the level of the junctions between folds or in case of local depth minima along the fold bottom indicating the fusion between several primal folding entities. A sibling approach restricts the skeletonization to the bounding hull of the cortex (Le Goualher et al., 1999). The resulting 2-D skeleton is split at junction points in order to define the superior trace of each fold. Note that the few folds that do not reach the bounding hull are missed by this process. Then, the initial curve grows until the bottom of the fold in order to yield a 2-D parameterized surface, which is the strength of the approach (Vaillant & Davatzikos, 1997). Finally, these different methods represent the topography of the folding pattern as
a graph: The nodes are the elementary folds and the links represent junctions or the fact that two parallel folds build up a gyrus. Some approaches put the focus on the deepest part of the 3-D folding geometry. For this purpose, the representation can rely on the bottom lines of the folds defined from the 3-D skeleton using simple topological considerations (Lohmann, 1998; Mangin, Re´gis, et al., 1995; Mangin, Frouin, Bloch, Re´gis, & Lo´pez-Krahe, 1995). When the key focus is not even the bottom lines but the deepest points of the folding, an alternative to the skeleton-based strategy lies in depth-based processing. For instance, a watershed-based algorithm using depth as altitude can be used to split the negative mold of whiter matter in the so-called sulcal basins associated with depth local maxima (Lohmann & von Cramon, 2000). The main difficulty is the pruning of the basins in order to keep only the morphologically meaningful ones. The end of the nineties coincided with a shift from volumetric processing to surface-based processing, thanks to the maturity of the pipelines generating spherical cortical surfaces (Dale, Fischl, & Sereno, 1999). This transition has increased the trend to focus on the depth of the folding, because the embedding of the cortical surface into the 3-D space is less accessible once the cortex is represented as a 2-D mesh. When dealing with cortical surface meshes, the bottom lines of the sulci are detected using variants of surface-based skeletonization algorithms acting on buried regions defined from depth or curvature (Kao et al., 2007; Li, Guo, Nie, & Liu, 2010; Seong et al., 2010; Shi, Thompson, Dinov, & Toga, 2008). These methods differ mainly not only in the way they combine depth and curvature to achieve reliable localization of the bottom lines but also in their pruning strategy to get rid of spurious detections. Semiautomatic approaches have been proposed to delineate the optimal sulcus bottom line from manually selected extremities (Le Troter, Auzias, & Coulon, 2012; Shattuck et al., 2009). A strong advantage of the
368
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
surface-based strategies is their sensitivity to dimples, namely, primal sketches of the folding visible in depth or curvature maps. The dimples cannot be detected by 3-D skeletonization algorithms because they are not completely folded (see Figure 4(b)), which is problematic for studying the developing brain (Dubois, Benders, Borradori-Tolsa, et al., 2008; Dubois, Benders, Cachia, et al., 2008) or tertiary sulci. While most of the bottom-up strategies aim at designing methods performing automatic recognition of the sulci of the standard nomenclature, a few approaches are dedicated to research programs questioning the current models of the sulcal anatomy. These approaches aim at inferring a new model of the folding pattern overcoming the ambiguities raised by the interindividual variability (see Figure 4). They mainly target local maxima of cortical depth, which are supposed to result from gyri buried into the folds (see Figure 5). The variable depth of these buried gyri partly explains interindividual variability (Regis et al., 2005). Thanks to the spherical topology of the cortical surface, the analogy with geology leads to simple adaptations of the watershed notion to segregate the surface into depth-based patches, the sulcal basins, which mimic catchment basins, namely, geographic areas with a common outlet for the surface runoff (Rettmann, Han, Xu, & Prince, 2002; Yang & Kruggel, 2008). The most recent approaches do not even pay attention to the spatial extent of the catchment basins but deal with simplistic representations based on the deepest points called sulcal pits (Im et al., 2010; Lohmann, von Cramon, & Colchester, 2008; Meng, Li, Lin, Gilmore, & Shen, 2014; Operto et al., 2012). In order to enrich the representations and to postpone the pruning to the sulcus recognition or model inference stage, a hierarchical strategy inspired by the field of scale space has
(a)
been proposed (Cachia et al., 2001). In the same spirit, graph-based approaches merging surface-based, line-based, and point-based representations are in gestation (Bao, Giard, Tourville, & Klein, 2012). Finally, in the context of developmental studies, an original approach dedicated to longitudinal data has been designed to focus on the seeds of the folding process, namely, the points of the cortical surface corresponding to local maxima of the folding rate (Lefe`vre et al., 2009).
Sulcus Recognition Most of the graphic representations yielded by the pipelines of the previous section include oversegmentation of the sulci of the nomenclature. Indeed, the geometry of a sulcus often includes subdivisions related to branches or interruptions. Then, the sulcus recognition requires a reconstruction process from the set of building blocks listed in the representation, which amounts to a labeling with the sulcus names of the nomenclature (see Figures 2 and 4). The number of names involved in the labeling ranges from 10 to 65 in each hemisphere, according to the richness of the sulcus model. The labeling can be viewed as a many-to-one matching between the representation built for a given subject and the model of the sulci. The model of the sulci can be simply a learning data set of individual representations labeled by a human expert (Lyu et al., 2010) or a probabilistic representation of the sulcus variability inferred from this learning data set: maps of the spatial variability of each sulcus after optimal alignment across the data set (Perrot, Rivie`re, & Mangin, 2011) or different kinds of random graph modeling the joint variability of pairs of sulci (Mangin, Re´gis, et al., 1995; Mangin, Frouin, Bloch, Re´gis, &
1
(b)
(c)
Figure 4 Standard sulcus nomenclature. (a) Three frontal lobes; (b) the skeleton of the negative brain mold used to detect folds; (1) undetected dimples; (c) labeling of the folds with the standard sulcus nomenclature (red, central; yellow, precentral; green, superior frontal; cyan, intermediate frontal; violet, inferior frontal; blue, sylvian valley, etc.).
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
369
2
(a)
(b)
2
(c)
Figure 5 Toward an alphabet of the folding pattern. (a) White matter of the Figure 4 frontal lobes; (2) buried gyri also called ‘plis de passage’; (b, c) labeling of the folds with the sulcal roots nomenclature corresponding to putative indivisible entities. Reproduced from Regis, J., Mangin, J., Ochiai, T., Frouin, V., Riviere, D., Cachia, A., Tamura, M., & Samson, Y. (2005). "Sulcal root" generic model: A hypothesis to overcome the variability of the human cortex folding patterns. Neurologia Medico-Chirurgica, 45(1), 1–17.
Lo´pez-Krahe, 1995; Riviere et al., 2002; Shi et al., 2009; Yang & Kruggel, 2009). The labeling of the building blocks is driven by an optimization process often casted into a Bayesian framework aiming at maximizing the similarity between the sulci defined by the labeling and their model. With this regard, the multiatlas strategy of Lyu et al. has a specific status since the model is the set of bottom lines of several instances of each sulcus efficiently matched with the candidate sulci using a spectral method. For the other approaches, the global similarity measure to be optimized is a sum of local similarity measures. These local measures can stem from local registrations between a fold and probabilistic maps of the sulci (Perrot et al., 2011), the output of a multilayer perceptron fed with features describing the shape of sulci or pairs of sulci and trained on the learning database (Riviere et al., 2002), and more standard potential functions acting on shape features like localization, orientation, length, moments, or wavelet coefficients (Mangin, Re´gis, et al., 1995; Mangin, Frouin, Bloch, Re´gis, & Lo´pezKrahe, 1995; Shi, Tu, et al., 2009; Yang & Kruggel, 2009). For all these methods, dealing with graphic representations rather than images allows the optimization process to rely on sophisticated schemes like simulated annealing (Riviere et al., 2002), genetic algorithms (Yang & Kruggel, 2009), or belief propagation (Shi, Tu, et al., 2009b). The lack of gold standard and the fact that only two of these methods using the same bottom-up pipeline are distributed to the community prevent simple comparisons (http://brainvisa. info) (Perrot et al., 2011; Riviere et al., 2002). The leave-oneout validation versus manual labeling of the method of Perrot et al. has shown an 86% mean recognition rate across the 65
sulci provided in each hemisphere. The performance varies from 95% for the largest primary sulci to 70% for the most variable secondary sulci. Users of this method interested in morphometry studies report the processing of more than 10 000 subjects. Note that the main risk of the bottom-up strategies is undersegmentation, namely, situations where a building block includes the frontier between two sulci. Overcoming this situation probably requires a top-down complementary strategy that is not explicitly embedded in current methods, except in the multiatlas strategy that allows refinement of the individual representations by analogy with the closest atlas (Lyu et al., 2010). Note also that the strategy of Shi et al. aims at defining sulci as continuous curves, to mimic manual tracing, which leads the method to fill up the gaps existing in bottom-up representations (Shi, Tu, et al., 2009b). The methods dealing with point-based representations shall be protected from oversegmentation. In reality, they are prone to a sibling issue: the sulcal pits or sulcal basin centroids of the model are not detected in all brains. Nevertheless, dealing with points is more comfortable than dealing with sulci because the many-toone matching problem becomes a one-to-one matching problem, resulting in simpler formulations (Im et al., 2011; Lohmann & von Cramon, 2000).
Inference of New Models of the Folding Pattern? The success of the multiatlas strategy in neuroimaging (Heckemann et al., 2006) could lead to consider that the future of sulcus recognition is in pattern matching methods informed by a very large data set of manually labeled sulci, in the spirit of
370
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
the approach of Lyu et al. (2010). This data set would play the same role as the huge pools of translated documents of institutions like the European Union for automatic language translation. Indeed, today’s most advanced translation services rely on statistical pattern matching rather than on teaching the rules of human language to the computer. To deal with the lack of gold standard, the learning dataset could stem from a consensus-based labeling effort of the community of neuroanatomists in order to generate the body of information required for the machine to do the job correctly whatever the brain idiosyncrasies. But with our current understanding of the variability, would the consensual labeling of unusual patterns be really reliable? Unlike the field of language translation, mimicking human behavior is not necessarily the best strategy for the computer. It should be noted that the sulcal pits model used to label individual sulcal pits does not stem from the anatomical literature but from the field of computational anatomy. The pits in the model are clusters of individual pits detected after nonlinear alignment of a large set of cortical surfaces (Im et al., 2010). The sulcal pits model is in good agreement with the sulcal roots model proposed by a human anatomist as an alphabet of putative indivisible atomic entities supposed to be stable across individuals because of a developmental origin (see Figure 5; Regis et al., 2005). The sulcal pits model provides a good foretaste of the potential of computational anatomy for paving the way toward new more objective models of the folding patterns. Advances in the understanding of the folding dynamics could also largely contribute to this research program. For instance, the intrinsic geometry of the cortical surface provided by the first eigenvectors of the Laplace–Beltrami operator may be intimately associated with the folding process. The associated coordinate system could be the ideal alignment between subjects before model inference or sulcus recognition (Shi, Dinov, & Toga, 2009; Shi, Sun, Lai, Dinov, & Toga, 2010; Shi, Tu, et al., 2009). The convoluted shape of the cerebral cortex is a challenge for human binocular vision. In return, computer vision systems can be endowed with a dedicated architecture including 3-D retina and 3-D higher-level vision areas. Furthermore, this vision architecture dedicated to the cortical surface can be duplicated without limit, which overcomes the working memory overload disturbing human experts trying to model the folding pattern variability. Manifold learning technology applied on massive databases in the spirit of Ono et al. could help us to segregate the different folding patterns existing in the population, in order to trigger a research program aiming at matching these patterns according to architectural clues provided by other imaging modalities (Ono et al., 1990; Sun, Perrot, Tucholka, Riviere, & Mangin, 2009). Therefore, computational anatomy should be the perfect assistant to support the neuroanatomists in their quest for a better model of the variability.
Acknowledgments This work was supported by the European FET Flagship project ‘Human Brain Project’ (SP2), the French Agence Nationale de la Recherche (ANR-09-BLAN-0038-01 ‘BrainMorph,’ ANR-14
‘APEX,’ and ANR-12JS03-001-01 ‘MoDeGy’), and the French ‘Plan Alzheimer’ Foundation (CATI multicenter imaging platform).
See also: INTRODUCTION TO ANATOMY AND PHYSIOLOGY: Development of the Basal Ganglia and the Basal Forebrain; Embryonic and Fetal Development of the Human Cerebral Cortex; Fetal and Postnatal Development of the Cortex: MRI and Genetics; Gyrification in the Human Brain; Sulci as Landmarks; INTRODUCTION TO METHODS AND MODELING: Automatic Labeling of the Human Cerebral Cortex.
References Ashburner, J. (2007). A fast diffeomorphic image registration algorithm. NeuroImage, 38(1), 95–113. Auzias, G., Colliot, O., Glaune`s, J. A., Perrot, M., Mangin, J. F., Trouve´, A., et al. (2011). Diffeomorphic brain registration under exhaustive sulcal constraints. IEEE Transactions on Medical Imaging, 30(6), 1214–1227. Auzias, G., Lefe`vre, J., Le Troter, A., Fischer, C., Perrot, M., Re´gis, J., et al. (2013). Model-driven harmonic parameterization of the cortical surface: HIP-HOP. IEEE Transactions on Medical Imaging, 32(5), 873–887. Bao, F., Giard, J., Tourville, J., & Klein, A. (2012). Automated extraction of nested sulcus features from human brain MRI data. In IEEE Engineering in Medicine and Biology Society (pp. 4429–4433). Cachia, A., Borst, G., Vidal, J., Fischer, C., Pineau, A., Mangin, J. F., et al. (2014). The shape of the ACC contributes to cognitive control efficiency in preschoolers. Journal of Cognitive Neuroscience, 26(1), 96–106. Cachia, A., Mangin, J., Rivie`re, D., Boddaert, N., Andrade, A., Kherif, F., et al. (2001). A mean curvature based primal sketch to study the cortical folding process from antenatal to adult brain. In Medical image computing and computer-assisted intervention–MICCAI 2001 (pp. 897–904). Dale, A. M., Fischl, B., & Sereno, M. I. (1999). Cortical surface-based analysis. I. Segmentation and surface reconstruction. NeuroImage, 9(2), 179–194. Destrieux, C., Fischl, B., Dale, A., & Halgren, E. (2010). Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. NeuroImage, 53(1), 1–15. Dubois, J., Benders, M., Borradori-Tolsa, C., Cachia, A., Lazeyras, F., Leuchter, R. H.-V., et al. (2008). Primary cortical folding in the human newborn: An early marker of later functional development. Brain, 131(8), 2028–2041. Dubois, J., Benders, M., Cachia, A., Lazeyras, F., Leuchter, R. H.-V., Sizonenko, S. V., et al. (2008). Mapping the early cortical folding process in the preterm newborn brain. Cerebral Cortex, 18(6), 1444–1454. Fischl, B., Rajendran, N., Busa, E., Augustinack, J., Hinds, O., Yeo, B. T., et al. (2008). Cortical folding patterns and predicting cytoarchitecture. Cerebral Cortex, 18(8), 1973–1980. Germanaud, D., Lefe`vre, J., Toro, R., Fischer, C., Dubois, J., Hertz-Pannier, L., et al. (2012). Larger is twistier: Spectral analysis of gyrification (SPANGY) applied to adult brain size polymorphism. NeuroImage, 63(3), 1257–1272. Heckemann, R. A., Hajnal, J. V., Aljabar, P., Rueckert, D., & Hammers, A. (2006). Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. NeuroImage, 33(1), 115–126. Im, K., Choi, Y. Y., Yang, J. J., Lee, K. H., Kim, S. I., Grant, P. E., et al. (2011). The relationship between the presence of sulcal pits and intelligence in human brains. NeuroImage, 55(4), 1490–1496. Im, K., Jo, H. J., Mangin, J.-F., Evans, A. C., Kim, S. I., & Lee, J.-M. (2010). Spatial distribution of deep sulcal landmarks and hemispherical asymmetry on the cortical surface. Cerebral Cortex, 20(3), 602–611. Joshi, A. A., Pantazis, D., Li, Q., Damasio, H., Shattuck, D. W., Toga, A. W., et al. (2010). Sulcal set optimization for cortical surface registration. NeuroImage, 50(3), 950–959. Kao, C. Y., Hofer, M., Sapiro, G., Stem, J., Rehm, K., & Rottenberg, D. A. (2007). A geometric method for automatic extraction of sulcal fundi. IEEE Transactions on Medical Imaging, 26(4), 530–540. Le Goualher, G., Procyk, E., Collins, D., Venugopal, R., Barillot, C., & Evans, A. (1999). Automated extraction and variability analysis of sulcal neuroanatomy. IEEE Transactions on Medical Imaging, 18(3), 206–217. Le Troter, A., Auzias, G., & Coulon, O. (2012). Automatic sulcal line extraction on cortical surfaces using geodesic path density maps. NeuroImage, 61(4), 941–949.
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
Lefe`vre, J., Leroy, F., Khan, S., Dubois, J., Huppi, P., Baillet, S., et al. (2009). Identification of growth seeds in the neonate brain through surfacic Helmholtz decomposition. Information Processing in Medical Imaging, 21, 252–263. Lefe`vre, J., & Mangin, J.-F. (2010). A reaction–diffusion model of human brain development. PLoS Computational Biology, 6(4), e1000749. Li, G., Guo, L., Nie, J., & Liu, T. (2010). An automated pipeline for cortical sulcal fundi extraction. Medical Image Analysis, 14(3), 343–359. Lohmann, G. (1998). Extracting line representations of sulcal and gyral patterns in MR images of the human brain. IEEE Transactions on Medical Imaging, 17(6), 1040–1048. Lohmann, G., & von Cramon, D. Y. (2000). Automatic labelling of the human cortical surface using sulcal basins. Medical Image Analysis, 4(3), 179–188. Lohmann, G., von Cramon, D. Y., & Colchester, A. C. (2008). Deep sulcal landmarks provide an organizing framework for human cortical folding. Cerebral Cortex, 18(6), 1415–1420. Lyu, I., Seong, J. K., Shin, S. Y., Im, K., Roh, J. H., Kim, M. J., et al. (2010). Spectralbased automatic labeling and refining of human cortical sulcal curves using expertprovided examples. NeuroImage, 52(1), 142–157. Mangin, J.-F., Frouin, V., Bloch, I., Re´gis, J., & Lo´pez-Krahe, J. (1995). From 3D magnetic resonance images to structural representations of the cortex topography using topology preserving deformations. Journal of Mathematical Imaging and Vision, 5(4), 297–318. Mangin, J. F., Jouvent, E., & Cachia, A. (2010). In-vivo measurement of cortical morphology: Means and meanings. Current Opinion in Neurology, 23(4), 359–367. Mangin, J., Re´gis, J., Bloch, I., Frouin, V., Samson, Y., & Lo´pez-Krahe, J. (1995). A MRF based random graph modelling the human cortical topography. In Computer vision, virtual reality and robotics in medicine (pp. 177–183). Mangin, J. F., Riviere, D., Cachia, A., Duchesnay, E., Cointepas, Y., Papadopoulos-Orfanos, D., et al. (2004). Object-based morphometry of the cerebral cortex. IEEE Transactions on Medical Imaging, 23(8), 968–982. Meng, Y., Li, G., Lin, W., Gilmore, J. H., & Shen, D. (2014). Spatial distribution and longitudinal development of deep cortical sulcal landmarks in infants. NeuroImage, 100, 206–218. Ono, M., Kubik, S., & Abarnathey, C. D. (1990). Atlas of the cerebral sulci. New York: Georg Thieme. Operto, G., Auzias, G., Le Troter, A., Perrot, M., Riviere, D., Dubois, J., et al. (2012). Structural group analysis of cortical curvature and depth patterns in the developing brain. In IEEE ISBI (pp. 422–425). Perrot, M., Rivie`re, D., & Mangin, J. F. (2011). Cortical sulci recognition and spatial normalization. Medical Image Analysis, 15(4), 529–550. Petrides, M. (2012). The human cerebral cortex: An MRI atlas of the sulci and gyri in MNI stereotaxic space. Amsterdam: Academic Press. Plaze, M., Paille`re-Martinot, M. L., Penttila¨, J., Januel, D., de Beaurepaire, R., Bellivier, F., et al. (2011). "Where do auditory hallucinations come from?" – A brain morphometry study of schizophrenia patients with inner or outer space hallucinations. Schizophrenia Bulletin, 37(1), 212–221. Regis, J., Mangin, J., Ochiai, T., Frouin, V., Riviere, D., Cachia, A., et al. (2005). "Sulcal root" generic model: A hypothesis to overcome the variability of the human cortex folding patterns. Neurologia Medico-Chirurgica, 45(1), 1–17. Reillo, I., de Juan Romero, C., Garcı´a-Cabezas, M., & Borrell, V. (2011). A role for intermediate radial glia in the tangential expansion of the mammalian cerebral cortex. Cerebral Cortex, 21(7), 1674–1694. Rettmann, M. E., Han, X., Xu, C., & Prince, J. L. (2002). Automated sulcal segmentation using watersheds on the cortical surface. NeuroImage, 15(2), 329–344. Riviere, D., Mangin, J., Papadopoulos-Orfanos, D., Martinez, J., Frouin, V., & Regis, J. (2002). Automatic recognition of cortical sulci of the human brain using a congregation of neural networks. Medical Image Analysis, 6(2), 77–92.
371
Schaer, M., Cuadra, M. B., Tamarit, L., Lazeyras, F., Eliez, S., & Thiran, J. P. (2008). A surface-based approach to quantify local cortical gyrification. IEEE Transactions on Medical Imaging, 27(2), 161–170. Seong, J. K., Im, K., Yoo, S. W., Seo, S. W., Na, D. L., & Lee, J. M. (2010). Automatic extraction of sulcal lines on cortical surfaces based on anisotropic geodesic distance. NeuroImage, 49(1), 293–302. Shattuck, D. W., Joshi, A. A., Pantazis, D., Kan, E., Dutton, R. A., Sowell, E. R., et al. (2009). Semi-automated method for delineation of landmarks on models of the cerebral cortex. Journal of Neuroscience Methods, 178(2), 385–392. Shi, Y., Dinov, I., & Toga, A. W. (2009). Cortical shape analysis in the Laplace-Beltrami feature space. Medical Image Computing and Computer-Assisted Intervention, 12(Pt 2), 208–215. Shi, Y., Sun, B., Lai, R., Dinov, I., & Toga, A. W. (2010). Automated sulci identification via intrinsic modeling of cortical anatomy. Medical Image Computing and Computer-Assisted Intervention, 13(Pt 3), 49–56. Shi, Y., Thompson, P. M., Dinov, I., & Toga, A. W. (2008). Hamilton-Jacobi skeleton on cortical surfaces. IEEE Transactions on Medical Imaging, 27(5), 664–673. Shi, Y., Tu, Z., Reiss, A. L., Dutton, R. A., Lee, A. D., Galaburda, A. M., et al. (2009). Joint sulcal detection on cortical surfaces with graphical models and boosted priors. IEEE Transactions on Medical Imaging, 28(3), 361–373. Sun, T., & Hevner, R. F. (2014). Growth and folding of the mammalian cerebral cortex: From molecules to malformations. Nature Reviews Neuroscience, 15(4), 217–232. Sun, Z. Y., Klo¨ppel, S., Rivie`re, D., Perrot, M., Frackowiak, R., Siebner, H., et al. (2012). The effect of handedness on the shape of the central sulcus. NeuroImage, 60(1), 332–339. Sun, Z., Perrot, M., Tucholka, A., Riviere, D., & Mangin, J. (2009). Constructing a dictionary of human brain folding patterns. In: MICCAI Proceedings, Berlin, Hiedelberg: Springer Verlag, LNCS-5762. Taber, L. A. (2014). Morphomechanics: Transforming tubes into organs. Current Opinion in Genetics and Development, 27C, 7–13. Thompson, P., & Toga, A. W. (1996). A surface-based technique for warping threedimensional images of the brain. IEEE Transactions on Medical Imaging, 15(4), 402–417. Toga, A. W., & Thompson, P. M. (2003). Temporal dynamics of brain anatomy. Annual Review of Biomedical Engineering, 5, 119–145. Toro, R., & Burnod, Y. (2005). A morphogenetic model for the development of cortical convolutions. Cerebral Cortex, 15(12), 1900–1913. Toro, R., Perron, M., Pike, B., Richer, L., Veillette, S., Pausova, Z., et al. (2008). Brain size and folding of the human cerebral cortex. Cerebral Cortex, 18(10), 2352–2357. Vaillant, M., & Davatzikos, C. (1997). Finding parametric representations of the cortical sulci using an active contour model. Medical Image Analysis, 1(4), 295–315. Van Essen, D. C. (1997). A tension-based theory of morphogenesis and compact wiring in the central nervous system. Nature, 385(6614), 313–318. Van Essen, D. C., Drury, H. A., Joshi, S., & Miller, M. I. (1998). Functional and structural mapping of human cerebral cortex: Solutions are in the surfaces. Proceedings of the National Academy of Sciences of the United States of America, 95(3), 788–795. Weiner, K. S., Golarai, G., Caspers, J., Chuapoco, M. R., Mohlberg, H., Zilles, K., et al. (2014). The mid-fusiform sulcus: A landmark identifying both cytoarchitectonic and functional divisions of human ventral temporal cortex. NeuroImage, 84, 453–465. Yang, F., & Kruggel, F. (2008). Automatic segmentation of human brain sulci. Medical Image Analysis, 12(4), 442–451. Yang, F., & Kruggel, F. (2009). A graph matching approach for labeling brain sulci using location, orientation, and shape. Neurocomputing, 73, 179–190. Zilles, K., Palomero-Gallagher, N., & Amunts, K. (2013). Development of cortical folding during evolution and ontogeny. Trends in Neurosciences, 36(5), 275–284.
Before the advent of MRI, the cortical folding pattern of patients was out of sight, out of mind. The sulcus nomenclature, which was taught in the anatomy classes, had no real use except for neuroanatomists and to some extent neurosurgeons. Few people were aware of the considerable interindividual variability of the folding pattern and of the complete lack of understanding of the origin of this variability. The fantastic development of MRI could have triggered a renewal of interest, but the actual scientific drive was functional imaging, which requires morphological variability to be removed to simplify statistical group analysis. Hence, for the brain mapping community, the variability of the folding pattern is mainly an impediment to perfect spatial normalization. Today, spatial normalization technologies have reached a very efficient stage where primary sulci, which are good landmarks of primary architectural areas, are reasonably aligned across subjects (Ashburner, 2007; Fischl et al., 2008). It is tempting to consider that the variability of the folding pattern is an epiphenomenon of low interest that can be forgotten. However, the normalization paradigm has provided the opportunity to quantify the morphological variability, which has raised awareness that this variability can be a very valuable source of information. For instance, techniques like voxelbased morphometry or cortical thickness analysis have provided a myriad of insights about the impact of development, pathologies, or cognitive skills on brain structures (Toga & Thompson, 2003). Hence, the variability of the folding pattern has received more and more attention. A wave of research programs aiming at explaining the origin and the dynamics of the folding process was triggered (Lefe`vre & Mangin, 2010; Regis et al., 2005; Reillo, de Juan Romero, Garcı´a-Cabezas, & Borrell, 2011; Sun & Hevner, 2014; Taber, 2014; Toro & Burnod, 2005; Van Essen, 1997). These research programs propose various hypotheses leading to consider the folding pattern as a proxy of the underlying architecture of the cerebral cortex. Then, the sulci geometry could provide biomarkers of abnormal development or the signature of specific architectures (Cachia et al., 2014; Dubois et al., 2008; Plaze et al., 2011; Sun et al., 2012; Weiner et al., 2014).
The Need for Computational Anatomy The quantification of the folding pattern is a difficult issue. The simplest strategies rely on global or local gyrification indexes measuring the amount of cortex buried into the folds (Schaer et al., 2008; Toro et al., 2008; Zilles, Palomero-Gallagher, & Amunts, 2013). A spectral approach can provide a richer insight
Brain Mapping: An Encyclopedic Reference
of the folding global features (Germanaud et al., 2012). Following the footsteps of the neuroanatomists is more ambitious because it requires the identification of the sulci described in the literature. Very few experts can perform this difficult and tedious task for the complete cerebral cortex. With the usual radiological point of view, namely, a series of slices, even the largest sulci can be difficult to recognize, which explains the scarcity of relevant knowledge to overcome the interindividual variability (see Figure 1). 3-D rendering of the cortical surface is of great help, but it is often insufficient to deal with unusual folding patterns that require inspecting the shape of the fold depths (Regis et al., 2005). Hence, combining 3-D and 2-D views is often mandatory. Cortical inflation is an attractive alternative to exhibit the buried cortex (Van Essen, Drury, Joshi, & Miller, 1998), but the deformations from the actual folding geometry can be disturbing for sulcus identification. Whatever the visualization strategy, our ignorance with respect to the origin of the variability prevents us to decode safely configurations where the main sulci are split into pieces and reorganized into unusual patterns. A dedicated atlas describing the most usual patterns is probably the best guideline for sulcus identification, but this atlas is not comprehensive because it stems from the study of only 25 brains (Ono, Kubik, & Abarnathey, 1990). The reliable identification of secondary and tertiary folds (Petrides, 2012) is beyond reach with the current state of knowledge. The complexity of the cortical folding pattern is overwhelming for human experts. Hence, computational anatomy is now a key player to help the field to harness the folding variability. Alignment with a single-subject cortical surface atlas has the merit to provide the rough localization of primary sulci (Destrieux, Fischl, Dale, & Halgren, 2010), but is not sufficient to obtain accurate definition of the sulci for shape analysis. The idiosyncrasies of the template brain are not a good model for any other brain. The solution could lie into multisubject atlases (Heckemann, Hajnal, Aljabar, Rueckert, & Hammers, 2006), but this approach may not be flexible enough to overcome the ambiguities hampering a reliable pairwise alignment between cortical patterns. Hence, the community developed alternatives with a stronger computer vision flavor. First, bottom-up processing pipelines convert standard MR images into synthetic representations of individual folding geometries; and second, pattern recognition techniques match such representations with a model of the sulci (see Figure 2). Automatic recognition of the sulci provides a range of opportunities for morphometry (see Figure 3; Mangin, Jouvent, & Cachia, 2010) or sulcusdriven spatial normalization (Auzias et al., 2011, 2013; Joshi et al., 2010; Thompson & Toga, 1996).
http://dx.doi.org/10.1016/B978-0-12-397025-1.00307-9
365
Figure 1 Sulci are very difficult to recognize in a stack of brain slices.
S.Pe.C.median. F.C.L.r.diag. F.C.L.r.asc. F.C.L.r.ant.
‘An artificial neuroanatomis’
S.Pe.C.marginal. S.Pe.C.sup. S.Pe.C.inter. S.Pe.C.inf.
S.C. F.C.L.r.retroC.tr. F.I.P.Po.C.inf. S.Po.C.sup. S.Pa.sup. S.GSM.
S.F.sup.
F.I.P.r.int.1 S.T.s.ter.asc.ant F.I.P.r.int.2
S.F.inter.
3-D Retina
Elementary folds
S.F.inf.
F.I.P. S.T.s.ter.asc.post.
S.F.inf.ant. S.F.marginal.
Occipital
S.F.orbitaire. S.Or. F.C.L.a. F.C.L.p. Insula S.T.pol.
S.T.i.post. S.O.T.lat.post. S.O.T.lat.med. F.C.L.r.sc.post. S.T.i.ant. S.C.sylvian. S.T.s. F.C.L.r.sc.ant.
Sulcus model (a)
(d)
(e)
(b)
Bottom-up processing
Pattern recognition
Sulcus recognition (c)
(f)
Figure 2 A computer vision pipeline mimicking a human anatomist. Its 3-D retina is the standard space of the brain mapping community. After detection of the building blocks of the folding pattern from a negative mold of the brain, the sulci of the standard nomenclature are reassembled according to a model inferred from a learning database. Reproduced from Perrot, M., Rivie`re, D., and Mangin, J. F. (2011). Cortical sulci recognition and spatial normalization. Medical Image Analysis, 15(4), 529–550.
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
367
Figure 3 Once identified, a sulcus provides various morphometric features like length (red), depth (yellow), surface area (blue), or average span between its walls (right).
Bottom-Up Representations of the Folding Pattern This section describes examples of bottom-up strategies. Each approach aims at converting the implicit encoding of the cortical folding pattern embedded in the geometry of the cortical surface into a synthetic graphical representation. Hence, the pattern recognition system dedicated to sulcus recognition can deal with more abstract representations than images. Abstract representations open the door to machine learning strategies beyond reach for image registration. The first generation of techniques relied on voxel-based representations of the cortex. After a pipeline leading to a classification of gray matter and white matter, each elementary fold is detected and represented by its median surface (see Figure 2(d)). These pieces of surface stem, for instance, from the 3-D skeletonization of a negative mold of white matter (see Figure 2(c); Mangin et al., 1995; Mangin, Frouin, Bloch, Re´gis, & Lo´pez-Krahe, 1995). This skeleton is a set of connected surfaces generated from iterative homotopic erosions scalping the mold while preserving its initial topology. This process can be combined with a watershed algorithm using MR intensities as altitude in order to impose the localization of the skeleton in the ‘crevasse’ corresponding to the cerebrospinal fluid filling up the folds (Mangin et al., 2004). The skeleton is finally split into elementary pieces at the level of the junctions between folds or in case of local depth minima along the fold bottom indicating the fusion between several primal folding entities. A sibling approach restricts the skeletonization to the bounding hull of the cortex (Le Goualher et al., 1999). The resulting 2-D skeleton is split at junction points in order to define the superior trace of each fold. Note that the few folds that do not reach the bounding hull are missed by this process. Then, the initial curve grows until the bottom of the fold in order to yield a 2-D parameterized surface, which is the strength of the approach (Vaillant & Davatzikos, 1997). Finally, these different methods represent the topography of the folding pattern as
a graph: The nodes are the elementary folds and the links represent junctions or the fact that two parallel folds build up a gyrus. Some approaches put the focus on the deepest part of the 3-D folding geometry. For this purpose, the representation can rely on the bottom lines of the folds defined from the 3-D skeleton using simple topological considerations (Lohmann, 1998; Mangin, Re´gis, et al., 1995; Mangin, Frouin, Bloch, Re´gis, & Lo´pez-Krahe, 1995). When the key focus is not even the bottom lines but the deepest points of the folding, an alternative to the skeleton-based strategy lies in depth-based processing. For instance, a watershed-based algorithm using depth as altitude can be used to split the negative mold of whiter matter in the so-called sulcal basins associated with depth local maxima (Lohmann & von Cramon, 2000). The main difficulty is the pruning of the basins in order to keep only the morphologically meaningful ones. The end of the nineties coincided with a shift from volumetric processing to surface-based processing, thanks to the maturity of the pipelines generating spherical cortical surfaces (Dale, Fischl, & Sereno, 1999). This transition has increased the trend to focus on the depth of the folding, because the embedding of the cortical surface into the 3-D space is less accessible once the cortex is represented as a 2-D mesh. When dealing with cortical surface meshes, the bottom lines of the sulci are detected using variants of surface-based skeletonization algorithms acting on buried regions defined from depth or curvature (Kao et al., 2007; Li, Guo, Nie, & Liu, 2010; Seong et al., 2010; Shi, Thompson, Dinov, & Toga, 2008). These methods differ mainly not only in the way they combine depth and curvature to achieve reliable localization of the bottom lines but also in their pruning strategy to get rid of spurious detections. Semiautomatic approaches have been proposed to delineate the optimal sulcus bottom line from manually selected extremities (Le Troter, Auzias, & Coulon, 2012; Shattuck et al., 2009). A strong advantage of the
368
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
surface-based strategies is their sensitivity to dimples, namely, primal sketches of the folding visible in depth or curvature maps. The dimples cannot be detected by 3-D skeletonization algorithms because they are not completely folded (see Figure 4(b)), which is problematic for studying the developing brain (Dubois, Benders, Borradori-Tolsa, et al., 2008; Dubois, Benders, Cachia, et al., 2008) or tertiary sulci. While most of the bottom-up strategies aim at designing methods performing automatic recognition of the sulci of the standard nomenclature, a few approaches are dedicated to research programs questioning the current models of the sulcal anatomy. These approaches aim at inferring a new model of the folding pattern overcoming the ambiguities raised by the interindividual variability (see Figure 4). They mainly target local maxima of cortical depth, which are supposed to result from gyri buried into the folds (see Figure 5). The variable depth of these buried gyri partly explains interindividual variability (Regis et al., 2005). Thanks to the spherical topology of the cortical surface, the analogy with geology leads to simple adaptations of the watershed notion to segregate the surface into depth-based patches, the sulcal basins, which mimic catchment basins, namely, geographic areas with a common outlet for the surface runoff (Rettmann, Han, Xu, & Prince, 2002; Yang & Kruggel, 2008). The most recent approaches do not even pay attention to the spatial extent of the catchment basins but deal with simplistic representations based on the deepest points called sulcal pits (Im et al., 2010; Lohmann, von Cramon, & Colchester, 2008; Meng, Li, Lin, Gilmore, & Shen, 2014; Operto et al., 2012). In order to enrich the representations and to postpone the pruning to the sulcus recognition or model inference stage, a hierarchical strategy inspired by the field of scale space has
(a)
been proposed (Cachia et al., 2001). In the same spirit, graph-based approaches merging surface-based, line-based, and point-based representations are in gestation (Bao, Giard, Tourville, & Klein, 2012). Finally, in the context of developmental studies, an original approach dedicated to longitudinal data has been designed to focus on the seeds of the folding process, namely, the points of the cortical surface corresponding to local maxima of the folding rate (Lefe`vre et al., 2009).
Sulcus Recognition Most of the graphic representations yielded by the pipelines of the previous section include oversegmentation of the sulci of the nomenclature. Indeed, the geometry of a sulcus often includes subdivisions related to branches or interruptions. Then, the sulcus recognition requires a reconstruction process from the set of building blocks listed in the representation, which amounts to a labeling with the sulcus names of the nomenclature (see Figures 2 and 4). The number of names involved in the labeling ranges from 10 to 65 in each hemisphere, according to the richness of the sulcus model. The labeling can be viewed as a many-to-one matching between the representation built for a given subject and the model of the sulci. The model of the sulci can be simply a learning data set of individual representations labeled by a human expert (Lyu et al., 2010) or a probabilistic representation of the sulcus variability inferred from this learning data set: maps of the spatial variability of each sulcus after optimal alignment across the data set (Perrot, Rivie`re, & Mangin, 2011) or different kinds of random graph modeling the joint variability of pairs of sulci (Mangin, Re´gis, et al., 1995; Mangin, Frouin, Bloch, Re´gis, &
1
(b)
(c)
Figure 4 Standard sulcus nomenclature. (a) Three frontal lobes; (b) the skeleton of the negative brain mold used to detect folds; (1) undetected dimples; (c) labeling of the folds with the standard sulcus nomenclature (red, central; yellow, precentral; green, superior frontal; cyan, intermediate frontal; violet, inferior frontal; blue, sylvian valley, etc.).
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
369
2
(a)
(b)
2
(c)
Figure 5 Toward an alphabet of the folding pattern. (a) White matter of the Figure 4 frontal lobes; (2) buried gyri also called ‘plis de passage’; (b, c) labeling of the folds with the sulcal roots nomenclature corresponding to putative indivisible entities. Reproduced from Regis, J., Mangin, J., Ochiai, T., Frouin, V., Riviere, D., Cachia, A., Tamura, M., & Samson, Y. (2005). "Sulcal root" generic model: A hypothesis to overcome the variability of the human cortex folding patterns. Neurologia Medico-Chirurgica, 45(1), 1–17.
Lo´pez-Krahe, 1995; Riviere et al., 2002; Shi et al., 2009; Yang & Kruggel, 2009). The labeling of the building blocks is driven by an optimization process often casted into a Bayesian framework aiming at maximizing the similarity between the sulci defined by the labeling and their model. With this regard, the multiatlas strategy of Lyu et al. has a specific status since the model is the set of bottom lines of several instances of each sulcus efficiently matched with the candidate sulci using a spectral method. For the other approaches, the global similarity measure to be optimized is a sum of local similarity measures. These local measures can stem from local registrations between a fold and probabilistic maps of the sulci (Perrot et al., 2011), the output of a multilayer perceptron fed with features describing the shape of sulci or pairs of sulci and trained on the learning database (Riviere et al., 2002), and more standard potential functions acting on shape features like localization, orientation, length, moments, or wavelet coefficients (Mangin, Re´gis, et al., 1995; Mangin, Frouin, Bloch, Re´gis, & Lo´pezKrahe, 1995; Shi, Tu, et al., 2009; Yang & Kruggel, 2009). For all these methods, dealing with graphic representations rather than images allows the optimization process to rely on sophisticated schemes like simulated annealing (Riviere et al., 2002), genetic algorithms (Yang & Kruggel, 2009), or belief propagation (Shi, Tu, et al., 2009b). The lack of gold standard and the fact that only two of these methods using the same bottom-up pipeline are distributed to the community prevent simple comparisons (http://brainvisa. info) (Perrot et al., 2011; Riviere et al., 2002). The leave-oneout validation versus manual labeling of the method of Perrot et al. has shown an 86% mean recognition rate across the 65
sulci provided in each hemisphere. The performance varies from 95% for the largest primary sulci to 70% for the most variable secondary sulci. Users of this method interested in morphometry studies report the processing of more than 10 000 subjects. Note that the main risk of the bottom-up strategies is undersegmentation, namely, situations where a building block includes the frontier between two sulci. Overcoming this situation probably requires a top-down complementary strategy that is not explicitly embedded in current methods, except in the multiatlas strategy that allows refinement of the individual representations by analogy with the closest atlas (Lyu et al., 2010). Note also that the strategy of Shi et al. aims at defining sulci as continuous curves, to mimic manual tracing, which leads the method to fill up the gaps existing in bottom-up representations (Shi, Tu, et al., 2009b). The methods dealing with point-based representations shall be protected from oversegmentation. In reality, they are prone to a sibling issue: the sulcal pits or sulcal basin centroids of the model are not detected in all brains. Nevertheless, dealing with points is more comfortable than dealing with sulci because the many-toone matching problem becomes a one-to-one matching problem, resulting in simpler formulations (Im et al., 2011; Lohmann & von Cramon, 2000).
Inference of New Models of the Folding Pattern? The success of the multiatlas strategy in neuroimaging (Heckemann et al., 2006) could lead to consider that the future of sulcus recognition is in pattern matching methods informed by a very large data set of manually labeled sulci, in the spirit of
370
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
the approach of Lyu et al. (2010). This data set would play the same role as the huge pools of translated documents of institutions like the European Union for automatic language translation. Indeed, today’s most advanced translation services rely on statistical pattern matching rather than on teaching the rules of human language to the computer. To deal with the lack of gold standard, the learning dataset could stem from a consensus-based labeling effort of the community of neuroanatomists in order to generate the body of information required for the machine to do the job correctly whatever the brain idiosyncrasies. But with our current understanding of the variability, would the consensual labeling of unusual patterns be really reliable? Unlike the field of language translation, mimicking human behavior is not necessarily the best strategy for the computer. It should be noted that the sulcal pits model used to label individual sulcal pits does not stem from the anatomical literature but from the field of computational anatomy. The pits in the model are clusters of individual pits detected after nonlinear alignment of a large set of cortical surfaces (Im et al., 2010). The sulcal pits model is in good agreement with the sulcal roots model proposed by a human anatomist as an alphabet of putative indivisible atomic entities supposed to be stable across individuals because of a developmental origin (see Figure 5; Regis et al., 2005). The sulcal pits model provides a good foretaste of the potential of computational anatomy for paving the way toward new more objective models of the folding patterns. Advances in the understanding of the folding dynamics could also largely contribute to this research program. For instance, the intrinsic geometry of the cortical surface provided by the first eigenvectors of the Laplace–Beltrami operator may be intimately associated with the folding process. The associated coordinate system could be the ideal alignment between subjects before model inference or sulcus recognition (Shi, Dinov, & Toga, 2009; Shi, Sun, Lai, Dinov, & Toga, 2010; Shi, Tu, et al., 2009). The convoluted shape of the cerebral cortex is a challenge for human binocular vision. In return, computer vision systems can be endowed with a dedicated architecture including 3-D retina and 3-D higher-level vision areas. Furthermore, this vision architecture dedicated to the cortical surface can be duplicated without limit, which overcomes the working memory overload disturbing human experts trying to model the folding pattern variability. Manifold learning technology applied on massive databases in the spirit of Ono et al. could help us to segregate the different folding patterns existing in the population, in order to trigger a research program aiming at matching these patterns according to architectural clues provided by other imaging modalities (Ono et al., 1990; Sun, Perrot, Tucholka, Riviere, & Mangin, 2009). Therefore, computational anatomy should be the perfect assistant to support the neuroanatomists in their quest for a better model of the variability.
Acknowledgments This work was supported by the European FET Flagship project ‘Human Brain Project’ (SP2), the French Agence Nationale de la Recherche (ANR-09-BLAN-0038-01 ‘BrainMorph,’ ANR-14
‘APEX,’ and ANR-12JS03-001-01 ‘MoDeGy’), and the French ‘Plan Alzheimer’ Foundation (CATI multicenter imaging platform).
See also: INTRODUCTION TO ANATOMY AND PHYSIOLOGY: Development of the Basal Ganglia and the Basal Forebrain; Embryonic and Fetal Development of the Human Cerebral Cortex; Fetal and Postnatal Development of the Cortex: MRI and Genetics; Gyrification in the Human Brain; Sulci as Landmarks; INTRODUCTION TO METHODS AND MODELING: Automatic Labeling of the Human Cerebral Cortex.
References Ashburner, J. (2007). A fast diffeomorphic image registration algorithm. NeuroImage, 38(1), 95–113. Auzias, G., Colliot, O., Glaune`s, J. A., Perrot, M., Mangin, J. F., Trouve´, A., et al. (2011). Diffeomorphic brain registration under exhaustive sulcal constraints. IEEE Transactions on Medical Imaging, 30(6), 1214–1227. Auzias, G., Lefe`vre, J., Le Troter, A., Fischer, C., Perrot, M., Re´gis, J., et al. (2013). Model-driven harmonic parameterization of the cortical surface: HIP-HOP. IEEE Transactions on Medical Imaging, 32(5), 873–887. Bao, F., Giard, J., Tourville, J., & Klein, A. (2012). Automated extraction of nested sulcus features from human brain MRI data. In IEEE Engineering in Medicine and Biology Society (pp. 4429–4433). Cachia, A., Borst, G., Vidal, J., Fischer, C., Pineau, A., Mangin, J. F., et al. (2014). The shape of the ACC contributes to cognitive control efficiency in preschoolers. Journal of Cognitive Neuroscience, 26(1), 96–106. Cachia, A., Mangin, J., Rivie`re, D., Boddaert, N., Andrade, A., Kherif, F., et al. (2001). A mean curvature based primal sketch to study the cortical folding process from antenatal to adult brain. In Medical image computing and computer-assisted intervention–MICCAI 2001 (pp. 897–904). Dale, A. M., Fischl, B., & Sereno, M. I. (1999). Cortical surface-based analysis. I. Segmentation and surface reconstruction. NeuroImage, 9(2), 179–194. Destrieux, C., Fischl, B., Dale, A., & Halgren, E. (2010). Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. NeuroImage, 53(1), 1–15. Dubois, J., Benders, M., Borradori-Tolsa, C., Cachia, A., Lazeyras, F., Leuchter, R. H.-V., et al. (2008). Primary cortical folding in the human newborn: An early marker of later functional development. Brain, 131(8), 2028–2041. Dubois, J., Benders, M., Cachia, A., Lazeyras, F., Leuchter, R. H.-V., Sizonenko, S. V., et al. (2008). Mapping the early cortical folding process in the preterm newborn brain. Cerebral Cortex, 18(6), 1444–1454. Fischl, B., Rajendran, N., Busa, E., Augustinack, J., Hinds, O., Yeo, B. T., et al. (2008). Cortical folding patterns and predicting cytoarchitecture. Cerebral Cortex, 18(8), 1973–1980. Germanaud, D., Lefe`vre, J., Toro, R., Fischer, C., Dubois, J., Hertz-Pannier, L., et al. (2012). Larger is twistier: Spectral analysis of gyrification (SPANGY) applied to adult brain size polymorphism. NeuroImage, 63(3), 1257–1272. Heckemann, R. A., Hajnal, J. V., Aljabar, P., Rueckert, D., & Hammers, A. (2006). Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. NeuroImage, 33(1), 115–126. Im, K., Choi, Y. Y., Yang, J. J., Lee, K. H., Kim, S. I., Grant, P. E., et al. (2011). The relationship between the presence of sulcal pits and intelligence in human brains. NeuroImage, 55(4), 1490–1496. Im, K., Jo, H. J., Mangin, J.-F., Evans, A. C., Kim, S. I., & Lee, J.-M. (2010). Spatial distribution of deep sulcal landmarks and hemispherical asymmetry on the cortical surface. Cerebral Cortex, 20(3), 602–611. Joshi, A. A., Pantazis, D., Li, Q., Damasio, H., Shattuck, D. W., Toga, A. W., et al. (2010). Sulcal set optimization for cortical surface registration. NeuroImage, 50(3), 950–959. Kao, C. Y., Hofer, M., Sapiro, G., Stem, J., Rehm, K., & Rottenberg, D. A. (2007). A geometric method for automatic extraction of sulcal fundi. IEEE Transactions on Medical Imaging, 26(4), 530–540. Le Goualher, G., Procyk, E., Collins, D., Venugopal, R., Barillot, C., & Evans, A. (1999). Automated extraction and variability analysis of sulcal neuroanatomy. IEEE Transactions on Medical Imaging, 18(3), 206–217. Le Troter, A., Auzias, G., & Coulon, O. (2012). Automatic sulcal line extraction on cortical surfaces using geodesic path density maps. NeuroImage, 61(4), 941–949.
INTRODUCTION TO METHODS AND MODELING | Sulcus Identification and Labeling
Lefe`vre, J., Leroy, F., Khan, S., Dubois, J., Huppi, P., Baillet, S., et al. (2009). Identification of growth seeds in the neonate brain through surfacic Helmholtz decomposition. Information Processing in Medical Imaging, 21, 252–263. Lefe`vre, J., & Mangin, J.-F. (2010). A reaction–diffusion model of human brain development. PLoS Computational Biology, 6(4), e1000749. Li, G., Guo, L., Nie, J., & Liu, T. (2010). An automated pipeline for cortical sulcal fundi extraction. Medical Image Analysis, 14(3), 343–359. Lohmann, G. (1998). Extracting line representations of sulcal and gyral patterns in MR images of the human brain. IEEE Transactions on Medical Imaging, 17(6), 1040–1048. Lohmann, G., & von Cramon, D. Y. (2000). Automatic labelling of the human cortical surface using sulcal basins. Medical Image Analysis, 4(3), 179–188. Lohmann, G., von Cramon, D. Y., & Colchester, A. C. (2008). Deep sulcal landmarks provide an organizing framework for human cortical folding. Cerebral Cortex, 18(6), 1415–1420. Lyu, I., Seong, J. K., Shin, S. Y., Im, K., Roh, J. H., Kim, M. J., et al. (2010). Spectralbased automatic labeling and refining of human cortical sulcal curves using expertprovided examples. NeuroImage, 52(1), 142–157. Mangin, J.-F., Frouin, V., Bloch, I., Re´gis, J., & Lo´pez-Krahe, J. (1995). From 3D magnetic resonance images to structural representations of the cortex topography using topology preserving deformations. Journal of Mathematical Imaging and Vision, 5(4), 297–318. Mangin, J. F., Jouvent, E., & Cachia, A. (2010). In-vivo measurement of cortical morphology: Means and meanings. Current Opinion in Neurology, 23(4), 359–367. Mangin, J., Re´gis, J., Bloch, I., Frouin, V., Samson, Y., & Lo´pez-Krahe, J. (1995). A MRF based random graph modelling the human cortical topography. In Computer vision, virtual reality and robotics in medicine (pp. 177–183). Mangin, J. F., Riviere, D., Cachia, A., Duchesnay, E., Cointepas, Y., Papadopoulos-Orfanos, D., et al. (2004). Object-based morphometry of the cerebral cortex. IEEE Transactions on Medical Imaging, 23(8), 968–982. Meng, Y., Li, G., Lin, W., Gilmore, J. H., & Shen, D. (2014). Spatial distribution and longitudinal development of deep cortical sulcal landmarks in infants. NeuroImage, 100, 206–218. Ono, M., Kubik, S., & Abarnathey, C. D. (1990). Atlas of the cerebral sulci. New York: Georg Thieme. Operto, G., Auzias, G., Le Troter, A., Perrot, M., Riviere, D., Dubois, J., et al. (2012). Structural group analysis of cortical curvature and depth patterns in the developing brain. In IEEE ISBI (pp. 422–425). Perrot, M., Rivie`re, D., & Mangin, J. F. (2011). Cortical sulci recognition and spatial normalization. Medical Image Analysis, 15(4), 529–550. Petrides, M. (2012). The human cerebral cortex: An MRI atlas of the sulci and gyri in MNI stereotaxic space. Amsterdam: Academic Press. Plaze, M., Paille`re-Martinot, M. L., Penttila¨, J., Januel, D., de Beaurepaire, R., Bellivier, F., et al. (2011). "Where do auditory hallucinations come from?" – A brain morphometry study of schizophrenia patients with inner or outer space hallucinations. Schizophrenia Bulletin, 37(1), 212–221. Regis, J., Mangin, J., Ochiai, T., Frouin, V., Riviere, D., Cachia, A., et al. (2005). "Sulcal root" generic model: A hypothesis to overcome the variability of the human cortex folding patterns. Neurologia Medico-Chirurgica, 45(1), 1–17. Reillo, I., de Juan Romero, C., Garcı´a-Cabezas, M., & Borrell, V. (2011). A role for intermediate radial glia in the tangential expansion of the mammalian cerebral cortex. Cerebral Cortex, 21(7), 1674–1694. Rettmann, M. E., Han, X., Xu, C., & Prince, J. L. (2002). Automated sulcal segmentation using watersheds on the cortical surface. NeuroImage, 15(2), 329–344. Riviere, D., Mangin, J., Papadopoulos-Orfanos, D., Martinez, J., Frouin, V., & Regis, J. (2002). Automatic recognition of cortical sulci of the human brain using a congregation of neural networks. Medical Image Analysis, 6(2), 77–92.
371
Schaer, M., Cuadra, M. B., Tamarit, L., Lazeyras, F., Eliez, S., & Thiran, J. P. (2008). A surface-based approach to quantify local cortical gyrification. IEEE Transactions on Medical Imaging, 27(2), 161–170. Seong, J. K., Im, K., Yoo, S. W., Seo, S. W., Na, D. L., & Lee, J. M. (2010). Automatic extraction of sulcal lines on cortical surfaces based on anisotropic geodesic distance. NeuroImage, 49(1), 293–302. Shattuck, D. W., Joshi, A. A., Pantazis, D., Kan, E., Dutton, R. A., Sowell, E. R., et al. (2009). Semi-automated method for delineation of landmarks on models of the cerebral cortex. Journal of Neuroscience Methods, 178(2), 385–392. Shi, Y., Dinov, I., & Toga, A. W. (2009). Cortical shape analysis in the Laplace-Beltrami feature space. Medical Image Computing and Computer-Assisted Intervention, 12(Pt 2), 208–215. Shi, Y., Sun, B., Lai, R., Dinov, I., & Toga, A. W. (2010). Automated sulci identification via intrinsic modeling of cortical anatomy. Medical Image Computing and Computer-Assisted Intervention, 13(Pt 3), 49–56. Shi, Y., Thompson, P. M., Dinov, I., & Toga, A. W. (2008). Hamilton-Jacobi skeleton on cortical surfaces. IEEE Transactions on Medical Imaging, 27(5), 664–673. Shi, Y., Tu, Z., Reiss, A. L., Dutton, R. A., Lee, A. D., Galaburda, A. M., et al. (2009). Joint sulcal detection on cortical surfaces with graphical models and boosted priors. IEEE Transactions on Medical Imaging, 28(3), 361–373. Sun, T., & Hevner, R. F. (2014). Growth and folding of the mammalian cerebral cortex: From molecules to malformations. Nature Reviews Neuroscience, 15(4), 217–232. Sun, Z. Y., Klo¨ppel, S., Rivie`re, D., Perrot, M., Frackowiak, R., Siebner, H., et al. (2012). The effect of handedness on the shape of the central sulcus. NeuroImage, 60(1), 332–339. Sun, Z., Perrot, M., Tucholka, A., Riviere, D., & Mangin, J. (2009). Constructing a dictionary of human brain folding patterns. In: MICCAI Proceedings, Berlin, Hiedelberg: Springer Verlag, LNCS-5762. Taber, L. A. (2014). Morphomechanics: Transforming tubes into organs. Current Opinion in Genetics and Development, 27C, 7–13. Thompson, P., & Toga, A. W. (1996). A surface-based technique for warping threedimensional images of the brain. IEEE Transactions on Medical Imaging, 15(4), 402–417. Toga, A. W., & Thompson, P. M. (2003). Temporal dynamics of brain anatomy. Annual Review of Biomedical Engineering, 5, 119–145. Toro, R., & Burnod, Y. (2005). A morphogenetic model for the development of cortical convolutions. Cerebral Cortex, 15(12), 1900–1913. Toro, R., Perron, M., Pike, B., Richer, L., Veillette, S., Pausova, Z., et al. (2008). Brain size and folding of the human cerebral cortex. Cerebral Cortex, 18(10), 2352–2357. Vaillant, M., & Davatzikos, C. (1997). Finding parametric representations of the cortical sulci using an active contour model. Medical Image Analysis, 1(4), 295–315. Van Essen, D. C. (1997). A tension-based theory of morphogenesis and compact wiring in the central nervous system. Nature, 385(6614), 313–318. Van Essen, D. C., Drury, H. A., Joshi, S., & Miller, M. I. (1998). Functional and structural mapping of human cerebral cortex: Solutions are in the surfaces. Proceedings of the National Academy of Sciences of the United States of America, 95(3), 788–795. Weiner, K. S., Golarai, G., Caspers, J., Chuapoco, M. R., Mohlberg, H., Zilles, K., et al. (2014). The mid-fusiform sulcus: A landmark identifying both cytoarchitectonic and functional divisions of human ventral temporal cortex. NeuroImage, 84, 453–465. Yang, F., & Kruggel, F. (2008). Automatic segmentation of human brain sulci. Medical Image Analysis, 12(4), 442–451. Yang, F., & Kruggel, F. (2009). A graph matching approach for labeling brain sulci using location, orientation, and shape. Neurocomputing, 73, 179–190. Zilles, K., Palomero-Gallagher, N., & Amunts, K. (2013). Development of cortical folding during evolution and ontogeny. Trends in Neurosciences, 36(5), 275–284.