- Email: [email protected]
Cortical feature maps via geometric models
Journal of Physiology - Paris 103 (2009) 46–51
Contents lists available at ScienceDirect
Journal of Physiology - Paris journal homepage: www.elsevier.com/locate/jphysparis
Cortical feature maps via geometric models Scott D. Pauls Department of Mathematics, Dartmouth College, Hanover NH 03755, United States
a r t i c l e
i n f o
Keywords: Visual cortex Sub-Riemannian geometry Cortical maps
a b s t r a c t We present a new model for feature map formation in the primary visual cortex, building on dimension reduction/wire length minimization techniques. We create a model space of feature parameters, endowed with various geometries picked to reflect physical or experimental data and search for a map from the parameter space to the cortical sheet which minimizes distortions. Upon simulating these maps, we find a family of Riemannian and sub-Riemannian geometries which give rise to feature maps which reflect known experimental data concerning (1) the qualitative arrangement of orientation maps and (2) the distribution of connections. One of the main findings is that experimental data showing both elongated and non-elongated connection patterns are represented within our family of models. Ó 2009 Elsevier Ltd. All rights reserved.
1. Introduction Within the primary visual cortex there are cells with preferential responses to a variety of stimulus features. The pattern of these cells form overlayed maps for each stimulus feature, such as position, orientation and ocular dominance. Numerous models have been devised to help understand the formation of feature maps in accordance with experimental data. Central to many of these models is the notion of dimension reduction, due to Durbin and Mitchison (1990), who view the maps as arising from a mapping from a two dimensional sheet of cortex, C to a high dimensional parameter space, P, of features. Most dimension reduction schemes impose reasonable (and experimentally observed) constraints on the mapping such as smoothness of the resulting feature map and completeness of coverage. In particular, we recall the elasticnet algorithm (Durbin and Willshaw, 1987) and the self-organizing feature map algorithm (Kohonen, 2001) which were both successful in producing feature maps qualitatively consistent with experimentally observed feature maps. These techniques and their variants have received a great deal of attention (see, for example, Swindale (2004), Yu et al. (2005) and the references therein). In a related direction, Koulakov and Chklovskii (2001) added an additional hypothesis (first proposed by Hubel and Weisel (1977)), that of wire length minimization, that coupled with the self-organizing algorithm formed different types of feature maps depending on the nature of the connection distribution of each neuron. The purpose of this work is to further study the relation between the distribution of connections between neurons in the primary visual cortex and the feature maps via a dimension reduction/wire length minimization model which seeks to find an appropriate geo-
E-mail address: [email protected] 0928-4257/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.jphysparis.2009.05.003
metric structure on the parameter space P which will reflect both the connection distribution and produce plausible (with respect to experimental evidence) feature maps. Of the main experimental results of interest in our model are the findings of patchy patterns of connections between neurons in the visual cortex which preferentially connect to neurons of similar preferred orientation (in the cat (Gilbert and Wiesel, 1993; Kisvarday et al., 1997), the tree shrew (Bosking et al., 2002) and the macaque (Malach et al., 1993)). Moreover, in some mammals, the connection patterns are elongated along a direction corresponding to the preferred orientation of the injection site (tree shrew (Bosking et al., 2002), cat (Schmidt et al., 1997) and squirrel monkey (Sincich and Blasdel, 2001)). While this anisotropy is quite pronounced, we recall that in mammals with well defined ocular dominance columns, the majority of the anisotropy can be accounted for by the separation imposed by the ocular dominance columns. Our method, described in detail in the next sections, focuses on a parameter space P ¼ P ½0; 1 ¼ ðR2 S1 Þ ½0; 1 where R2 gives the spatial position, and ðcosðhÞ; sinðhÞÞ 2 S1 gives the preferred orientation angle h and q 2 R gives the tuning strength. We take a disb , respectively, and create a b and C crete sampling of P and C; P b which satisfies both the goal b ! C dimension reduction map, f : P of smoothness and wire length minimization. To do this we propose a new method: we equip P and C with metrics and seek a map which minimizes metric distortion. This has two consequences. First, it links as closely as possible the geometry of the network of connections in the cortex and the geometry of the parameter space. Second, as discussed below, the distortion minimization imposes both smoothness and wire length minimization on the resulting cortical feature maps. We adapt the metric dP , and hence the geometry of P, to the available experimental evidence. More specifically, we find that to produce plausible feature maps, dP must have three properties:
47
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
a. It must be invariant under the group of rotations and translations of the plane. b. dP is degenerate, i.e. there are distinct points x; x0 2 P with dP ðx; x0 Þ ¼ 0. More specifically, dP may not depend on the variable q. c. Metric balls exhibit an anisotropy reflecting the preferential connection of neurons with similar feature parameters. The first condition is motivated by the ability of the visual cortex to process stimuli equally well under the same invariance: the rotation or translation of an image is just as readily processed by V1 as the untransformed version. In the literature, this is often called a ‘‘shift-twist” invariance and has had wide and fruitful use in understanding aspects of the visual system (see, for example, Bressloff et al. (2001), Ben-Shahar and Zucker (2004), Citti and Sarti (2006), Sarti et al. (2008)) and applications to computer vision and image processing. The second condition is crucial to the formation of feature maps reflecting the experimental findings of Maldonado et al. (1997) while the third condition is essential in reflecting of experimental literature describing the connection distributions in various mammals. We use a Metropolis–Monte Carlo technique to deform random cortical maps towards distortion minimizers using two different families of metrics on P ¼ R2 S1 : an invariant Riemannian metric and an invariant sub-Riemannian metric. These metrics are lifted, via projection, to degenerate metrics on P. Our simulation indicates that the sub-Riemannian metric on the parameter space induces a more plausible cortical orientation map than the Riemannian metric and exhibits patchy connection patterns and, under suitable choices of the parameters in the definition of the metric, both elongated and non-elongated connection maps. In particular, this complements the work of Citti and Sarti (2006), Petitot (2003) and Petitot and Tondut (1998), who also found and exploited sub-Riemannian models from different perspectives.
2. Dimension reduction, wire length minimization and smoothness As the visual cortex encodes information from a high dimensional space of parameters in what is essentially a two dimensional sheet, it is a fundamental problem to understand and/or model mechanisms by which the feature maps are created. In 1977, Hubel and Weisel (1977) suggested that the structure of orientation preference maps arise from evolutionary pressure to minimize cortical volume while maintaining complete parameter coverage. Durbin and Mitchison (1990) proposed a ‘‘dimension reduction” approach to this optimization, viewing cortical maps as mappings from parameter space to a cortical sheet and attempting to create maps which achieve complete coverage while minimizing total connection length. As outlined in the introduction, a number of authors have used a variety of algorithms to produce simulated feature maps. Our approach falls in the category of dimension reduction/wire length minimizing schemes but has significant novel features. In particular, we point out four new aspects of our approach: i. In the spirit of Cowey’s original argument (Cowey, 1979), we build a map from the space of parameters to the cortex whereas other work builds a map from the cortex to parameter space. ii. We use the geometry of the parameter space, determined by its metric, to define connections between neurons (or neuron populations) described by those parameters. iii. We impose an invariance under rotations and translations on the geometry of the parameter space.
iv. We attempt to optimize our map by ensuring both minimal wire length and smoothness by enforcing compatibility between the geometry of the parameter space and that of the cortex. To fix notation, we denote the parameter space by P and the cortical sheet by C. As noted in Durbin and Mitchison (1990), there are two competing ideas which guide the formation of such maps. First, the experimentally observed smoothness of the orientation maps would imply that nearby cortical positions should correspond to nearby parameter values. Second, we require that close parameter values should map to close cortical positions. This condition was first put forth by Cowey (1979). One way to express these two ideas is in terms of metrics on P and C, denoted dP and dC , respectively. For example, the smoothness condition can be expressed as a co-Lipschitz condition:
LdP ðx; x0 Þ 6 dC ðf ðxÞ; f ðx0 ÞÞ
ð1Þ
where f : P ! C and L is a constant. Similarly, the Cowey condition can be expressed as a Lipschitz condition:
dC ðf ðxÞ; f ðx0 ÞÞ 6 L0 dP ðx; x0 Þ
ð2Þ
where, again, L0 is a constant. We make the following trivial observation: maps satisfying (1) may have distant cortical areas with close parameter values while those satisfying (2) may have distant parameter values mapped to nearby cortical areas. Since pinwheels and fractures in orientation maps correspond to mappings of relatively distant parameter values to adjacent cortical areas, we see that maps exhibiting such singularities would likely violate (1) with L close to 1 but could easily satisfy (2) with L0 close to 1. Of course, the existence of a biLipschitz map on continuous spaces P and C of different dimension cannot exist but such maps are attainable on fib be fixed disb and C nite discrete samplings of these spaces. We let P b a b ! C crete subsets of P and C, respectively and call a map f : P cortical map. In terms of cortical wire length minimization, we assume that neural connections are determined by proximity in parameter space – i.e. cells with similar parameter values are more likely to be connected than those with disparate values. This is consistent with experimental data showing that neurons are connected to other neurons in V1 with similar response characteristics. Gib , we define the conven a cortical map f and a fixed point x 2 P b as the image of a nections between f ðxÞ and other points of C metric ball containing x; f ðBP ðx; rÞÞ (the radius of the ball, r, will be determined later via examination of the relative scales of b ðP; dP Þ and ðC; dC Þ). If we denote the connection relation in C b rÞ, the total cortical wire b by Bðx; by f ðxÞ f ðx0 Þ and BP ðx; rÞ \ P length for the
X
X
f ðxÞ2b C
f ðx0 Þf ðxÞ
dC ðf ðxÞ; f ðx0 ÞÞ ¼
X b x2 P
x0 2b B ðx;rÞ
dC ðf ðxÞ; f ðx0 ÞÞ 6
X
L0 dP ðx; x0 Þ
b x2 P
x0 2b B ðx;rÞ
Thus, under the assumption that the geometry of the parameter space models the connections well, (2) with L0 as small as possible ensures low total cortical wire length. We also wish to allow the possibility of degenerate metrics on P. If P ¼ P P 0 and
dP ðx; x0 Þ ¼ dP ðpðxÞ; pðx0 ÞÞ where dP is a (non-degenerate) metric on P and p is the projection of P onto P, the Cowey condition (2) takes the following form: For b with pðxÞ – pðx0 Þ: every x; x0 2 P
dC ðf ðxÞ; f ðx0 ÞÞ 6 L0 dP ðpðxÞ; pðx0 ÞÞ 0
where L is a constants.
ð3Þ
48
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
As in the non-degenerate case, we consider the choice of metric, dP , on P, in this context, equivalent to specifying the number and nature of the connections between a fixed neuron and other neurons in the cortex. Fixing a radius r, we define the set of neurons b . We again b to be BP ðx; rÞ \ P connected to a fixed neuron x 2 P emphasize that condition (3) with L0 close to one, coupled with a bound on
X
X
bÞ y2pð P
x;x0 2p1 ðyÞ
dC ðf ðxÞ; f ðx0 ÞÞ
ð4Þ
ensures that the total wire length is as small as possible. We note that this approach is quite different from that of Koulakov and Chklovskii (2001) where the authors specify a connection distribution only and create a map which minimizes total wirelength with respect to the L2 energy using Euclidean metrics on both the parameter and cortical spaces. Our approach differs in that we allow for different metrics on P and C and use the metric on P to define the connections themselves and, as a consequence, the distribution of connections. Moreover, (Koulakov and Chklovskii (2001)) works with a single parameter, orientation, while our model deal with four: two spatial variables, orientation and tuning strength.
coordinates ðx; y; hÞ. Each point ðx; y; hÞ is thought of as the composition of two transformations, a rotation by h,
Rh ðx; y; h0 Þ ¼
sinðhÞ
and a translation T ðx;yÞ ðx0 ; y0 ; h0 Þ ¼ ðx þ x0 ; y þ y0 ; h0 Þ. The group action is given by composition:
ðx; y; hÞ ðx0 ; y0 ; h0 Þ ¼ T ðx;yÞ Rh T ðx0 ;y0 Þ Rh0 ¼ T ðx;yÞ ðx0 cosðhÞ y0 sinðhÞ; x0 sinðhÞ þ y0 cosðhÞ; h0 þ hÞ ¼ ðx þ x0 cosðhÞ y0 sinðhÞ; y þ x0 sinðhÞ þ y0 cosðhÞ; h0 þ hÞ
e h ðx; yÞt ; hÞk jðx; y; hÞjRT ¼ kð R cosðhÞ 1 sinðhÞ . Then, we can define a left invarisinðhÞ cosðhÞ ant metric based on this norm as follows: eh ¼ where R
3. Parameter space
3.1. The roto-translation group Before describing the various metrics, we recall an important property of the visual system: invariance under rotations and translations of the plane. This is often modeled by imposing a group invariance on the structure of the retinotopic and orientation maps. We will incorporate this feature by defining metrics which are invariant under this group operation. To begin, we define the group itself: Definition 1. We call the group, RT, of rotations and translations acting on the two dimensional plane the rototranslation group and represent it as follows: RT is diffeomorphic to R2 S1 with
ð5Þ
We may equip RT with a norm which is invariant under the group action by specifying a fixed norm k k on R3 and defining
dRT ððx; y; hÞ; ðx0 ; y0 ; h0 ÞÞ ¼
We turn next to the description of the parameter space and, importantly, of its geometry via the choice of metric. In contrast to the space C, which is always viewed as a subset of R2 with the Euclidean metric, our choice of metric on the parameter space will vary. We use the parameter space P ¼ P ½0; 1 ¼ ðR2 S1 Þ ½0; 1 denoted by variables fx; y; h; qg where ðx; yÞ denotes position, h denotes orientation tuning and q denotes the strength of orientation tuning. We introduce two families of metrics on P ¼ R2 S1 based on Riemannian and sub-Riemannian geometries to study their effect of this choice on the resulting cortical maps. We intentionally omit a dependence on the tuning strength, q. Thus, technically, we will define degenerate metrics on P, based on nondegenerate metrics on P ¼ R2 S1 . We make this choice in order to again fit with experimental evidence. Upon including the tuning strength parameter in the metrics, where the metric on S1 was extended, via polar coordinates, to a metric on S1 ½0; 1, our simulations reveal a ‘‘tuning strength map” with very regular features, which is inconsistent with Maldonado et al. (1997). Specifically, the tuning strength maps exhibit a high correlation between pinwheel centers and lower tuning strength, whereas (Maldonado et al. (1997)) shows cells of strong tuning close to pinwheel centers. We note further that, when including a dependence on tuning strength, the orientation maps for the Riemannian metrics no longer resembled experimentally observed maps while the orientation maps for the sub-Riemnannian metrics remained consistent with experimental data.
x ; h0 þ h y cosðhÞ
cosðhÞ sinðhÞ
1 ðjðx; y; hÞ1 ðx0 ; y0 ; h0 ÞjRT 2 þ jðx0 ; y0 ; h0 Þ1 ðx; y; hÞjRT Þ
e h ðx; yÞt ; hÞ and it Direct computation shows that ðx; y; hÞ1 ¼ ð R then follows easily that dRT is invariant under the action of RT on itself and under inversion. We review another way of creating invariant metrics on R using a different presentation. Identifying RT with P ¼ R2 S1 with coordinates ðx; y; hÞ, we define the following three vector fields which span the Lie algebra, rt, of RT:
X 1 ¼ cosðhÞ@ x þ sinðhÞ@ y X2 ¼ @h
ð6Þ
X 3 ¼ sinðhÞ@ x þ cosðhÞ@ y Notice that ½X 1 ; X 2 ¼ X 3 and ½X 3 ; X 2 ¼ X 1 are the only nontrivial bracket relations. If we choose a norm, nðÞ, on rt we define a left invariant metric associated to n by
dF ðx; yÞ ¼ inf
Z
c2A
nðc_ Þ
c
where A is the set of all absolutely continuous paths connecting x to y. This metric is, in general, a Finsler metric but if n is derived from an inner product, it is Riemannian. We may define a sub-Riemannian metric on RT by placing an inner product, h; i on a bracket generating subbundle of the tangent bundle, spanned by fX 1 ; X 2 g and defining
dsR ðx; yÞ ¼ inf
c2A0
Z
1
hc_ ; c_ i2
c
where A0 is the set of all absolutely continuous paths connecting x to y so that their derivatives, when they exist, are in spanfX 1 ; X 2 g. We now define two families of metrics on P ¼ R2 S1 . The first is the infinitesimal form of a family of Riemannian metrics and the second is the infinitesimal form of a family of sub-Riemannian metrics. By abuse of notation, we will refer to them as the Riemannian and sub-Riemannian metrics, respectively. As we shall see in the next section, we use the infinitesimal forms for computational convienience. We note that sub-Riemannian metrics in this setting can be realized as the limiting cases of families of Riemannian metrics where the length of one vector field, X 3 , tends to infinity. Thus, these two families could also be seen as one family of Riemannian metrics with certain limiting cases included.
49
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
3.2. Riemannian metric
4. Simulation methods
The first metric is infinitesimally equivalent to a standard Riemannian metric on P. We pick constants fa1 ; a2 ; a3 g
In each simulation, we begin with a random map from a b , taken from 1 ; 1 2 ½0; 2p ½0; 1 P 8 8 8 8 grid, P 2 2 1 1 2 b taken from ; ¼ P ½0; 1 to a 64 64 grid, C C. We select
dP;1 ððx; y; h; qÞ; ðx0 ; y0 ; h0 ; q0 ÞÞ 1 ¼ a21 ððx x0 Þ cosðhÞ þ ðy y0 Þ sinðhÞ þ ðx x0 Þ cosðh0 Þ 2 1 2 ððx x0 Þ sinðhÞ þ ðy y0 Þ cosðhÞ þðy y0 Þ sinðh0 ÞÞÞ þ a22 2 2
ðx x0 Þ sinðh0 Þ þ ðy y0 Þ cosðh0 ÞÞÞ þ a23 ððcosðhÞ0 cosðh0 ÞÞ2 12 þðsinðhÞ0 sinðh0 ÞÞ2 Þ ð7Þ Extending the action of R to P via T ðx;yÞ Rh ðx; y; h; qÞ ¼ ððx; yÞt e h ðx0 ; y0 Þt ; h þ h0 ; qÞ, we see that the dP;1 is invariant. We emphasize þR that while the standard Euclidean metric is also invariant under the group action but that the Riemannian metric has the advantage that the group action acts nicely on the basis of vector fields described above, making the effect of the variation of the parameters apparent. Specifically, we point out that varying the weights fa1 ; a2 g dilates vector fields fX 1 ; X 3 g respectively while varying a3 dilates the scale of the factor S1 of P. This will be particularly helpful when discussing the application of this model to the experimental data of concerning the shape of the distribution of the horizontal long range connection in the primary visual cortex. 3.3. Sub-Riemannian metric We define a second metric which is infinitesimally equivalent to a sub-Riemannian metric on P:
dP;2 ððx; y; h; qÞ; ðx0 ; y0 ; h0 ; q0 ÞÞ 1 ¼ a21 ððx x0 Þ cosðhÞ þ ðy y0 Þ sinðhÞ þ ðx x0 Þ cosðh0 Þ 2 1 2 þðy y0 Þ sinðh0 ÞÞÞ þ a22 ðððx x0 Þ sinðhÞ þ ðy y0 Þ cosðhÞÞ 2 ðx x0 Þ sinðh0 Þ þ ðy y0 Þ cosðh0 ÞÞj þ a23 ððcosðhÞ0 cosðh0 ÞÞ2 12 þðsinðhÞ0 sinðh0 ÞÞ2 Þ ð8Þ The j j in the term with coefficient a22 is simply the Euclidean absolute value. Just as with the Riemannian metric, the sub-Riemannian metric is invariant under the action of rotations and translations. Recalling the definition of the vector fields on P, this metric has the following rough characterization: starting at a point ðx; y; hÞ, travel is allowed in the h direction and in a spatial direction given by ðcosðhÞ; sinðhÞÞ and is severely restricted in the direction given by ð sinðhÞ; cosðhÞÞ. This property, together with the group invariance, is crucial to the choice of the metric as it reflects some experimental evidence, specifically the observed elongation of the connection pattern in the direction of the preferred orientation of the target neuron is echoed in the anisotropy of the metric. The sub-Riemannian metric reflects this as a metric ball centered at a point is elongated in the same fashion. As further motivation, we point out that the mixture of local and lateral connections used in Bressloff et al. (2001) can be interpreted as an approximation of the sub-Riemannian metric. Also, Citti and Sarti (2006), in modeling the network of simple cells in V1, have shown that the sub-Riemannian metric appears naturally in this context. Last, we again point out that the sub-Riemannian metric is the limit of a sequence of Riemannian metrics and hence, the second metric can be seen as a limit, in a certain sense, of a sequence of metrics related to the first metric (see, for example, Capogna et al. (2006) or Koranyi and Reimann (1995)).
2 2
a metric on P; dP , from the families of metrics described in the previous section. As discussed above, define the degenerate metric on P by dP ðx; x0 Þ ¼ dP ðpðxÞ; pðx0 ÞÞ where p is the projection of P to P. We use the following L2 energy to measure the distortion of the b: b ! C map f : P
0
112 X B C Eðf Þ ¼ @ ðdP ðx; x0 Þ dC ðf ðxÞ; f ðx0 ÞÞÞ2 A b 0 x;x 2 P We note that if we minimize this energy, we force the constants L; L0 in (1) and (3) to be as close as possible to one and to optimally bound (4). Thus, minimizing the energy ensures a combination of maximal smoothness and minimal wire length. We then use a standard Metropolis–Monte Carlo scheme to randomly perturb the map towards a minimum of this energy while slowly reducing the annealing temperature to zero over at least 3,000,000 trials. At this point, we assume we are in a neighborhood of a (local) minimum and switch to an exhaustive search of the map space to further decrease the energy. To do this we step b and calculate the energy change if that through each element of P b . If there is are element is swapped with each other element in P swaps which decrease energy, the swap of maximal decrease is selected. This procedure is repeated until there are no swaps which decrease energy. We emphasize that we do not place periodic boundary condib . The procedure requires, on average, roughly 1 b and C tions on P day of CPU time on a Thinkmate quad core AMD 2.66 GHz Opteron 64.
5. Results 5.1. Feature map formation For each metric, we perform the optimization on two different scales, on a small spatial scale where the h direction is larger than the spatial directions (i.e. a1 ; a2 a3 ) and again on a large spatial scale where the h direction is comparable to the spatial directions. The first simulation is to determine the local interaction between the orientation and retinotopic maps while the second simulation describes a coarser structure. 5.1.1. Small spatial scale In the regime of small spatial scales, all versions of the two metrics induced to the same feature map formation. As we see in Fig. 1a, we see a distinctive pinwheel form in the orientation map while the simulation produces significant distortion in the retinotopic map (Fig. 1b and c). This specific figure was generated using the sub-Riemannian metric with small spatial scale. 5.1.2. Large spatial scale For the Riemannian metric with a1 ¼ a2 ¼ a3 ¼ 18, the orientation map shows a regular (almost periodic) feature map comprised of interlocking pinwheel structures, Fig. 2a. In this case, the spatial maps exhibit a regular grid structure. Other trials in the same regime yield qualitatively similar maps. For the sub-Riemannian metric with a1 ¼ a2 ¼ a3 ¼ 18, the orientation map also produces interlocking pinwheel structures but the
50
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
Fig. 1. Small scale feature maps.
Fig. 2. Large scale feature maps.
resulting map is less regular. In addition, it includes longer ‘‘linear” regions as well as fractures (Fig. 2b). Looking at a different set of 1 , we see a similar picture to that of parameters a1 ¼ a2 ¼ 18 ; a3 ¼ 64
the initial sub-Riemannian case (Fig. 2c), indicating that the relative scales of a1 ; a2 and a3 do not change the qualitative structure of the maps.
Fig. 3. Connection patterns.
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
We note that, contrary to experimental evidence (see, for example, Das and Gilbert (1997)), our large scale simulations do not produce dislocations in the spatial maps. Given the distortion evident in the small scale maps, we suspect that this is a consequence of the coarseness of the sample in the simulation. In addition, our minimization method is a global optimization of the map. For a larger scale simulation (e.g. on the order of tens of thousands of sample points on a larger portion of cortical sheet), it may be more reasonable to instead use local rather than global optimization. We expect that a large scale simulation with finer sampling and local optimization will allow us to test these models against this particular data. 5.2. Connection patterns The connection patterns created by considering the images of metric balls in P under the map f result in elongated connection patterns along the axis of orientation, consistent with the tree shrew, cat and macaque data. Fig. 3 shows the images of metric balls and the distribution of orientations for the three different metrics described in the previous section. In the images of the balls, the solid black line is the image of a line in the direction of the preferred orientation of the center of the ball. The histograms are centered at the preferred orientation of the center of the ball and have bin widths of p8 radians. Fig. 3a shows the results for the Riemannian metric ð~ a ¼ ð1=8; 1=8; 1=8ÞÞ, yielding patchy long range connections which are virtually isotropic (although roughly rectangular). Fig. 3b shows the results from the first sub-Riemannian metric ð~ a ¼ ð1=8; 1=8; 1=8ÞÞ, yielding patchy connections and a distinct elongation along the axis of preferred orientation. Fig. 3c shows the results from the second sub-Riemannian metric ð~ a ¼ ð1=8; 1=8; 1=64ÞÞ, again yielding patchy connections but with virtually isotropic connection pattern. Taking into account the distortion due to ocular dominance regions (when present), these variants provide models consistent with the known observed data. 6. Conclusions In this work, we present a new geometric model for feature map formation in the primary visual cortex. We follow an idea of Cowey and build a map from a parameter space to the cortical sheet preserving smoothness and minimizing distortion. The main new aspect of our model is that we encode a geometry for the parameter space via a choice of metric and attempt to find a map which is as close as possible to an isometry, hence ensuring smoothness and the Cowey condition. Using two families of Riemannian and sub-Riemannian metrics, we find that various instances of these metrics reflect the basic small scale structure (in the form of pinwheel orientation maps) as well as observed larger scale structure. Moreover, different choices of metric lead to different patchy connection patterns in the model cortex. Specifically, the isotropic Riemannian metric produces an almost ‘‘crystalline” orientation map and highly isotropic connection patterns. Highly anisotropic Riemannian metrics are well approximated by sub-Riemannian metrics, where we find pronounced anisotropy of the connection patterns along the direction of preferred orientation. We also find that in varying the parameters of the sub-Riemannian metrics, we can easily change the level of distortion of the connection pattern from highly anisotropic to almost isotropic. The range of these models allows us to
51
capture all aspects concerning elongation of connection patterns of currently observed data. These models provide a mechanism for translating dynamical processes in the visual cortex into dynamical processes in the continuous model spaces. We expect that new analyses are possible in the model spaces due to the wealth of differential geometric tools available. As examples, we point out that (Citti and Sarti, 2006; Hladky and Pauls, preprint; Sarti et al., 2008) all use such tools on model parameter spaces to investigate contour completion, infilling and perceptual completion in the primary visual cortex. Our results support the connection between the network structure of the visual cortex and the continuous model spaces and provide finer detail as to the metric structures on the model spaces which correspond to experimental data. References Ben-Shahar, O., Zucker, S., 2004. Geometrical computations explain projection patterns of long-range horizontal connections in the visual cortex. Neural Comput. 16, 445–476. Bosking, W.H., Crowley, J.C., Fitzpatrick, D., 2002. Spatial coding of position and orientation in primary visual cortex. Nat. Neurosci. 5, 874–882. Bressloff, P., Cowan, J., Golubitsky, M., Thomas, P., Wiener, M., 2001. Geometric visual hallucinations euclidean symmetry and the functional architecture of the striate cortex. Philos. Trans. R. Soc. Lond. B 356, 299–330. Capogna, L., Danielli, D., Pauls, S., Tyson, J., 2006. An introduction to the Heisenberg group and the sub-Riemannian isoperimetric problem. Progress in Mathematics, vol. 259. Birkhauser. Citti, G., Sarti, A., 2006. A cortical based model of perceptual completion in the rototranslation space. J. Math. Imaging Vis. 24 (3), 307–326. Cowey, A., 1979. Cortical maps and visual perception. Quart. J. Exp. Psychol. 31, 1– 17. Das, A., Gilbert, C.D., 1997. Distortions of the visuotopic map match orientation singularities in primary visual cortex. Nature 387, 594–598. Durbin, R., Mitchison, G., 1990. A dimension reduction framework for understanding cortical maps. Nature 343, 644–647. Durbin, R., Willshaw, D., 1987. An analogue approach to the travelling salesman problem using an elastic net method. Nature 326, 689–691. Gilbert, C.D., Wiesel, T.N., 1993. Clustered intrinsic connections in cat visual cortex. J. Neurosci. 3, 1116–1133. Hladky, R., Pauls, S.D., preprint. Minimal surfaces in the roto-translation group with applications to a neuro-biological image completion model. Hubel, D.H., Weisel, T.N., 1977. Functional architecture of macaque monkey visual cortex. Proc. R. Soc. Lond. (Biol.) 198, 1–59. Kisvarday, Z., Toth, E., Rausch, M., Eysel, U., 1997. Orientation-specific relationship between populations of excitatory and inhibitory lateral connections in the visual cortex of the cat. Cereb. Cortex 7, 605–618. Kohonen, T., 2001. Self Organizing Maps. Springer-Verlag. Koranyi, A., Reimann, M., 1995. Foundations for the theory of the quasi-conformal mappings of the Heisenberg group. Adv. Math. III (1), 1–87. Koulakov, A., Chklovskii, D., 2001. Orientation preference patterns in mammalian visual cortex: a wire length minimization approach. Neuron 29, 519–527. Malach, R., Amir, Y., Harel, M., Grinvald, A., 1993. Relationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo targeted biocytin injections in primate striate cortex. PNAS 90, 10469–10473. Maldonado, P., Godecke, I., Gray, C., Bonhoeffer, T., 1997. Orientation selectivity in pinwheel centers in cat striate cortex. Science 276 (5318), 1551–1555. Petitot, J., 2003. The neurogeometry of pinwheels as a sub-Riemannian contact structure. J. Physiol. 97, 265–309. Petitot, J., Tondut, Y., 1998. Vers une neuro-geometrie. fibrations corticales, structures de contact et contours subjectifs modaux. Math., Inform. Sci. Humaine, EHESS, Paris 145, 5–101. Sarti, A., Citti, G., Petitot, J., 2008. The symplectic structure of the primary visual cortex. Biol. Cybern. 98, 33–48. Schmidt, K., Goebel, R., Lowel, S., Singer, W., 1997. The perceptual grouping criterion of colinearity is reflected by anisotropies of connections in the primary visual cortex. Eur. J. Neurosci. 9 (5), 1083. Sincich, L., Blasdel, G., 2001. Oriented axon projections in primary visual cortex of the monkey. J. Neurosci. 21 (12), 4416–4426. Swindale, N.V., 2004. How different feature spaces may be represented in cortical maps. Comput. Neural Syst. 15, 217–242. Yu, H., Farley, B., Jin, D., Sur, M., 2005. The coordinated mapping of visual space and response features in visual cortex. Neuron 47, 267–280.
Contents lists available at ScienceDirect
Journal of Physiology - Paris journal homepage: www.elsevier.com/locate/jphysparis
Cortical feature maps via geometric models Scott D. Pauls Department of Mathematics, Dartmouth College, Hanover NH 03755, United States
a r t i c l e
i n f o
Keywords: Visual cortex Sub-Riemannian geometry Cortical maps
a b s t r a c t We present a new model for feature map formation in the primary visual cortex, building on dimension reduction/wire length minimization techniques. We create a model space of feature parameters, endowed with various geometries picked to reflect physical or experimental data and search for a map from the parameter space to the cortical sheet which minimizes distortions. Upon simulating these maps, we find a family of Riemannian and sub-Riemannian geometries which give rise to feature maps which reflect known experimental data concerning (1) the qualitative arrangement of orientation maps and (2) the distribution of connections. One of the main findings is that experimental data showing both elongated and non-elongated connection patterns are represented within our family of models. Ó 2009 Elsevier Ltd. All rights reserved.
1. Introduction Within the primary visual cortex there are cells with preferential responses to a variety of stimulus features. The pattern of these cells form overlayed maps for each stimulus feature, such as position, orientation and ocular dominance. Numerous models have been devised to help understand the formation of feature maps in accordance with experimental data. Central to many of these models is the notion of dimension reduction, due to Durbin and Mitchison (1990), who view the maps as arising from a mapping from a two dimensional sheet of cortex, C to a high dimensional parameter space, P, of features. Most dimension reduction schemes impose reasonable (and experimentally observed) constraints on the mapping such as smoothness of the resulting feature map and completeness of coverage. In particular, we recall the elasticnet algorithm (Durbin and Willshaw, 1987) and the self-organizing feature map algorithm (Kohonen, 2001) which were both successful in producing feature maps qualitatively consistent with experimentally observed feature maps. These techniques and their variants have received a great deal of attention (see, for example, Swindale (2004), Yu et al. (2005) and the references therein). In a related direction, Koulakov and Chklovskii (2001) added an additional hypothesis (first proposed by Hubel and Weisel (1977)), that of wire length minimization, that coupled with the self-organizing algorithm formed different types of feature maps depending on the nature of the connection distribution of each neuron. The purpose of this work is to further study the relation between the distribution of connections between neurons in the primary visual cortex and the feature maps via a dimension reduction/wire length minimization model which seeks to find an appropriate geo-
E-mail address: [email protected] 0928-4257/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.jphysparis.2009.05.003
metric structure on the parameter space P which will reflect both the connection distribution and produce plausible (with respect to experimental evidence) feature maps. Of the main experimental results of interest in our model are the findings of patchy patterns of connections between neurons in the visual cortex which preferentially connect to neurons of similar preferred orientation (in the cat (Gilbert and Wiesel, 1993; Kisvarday et al., 1997), the tree shrew (Bosking et al., 2002) and the macaque (Malach et al., 1993)). Moreover, in some mammals, the connection patterns are elongated along a direction corresponding to the preferred orientation of the injection site (tree shrew (Bosking et al., 2002), cat (Schmidt et al., 1997) and squirrel monkey (Sincich and Blasdel, 2001)). While this anisotropy is quite pronounced, we recall that in mammals with well defined ocular dominance columns, the majority of the anisotropy can be accounted for by the separation imposed by the ocular dominance columns. Our method, described in detail in the next sections, focuses on a parameter space P ¼ P ½0; 1 ¼ ðR2 S1 Þ ½0; 1 where R2 gives the spatial position, and ðcosðhÞ; sinðhÞÞ 2 S1 gives the preferred orientation angle h and q 2 R gives the tuning strength. We take a disb , respectively, and create a b and C crete sampling of P and C; P b which satisfies both the goal b ! C dimension reduction map, f : P of smoothness and wire length minimization. To do this we propose a new method: we equip P and C with metrics and seek a map which minimizes metric distortion. This has two consequences. First, it links as closely as possible the geometry of the network of connections in the cortex and the geometry of the parameter space. Second, as discussed below, the distortion minimization imposes both smoothness and wire length minimization on the resulting cortical feature maps. We adapt the metric dP , and hence the geometry of P, to the available experimental evidence. More specifically, we find that to produce plausible feature maps, dP must have three properties:
47
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
a. It must be invariant under the group of rotations and translations of the plane. b. dP is degenerate, i.e. there are distinct points x; x0 2 P with dP ðx; x0 Þ ¼ 0. More specifically, dP may not depend on the variable q. c. Metric balls exhibit an anisotropy reflecting the preferential connection of neurons with similar feature parameters. The first condition is motivated by the ability of the visual cortex to process stimuli equally well under the same invariance: the rotation or translation of an image is just as readily processed by V1 as the untransformed version. In the literature, this is often called a ‘‘shift-twist” invariance and has had wide and fruitful use in understanding aspects of the visual system (see, for example, Bressloff et al. (2001), Ben-Shahar and Zucker (2004), Citti and Sarti (2006), Sarti et al. (2008)) and applications to computer vision and image processing. The second condition is crucial to the formation of feature maps reflecting the experimental findings of Maldonado et al. (1997) while the third condition is essential in reflecting of experimental literature describing the connection distributions in various mammals. We use a Metropolis–Monte Carlo technique to deform random cortical maps towards distortion minimizers using two different families of metrics on P ¼ R2 S1 : an invariant Riemannian metric and an invariant sub-Riemannian metric. These metrics are lifted, via projection, to degenerate metrics on P. Our simulation indicates that the sub-Riemannian metric on the parameter space induces a more plausible cortical orientation map than the Riemannian metric and exhibits patchy connection patterns and, under suitable choices of the parameters in the definition of the metric, both elongated and non-elongated connection maps. In particular, this complements the work of Citti and Sarti (2006), Petitot (2003) and Petitot and Tondut (1998), who also found and exploited sub-Riemannian models from different perspectives.
2. Dimension reduction, wire length minimization and smoothness As the visual cortex encodes information from a high dimensional space of parameters in what is essentially a two dimensional sheet, it is a fundamental problem to understand and/or model mechanisms by which the feature maps are created. In 1977, Hubel and Weisel (1977) suggested that the structure of orientation preference maps arise from evolutionary pressure to minimize cortical volume while maintaining complete parameter coverage. Durbin and Mitchison (1990) proposed a ‘‘dimension reduction” approach to this optimization, viewing cortical maps as mappings from parameter space to a cortical sheet and attempting to create maps which achieve complete coverage while minimizing total connection length. As outlined in the introduction, a number of authors have used a variety of algorithms to produce simulated feature maps. Our approach falls in the category of dimension reduction/wire length minimizing schemes but has significant novel features. In particular, we point out four new aspects of our approach: i. In the spirit of Cowey’s original argument (Cowey, 1979), we build a map from the space of parameters to the cortex whereas other work builds a map from the cortex to parameter space. ii. We use the geometry of the parameter space, determined by its metric, to define connections between neurons (or neuron populations) described by those parameters. iii. We impose an invariance under rotations and translations on the geometry of the parameter space.
iv. We attempt to optimize our map by ensuring both minimal wire length and smoothness by enforcing compatibility between the geometry of the parameter space and that of the cortex. To fix notation, we denote the parameter space by P and the cortical sheet by C. As noted in Durbin and Mitchison (1990), there are two competing ideas which guide the formation of such maps. First, the experimentally observed smoothness of the orientation maps would imply that nearby cortical positions should correspond to nearby parameter values. Second, we require that close parameter values should map to close cortical positions. This condition was first put forth by Cowey (1979). One way to express these two ideas is in terms of metrics on P and C, denoted dP and dC , respectively. For example, the smoothness condition can be expressed as a co-Lipschitz condition:
LdP ðx; x0 Þ 6 dC ðf ðxÞ; f ðx0 ÞÞ
ð1Þ
where f : P ! C and L is a constant. Similarly, the Cowey condition can be expressed as a Lipschitz condition:
dC ðf ðxÞ; f ðx0 ÞÞ 6 L0 dP ðx; x0 Þ
ð2Þ
where, again, L0 is a constant. We make the following trivial observation: maps satisfying (1) may have distant cortical areas with close parameter values while those satisfying (2) may have distant parameter values mapped to nearby cortical areas. Since pinwheels and fractures in orientation maps correspond to mappings of relatively distant parameter values to adjacent cortical areas, we see that maps exhibiting such singularities would likely violate (1) with L close to 1 but could easily satisfy (2) with L0 close to 1. Of course, the existence of a biLipschitz map on continuous spaces P and C of different dimension cannot exist but such maps are attainable on fib be fixed disb and C nite discrete samplings of these spaces. We let P b a b ! C crete subsets of P and C, respectively and call a map f : P cortical map. In terms of cortical wire length minimization, we assume that neural connections are determined by proximity in parameter space – i.e. cells with similar parameter values are more likely to be connected than those with disparate values. This is consistent with experimental data showing that neurons are connected to other neurons in V1 with similar response characteristics. Gib , we define the conven a cortical map f and a fixed point x 2 P b as the image of a nections between f ðxÞ and other points of C metric ball containing x; f ðBP ðx; rÞÞ (the radius of the ball, r, will be determined later via examination of the relative scales of b ðP; dP Þ and ðC; dC Þ). If we denote the connection relation in C b rÞ, the total cortical wire b by Bðx; by f ðxÞ f ðx0 Þ and BP ðx; rÞ \ P length for the
X
X
f ðxÞ2b C
f ðx0 Þf ðxÞ
dC ðf ðxÞ; f ðx0 ÞÞ ¼
X b x2 P
x0 2b B ðx;rÞ
dC ðf ðxÞ; f ðx0 ÞÞ 6
X
L0 dP ðx; x0 Þ
b x2 P
x0 2b B ðx;rÞ
Thus, under the assumption that the geometry of the parameter space models the connections well, (2) with L0 as small as possible ensures low total cortical wire length. We also wish to allow the possibility of degenerate metrics on P. If P ¼ P P 0 and
dP ðx; x0 Þ ¼ dP ðpðxÞ; pðx0 ÞÞ where dP is a (non-degenerate) metric on P and p is the projection of P onto P, the Cowey condition (2) takes the following form: For b with pðxÞ – pðx0 Þ: every x; x0 2 P
dC ðf ðxÞ; f ðx0 ÞÞ 6 L0 dP ðpðxÞ; pðx0 ÞÞ 0
where L is a constants.
ð3Þ
48
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
As in the non-degenerate case, we consider the choice of metric, dP , on P, in this context, equivalent to specifying the number and nature of the connections between a fixed neuron and other neurons in the cortex. Fixing a radius r, we define the set of neurons b . We again b to be BP ðx; rÞ \ P connected to a fixed neuron x 2 P emphasize that condition (3) with L0 close to one, coupled with a bound on
X
X
bÞ y2pð P
x;x0 2p1 ðyÞ
dC ðf ðxÞ; f ðx0 ÞÞ
ð4Þ
ensures that the total wire length is as small as possible. We note that this approach is quite different from that of Koulakov and Chklovskii (2001) where the authors specify a connection distribution only and create a map which minimizes total wirelength with respect to the L2 energy using Euclidean metrics on both the parameter and cortical spaces. Our approach differs in that we allow for different metrics on P and C and use the metric on P to define the connections themselves and, as a consequence, the distribution of connections. Moreover, (Koulakov and Chklovskii (2001)) works with a single parameter, orientation, while our model deal with four: two spatial variables, orientation and tuning strength.
coordinates ðx; y; hÞ. Each point ðx; y; hÞ is thought of as the composition of two transformations, a rotation by h,
Rh ðx; y; h0 Þ ¼
sinðhÞ
and a translation T ðx;yÞ ðx0 ; y0 ; h0 Þ ¼ ðx þ x0 ; y þ y0 ; h0 Þ. The group action is given by composition:
ðx; y; hÞ ðx0 ; y0 ; h0 Þ ¼ T ðx;yÞ Rh T ðx0 ;y0 Þ Rh0 ¼ T ðx;yÞ ðx0 cosðhÞ y0 sinðhÞ; x0 sinðhÞ þ y0 cosðhÞ; h0 þ hÞ ¼ ðx þ x0 cosðhÞ y0 sinðhÞ; y þ x0 sinðhÞ þ y0 cosðhÞ; h0 þ hÞ
e h ðx; yÞt ; hÞk jðx; y; hÞjRT ¼ kð R cosðhÞ 1 sinðhÞ . Then, we can define a left invarisinðhÞ cosðhÞ ant metric based on this norm as follows: eh ¼ where R
3. Parameter space
3.1. The roto-translation group Before describing the various metrics, we recall an important property of the visual system: invariance under rotations and translations of the plane. This is often modeled by imposing a group invariance on the structure of the retinotopic and orientation maps. We will incorporate this feature by defining metrics which are invariant under this group operation. To begin, we define the group itself: Definition 1. We call the group, RT, of rotations and translations acting on the two dimensional plane the rototranslation group and represent it as follows: RT is diffeomorphic to R2 S1 with
ð5Þ
We may equip RT with a norm which is invariant under the group action by specifying a fixed norm k k on R3 and defining
dRT ððx; y; hÞ; ðx0 ; y0 ; h0 ÞÞ ¼
We turn next to the description of the parameter space and, importantly, of its geometry via the choice of metric. In contrast to the space C, which is always viewed as a subset of R2 with the Euclidean metric, our choice of metric on the parameter space will vary. We use the parameter space P ¼ P ½0; 1 ¼ ðR2 S1 Þ ½0; 1 denoted by variables fx; y; h; qg where ðx; yÞ denotes position, h denotes orientation tuning and q denotes the strength of orientation tuning. We introduce two families of metrics on P ¼ R2 S1 based on Riemannian and sub-Riemannian geometries to study their effect of this choice on the resulting cortical maps. We intentionally omit a dependence on the tuning strength, q. Thus, technically, we will define degenerate metrics on P, based on nondegenerate metrics on P ¼ R2 S1 . We make this choice in order to again fit with experimental evidence. Upon including the tuning strength parameter in the metrics, where the metric on S1 was extended, via polar coordinates, to a metric on S1 ½0; 1, our simulations reveal a ‘‘tuning strength map” with very regular features, which is inconsistent with Maldonado et al. (1997). Specifically, the tuning strength maps exhibit a high correlation between pinwheel centers and lower tuning strength, whereas (Maldonado et al. (1997)) shows cells of strong tuning close to pinwheel centers. We note further that, when including a dependence on tuning strength, the orientation maps for the Riemannian metrics no longer resembled experimentally observed maps while the orientation maps for the sub-Riemnannian metrics remained consistent with experimental data.
x ; h0 þ h y cosðhÞ
cosðhÞ sinðhÞ
1 ðjðx; y; hÞ1 ðx0 ; y0 ; h0 ÞjRT 2 þ jðx0 ; y0 ; h0 Þ1 ðx; y; hÞjRT Þ
e h ðx; yÞt ; hÞ and it Direct computation shows that ðx; y; hÞ1 ¼ ð R then follows easily that dRT is invariant under the action of RT on itself and under inversion. We review another way of creating invariant metrics on R using a different presentation. Identifying RT with P ¼ R2 S1 with coordinates ðx; y; hÞ, we define the following three vector fields which span the Lie algebra, rt, of RT:
X 1 ¼ cosðhÞ@ x þ sinðhÞ@ y X2 ¼ @h
ð6Þ
X 3 ¼ sinðhÞ@ x þ cosðhÞ@ y Notice that ½X 1 ; X 2 ¼ X 3 and ½X 3 ; X 2 ¼ X 1 are the only nontrivial bracket relations. If we choose a norm, nðÞ, on rt we define a left invariant metric associated to n by
dF ðx; yÞ ¼ inf
Z
c2A
nðc_ Þ
c
where A is the set of all absolutely continuous paths connecting x to y. This metric is, in general, a Finsler metric but if n is derived from an inner product, it is Riemannian. We may define a sub-Riemannian metric on RT by placing an inner product, h; i on a bracket generating subbundle of the tangent bundle, spanned by fX 1 ; X 2 g and defining
dsR ðx; yÞ ¼ inf
c2A0
Z
1
hc_ ; c_ i2
c
where A0 is the set of all absolutely continuous paths connecting x to y so that their derivatives, when they exist, are in spanfX 1 ; X 2 g. We now define two families of metrics on P ¼ R2 S1 . The first is the infinitesimal form of a family of Riemannian metrics and the second is the infinitesimal form of a family of sub-Riemannian metrics. By abuse of notation, we will refer to them as the Riemannian and sub-Riemannian metrics, respectively. As we shall see in the next section, we use the infinitesimal forms for computational convienience. We note that sub-Riemannian metrics in this setting can be realized as the limiting cases of families of Riemannian metrics where the length of one vector field, X 3 , tends to infinity. Thus, these two families could also be seen as one family of Riemannian metrics with certain limiting cases included.
49
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
3.2. Riemannian metric
4. Simulation methods
The first metric is infinitesimally equivalent to a standard Riemannian metric on P. We pick constants fa1 ; a2 ; a3 g
In each simulation, we begin with a random map from a b , taken from 1 ; 1 2 ½0; 2p ½0; 1 P 8 8 8 8 grid, P 2 2 1 1 2 b taken from ; ¼ P ½0; 1 to a 64 64 grid, C C. We select
dP;1 ððx; y; h; qÞ; ðx0 ; y0 ; h0 ; q0 ÞÞ 1 ¼ a21 ððx x0 Þ cosðhÞ þ ðy y0 Þ sinðhÞ þ ðx x0 Þ cosðh0 Þ 2 1 2 ððx x0 Þ sinðhÞ þ ðy y0 Þ cosðhÞ þðy y0 Þ sinðh0 ÞÞÞ þ a22 2 2
ðx x0 Þ sinðh0 Þ þ ðy y0 Þ cosðh0 ÞÞÞ þ a23 ððcosðhÞ0 cosðh0 ÞÞ2 12 þðsinðhÞ0 sinðh0 ÞÞ2 Þ ð7Þ Extending the action of R to P via T ðx;yÞ Rh ðx; y; h; qÞ ¼ ððx; yÞt e h ðx0 ; y0 Þt ; h þ h0 ; qÞ, we see that the dP;1 is invariant. We emphasize þR that while the standard Euclidean metric is also invariant under the group action but that the Riemannian metric has the advantage that the group action acts nicely on the basis of vector fields described above, making the effect of the variation of the parameters apparent. Specifically, we point out that varying the weights fa1 ; a2 g dilates vector fields fX 1 ; X 3 g respectively while varying a3 dilates the scale of the factor S1 of P. This will be particularly helpful when discussing the application of this model to the experimental data of concerning the shape of the distribution of the horizontal long range connection in the primary visual cortex. 3.3. Sub-Riemannian metric We define a second metric which is infinitesimally equivalent to a sub-Riemannian metric on P:
dP;2 ððx; y; h; qÞ; ðx0 ; y0 ; h0 ; q0 ÞÞ 1 ¼ a21 ððx x0 Þ cosðhÞ þ ðy y0 Þ sinðhÞ þ ðx x0 Þ cosðh0 Þ 2 1 2 þðy y0 Þ sinðh0 ÞÞÞ þ a22 ðððx x0 Þ sinðhÞ þ ðy y0 Þ cosðhÞÞ 2 ðx x0 Þ sinðh0 Þ þ ðy y0 Þ cosðh0 ÞÞj þ a23 ððcosðhÞ0 cosðh0 ÞÞ2 12 þðsinðhÞ0 sinðh0 ÞÞ2 Þ ð8Þ The j j in the term with coefficient a22 is simply the Euclidean absolute value. Just as with the Riemannian metric, the sub-Riemannian metric is invariant under the action of rotations and translations. Recalling the definition of the vector fields on P, this metric has the following rough characterization: starting at a point ðx; y; hÞ, travel is allowed in the h direction and in a spatial direction given by ðcosðhÞ; sinðhÞÞ and is severely restricted in the direction given by ð sinðhÞ; cosðhÞÞ. This property, together with the group invariance, is crucial to the choice of the metric as it reflects some experimental evidence, specifically the observed elongation of the connection pattern in the direction of the preferred orientation of the target neuron is echoed in the anisotropy of the metric. The sub-Riemannian metric reflects this as a metric ball centered at a point is elongated in the same fashion. As further motivation, we point out that the mixture of local and lateral connections used in Bressloff et al. (2001) can be interpreted as an approximation of the sub-Riemannian metric. Also, Citti and Sarti (2006), in modeling the network of simple cells in V1, have shown that the sub-Riemannian metric appears naturally in this context. Last, we again point out that the sub-Riemannian metric is the limit of a sequence of Riemannian metrics and hence, the second metric can be seen as a limit, in a certain sense, of a sequence of metrics related to the first metric (see, for example, Capogna et al. (2006) or Koranyi and Reimann (1995)).
2 2
a metric on P; dP , from the families of metrics described in the previous section. As discussed above, define the degenerate metric on P by dP ðx; x0 Þ ¼ dP ðpðxÞ; pðx0 ÞÞ where p is the projection of P to P. We use the following L2 energy to measure the distortion of the b: b ! C map f : P
0
112 X B C Eðf Þ ¼ @ ðdP ðx; x0 Þ dC ðf ðxÞ; f ðx0 ÞÞÞ2 A b 0 x;x 2 P We note that if we minimize this energy, we force the constants L; L0 in (1) and (3) to be as close as possible to one and to optimally bound (4). Thus, minimizing the energy ensures a combination of maximal smoothness and minimal wire length. We then use a standard Metropolis–Monte Carlo scheme to randomly perturb the map towards a minimum of this energy while slowly reducing the annealing temperature to zero over at least 3,000,000 trials. At this point, we assume we are in a neighborhood of a (local) minimum and switch to an exhaustive search of the map space to further decrease the energy. To do this we step b and calculate the energy change if that through each element of P b . If there is are element is swapped with each other element in P swaps which decrease energy, the swap of maximal decrease is selected. This procedure is repeated until there are no swaps which decrease energy. We emphasize that we do not place periodic boundary condib . The procedure requires, on average, roughly 1 b and C tions on P day of CPU time on a Thinkmate quad core AMD 2.66 GHz Opteron 64.
5. Results 5.1. Feature map formation For each metric, we perform the optimization on two different scales, on a small spatial scale where the h direction is larger than the spatial directions (i.e. a1 ; a2 a3 ) and again on a large spatial scale where the h direction is comparable to the spatial directions. The first simulation is to determine the local interaction between the orientation and retinotopic maps while the second simulation describes a coarser structure. 5.1.1. Small spatial scale In the regime of small spatial scales, all versions of the two metrics induced to the same feature map formation. As we see in Fig. 1a, we see a distinctive pinwheel form in the orientation map while the simulation produces significant distortion in the retinotopic map (Fig. 1b and c). This specific figure was generated using the sub-Riemannian metric with small spatial scale. 5.1.2. Large spatial scale For the Riemannian metric with a1 ¼ a2 ¼ a3 ¼ 18, the orientation map shows a regular (almost periodic) feature map comprised of interlocking pinwheel structures, Fig. 2a. In this case, the spatial maps exhibit a regular grid structure. Other trials in the same regime yield qualitatively similar maps. For the sub-Riemannian metric with a1 ¼ a2 ¼ a3 ¼ 18, the orientation map also produces interlocking pinwheel structures but the
50
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
Fig. 1. Small scale feature maps.
Fig. 2. Large scale feature maps.
resulting map is less regular. In addition, it includes longer ‘‘linear” regions as well as fractures (Fig. 2b). Looking at a different set of 1 , we see a similar picture to that of parameters a1 ¼ a2 ¼ 18 ; a3 ¼ 64
the initial sub-Riemannian case (Fig. 2c), indicating that the relative scales of a1 ; a2 and a3 do not change the qualitative structure of the maps.
Fig. 3. Connection patterns.
S.D. Pauls / Journal of Physiology - Paris 103 (2009) 46–51
We note that, contrary to experimental evidence (see, for example, Das and Gilbert (1997)), our large scale simulations do not produce dislocations in the spatial maps. Given the distortion evident in the small scale maps, we suspect that this is a consequence of the coarseness of the sample in the simulation. In addition, our minimization method is a global optimization of the map. For a larger scale simulation (e.g. on the order of tens of thousands of sample points on a larger portion of cortical sheet), it may be more reasonable to instead use local rather than global optimization. We expect that a large scale simulation with finer sampling and local optimization will allow us to test these models against this particular data. 5.2. Connection patterns The connection patterns created by considering the images of metric balls in P under the map f result in elongated connection patterns along the axis of orientation, consistent with the tree shrew, cat and macaque data. Fig. 3 shows the images of metric balls and the distribution of orientations for the three different metrics described in the previous section. In the images of the balls, the solid black line is the image of a line in the direction of the preferred orientation of the center of the ball. The histograms are centered at the preferred orientation of the center of the ball and have bin widths of p8 radians. Fig. 3a shows the results for the Riemannian metric ð~ a ¼ ð1=8; 1=8; 1=8ÞÞ, yielding patchy long range connections which are virtually isotropic (although roughly rectangular). Fig. 3b shows the results from the first sub-Riemannian metric ð~ a ¼ ð1=8; 1=8; 1=8ÞÞ, yielding patchy connections and a distinct elongation along the axis of preferred orientation. Fig. 3c shows the results from the second sub-Riemannian metric ð~ a ¼ ð1=8; 1=8; 1=64ÞÞ, again yielding patchy connections but with virtually isotropic connection pattern. Taking into account the distortion due to ocular dominance regions (when present), these variants provide models consistent with the known observed data. 6. Conclusions In this work, we present a new geometric model for feature map formation in the primary visual cortex. We follow an idea of Cowey and build a map from a parameter space to the cortical sheet preserving smoothness and minimizing distortion. The main new aspect of our model is that we encode a geometry for the parameter space via a choice of metric and attempt to find a map which is as close as possible to an isometry, hence ensuring smoothness and the Cowey condition. Using two families of Riemannian and sub-Riemannian metrics, we find that various instances of these metrics reflect the basic small scale structure (in the form of pinwheel orientation maps) as well as observed larger scale structure. Moreover, different choices of metric lead to different patchy connection patterns in the model cortex. Specifically, the isotropic Riemannian metric produces an almost ‘‘crystalline” orientation map and highly isotropic connection patterns. Highly anisotropic Riemannian metrics are well approximated by sub-Riemannian metrics, where we find pronounced anisotropy of the connection patterns along the direction of preferred orientation. We also find that in varying the parameters of the sub-Riemannian metrics, we can easily change the level of distortion of the connection pattern from highly anisotropic to almost isotropic. The range of these models allows us to
51
capture all aspects concerning elongation of connection patterns of currently observed data. These models provide a mechanism for translating dynamical processes in the visual cortex into dynamical processes in the continuous model spaces. We expect that new analyses are possible in the model spaces due to the wealth of differential geometric tools available. As examples, we point out that (Citti and Sarti, 2006; Hladky and Pauls, preprint; Sarti et al., 2008) all use such tools on model parameter spaces to investigate contour completion, infilling and perceptual completion in the primary visual cortex. Our results support the connection between the network structure of the visual cortex and the continuous model spaces and provide finer detail as to the metric structures on the model spaces which correspond to experimental data. References Ben-Shahar, O., Zucker, S., 2004. Geometrical computations explain projection patterns of long-range horizontal connections in the visual cortex. Neural Comput. 16, 445–476. Bosking, W.H., Crowley, J.C., Fitzpatrick, D., 2002. Spatial coding of position and orientation in primary visual cortex. Nat. Neurosci. 5, 874–882. Bressloff, P., Cowan, J., Golubitsky, M., Thomas, P., Wiener, M., 2001. Geometric visual hallucinations euclidean symmetry and the functional architecture of the striate cortex. Philos. Trans. R. Soc. Lond. B 356, 299–330. Capogna, L., Danielli, D., Pauls, S., Tyson, J., 2006. An introduction to the Heisenberg group and the sub-Riemannian isoperimetric problem. Progress in Mathematics, vol. 259. Birkhauser. Citti, G., Sarti, A., 2006. A cortical based model of perceptual completion in the rototranslation space. J. Math. Imaging Vis. 24 (3), 307–326. Cowey, A., 1979. Cortical maps and visual perception. Quart. J. Exp. Psychol. 31, 1– 17. Das, A., Gilbert, C.D., 1997. Distortions of the visuotopic map match orientation singularities in primary visual cortex. Nature 387, 594–598. Durbin, R., Mitchison, G., 1990. A dimension reduction framework for understanding cortical maps. Nature 343, 644–647. Durbin, R., Willshaw, D., 1987. An analogue approach to the travelling salesman problem using an elastic net method. Nature 326, 689–691. Gilbert, C.D., Wiesel, T.N., 1993. Clustered intrinsic connections in cat visual cortex. J. Neurosci. 3, 1116–1133. Hladky, R., Pauls, S.D., preprint. Minimal surfaces in the roto-translation group with applications to a neuro-biological image completion model. Hubel, D.H., Weisel, T.N., 1977. Functional architecture of macaque monkey visual cortex. Proc. R. Soc. Lond. (Biol.) 198, 1–59. Kisvarday, Z., Toth, E., Rausch, M., Eysel, U., 1997. Orientation-specific relationship between populations of excitatory and inhibitory lateral connections in the visual cortex of the cat. Cereb. Cortex 7, 605–618. Kohonen, T., 2001. Self Organizing Maps. Springer-Verlag. Koranyi, A., Reimann, M., 1995. Foundations for the theory of the quasi-conformal mappings of the Heisenberg group. Adv. Math. III (1), 1–87. Koulakov, A., Chklovskii, D., 2001. Orientation preference patterns in mammalian visual cortex: a wire length minimization approach. Neuron 29, 519–527. Malach, R., Amir, Y., Harel, M., Grinvald, A., 1993. Relationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo targeted biocytin injections in primate striate cortex. PNAS 90, 10469–10473. Maldonado, P., Godecke, I., Gray, C., Bonhoeffer, T., 1997. Orientation selectivity in pinwheel centers in cat striate cortex. Science 276 (5318), 1551–1555. Petitot, J., 2003. The neurogeometry of pinwheels as a sub-Riemannian contact structure. J. Physiol. 97, 265–309. Petitot, J., Tondut, Y., 1998. Vers une neuro-geometrie. fibrations corticales, structures de contact et contours subjectifs modaux. Math., Inform. Sci. Humaine, EHESS, Paris 145, 5–101. Sarti, A., Citti, G., Petitot, J., 2008. The symplectic structure of the primary visual cortex. Biol. Cybern. 98, 33–48. Schmidt, K., Goebel, R., Lowel, S., Singer, W., 1997. The perceptual grouping criterion of colinearity is reflected by anisotropies of connections in the primary visual cortex. Eur. J. Neurosci. 9 (5), 1083. Sincich, L., Blasdel, G., 2001. Oriented axon projections in primary visual cortex of the monkey. J. Neurosci. 21 (12), 4416–4426. Swindale, N.V., 2004. How different feature spaces may be represented in cortical maps. Comput. Neural Syst. 15, 217–242. Yu, H., Farley, B., Jin, D., Sur, M., 2005. The coordinated mapping of visual space and response features in visual cortex. Neuron 47, 267–280.