Mosaic model for color vision

J. theor. Biol. (1975) 53, 177-197

Mosaic Model for Color Vision WHITMAN RICHARDS Massachusetts

Institute of Technology, Department Cambridge, Mass. 02139, U.S.A.

(Received 15 July 1974, and in revised,form

of Psychology,

4 December 1974)

Color vision in man is based upon three different cone types, which are quite likely arranged in a semi-ordered array in the retina. The model proposes that this ordering is an inherent part of the genetic code that sets up the color vision mechanism, and that the specification for each cone type (red, green or blue) also includes a specification for its place in the larger structure of which it is a part. One possible positional mosaic for the three cone types is proposed, together with its degeneracies into anomalous (red-green) color mechanisms. Assuming only one fixed probability for a degenerate transition, the population frequencies for color anomalies predicted from the model agree closely with the observed frequencies. 1. General Hypothesis Normal color vision in man is generally considered to be trichromatic, being based upon the relative absorptions of three different types of cone photopigments (Maxwell, 1855; Wright, 1928; Guild, 1931; Stiles & Burch, 1959; Speranskaya, 1959; Brown & Wald, 1964; Marks, Dobelle & MacNichol, 1964). Although rods do participate, their role is relatively minor and is observed only under special circumstances (Stiles, 1955; Bongard, Smirnov & Friedrich, 1958; Blackwell, 1961; Willmer, 1961; Clarke, 1963; Richards & Luria, 1964; Trezona, 1970, 1973; Palmer, 1972; McCann, 1972; Makous & Boothe, 1974). For the exposition of our model we will assume that the three types of cones are the sole mediators of color vision and that rods merely degrade the color signal. For the brain to detect differences in the activations of the different cone types it is necessary to encode the activities of the three cone types in three different ways, or “channels”, where each channel is to some degree independent of the other two. The proof that such an encoding takes place is the fact that three dimensions are necessary and sufficient to describe the space of all colors (the Munsell Color system is one example of many; Indow, 1974). Although it is not necessary that each cone type feed its own 177 T.B. 12

178

1%‘.

RICHARDS

independent channel, there must be some degree of organization in the convergence of cone types into the three channels, otherwise the original trichromacy of color would be lost. In particular, we would not expect that cones converge randomly onto various neural channels, but instead must have their neural connections specified to some degree. Such a specification of the neural linkages of the three types of cones need not require a chemical specificity that causes axons to reach out to their appropriate neural neighbor (Weiss, 1937; Sperry, 1945; Gaze, 1970). Instead, the proper positioning of local elements in the global retinal mosaic could produce the same result (Wolpert, 1969; Goodwin & Cohen, 1969; Crick, 1971). In fact, this is my general hypothesis, namely that when a genetic component is specified, the specification includes not only plans for its own formation, but also includes a specification for its place in the larger structure of which it is a part. Without proper organization of the elements in the larger structure, the encoding offered by each element will be worthless. As an analogy, an isolated letter has no meaning. To be useful the letter must have a specified position in a word, which in turn must be appropriately located in its sentence, etc. My fundamental hypothesis is that the genetic code for each cone type also includes a specification for the position of the cone in a larger array. The genetic code would thus specify an ideal retinal mosaic of the three cone types (plus rods), to be laid down on top of an appropriate neural network that follows the same global code for position. In practice, random fluctuations in position or even imposed gradients may distort the ideal cone mosaic. (The increasing percentage of rods as one proceeds away from the fovea to the peripheral retina could be considered an imposed gradient.) The genetic code only defines the best possible pattern; departures from this pattern, either random or non-random, cannot improve the coding, but only degrade the color signal. When errors in the genetic code itself occur, i.e. when the code fails, then the general properties of the aberrant mosaic will be reliably predictable, except for random fluctuations. The objective of this report is to present the major classes of cone mosaics and to describe their relation to normal and defective (red-green) color vision. 2. An Ideal Mosaic:

The Major Hypotheses

Because the retina is a two-dimensional sheet, the code for the placement of cone types must have two variables for position and one for pigment type. The two-dimensional positional code, however, need not formally define an x,y position on the retinal surface. Instead, the code could define the sequence of cone types in a string. However, in this kind of specification at least

MOSAIC

MODEL

FOR

COLOR

VISION

179

two such strings would then be necessary to define the complete twodimensional spatial relationships. For descriptive simplicity, I have chosen the “string” specification method for constructing a suitable ideal mosaic of three cone types. In the long run, many factors will constrain the final choice for the arrangement of cone types in a string. However, two factors that strongly influenced my present choice of strings were first the desire to generate a matrix with hexagonal symmetry, and secondly, the need to reflect a relative density of red, green and blue cones consistent with available data. Clearly, in the interests of parsimony, a two-dimensional matrix with some form of symmetry is desirable. Although there is no specific evidence favoring the chosen hexagonal mosaic, there is abundant evidence of regular packing of

C

.. b - .b

Close

to the yellow

spot

Periphery

FIG. 1. End-on appearance of rod (b) and cone (c) mosaic in a fresh preparation at the macular edge and peripheral retina (after Schultze, 1866, as shown in Pirenne, 1967).

receptors in human and other retinae (Hannover, 1843; Schultze, 1866; Denton & Wyllie, 1955; Polyak, 1957; Lyall, 1957; Dunn, 1966). Adams, Perez & Hawthorne (1974) presented photographs of fovea1 and peripheral receptor mosaics that look remarkably similar to some early sketches of the human retina prepared by Schultze in 1866. Figure 1 illustrates the honeycomb-like arrangement of receptors that the ‘(string” specification should approximate. The construction of’ the strings should also be dependent upon the population distribution of the three cone types. In the fovea1 region of greatest color discrimination, the density of “blue” cones is about one-twelfth that of the “red” and “green” cones (Walraven, 1962; Wald, 1964). In this same region, the density of red and green cones is roughly equal. To reflect these

180

W.

RICHARDS

distributions in the ideal two-dimensional matrix, two types of one-dimensional sequences of cones are assumed : one string (1) will be defmed in terms of red, green and blue cones, whereas the second string (2) will contain only red and green cones. These two strings, which are otherwise chosen post hoc, will then be integrated into a two-dimensional mosaic. Clearly, with equal arbitrariness we could have started by defining a regular twodimensional mosaic at the outset and then proceeded to point out its “string” components. By starting with the strings themselves, however, the relations between the components of the mosaic will be seen more clearly, and will be easier to relate to a proposed one-dimensional genetic code. Furthermore, individual perturbations in the local order can be described more simply, which will be of great advantage when “degenerate” mosaics are presented. Of the two basic strings proposed to illustrate the potential power of the mosaic hypothesis, the simplest is the string made up of only red and green cones : rrgrrgrrgrrgrr Quite simply, each green cone, g, is surrounded by two red cones, r. This string will be identified as string 2, because in the mosaic framework it is the secondary string. The primary string, about which the hexagonal structure is organized, is defined as follows : R

glo

&b

B

glb

810

R

ho

&lb

B

glb

glo

R.

Thus, two green cones, glb, appear on each side of the blue cone B, and these g,, cones are then abutted by two additional green cones g,, that lie next to the red cone R (subscripts are added for subsequent clarification of position). To emphasize the unique location of the red cone R, which will become the focal point of one part of a hexagonal array, the cone in this position is identified by a capital letter. The pigment type remains the same as the letter designation however, being selected from one of the three categories “red “, “green” or “blue”. For purposes of exposition, the maxima of these three photopigments will be assumed to be respectively near 570, 535 and 440 nm; an exact specification is not required for the model. The strings 1 and 2 may now be interlaced in such a manner to facilitate packing and to preserve symmetry. Such an interlacing requires that adjacent strings be shifted by one-half element with respect to each other. Furthermore, to distribute the blue cones more uniformly over the retina, alternate rows of string 1 have been shifted by one-half the length of a subsequence as shown in Fig. 2. The resultant pattern has hexagonal packing with onedimensional symmetry. Although other suitable mosaics could be constructed having similar properties, this particular mosaic has been chosen as an ideal

MOSAIC

MODEL

FOR

COLOR

181

VISION

pattern that could be obtained with complete genetic specification of cone type and position. This mosaic will be called a class I mosaic. An abstracted representation of the mosaic appears on the right side of Fig. 2, making the substitutions r = *, g = x, R = o (the substitution G = * will appear later). The abstracted version is included only to make the packing arrangement more apparent. The class I mosaic could have been constructed by specifying the neighbors for each of the cone types, for example by specifying that each B cone be String Siring Siring Siring String

2 I 2 I+ 2

rryrrgrr gRgyBggRg rrgrryrr gBygRggBg rrgrrgrr

l

-

xi-‘\x

xox~B,xxox

l

l

z ,

(i>x>p.Y.yj>

“Oss

I

FIG. 2. Class I mosaic, produced by shifting alternate rows of string 1 by one-half the length of a subsequence.

surrounded by two g and four r cones. However, the string specification has been introduced because it may provide further insight into how a twodimensional mosaic may be specified genetically in terms of sequences of elements -and relations between sequences. Such a higher order specification would, if impaired, permit larger areas of aberrations than if the specification were on an element by element basis. In effect, one would have a hierarchy of genetic coding where local order could be preserved but global order impaired, as might happen in certain heterozygotes (Vogt, 1942; Falls, 1951; Wald, Wooten & Gilligan, 1974). Regardless of the means for arriving at the class I mosaic, it should be noted that the mosaic does satisfy the constraint that red and green cones should outnumber the blue cones by about 12 to 1 (Walraven, 1962; Wald, 1964). The red-green-blue cone ratio is 25 : 30 : 5. These ratios are also appropriate for describing the photopic sensitivity of the normal retina (Wald, 1964; Vos & Walraven, 1972). 3. Reduced Mosaics The construction of a two-dimensional mosaic similar to the class I mosaic requires that six genetic elements be encoded to specify both photopigment type and position. Whenever the code fails, either by an incomplete or an improper specification, then the mosaic will be altered. In terms of the two-string characterization of a mosaic, these errors could be expressed in several different ways: (i) the absence of a receptor, (ii) the replacement of a receptor by a degenerate receptor-type, (iii) the replacement of one

182

W.

RICHARDS

receptor (i.e. red) by another (i.e. green), or (iv) by a displacement or exchange of receptor types along the string. Of these possibilities, some form of replacement of one receptor type by another is the favored form for introducing aberrations in the mosaic. In the case of the obvious need to alter the cone mosaic as one moves from the rod-free fovea to the rod-rich periphery, the simplest alteration is merely to replace certain (or all) cone types by rods with increasing frequency as eccentricity increases. The alteration is thus a programmed replacement of some receptor types (i.e. red and green cones) by another (more primitive?) receptor type (i.e. rod). Degenerate fovea1 mosaics could be constructed in the same manner, by proposing that whenever the genetic code fails, one of the (six) cone receptor elements in the mosaic is replaced by the degenerate receptor type (Schouten, 1935, 1937; MacLeod & Hayhoe, 1974). Although such reduced or degenerate mosaics can be shown to have interesting properties like those described below, an alternate form of the presumed nature of the genetic aberration is preferred to describe the properties and interrelations between various alterations in the class I mosaic. The preferred form of aberration is the replacement of one cone type by another “normal” cone type (i.e. a cone carrying one of the characteristic cone photopigments). Such a replacement will also include all cases of a displacement or an exchange of receptor types along a string. Furthermore, if the replacement option is restricted only to red and green cone types, then the replacement of one cone type by another can be viewed as merely a complementation of cone types (i.e. “red” photopigment is replaced by the characteristic “green” photopigment or vice versa). By restricting the alterations in cone mosaics to only complementation of the “red” and “green” cone types, a taxonomy of mosaics is greatly simplified. Yet, in spite of these simplifications, the mosaics will have surprising properties. (A)

COLOR

NORMAL

CLASS

II

Because the properties of color matches are constrained by the quanta1 absorptions of the cone photopigments, the trivariance of normal color matches will be based upon the spectral absorption of the three characteristic photopigments (Wyszecki & Stiles, 1967). As long as these three photopigments are present, then regardless of their relative proportions, color matches made by one color normal should in principle produce a satisfactory match for any other color normal (disregarding differences in the spectral absorption properties of the ocular media). Although this statement is not completely true, it is correct to a good first approximation for fovea1 vision (Stiles, 1955; Crawford, 1965; Richards, 1967). Thus, we may eliminate at least one class of red cones, R, from the mosaic I and still have an observer

MOSAIC

MODEL

FOR

COLOR

183

VISION

who accepts normal color matches. This observer, where the R cone in string 1 is now presumed to be complemented to a G cone type (symbolized by *), shall be designated a class II observer. Keeping the same symbolic notation, a class II observer would thus have a mosaic of the form shown in Fig. 3. Siring Siring Sfring String String

2 I 2 It 2

rrgrrgrr gGggBggGg rrgrrgrr gBggGggBg rrgrrgrr

*

5

FIG.

*

xi-‘\x

*

l

x*x fB,x x *x (,\,xX*; ,X(& .-.’ y, . . x .-I

Class II.

3. Class II mosaic.

Considering that only one-tenth of the cone receptors have been complemented, such a change should be hardly noticeable. Certainly color matches would be unaffected by this change. On the other hand, if the receptor R occupied a key position in the neural coding of color, then the replacement of R by G could lead to quite noticeable changes in color specification (such as color appearances and color naming). Appendix A gives an example of an underlying neural network whose color encoding properties could be seriously affected by complementing R to G. The basis for bringing about the change in the coding of color rests upon the assumption that the R location (0) is the focal point of the yellow-blue opponent color channel. If the spectral sensitivity of this focal point is changed from “red” to “green”, then the color coding properties of the network will be altered. (B)

COLOR

DEFECTIVE

MOSAICS

Classically, four types of red-green blindness and one type of blueblindness are described (Wright, 1946; Wyszecki & Stiles, 1967). Blueblindness, or tritanopia, is rare, appearing in the order of l/10,000; is not sex-linked (Wright, 1952; Henry, Cole & Nathan, 1964; Kalmus, 1965) and may occur as a result of a retinal disease (Krill, Smith & Pokorny, 1971). Blue-blindness appears to have an entirely different etiology from the more common red-green blindness, which is sex-linked. Because blue-blindness is a special type of blindness, special rules must be invoked. Although the resulting mosaic will be obvious (i.e. absence of the B cone), the derivation or probability considerations are obscure. Hence only red-green blindness will be considered in the mosaic model. Furthermore, because the red-green defects are sex-linked, the mosaic reductions will be illustrated only for males. When heterozygote females are encountered, the more complicated “salt and pepper” patchwork of the combination of two mosaics must be expected (Vogt, 1942; Falls, 1951; Wald et al., 1974).

184

W.

RICHARDS

(i) Protanotnalies An extensive loss of red cones will generally lead to either protanopia or protanomaly, depending upon the extent and distribution of the remaining red cones. According to the mosaic model, the critical property of protanomalies is that the red-green (rg) channel fails to encode chromatic information properly. The focal point of the rg channel is the blue receptor (see Appendix A). For the rg process to fail, all cones immediately adjacent to B must be green cones. In addition, the R cone must also be complemented to G, because this receptor also appears in the surround of the rg process. According to the model, the principal distinction between protanopia, where the observer is a complete dichromat and protanomaly, where he is a poor trichromat, is whether a few red cones remain in the neighborhood of the G receptor, thus contributing some “red” sensitivity to the surround of the red-green channel (see Appendix A). If they do, then the observer will require three primaries for all color matches and be classed as an anomalous trichromat even though two primaries might suffice for most matches. In Fig. 4, the anomalous and anopic mosaics may be compared (P-an and P-ope). In protanopia (bottom) all red cones are complemented to green cones and all original class I green cones remain uncomplemented. The mosaic thus has only green and blue cones. For protanomaly, some red cones remain. None of these cones are those found in the prototypic class I observer, however. The original (r) red cones in string 2 must be complemented in order that the rg channel cease to encode red-green information. In protanomaly the remaining red cones arise from complemented green cones that do not directly activate the rg channel. Two such green cone types satisfy this constraint, namely g,, adjacent to G at position (0) feeding the yb channel and g, which lies equidistant from B but in string 2. Both positions provide only minimal rg chromatic information. When green cones g,, in string 1 are complemented, the mosaic would appear as P-an,. When green cones (gJ in string 2 are complemented, the mosaic is P-an,, as shown in Fig. 4. In protanomalous observers, it is generally believed that the residual red receptor does not have an absorption spectrum typical of the red receptor found in normals or in dichromats (Wright, 1946; Pokorny, Smith & Katz, 1973). This possibility will be considered subsequently when the genetic code for the strings is characterized. (ii) Deuteranomalies The second kind of complete red-green reduction occurring in males is the absence of green cones leading to deuteranopia. Like protanomalies,

MOSAIC

MODEL rrgrrgrr gBggRggBg rrgrrgrr gRggBggRg r rgrrgrr

FOR

COLOR

185

VISION

~yf,“,‘,“~,~ ‘.J x .-. xoxx/B\xxo~ . . x ‘.-.’

.-.

x 1-1 x . .

x . . x .-•

II

rrgrrgrr gBggGggBg rrgrrgrr gGggBggGg rrgrrgrr

D-an,

rrrrrrrr rBrgRgrBr rrrrrrrr

.-. . . . . .(B> x o x {B) .. . . . . . .-

D-an,

rrgrrgrr rRrrBrrRr rrgrrgrr

.o..

D-ope

.

.

x

.-

.

.

x;B)x..o.-

x

.

.

rrrrrrrr rBrrRrrBr rrrrrrrr

P-an,

P-an,

P-ope

ggrggrgg gGggBggGg ggrggrgg

* x-x xx x*x,!B)(x*x x x ’ x-x

* x x

4999499g gBggGggBg gggggggg

,%X x x x x jy x\Bkx*xxBjc x-ix x x xl-x

* x x

FIG. 4. Mosaics for the six major classes of color normal and color anomalous males. B = blue cone (B) ; . = red cone (r); o = red cone (R); x = green cone (6); * = green cone (G).

the critical property to encode chromatic focal point of the rg all cones immediately that green cones g,,

of deuteranomaly is also a failure in the rg channel information. Once again taking the B cone as the channel (see Appendix A), we now must require that adjacent to the B cone be red cones. This requires must be complemented. For deuteranopia, all other

186

W.

RICHARDS

green cones g,, and g, are also complemented to red cones, leaving a degenerate mosaic of red and blue cones. In a manner analogous to protanomaly, the principal distinction between the anopic and anomalous observer is whether or not a few green cones remain outside the center of the rg channel. For deuteranomal type 1 (D-an,), the g2 cones in string 2 are complemented, but in string 1 the original gr,, cones adjacent to R remain intact, thus leaving the deuteranomalous observer with three cone photopigments but only one normally functional chromatic channel. For the type 2 deuteranomal (D-an,), all the g, cones in string 1 are complemented to r, while the g, cones remain unchanged. Again, like protanomaly, the residual normal green pigment in deuteranomalous observers need not have a characteristically normal absorption spectrum. Its &,aX, if abnormal, could be shifted to longer wavelengths by a distortion in the fabrication of its opsin by the gene sequence. This possibility will be elaborated shortly. Unlike protanopes, deuteranopes and deuteranomals have a great likelihood of capitalizing on rod function because the A,,, for rods and red cones are quite distant. Random aberrations in the execution of the genetic code that affect rod placement could thus lead to many variations from the nonideal mosaic. Not unexpected, therefore, is to find a range of severity and complexity of deuteranomaly that goes beyond the proposed type 1 and 2 difference (Wright, 1946 ; Pickford, 1968; Bender, Ruddock, De VriesDe Mol & Went, 1972). As long as the rod response is not inhibited by the active cones (Makous & Boothe, 1974), even a red-yellow-green discrimination might be elicited from a deuteranope (Jaeger & Kroker, 1952). However, the “yellow” neutral point must then be shifted toward shorter wavelengths due to the shortwave shift in A,,,, of g to the rods. This appears to be the case in the most extensively studied deuteranope (Luria, 1967). 4. The Gene Code

The mosaics illustrated in Fig. 4 are the result of an elaborate machinery that builds its product from a simple code word, properly expressed and organized. This code word, a genetic one, is a one-dimensional sequence. How might it be characterized ? The simplest characterization is merely to present the elements of the code in the sequence in which they will appear in the mosaic. Putting aside instructions for replicating the sequence, a possible characterization is given in Fig. 5. Here, the strings 1 and 2 of the class I mosaic are characterized (in an abstract sense only!) as lying in a line with an appropriate sequence of the six red or green elements.

MOSAIC

MODEL

FOR

COLOR Gene

CIOSS

I

m B

I

rglb

1 glo

II

code String

Siring

x

187

VISION

i

R’

l

9; i

2

‘20

[

‘Zb

x

0

x

-

*

gin,

G

ge

rzo

‘2b

0

D-an,

546 D-on,

D- ope

P-an,

P-an2

420

g2b

%

42

557 P-ope

m

B

g,

i

91

G

gz

535

FIG. 5. Characterization of a gene segment encoding cone type and position for the various red-green mosaics. Hypothetical 3,max are shown below several code elements to indicate the effect of an energy bond shift produced by embedding an isolated r or g element in a large number of its complements.

At this level, the critical transition from a class I to II observer comes about by complementing R to G. The remainder of the sequence is unchanged. For the anomalous observers, several complementations are required, as indicated. These all serve to produce a reduced mosaic according to Fig. 4. Here, however, the transitions are easier to observe. Note that in some cases, such as in string 2 of P-anz, a green element is complemented to red while its red neighbor is complemented to green. This positional specification of the element is an essential feature of the mosaic model. For class I observers as well as for D-an’s and P-an’s, the code sequence of Fig. 5 has at least one element embedded between its complement. These types of mosaic specification are the only strong instances of such embedding. Thus, for D-an,, the g10 green receptor element (adjacent to R) is embedded between two red receptor elements. Under these circumstances, it is possible

188

W.

RICHARDS

that the replication of the photopigment g may not proceed normally, with a slightly deformed opsin causing an energy bond shift in the isomerization process, thereby yielding an anomalous green pigment (Dartnall & Lythgoe, 1965). This possibility is indicated by the prime as well as by the i,, notation below the element. The exact specification for such a construction must remain ambiguous, however, because of the great difficulties encountered in accurately describing the “anomalous” pigments of the anomalous trichromats (Wald, 1966; Alpern & Torii, 1968; Brindley, 1970; Rushton, Powell & White, 1973; Pokorny et al., 1973). 5. Population Frequencies of Mosaic Types One of the striking properties of the mosaic model is the prediction of the population frequencies of red-green color deficits in males from only one mutation rate applied to both red and green cones. In particular these frequencies can be predicted by making the simplifying assumption that the population frequency for encountering a red to green complementation will be the same as for encountering the opposite green to red complementation. As seen from the viewpoint of the steady-state population, the model’s probability of encountering the complementation of a red or green cone is close to 0.3 (among males). The near identity of this statistic raises the interesting question of whether or not the breakdown of each cone type has a similar basis, as might be expected from their proximity on the genetic substrate (Jackson et al., 1964). Certainly all evidence points to a close linkage between the sex-linked red-green receptor types, while the autosomal dominant blue receptor remains distant. By our definition, the protanopic observer has neither the R nor the r type red receptor, but retains all g green receptors. This definition may be symbolically represented as [R, i?; g, G] with the bar above the receptor type indicating its absence by complementation. (The notation is intentionally redundant for clarity; the semi-colon separates the redundant symbols, colons will indicate exceptions to preceding notations within the brackets.) Assuming the probability for complementing a red or green receptor is X, then the probability of complementing all red receptors (r, rZO and rZb) but retaining all green receptors (glO, g,, and g2) will be x3fl -x-J”. If x = 0.3, th e probability of protanopia is thus about 0.01. Protanomaly is similar to protanopia in requiring the presence of the green receptor g,, adjacent to R = G, thus impairing color processing by the rg channel. In addition, however, protanomaly also has a residual red receptor arising by complementing either g, or g,, (the green cone adjacent to G at position 0). Symbolically, protanomaly may thus be represented

MOSAIC

MODEL

FOR COLOR

as [a, glb, 3: g, n gl,], Its predicted population

VISION

probability

189

is

2.x4. [l -x-J” = 0,008 for x = 0.3.

Table 1 summarizes these results, and also includes in the final column the average population estimates taken from LeGrand (1968). The agreement is good, particularly considering that the incidence of protanopia is estimated anywhere from 0-009-O-016 while the range from protanomaly is 0*006-0~013. TABLE 1 Population frequencies for normal and for anomalous red-green color blindness predicted from the mosaic model. The transition probability for red-green cone complementation is assumed fixed at 0.3. Semicolons separate redundant symbols; colons indicate e-xceptions to preceding notations within the brackets Class

Symbolic

P-ope P-an D-ope D-an II I

CR, f-i g, G) m, .%b,f: F& n 83 (R, r; g, G) CR, r, S: g, n g,,)
Probability x3.(1 - x)” 2*x4*(1 - x)2 x3-(1 - x)” 2.x2.(1 - x)” x+[l - x.(1 - x)Z] + x3.(1 - x)3

(1 -x).[l

- (1-x)x”] +x3.(1 -x)3

Predicted Observed oGo9 0408 0.009 0.043 0.26 0.67 1.oo

0.012 0+09 0.014 0.047 0.25 0.67 __ 1.00

Deuteranopia may be defined as a class in a manner similar to protanopia, but interchanging the r and g symbols. Thus the deuteranope would possess R and r receptor types and would lack all g receptors: [R, r; g, C5] = x3 ‘(1 -x)~. The probability of encountering this class is 0.010, to be compared with the observed figure of 0.014. The proposed matrix for the deuteranomalous observer is to eliminate the chromatic content of the rg channel by surrounding B by all r receptors, but to permit residual green cones either adjacent to R or at location g2. Symbolically this may be expressed as [R, r, 8: g, n g,,]. The probability is 2~~~~(1-x)~ = 0.043 for x = 0.3. This value is to be compared with an observed mean value of O-047. Again one must evaluate these predictions in terms of the range of the observed values, which lie from 0*040-O-051. Turning now to the color normal observers, two major categories of color normals have been identified (Rubin, 1961; Richards, 1967; Waaler, 1973). The class I and class II color normals differ, according to the mosaic model by the absence of the R receptor type in the class II observer. The

190

W.

RICHARDS

class II observer thus has no R, but at least one other appropriately placed red receptor to retain the chromatic function of the rg channel: [R rZO n rzb n g,,]. The expected population probability among males is thus x- [l -x*(1 -x)“] = 0.25, close to the observed value. The remaining class I probability, calculated by requiring at least R and one neighboring g, is finally 0.66. These calculations and results are summarized in Table 1. 6. Discussion

Regardless of the degree of validity of the mosaic concept, the model raises many questions about color vision for which there are no data. Such an ambiguous position has both strength and weakness. Its strength lies partly in suggesting new approaches to color vision, in particular showing that color vision must be considered in terms of the organization of a twodimensional retina. On the other hand, the model fails in certain areas where data are not sufficient or contradictory. One major weakness of the model is that color anomalies must be interpreted not only in terms of a breakdown in the mosaic, but also in terms of an assumed underlying neural network. Some sort of description of the encoding of chromatic information is necessary in order to explain why color normals may be divided into two classes, and why red-green dichromats have only yellow-blue sensations. If these distinctions are brushed aside, then speculations about the neural processing of color information may also be avoided. Then, however, the success of a mosaic-type model would require that both red and green cones degenerate into a common degenerate receptor type. Furthermore, both kinds of anomalous trichromacy would then have to be based upon only one and the same anomalous pigment, which seems unlikely (Pokorny et al., 1973; Piantanida & Sperling, 1973). A further disadvantage of the model is that although only six types of color vision are discussed and summarized in Table 1, there are clearly a large number of subcategories among the normals in particular, some of which might quite likely be misclassified. Until more is understood about the neural linkages following the receptors, this ambiguity must persist (Ingling & Drum, 1973). But once again, we lack knowledge, for in spite of the advances of electrophysiology there have been only minor gains in our understanding of the color coding process. One hopeful sign is the recent quantification of electrophysiological data in the attempt to predict psychophysical results (De Valois, Abramov & Jacobs, 1966; Abramov, 1968). Such modeling may eventually yield formal neural networks not already envisioned by the psychophysicist. The precise form of the encoding of the yellow-blue and red-green color channels still remains obscure. However,

MOSAIC

MODEL

FOR

COLOR

191

VISION

the present evidence does not exclude the color network described in Appendix A. In some respects, the unanswered weaknesses of the model also represent an element of strength, for they point out where data are needed. But aside from this heuristic value, the model has two very strong assets. The first and most important is that it illustrates the need to consider color vision in terms of a two-dimensional array of receptors. The retina is not a random mosaic but instead is ordered. The genetic code must thus also be ordered, and the position (or linkages) of the three cone types must be specified in some manner in advance. But such a specification of three cone types in a two-dimensional array will lead to a further subclassification within each cone type, defining its position in the array. This is equivalent to the specification of string 1 or string 2 cones. A failure in the code may thus occur for either pigment or for position. But if these two are encoded together, then such failures will lead to predictable degeneracies in the mosaic. These are the color defective retinae. Quite surprisingly, the probability for encountering these degenerate cases may be predicted from only one statistic. In this respect at least, the two dimensions of the mosaic model has led to an unexpected simplification over our earlier concepts based solely upon three cardinal groups. Dr C. R. Ingling provided most valuable criticism of all aspects of the manuscript, and his help was greatly appreciated. The important concept of thresholded inhibition of one stage in the opponent-process model is entirely Ingling’s, although his neural model is not a two-stage model. Drs J. Pokorny, V. Smith and G. Waaler also offered very sensible advice in selected areas, which resulted in important modifications. Finally, those readers familiar with Dr Waaler’s ideas will find a similar spirit here, for I have been heavily influenced by his notions and am greatly indebted to him. This research was supported by N.E.I. under EYO0742 and by A.R.P.A. under an AFOSR contract F44620-74-C-0076. REFERENCES ABRAMOV, I. (1968). J. opt. Sot. Am. 58, 574. ADAMS, K., PEREZ, J. M. &HAWTHORNE, M. N. (1974). Invest. Ophthal. ALPERN, M. & TORII, S. (1968). J. gen. Physiol. 52,717. BENDER, B. G., RUDDOCK, K. H., DE VRIES-DE MOL, E. C. & WENT, L.

13, 885. N. (1972).

Vision

Res. 12, 2035. BLACKWELL, H. R. & BLACKWELL, 0. M. (1961). Vision Res. BONGARD, M. M., S~VWNOV, M. S. & FRIEDRICH, L. (1958). In

p. 325. London: Her Majesty’s Stationery Office.

BRINDLEY, Williams BROWN, P. CLARKE, F. CRAWFORD,

1, 63. Visual Problems of Colour Z,

G. S. (1970). Physiology of the Retina and Visual Pathway, p. 229. Baltimore:

& Wilkins. K. & WALD, G. (1964). Science, N. J. J. (1963). Optica Acta 10, 257. B. H. (1965). Vision Res. 5,71.

Y. 144,4.5.

192

W.

RICHARDS

CRICK, F. (1971). Symp. Sot. exp. Biol. DARTNALL, H. J. A. & LYTHGOE, J. N.

25,429. (1965). Vision Res. 5, 81. DENTON, E. J. & WYLLIE, J. H. (1955). J. Physiol., Land. 127, 81. DE VALOIS, R. L., ABRAMOV, I. & JACOBS, G. H. (1966). J. opt. Sot. Am. 56,966. DUNN, R. F. (1966). J. Ultrastruct. Res. 16, 672. FALLS, H. F. (1951). Am. J. Ophthat. 34,41. GAZE, R. M. (1970). i’%e Formation of Nerve Connections. New York: Academic Press. GOODWIN, B. C. & COHEN, M. H. (1969). J. theor. Biol. 25,49. GUILD. J. (1931). Phil. Trans. R. Sot. Ser. A 230. 149. HANN~VER; A. il843). Vid. Sel. Naturvid. og Mathen. Afa. 10, 9. HENRY, G. H., COLE, B. L. & NATHAN, J. (1964). Ann. hum. Genet. 27, 219. HURVICH, L. M., JAMESON, D. & COHEN, J. D. (1968). Percept. Psychophys. 4, 65. INDOW, T. (1974). In Cotor ‘73: Proceedings of the 2nd Congress, Assoc. Znt. de la Coateur, p. 137. London: Adam Hilger. INGLING, C. R., JR. St DRUM, B. A. (1973). J. opt. Sot. Am. 63, 369. INGLING, C. R., TSON, B. H. P. & DRIJM, B. A. (1974). The spectral sensitivity of the opponent-color channels. (Preliminary manuscript communicated personally.) JACOBS, G. H. & WASCHER, T. C. (1967). J. opt. Sot. Am. 57, 1155. JAEGER, W. & KROKER, K. (1952). Klin. Mbl. Augenheiik. 121,445. JAMESON, D. (1972). In Handbook of Sensory Physiology, vol. 7, chap. 4, p. 381 (D. Jameson & L. M. Hurvich, eds). Berlin: Springer Verlag. JUDD, D. B. (1949). J. Res. natn. Bur. Stand. 42, 1. KALMUS, H. (1965). Diagnosis and Genetics of Color Vision. New York: Pergamon Press. KALMUS, H. & CASE, R. M. (1972). Ann. hum. Genet. 35, 369. KRILL, A. E., SMITH, V. C. & POKORNY, J. (1971). Invest. Ophthat. 10, 457. LE GRAND, Y. (1968). Light, Colour and Vision. London: Chapman & Hall. L~RIA, S. M. (1967). Am. J. Psychol. 80, 14. LYALL, A. H. (1957). Q. JI microsc. Sci. 98, 189. MACLEOD, D. A. & HAYHOE, M. (1974). J. opt. Sot. Am. 64,92. MAKOUS, W. & BOOTHE, R. (1974). Vision Res. 14, 285. MARKS, W. B., DOBELLE, W. H. & MACNICHOL, E. F. (1964). Science, N. Y. 143, 1181. MAXWELL, J. C. (185s). Trans. R. Sot. Edin. 21, 275. MCCANN, J. J. (1972). Science, N. Y. 176, 1255. MULLER, G. E. (1930). 2. Psychol. Erganzeingsbande 17 & 18. NELSON, J. H. (1938). Proc. phys. Sot. 50, 661. PALMER, D. A. (1972). J. opt. Sot. Am. 62, 828. PIANTANIDA, T. P. & SPERLING, H. G. (1973). Visiofl Res. 13, 2033. PICKFORD, R. W. (1968). Vision Res. 8,469. PIRENNE. M. (1967). Vision and the Eve. London: Chanman & Hall. PITT, F..H. G. (1944). Proc. R. Sot. % 132, 101. POKORNY, J., SMITH, V. C. & KATZ, I. (1973). J. opt. Sot. Am. 63, 232. POLYAK, S. (1957). In The Vertebrate Visual System (H. Kluver, ed.), p. 265. Chicago: Univ. Chicago Press. RICHARDS, W. (1967). J. opt. Soe. Am. 57, 1047. RICHARDS, W. & LURIA, S. M. (1964). Vision Res. 4, 281. RUBIN, M. L. (1961). Am. J. Ophthal. 52, 166. RUSHTON, W. A. H., POWELL, D. S. & WHITE, K. D. (1973). Vision Res. 13, 2017. SCHOUTEN, 5. F. (1935). Proc. K. ned. Akad. Wet. 38, 3. SCHOUTEN, J. F. (1937). Thesis (Utrecht). SCHULTZE, M. (1866). Arch. microsk. Anat. EntwMech. 2, 175. SPERANSKAYA, N. I. (1959). Optics Spectrosc. 7, 424. SPERRY, R. W. (1945). J. Neurophysiol8, 15. STILES, W. S. (1955). In Phys. Sot. Yearbook, p. 44. London: Her Majesty’s Stationery OffiCe. STIL.ES, W. S. & BURCH, J. M. (1959). Optica Acta 6, 1.

MOSAIC

MODEL

FOR

COLOR

193

VISION

P. W. (1970). TREZONA, P. W. (1973). VOGT, A. (1942). Atlast

Vision Res. 10, 317. Vision Res. 13, 9. Spaltlampen-Mikroskopie 3, 846. Vos, J. J. & WALRAVEN, P. L. (1972). Vision Res. 12, 1327. WAAL.ER, G. H. M. (1967). In Mat. Naturv. Klasse, Ny Ser.,

TRJZZONA,

vol. 9, p. 1. Oslo: D. Norske

Vidensk-Ak. WAALER,

G.

M.

(1968).

Mat.

Natwv.

Klasse,

Ny

Ser.

vol.

11,

p. 1. Oslo: D. Norske

Vidensk-Ak.

WAALER, G. H. M. (1973). Acta Ophthal. Sappl. 122, p. 1. WAALER, G. H. M. (1974). Acta Ophthal. Suppl. 124, p. 5. WALD, G. (1964). Science, h? Y. 145, 1007. WALD, G. (1966). Proc. natn Acad. Sci. U.S.A. 45, 100. WALD, G., WOOTEN, B. R. & GILLIGAN, K. (1974). Assoc. Program 27. WALRAVEN, P. L. (1962). On the Mechanisms of Colour Vision.

Res.

Vision

Ophthal.

The Netherlands:

1974

Institute

for Perception. WEISS, P. (1937). J. camp. Neural. 67, 269. WILLMER, E. N. (1961). J. theor. Biol. 2, 141. WOLPERT, L. (1969). J. theor. Biol. 25, 1. WRIGHT, W. D. (1928). Trans. opt. Sot. 30, 141. WRIGHT, W. D. (1946). Researches on Normal and H. Kimpton. WRIGIS, W. D. (1952). J. opt. Sot. Am. 42, 509. WYSZECKI, G. & STILES, W. S. (1967). Color Science.

APPENDIX

Defective

Color

Vision.

London:

New York: Wiley & Sons.

A

Color Normal Classesand the PossibleNeural Organization of Opponent-processPairs

Color normals may be divided into at least two major classes (Richards, 1967; Waaler, 1967, 1974). The simplest method for differentiating between these two classes is by the manner in which observers assign color names to desaturated monochromatic lights in the region from 490-540 nm. Under the desaturated conditions when the monochromatic light appears on top of a white surround, class I males identify 515 nm as a green having no preponderance of yellow or blue, whereas class II observers choose 527 nm (Rubin, 1961; Richards, 1967; Waaler, 1967, 1968; Jacobs & Wascher, 1967). The same I and II notation has been applied to the two “normal” mosaics shown in Fig. 4 because certain properties of these two mosaics can be interpreted to underlie the behavioral classifications. Such an interpretation, however, rests upon an understanding of the neural encoding of color, which must be different in the class I and class II observers otherwise they would have picked the same wavelength for the unique “green”. This as well as other differences between the class I and class II (male) I.B. 13

194

W.

RICHARDS

observer may be explained in terms of two variations in the normal construction of the yellow-blue color channel. The encoding of color information beyond the receptors is generally believed to take place in two channels constructed from spatially-opponent pairs (Jameson, 1972). These opponent pairs are identified as red-green (rg) and yellow-blue (yb). The present mosaic model proposes that the rg process is organized or centered around the receptor B, whereas the yb process is centered about the position R (or G for class II observers). As seen from the cone mosaic, the proposed rg and yb fields will appear as yb

rg x0x x i-v, x x x Bx x * ‘.J x x ox

(0) rg

yb

b) FIG. 6. (a) Proposed rg and yb fields. (b) Spatial organization of these fields in terms of red, green, yellow and blue labels.

shown in Fig. 6(a). In terms of red, green, yellow and blue labels, the spatial of these fields could be crudely illustrated as shown in Fig. 6(b) for the class I observer. The notation below each diagram in Fig. 6(b) describes the inhibitory linkages between the components of the fields. These will be elaborated shortly. The interactions between the components of each field may be described in terms of the relative contributions of the receptors r, g, B, G or R. For the rg process, the major interactions will be between Y and G,, where Y = 4r+2g and G, = lOg+2R. The contribution of B to this channel is organization

MOSAIC

MODEL

FOR

COLOR

VISION

195

assumed to be via an inhibition of G,. However, following the suggestion of Ingling et al. (1973) and Ingling, Tson & Drum (1974), this contribution of B can occur only when the activity of G, is greater than B. Thus the stimulation of B in isolation has no effect upon the rg process unless some G, activity is already present. Further details describing the possible effects of such thresholded inhibition will be forthcoming (Ingling, personal communication). For our purposes, the activity of B is not felt unless the stimuli are desaturated (i.e. not monochromatic). Under desaturated conditions, the response of the rg channel may then be approximated by: (4r+2g+B)-(lOg+2R) where the net effect of B’s inhibition of G, is taken as excitation of Y, which is equal to 4r + 2g. Assuming linear processes, these receptor responses may then be normalized to unity within the brackets (so a diffuse white field will yield no net response) to give the following: rg, = (0.40r +0*14B) - (0.55g). For the class II observer, who has R replaced by G, the net result is rg,, = (0*57r + 0*14B) - (0.72g). This difference between rg, and rg,, is not appreciable, as may be proven by substituting for r, g and B appropriate approximations to the fundamentals, such as those derived by Pitt (1944) or Vos & Walraven (1972). Similarly, the response of the yb channel may also be approximated, assuming desaturated stimuli where the thresholded inhibition by R in this case is active. Now, however, the replacement of R by G will seriously alter the spectral response characteristics of the yb channel. In terms of receptor inputs, the major inhibition in the yb channel is between Y and B, with a minor contribution from R which has the effect of adding to B (for the class I observer). The response of the yb channel may be approximated by: (4r+2g)-(4B+R). Again normalizing to unity within the brackets so that a white field will yield no response, we find: yb, = (0.47r + 0*33g) - (0*8B). For the class I observer this is a typical yellow-blue opponent process. On the other hand, if R is replaced by G to characterize a class II observer, then the reduction leads to Here, the yellow-blue

yb,, = (0+67r +0*13g) - (0*8B). opponent process becomes almost a red-blue process.

196

w*

RICHARDS

Clearly, an opponent channel based upon red versus blue rather than yellow versus blue would require a longer wavelength light to balance the opponent process. Hence the class II observer will choose a “green” containing a balance between yellow and blue at a longer wavelength than the class I observer. The other known differences between the class I and II color normals may also be interpreted in terms of the model, but such interpretations go beyond the scope of this paper. Most important is that many of these explanations require two neural stages of inhibition (Muller, 1930; Judd, 1949), one of which is thresholded in a manner first suggested by Ingling (personal communication). If the test field appears on a dark background, then the critical stage of thresholded inhibition will be lost and the effect of replacing R by G will be too subtle to be seen. The diagnostic test will thus fail (Hurvich, Jameson & Cohen, 1968; Kalmus & Case, 1972). Finally, it should be obvious that the positional code for the neural network must be the same as that for the receptor mosaic, at least in its global features, in order that the two levels of information processing be in register and act in concert. A scheme similar to that devised for the receptor layer could thus be deduced for the underlying neural layer. However at present, the greatest constraints upon the global positional code come from our knowledge of the receptors, which impose the first functional constraint. Thus the organization of the receptor layer appears to be a practical starting point for deducing subsequent (hierarchical) organizations.

APPENDIX Invariant

B

Ratios of the Frequencies of Color Anomals

The population frequencies for male red-green color defectives presented in Table 1 are derived from statistics whose ratios yield invariant numbers. Thus, regardless of the actual value of x, these ratios will remain unchanged. The first invariant ratio is that between the incidence of the two types of dichromats : R = % deuteranopes = x3.(1 --x)~ = 1 1 x3*(1 -x)3 * oAprotanopes In practice, this ratio R, ranges from 0.93 to 1.8 with a mean value near 1.2 (LeGrand, 1968).

MOSAIC

MODEL

FOR

COLOR

VISION

197

The second invariant ratio is more complex and involves anomals and anopes from both classes: R = % protanomals * % deuteranomals 2 % protanopes*% deuteranopes 2*x4*(1-x)2.2*xy1-x)4=4 =

x3.(&x)3.x3.(~-.43

*

Values of R, calculated from LeGrand (1968) range from 1.1-5.7 with a mean of 3.1. Nelson’s (1938) values for R, and R, respectively are 4.2 and O-95. Also from LeGrand’s tabulation, it is possible to estimate x by comparing the average ratio of 5.4 of deuteranomals to protanomals. From Table 1 this ratio equals (1 -x)~/x~, yielding x = 0.30. This is the value assumed to calculate the percentage of males in each class in Table 1.