Nonlinear distortion of gratings at the foveal resolution limit

and low

contrast region, rcspcctively. The cffccts for both brightness and hue in the high contrast

region are remarkably

whereas those in the low contest brightness

of

the high

contrast

large

region are small. The region

was reduced

to

IO-2546 of its original brightness level and the perceived hue of the high contrast

region shifted nearly

green under maximum

100 nm toward

=k[(g(x,L’)*a(.r,y)).lPnl(.~.y);

(3)

modulation.

stripe pattern can be seen. (The black and white simulation does not capture the red-green hut variations reported in the psychophysical observations. However, it is easy to elaborate the model to account for the hue shift as well. For example, a 632.8 nm interference fringe stimu-

where the cone mosaic function, ~(x,J), is expressed by a sum of delta functions, each of which corresponds to a cone center (see Williams, 1988). Equation (3) shows that the nonlinearity can be applied directly to the input stimulus after it has been blurred by the cone aperture. That is the order of the sampling and

Fig. 8, Silaulatian

of the wrondsry

zebra stripes produc&

by the contrast hy~tk~s~s.

i&g of the primstc fovea sampting a unity contrast 5inusoidaI fhc And stageof the simu~;tti~n was a Gaussian tow-P;ISS filter

Each pant9 shuws

the cerltr3l I I

grating ofnuar

frcqucncy.

lhat tcmoved spatial frequcnei~z

abuvc the cone Nyquist

frcqucncy. (al In this cast the image. after sampling by the cant mosaic, has been

passed thr ,ough an intensive nonli~~arity, the nanfin tcarity us& fourth

and then low pass liltsrcd.

here was exaggerated to product

po wcr of the input sensitivity.)

The comprcssivc

Tkc resulting

imep

For the purposes of d~rno~st~t~~n,

a high contrast nonliaerrity

much less scvcre than this, tend would have generated a zebra strip threshold.

the cone Nyquist

zebra stripe. (WC computed the

that rcsidus in the visual system is in the m&l

that was near contrast

shows the tucat diffcrunce in contrast that has tkc spatial configuration

of

zebra strir 9s. (b) in this case the grating was sampled by the cone mosaic and few-pass fittered in the same way a5 in a above, but the nonfincarity

was not appticd. showing

produce secondary zebra stripes

that the nonlj~~a~ty

and that they do not arise from sampling

is required

alone.

t(r

Nonlinear

distortion

nonlinear process is not critical.7 This is a useful result because it allows us to more easily predict the stimulus conditions that will generate the secondary zebra stripes. The input interference fringe is modified by the nonlinear function and can be represented by the Fourier series expansion. For simplicity, we describe it with a one dimensional representation: r’(x) = k {L(l + CR,(/) cos 27cf~))P = k f qcos Zn#?;

(4) (5)

j-0

where R,(f) represents the contrast reduction caused by the cone aperture as a function of the input spatial frequency./: L and c represent the mean luminance level and contrast of an input fringe, respectively and J’ indicates the order of the harmonic. This shows that the nonlinear process produces higher order harmonics of the original spatial frequency. For example, if the input spatial frequency is near the cone Nyquist frequency, the nonlinear process will generate a number of frequency components. The one of particular interest for the present purpose is the frcqucncy doubled component which is near the cone sampling frequency. or about I20 c/dcg. Since equation (3) shows that we can treat the nonlinear process as preceding the sampling operation, it bccomcs clear that this frequencydoubled harmonic will produce a zebra stripe near zero spatial frequency just as a real 120c/deg grating would (Williams, 1988). Consequently, the contrast hypothesis generates secondary zebra stripes with moirb zeroes at the same orientation, but half the spatial frequency of the moirf zeroes of the primary zebra stripes as shown in Fig. 6d. This is consistent with the psychophysical results described in Fig. 2. EXPERIMENTAL

TESTS

AND CHROMATIC

OF

THE

CONTRAST

ALlASING HYPOTHESES

The chromatic aliasing hypothesis and contrast hypothesis described above make very tThe cone mosaic function. m(.r. J-). is described by a set of delta functions. (Williams.

each of which represents cone locations

1988). Thus (m(x. y))’ requires a definition of

(6(x. v))p (0
< I). Strictly speaking.

it is impossible

to define this mathematically.

However,

usage of the delta

as just indicating

functions

cations

of cones,

applied

to the delta

locations. m(.r. Y).

we may consider functions

if we restrict the that

any

the lopowers

have no effect on their

In this sense. {m(.v,).)}P

can be reduced

to

at resolution

limit

82-l

similar predictions, and both could account for the experimental data described so far. However, the two hypotheses do make different predictions under other experimental conditions, and we devised two different experimental tests to distinguish between them. (I) Can dichromats see secondary zebra stripes? The chromatic aliasing hypothesis predicts that secondary zebra stripes should be absent in dichromats who lack either the L or the M cones, since the hypothesis is based on spatial sampling by at least two interleaved submosaics. However, the contrast hypothesis predicts that the secondary zebra stripes should be visible in dichromats, at least if they exhibit the same intensive nonlinearity found in color normal observers. Though one might not expect a dichromat to report hue variations in the secondary zebra stripes like the normal, the contrast hypothesis predicts that dichromats should be able to see secondary zebra stripes as variations in brightness. Subjects were four dichromats. three protanopes and one dcutcranopc, diagnosed by Raylcigh matches. Maxwell matches. and neutral point matches (Alpern & Wake, 1977; Wyszccki & Stiles, 1982). These observers reported low frequency distortion products when viewing the sum of two high frcqucncy interfcrence fringes, suggesting that, though they lack a cone type, they possess a compressive nonlinearity akin to that found in color normals. The dichromats reported the contrast-dependent brightness shift but no contrast-dependent hue shift. They were all inexperienced observers and were naive concerning the purpose of psychophysical experiments. They viewed interference fringes spanning extensive ranges of spatial frequencies and orientations were asked for subjective reports, without coaching. Four of the five criteria for identifying secondary zebra stripes in normals, described above, were applied to determine if a dichromat could see the secondary zebra stripes. (The requirement that they see color variations was not included on account of their color deficiency.) All four dichromats reported the primary zebra stripes at the fovea1 center for spatial frequencies near I lOc/deg. just like most normals. One of these observers, protanope TH, reported the presence of the zebra stripes at about 55 c/deg. He described the secondary zebra stripes as variations in brightness.

828

NO~WOSIU SEKJGLCHI et al.

The spatial frequency and orientation adjustment experiment described earlier was conducted on this protanope. Figure 9 shows the results. This observer’s settings were much more variable and there is no evidence for the six clusters of settings that would reflect triangular packing. However, the failure to observe these clusters is unlikely to be characteristic of dichromats; many color normal observers do not show clear evidence for triangular packing though primary zebra stripes are reported (Williams. 1988). What is clear in the data is the factor of 2 relationship between the spatial frequencies generating the primary and secondary moire zeroes, which is similar to normals. The moire zeroes of the secondary and primary zebra stripes lie in the range of 48-60c/deg and 94-122c/deg, respectively and the ratio of the mean spatial frequency settings for primary and secondary moire zeroes is 1.99. Though only one of the four dichromats tested produced clear evidence for secondary zebra stripes, this low yield is not surprising given that some fraction of normals do not report this subtle effect and given that the dichromats were not expericnccd observers. This result indicates that secondary zebra stripes do not require the presence of both L and M cones so that chromatic aliasing is not a ncccssary condition for their production. The results are consistent with the contrast hypothesis. (2) Aw secondury xhru grutings of neutral hue?

stripes

cisible with

The chromatic aliasing hypothesis predicts that the secondary zebra stripes should still appear chromatic when viewing a white instead of a red interference fringe. A white fringe of

Fig. 9. Settings made by the dichromatic the two-dimensional

observer (TH)

adjustment experiment.

in

Solid and open

circles represent the settings for moirk zeroes of the primary and secondary zebra-stripes.

respectively.

the appropriate spatial frequency will shift in and out of register with the L and M cones in the same way that the red fringe would. Indeed, because red-green color discrimination is optimal when the background hue is neutral (Le Grand, 1949), the use of a white interference fringe should, if anything, enhance the visibility of the hue variations associated with the secondary zebra stripes. Two color normal observers (DW and NS) observed a white interference fringe produced with a point source of tungsten light and a Ronchi ruling. A pair of first order spectra from the diffraction pattern of the Ronchi ruling were used as two point sources at the entrance pupil of the observer’s eye, forming the white interference fringe on the retina. The fringe spatial frequency was set at about 60c/deg and orientation was horizontal. Space-averaged luminance was about 1500 td. Each observer independently reported the secondary zebra stripes and described their appearance. Both observers described the secondary zebra stripes as variations in brightness alone without the rod-green hue variation seen with red intcrfcrcnce fringes. This same result was obtained with a 594 nm interference fringe obtained with a Hc-Nc Inscr. Though 594 nm is slightly reddish. it is much less red than 632.8 nm. and it produced a large diminution in the hue variations associated with the secondary zebra stripes for both observers. These observations arc inconsistent with the chromatic aliasing hypothesis, since it would predict a variation in hue. The contrast hypothesis makes the opposite prediction, namely that the hue variations in the secondary zebra stripes should be reduced. The prediction is based on independent observations of the hue shift that is essential to the contrast hypothesis. The same observers viewed a vertical contrast-modulated interference fringe, which was produced by adding two white interference fringes (both were 56c/deg but slightly different orientation from horizontal). The spatial frequency of the contrast was 8 cideg. Both observers modulation independently described the 8 c/deg distortion product as defined by variations in brightness, with no variation in hue, though the red-green hue variations could be seen by introducing a red Wratten (no. 26) filter before the eye. Whatever the mechanism for the generation of the contrast-dependent hue shift, it is not apparent with neutral stimuli. Since the secondary zebra

Nonlinear distortion at resolution limit

stripes also acquire a neutral appearance with white stimuli, their behavior is entirely consistent with the contrast hypothesis, though not the chromatic aliasing hypothesis.

DISCUSSION T’he secondary zebra stripes are subtle even under the best conditions, and the low contrast sensitivity to them makes it an exceedingly difficult phenomenon to study. Nonetheless, the evidence we have been able to obtain from a dichromat suggests that chromatic aliasing is not necessary for their production. Furthermore, the fact that the hue variations in the secondary zebra stripes are diminished when a neutral interference fringe is used instead of a red one also argues against chromatic aliasing as the explanation. The available evidence points toward the contrast hypothesis to account for secondary zebra stripes. Indeed, the calculations below suggest that the form of the compressive nonlinearity described by Makous et al. (1985) is just what is required to predict the ratio of contrast sensitivities observed psychophysically for primary and secondary zebra stripes. According to equation (3) the contrast hypothesis predicts that the output of the nonlinear process depends only on the attenuation by the cone aperture and the form of the nonlinearity. Thus, these two factors alone should account for the difference in contrast sensitivity to primary and secondary zebra stripes. If the attenuation by the cone aperture and the form of the compressive nonlinearity are sufficient to explain the observed ratio of contrast sensitivities, the calculated ratio of the amplitude of the fundamental component for the primary zebra stripes to that of the second harmonic for the secondary zebra stripes should be unity. Here we rewrite equations (4) and (5) described above.

r’(x)=k{L(l

+cR,(/)c0s27~~~)}“

(4)

5

= k 1 “L/cos 2nit3.

(5)

J-0

a, in equation (5) provides the amplitude of each frequency component in the signal resulting from attenuation by the cone aperture and the effect of the nonlinearity. (The MTF of the cone aperture was assumed to be a Bessel function of the first order, Snyder & Miller, 1977.) Measurements of the power function describing the

819

compressive nonlinearity (p = 0.241) as well as the diameter of the fovea1 cone aperture (I .7pm) were available from previous work for one observer (DW) (MacLeod et al., 1985; Makous et al., 1985). We wish to calculate the ratio, Q, (at 112 c/deg)/a, (at 56 c/deg). a,, the amplitude of the fundamental frequency of the distorted signal resulting from an interference fringe with a spatial frequency equal to the cone sampling frequency was calculated from equations (4) and (5) by setting c to 20% at a spatial frequency (f ) of 112 c/deg, as measured psychophysically in observer DW. a, calculated here is the amplitude required to generate a threshold percept of the primary zebra stripes. We also wish to calculate a2 for the second harmonic of the distorted signal that would result from a fringe at the Nyquist frequency. The contrast of the input fringe in this case is set to the threshold contrast for secondary zebra stripes (75% at 56c/deg). According to the contrast hypothesis, this aI is the amplitude of the signal available to generate a threshold percept of the secondary zebra stripes. The ratio of amplitudes calculated from threshold data was I.12 which corresponds to 0.05 log units and is quite close to the ratio of 1.0. Thus the attenuation by the cone aperture and the form of the compressive nonlinearity explain the observed ratio of contrast sensitivities for primary and secondary zebra stripes, which is predicted by the contrast hypothesis. The calculation shows that the compressive nonlinearity, for which there is substantial independent evidence, ought to produce suprathreshold secondary zebra stripes when viewing high contrast interference fringes that are near the Nyquist frequency. Another prediction of the contrast hypothesis is that the primary and secondary zebra stripes should have a similar spatial configuration, when the spatial frequency used to produce the primary zebra stripes is exactly twice that producing the secondary zebra stripes. However, though both observers (DW and NS) saw similarities in the two patterns, they did not seem to be identical. We have not taken this as compelling evidence against the contrast hypothesis, however, because the fringes required to produce the two effects could be subject to different phase distortions from the optics of the eye. Phase distortion would alter the spatial configuration of both kinds of zebra stripes. Different phase distortions are plausible because the light forming the fringe required to produce

the primary zebra stripes enters through different points of the pupil than that which produces the primary zebra stripes. A central prediction of the contrast hypothesis is that. at any retinal location. the spatial frequency that produces a moirk zero for the primary zebra stripes is twice that for the secondary zebra stripes. The data agree fairly well with this prediction: at the fovea the ratio is 2.02 and 1.87 for two observers. However, there is a slight tendency for the ratio to be less than two overall (for example, see Fig. 3). The reason for this discrepancy is not known. It is possible that the settings observers make of the primary moire zeroes are somewhat biased toward lower spatial frequencies, due to the reduction in contrast

or the reduction

in the area over which

zebra stripes can be seen as spatial frequency incrcascs. Also, the moir6 zero settings for the secondary zebra stripes, which occur right at the resolution limit. may be biased upwards in spatial frequency. At lower Spatial frcquencics, the original intcrfcrenco fringe can bc resolved which

may

stripes,

mask

forcing

I’hc

the subtle

slightly

secondary

higher

zebra

settings.

could in principle product moirk zcrocs like the scc“spurious”

ondary

rcbra

others would ;I spatial product which

A simple

cxamplc

bc frcqucncy-tripling

frcqucncy

of

;I distortion could

qucncy

stripes.

also

zebra stripe.

the third harmonic

4Oc/dcg

product

product produced

could

I20 c/dcg.

at

a low

However,

among

in which cast

about

spatial

frc-

the amplitude

of

by the nonlinearity

is much less than that of the second harmonic,

zeroes protfuccd in this way was unsuccessful. This is not rcry surprising, given the Fact that the secondary and ;I starch

Ara

for additional

of conditions. Williams (1988) developed that

volution

consisted

of

under the best

three

by the cone aperature,

of fovea1

stages:

sampling

conby the

of

is csscntially to intcrpolatc between the cones. The parameters of this stage specify which alias (of many) will actually be chosen by the visual

system as the interpretation

Our

original

ondary

is known

to

be inherently

motivation

for

studying

sec-

zebra stripes was that they might

the packing

arrangement

reveal

L cones.

of M and

However,

since we have been lead to reject the

secondary

zebra stripes as a candidate

matic not

aliasing. provide

and

this phenomenon information

geometry. exclusive,

cones. Moreover.

are

results

of chromatic

Nerger,

with

we found

fine,

1991). Brcwstcr and

gratings,

for cxumplc

lurgcr

than

chromatic

patterns

WC confirm

that

that

appear

20c/dcg.

can appear

white

covcrcd

with

that are scvcral

the bars of the grating. zebra stripes.

Brainard

(1832) reported

red and green splotches secondary

for Haake.

achromatic

colored,

of

stimulus.

phenomenon

candidate Sekiguchi.

periodic,

appear

an example

a punctate

an additional

(Williams,

9r Packer.

the

and M

1964, 1978; Cicerone

1989) is probably

is a promising

aliasing

by the L

at the fovea sometimes

aliasing

Recently

mutually

do not preclude

aliasing

red or green (Krauskopf, &

aliasing

not

the fact that tiny, brief yellow

flashes presented

chromatic

does

the submosaic

the chromatic

hypotheses

and our

existence

for chro-

probably

about

Nonetheless,

contrast

times

Unlike

the

this is just the kind

effect one might

expect from chromatic

by a disordered

array

Ack,row/r~.~~~lenrs--Wc Curcio and Walter and

Drs

of

aliasing

of M and L cones.

of the stimulus.

For

simplicity. the spatial filtering stage was rcpresented as a linear spatial filter. The existence of secondary zebra stripes that arise from an early nonlinearity in the visual system requires modification of the linear model. At the very Icast, a nonlinear stage would need to be

to

are

Makous

Albert

Brainard

grateful

to

for providing

Ahumada,

Mary

Drs

Christine

USvaluclblrdata

Hayhoe

and

David

for useful suggestions on an earlier version of this

manuscript.

a model

cone locations, and tinally low pass spatial tiltcring. The function of the final stage array

which

nonlinear.

moirb

stripes are only just visible

aliasing

receptors,

that

nonlinearity

additional

interposed between the sampling operation and the linear spatial filter. This kind of modification should not be too surprising. since the final stage of the model is a crude representation of the entire visual system beyond the photo-

Wr also thank Bill Haake for his contributions

this work

technical

and

Alan

assistance.

AFOSRLtS-0019.

Russell

This

and

research

EYOO269,

Clill was

EYO4367.

Conklin supported

EY06l

I7

for by and

EYOl319.

REFERENCES Ahumada. and

A. Jr (1986).

medium

Models for the arrangement

wavclcngth

cows

Journul of the Oprical Society Alpcrn.

M. & Wake.

dcutan

color

T. (1977).

vision

in the central

of America, Journal

fovea.

3 (SuppI.).

Cone pigment

defects.

of long

of

92.

in human Physiology.

London. 266. 595-6 12. Bcrgmann.

C. (I 858).

iibcr die Netzhaut (Eds.).

Zeifschri/r

pp. 83-1081.

Anatomisches

und Physiologisches

des Auges. In Hcnle. J. Kc Pfeufer, C. /i”r

Rarionek

Medicin

[vol. 11(3).

Nonlinear distortion at resolution limit flrainard. 0. H.. Wandell, 8. A. k Poirson. A. B. (1989). Discrete analysis of spatial and spectral aliasing. Imestigattie Ophrhatmotogy and Visual Science (Suppi.). 30.53. Brewster, D. (1832).On the undulations excited in the retina by the action of luminous points and lines. The London and Edinburgh

Phytosophicat

Maga-ine

and Journal

of

Science. I, 169-174. Burton, G. J. (1973). Evidence for non-linear response processesin the human visual system from measurements on the threshold of spatial beat frquencies. Vision Research. 13. 121l-1225. Byram, G. M. (1944). The physical and photochemical basis of visual resolving power. Journal of the Oprical Society of America,

47, 7 18-737.

Campbell. F. W. &. Green. D. G. (1965). Optical and retinal factors affecting visual resolution. Journal of Physiotogp London.

181. 576-593.

Cicerone. C. M. & Nerger. J. L. (1989). The relative numbers of long-wavelength-sensitive to middle-wavclength-sensitivecones in the human fovea centralis. Vision Research. -y. 1I S-128. Colerta, N. J. & Williams. 13. R. (1987). Psychophysical estimate of extrafoveal cone spacing. lournaf of the Oprical Soci+v of America. 4, 1503-t 5 I3. Coletla. N. J.. Williams. D. R. & Tiana. C. (1990). Consequences of spatial sampling for human motion perception. Vi.sion Research. .W, 163 I -I 648. Curcio. C. A.. Sloan, K. R.. Kalina, R. E. & Hendrickson, A. E. f IWO).Human ph~~torL~ep~ortopography. Journal (I$’ Cumparurirr

Neurolqqy.

292. 497-523.

Krauskopf. .I. (IY64).Color appcarancc of small stimuli and the sparial distrihulion of color rcccptors. Journal of the Opricat .%cb~y of America,

JJ.

t I? I

Krauskopf, J. (l97X). On identifying dctcctors. In Arming ton. J. C., Krauskopf. J. & Wooten, B. R. (Eds.). Vi.ruat p.~y~hriptf~.~i~.~and phy.~;fjl~lgy (pp. 283 -295). New York: hcadcnric Prcvs . . Lc Grand. Y. (1949) Lcs scuilr diiTSrrnlicls de couleurs duns la Ihkxic ds Young. Rrtwc d’opriqur Throrique et tnstrumrnrul~~. 28, 261-218. MacLeod. D. I. A., Williams, D. R. & Makous. W. (1985). Difference frequency gratings above Ihe resolution limit. /n~~~f~~{if;~r~pi~~i~~~t~~utu~~ and Visuut Scirnce (Suppt. ), 26,

II.

Makous. W., Williams, D. R. & Mactcod. D. 1. A. (1985). Nonlinear transformation in human vision. Journut of the Opricul So&y

of America.

2 (Suppt.). 80.

Ohzu, H.. Enoch. J. M. & O’Hsir. J. C. ((972). Optical m~ulalion by the isolated retina and retinal receptors. i’isiun Research, 12, 23 I -2-M. Chcrbcrg. G. (1935) Topography of the layer of rods and

831

cones in the human retina. Acra Ophf~tmotogtca (Suppl.). 6. I-103. Smih V. C. & Pokomy. J. (1975). Spectral sensitivity of the fovea1 cone pigments between 400 and SOOnm. Vision Research, IS. 161-171. Snyder. A. W. & Miller. W. H. (1977). Photoreceptor diameter and spacing for highest resolving power. Journal of fhe Oprical

Soctefy of America.

6% 696-698.

Tiana. C.. Williams, D. R.. Coletta, N. J. & Haake. P. W. (1990. A model of aliasing in extrafoveal human vision. In Landy, M. & Movshon. A. (Eds.). Computation mo&ts of visual processing. Cambridge, Mass.: MIT Press. In press. Vimal, R. L. P., Pokomy, J.. Smith, V. C. & Shevell. S. K. (1989). Fovea1 cone thresholds. Vision Research, 29.

61-78. Vos. J. J. & Walraven, P. L. (1971). On the deviation of the fovea1 receptor primaries. Vision Research. It. 799-818. Walraven, P. L. (1974). A closer look at the tritanopic convergence point. Vision Research, fJ, 1339-1343. Williams, D. R. (1985). AIiasing in human fovea1 vision. Vision Research, 25, 195-205. Williams, D. R. (1986). Seeing through the photoreceptor mosaic. Trends in Neuroscicnre. 9. 193-198. Williams, D. R. (1988). Topography of the fovea1 cone mosaic in the living human eye. Vivian Research, 28. 433454.

Williams, D. R. (1990). The invisible cone mosaic. Proceedings from

ihe Na&mal

Research Council S~ni~~.~iurn on

DC: National Academy Press. Williams. D. R. % Collier, R. J. (1983). Consequences of spatial sampling by a human pholorLzcptor mo.saic. Phororeceprion.

Washington.

Science, 221, 385-387.

Williams. D. R., MacLeod. D. I. A. & Hayhoc, M. M. (198la). Punctatc sensitivity of the blue-sensitive mcuhunism. Vi&n Research. 31, 1205-1210. Wilhams, D. R.. MacLeod. D. I. A. & lfayhoe. M. M. (I981 b). Fove~l lritanopia. Vision Research, ,7/, 1341-1356.

Williams. D. R., Cotiicr. R. J. & Thompson, B. J. (1983). Spatial resolution of the short-wavelength mechanism. In Mellon, J. D. & Sharpe. L. T. (Eds.), Color vision (pp. 487-503). London: Academic Press. Williams, D. R.. Sekiguchi. N., Haake. W., Brainard, D. & Packer. 0. (1991). The cost of trichromacy for spatial vision. In Lee, 0. (Ed.), Advunces in understanding visual procrsses: Convergence of neurophysiotogicai

and psycho-

pi~ysicat evidence. New York: Plenum Press. In press.

Wyszccki. G. & S&s, York: Wiley.

W. S. (t982).

Color science. New