- Email: [email protected]
The contributions of figure and ground textures to segmentation
Vision Res. Vol. 32, No. 9, pp. 1793-1800, 1992 Printed in Great Britain. All rights reserved
Copyright
0
0042s6989/92 $5.00 + 0.00 1992 Pergamon Press Ltd
The Contributions of Figure and Ground Textures to Segmentation MARY J. BRAVO,*?
RANDOLPH
BLAKE*$
Received 9 December 1991
Several models of texture segmentation use spatial gradients in the activity of early filters to locate texture boundaries. The models assmne that these filters are identical to those involved in the detection and discrimination of near threshold patterns. The models diifer in how activity gradients from different types of filters are combined. We examined this question by measuring the respective contributions of a figure and a ground texture to segmentation. Vertical and horizontal line segments were used to construct two perfectly discriminable textures and these textures were used to construct four types of displays. Each display contained an obliquely oriented figure, but the displays diiered in the way this figure was defined. Displays consisted of either (1) a horizontally textured figure on a blank background, (2) a blank figure on a vertically textured background, (3) a horizontally textured figure on a vertically textured background or (4) a figure with a mixed texture (50% vertical lines, 50% horizontal lines) on a blank background. In a two-alternative forced-choice experiment, observers were asked to judge the figure’s orientation (right or left oblique), and the contrast of the textures was varied across trials. The resulting psychometric functions for segmentation were very similar for the four types of displays, suggesting ways in which a simple mode1of segmentation should be modified. Texture
segmentation
Contrast
threshold
Orientation
INTRODUCTION Results of experiments on the detection and discrimination of spatial patterns imply that the visual system analyzes such patterns with spatial frequency-, orientation-, and space-tuned filters (for a review see Graham, 1989). And while the specifics of this analysis are still being investigated, it is reasonable to ask what happens next? How does the visual system use the output of these filters? In particular, how does the visual system begin to construct a representation of figures and surfaces from this highly analytical representation? There is good reason to believe that the outputs of these filters are used in computations subserving motion (Watson & Ahumada, 1985; Adelson & Bergen, 1985) and stereopsis (Marr & Poggio, 1979; Blake & Wilson, 1991). In addition, several researchers have suggested that the visual system uses the output of these filters directly for one form of figure-ground segregation, texture segmentation (Caelli, 1985; Turner, 1986; Bergen & Adelson, 1988; Sutter, Beck & Graham, 1989; Fogel & Sagi, 1989; Malik & Perona, 1990; Landy & Bergen, 1991). It is known that texture segmentation can be both precise and essentially instantaneous (Bergen & Julesz, *Northwestern University, Evanston, IL 60208, U.S.A. tThe Smith-Kettlewell Eye Research Institute, 2232 Webster St., San Francisco, CA 94115, U.S.A. $.Department of Psychology, Vanderbilt University, Nashville, TN 37240, U.S.A.
1983; Bravo & Blake, 1990). This speed and efficiency suggest that the visual system performs a spatially parallel analysis of local stimulus properties and then locates discontinuities in these properties. These local stimulus properties, the computational models suggest, correspond to the outputs of the early filters. Although any one of these filters analyzes a restricted spatial location, there are many similarly tuned filters that respond to different locations. Taken together these similarly tuned, spatially distributed filters form a map for a given spatial frequency and orientation. Within a map, changes in filter output signal changes in the image composition. The computational models assume that two textures will segment if they produce a different response in at least one type of filter. That is, the models assume that segmentation will occur when there is a spatial activity gradient within at least one map. Since segmentation yields a delineation of texture regions, some of these models include further stages of processing (Malik & Perona, 1990; Landy & Bergen, 1991; Sutter et al., 1989; Caelli, 1985). In their most generic form (Fig. l), these additional stages include a rectification followed by local averaging or smoothing. The spatial gradients in these rectified and smoothed filter responses are then extracted using oriented, oddsymmetric filters. Finally, the responses to gradients in different maps are summed, and, if the result exceeds some threshold, a texture boundary is registered. These models predict whether or not observers will discriminate two textures presented side-by-side. However, there
1793
I794
MARY
J. BRAVO
and RANDOLPH
Responses of a set of filters tuned to vertical
BLAKE
Responses of a set of filters tuned to horizontal
Summation
FIGURE 1. A simple model of texture segmentation. At top, center is the input image. The right and left topmost squares represent the responses within two of many filter maps. The remaining squares depict the way in which these responses are transformed by the different stages of the model. White corresponds to a positive response, black to a negative response, and gray to no response.
are texture pairs that are discriminable and yet do not segregate into distinct perceptual regions (see e.g. Marr, 1982 pp. 93-96). Thus shape discrimination of the texture-defined region is taken as a more stringent measure of segmentation. Landy and Bergen (1991) model shape discrimination performance by including, as a final processing stage, the cross-correlation of the texture boundaries with templates used for shape recognition. In the generic model depicted in Fig. 1, the activity in different maps is combined only in the last step. Several models deviate from this generic model by combining the activity in different maps at an earlier stage in an attempt to suppress weak signals and thus eliminate spurious activity gradients. For example, Landy and Bergen (1991) subtract the activity in maps which encode the same spatial frequency but orthogonal orientations. This difference is then divided by the sum of the activity in all maps responding to the same spatial frequency. Malik and Perona (1990) include local inhibition across all maps in their model. Also, rather than summing the gradients in different maps, they allow the largest gradient to determine the perceived texture boundary. Most computational models of segmentation are based more on theoretical considerations than on empirical observations, and, in truth, there is little data available to distinguish among these models. This is because research on texture segmentation has focused primarily on determining the stimulus properties that are
measured at each location in the image, and not on how these local measurements are then compared and combined. This focus is certainly logical, for without a detailed answer to the first question (the local measurements), the second question (their combination) appears intractable. The present experiment attempts to side-step the first question, the exact nature of the local measurements, by using perfectly discriminable patterns (i.e. patterns that are discriminable whenever they are detectable). Such patterns are assumed to stimulate non-overlapping sets of filters at detection threshold (Watson & Robson, 1981; Thomas & Gilles, 1979). Now, if indeed the filters involved in segmentation are identical to those involved in detection and discrimination, then, at very low contrasts, perfectly discriminable textures will stimulate non-overlapping sets of filters for segmentation. The trick is to generate perfectly discriminable textures, that is, textures that stimulate different filters. The experiment reported in this paper compares (1) the contrast threshold for the segmentation of a textured figure on a perfectly discriminable, textured ground, with (2) the contrast threshold for each texture region presented separately. (These conditions are represented on the right side of Fig. 2; actual display elements had the same mean luminance as the background.) Thus observers identified the orientation of a rectangle defined by either one or two textures, with the contrast of the textures varied randomly across trials.
SEGMENTATION
OF FIGURE
AND GROUND
1795
TEXTURES
100 90 80 70 60
G ‘I ‘i’l’l’l ‘III’ ‘I ‘I’ ‘1’1 I I II
‘I
I’I I’,‘, ’ II Il I
F&G ‘I
ll’l’l’l ‘-1, ’, I-__, IIII’ ’ 11 l,l,I ‘I’1 I, I
’,’,
I, I
. I
I
-1.0
I
I
- 0.8
I
I
- 0.6
I
1
-1.0
I
I
- 0.8
I
1
- 0.6
log contrast FIGURE 2. (Right) Schematics of the three display conditions. Note that the actual Clisplays were composed of lines having . . 3 the same average luminance as the background. Note also that this figure depicts only the corner of the display that conramea the rectangle. (Left) Psychometric functions for two observers on the three display conditions. The percentage of correct judgements of the rectangle’s orientation are plotted against the log of the texture contrast.
The generic model outlined above (Fig. 1) makes a clear prediction about the outcome of this experiment: the combined figure and ground textures (F&G) should segment at lower contrast than when either the figure texture (F) or ground texture (G) is presented alone. The reason is as follows. Because the perfectly discriminable textures stimulate different sets of filters, they will generate gradients in different maps. Since the textures do not stimulate a common map, the gradients generated by the F&G conditions will simply be the sum of the gradients generated by the F and G conditions, respectively. According to the generic model, segmentation is based on the summation of gradients across maps, and so a texture boundary that elicits gradients in two independent maps will be detected at lower contrasts than one that elicits gradients in a single map. If this model were modified such that segmentation were based on the largest gradient rather than the sum of gradients (as in the Malik & Perona model), probability summation would still give an advantage to the F&G condition over the F and G conditions. Before conducting the main experiment, we did three preliminary experiments to ensure that the displays would have the necessary characteristics. The first experiment determined the appropriate size for the texturedefined figure. The second experiment eliminated the brightness gradient along the texture boundaries in the F and G displays. And the third experiment tested
whether the textures were discriminable threshold.
at detection
OBSERVERS
In total, four observers participated in the preliminary and main experiments. One observer, MB, is a practiced psychophysical observer. The other three observers had never participated in a vision experiment and were unaware of the purpose of this study. The three naive observers were paid for their participation. One naive observer was found to be differentially sensitive to the vertical and horizontal textures and was dropped from the study. The remaining three observers had normal or corrected-to-normal acuity along all meridians.
PRELIMINARY EXPERIMENT #l: SIZE OF THE TEXTURE-DEFINED REGION
One of the important decisions in the design of the main experiment concerned the size of the texture defined region. In this preliminary experiment we used a small rectangle, 2 x 3 elements in size. Because each element of the figure texture is adjacent to an element in the ground texture, we expected that use of this small figure would maximize the likelihood of observing an interaction between the figure and ground textures.
I796
MARY
J. BRAVO
and RANDOLPH
Stimulus Stimuli were displayed on a Conrac monochrome monitor (P4 phosphor), located 1.7 m from the observer in a darkened room. The fine structure (texture) and coarse structure (figure/ground regions) of the stimuli are described below. (a) Texture. In this preliminary experiment we used lines that had the same mean luminance as the background, that is, they were physically balanced (similar to the balanced dots of Carlson, Moeller & Anderson, 1984). These lines consisted of two spatial regions: a bright center line (2’ x 22’) surrounded by a dark surround rectangle (6’ x 26’). The luminances of the center and surround components were adjusted in accordance with their respective areas so that the space averaged luminance of the element was equal to the screen luminance of 18.2 cd/m2. A photometer was used to confirm that textured and untextured regions of the display had the same luminance. Contrast, defined as the maximum deviation from the mean luminance divided by the mean luminance (after King-Smith & Kulikowski, 1975) was varied by changing the luminance of the center and surround components while keeping their proportions constant and, hence, the stimulus physically balanced. The elements were arranged in rows with alternate rows indented by half the horizontal spacing of the elements. Nearest neighbor elements were separated by an average of 28’ along the diagonal. The positions of the texture elements were jittered by up to 25% of the vertical inter-element spacing. (b) Figure regions. As depicted on the right side of Fig. 2, the figures were rectangles 2 x 3 elements in size. The long axis of the rectangle was oriented at either 45 or 135 deg. Each display contained a single figure located, at random, in any of the four corners of the screen, 3.7 deg of visual angle from the screen center. The ground texture filled the remainder of the display in the G and F&G conditions. Procedure The observer initiated trials when looking at a fixation point located at the center of a blank field that had the same luminance as the background of the texture displays (18 cd/m2). After a 250 msec delay, a display was presented and then replaced 167 msec later by the blank field and fixation spot. The observer’s task was to determine the orientation of the texture-defined rectangle and press the appropriate response key. Runs were blocked by condition (F, G and F&G) and the ordering of these runs was randomly selected for each session. Seven contrasts, ranging from 8 to 32% in 2 dB steps, were presented in random order during a run. The location of the rectangle was also randomized across trials. Each run was preceded by five practice trials. Result and Discussion The data were analyzed separately for each observer, but were pooled across the four figure locations. For each condition the percent correct of 80 trials at each
BLAKE
contrast level was corrected for guessing and then transformed into a z-score. The regression statistics of these scores were computed to get best-fitting lines. These lines were then inversely transformed to get cumulative Gaussian fits to the data. These psychometric functions. with the original percent correct data, are shown in Fig. 2. left. For both observers the psychometric function for the segmentation of the F and F&G displays are quite similar; the addition of the ground texture has essentially no effect on the segmentation of the textured figure. On the other hand, both observers require higher contrasts to segment the rectangle in the G condition. And while this difference is small for one observer, it is quite large for the other. This inequality in the effectiveness of the figure and ground textures for segmentation clearly makes it impossible to compare their respective contributions to segmentation. For our main experiment to succeed, this difference must be eliminated. One factor that could contribute to the relative ineffectiveness of the G condition in supporting segmentation is that at low contrasts it is likely that only some of the elements are detected. In the F condition. the detection of one element is sufficient to indicate the corner of the display in which the rectangle is located. Knowing the location of the rectangle allows the observer to focus his/her attention at that location, and this would presumably aid in the orientation judgment. However, the non-detection of an element is insufficient to indicate the location of the rectangle in the G condition. since non-detection could be due to the absence of an element or to local internal noise. A related figure and ground asymmetry has been reported for high contrast displays. The discriminability of two textures may depend on which texture is assigned to the figure and which to the ground. Such asymmetries have been attributed to the greater potential for spurious boundaries with large texture regions (Gurnsey & Browse, 1987: Rubenstein 8~ Sagi, 1990). This disadvantage of the G condition can be partially off-set by using a larger rectangle. PRELIMINARY
EXPERIMENT LINES
# 2: BALANCED
Although the texture elements used in the preliminary study were physically balanced it is possible that they were perceptually unbalanced. A nonlinearity in the visual system’s encoding of these elements could cause the space averaged brightness of the elements to differ from that of the background. Any brightness cue of this sort would favor the segmentation of the F and G conditions over the F&G condition. To test for a possible brightness cue, several modified displays were created in which the texture elements were systematically unbalanced. A range of surround luminances were paired with a particular center luminance, and segmentation performance was then measured for each surround luminance. Figure 3 (top) shows the luminance profile of an element and the various lines indicate the surround luminances that were used to create a range of unbal-
SEGMENTATION
c &-
OF FIGURE
balanced luminance balanced brightness
AND
GROUND
left or right side of the display, guessing if necessary. In the discrimination task, observers determined whether the elements were vertically or horizontally oriented, again guessing if necessary. A comparison of the detection results for the two orientations showed that one of the four observers was differentially sensitive to the two element orientations. This observer’s results were excluded from the study. The combined detection results for the other three observers were then compared to the results of the discrimination experiment (Fig. 4). The psychometric functions for two of the observers were very similar. The third observer, GH, performed slightly better on the detection experiment relative to the discrimination experiment. MAIN
*
LTilmml ’ ATeamlrmnd
ALum
l
center
Area
center
FIGURE 3. (Top) Luminance profile of a texture element. Across trials different surround luminances were paired with a particular center luminance. (Bottom) Segmentation performance for different contrast values (ranging from 1O-o.9 to 10-06) for the G (lower left graph) and F conditions. Performance reached a minimum for a 1.66 ratio of surround (area*Alum) to center (area* Alum).
anced elements. Figure 3 (bottom) shows how performance was affected by these different surround luminances. Three graphs show data from the F condition and one graph (bottom, left) shows data from the G condition. Performance reached the lowest point for a center surround ratio that was not physically balanced. Fortunately, this ratio was the same for all observers and all contrasts and is represented by the lighter broad line in Fig. 3. These perceptually balanced lines were used in the main experiment.
PRELIMINARY EXPERIMENT # 3: IDENTIFICATION AT DETECTION THRESHOLD
A final preliminary experiment checked that observers could identify the orientation of the texture elements at detection threshold. This control consisted of two experiments, one to generate a psychometric function for element detection and the other to generate a function for element identification. Each display contained a small rectangle (2 x 3 elements) located in one of the four corners of an otherwise blank field. The elements composing the texture were perceptually balanced and were oriented horizontally in one half of the displays and vertically in the other half. The same set of stimuli were used in the detection and identification experiments, but the task differed in the two experiments. In the detection experiment, observers indicated whether the elements appeared on the
1797
TEXTURES
STUDY
The results of the three preliminary experiments were incorporated into the design of the main study; otherwise the main experiment was similar to the first preliminary experiment. The main experiment also included an additional condition which consisted of a blank ground and figure texture composed of 50% vertical and 50% horizontal elements. This condition, called the mix-F condition, was added to determine whether vertical and horizontal elements would group at the same contrast that they segment. Thus, this main experiment compared the contrast necessary to segment each of the four conditions shown schematically in Fig. 5, right. In sum, the main experiment was identical to the first preliminary experiments except that (1) larger rectangles were used, (2) the elements were perceptually balanced, and (3) a mix-F condition was included. Results and Discussion Figure 5 shows the results for three observers on the four conditions. Texture segmentation was measured as the percentage of correct responses for the identification of the rectangle orientation. Segmentation performance is plotted against texture contrast. Superimposed on each data set are best-fit cumulative Gaussian functions and a shaded bar indicating the contrast, plus and minus one standard error, associated with 75% performance. While there was considerable variation across observers, each observer produced very similar psychometric functions for the four stimulus conditions with the exception of the ground condition for observer GH. Before discussing these results, we must reiterate the assumption that makes this experiment interpretable in the context of the computational models discussed earlier. We assume that the vertical and horizontal texture are stimulating non-overlapping sets of filters, and thus producing gradients in different filter maps. This assumption is essential, for if the F and G textures produce gradients in both different and common maps, then when these two textures are presented side-by-side, gradients would be both added and eliminated. Without knowing the number and relative weights of these gradients it would be impossible to determine how the
179x
J. BRAVO
MARY
and RANDOLPH
BLAKE
1 0 l
detection discrimination
.t MB I -1.0
I
1
d-./i 0
.
.
I - 0.6
- 0.8
: 0 .* ’ . .”
-1.0
- 0.8
,+ : *,& i : :
*’ a’ 8
GH
LH
.
- 0.6
-1.0
- 0.8
:
- 0.6
log contrast FIGURE 4. Psychometric function for three observers for a two-alternative, spatial forced-choice decision location of the texture (detection, open circles) and (b) the orientation of the texture elements (discrimination, Performance is plotted against the log contrast of the texture elements.
gradients from different maps are combined prior to segmentation. So we must consider the likelihood that these textures are stimulating non-overlapping sets of filters. This assumption is reasonable provided that two conditions are satisfied. The first condition is that whenever the textures are detected they can be discriminated. This was demonstrated for two of the three
about (a) the solid circles).
subjects in the last preliminary experiment. The second condition is that segmentation occur at contrasts very near the detection threshold. This second requirement is less readily demonstrated. Because of probability summation across space, the detection threshold for an eccentrically viewed grating depends on the number of grating cycles that are presented, with more cycles producing a lower threshold (Robson & Graham, 198 1).
100
F
90
--- --_-_ ___-- _-_- _--_ --_ ---
80 70 60 50
G
100 90
‘,‘I’1‘l’l’l’,l,’
80
I I’,‘,’ 1’11 )I
70
,‘I I
’
I /I/I I I
3
100
8
90
2
‘I’,’
I’ , ’
I
I
I’Il ,lyll;
I I
l’I’,‘I’I I 1 I
I
I
F&G
70
I ,l,l, Ill, I ,l,l,! I ,I,‘, ‘---1,1,1 ,‘,I,‘------_I , ,I , i-‘-_: _T, I / I
60
, l---f’,‘,’ ,‘,I-.27
50
I
80
I
I
I
I 1 ’ 1-1’ I ‘( ’ I ’ I 1 1 I 1 I I I
100
Mix-F
90 80 70 60 50 -1.0
- 0.8
- 0.6
-1.0
- 0.8
- 0.6
-1.0
- 0.8
- 0.6
log Contrast FIGURE 5. (Right) Schematic representation of the four conditions used in the main experiment. (Actual had the same average brightness as the background.) (Left) Psychometric functions for three observers on conditions. Percentage of correct judgements of the rectangle’s orientation plotted against the log of the texture are best-fit cumulative Gaussians and the shaded bars correspond to the contrast, plus and minus one standard with 75% correct performance
texture elements the four display contrast. Curves error, associated
SEGMENTATION
OF FIGURE
This implies that the detection threshold for our texture will depend on the number of texture elements presented. It is assumed that detection requires only that one filter generate a signal in response to the texture, but we do not know the number of filters that must generate a signal in order for the shape of the textured figure to be identified (our measure of segmentation). For this reason we cannot directly relate the detection and segmentation thresholds. Nevertheless, if this comparison is made for the present experiment (i.e. cf. Figs 4 and 5, Fcondition) one finds that the segmentation threshold is lower than the detection threshold This must surely be due to the much larger figure in the segmentation experiment (32 elements) relative to the detection experiment (6 elements). The contrasts are certainly similar in the two experiments, and thus we think it likely that the segmentation experiment was conducted at sufficiently low contrasts. Given that our assumption is reasonable, we can interpret the results of the main experiment. Considering the top three curves for each observer, it appears that the addition of a vertically textured background neither enhances nor degrades segmentation of the horizontally textured rectangle. If the figure and ground textures stimulate independent maps, and so provide independent sources of information about the rectangle boundary, then observers appear to process selectively only one source of info~ation for segmentation. For the reasons described below, we think that this strategy is reasonable given the conditions of our experiment. By necessity the experiment was done at low contrasts. In addition, the various conditions were run in different blocks of trials so the observer knew before each trial the sort of texture display that would be presented. Is it possible that these two factors influenced our findings? Perhaps an observer presented with low contrast displays sets a very low criterion for deciding that a texture boundary is present. Of course the disadvantage of setting a low criterion is the acceptance of surpious gradients. Observers could reduce the acceptance rate of spurious gradients by only processing info~ation from the map that is most likely to contain the relevant information. This strategy was possible in our experiment because the different conditions were run in different blocks. By selectively attending to certain maps, information from other maps which might normally be summed would instead be discarded. Thus our results may be limited to conditions where observers have foreknowledge of, and can thus attend to, the appropriate filters for se~entation. Note, however, that foreknowledge and attention cannot account for the finding that the mix-F condition (bottom row) segments at similar contrasts as the F condition. If segmentation were based on the information carried by just one type of texture element, then clearly the mix-F condition should be more difficult than the F condition: in the mix-F condition the density of similar elements is half that of the F condition and these elements are arranged in a haphazard way within the figure. The filters responding to these elements would
AND GROUND
TEXTURES
1799
generate a relatively weak and inconsistent gradient along the figure boundary in the mix-F condition relative to the F condition. The similarity of these two conditions indicates that the visual system combines information from different maps before texture boundaries are registered. Taking together the results from the four conditions, our experiment suggests that there are at least two alternate ways in which information from different maps may be combined prior to segmentation. However, we should consider an alternative explanation for the similarity between the mix-F and F conditions. The model of Malik and Perona (1990) includes circularly symmetric filters among the early filters involved in segmentation. Such filters would respond to both elements in the mix-F condition, and thus would generate a consistent activity gradient along the figure boundary. To account for our results, these circularly symmetric filters must have the same contrast threshold as orientation selective filters. There is ample evidence (including the third control experiment) that the filters involved in detection are oriented (e.g. Kulikowski, Abadi & King-Smith, 1973; see Graham, 1989 for a comprehensive list). This leads to the conclusion that the filters involved in detection and segmentation are not identical and that observers should be able to segment textures that they cannot detect. This conclusion seems unlikely since detection can be based on any signal, including a segmentation signal. One might still conjecture that circularly symmetric filters with a high contrast threshold are involved in the segmentation of high contrast textures. Such filters would serve as detectors of line-crossings (Jules2 & Bergen, 1983), a feature for which there is conflicting evidence (Gurnsey & Browse, 1987). However, the inclusion of such filters undermines a central virtue of these computational models, a virtue which distinguishes them from other texture segmentation models, such as those of Julesz and Bergen (1983) and Beck (1982). These latter models offer accounts of texture segmentation that are based on the detection of abstract features such as lines and line endings. Since texture segmentation is one of the principle methods used to determine the list of abstract features, the models are difficult to test since an unpredicted result is taken as evidence that the list of features should be revised. Models based on the output of early filters are truly testable because the filters have been defined in a different context.
REFERENCES Adelson, E. H. & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, AZ, 284-299. Beck, J. (1982). Textural segmentation. In Beck, K. (Ed.), ~rgu~~zar~o~ and representation in perception. Hillside, N.J.: Erlbaum. Bergen, J. R. & Adelson, E. H. (1988). Early vision and texture perception. Nature, 333, 363-364. Bergen, J. R. & Julesz, B. (1983). Rapid discrimination of visual patterns. I.E.E.E. Transac~jons on Systems, Man, and Cybernetics, SMC-13,
857-863.
1800
MARY
J. BRAVO
and RANDOLPH
Blake, R. & Wilson, H. R. (1991). Neural models of stereoscopic vision. Tren& in Neurosciences, 14, 445452. Bravo, M. & Blake, R. (1990). Preattentive processing and perceptual groups. Perception, 19, 515-552. Caelli, T. (1985). Three processing characteristics of visual texture segmentation. Spatial Vision, I, 19-30. Carlson, C., Moeller, J. & Anderson, C. (1984). Visual illusions without low spatial frequencies. Vision Research, 24, 1407-1413. Fogel, I. & Sagi, D. (1989). Gabor filters as texture discriminators. Biological Cybernetics, 61, 103-I 13. Graham, N. (1989). Visual pattern analyzers. New York: Oxford University Press. Gurnsey, R. & Browse, R. A. (1987). Micropattern properties and presentation conditions influencing visual texture discrimination. Perception and Psychophysics, 41, 239-252. Julesz, B. & Bergen, J. R. (1983). Textons, the fundamental elements in preattentive vision and the perception of textures. Bell Systems Technical Journal, 62, 1619-1645. King-Smith, P. E. & Kulikowski, J. J. (1975). Pattern and flicker detection analyzed by subthreshold summation. Vision Research, 249, 519-548. Kulikowski, J. J., Abadi, R. &King-Smith, P. E. (1973). Orientational selectivity of grating and line detectors in human vision. Vision Research, 13, 1479-1486. Landy, M. S. & Bergen, J. R. (1991). Texture segregation and orientation gradient. Vision Research, 31, 679691. Malik, J. & Perona, P. (1990). Preattentive texture discrimination with early vision mechanisms. Journal of the Opiical Society of America, A 7, 923-932.
BLAKE
Marr, D. (1982). Vision. San Francisco, Calif.: W. H. Freeman & C i) Marr, D. & Poggio, T. (1979). A computational theory of human stereo vision. Proceedings o-f the Royal Society of London B. 514. 301l328. Robson. J. G. & Graham, N. (1981). Probabdity summation and regional variation in contrast sensitivity across the visual field Vision Research, 21, 409418. Rubenstein, B. S. & Sagi, D. (1990). Spatial variability of as a limiting factor in texture-discrimination tasks: Implications for performance asymmetries. Journal of the Optical Sociery o/ America. .4 7. 1632-1643. Sutter, A., Beck, J. & Graham, N. (1989). Contrast and spatial variables in texture segregation: Testing a simple spatial frequency maps model. Perception and Psychophysics, 46, 312-332 Thomas, J. P. & Gille, J. (1979). Bandwidths of orientation detectors in human vision, Journal of the Optical Society of America 61, 11761186. Turner, M. R. (1986). Texture discrimination of Gabor functions. Biological Cybernetics, 55, 7 l--82. Watson, A. B. & Ahumada, A. J. (1985). Model of human visualmotion sensing. Journal of the Optical Socief\l of .-lmerica. 2, 3222341. Watson, A. B. & Robson, J. G. (1981). Discrimination at threshold: labelled detectors in human vision. Vision Research. 21. 1115-1122.
Acknowledgements-We thank Jitendra Malik for reading an earlier draft of this manuscript. This work was supported by grants F32 EY06155 from the NE1 and BNS8617204 from the NSF.
Copyright
0
0042s6989/92 $5.00 + 0.00 1992 Pergamon Press Ltd
The Contributions of Figure and Ground Textures to Segmentation MARY J. BRAVO,*?
RANDOLPH
BLAKE*$
Received 9 December 1991
Several models of texture segmentation use spatial gradients in the activity of early filters to locate texture boundaries. The models assmne that these filters are identical to those involved in the detection and discrimination of near threshold patterns. The models diifer in how activity gradients from different types of filters are combined. We examined this question by measuring the respective contributions of a figure and a ground texture to segmentation. Vertical and horizontal line segments were used to construct two perfectly discriminable textures and these textures were used to construct four types of displays. Each display contained an obliquely oriented figure, but the displays diiered in the way this figure was defined. Displays consisted of either (1) a horizontally textured figure on a blank background, (2) a blank figure on a vertically textured background, (3) a horizontally textured figure on a vertically textured background or (4) a figure with a mixed texture (50% vertical lines, 50% horizontal lines) on a blank background. In a two-alternative forced-choice experiment, observers were asked to judge the figure’s orientation (right or left oblique), and the contrast of the textures was varied across trials. The resulting psychometric functions for segmentation were very similar for the four types of displays, suggesting ways in which a simple mode1of segmentation should be modified. Texture
segmentation
Contrast
threshold
Orientation
INTRODUCTION Results of experiments on the detection and discrimination of spatial patterns imply that the visual system analyzes such patterns with spatial frequency-, orientation-, and space-tuned filters (for a review see Graham, 1989). And while the specifics of this analysis are still being investigated, it is reasonable to ask what happens next? How does the visual system use the output of these filters? In particular, how does the visual system begin to construct a representation of figures and surfaces from this highly analytical representation? There is good reason to believe that the outputs of these filters are used in computations subserving motion (Watson & Ahumada, 1985; Adelson & Bergen, 1985) and stereopsis (Marr & Poggio, 1979; Blake & Wilson, 1991). In addition, several researchers have suggested that the visual system uses the output of these filters directly for one form of figure-ground segregation, texture segmentation (Caelli, 1985; Turner, 1986; Bergen & Adelson, 1988; Sutter, Beck & Graham, 1989; Fogel & Sagi, 1989; Malik & Perona, 1990; Landy & Bergen, 1991). It is known that texture segmentation can be both precise and essentially instantaneous (Bergen & Julesz, *Northwestern University, Evanston, IL 60208, U.S.A. tThe Smith-Kettlewell Eye Research Institute, 2232 Webster St., San Francisco, CA 94115, U.S.A. $.Department of Psychology, Vanderbilt University, Nashville, TN 37240, U.S.A.
1983; Bravo & Blake, 1990). This speed and efficiency suggest that the visual system performs a spatially parallel analysis of local stimulus properties and then locates discontinuities in these properties. These local stimulus properties, the computational models suggest, correspond to the outputs of the early filters. Although any one of these filters analyzes a restricted spatial location, there are many similarly tuned filters that respond to different locations. Taken together these similarly tuned, spatially distributed filters form a map for a given spatial frequency and orientation. Within a map, changes in filter output signal changes in the image composition. The computational models assume that two textures will segment if they produce a different response in at least one type of filter. That is, the models assume that segmentation will occur when there is a spatial activity gradient within at least one map. Since segmentation yields a delineation of texture regions, some of these models include further stages of processing (Malik & Perona, 1990; Landy & Bergen, 1991; Sutter et al., 1989; Caelli, 1985). In their most generic form (Fig. l), these additional stages include a rectification followed by local averaging or smoothing. The spatial gradients in these rectified and smoothed filter responses are then extracted using oriented, oddsymmetric filters. Finally, the responses to gradients in different maps are summed, and, if the result exceeds some threshold, a texture boundary is registered. These models predict whether or not observers will discriminate two textures presented side-by-side. However, there
1793
I794
MARY
J. BRAVO
and RANDOLPH
Responses of a set of filters tuned to vertical
BLAKE
Responses of a set of filters tuned to horizontal
Summation
FIGURE 1. A simple model of texture segmentation. At top, center is the input image. The right and left topmost squares represent the responses within two of many filter maps. The remaining squares depict the way in which these responses are transformed by the different stages of the model. White corresponds to a positive response, black to a negative response, and gray to no response.
are texture pairs that are discriminable and yet do not segregate into distinct perceptual regions (see e.g. Marr, 1982 pp. 93-96). Thus shape discrimination of the texture-defined region is taken as a more stringent measure of segmentation. Landy and Bergen (1991) model shape discrimination performance by including, as a final processing stage, the cross-correlation of the texture boundaries with templates used for shape recognition. In the generic model depicted in Fig. 1, the activity in different maps is combined only in the last step. Several models deviate from this generic model by combining the activity in different maps at an earlier stage in an attempt to suppress weak signals and thus eliminate spurious activity gradients. For example, Landy and Bergen (1991) subtract the activity in maps which encode the same spatial frequency but orthogonal orientations. This difference is then divided by the sum of the activity in all maps responding to the same spatial frequency. Malik and Perona (1990) include local inhibition across all maps in their model. Also, rather than summing the gradients in different maps, they allow the largest gradient to determine the perceived texture boundary. Most computational models of segmentation are based more on theoretical considerations than on empirical observations, and, in truth, there is little data available to distinguish among these models. This is because research on texture segmentation has focused primarily on determining the stimulus properties that are
measured at each location in the image, and not on how these local measurements are then compared and combined. This focus is certainly logical, for without a detailed answer to the first question (the local measurements), the second question (their combination) appears intractable. The present experiment attempts to side-step the first question, the exact nature of the local measurements, by using perfectly discriminable patterns (i.e. patterns that are discriminable whenever they are detectable). Such patterns are assumed to stimulate non-overlapping sets of filters at detection threshold (Watson & Robson, 1981; Thomas & Gilles, 1979). Now, if indeed the filters involved in segmentation are identical to those involved in detection and discrimination, then, at very low contrasts, perfectly discriminable textures will stimulate non-overlapping sets of filters for segmentation. The trick is to generate perfectly discriminable textures, that is, textures that stimulate different filters. The experiment reported in this paper compares (1) the contrast threshold for the segmentation of a textured figure on a perfectly discriminable, textured ground, with (2) the contrast threshold for each texture region presented separately. (These conditions are represented on the right side of Fig. 2; actual display elements had the same mean luminance as the background.) Thus observers identified the orientation of a rectangle defined by either one or two textures, with the contrast of the textures varied randomly across trials.
SEGMENTATION
OF FIGURE
AND GROUND
1795
TEXTURES
100 90 80 70 60
G ‘I ‘i’l’l’l ‘III’ ‘I ‘I’ ‘1’1 I I II
‘I
I’I I’,‘, ’ II Il I
F&G ‘I
ll’l’l’l ‘-1, ’, I-__, IIII’ ’ 11 l,l,I ‘I’1 I, I
’,’,
I, I
. I
I
-1.0
I
I
- 0.8
I
I
- 0.6
I
1
-1.0
I
I
- 0.8
I
1
- 0.6
log contrast FIGURE 2. (Right) Schematics of the three display conditions. Note that the actual Clisplays were composed of lines having . . 3 the same average luminance as the background. Note also that this figure depicts only the corner of the display that conramea the rectangle. (Left) Psychometric functions for two observers on the three display conditions. The percentage of correct judgements of the rectangle’s orientation are plotted against the log of the texture contrast.
The generic model outlined above (Fig. 1) makes a clear prediction about the outcome of this experiment: the combined figure and ground textures (F&G) should segment at lower contrast than when either the figure texture (F) or ground texture (G) is presented alone. The reason is as follows. Because the perfectly discriminable textures stimulate different sets of filters, they will generate gradients in different maps. Since the textures do not stimulate a common map, the gradients generated by the F&G conditions will simply be the sum of the gradients generated by the F and G conditions, respectively. According to the generic model, segmentation is based on the summation of gradients across maps, and so a texture boundary that elicits gradients in two independent maps will be detected at lower contrasts than one that elicits gradients in a single map. If this model were modified such that segmentation were based on the largest gradient rather than the sum of gradients (as in the Malik & Perona model), probability summation would still give an advantage to the F&G condition over the F and G conditions. Before conducting the main experiment, we did three preliminary experiments to ensure that the displays would have the necessary characteristics. The first experiment determined the appropriate size for the texturedefined figure. The second experiment eliminated the brightness gradient along the texture boundaries in the F and G displays. And the third experiment tested
whether the textures were discriminable threshold.
at detection
OBSERVERS
In total, four observers participated in the preliminary and main experiments. One observer, MB, is a practiced psychophysical observer. The other three observers had never participated in a vision experiment and were unaware of the purpose of this study. The three naive observers were paid for their participation. One naive observer was found to be differentially sensitive to the vertical and horizontal textures and was dropped from the study. The remaining three observers had normal or corrected-to-normal acuity along all meridians.
PRELIMINARY EXPERIMENT #l: SIZE OF THE TEXTURE-DEFINED REGION
One of the important decisions in the design of the main experiment concerned the size of the texture defined region. In this preliminary experiment we used a small rectangle, 2 x 3 elements in size. Because each element of the figure texture is adjacent to an element in the ground texture, we expected that use of this small figure would maximize the likelihood of observing an interaction between the figure and ground textures.
I796
MARY
J. BRAVO
and RANDOLPH
Stimulus Stimuli were displayed on a Conrac monochrome monitor (P4 phosphor), located 1.7 m from the observer in a darkened room. The fine structure (texture) and coarse structure (figure/ground regions) of the stimuli are described below. (a) Texture. In this preliminary experiment we used lines that had the same mean luminance as the background, that is, they were physically balanced (similar to the balanced dots of Carlson, Moeller & Anderson, 1984). These lines consisted of two spatial regions: a bright center line (2’ x 22’) surrounded by a dark surround rectangle (6’ x 26’). The luminances of the center and surround components were adjusted in accordance with their respective areas so that the space averaged luminance of the element was equal to the screen luminance of 18.2 cd/m2. A photometer was used to confirm that textured and untextured regions of the display had the same luminance. Contrast, defined as the maximum deviation from the mean luminance divided by the mean luminance (after King-Smith & Kulikowski, 1975) was varied by changing the luminance of the center and surround components while keeping their proportions constant and, hence, the stimulus physically balanced. The elements were arranged in rows with alternate rows indented by half the horizontal spacing of the elements. Nearest neighbor elements were separated by an average of 28’ along the diagonal. The positions of the texture elements were jittered by up to 25% of the vertical inter-element spacing. (b) Figure regions. As depicted on the right side of Fig. 2, the figures were rectangles 2 x 3 elements in size. The long axis of the rectangle was oriented at either 45 or 135 deg. Each display contained a single figure located, at random, in any of the four corners of the screen, 3.7 deg of visual angle from the screen center. The ground texture filled the remainder of the display in the G and F&G conditions. Procedure The observer initiated trials when looking at a fixation point located at the center of a blank field that had the same luminance as the background of the texture displays (18 cd/m2). After a 250 msec delay, a display was presented and then replaced 167 msec later by the blank field and fixation spot. The observer’s task was to determine the orientation of the texture-defined rectangle and press the appropriate response key. Runs were blocked by condition (F, G and F&G) and the ordering of these runs was randomly selected for each session. Seven contrasts, ranging from 8 to 32% in 2 dB steps, were presented in random order during a run. The location of the rectangle was also randomized across trials. Each run was preceded by five practice trials. Result and Discussion The data were analyzed separately for each observer, but were pooled across the four figure locations. For each condition the percent correct of 80 trials at each
BLAKE
contrast level was corrected for guessing and then transformed into a z-score. The regression statistics of these scores were computed to get best-fitting lines. These lines were then inversely transformed to get cumulative Gaussian fits to the data. These psychometric functions. with the original percent correct data, are shown in Fig. 2. left. For both observers the psychometric function for the segmentation of the F and F&G displays are quite similar; the addition of the ground texture has essentially no effect on the segmentation of the textured figure. On the other hand, both observers require higher contrasts to segment the rectangle in the G condition. And while this difference is small for one observer, it is quite large for the other. This inequality in the effectiveness of the figure and ground textures for segmentation clearly makes it impossible to compare their respective contributions to segmentation. For our main experiment to succeed, this difference must be eliminated. One factor that could contribute to the relative ineffectiveness of the G condition in supporting segmentation is that at low contrasts it is likely that only some of the elements are detected. In the F condition. the detection of one element is sufficient to indicate the corner of the display in which the rectangle is located. Knowing the location of the rectangle allows the observer to focus his/her attention at that location, and this would presumably aid in the orientation judgment. However, the non-detection of an element is insufficient to indicate the location of the rectangle in the G condition. since non-detection could be due to the absence of an element or to local internal noise. A related figure and ground asymmetry has been reported for high contrast displays. The discriminability of two textures may depend on which texture is assigned to the figure and which to the ground. Such asymmetries have been attributed to the greater potential for spurious boundaries with large texture regions (Gurnsey & Browse, 1987: Rubenstein 8~ Sagi, 1990). This disadvantage of the G condition can be partially off-set by using a larger rectangle. PRELIMINARY
EXPERIMENT LINES
# 2: BALANCED
Although the texture elements used in the preliminary study were physically balanced it is possible that they were perceptually unbalanced. A nonlinearity in the visual system’s encoding of these elements could cause the space averaged brightness of the elements to differ from that of the background. Any brightness cue of this sort would favor the segmentation of the F and G conditions over the F&G condition. To test for a possible brightness cue, several modified displays were created in which the texture elements were systematically unbalanced. A range of surround luminances were paired with a particular center luminance, and segmentation performance was then measured for each surround luminance. Figure 3 (top) shows the luminance profile of an element and the various lines indicate the surround luminances that were used to create a range of unbal-
SEGMENTATION
c &-
OF FIGURE
balanced luminance balanced brightness
AND
GROUND
left or right side of the display, guessing if necessary. In the discrimination task, observers determined whether the elements were vertically or horizontally oriented, again guessing if necessary. A comparison of the detection results for the two orientations showed that one of the four observers was differentially sensitive to the two element orientations. This observer’s results were excluded from the study. The combined detection results for the other three observers were then compared to the results of the discrimination experiment (Fig. 4). The psychometric functions for two of the observers were very similar. The third observer, GH, performed slightly better on the detection experiment relative to the discrimination experiment. MAIN
*
LTilmml ’ ATeamlrmnd
ALum
l
center
Area
center
FIGURE 3. (Top) Luminance profile of a texture element. Across trials different surround luminances were paired with a particular center luminance. (Bottom) Segmentation performance for different contrast values (ranging from 1O-o.9 to 10-06) for the G (lower left graph) and F conditions. Performance reached a minimum for a 1.66 ratio of surround (area*Alum) to center (area* Alum).
anced elements. Figure 3 (bottom) shows how performance was affected by these different surround luminances. Three graphs show data from the F condition and one graph (bottom, left) shows data from the G condition. Performance reached the lowest point for a center surround ratio that was not physically balanced. Fortunately, this ratio was the same for all observers and all contrasts and is represented by the lighter broad line in Fig. 3. These perceptually balanced lines were used in the main experiment.
PRELIMINARY EXPERIMENT # 3: IDENTIFICATION AT DETECTION THRESHOLD
A final preliminary experiment checked that observers could identify the orientation of the texture elements at detection threshold. This control consisted of two experiments, one to generate a psychometric function for element detection and the other to generate a function for element identification. Each display contained a small rectangle (2 x 3 elements) located in one of the four corners of an otherwise blank field. The elements composing the texture were perceptually balanced and were oriented horizontally in one half of the displays and vertically in the other half. The same set of stimuli were used in the detection and identification experiments, but the task differed in the two experiments. In the detection experiment, observers indicated whether the elements appeared on the
1797
TEXTURES
STUDY
The results of the three preliminary experiments were incorporated into the design of the main study; otherwise the main experiment was similar to the first preliminary experiment. The main experiment also included an additional condition which consisted of a blank ground and figure texture composed of 50% vertical and 50% horizontal elements. This condition, called the mix-F condition, was added to determine whether vertical and horizontal elements would group at the same contrast that they segment. Thus, this main experiment compared the contrast necessary to segment each of the four conditions shown schematically in Fig. 5, right. In sum, the main experiment was identical to the first preliminary experiments except that (1) larger rectangles were used, (2) the elements were perceptually balanced, and (3) a mix-F condition was included. Results and Discussion Figure 5 shows the results for three observers on the four conditions. Texture segmentation was measured as the percentage of correct responses for the identification of the rectangle orientation. Segmentation performance is plotted against texture contrast. Superimposed on each data set are best-fit cumulative Gaussian functions and a shaded bar indicating the contrast, plus and minus one standard error, associated with 75% performance. While there was considerable variation across observers, each observer produced very similar psychometric functions for the four stimulus conditions with the exception of the ground condition for observer GH. Before discussing these results, we must reiterate the assumption that makes this experiment interpretable in the context of the computational models discussed earlier. We assume that the vertical and horizontal texture are stimulating non-overlapping sets of filters, and thus producing gradients in different filter maps. This assumption is essential, for if the F and G textures produce gradients in both different and common maps, then when these two textures are presented side-by-side, gradients would be both added and eliminated. Without knowing the number and relative weights of these gradients it would be impossible to determine how the
179x
J. BRAVO
MARY
and RANDOLPH
BLAKE
1 0 l
detection discrimination
.t MB I -1.0
I
1
d-./i 0
.
.
I - 0.6
- 0.8
: 0 .* ’ . .”
-1.0
- 0.8
,+ : *,& i : :
*’ a’ 8
GH
LH
.
- 0.6
-1.0
- 0.8
:
- 0.6
log contrast FIGURE 4. Psychometric function for three observers for a two-alternative, spatial forced-choice decision location of the texture (detection, open circles) and (b) the orientation of the texture elements (discrimination, Performance is plotted against the log contrast of the texture elements.
gradients from different maps are combined prior to segmentation. So we must consider the likelihood that these textures are stimulating non-overlapping sets of filters. This assumption is reasonable provided that two conditions are satisfied. The first condition is that whenever the textures are detected they can be discriminated. This was demonstrated for two of the three
about (a) the solid circles).
subjects in the last preliminary experiment. The second condition is that segmentation occur at contrasts very near the detection threshold. This second requirement is less readily demonstrated. Because of probability summation across space, the detection threshold for an eccentrically viewed grating depends on the number of grating cycles that are presented, with more cycles producing a lower threshold (Robson & Graham, 198 1).
100
F
90
--- --_-_ ___-- _-_- _--_ --_ ---
80 70 60 50
G
100 90
‘,‘I’1‘l’l’l’,l,’
80
I I’,‘,’ 1’11 )I
70
,‘I I
’
I /I/I I I
3
100
8
90
2
‘I’,’
I’ , ’
I
I
I’Il ,lyll;
I I
l’I’,‘I’I I 1 I
I
I
F&G
70
I ,l,l, Ill, I ,l,l,! I ,I,‘, ‘---1,1,1 ,‘,I,‘------_I , ,I , i-‘-_: _T, I / I
60
, l---f’,‘,’ ,‘,I-.27
50
I
80
I
I
I
I 1 ’ 1-1’ I ‘( ’ I ’ I 1 1 I 1 I I I
100
Mix-F
90 80 70 60 50 -1.0
- 0.8
- 0.6
-1.0
- 0.8
- 0.6
-1.0
- 0.8
- 0.6
log Contrast FIGURE 5. (Right) Schematic representation of the four conditions used in the main experiment. (Actual had the same average brightness as the background.) (Left) Psychometric functions for three observers on conditions. Percentage of correct judgements of the rectangle’s orientation plotted against the log of the texture are best-fit cumulative Gaussians and the shaded bars correspond to the contrast, plus and minus one standard with 75% correct performance
texture elements the four display contrast. Curves error, associated
SEGMENTATION
OF FIGURE
This implies that the detection threshold for our texture will depend on the number of texture elements presented. It is assumed that detection requires only that one filter generate a signal in response to the texture, but we do not know the number of filters that must generate a signal in order for the shape of the textured figure to be identified (our measure of segmentation). For this reason we cannot directly relate the detection and segmentation thresholds. Nevertheless, if this comparison is made for the present experiment (i.e. cf. Figs 4 and 5, Fcondition) one finds that the segmentation threshold is lower than the detection threshold This must surely be due to the much larger figure in the segmentation experiment (32 elements) relative to the detection experiment (6 elements). The contrasts are certainly similar in the two experiments, and thus we think it likely that the segmentation experiment was conducted at sufficiently low contrasts. Given that our assumption is reasonable, we can interpret the results of the main experiment. Considering the top three curves for each observer, it appears that the addition of a vertically textured background neither enhances nor degrades segmentation of the horizontally textured rectangle. If the figure and ground textures stimulate independent maps, and so provide independent sources of information about the rectangle boundary, then observers appear to process selectively only one source of info~ation for segmentation. For the reasons described below, we think that this strategy is reasonable given the conditions of our experiment. By necessity the experiment was done at low contrasts. In addition, the various conditions were run in different blocks of trials so the observer knew before each trial the sort of texture display that would be presented. Is it possible that these two factors influenced our findings? Perhaps an observer presented with low contrast displays sets a very low criterion for deciding that a texture boundary is present. Of course the disadvantage of setting a low criterion is the acceptance of surpious gradients. Observers could reduce the acceptance rate of spurious gradients by only processing info~ation from the map that is most likely to contain the relevant information. This strategy was possible in our experiment because the different conditions were run in different blocks. By selectively attending to certain maps, information from other maps which might normally be summed would instead be discarded. Thus our results may be limited to conditions where observers have foreknowledge of, and can thus attend to, the appropriate filters for se~entation. Note, however, that foreknowledge and attention cannot account for the finding that the mix-F condition (bottom row) segments at similar contrasts as the F condition. If segmentation were based on the information carried by just one type of texture element, then clearly the mix-F condition should be more difficult than the F condition: in the mix-F condition the density of similar elements is half that of the F condition and these elements are arranged in a haphazard way within the figure. The filters responding to these elements would
AND GROUND
TEXTURES
1799
generate a relatively weak and inconsistent gradient along the figure boundary in the mix-F condition relative to the F condition. The similarity of these two conditions indicates that the visual system combines information from different maps before texture boundaries are registered. Taking together the results from the four conditions, our experiment suggests that there are at least two alternate ways in which information from different maps may be combined prior to segmentation. However, we should consider an alternative explanation for the similarity between the mix-F and F conditions. The model of Malik and Perona (1990) includes circularly symmetric filters among the early filters involved in segmentation. Such filters would respond to both elements in the mix-F condition, and thus would generate a consistent activity gradient along the figure boundary. To account for our results, these circularly symmetric filters must have the same contrast threshold as orientation selective filters. There is ample evidence (including the third control experiment) that the filters involved in detection are oriented (e.g. Kulikowski, Abadi & King-Smith, 1973; see Graham, 1989 for a comprehensive list). This leads to the conclusion that the filters involved in detection and segmentation are not identical and that observers should be able to segment textures that they cannot detect. This conclusion seems unlikely since detection can be based on any signal, including a segmentation signal. One might still conjecture that circularly symmetric filters with a high contrast threshold are involved in the segmentation of high contrast textures. Such filters would serve as detectors of line-crossings (Jules2 & Bergen, 1983), a feature for which there is conflicting evidence (Gurnsey & Browse, 1987). However, the inclusion of such filters undermines a central virtue of these computational models, a virtue which distinguishes them from other texture segmentation models, such as those of Julesz and Bergen (1983) and Beck (1982). These latter models offer accounts of texture segmentation that are based on the detection of abstract features such as lines and line endings. Since texture segmentation is one of the principle methods used to determine the list of abstract features, the models are difficult to test since an unpredicted result is taken as evidence that the list of features should be revised. Models based on the output of early filters are truly testable because the filters have been defined in a different context.
REFERENCES Adelson, E. H. & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, AZ, 284-299. Beck, J. (1982). Textural segmentation. In Beck, K. (Ed.), ~rgu~~zar~o~ and representation in perception. Hillside, N.J.: Erlbaum. Bergen, J. R. & Adelson, E. H. (1988). Early vision and texture perception. Nature, 333, 363-364. Bergen, J. R. & Julesz, B. (1983). Rapid discrimination of visual patterns. I.E.E.E. Transac~jons on Systems, Man, and Cybernetics, SMC-13,
857-863.
1800
MARY
J. BRAVO
and RANDOLPH
Blake, R. & Wilson, H. R. (1991). Neural models of stereoscopic vision. Tren& in Neurosciences, 14, 445452. Bravo, M. & Blake, R. (1990). Preattentive processing and perceptual groups. Perception, 19, 515-552. Caelli, T. (1985). Three processing characteristics of visual texture segmentation. Spatial Vision, I, 19-30. Carlson, C., Moeller, J. & Anderson, C. (1984). Visual illusions without low spatial frequencies. Vision Research, 24, 1407-1413. Fogel, I. & Sagi, D. (1989). Gabor filters as texture discriminators. Biological Cybernetics, 61, 103-I 13. Graham, N. (1989). Visual pattern analyzers. New York: Oxford University Press. Gurnsey, R. & Browse, R. A. (1987). Micropattern properties and presentation conditions influencing visual texture discrimination. Perception and Psychophysics, 41, 239-252. Julesz, B. & Bergen, J. R. (1983). Textons, the fundamental elements in preattentive vision and the perception of textures. Bell Systems Technical Journal, 62, 1619-1645. King-Smith, P. E. & Kulikowski, J. J. (1975). Pattern and flicker detection analyzed by subthreshold summation. Vision Research, 249, 519-548. Kulikowski, J. J., Abadi, R. &King-Smith, P. E. (1973). Orientational selectivity of grating and line detectors in human vision. Vision Research, 13, 1479-1486. Landy, M. S. & Bergen, J. R. (1991). Texture segregation and orientation gradient. Vision Research, 31, 679691. Malik, J. & Perona, P. (1990). Preattentive texture discrimination with early vision mechanisms. Journal of the Opiical Society of America, A 7, 923-932.
BLAKE
Marr, D. (1982). Vision. San Francisco, Calif.: W. H. Freeman & C i) Marr, D. & Poggio, T. (1979). A computational theory of human stereo vision. Proceedings o-f the Royal Society of London B. 514. 301l328. Robson. J. G. & Graham, N. (1981). Probabdity summation and regional variation in contrast sensitivity across the visual field Vision Research, 21, 409418. Rubenstein, B. S. & Sagi, D. (1990). Spatial variability of as a limiting factor in texture-discrimination tasks: Implications for performance asymmetries. Journal of the Optical Sociery o/ America. .4 7. 1632-1643. Sutter, A., Beck, J. & Graham, N. (1989). Contrast and spatial variables in texture segregation: Testing a simple spatial frequency maps model. Perception and Psychophysics, 46, 312-332 Thomas, J. P. & Gille, J. (1979). Bandwidths of orientation detectors in human vision, Journal of the Optical Society of America 61, 11761186. Turner, M. R. (1986). Texture discrimination of Gabor functions. Biological Cybernetics, 55, 7 l--82. Watson, A. B. & Ahumada, A. J. (1985). Model of human visualmotion sensing. Journal of the Optical Socief\l of .-lmerica. 2, 3222341. Watson, A. B. & Robson, J. G. (1981). Discrimination at threshold: labelled detectors in human vision. Vision Research. 21. 1115-1122.
Acknowledgements-We thank Jitendra Malik for reading an earlier draft of this manuscript. This work was supported by grants F32 EY06155 from the NE1 and BNS8617204 from the NSF.