- Email: [email protected]
Sparse, opaque three-dimensional texture, 2B: Photometry
Pattern Recognition, Vol. 29, No. 2, pp. 297-313, 1996 Elsevier Science Ltd Copyright © 1996 Pattern Recognition Society Printed in Great Britain. All rights reserved 0031 3203~96 $15.00+.00
Pergamon
0031-3203(95)00080-1
SPARSE, OPAQUE THREE-DIMENSIONAL TEXTURE, 2B: PHOTOMETRY A D L A I W A K S M A N and A Z R I E L R O S E N F E L D * Computer Vision Laboratory, Center for Automation Research, University of Maryland, College Park, MD 20742-3275, U.S.A.
(Received 18 November 1994; in revisedform 2 May 1995; receivedfor publication 2 June 1995) Abstract--This paper deals with three-dimensional (3D) textures composed of approximately planar texels distributed over a volume of space ("leafy" textures). It studies the gray-level histograms of images of such textures (specifically houseplants) under illumination by a compact light source. The variation of such histograms with light source direction resembles that of synthetic histograms generated using a Phong-type reflectance model and a uniform texel orientation model, and ignoring transmittance, interreflection and shadows. Histogram
Houseplant
Photometry
Reflectance
1. INTRODUCTION Natural scenes contain many phenomena, such as leaves in a tree crown and falling snowflakes, which involve discrete, relatively sparse and opaque elements or "texels" distributed over a given volume of space. In this and two companion papers, we study three questions about such 3D textures: (a) (Visibility) H o w far into the texture can we see from a given viewpoint? W h a t is the probability that a given viewing ray will pass all the way through it? (b) (Photometry) Assuming that we know how light is reflected from (and possibly transmitted by) the texels, and given an illumination model, what will the gray-level histogram of the texture look like in an image taken from a given viewpoint? (c) (Recovery) What properties of the texel population can be recovered from single images? In these papers we regard textures such as snow and tree crowns as composed primarily of (approximately) planar texels (flakes or leaves) randomly distributed in space. In an image of a sparse 3D texture, both texels and background will be visible. In order to study questions such as those posed above, we must assume that the
This work was supported by the Defense Advanced Research Projects Agency (ARPA Order No. 8459) and the U.S. Army Topographic Engineering Center under Contract DACA76-92-C-0009. The authors thank Sandy German for providing the houseplant and for her expert help in preparing this paper. * Author to whom correspondence should be addressed. 297
3D texture
texels can be distinguished from the background. This is sometimes easy even for natural scenes--for example, in the case of a tree crown imaged against the sky. The examples used in this paper are indoor images of plants taken against a black background; these were more convenient to acquire and digitize than outdoor images. We will usually refer to our examples, for brevity, as "leafy" textures. This paper addresses the photometry question posed in (b) above. (The visibility question posed in (a) was treated in an earlier paper. ~1) Recovery questions, which will be treated in a forthcoming paper, will be illustrated by studying the use of run length statistics of one-dimensional image "cross-sections" to obtain information about the distribution of texel sizes.) Section 2 describes our assumptions about scene illumination, and also discusses the need for camera calibration when computing gray-level histograms. Section 3 discusses reflectance and transmittance models; we find that the reflectance of leaves can be described fairly accurately by a Phong-type model, and we also suggest a model for leaf transmittance (which, however, is much smaller than leaf reflectance). Section 4 studies the gray-level histograms of leafy textures, both real and synthetic, under various conditions of illumination. As we shall see, the dependence of the real histograms on the direction of illumination resembles that of the synthetic histograms derived from a uniform distribution of leaf orientations. An earlier treatment of the gray-level probability densities of natural textures such as pebbles and a bush (wtih a dark background) can be found in Richards. ~z)
298
A. WAKSMAN and A. ROSENFELD 2. ILLUMINATION, SHADOWS AND CAMERA CALIBRATION
2.1. Illumination, background and shadows In an o u t d o o r scene, even when the sky is overcast, the illumination is not completely uniform; the sky is brightest in the direction of the sun 13) (if the overcast is not too heavy), a n d in any case the g r o u n d is m u c h less bright t h a n the sky. (4) Thus, if the sun is visible or the sky is only lightly overcast, most of the illumination
comes from a single c o m p a c t light source. M o s t of the examples in this p a p e r used illumination of this type, which is relatively easy to produce indoors. Figure 1 shows a set of i n d o o r images of a h o u s e p l a n t ill u m i n a t e d by a c o m p a c t light source, located at a distance of ca 2 m, either b e h i n d the c a m e r a or 90 ° away from it, at one of three elevations: 0, 30 and 45 :. In all six images, the c a m e r a position is the same, b u t the change in light source direction gives rise to substantial changes in the a p p e a r a n c e of the images.
o
-
t
£
¢ a. (0 °, 0°).
d, (90 °, 0°).
b. (0 °, 30°).
e. (90 °, 30°).
c. (0 °, 45°).
f. (90 °, 45°).
Fig. 1. A houseplant illuminated by a compact light source from various directions, specified by azimuth 0 or 90 ° and elevation 0, 30 or 45 °.
Photometry
~'i!.: ~ ~ ~
299
..
2
-~ •
Fig. 2. Plant image (0, 30°): (a) original; (b) with the background masked out.
In Fig. 1, the plant is imaged against a dark background; this produces sharp, tall peaks at the low ends of the histograms of the images. Since we want to study the parts of the. histograms produced by the texture, it is desirable to suppress these peaks. We can do this by choosing a threshold which eliminates the background but almost none of the plant in the frontally lit image [Fig. l(a)]; the below-threshold pixels can then be used to create a mask to eliminate the background from all the images. An example of the results of eliminating the background and major stems is shown in Fig. 2. (We did not attempt to completely eliminate the stems, but they constitute only a small percentage of the plant area.) Note that in an outdoor image the background can be arbitrary and is not easy to eliminate unless it is of a very simple nature (e.g. the sky). The histograms of the images in Fig. 1, after masking, are shown in Fig. 3. A compact light source produces relatively sharp shadows. A point P is in shadow if the light source is not visible from P; the probability of this happening is greater if P is deeper inside the texture. In Fig. l(d), the light sourceis 90 ° to the left of the viewing direction; we can see that there are more shadows towards the right-hand side of the plant. To confirm this, Fig. 4 plots the mean gray level in a sliding vertical strip of Fig. l(d); the darkening is quite apparent. Note that in Fig. l(a), the light source is behind the camera, so that shadows are not visible. Plant images obtained under diffuse lighting tend to look quite different from Figs 1 and 2. Figure 5(a) shows an image ofa houseplant obtained by occluding the (compact) light source with a screen, so that the main source of illumination was the light reflected from the walls behind and on both sides of the plant. This setup resulted in a diffusely side- and backlit image with a bright ("skylike') background. The backlighting gave rise to highlights at the edges of the stems and m a n y of the leaves; these are quite conspicuous when we mask out the background and redisplay it as black [Fig. 5(b)]. The histogram of the masked image is shown in Fig. 6; note that the gray levels cover a much wider range than those of the compactly illuminated images. (The peak that remains at the high
end of the histogram after masking represents the highlights at the edges of the leaves and stems, which result from the side- and backlighting.) In the rest of this paper we will use examples involving compact illumination in order to study the effects of direction of illumination on the histogram. 2.2. Camera calibration If the camera is not calibrated, the pixel values will vary monotonically, but not linearly, with light intensity; as a result, intensity measurements as well as the shapes of image histograms will be distorted. However, the values can be calibrated by measuring pixel values obtained from the same point on a brightly lit white surface, for different distances of the light from the surface or for different f-stop settings. Moving the light source 1.4 times as far from the surface reduces the illumination by one-half. Similarly, closing the camera's aperture by one f-stop (e.g. f/1.4 to 1"/2, or f/2 to 1"/2.8) reduces the aperture area by one-half. Thus, in both cases, the pixel intensity values should be halved. To calibrate our RCA C C D camera, we fitted a pixel value correction function f(x) to triples of pixel values obtained using three consecutive f-stops; these values should be in images obtained at approximately a 4:2:1 ratio. For our camera we found that f(x) = 42 + 0.51x + 0.0013x 2, as shown in Fig. 7, provided a good fit. The same f(x) also provided a good fit to the inverse-square function when the surface was illuminated from different known distances.
3. R E F L E C T A N C E A N D T R A N S M I T T A N C E
MODELS
The brightness of a texel at a point depends on the a m o u n t of incident light reflected or transmitted by the texel at that point. (For a comprehensive presentation of reflectance and transmittance terminology, see the detailed glossary by ASTMJ 5)) In this section we discuss some standard models for reflectance, and also fit such a model to the reflectance data obtained from a real leaf. We also define a leaf transmittance model and fit it to real leaf data.
300
A. WAKSMAN and A. ROSENFELD
t
5O
1130
150
200
50
(o °, 0°).
~L 100
150
200
d. (90 °, 0°).
I
150
200
50
b. (0 °, 30°).
150
100
150
200
e. (90 °, 30°1 .
I
100
I00
200
c. (0 °, 45°).
50
L 100
, 150
200
f. (90 °, 45°).
Fig. 3. Histograms of the images in Fig 1 with the background deleted.
3.1.
Reflectance
A reflectance model gives the reflectance r (the ratio of reflected to incident light intensity) at a point P, taking into account the normalized unit vectors N, the surface normal, L, the light source direction, and E, the viewpoint ("eye") direction, as well as the surface properties at P. (Color can be handled by also taking into account the wavelengths of light involved, but color will not be considered here.).Clearly, reflected light can
(directly) reach the viewer only when the light source and the viewer are on the same side of the surface ((N. L) (N- E) > 0). • Lambert's law ¢6'7) (see Fig. 8) models diffuse reflection from a surface, where light is reflected equally in all directions. As the incidence angle of the light (the angle between the beam of light and the surface normal) increases, the light is spread over a larger surface area. The fraction of light reflected is proportional to
Photometry
76
i
i
f
i
301
i
,
,
i
72
70 68 66 64 62 60 58 56 50
,
t
,
100
150
200
,
,
,
250 300 350 Position of strip center
,
400
450
500
Fig. 4. Plot of vertical strip gray-level averages for Fig. l(d).
Fig. 5. A houseplant under diffuse lighting: (a) original image; (b) imave with background masked out and redisplayed as black.
the ratio of the cross-section of the b e a m to that of the area it illuminates. In other words: zlcos/_NL[ = ~c,I(N-L)I
r=
50
100
150
200
250
Fig. 6. Histogram of Fig. 5, with the background deleted.
{0
if ( N ' L ) ( N ' E ) > 0 otherwise.
Here K~, k n o w n as the albedo, is a p r o p e r t y of the surface at P. N o t e t h a t the fraction of light reflected is i n d e p e n d e n t of the viewing direction E. • Models that i n c o r p o r a t e specular reflection allow for shiny or glossy highlights. They c o n c e n t r a t e reflected light near the mirror-reflection direction R (whose angle with N, called the specular angle, is equal to a n d opposite the angle betwen N a n d L), but with some spread a r o u n d t h a t direction [see Fig. 9(a)].
302
A. WAKSMAN and A. ROSENFELD 256 224 192 160 128 96 64 32 0
t
i
i
t
i
i
i
32
64
96
128
160
192
224
256
Fig. 7. Pixel value correction function.
form
N
(~llcos/_NL]+xpmax{cos~,0} v if ( N ' L ) ( N ' E ) > 0 r=
,/ ZNL
a 2
al g2
=
cos/NL
Fig. 8. Lambert's law (diffuse reflection), al is the beam crosssection and a z is the cross-section of the illuminated patch.
The P h o n g model (s) is perhaps the simplest widely used model of this type. The reflectance r has both a diffuse and a specular component; the specular component is r s = tcpmax{E-R, 0} p = lop max{cos~,0} p, where ~ is the angle between the mirror-reflection direction R and the viewing direction E [see Fig. 9(b)] and p is a measure of the specularity or "glossiness" of the surface. Thus, the total reflectance is of the
otherwise.
(As in the Lambertian case, we take the reflectance to be nonzero only when the light source and the viewer are on the same side of the reflecting plane.) Higher values of p produce sharper highlights; the limit of p = oe represents perfect mirror reflection. Figure 10 shows r s for - 9 0 ° < c~< 90 ° and various values of p. To measure the reflectance of a real leaf, we mounted the leaf on a turntable (9) and took digital images under illumination by a compact light source ca 0.5 m away at azimuth 90 ° (relative to the camera axis) and elevation 0 °, as shown schematically in Fig. 11. We computed the average gray level of a circular patch near the center of the leaf; the patch diameter was ca 1 cm, or 50 pixels, in the digitized image. The camera was calibrated using the function f ( x ) defined in Section 2.2. The average corrected values are plotted as a function of turntable angle in Fig. 12. The curve in Fig. 12 is the P h o n g reflectance function corresponding to K~= 105, tcv = 96 and p = 13.3; it provides a very good fit to the plotted points.
N
N
Fig. 9. Near-specular reflection. (b) Angle a in the Phong model.
Photometry
303
M o r e sophisticated models (for example Torrance and Sparrow (lm and Beckmann and Spizzichinoll~; see N a y a r 17~)take other factors such as the light source direction L into account in computing the specular component, but we will not consider them here. 3.2.
~-90
-60
-30
0 a
30
60
Transmittance
When the light source and the viewer are on opposite sides of the leaf plane ( ( N - L ) ( N . E ) < 0), the leaves transmit some light, but typically much less than they reflect. F o r example, when we illuminated a leaf frontally, the reflected intensity was about four times the transmitted intensity (measured with the light source at the same distance on the opposite side of the leaf). To model leaf transmittance, we assume that a leaf has a cellular structure, as illustrated in Fig. 13. Light entering the leaf at angle 0 to the normal is attenuated by an amount that is exponential in the thickness of the leaf in the direction O, i.e. an a m o u n t proportional to A sec°,where A is the attenuation at 0 = 0. If we assume for simplicity that the cell walls perpendicular to the leaf surface are opaque, they block a fraction of the light given by 1 - t/sltan 01,where t is the leaf thickness and s is the spacing of the walls. Thus, the total attenuation is of the form
90
Fig. 10. The specular component of the Phong model as a function of c~(p = 1, 2, 5, 10, 20, 50, 100, 200, 500, 1000). Narrower curves correspond to higher values of p.
-~@C ..................
A~eC°(1-~tanO).
Fig. ll.
Experimental
We fitted this transmittance model to data obtained from a real leaf mounted on a turntable, with the light source and camera on opposite sides of the leaf, as illustrated in Fig. 14. The formula predicted the measured transmittance values quite well over turntable
arrangement for measuring leaf reflectance.
180 q'/
160 140 /
120
~k
o,,' /
*/ 100 80 Q r)
60
o ,, "-..
40
""-... 20 "'. 0
i
i
i
10
20
30
i
i
i
40 50 60 Turntable angle, degrees
i
i
70
80
Fig. 12. Phong model fit to measured leaf reflectance values. PR zg-2-F
90
304
A. WAKSMAN and A. ROSENFELD
I~
s
(2) the texels are planar and of constant size (usually we used disks); (3) the illumination comes from a distant point light source; (4) the texels have the reflectance function described in Section 3. (We will ignore transmittance, since as already mentioned, it is typically much smaller than the reflectance.)
P-I
Fig. 13. Cell wall model of a leaf.
Fig. 14. Experimental arrangement for measuring leaf transmittance.
angles in the range - 4 0 ° _ < 0 _ < 4 0 ° when we set A = 0.19703 and t/s = 0.26945, as shown in Fig. 15. 4. HISTOGRAMSOF LEAFY TEXTURES The gray-level histogram of a leafy texture (with the background eliminated) depends on (1) the distribution of the positions and orientations of the texels; (2) the distribution of the sizes and shapes of the texels; (3) the illumination; (4) the reflectance and transmittance of the texels. In the synthetic examples used in Section 4.1: (1) the texel positions are randomly distributed in the volume of space being viewed by the camera;
We shall see in Section 4.1 how the shape of the histogram depends on the direction of illumination, for three simple models of the distribution of texel orientations. We shall also see that the histogram shape is relatively insensitive to the spatial density of the texels and to their distribution of sizes and shapes. (We have found (I) that visibility through such a texture is also quite insensitive to texel shape, but is quite sensitive to spatial density and texel size.) In Section 4.2 we shall see that the plant image histograms shown in Fig. 3 behave similarly, under changes in light source direction, to the synthetic histograms obtained using a uniform distribution of texel orientations.
4.1. Histograms of synthetic textures To study the dependence of a leafy texture's histogram on illumination, we generated a set of synthetic leafy textures, using texels which were either disks or triangles of constant size as in our earlier paper. ") We used three models for the orientations of the texels: (a) uniform over the Gaussian sphere; (b) "drooping", with all the texels vertical (leaving one rotational degree of freedom); and (c) "sagging", where the texels' surface normals make a given angle with the vertical (here 45°). We modeled the illumination as coming from a single infinitely distant point source at azimuth 0 or 90 °, and at elevation 0, 30 or 45 °. The texels' reflectance was given by a Phong model as in Section 3 with
200 .9" // 150
"-t,
I/"
,?,
"Ib,
/ I/
,
I00 /•,' o /'/
',, •
50
0
I
-60
-40
-20
I
i
I
0
20
40
Turntable angle, degrees
Fig. 15. Leaf transmittance model fit to measured values.
60
Photometry
305
' a . (0 °, 0°).
d. (90 °, 0°).
b. (0 °, 30 ° ).
e. (90 °, 30 °).
c. (0 °, 45°).
f. (90 °, 45°).
Fig. 16. A texture with uniform texel orientation distribution, under point source illumination from different directions.
p = 13.3, corresponding to our actual leaf measurements. Shadows were not incorporated into the simulation. Figure 16 shows synthetic textures generated using the uniform orientation distribution and the six light source directions. The texels are disk-shaped and have a 13-pixel radius; the probability that an image pixel is the center of a texel is 0.0022. The histograms of these
textures (not including the background) are shown in Fig. 17. To see that the shapes of the histograms are not sensitive to the texels' sizes, shapes or spatial density, we generated textures in which the texels are diskshaped but have radius 9 pixels; triangular-shaped, with the same area as the original disks; and diskshaped With the original radius, but with probability
306
A. WAKSMAN and A. ROSENFELD
50
1(30
150
200
250
0
50
(o °, o°).
50
100
150
100
150
lfiO
200
250
200
250
200
250
d. (90 °, 0°).
200
250
50
I00
150
e. (90 °, 30°).
b. (o o, 3o°1.
50
100
200
250
50
100
150
f. (90 °, 45°).
c. (0 °, 45°1 . Fig. 17. Histograms for Fig. 16.
0.0011 of a texel center at each image pixel. These variants are shown in Fig. 18 for two light source positions: azimuth 0 °, elevation 0 ° and azimuth 90 °, elevation 0 °. The histograms of the variants are shown in Fig. 19; we see that they closely resemble the histograms in Figs 17(a) and (d) respectively. This result is not surprising since the leaf orientations in our examples are independent of position, so the visible
leaf pixels are a representative sample of the leaves as a whole. The effects of the light source position are more significant. F o r azimuth 0 °, as the light's elevation increases, there is a noticeable tendency for the histogram to shift towards darker gray levels [Fig. 17(a)(c)]. This is to be expected since the largest texel areas are frontal and become less bright as the elevation
Photometry
A z i m u t h 0 °, e l e v a t i o n 0 °
~,
...... ~/Jlll~_
307
A z i m u t h 90 °, e l e v a t i o n 0 °
q "'~, Q#
Fig. 18. Texture variants with uniform orientation distribution.
of the light source (and hence the incidence angle) increases. For azimuth 90 °, on the other hand, the incident light rays are perpendicular to the viewing direction and this remains true as the light's elevation changes; since a uniform orientation distribution is isotropic, the relationship between visible texel area and incidence angle (and hence reflected intensity) remains approximately the same for all elevations. Analogous observations about the histograms can be made for the other two texel orientation models.
The images ~nd histograms for the six light source positions are shown in Fig. 20 and 21 for the "drooping" orientation model and in Fig. 22 and 23 for the "sagging" model. In the case of the drooping model, for both azimuths (0 and 90 °) there is substantial compression of the histogram with increasing elevation of the light source. This is to be expected, since for light source elevation 0 ° the drooping model yields a full range of incidence angles, but for higher elevations this range is compressed. For the sagging model, this situation is reversed: we have a full range of
308
A. WAKSMAN and A. ROSENFELD
A z i m u t h 0 °, e l e v a t i o n 0 °
0
A z i m u t h 90 °, e l e v a t i o n 0 °
50
I00
150
50
I00
150
200
250
50
100
150
200
250
50
100
150
200
:250
O"
50
100
150
201)
250
0
50
le~
150
200
Fig. 19. Histograms of the texture variants.
incidence angles only for light source elevation 45 °, but a compressed range for lower elevations, so that the histogram range expands as the elevation increases from 0 to 45 °. (The range would shrink again for higher elevations, which are not shown here.) Thus, for all three models, the effects of the light source position on the shape of the histogram are qualitatively predictable.
4.2. Histograms of real textures The real plant images shown in Fig. 1 do not satisfy any of our simple texel orientation models, but the qualitative observations of Section 4.1 hold for the histograms of the real images (with backgrounds masked out). In particular, for azimuth 0 ° [Figs 3(a) (c)] the histogram compresses with increasing light source
Photometry
309
,_
a. (0 °, 0°).
d. (90 °, 0°).
b. (0 °, 30°).
e. (90 °, 30°).
c. (0 °, 45°).
f. (90 °, 45°).
Fig. 20. A texture with "drooping" texel orientation distribution, under point source illumination from different directions.
elevation and for azimuth 90 ° [Figs 3(d)-(f)], there is little change with elevation; thus, the behavior of the real histograms under change in light source position is analogous to that of the synthetic histograms obtained from the uniform orientation model. It is not surprising that the histograms obtained from the drooping and sagging models behave differently than those of the real plant images; for those models, the
texel orientations are much less isotropic indeed, they have only one degree of freedom. Note that the real images contain shadows and interreflections, which were not modeled in the synthetic images; but the behavior of the histograms is actually reinforced by the presence of the shadows. Indeed, for azimuth 0 ° and elevation 0 °, the shadows are not visible because the light source is directly
310
A. W A K S M A N and A. R O S E N F E L D
50
1OO
150
200
250
0
50
~. (o °, o°).
50
IO0
i 150
,
i
100
150
~. (o °, 45°).
150
200
250
200
250
200
250
d. (90 °, 0°).
2(30
250
50
b. (o o, 30°).
50
I00
100
i 150
e. (90 °, 30°).
L
200
250
50
100
150
f. (90 o, 45o/. Fig. 21. Histograms of a "drooping" texture.
Photometry
v
i V
3
~
v
i V
;Y_.¢. ~. (0 o, 0o).
d. (90 o, 0o).
S qlp~
~\
~
..t b. (0 °, 30°). v
p\
O ~
e. (90 °, 30°).
O ~v "
at c. (0 °, 45°).
%
v
O~j J,
/ f. (90 °, 45°).
Fig. 22. A texture with "sagging" texel orientation distribution, under point source illumination from different directions.
312
A. WAKSMAN and A. ROSENFELD
I
50
I00
&,
50
200
150
250
50
(oo, oo).
IUO
100
150
200
250
200
250
2OO
250
d. (90 °, 0°).
150
200
250
50
b. (o o, 30°).
I00
150
e. (90 °, 3o°1 .
i
100
150
200
250
50
100
150
f. (90 °, 45°).
c. (0 °, 45°).
Fig. 23. Histograms of a "sagging" texture.
behind the viewer; as the elevation increases, shadows are increasingly visible, which contributes to a shift of the histogram towards dark gray levels. For azimuth 90 °, on the other hand, shadows are visible for all elevations. 5. C O N C L U D I N G
REMARKS
Quantitative modeling of the gray-level histograms of real leafy textures would, be difficult, because the
illumination is more complex (sun and sky, in outdoor images; multiple nonpoint-like light sources, in indoor situations); shadows, interreflections and transmittance must be taken into account; the texels are often not planar; and they do not have a simple distribution of orientations. Nevertheless, we have seen that it is possible to qualitatively predict how the histograms of real plant images vary with the direction of (predominant) illumination. In fact, we have found that this variation resembles that of synthetic histograms oh-
Photometry
tained from p o i n t source illumination a n d a uniform o r i e n t a t i o n model, ignoring t r a n s m i t t a n c e , interreflections a n d shadows.
5. 6.
REFERENCES
1. A. Waksman and A. Rosenfeld, Sparse, opaque threedimensional texture, 2a: Visibility, Graphical Models and Image Processing, in press; also available as Technical Report CAR-TR-729 (CS-TR-3333), Center for Automation Research, University of Maryland at College Park (1994). 2. W.A. Richards, Lightness scale from image intensity distributions, Appl. Opt. 21(4), 2569-2582 (15 July 1982). 3. L. Levi, Applied Optics: A Guide to Optical System Design, Vol. 2, p. 105. J. Wiley, New York (1968). 4. M. S. Langer and S. W. Zucker, Shape from Shading on a Cloudy Day, Technical Report TR-CIM-91-7, Computer Vision and Robotics Laboratory, Research Centre
7. 8. 9. 10. 11.
for Intelligent Machines, McGill University, Montr6al, Qu6bec, Canada (May 1992). American Society for Testing and Materials, Standard Terminology of Appearance, E284-91, Annual Book oj A S T M Standards. Philadelphia, PA (1991). J. H. Lambert, Photometria sire de mensura de gratibus luminis, colorum et umbrae. Eberhard Klett, Augsberg, Germany (1760). S.K. Nayar, K. Ikeuchi and T. Kanade, Surface reflection: physical and geometrical perspectives, IEEE Trans. Pattern Anal. Math. Intell. 13, 611-634 (1991). B. Phong, Illumination for computer generated pictures, Commun. ACM. 18, 311-317 (1975). S.J. Sickels, A Virtual Instrument Interface for Camera Motion Control, M.S. thesis, Department of Electrical Engineering, University of Maryland at College Park (1994). K. Torrance and E. Sparrow, Theory for off-specular reflection from roughened surfaces, J. Opt. Soc. Am. 57, 1105-1114 (1967). P. Beckmann and A. Spizzichino, The Scattering of Electromagnetic Waves from Rough Surfaces. Pergamon Press, New York (1963).
About the Author--ADLAI WAKSMAN is a Ph.D. candidate in the Computer and Information Science
Department at the University of Pennsylvania under Ruzena Bajcsy. He is currently doing research with Azriel Rosenfeld at the University of Maryland at College Park.
AZRIEL ROSENFELD is Director of the Center for Automation Research, a department-level unit of the College of Computer, Mathematical and Physical Sciences at the University of Maryland at College Park. Professor Rosenfeld's research interests include many aspects of image analysis and computer vision. About the Author
313
Pergamon
0031-3203(95)00080-1
SPARSE, OPAQUE THREE-DIMENSIONAL TEXTURE, 2B: PHOTOMETRY A D L A I W A K S M A N and A Z R I E L R O S E N F E L D * Computer Vision Laboratory, Center for Automation Research, University of Maryland, College Park, MD 20742-3275, U.S.A.
(Received 18 November 1994; in revisedform 2 May 1995; receivedfor publication 2 June 1995) Abstract--This paper deals with three-dimensional (3D) textures composed of approximately planar texels distributed over a volume of space ("leafy" textures). It studies the gray-level histograms of images of such textures (specifically houseplants) under illumination by a compact light source. The variation of such histograms with light source direction resembles that of synthetic histograms generated using a Phong-type reflectance model and a uniform texel orientation model, and ignoring transmittance, interreflection and shadows. Histogram
Houseplant
Photometry
Reflectance
1. INTRODUCTION Natural scenes contain many phenomena, such as leaves in a tree crown and falling snowflakes, which involve discrete, relatively sparse and opaque elements or "texels" distributed over a given volume of space. In this and two companion papers, we study three questions about such 3D textures: (a) (Visibility) H o w far into the texture can we see from a given viewpoint? W h a t is the probability that a given viewing ray will pass all the way through it? (b) (Photometry) Assuming that we know how light is reflected from (and possibly transmitted by) the texels, and given an illumination model, what will the gray-level histogram of the texture look like in an image taken from a given viewpoint? (c) (Recovery) What properties of the texel population can be recovered from single images? In these papers we regard textures such as snow and tree crowns as composed primarily of (approximately) planar texels (flakes or leaves) randomly distributed in space. In an image of a sparse 3D texture, both texels and background will be visible. In order to study questions such as those posed above, we must assume that the
This work was supported by the Defense Advanced Research Projects Agency (ARPA Order No. 8459) and the U.S. Army Topographic Engineering Center under Contract DACA76-92-C-0009. The authors thank Sandy German for providing the houseplant and for her expert help in preparing this paper. * Author to whom correspondence should be addressed. 297
3D texture
texels can be distinguished from the background. This is sometimes easy even for natural scenes--for example, in the case of a tree crown imaged against the sky. The examples used in this paper are indoor images of plants taken against a black background; these were more convenient to acquire and digitize than outdoor images. We will usually refer to our examples, for brevity, as "leafy" textures. This paper addresses the photometry question posed in (b) above. (The visibility question posed in (a) was treated in an earlier paper. ~1) Recovery questions, which will be treated in a forthcoming paper, will be illustrated by studying the use of run length statistics of one-dimensional image "cross-sections" to obtain information about the distribution of texel sizes.) Section 2 describes our assumptions about scene illumination, and also discusses the need for camera calibration when computing gray-level histograms. Section 3 discusses reflectance and transmittance models; we find that the reflectance of leaves can be described fairly accurately by a Phong-type model, and we also suggest a model for leaf transmittance (which, however, is much smaller than leaf reflectance). Section 4 studies the gray-level histograms of leafy textures, both real and synthetic, under various conditions of illumination. As we shall see, the dependence of the real histograms on the direction of illumination resembles that of the synthetic histograms derived from a uniform distribution of leaf orientations. An earlier treatment of the gray-level probability densities of natural textures such as pebbles and a bush (wtih a dark background) can be found in Richards. ~z)
298
A. WAKSMAN and A. ROSENFELD 2. ILLUMINATION, SHADOWS AND CAMERA CALIBRATION
2.1. Illumination, background and shadows In an o u t d o o r scene, even when the sky is overcast, the illumination is not completely uniform; the sky is brightest in the direction of the sun 13) (if the overcast is not too heavy), a n d in any case the g r o u n d is m u c h less bright t h a n the sky. (4) Thus, if the sun is visible or the sky is only lightly overcast, most of the illumination
comes from a single c o m p a c t light source. M o s t of the examples in this p a p e r used illumination of this type, which is relatively easy to produce indoors. Figure 1 shows a set of i n d o o r images of a h o u s e p l a n t ill u m i n a t e d by a c o m p a c t light source, located at a distance of ca 2 m, either b e h i n d the c a m e r a or 90 ° away from it, at one of three elevations: 0, 30 and 45 :. In all six images, the c a m e r a position is the same, b u t the change in light source direction gives rise to substantial changes in the a p p e a r a n c e of the images.
o
-
t
£
¢ a. (0 °, 0°).
d, (90 °, 0°).
b. (0 °, 30°).
e. (90 °, 30°).
c. (0 °, 45°).
f. (90 °, 45°).
Fig. 1. A houseplant illuminated by a compact light source from various directions, specified by azimuth 0 or 90 ° and elevation 0, 30 or 45 °.
Photometry
~'i!.: ~ ~ ~
299
..
2
-~ •
Fig. 2. Plant image (0, 30°): (a) original; (b) with the background masked out.
In Fig. 1, the plant is imaged against a dark background; this produces sharp, tall peaks at the low ends of the histograms of the images. Since we want to study the parts of the. histograms produced by the texture, it is desirable to suppress these peaks. We can do this by choosing a threshold which eliminates the background but almost none of the plant in the frontally lit image [Fig. l(a)]; the below-threshold pixels can then be used to create a mask to eliminate the background from all the images. An example of the results of eliminating the background and major stems is shown in Fig. 2. (We did not attempt to completely eliminate the stems, but they constitute only a small percentage of the plant area.) Note that in an outdoor image the background can be arbitrary and is not easy to eliminate unless it is of a very simple nature (e.g. the sky). The histograms of the images in Fig. 1, after masking, are shown in Fig. 3. A compact light source produces relatively sharp shadows. A point P is in shadow if the light source is not visible from P; the probability of this happening is greater if P is deeper inside the texture. In Fig. l(d), the light sourceis 90 ° to the left of the viewing direction; we can see that there are more shadows towards the right-hand side of the plant. To confirm this, Fig. 4 plots the mean gray level in a sliding vertical strip of Fig. l(d); the darkening is quite apparent. Note that in Fig. l(a), the light source is behind the camera, so that shadows are not visible. Plant images obtained under diffuse lighting tend to look quite different from Figs 1 and 2. Figure 5(a) shows an image ofa houseplant obtained by occluding the (compact) light source with a screen, so that the main source of illumination was the light reflected from the walls behind and on both sides of the plant. This setup resulted in a diffusely side- and backlit image with a bright ("skylike') background. The backlighting gave rise to highlights at the edges of the stems and m a n y of the leaves; these are quite conspicuous when we mask out the background and redisplay it as black [Fig. 5(b)]. The histogram of the masked image is shown in Fig. 6; note that the gray levels cover a much wider range than those of the compactly illuminated images. (The peak that remains at the high
end of the histogram after masking represents the highlights at the edges of the leaves and stems, which result from the side- and backlighting.) In the rest of this paper we will use examples involving compact illumination in order to study the effects of direction of illumination on the histogram. 2.2. Camera calibration If the camera is not calibrated, the pixel values will vary monotonically, but not linearly, with light intensity; as a result, intensity measurements as well as the shapes of image histograms will be distorted. However, the values can be calibrated by measuring pixel values obtained from the same point on a brightly lit white surface, for different distances of the light from the surface or for different f-stop settings. Moving the light source 1.4 times as far from the surface reduces the illumination by one-half. Similarly, closing the camera's aperture by one f-stop (e.g. f/1.4 to 1"/2, or f/2 to 1"/2.8) reduces the aperture area by one-half. Thus, in both cases, the pixel intensity values should be halved. To calibrate our RCA C C D camera, we fitted a pixel value correction function f(x) to triples of pixel values obtained using three consecutive f-stops; these values should be in images obtained at approximately a 4:2:1 ratio. For our camera we found that f(x) = 42 + 0.51x + 0.0013x 2, as shown in Fig. 7, provided a good fit. The same f(x) also provided a good fit to the inverse-square function when the surface was illuminated from different known distances.
3. R E F L E C T A N C E A N D T R A N S M I T T A N C E
MODELS
The brightness of a texel at a point depends on the a m o u n t of incident light reflected or transmitted by the texel at that point. (For a comprehensive presentation of reflectance and transmittance terminology, see the detailed glossary by ASTMJ 5)) In this section we discuss some standard models for reflectance, and also fit such a model to the reflectance data obtained from a real leaf. We also define a leaf transmittance model and fit it to real leaf data.
300
A. WAKSMAN and A. ROSENFELD
t
5O
1130
150
200
50
(o °, 0°).
~L 100
150
200
d. (90 °, 0°).
I
150
200
50
b. (0 °, 30°).
150
100
150
200
e. (90 °, 30°1 .
I
100
I00
200
c. (0 °, 45°).
50
L 100
, 150
200
f. (90 °, 45°).
Fig. 3. Histograms of the images in Fig 1 with the background deleted.
3.1.
Reflectance
A reflectance model gives the reflectance r (the ratio of reflected to incident light intensity) at a point P, taking into account the normalized unit vectors N, the surface normal, L, the light source direction, and E, the viewpoint ("eye") direction, as well as the surface properties at P. (Color can be handled by also taking into account the wavelengths of light involved, but color will not be considered here.).Clearly, reflected light can
(directly) reach the viewer only when the light source and the viewer are on the same side of the surface ((N. L) (N- E) > 0). • Lambert's law ¢6'7) (see Fig. 8) models diffuse reflection from a surface, where light is reflected equally in all directions. As the incidence angle of the light (the angle between the beam of light and the surface normal) increases, the light is spread over a larger surface area. The fraction of light reflected is proportional to
Photometry
76
i
i
f
i
301
i
,
,
i
72
70 68 66 64 62 60 58 56 50
,
t
,
100
150
200
,
,
,
250 300 350 Position of strip center
,
400
450
500
Fig. 4. Plot of vertical strip gray-level averages for Fig. l(d).
Fig. 5. A houseplant under diffuse lighting: (a) original image; (b) imave with background masked out and redisplayed as black.
the ratio of the cross-section of the b e a m to that of the area it illuminates. In other words: zlcos/_NL[ = ~c,I(N-L)I
r=
50
100
150
200
250
Fig. 6. Histogram of Fig. 5, with the background deleted.
{0
if ( N ' L ) ( N ' E ) > 0 otherwise.
Here K~, k n o w n as the albedo, is a p r o p e r t y of the surface at P. N o t e t h a t the fraction of light reflected is i n d e p e n d e n t of the viewing direction E. • Models that i n c o r p o r a t e specular reflection allow for shiny or glossy highlights. They c o n c e n t r a t e reflected light near the mirror-reflection direction R (whose angle with N, called the specular angle, is equal to a n d opposite the angle betwen N a n d L), but with some spread a r o u n d t h a t direction [see Fig. 9(a)].
302
A. WAKSMAN and A. ROSENFELD 256 224 192 160 128 96 64 32 0
t
i
i
t
i
i
i
32
64
96
128
160
192
224
256
Fig. 7. Pixel value correction function.
form
N
(~llcos/_NL]+xpmax{cos~,0} v if ( N ' L ) ( N ' E ) > 0 r=
,/ ZNL
a 2
al g2
=
cos/NL
Fig. 8. Lambert's law (diffuse reflection), al is the beam crosssection and a z is the cross-section of the illuminated patch.
The P h o n g model (s) is perhaps the simplest widely used model of this type. The reflectance r has both a diffuse and a specular component; the specular component is r s = tcpmax{E-R, 0} p = lop max{cos~,0} p, where ~ is the angle between the mirror-reflection direction R and the viewing direction E [see Fig. 9(b)] and p is a measure of the specularity or "glossiness" of the surface. Thus, the total reflectance is of the
otherwise.
(As in the Lambertian case, we take the reflectance to be nonzero only when the light source and the viewer are on the same side of the reflecting plane.) Higher values of p produce sharper highlights; the limit of p = oe represents perfect mirror reflection. Figure 10 shows r s for - 9 0 ° < c~< 90 ° and various values of p. To measure the reflectance of a real leaf, we mounted the leaf on a turntable (9) and took digital images under illumination by a compact light source ca 0.5 m away at azimuth 90 ° (relative to the camera axis) and elevation 0 °, as shown schematically in Fig. 11. We computed the average gray level of a circular patch near the center of the leaf; the patch diameter was ca 1 cm, or 50 pixels, in the digitized image. The camera was calibrated using the function f ( x ) defined in Section 2.2. The average corrected values are plotted as a function of turntable angle in Fig. 12. The curve in Fig. 12 is the P h o n g reflectance function corresponding to K~= 105, tcv = 96 and p = 13.3; it provides a very good fit to the plotted points.
N
N
Fig. 9. Near-specular reflection. (b) Angle a in the Phong model.
Photometry
303
M o r e sophisticated models (for example Torrance and Sparrow (lm and Beckmann and Spizzichinoll~; see N a y a r 17~)take other factors such as the light source direction L into account in computing the specular component, but we will not consider them here. 3.2.
~-90
-60
-30
0 a
30
60
Transmittance
When the light source and the viewer are on opposite sides of the leaf plane ( ( N - L ) ( N . E ) < 0), the leaves transmit some light, but typically much less than they reflect. F o r example, when we illuminated a leaf frontally, the reflected intensity was about four times the transmitted intensity (measured with the light source at the same distance on the opposite side of the leaf). To model leaf transmittance, we assume that a leaf has a cellular structure, as illustrated in Fig. 13. Light entering the leaf at angle 0 to the normal is attenuated by an amount that is exponential in the thickness of the leaf in the direction O, i.e. an a m o u n t proportional to A sec°,where A is the attenuation at 0 = 0. If we assume for simplicity that the cell walls perpendicular to the leaf surface are opaque, they block a fraction of the light given by 1 - t/sltan 01,where t is the leaf thickness and s is the spacing of the walls. Thus, the total attenuation is of the form
90
Fig. 10. The specular component of the Phong model as a function of c~(p = 1, 2, 5, 10, 20, 50, 100, 200, 500, 1000). Narrower curves correspond to higher values of p.
-~@C ..................
A~eC°(1-~tanO).
Fig. ll.
Experimental
We fitted this transmittance model to data obtained from a real leaf mounted on a turntable, with the light source and camera on opposite sides of the leaf, as illustrated in Fig. 14. The formula predicted the measured transmittance values quite well over turntable
arrangement for measuring leaf reflectance.
180 q'/
160 140 /
120
~k
o,,' /
*/ 100 80 Q r)
60
o ,, "-..
40
""-... 20 "'. 0
i
i
i
10
20
30
i
i
i
40 50 60 Turntable angle, degrees
i
i
70
80
Fig. 12. Phong model fit to measured leaf reflectance values. PR zg-2-F
90
304
A. WAKSMAN and A. ROSENFELD
I~
s
(2) the texels are planar and of constant size (usually we used disks); (3) the illumination comes from a distant point light source; (4) the texels have the reflectance function described in Section 3. (We will ignore transmittance, since as already mentioned, it is typically much smaller than the reflectance.)
P-I
Fig. 13. Cell wall model of a leaf.
Fig. 14. Experimental arrangement for measuring leaf transmittance.
angles in the range - 4 0 ° _ < 0 _ < 4 0 ° when we set A = 0.19703 and t/s = 0.26945, as shown in Fig. 15. 4. HISTOGRAMSOF LEAFY TEXTURES The gray-level histogram of a leafy texture (with the background eliminated) depends on (1) the distribution of the positions and orientations of the texels; (2) the distribution of the sizes and shapes of the texels; (3) the illumination; (4) the reflectance and transmittance of the texels. In the synthetic examples used in Section 4.1: (1) the texel positions are randomly distributed in the volume of space being viewed by the camera;
We shall see in Section 4.1 how the shape of the histogram depends on the direction of illumination, for three simple models of the distribution of texel orientations. We shall also see that the histogram shape is relatively insensitive to the spatial density of the texels and to their distribution of sizes and shapes. (We have found (I) that visibility through such a texture is also quite insensitive to texel shape, but is quite sensitive to spatial density and texel size.) In Section 4.2 we shall see that the plant image histograms shown in Fig. 3 behave similarly, under changes in light source direction, to the synthetic histograms obtained using a uniform distribution of texel orientations.
4.1. Histograms of synthetic textures To study the dependence of a leafy texture's histogram on illumination, we generated a set of synthetic leafy textures, using texels which were either disks or triangles of constant size as in our earlier paper. ") We used three models for the orientations of the texels: (a) uniform over the Gaussian sphere; (b) "drooping", with all the texels vertical (leaving one rotational degree of freedom); and (c) "sagging", where the texels' surface normals make a given angle with the vertical (here 45°). We modeled the illumination as coming from a single infinitely distant point source at azimuth 0 or 90 °, and at elevation 0, 30 or 45 °. The texels' reflectance was given by a Phong model as in Section 3 with
200 .9" // 150
"-t,
I/"
,?,
"Ib,
/ I/
,
I00 /•,' o /'/
',, •
50
0
I
-60
-40
-20
I
i
I
0
20
40
Turntable angle, degrees
Fig. 15. Leaf transmittance model fit to measured values.
60
Photometry
305
' a . (0 °, 0°).
d. (90 °, 0°).
b. (0 °, 30 ° ).
e. (90 °, 30 °).
c. (0 °, 45°).
f. (90 °, 45°).
Fig. 16. A texture with uniform texel orientation distribution, under point source illumination from different directions.
p = 13.3, corresponding to our actual leaf measurements. Shadows were not incorporated into the simulation. Figure 16 shows synthetic textures generated using the uniform orientation distribution and the six light source directions. The texels are disk-shaped and have a 13-pixel radius; the probability that an image pixel is the center of a texel is 0.0022. The histograms of these
textures (not including the background) are shown in Fig. 17. To see that the shapes of the histograms are not sensitive to the texels' sizes, shapes or spatial density, we generated textures in which the texels are diskshaped but have radius 9 pixels; triangular-shaped, with the same area as the original disks; and diskshaped With the original radius, but with probability
306
A. WAKSMAN and A. ROSENFELD
50
1(30
150
200
250
0
50
(o °, o°).
50
100
150
100
150
lfiO
200
250
200
250
200
250
d. (90 °, 0°).
200
250
50
I00
150
e. (90 °, 30°).
b. (o o, 3o°1.
50
100
200
250
50
100
150
f. (90 °, 45°).
c. (0 °, 45°1 . Fig. 17. Histograms for Fig. 16.
0.0011 of a texel center at each image pixel. These variants are shown in Fig. 18 for two light source positions: azimuth 0 °, elevation 0 ° and azimuth 90 °, elevation 0 °. The histograms of the variants are shown in Fig. 19; we see that they closely resemble the histograms in Figs 17(a) and (d) respectively. This result is not surprising since the leaf orientations in our examples are independent of position, so the visible
leaf pixels are a representative sample of the leaves as a whole. The effects of the light source position are more significant. F o r azimuth 0 °, as the light's elevation increases, there is a noticeable tendency for the histogram to shift towards darker gray levels [Fig. 17(a)(c)]. This is to be expected since the largest texel areas are frontal and become less bright as the elevation
Photometry
A z i m u t h 0 °, e l e v a t i o n 0 °
~,
...... ~/Jlll~_
307
A z i m u t h 90 °, e l e v a t i o n 0 °
q "'~, Q#
Fig. 18. Texture variants with uniform orientation distribution.
of the light source (and hence the incidence angle) increases. For azimuth 90 °, on the other hand, the incident light rays are perpendicular to the viewing direction and this remains true as the light's elevation changes; since a uniform orientation distribution is isotropic, the relationship between visible texel area and incidence angle (and hence reflected intensity) remains approximately the same for all elevations. Analogous observations about the histograms can be made for the other two texel orientation models.
The images ~nd histograms for the six light source positions are shown in Fig. 20 and 21 for the "drooping" orientation model and in Fig. 22 and 23 for the "sagging" model. In the case of the drooping model, for both azimuths (0 and 90 °) there is substantial compression of the histogram with increasing elevation of the light source. This is to be expected, since for light source elevation 0 ° the drooping model yields a full range of incidence angles, but for higher elevations this range is compressed. For the sagging model, this situation is reversed: we have a full range of
308
A. WAKSMAN and A. ROSENFELD
A z i m u t h 0 °, e l e v a t i o n 0 °
0
A z i m u t h 90 °, e l e v a t i o n 0 °
50
I00
150
50
I00
150
200
250
50
100
150
200
250
50
100
150
200
:250
O"
50
100
150
201)
250
0
50
le~
150
200
Fig. 19. Histograms of the texture variants.
incidence angles only for light source elevation 45 °, but a compressed range for lower elevations, so that the histogram range expands as the elevation increases from 0 to 45 °. (The range would shrink again for higher elevations, which are not shown here.) Thus, for all three models, the effects of the light source position on the shape of the histogram are qualitatively predictable.
4.2. Histograms of real textures The real plant images shown in Fig. 1 do not satisfy any of our simple texel orientation models, but the qualitative observations of Section 4.1 hold for the histograms of the real images (with backgrounds masked out). In particular, for azimuth 0 ° [Figs 3(a) (c)] the histogram compresses with increasing light source
Photometry
309
,_
a. (0 °, 0°).
d. (90 °, 0°).
b. (0 °, 30°).
e. (90 °, 30°).
c. (0 °, 45°).
f. (90 °, 45°).
Fig. 20. A texture with "drooping" texel orientation distribution, under point source illumination from different directions.
elevation and for azimuth 90 ° [Figs 3(d)-(f)], there is little change with elevation; thus, the behavior of the real histograms under change in light source position is analogous to that of the synthetic histograms obtained from the uniform orientation model. It is not surprising that the histograms obtained from the drooping and sagging models behave differently than those of the real plant images; for those models, the
texel orientations are much less isotropic indeed, they have only one degree of freedom. Note that the real images contain shadows and interreflections, which were not modeled in the synthetic images; but the behavior of the histograms is actually reinforced by the presence of the shadows. Indeed, for azimuth 0 ° and elevation 0 °, the shadows are not visible because the light source is directly
310
A. W A K S M A N and A. R O S E N F E L D
50
1OO
150
200
250
0
50
~. (o °, o°).
50
IO0
i 150
,
i
100
150
~. (o °, 45°).
150
200
250
200
250
200
250
d. (90 °, 0°).
2(30
250
50
b. (o o, 30°).
50
I00
100
i 150
e. (90 °, 30°).
L
200
250
50
100
150
f. (90 o, 45o/. Fig. 21. Histograms of a "drooping" texture.
Photometry
v
i V
3
~
v
i V
;Y_.¢. ~. (0 o, 0o).
d. (90 o, 0o).
S qlp~
~\
~
..t b. (0 °, 30°). v
p\
O ~
e. (90 °, 30°).
O ~v "
at c. (0 °, 45°).
%
v
O~j J,
/ f. (90 °, 45°).
Fig. 22. A texture with "sagging" texel orientation distribution, under point source illumination from different directions.
312
A. WAKSMAN and A. ROSENFELD
I
50
I00
&,
50
200
150
250
50
(oo, oo).
IUO
100
150
200
250
200
250
2OO
250
d. (90 °, 0°).
150
200
250
50
b. (o o, 30°).
I00
150
e. (90 °, 3o°1 .
i
100
150
200
250
50
100
150
f. (90 °, 45°).
c. (0 °, 45°).
Fig. 23. Histograms of a "sagging" texture.
behind the viewer; as the elevation increases, shadows are increasingly visible, which contributes to a shift of the histogram towards dark gray levels. For azimuth 90 °, on the other hand, shadows are visible for all elevations. 5. C O N C L U D I N G
REMARKS
Quantitative modeling of the gray-level histograms of real leafy textures would, be difficult, because the
illumination is more complex (sun and sky, in outdoor images; multiple nonpoint-like light sources, in indoor situations); shadows, interreflections and transmittance must be taken into account; the texels are often not planar; and they do not have a simple distribution of orientations. Nevertheless, we have seen that it is possible to qualitatively predict how the histograms of real plant images vary with the direction of (predominant) illumination. In fact, we have found that this variation resembles that of synthetic histograms oh-
Photometry
tained from p o i n t source illumination a n d a uniform o r i e n t a t i o n model, ignoring t r a n s m i t t a n c e , interreflections a n d shadows.
5. 6.
REFERENCES
1. A. Waksman and A. Rosenfeld, Sparse, opaque threedimensional texture, 2a: Visibility, Graphical Models and Image Processing, in press; also available as Technical Report CAR-TR-729 (CS-TR-3333), Center for Automation Research, University of Maryland at College Park (1994). 2. W.A. Richards, Lightness scale from image intensity distributions, Appl. Opt. 21(4), 2569-2582 (15 July 1982). 3. L. Levi, Applied Optics: A Guide to Optical System Design, Vol. 2, p. 105. J. Wiley, New York (1968). 4. M. S. Langer and S. W. Zucker, Shape from Shading on a Cloudy Day, Technical Report TR-CIM-91-7, Computer Vision and Robotics Laboratory, Research Centre
7. 8. 9. 10. 11.
for Intelligent Machines, McGill University, Montr6al, Qu6bec, Canada (May 1992). American Society for Testing and Materials, Standard Terminology of Appearance, E284-91, Annual Book oj A S T M Standards. Philadelphia, PA (1991). J. H. Lambert, Photometria sire de mensura de gratibus luminis, colorum et umbrae. Eberhard Klett, Augsberg, Germany (1760). S.K. Nayar, K. Ikeuchi and T. Kanade, Surface reflection: physical and geometrical perspectives, IEEE Trans. Pattern Anal. Math. Intell. 13, 611-634 (1991). B. Phong, Illumination for computer generated pictures, Commun. ACM. 18, 311-317 (1975). S.J. Sickels, A Virtual Instrument Interface for Camera Motion Control, M.S. thesis, Department of Electrical Engineering, University of Maryland at College Park (1994). K. Torrance and E. Sparrow, Theory for off-specular reflection from roughened surfaces, J. Opt. Soc. Am. 57, 1105-1114 (1967). P. Beckmann and A. Spizzichino, The Scattering of Electromagnetic Waves from Rough Surfaces. Pergamon Press, New York (1963).
About the Author--ADLAI WAKSMAN is a Ph.D. candidate in the Computer and Information Science
Department at the University of Pennsylvania under Ruzena Bajcsy. He is currently doing research with Azriel Rosenfeld at the University of Maryland at College Park.
AZRIEL ROSENFELD is Director of the Center for Automation Research, a department-level unit of the College of Computer, Mathematical and Physical Sciences at the University of Maryland at College Park. Professor Rosenfeld's research interests include many aspects of image analysis and computer vision. About the Author
313