Current Biology Vol 21 No 22 R914
Dispatches
Visual Perception: Lightness in a High-Dynamic-Range World How the visual system detects surface reflectance — lightness — has puzzled researchers for centuries. A new study of contexts that capture the characteristic luminance range of natural scenes rules out a long-standing theory of lightness perception. Daniel J. Graham The sun’s daily blast of visible wavelength photons makes vision possible, but it also poses challenges for visual systems. In natural scenes, the ratio of the highest to lowest light intensity — the dynamic range of luminance — is often 1000:1 or more, due mostly to variations in the illumination of surfaces (Figure 1A,B). Neurons respond over a far smaller range, necessitating a variety of adaptation mechanisms. Beyond the challenges they pose for neural coding, variations in illumination raise deep questions about visual perception, especially in terms of lightness — the judgment of surface reflectance. Reflectance itself — a physical property of materials — is not directly accessible, since the eye can only measure luminance, but detecting reflectance is thought to be essential for object recognition. The lightness of a surface is roughly constant in the face of huge variation in the luminance reaching our eyes (for example, see [1]). The question of how humans perceive lightness has puzzled scientists for centuries [2]. A long-standing theory — Wallach’s ratio rule [3] — has been challenged [4] but is still considered ‘‘almost axiomatic’’ [5]. Wallach’s ratio rule holds that when a surface S1 of reflectance R1 is compared to a background surface S2 of reflectance R2, S1 will appear to be the same lightness as long as the ratio of the luminances for those surfaces, L1 / L2, remains the same. That is, two luminances that differ by a fixed ratio (L1 = k*L2) should be perceived to match two reflectances that differ by the same ratio (R1 = k*R2). In this case, the ‘mapping function’ relating luminance to lightness is linear and has a slope of one on a log–log scale, as
functions of the form y = kx have a slope of one on log–log axes. If the mapping function has a slope different from one, the function is nonlinear — because the slope of a function that is linear on log–log scales corresponds to its exponent — and the ratio rule would not hold. Following Wallach, much evidence has supported the ratio rule, and it has been extended to more complex arrays [6]. In this issue of Current Biology, Radonjic et al. [7] report findings that call for a reevaluation of the ratio rule. They present a study of lightness perception in contexts that approximate the high dynamic range of luminance characteristic of natural scenes. Radonjic et al. [7] show that luminance-to-lightness mappings in such contexts rule out ratio-based explanations. They demonstrate that humans perceive an orderly progression of lightness over the entire range of luminances, and that the brain can perform massive nonlinear compression in judging lightness. While it has long been possible to sequentially present luminances that span a high dynamic range [8], the technical innovation introduced by Radonjic et al. [7] is to present test luminances in a spatial context that itself spans this range. This paradigm has not previously been implemented, because it has not been possible to display a context comprising both high and low intensities in the laboratory. Cathode ray tube displays can produce low intensities, as each pixel can be selectively turned ‘off’, but high intensities damage the phosphors and produce x-rays. Liquid crystal displays can safely produce high intensities, but because the illumination behind the liquid crystal is uniform, striking both ‘white’ and ‘black’ pixels, significant amounts of light leak through pixels
that should be fully ‘black’. Both types of conventional display are limited to ranges of around 300:1 [9]. Engineers have devised a way to radically expand dynamic range with liquid crystal arrays: exchange uniform illumination for selective illumination [10]. In practice, this means projecting a high-intensity illumination image through a standard array. In this way, ‘black’ pixels can be truly black, while ‘white’ pixels can emit high luminance. The device Radonjic et al. [7] built, which used a commercial digital projector for illumination, displayed images spanning four log10 units of luminance. Radonjic et al. [7] presented a checkerboard of luminances to observers (Figure 1C). A test luminance was shown in the central square while the other squares provided the high dynamic range ‘context’, giving the observer a sense of how each test luminance related to the full dynamic range. The test luminance was randomly varied among the values defining the context. Subjects compared test luminances to a separate chamber housing a reflectance palette lighted by a diffuse illuminator and reported the best perceptual match (Figure 1D). Even for a 5905:1 range, the visual system was found to map these luminances to reflectances spanning a 100:1 range (90% to 0.9% reflectance). This represents massive compression of a kind that has not previously been demonstrated using psychophysics. It is striking that lightness judgments can be made over this range of luminances since the reflectances the visual system presumably seeks to recover cannot vary to such an extent. The empirical luminance-to-lightness mapping function derived by Radonjic et al. [7] displays a compressive nonlinearity — a slope considerably less than one on log–log axes (Figure 1E). This result directly undermines the ratio rule, and the lack of agreement was shown in three
Dispatch R915
further conditions: varying the luminance range holding maximum luminance constant; varying the maximum luminance holding range constant; and varying the luminance distribution holding maximum and minimum constant. Moreover, Radonjic et al. [7] demonstrated that presenting the test square on a uniform background, as in Wallach’s experiments, produced results that do conform to the ratio rule. Thus, displaying a naturalistic range of luminances all at once — even in the absence of meaningful image content or segmentation — has profound effects on lightness perception. This is further implied by Radonjic et al.’s [7] nonlinear model of their data, which is based on the Naka-Rushton equation describing neural adaptation [11]. The model treats perceptual judgment — the best-match reflectance for a given test luminance — as the response over which adaptation occurs, taking account of the context luminances. The model fits the data well under all conditions tested, thereby suggesting an efficient — perhaps optimal — use of the response range in the face of varying luminance ranges. However, the model leaves out some important factors influencing evolved efficient processing strategies, such as the distribution of natural luminances [12,13] and reflectances [14], and the role of noise, so a more complete model may be needed to demonstrate evolved optimality. Even so, the model is insightful. While this experiment captures the characteristic dynamic range of natural scenes, a checkerboard does not fully approximate natural images. As the authors suggest, there may be one-to-one luminance-to-lightness mappings within each separately illuminated portion of a real scene. Such effects can — and surely will — be studied with high-dynamic-range displays; however, some properties of natural scenes may be difficult to reproduce with such displays. The absolute luminances tested — and the maximum luminance possible with these displays [9] — are considerably lower than those found in natural scenes in sunlight. The question of whether absolute luminance contributes to lightness perception has long been debated [15] but barring technological advances, potential effects arising at high absolute
Natural scene luminances A
B 14000 cd/m2
2000:1 Dynamic range
22 cd/m2
1000:1 Dynamic range 7 cd/m2 22000 cd/m2
High-dynamic-range display
D
Reflectance palette 90%
.09%
C
100:1 Dynamic range
.03 cd/m2
TEST
10000:1 Dynamic range
Luminance-to-lightness mapping E
464 cd/m2 Current Biology
Figure 1. The high dynamic range of luminances in natural scenes is captured in a laboratory display, allowing measurement of perceptual luminance-to-lightness mappings. (A,B) The luminance range in natural scenes is often 1,000:1 or more. As most objects reflect between 4% and 80% of incident light, illumination plays the dominant role in generating the high dynamic range. Luminances are given in candelas per meter squared, as measured with a photometer. (C) A high dynamic range display is used to present a context image with a 10,000:1 dynamic range in [7]. Observers judged the lightness of the central test square and compared it to a palette of chips with known reflectance (D). (E) The empirical luminance-to-lightness mapping function found in [7] shows a compressive nonlinearity (slope < 1), thus ruling out a ratio rule explanation.
luminance may not be accessible with these displays. Can one measure lightness perception directly in natural scenes? Although maintaining controlled conditions is difficult, the task shares similarities with that of capturing a high dynamic range scene in a painting. Unlike a camera, the painter can choose the best perceptual match reflectance (pigment) for each luminance in the scene given its context. While stylistic imperatives may push a given painting away from the ‘best match’ criterion, recognizable
representations may on average possess luminance-mapping strategies [16] like those observed in the present study. Radonjic et al. [7] have demonstrated that high-dynamic-range displays are a powerful tool for studying lightness perception, one that can reveal fundamental effects not apparent under less naturalistic conditions. Iterating towards more and more natural conditions is a worthy goal for future research; this will propel efforts to devise comprehensive models of lightness
Current Biology Vol 21 No 22 R916
perception that invoke notions of evolved efficient processing. References 1. Adelson, E.H. (2000). Lightness perception and lightness illusions. In The New Cognitive Neurosciences, 2nd edn., M.S. Gazzaniga, ed. (Cambridge, MA: MIT Press), pp. 339–351. 2. Barlow, H.B., and Mollon, J.D. (1982). The Senses (Cambridge, UK: Cambridge University Press). 3. Wallach, H. (1948). Brightness constancy and the nature of achromatic colors. J. Exp. Psychol. 38, 310–324. 4. Jameson, D., and Hurvich, L.M. (1961). Complexities of perceived brightness. Science 133, 174–179. 5. Kingdom, F.A.A. (2011). Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy. Vision Res. 51, 652–673. 6. Land, E.H., and McCann, J.J. (1971). Lightness and retinex theory. J. Opt. Soc. Am. 61, 1–11.
7. Radonjic, A., Allred, S.R., Gilchrist, A.L., and Brainard, D.H. (2011). The dynamic range of human lightness perception. Curr. Biol. 21, 1931–1936. 8. Jacobsen, A., and Gilchrist, A. (1988). The ratio principle holds over a million-to-one range of illumination. Perception Psychophys. 43, 1–6. 9. Reinhard, E., Ward, G., Debevec, P., Pattanaik, S., Heidrich, W., and Myszkowski, K. (2010). High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (San Francisco: Morgan Kaufmann). 10. Seetzen, H., Heidrich, W., Stuezlinger, W., Ward, G., Whitehead, L., Trentacost, M., Ghosh, A., and Vorozcovs, A. (2004). High dynamic range display systems. ACM Trans. Graphics 23, 760–768. 11. Naka, K.I., and Rushton, W.A. (1966). S-potentials from colour units in the retina of fish (Cyprinidae). J. Physiol. 185, 536–555. 12. Richards, W.A. (1981). A lightness scale from image intensity distributions. Appl. Optics 21, 2569–2582. 13. Brady, N., and Field, D.J. (2000). Local contrast in natural images: normalisation and coding efficiency. Perception 29, 1041–1055.
Cellular Mechanotransduction: Filamin A Strains to Regulate Motility A new study suggests that mechanical strain through the actin-binding protein filamin A leads to increased linkage between the extracellular matrix and cytoskeleton and decreased actin dynamics. Christopher D. Lynch1 and Michael P. Sheetz1,2 Cellular mechanotransduction is the process by which cells detect external and internal mechanical signals and convert them into chemical responses. In the context of cell adhesion to a substrate, externally applied forces produce similar effects to internally generated forces [1]. Focal adhesions, the multi-molecular structures that connect the cell to a surrounding extracellular matrix, grow and mature, while motile velocity decreases [2]. How does the cell coordinate these responses at the molecular level? A new study published recently in Nature from Ehrlicher et al. [3] suggests that mechanical strain in a network of actin filaments crosslinked by the actin-binding protein filamin A alters important interactions between the network and the cell. Two critical aspects of cellular mechanics are addressed by this study — cytoskeletal network linkage to the extracellular matrix, and the dynamics of actin in the cytoskeleton. Both aspects could be responsive to strain on filamin A.
Filamin A is a large, rod-like protein composed of an amino-terminal actin-binding domain and 24 immunoglobulin G (IgG)-like domains that can bind numerous proteins. The first 15 IgG-like domains are referred to as rod 1. These domains interact end-to-end to produce an elongated structure that binds actin filaments along its length. Domains 16–23 make up rod 2 [4], a more compact region in which domains interact in complex ways to result in cryptic binding sites that are only exposed when the molecule is under tension [5]. The carboxy-terminal IgG-like domain allows the protein to homodimerize. Each subunit of a filamin A dimer is capable of binding lengthwise along an actin filament via interactions mediated by rod 1, thereby orthogonally crosslinking two actin filaments and creating a network of actin filaments. Although a filamin-crosslinked network is capable of transmitting forces over long distances [6], cohesive propagation of forces in cells between adhesions depends upon myosin contractility [7,8]. In cells, the loss of filamins results in the loss of normal focal adhesions
14. Attewell, D., and Baddeley, R.J. (2007). The distribution of reflectances within the visual environment. Vision Res. 47, 548–554. 15. Gilchrist, A., Kossifydis, C., Bonato, F., Agostini, T., Cataliotti, J., Li, X., Spehar, B., Annan, V., and Economou, E. (1999). An anchoring theory of lightness perception. Psychol. Rev. 106, 795–834. 16. Graham, D.J., Friedenberg, J.D., and Rockmore, D.N. (2009). Efficient visual system processing of spatial and luminance statistics in representational and non-representational art. Proc. SPIE: Human Vision and Electronic Imaging 7240, 72401N.
Faculty of Psychology, Department of Psychological Basic Research, University of Vienna, Vienna, Austria 1010. E-mail:
[email protected],
[email protected]
DOI: 10.1016/j.cub.2011.10.003
and reduced linkage between cytoskeletal compartments [9,10]. However, the filamin network can have other roles than just crosslinking. As a mechanotransducer, strain generated within this network (internally or externally) can strain the filamin A dimer at its crosslinks. One way to explain the effects of filamin A on many cell activities is that strain exerted on filamin could alter its binding affinity for other components, as has been shown for cell cytoskeletons in general [11]. Ehrlicher et al. [3] tested this hypothesis using a novel technique known as fluorescence loss after photoconversion (FLAC). Conceptually similar to fluorescence recovery after photobleaching (FRAP) [12], a given protein’s binding partner is tagged with a photoactivatable fluorophore that does not fluoresce until excited by a pulse of high-energy light. Once fluorescent, unbound proteins rapidly diffuse away from the site of excitation, while proteins bound to the actin–filamin A network must first release. The result is typically a two-component exponential decay in fluorescence intensity (a very rapid, unbound component and a slow, bound component). Although a high density of filamin A can result in rebinding and multiple release steps, small activation volumes and excess binding protein reduce this possibility. Thus, the slow decay component can be a measure of the off-rate constant for