- Email: [email protected]
Thresholds for detection and awareness of masked facial stimuli
Consciousness and Cognition 32 (2015) 68–78
Contents lists available at ScienceDirect
Consciousness and Cognition journal homepage: www.elsevier.com/locate/concog
Thresholds for detection and awareness of masked facial stimuli Frances Heeks, Paul Azzopardi ⇑ Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK
a r t i c l e
i n f o
Article history: Received 7 January 2014 Revised 9 September 2014 Accepted 12 September 2014 Available online 11 October 2014 Keywords: Blindsight Conscious awareness Signal detection Backward masking Face perception Low vision
a b s t r a c t It has been suggested that perception without awareness can be demonstrated by a dissociation between performance in objective (forced-choice) and subjective (yes–no) tasks, and such dissociations have been reported both for simple stimuli and more complex ones including faces. However, signal detection theory (SDT) indicates that the subjective measures used to assess awareness in such studies can be affected by response bias, which could account for the observed dissociation, and this was confirmed by Balsdon and Azzopardi (2015) using simple visual targets. However, this finding may not apply to all types of stimulus, as the detectability of complex targets such as faces is known to be affected by their configuration as well as by their stimulus energy. We tested this with a comparison of forced-choice and yes–no detection of facial stimuli depicting happy or angry or fearful expressions using a backward masking paradigm, and using SDT methods including correcting for unequal variances in the underlying signal distributions, to measure sensitivity independently of response criterion in 12 normal observers. In 47 out 48 comparisons there was no significant difference between sensitivity (da) in the two tasks: hence, across the range of expressions tested it appears that the configuration of complex stimuli does not enhance detectability independently of awareness. The results imply that, on the basis of psychophysical experiments in normal observers, there is no reason to postulate that performance and awareness are mediated by separate processes. Ó 2015 Published by Elsevier Inc.
1. Introduction A considerable body of research has been directed towards the question of whether perception can occur without awareness. This ongoing interest not only reflects the philosophical debate as to whether perception implies consciousness, but also has implications for understanding the functional architecture of the visual system. If performance can be dissociated from awareness, either through neurological damage as in the case of blindsight (Weiskrantz, 1986), or through degraded stimulus conditions presented to normal observers (e.g. Meeres & Graves, 1990) it would suggest that information can influence performance through different neural pathways than those mediating conscious awareness. Demonstrating perception without awareness requires a dissociation between performance on a measure of perception and a measure of awareness. However, there remains considerable controversy as to what constitutes an appropriate indicator of awareness. Cheesman and Merickle (1984) distinguished between the subjective threshold, which they defined as the stimulus energy level at which observers claim not to be able to discriminate perceptual information above chance, and the objective threshold, which they defined as the level at which observers are actually unable to discriminate perceptual ⇑ Corresponding author at: Department of Experimental Psychology, University of Oxford, South Parks Rd, Oxford OX1 3UD, UK. Fax: +44 1865 310447. E-mail address: [email protected] (P. Azzopardi). http://dx.doi.org/10.1016/j.concog.2014.09.009 1053-8100/Ó 2015 Published by Elsevier Inc.
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
69
information above chance. Whilst some researchers have argued that awareness should always be assessed using objective measures of perceptual discrimination (e.g. Eriksen, 1960; Holender, 1986), Cheesman and Merickle (1986) argued that objective measures provide conservative indicators of awareness, and that a subjectively defined awareness threshold best captures the distinction between conscious and unconscious experiences. Following from their finding that the objective threshold is associated with somewhat lower stimulus energy levels than the subjective threshold indicated by subjects’ self-reports (Cheesman & Merickle, 1984), they suggest that unconscious processing may occur only for stimuli presented at energy levels between the objective and the subjective thresholds, with stimuli above the subjective threshold being consciously processed, and stimuli below the objective threshold undergoing no processing whatsoever. According to this formulation, performance should reach chance level at a lower stimulus energy for objective (i.e. forced choice) tasks than for subjective (i.e. yes–no) tasks; hence there should be intermediate stimulus levels at which observers perform at chance in a subjective task but above chance in an objective task. Such dissociations have been reported in several studies: for example, Meeres and Graves (1990) found that observers were able to detect a masked target shape significantly above chance in a spatial forced-choice task (i.e. in which of two possible spatial locations a stimulus was presented) despite being unable to detect the stimuli in a yes–no task. There is, however, a potentially serious problem with dissociations which are based on percentage correct scores in subjective and objective tasks because, according to signal detection theory (SDT) (Green & Swets, 1966), percentage scores reflect not only an observers’ sensitivity to stimuli, but also their response bias. As illustrated in Fig. 1, SDT assumes that internal signals associated with presentation of a stimulus (S+) is normally distributed about a mean, and the internal signals associated with the absence of a stimulus (S) is also distributed around a lower mean value. Decisions about whether a particular internal signal was caused by S+ or S require an observer to set a criterion value, c, above which they respond S+ and below which they respond S As the two distributions overlap this will result in both correct responses (hits and correct rejections) and mistakes (false alarms and misses) with the proportions of correct responses and errors – and hence the percentage correct scores – depending not only on the observer’s sensitivity but also on the position of their response criterion. Use of a conservative response criterion in the yes–no task may depress an observer’s percentage correct score relative to a force choice task: the accuracy of observers’ decision about which of two spatial or temporal intervals contained a stimulus should be unaffected by response bias provided that the stimuli are randomly distributed with respect to the spatial or temporal interval. Hence any dissociation of performance and awareness based on differential accuracy in forced-choice compared to yes–no tasks may simply reflect the influence of response bias. SDT provides a measure of sensitivity (d0 ) which is independent of response criterion and therefore avoids the problem of response bias (Green & Swets, 1966). Assuming that the two underlying distributions of signals associated with S+ and S are normal and have equal variances, then the difference between the means of the distributions in units of standard deviation can be given by d0 = z(H) z(F), where H (hit rate) = (number of hits/number of S + s) and F (false alarms rate) = (number of false alarms/number of S s), and z is the inverse of the normal distribution function. Despite the fact that SDT is well established, the possibility that response bias can account for the difference between objective and subjective thresholds has been consistently ignored or actively dismissed by the majority of researchers in perception without awareness (e.g. Dixon, 1971, 1981). In some cases, SDT methods have been applied inappropriately in order to lend credibility to findings established through potentially biased methods. For example, Meeres and Graves (1990) conducted a post hoc analysis of their localization and detection data to calculate d0 reporting a dissociation of sensitivity in the two task consistent with their main finding. However, Meeres & Graves did not test for the equality of variances of the two underlying distributions. This is a critical assumption in calculating d0 : if the variances are not equal, d0 does not provide a reliable measure of sensitivity because the value obtained will vary according to the response criterion. Given that the widespread claims of dissociations between performance and awareness in normal observers are based on measures susceptible to response bias, it is of general interest to know whether the dissociations stand up when truly bias-
Fig. 1. Signal detection theory.
70
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
free measures are used. Balsdon & Azzopardi (2015) have shown, in an experiment using metacontrast masking, that they do not. However, it might may be premature to infer the equivalence of objective and subjective thresholds once response bias is taken into account on the basis of a single study. Indeed, an important consideration is the type of stimuli used in this and other studies. Whilst Balsdon’s & Azzopardi’s demonstration was restricted to the detection of simple targets (squares and diamonds), many other claims for dissociations relate to complex stimuli such as words (e.g. Cheesman & Merickle, 1984) or faces (e.g. Esteves & Öhman, 1993), and there are a number of reasons to suspect that processing of complex targets might be different. First, a context effect has been documented in tasks involving detection of complex stimuli. For example, the threshold for detection of masked normal faces is lower than that for stimuli composed of rearranged facial features – the ‘face detection effect’ (Purcell & Stewart, 1986, 1988). This implies that detection is influenced not only by stimulus energy but also by the configuration of components as a meaningful whole, and suggests that facial expressions may have some kind of privileged access to the system(s) mediating performance in this task. Secondly, various theories of emotion have postulated such an advantage for facial expressions through unconscious, or preattentive processing of emotionally significant stimuli (LeDoux, 1989; Zajonc, 1980; Öhman, 1986). LeDoux argues that ‘‘the computation of stimulus significance takes place prior to and independent of conscious awareness’’ (p. 267), with the affective system receiving inputs from the early stages of sensory processing, allowing emotional responses to be initiated rapidly. Such mechanisms might confer a privileged status to emotional stimuli and provide information to influence forced-choice responding that might not reach conscious awareness. Certainly, there is evidence that masked faces expressing fear or anger cause selective activation of areas of the brain associated with emotion – particularly the amygdala – when subjects are unable to report their presence (Morris, Öhman, & Dolan, 1999; Whalen, Rauch, Etcoff, et al., 1998). There is also some evidence that facial stimuli may be processed in a way that can influence forced-choice performance below the threshold for awareness. Esteves and Öhman (1993, Experiment 3) found that subjects were able to identify masked facial expressions as happy at lower stimulus onset asynchrony (SOA) than that necessary for accurate confidence ratings. However, all of these studies have relied on percentage correct scores to assess subjective awareness, and as the preceding discussion of bias indicates, dissociations based on such measured are potentially misleading. The purpose of the present experiment was therefore to test whether or not any dissociation between objective and subjective thresholds for the detection of faces can be accounted for by response bias, as has been demonstrated previously for simpler stimuli. 2. Methods Participants were required to detect images depicting facial expressions during yes–no and forced-choice tasks using a backward masking paradigm. 2.1. Participants 12 observers participated in the experiment: 5 male and 7 female. Apart form one member of senior staff, all were undergraduates at Oxford University. Their median age was 21 years. All had normal or corrected vision. 2.2. Apparatus Stimuli were presented 1.25 m in front of the seated observer on an Eizo T220 2000 monitor (Eizo Corporation, Ishikawa, Japan) with a non-interlaced frame rate of 120 Hz, driven by a VSG2/4 visual stimulus generator (Cambridge Research Systems, Rochester, U.K.). Presentation of stimuli and registration of responses was controlled by a custom program run on a Dan IBM PC-compatible computer (200 MHz Intel Pentium MMX processor) operated by the experimenter in the same room, which was dimly lit. The apparatus was calibrated by means of a photodiode and oscilloscope to ensure that the stimuli and masks were presented at the specified times and for the specified durations. 2.3. Stimuli Targets were 8-bit digitized pictures of faces from Ekman and Friesen’s (1976) Pictures of Facial Affect: The images were standardized in the following way using a graphical software (Core Photo-Paint 8, Corel Corporation, USA). First, an oval mask tool was used to remove background cues such as hair, jewellery, labels and borders; the intensity of the selected portion of the image was then adjusted to give a mean grey level of 128, and merged with a background also of grey level 128. Targets subtended a visual angle of 5.9° by 8.1°. Blank images consisted of a uniform background of grey level 128. The apparatus was adjusted so that a grey-level of 128 corresponded to a mean luminance of 10 cd m2, calibrated with a Minolta LC1500 photometer. The mask consisted of overlapping capital letters (N and O) in black type on a white background, similar to that in used in previous face detection studies by Purcell and Stewart (1986, 1988). It subtended a visual angle of 8.2° by 11.8°, completely covering the region in which target stimuli were presented. See Fig. 2(a–c) for illustrations.
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
71
Fig. 2. Stimuli. (a) Original format of facial stimuli from Pictures of Facial Affect (Ekman & Frisen, 1976). (b) Standardized formal of facial stimuli used in the present study. (c) Pattern mask used in the present study (Purcell & Stewart, 1986).
2.4. Procedure Each observer was tested on forced choice and yes–no detection of a single target stimulus throughout the experiment, with stimuli randomly allocated to observers. in total, the experiment included 4 images for each of the three expressions (happy, angry, fearful) for which dissociations had previously been reported in the literature; specific image numbers are given in Table 1. The forced choice task involved two temporal intervals: A target (30 ms) was shown in one interval, a blank (30 ms) in the other, with each interval followed immediately by the pattern mask (30 ms) a blank screen was shown for 2150 ms between the two intervals. After viewing both intervals, observers were asked to respond ‘1’ or ‘2’ to indicate in which interval the thought the target had been presented. The yes–no task involved only one interval: Either a target or a blank was shown (30 ms), followed immediately by the pattern mask (30 ms), and observers responded ‘yes’ or ‘no’ to indicate whether the thought a target had been presented, then rated their confidence in their decision on a scale of 1 (not at all certain) to 4 (certain). Observers were informed that a face target would be presented on 50% of trials. Responses were made verbally and recorded on computer by the experimenter. Target and mask durations were set at 30 ms each because Esteves and Öhman (1993) had reported these parameters to be below the threshold for conscious perception in their identification task. However, our pilot studies indicated that stimuli could be detected very accurately with these timings; hence contrast was reduced to ensure near-threshold performance. A PEST calibration procedure (Macmillan & Creelman, 1991; Taylor & Creelman, 1967) was used to determine the stimulus contrast at which each observer performed with 75% accuracy in the forced choice detection task for their allocated stimulus; in the experimental trials the target was then presented at a range of four contrasts bracketing the 75% threshold contrast. Each observer completed 4 blocks of forced-choice trials and 4 blocks of yes–no trials, alternating task type between every block; half the observers began with a yes–no block, the others with a forced choice block. For both tasks, blocks contained 160 trials, 40 at each of the 4 stimulus contrasts, randomised with respect to contrast level and order of targets/ blanks. 3. Results For the forced choice task, observers’ sensitivity was calculated as d0 directly from hit rate and false alarm rates and p adjusted by a factor of 1/ 2 to reflect the fact that the forced choice task allows two opportunities to detect the stimulus (Macmillan & Creelman, 1991). For the yes–no task, observers’ confidence ratings were used to construct ROCs (graphs of hit rate against false alarm rate) using a maximum-likelihood algorithm (Dorfman & Alf, 1969). ROCs for one observer are illustrated in Fig. 3. As shown in Table 1, the slopes of these ROCs (given as B) in many cases deviated significantly from 1.0 indicating that the variances of the two underlying distributions were not equal. Sensitivity in the yes–no task was therefore calculated as da which takes into account the possibility of unequal variances (Simpson & Fitter, 1973). ROC parameters are presented in Table 1. Figs. 4–6 show the sensitivity of each observer, measured at 4 contrasts for a particular happy, angry or fearful face in the forced choice and yes–no tasks, with error bars indicating the 95% confidence limits. The difference in sensitivity values for the two tasks was tested for significance using a Z-test (Marascuilo, 1970), onetailed since the sensitivity was predicted to be higher, if anything, in the objective compared to the subjective task. Of the 48 tests, 10 reached significance at a comparison-wise level of Pc = .05, but only one of them remained significance after a Bon-
72
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Table 1 da Values for yes–no and forced choice tasks, including ROC parameters for yes–no task. Expression
Obs
Con
Task
A
sA
B
sB
P (B)
da
sda
Happy (022)
FI
0.10
yn 2afc yn 2afc yn 2afc yn 2afc
0.0892 – 0.6084 – 0.9334 – 1.3942 –
0.1603 – 0.1627 – 0.1607 – 0.1888 –
0.9422 – 0.8631 – 0.6302 – 0.5386 –
0.1251 – 0.1256 – 0.1075 – 0.1212 –
NS – NS –
0.0918 0.1337 0.6513 0.7703 1.1168 1.1292 1.7359 1.6120
0.1653 0.1406 0.1702 0.1482 0.1819 0.1577 0.2078 0.1852
yn 2afc yn 2afc yn 2afc yn 2afc
0.0811 – 0.0576 – 0.2519 – 0.4712 –
0.1679 – 0.1609 – 0.1422 – 0.1536 –
1.0258 – 0.9336 – 0.6307 – 0.7765 –
0.1345 – 0.1250 – 0.0949 – 0.1055 –
0.0800 0.1570 0.0595 0.3408 0.3013 0.3813 0.5263 0.2931
0.1657 0.1410 0.1663 0.1425 0.1698 0.1420 0.1695 0.1418
yn 2afc yn 2afc yn 2afc yn 2afc
0.2899 – 0.2290 – 0.5042 – 0.6815 –
0.1585 – 0.1635 – 0.1661 – 0.1561 –
0.8593 – 0.9591 – 0.9007 – 0.7002 –
0.1200 – 0.1228 – 0.1225 – 0.1029 –
0.3109 0.1836 0.2338 0.4097 0.5298 0.5494 0.7895 0.8438
0.1692 0.1423 0.1659 0.1430 0.1702 0.1472 0.1749 0.1517
0.5419
0.1583 – 0.1467 – 0.1534 – 0.1534 –
0.8459 – 0.6554 – 0.6698 – 0.5006 –
0.1107 – 0.0927 – 0.0980 – 0.0824 –
0.5851 0.3382 0.6474 0.4741 0.8514 0.8997 1.2048 1.0572
0.1678 0.1459 0.1706 0.1430 0.1736 0.1533 0.1854 0.1568
0.1636 – 0.1720 – 0.2375 – 0.2938 –
0.9547 – 0.9794 – 0.8049 – 0.5334 –
0.1263 – 0.1298 – 0.1573 – 0.1506 –
0.2741 0.2910 0.5794 0.7673 1.8890 1.6710 2.5903 2.7274
0.1659 0.1414 0.1691 0.1479 0.2036 0.1822 0.2790 0.3043
0.1754 – 0.1593 – 0.1548 – 0.1395 –
1.0544 – 0.8702 – 0.7978 – 0.5695 –
0.1432 – 0.1197 – 0.1106 – 0.0822 –
0.3447 0.5658 0.3592 0.6433 0.4282 1.0316 0.4215 1.2281
0.1677 0.1466 0.1676 0.1457 0.1686 0.1563 0.1704 0.1635
0.1657 – 0.1492 – 0.2317 – 0.1816 –
0.8015 – 0.6552 – 0.7541 – 0.7502 –
0.1201 – 0.0963 – 0.1561 – 0.1249 –
0.7751 0.7239 0.6673 1.0249 1.7875 1.2943 1.3039 1.4663
0.1812 – 0.1904 – 0.2747 – 0.4214 –
0.9981 – 0.6028 – 0.8007 – 0.5025 –
0.1370 – 0.1122 – 0.1734 – 0.2011 –
0.1550 –
0.7596 –
0.1137 –
0.13 0.16 0.19 Happy (048)
DJ
0.08 0.11 0.14 0.17
Happy (057)
SP
0.08 0.11 0.14 0.17
Happy (066)
PA
0.16 0.18 0.20 0.22
Angry (003)
FH
0.03 0.04 0.05 0.06
Angry (025)
SG
0.25 0.30 0.35 0.40
Angry (069)
JB
0.14 0.18 0.22 0.26
Angry (105)
AH
0.18 0.22 0.26 0.30
Fearful (024)
JG
0.20
yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc
– 0.5473 – 0.7246 – 0.9527 – 0.2679 – 0.5735 – 1.7146 – 2.0759 – 0.3542 – 0.3367 – 0.3873 – 0.3430 – 0.7024 – 0.5641 – 1.5831 – 1.1526 – 0.7015 – 1.3553 – 1.9729 – 2.5150 – 0.4821 –
⁄⁄⁄
– ⁄⁄⁄
– NS – NS – ⁄⁄⁄
– ⁄
– NS – NS – NS – ⁄⁄
– NS – ⁄⁄⁄
– ⁄⁄⁄
– ⁄⁄⁄
– NS – NS – NS – ⁄⁄
– NS – NS – NS – ⁄⁄⁄
– NS – ⁄⁄⁄
– NS – ⁄
– NS – ⁄⁄⁄
– NS – ⁄
– ⁄
–
Z
P (Z)c
P (Z)e
1.039
NS
NS
0.527
NS
NS
0.052
NS
NS
0.445
NS
NS
0.354
NS
NS
1.828
⁄
NS
0.361
NS
NS
1.055
NS
NS
0.091
NS
NS
0.803
NS
NS
0.087
NS
NS
0.234
NS
NS
1.110
NS
NS
0.779
NS
NS
0.209
NS
NS
0.608
NS
NS
0.078
NS
NS
0.836
NS
NS
0.798
NS
NS
0.332
NS
NS
0.992
NS
NS
1.279
NS
NS
2.625
⁄⁄
NS
3.415
⁄⁄⁄
NS
0.1742 0.1644 0.1724 0.1718 0.2050 0.1937 0.1849 0.1815
0.214
NS
NS
1.469
NS
NS
⁄
NS
0.627
NS
NS
0.7021 0.7610 1.6415 1.8565 2.1779 2.9626 3.1781 3.7929
0.1713 0.1519 0.1989 0.2041 0.2161 0.3410 0.3532 0.3554
0.257
NS
NS
0.754
NS
NS
1.944
⁄
NS
1.227
NS
NS
0.5429 0.5943
0.1684 0.1531
0.226
NS
NS
–1.748
73
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78 Table 1 (continued) Expression
Obs
Con
Task
0.25
yn 2afc yn 2afc yn 2afc
0.30 0.35 Fearful (037)
GK
0.18 0.21 0.24 0.27
Fearful (095)
IB
0.17 0.20 0.23 0.26
Fearful (104)
IM
0.25 0.30 0.35 0.40
yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc
A 0.5470 – 0.4104 – 0.6929 – 0.5533 – 0.5771 – 1.2982 – 1.2010 – 0.0253 – 0.3324 – 0.6516 – 1.0872 – 0.5666 – 1.0125 – 1.0372 – 1.5885 –
sA
B
sB
P (B)
0.1592 – 0.1509 – 0.1685 –
0.8387 – 0.7753 – 0.9099 –
0.1240 – 0.1141 – 0.1403 –
NS –
0.1611 – 0.1558 – 0.1924 – 0.1719 –
0.8694 – 0.8080 – 0.8133 – 0.5824 –
0.1166 – 0.1064 – 0.1298 – 0.0994 –
NS – NS – NS –
0.1578 – 0.1592 – 0.1528 – 0.1625 –
0.9212 – 0.9134 – 0.6916 – 0.5709 –
0.1174 – 0.1169 – 0.0976 – 0.0966 –
0.1743 – 0.1664 – 0.1682 – 0.2034 –
1.0207 – 0.7054 – 0.6997 – 0.4072 –
0.1357 – 0.1096 – 0.1101 – 0.1023 –
⁄
– NS –
⁄⁄⁄
– NS – NS – ⁄⁄
– ⁄⁄⁄
– NS – ⁄⁄
– ⁄⁄
– ⁄⁄⁄
–
da
sda
P (Z)c
P (Z)e
1.540
NS
NS
1.983
⁄
NS
1.744
NS
NS
0.1693 0.1459 0.1671 0.1507 0.1845 0.1695 0.1911 0.2057
0.133
NS
NS
0.978
NS
NS
0.018
NS
NS
1.858
⁄
NS
0.0263 0.1777 0.3471 0.2003 0.7579 0.6421 1.3353 0.9264
0.1642 0.1405 0.1652 0.1407 0.1719 0.1455 0.1833 0.1516
0.701
NS
NS
0.677
NS
NS
0.514
NS
NS
1.691
⁄
NS
0.5607 0.2452 1.1701 0.7219 1.2018 1.0692 2.0806 1.6779
0.1668 0.1410 0.1782 0.1475 0.1790 0.1558 0.2356 0.1833
1.445
NS
NS
1.938
⁄
NS
0.559
NS
NS
1.349
NS
NS
0.5928 0.9447 0.4587 0.9120 0.7248 1.1343
0.1670 0.1560 0.1660 0.1572 0.1685 0.1635
0.5905 0.5607 0.6349 0.8550 1.4243 1.4287 1.4677 1.9894
Z
Key: SOA – stimulus onset asynchrony; A – y intercept of ROC plotted in z coordinates, equal to d0 when the variances of the two underlying distributions are equal; sA – standard deviation of A; B – gradient of ROC plotted in z coordinates, equivalent to the ratio of the variances of the two underlying distributions; sB – standard deviation of B; P(B) – probability of B deviating from a value of 1.0 by chance; da – measure of sensitivity, taking into account inequality of variances of the underlying distributions; sda – standard deviation of da; Z – the difference between da(yn) and da(2afc) expressed as a normal deviate; P(Z)c – comparison-wise probability of the difference between da(yn) and da(2afc) arising by chance; P(Z)e – experiment-wise probability of the difference between da(yn) and da(2afc) arising by chance. For P(B), P(Z)c and P(Z)e: ⁄ P < .05, ⁄⁄ P < .01, ⁄⁄⁄ P < .001. Each set of parameters is derived from 160 trials.
ferroni correction was applied to give an experiment-wise level of Pe = .05. Therefore, across a range of expressions and contrasts, these results provide virtually no evidence of a significant dissociation between sensitivity to facial stimuli in subjective and objective tasks. 4. Discussion This experiment was designed to demonstrate unconscious processing of facial expression according to the conditions outlined by Cheesman and Merickle (1986) by comparing forced-choice discrimination with observers’ reports of what they could see when using near-threshold stimuli. However, for the vast majority of cases there was no significant difference between sensitivity in yes–no and forced-choice detection of faces when bias-free measures were used. This applied across the range of expressions (happy, angry, fearful) for which dissociations had been previously reported in the literature. These findings suggest that for faces, as for simpler stimuli (Balsdon & Azzopardi, 2015), dissociations between subjective and objective measure may arise trivially as a result of response bias rather than as a result of unconscious pressing producing enhanced sensitivity in the objective task. Only one previous study has conducted a comparison of performance in subjective and objective tasks using masked facial stimuli. As mentioned earlier, Esteves and Öhman (1993) reported for happy faces that the threshold SOA required for accurate performance in their objective task was significantly lower than that for their subjective measure. Although there was no significant difference between the two thresholds when angry faces were tested, Esteves & Öhman suggest that this may reflect their failure to test an appropriately narrow range of SOAs, and conclude generally in support of Cheesman and Merickle’s (1986) hypothesis. However, performance in Esteves & Öhman’s study was assessed by percentage correct scores which may be influenced by response bias as discussed: this is one possible explanation for the discrepancy between their findings and those of the present study, which found no such dissociation when using bias-free measures. There is, however, another possible explanation for this discrepancy which relates to the use of different masks in the two studies:
74
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Fig. 3. Example ROCs fitted to the responses obtained from a single observer (FI) during yes–no detection. Key: H = hit rate; F = false alarm rate; z is the inverse of the normal distribution function; da = estimate of d0 corrected for unequal variances of underlying signal distributions (Simpson & Fitter, 1973).
whilst Esteves & Öhman used a neutral face to mask target facial expressions, the present study used an abstract pattern mask. The decision to use a pattern mask rather than a neutral face mask was intended to avoid any potentially confounding interactions or artefacts such as apparent motion arising from similarity between target facial expressions and a neutral face mask. For example, Esteves and Öhman (1993) note a confusion between angry and neutral faces (Hansen & Hansen, 1988) which may underlie their observers’ tendency to identify the neutral mask faces as angry more often than happy. Interactions between target and mask faces might produce effects which are independent of the processing of the target itself, and in this sense were seen as confounding factors to be excluded in the present experiment. However, it is possible that such interactions might have specific effects on facilitating forced choice performance independently of awareness, and
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
75
Fig. 4. Sensitivity, measured as da for the detection of happy faces in yes–no () and 2afc tasks ( ) plotted as a function of stimulus contrast for 4 observers. Numbers in brackets refer to the Ekman face presented to each observer.
could therefore have contributed to the dissociation observed in Esteves & Öhman’s study. Whilst there are no obvious mechanism by which the masking stimulus might influence the relationship between objective and subjective performance, this remains a possible explanation for the discrepancy between the two studies, and the present study might be replicated using a neutral face mask in order to assess whether there are any masking conditions in which a dissociation can be demonstrated using bias-free measures. Whilst the present study demonstrated equal sensitivity in subjective and objective tasks relating to detection of facial stimulus (present or absent), Esteves & Öhman’s tasks related to identification of facial expressions (as happy or angry): it remains to be seen whether observers would similarly show equal sensitivity in subjective and objective identification tasks if assess using bias-free measures. Clearly, identifying the emotion expressed by a face requires more detailed visual analysis than is required for simple presence/absence decisions: it therefore remains possible that identification performance could be selectively facilitated by unconscious processing of the emotional significance of stimuli as postulated by LeDoux (1989) and others. This possibility requires explicit testing, using bias-free measures of sensitivity to compare yes–no and forcedchoice identification performance. In contrast to the ‘forced choice’ identification task used by Esteves and Öhman (1993), where observers showed a bias in labelling faces as angry more often than as happy, a truly objective test of identification might involve participants in deciding in which of two temporal intervals a happy face (or angry, or fearful, etc.) was presented, given that a neutral face was presented in the other interval. It is necessary to present a neutral face rather than a blank screen in the other interval in order to asses observers’ perception of facial expressions (i.e. stimulus content) in isolation from their ability to detect the presence of a facial stimulus at all. Similarly, a subjective test of identification might involve a yes–no response (with
76
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Fig. 5. Sensitivity, measured as da for the detection of angry faces in yes–no () and 2afc tasks ( ) plotted as a function of stimulus contrast for 4 observers. Numbers in brackets refer to the Ekman face presented to each observer.
confidence ratings) as to whether a happy face was presented, with either a happy or a neutral face presented in each trial. As in the present study, a range of facial expressions should be tested to ensure that findings can be generalized. Observers’ responses in these tasks could then be used to calculate the sensitivity (da) as described in previous sections in order to compare yes–no and forced choice identification of facial expressions independently of response bias. The present study has demonstrated that sensitivity to facial stimuli backwardly masked by an abstract pattern does not differ significantly in subjective and objective detection tasks when calculated independently of response criterion. It remains to be seen whether this result depends on the particular task or masking parameters used in the present study, or whether it reflects a more generalized finding of equal sensitivity in subjective and objective tasks as long as bias-free measures are used. In addition, the present findings raise doubts about brain imaging studies purporting to demonstrate selective activation of brain areas in response to facial stimuli which are unavailable for conscious report (e.g. Morris et al., 1999; Whalen et al., 1998). Since these claims were also based on potentially biased measures of awareness, observers’ sensitivity to such stimuli may have been concealed by their response criterion. Such studies should be replicated at stimulus energy levels where observers show no sensitivity in tests of awareness using bias-free measures, in order to asses whether there remains any evidence of selective activation for truly ‘unconscious’ facial stimuli. The wider implications of a failure to demonstrate a dissociation between perception and awareness with bias-free measures relate to the functional architecture of the visual system as discussed earlier. Previously, dissociations demonstrated by psychophysical experiments or imaging studies have been taken as evidence for separate pathways mediating performance and awareness, or perception and action, or some other functional distinction. However the majority of these studies have
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
77
Fig. 6. Sensitivity, measured as da for the detection of fearful faces in yes–no () and 2afc tasks ( ) plotted as a function of stimulus contrast for 4 observers. Numbers in brackets refer to the Ekman face presented to each observer.
ignored or dismissed the possibility that response bias can account for the difference between objective and subjective thresholds. In the relatively few studies designed to include sufficient trials to allow calculation of sensitivity using rigorous SDT methods, comparisons have consistently shown – as in the present case – equally sensitivity in objective and subjective tasks for normal observers. Whilst SDT analysis indicates that performance can be dissociated from awareness following neurological damage (Azzopardi & Cowey, 1997, 1998, 2001), this depends on severe, qualitative disruption to normal functions of the visual system. It appears that for healthy observers, the same information is available to influence reports on phenomenological experience as for generating forced-choice responses; on these grounds, there is no reason to postulate that performance and awareness are modulated by separate neural pathways. Acknowledgment This work was supported by the United Kingdom Medical Research Council, Grant No. G971/387/B. References Azzopardi, P., & Cowey, A. (1997). Is blindsight like normal, near-threshold vision? Proceedings of the National Academy of Science of the United States of America, 94, 14190–14194. Azzopardi, P., & Cowey, A. (1998). Blindsight and visual awareness. Consciousness and Cognition, 7, 292–311.
78
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Azzopardi, P., & Cowey, A. (2001). Why blindsight is blind? In B. de Gelder, E. de Haan, & C. Heywood (Eds.), Varieties of unconscious processing: New findings and new models (pp. 3–19). Oxford: Oxford University Press. Balsdon, T., Azzopardi, P. (2015). Absolute and relative blindsight. Consciousness and Cognition, 32, 79–91. Cheesman, J., & Merickle, P. M. (1984). Priming with and without awareness. Perception & Psychophysics, 36, 387–395. Cheesman, J., & Merickle, P. M. (1986). Distinguishing conscious from unconscious perceptual processes. Canadian Journal of Psychology, 40, 343–367. Dixon, N. F. (1971). Subliminal perception: The nature of a controversy. London: McGraw-Hill. Dixon, N. F. (1981). Preconscious processing. Chichester: Wiley. Dorfman, D. D., & Alf, E. Jr., (1969). Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervalsrating-method data. Journal of Mathematical Psychology, 6, 487–496. Ekman, P., & Friesen, W. (1976). Pictures of Facial Affect. Palo Alto: Consulting Psychologists Press. Eriksen, C. W. (1960). Discrimination and learning without awareness: A methodological survey and evaluation. Psychological Review, 67, 279–300. Esteves, F., & Öhman, A. (1993). Masking the face: Recognitions of emotional facial expressions as a function of the paradigms of backward masking. Scandinavian Journal of Psychology, 34, 1–18. Green, D., & Swets, J. (1966). Signal detection theory and psychophysics. New York: Wiley. Hansen, C. H., & Hansen, R. D. (1988). Finding the face in the crowd: An anger superiority effect. Journal of Personality and Social Psychology, 54, 917–924. Holender, D. (1986). Semantic activation without conscious identification in dichotic listening, parafoveal vision, and visual masking: A survey and appraisal. Behavioural and Brain Sciences, 9, 1–66. LeDoux, J. (1989). Cognitive-emotional interactions in the brain. Cognition & Emotion, 3, 267–289. Macmillan, N., & Creelman, D. (1991). Signal detection theory: A user’s guide. Cambridge: Cambridge University Press. Marascuilo, L. (1970). Extensions of the significance test for one-parameter signal detection hypotheses. Psychometrika, 35, 237–243. Meeres, S. L., & Graves, R. (1990). Localisation of unseen visual stimuli by humans with normal vision. Neuropsychologia, 28, 1231–1237. Morris, J. S., Öhman, A., & Dolan, R. J. (1999). A subcortical pathway to the right amygdala mediating ‘‘unseen’’ fear. Proceedings of the National Academy of Science of the United States of America, 96, 1680–1685. Öhman, A. (1986). Face the beast and fear the face: Animal and social fears as prototypes for evolutionary analyses of emotion. Psychophysiology, 23, 123–145. Purcell, D. G., & Stewart, A. L. (1986). The face-detection effect. Bulletin of the Psychonomic Society, 24, 118–120. Purcell, D. G., & Stewart, A. L. (1988). The face-detection effect: configuration enhances detection. Perception and Psychophysics, 43, 355–366. Simpson, A., & Fitter, M. (1973). What is the best index of detectability? Psychological Bulletin, 80, 481–488. Taylor, M. M., & Creelman, C. D. (1967). PEST: Efficient estimates on probability functions. Journal of the Acoustical Society of America, 41, 782–787. Weiskrantz, L. (1986). Blindsight: A case study and its implications. Oxford: Oxford University Press. Reprinted with a new introduction, 1998. Whalen, P. J., Rauch, S. L., Etcoff, N. L., et al (1998). Masked presentations of emotional facial expressions modulated amygdala activity without explicit knowledge. Journal of Neuroscience, 18, 411–418. Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35, 151–175.
Contents lists available at ScienceDirect
Consciousness and Cognition journal homepage: www.elsevier.com/locate/concog
Thresholds for detection and awareness of masked facial stimuli Frances Heeks, Paul Azzopardi ⇑ Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK
a r t i c l e
i n f o
Article history: Received 7 January 2014 Revised 9 September 2014 Accepted 12 September 2014 Available online 11 October 2014 Keywords: Blindsight Conscious awareness Signal detection Backward masking Face perception Low vision
a b s t r a c t It has been suggested that perception without awareness can be demonstrated by a dissociation between performance in objective (forced-choice) and subjective (yes–no) tasks, and such dissociations have been reported both for simple stimuli and more complex ones including faces. However, signal detection theory (SDT) indicates that the subjective measures used to assess awareness in such studies can be affected by response bias, which could account for the observed dissociation, and this was confirmed by Balsdon and Azzopardi (2015) using simple visual targets. However, this finding may not apply to all types of stimulus, as the detectability of complex targets such as faces is known to be affected by their configuration as well as by their stimulus energy. We tested this with a comparison of forced-choice and yes–no detection of facial stimuli depicting happy or angry or fearful expressions using a backward masking paradigm, and using SDT methods including correcting for unequal variances in the underlying signal distributions, to measure sensitivity independently of response criterion in 12 normal observers. In 47 out 48 comparisons there was no significant difference between sensitivity (da) in the two tasks: hence, across the range of expressions tested it appears that the configuration of complex stimuli does not enhance detectability independently of awareness. The results imply that, on the basis of psychophysical experiments in normal observers, there is no reason to postulate that performance and awareness are mediated by separate processes. Ó 2015 Published by Elsevier Inc.
1. Introduction A considerable body of research has been directed towards the question of whether perception can occur without awareness. This ongoing interest not only reflects the philosophical debate as to whether perception implies consciousness, but also has implications for understanding the functional architecture of the visual system. If performance can be dissociated from awareness, either through neurological damage as in the case of blindsight (Weiskrantz, 1986), or through degraded stimulus conditions presented to normal observers (e.g. Meeres & Graves, 1990) it would suggest that information can influence performance through different neural pathways than those mediating conscious awareness. Demonstrating perception without awareness requires a dissociation between performance on a measure of perception and a measure of awareness. However, there remains considerable controversy as to what constitutes an appropriate indicator of awareness. Cheesman and Merickle (1984) distinguished between the subjective threshold, which they defined as the stimulus energy level at which observers claim not to be able to discriminate perceptual information above chance, and the objective threshold, which they defined as the level at which observers are actually unable to discriminate perceptual ⇑ Corresponding author at: Department of Experimental Psychology, University of Oxford, South Parks Rd, Oxford OX1 3UD, UK. Fax: +44 1865 310447. E-mail address: [email protected] (P. Azzopardi). http://dx.doi.org/10.1016/j.concog.2014.09.009 1053-8100/Ó 2015 Published by Elsevier Inc.
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
69
information above chance. Whilst some researchers have argued that awareness should always be assessed using objective measures of perceptual discrimination (e.g. Eriksen, 1960; Holender, 1986), Cheesman and Merickle (1986) argued that objective measures provide conservative indicators of awareness, and that a subjectively defined awareness threshold best captures the distinction between conscious and unconscious experiences. Following from their finding that the objective threshold is associated with somewhat lower stimulus energy levels than the subjective threshold indicated by subjects’ self-reports (Cheesman & Merickle, 1984), they suggest that unconscious processing may occur only for stimuli presented at energy levels between the objective and the subjective thresholds, with stimuli above the subjective threshold being consciously processed, and stimuli below the objective threshold undergoing no processing whatsoever. According to this formulation, performance should reach chance level at a lower stimulus energy for objective (i.e. forced choice) tasks than for subjective (i.e. yes–no) tasks; hence there should be intermediate stimulus levels at which observers perform at chance in a subjective task but above chance in an objective task. Such dissociations have been reported in several studies: for example, Meeres and Graves (1990) found that observers were able to detect a masked target shape significantly above chance in a spatial forced-choice task (i.e. in which of two possible spatial locations a stimulus was presented) despite being unable to detect the stimuli in a yes–no task. There is, however, a potentially serious problem with dissociations which are based on percentage correct scores in subjective and objective tasks because, according to signal detection theory (SDT) (Green & Swets, 1966), percentage scores reflect not only an observers’ sensitivity to stimuli, but also their response bias. As illustrated in Fig. 1, SDT assumes that internal signals associated with presentation of a stimulus (S+) is normally distributed about a mean, and the internal signals associated with the absence of a stimulus (S) is also distributed around a lower mean value. Decisions about whether a particular internal signal was caused by S+ or S require an observer to set a criterion value, c, above which they respond S+ and below which they respond S As the two distributions overlap this will result in both correct responses (hits and correct rejections) and mistakes (false alarms and misses) with the proportions of correct responses and errors – and hence the percentage correct scores – depending not only on the observer’s sensitivity but also on the position of their response criterion. Use of a conservative response criterion in the yes–no task may depress an observer’s percentage correct score relative to a force choice task: the accuracy of observers’ decision about which of two spatial or temporal intervals contained a stimulus should be unaffected by response bias provided that the stimuli are randomly distributed with respect to the spatial or temporal interval. Hence any dissociation of performance and awareness based on differential accuracy in forced-choice compared to yes–no tasks may simply reflect the influence of response bias. SDT provides a measure of sensitivity (d0 ) which is independent of response criterion and therefore avoids the problem of response bias (Green & Swets, 1966). Assuming that the two underlying distributions of signals associated with S+ and S are normal and have equal variances, then the difference between the means of the distributions in units of standard deviation can be given by d0 = z(H) z(F), where H (hit rate) = (number of hits/number of S + s) and F (false alarms rate) = (number of false alarms/number of S s), and z is the inverse of the normal distribution function. Despite the fact that SDT is well established, the possibility that response bias can account for the difference between objective and subjective thresholds has been consistently ignored or actively dismissed by the majority of researchers in perception without awareness (e.g. Dixon, 1971, 1981). In some cases, SDT methods have been applied inappropriately in order to lend credibility to findings established through potentially biased methods. For example, Meeres and Graves (1990) conducted a post hoc analysis of their localization and detection data to calculate d0 reporting a dissociation of sensitivity in the two task consistent with their main finding. However, Meeres & Graves did not test for the equality of variances of the two underlying distributions. This is a critical assumption in calculating d0 : if the variances are not equal, d0 does not provide a reliable measure of sensitivity because the value obtained will vary according to the response criterion. Given that the widespread claims of dissociations between performance and awareness in normal observers are based on measures susceptible to response bias, it is of general interest to know whether the dissociations stand up when truly bias-
Fig. 1. Signal detection theory.
70
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
free measures are used. Balsdon & Azzopardi (2015) have shown, in an experiment using metacontrast masking, that they do not. However, it might may be premature to infer the equivalence of objective and subjective thresholds once response bias is taken into account on the basis of a single study. Indeed, an important consideration is the type of stimuli used in this and other studies. Whilst Balsdon’s & Azzopardi’s demonstration was restricted to the detection of simple targets (squares and diamonds), many other claims for dissociations relate to complex stimuli such as words (e.g. Cheesman & Merickle, 1984) or faces (e.g. Esteves & Öhman, 1993), and there are a number of reasons to suspect that processing of complex targets might be different. First, a context effect has been documented in tasks involving detection of complex stimuli. For example, the threshold for detection of masked normal faces is lower than that for stimuli composed of rearranged facial features – the ‘face detection effect’ (Purcell & Stewart, 1986, 1988). This implies that detection is influenced not only by stimulus energy but also by the configuration of components as a meaningful whole, and suggests that facial expressions may have some kind of privileged access to the system(s) mediating performance in this task. Secondly, various theories of emotion have postulated such an advantage for facial expressions through unconscious, or preattentive processing of emotionally significant stimuli (LeDoux, 1989; Zajonc, 1980; Öhman, 1986). LeDoux argues that ‘‘the computation of stimulus significance takes place prior to and independent of conscious awareness’’ (p. 267), with the affective system receiving inputs from the early stages of sensory processing, allowing emotional responses to be initiated rapidly. Such mechanisms might confer a privileged status to emotional stimuli and provide information to influence forced-choice responding that might not reach conscious awareness. Certainly, there is evidence that masked faces expressing fear or anger cause selective activation of areas of the brain associated with emotion – particularly the amygdala – when subjects are unable to report their presence (Morris, Öhman, & Dolan, 1999; Whalen, Rauch, Etcoff, et al., 1998). There is also some evidence that facial stimuli may be processed in a way that can influence forced-choice performance below the threshold for awareness. Esteves and Öhman (1993, Experiment 3) found that subjects were able to identify masked facial expressions as happy at lower stimulus onset asynchrony (SOA) than that necessary for accurate confidence ratings. However, all of these studies have relied on percentage correct scores to assess subjective awareness, and as the preceding discussion of bias indicates, dissociations based on such measured are potentially misleading. The purpose of the present experiment was therefore to test whether or not any dissociation between objective and subjective thresholds for the detection of faces can be accounted for by response bias, as has been demonstrated previously for simpler stimuli. 2. Methods Participants were required to detect images depicting facial expressions during yes–no and forced-choice tasks using a backward masking paradigm. 2.1. Participants 12 observers participated in the experiment: 5 male and 7 female. Apart form one member of senior staff, all were undergraduates at Oxford University. Their median age was 21 years. All had normal or corrected vision. 2.2. Apparatus Stimuli were presented 1.25 m in front of the seated observer on an Eizo T220 2000 monitor (Eizo Corporation, Ishikawa, Japan) with a non-interlaced frame rate of 120 Hz, driven by a VSG2/4 visual stimulus generator (Cambridge Research Systems, Rochester, U.K.). Presentation of stimuli and registration of responses was controlled by a custom program run on a Dan IBM PC-compatible computer (200 MHz Intel Pentium MMX processor) operated by the experimenter in the same room, which was dimly lit. The apparatus was calibrated by means of a photodiode and oscilloscope to ensure that the stimuli and masks were presented at the specified times and for the specified durations. 2.3. Stimuli Targets were 8-bit digitized pictures of faces from Ekman and Friesen’s (1976) Pictures of Facial Affect: The images were standardized in the following way using a graphical software (Core Photo-Paint 8, Corel Corporation, USA). First, an oval mask tool was used to remove background cues such as hair, jewellery, labels and borders; the intensity of the selected portion of the image was then adjusted to give a mean grey level of 128, and merged with a background also of grey level 128. Targets subtended a visual angle of 5.9° by 8.1°. Blank images consisted of a uniform background of grey level 128. The apparatus was adjusted so that a grey-level of 128 corresponded to a mean luminance of 10 cd m2, calibrated with a Minolta LC1500 photometer. The mask consisted of overlapping capital letters (N and O) in black type on a white background, similar to that in used in previous face detection studies by Purcell and Stewart (1986, 1988). It subtended a visual angle of 8.2° by 11.8°, completely covering the region in which target stimuli were presented. See Fig. 2(a–c) for illustrations.
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
71
Fig. 2. Stimuli. (a) Original format of facial stimuli from Pictures of Facial Affect (Ekman & Frisen, 1976). (b) Standardized formal of facial stimuli used in the present study. (c) Pattern mask used in the present study (Purcell & Stewart, 1986).
2.4. Procedure Each observer was tested on forced choice and yes–no detection of a single target stimulus throughout the experiment, with stimuli randomly allocated to observers. in total, the experiment included 4 images for each of the three expressions (happy, angry, fearful) for which dissociations had previously been reported in the literature; specific image numbers are given in Table 1. The forced choice task involved two temporal intervals: A target (30 ms) was shown in one interval, a blank (30 ms) in the other, with each interval followed immediately by the pattern mask (30 ms) a blank screen was shown for 2150 ms between the two intervals. After viewing both intervals, observers were asked to respond ‘1’ or ‘2’ to indicate in which interval the thought the target had been presented. The yes–no task involved only one interval: Either a target or a blank was shown (30 ms), followed immediately by the pattern mask (30 ms), and observers responded ‘yes’ or ‘no’ to indicate whether the thought a target had been presented, then rated their confidence in their decision on a scale of 1 (not at all certain) to 4 (certain). Observers were informed that a face target would be presented on 50% of trials. Responses were made verbally and recorded on computer by the experimenter. Target and mask durations were set at 30 ms each because Esteves and Öhman (1993) had reported these parameters to be below the threshold for conscious perception in their identification task. However, our pilot studies indicated that stimuli could be detected very accurately with these timings; hence contrast was reduced to ensure near-threshold performance. A PEST calibration procedure (Macmillan & Creelman, 1991; Taylor & Creelman, 1967) was used to determine the stimulus contrast at which each observer performed with 75% accuracy in the forced choice detection task for their allocated stimulus; in the experimental trials the target was then presented at a range of four contrasts bracketing the 75% threshold contrast. Each observer completed 4 blocks of forced-choice trials and 4 blocks of yes–no trials, alternating task type between every block; half the observers began with a yes–no block, the others with a forced choice block. For both tasks, blocks contained 160 trials, 40 at each of the 4 stimulus contrasts, randomised with respect to contrast level and order of targets/ blanks. 3. Results For the forced choice task, observers’ sensitivity was calculated as d0 directly from hit rate and false alarm rates and p adjusted by a factor of 1/ 2 to reflect the fact that the forced choice task allows two opportunities to detect the stimulus (Macmillan & Creelman, 1991). For the yes–no task, observers’ confidence ratings were used to construct ROCs (graphs of hit rate against false alarm rate) using a maximum-likelihood algorithm (Dorfman & Alf, 1969). ROCs for one observer are illustrated in Fig. 3. As shown in Table 1, the slopes of these ROCs (given as B) in many cases deviated significantly from 1.0 indicating that the variances of the two underlying distributions were not equal. Sensitivity in the yes–no task was therefore calculated as da which takes into account the possibility of unequal variances (Simpson & Fitter, 1973). ROC parameters are presented in Table 1. Figs. 4–6 show the sensitivity of each observer, measured at 4 contrasts for a particular happy, angry or fearful face in the forced choice and yes–no tasks, with error bars indicating the 95% confidence limits. The difference in sensitivity values for the two tasks was tested for significance using a Z-test (Marascuilo, 1970), onetailed since the sensitivity was predicted to be higher, if anything, in the objective compared to the subjective task. Of the 48 tests, 10 reached significance at a comparison-wise level of Pc = .05, but only one of them remained significance after a Bon-
72
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Table 1 da Values for yes–no and forced choice tasks, including ROC parameters for yes–no task. Expression
Obs
Con
Task
A
sA
B
sB
P (B)
da
sda
Happy (022)
FI
0.10
yn 2afc yn 2afc yn 2afc yn 2afc
0.0892 – 0.6084 – 0.9334 – 1.3942 –
0.1603 – 0.1627 – 0.1607 – 0.1888 –
0.9422 – 0.8631 – 0.6302 – 0.5386 –
0.1251 – 0.1256 – 0.1075 – 0.1212 –
NS – NS –
0.0918 0.1337 0.6513 0.7703 1.1168 1.1292 1.7359 1.6120
0.1653 0.1406 0.1702 0.1482 0.1819 0.1577 0.2078 0.1852
yn 2afc yn 2afc yn 2afc yn 2afc
0.0811 – 0.0576 – 0.2519 – 0.4712 –
0.1679 – 0.1609 – 0.1422 – 0.1536 –
1.0258 – 0.9336 – 0.6307 – 0.7765 –
0.1345 – 0.1250 – 0.0949 – 0.1055 –
0.0800 0.1570 0.0595 0.3408 0.3013 0.3813 0.5263 0.2931
0.1657 0.1410 0.1663 0.1425 0.1698 0.1420 0.1695 0.1418
yn 2afc yn 2afc yn 2afc yn 2afc
0.2899 – 0.2290 – 0.5042 – 0.6815 –
0.1585 – 0.1635 – 0.1661 – 0.1561 –
0.8593 – 0.9591 – 0.9007 – 0.7002 –
0.1200 – 0.1228 – 0.1225 – 0.1029 –
0.3109 0.1836 0.2338 0.4097 0.5298 0.5494 0.7895 0.8438
0.1692 0.1423 0.1659 0.1430 0.1702 0.1472 0.1749 0.1517
0.5419
0.1583 – 0.1467 – 0.1534 – 0.1534 –
0.8459 – 0.6554 – 0.6698 – 0.5006 –
0.1107 – 0.0927 – 0.0980 – 0.0824 –
0.5851 0.3382 0.6474 0.4741 0.8514 0.8997 1.2048 1.0572
0.1678 0.1459 0.1706 0.1430 0.1736 0.1533 0.1854 0.1568
0.1636 – 0.1720 – 0.2375 – 0.2938 –
0.9547 – 0.9794 – 0.8049 – 0.5334 –
0.1263 – 0.1298 – 0.1573 – 0.1506 –
0.2741 0.2910 0.5794 0.7673 1.8890 1.6710 2.5903 2.7274
0.1659 0.1414 0.1691 0.1479 0.2036 0.1822 0.2790 0.3043
0.1754 – 0.1593 – 0.1548 – 0.1395 –
1.0544 – 0.8702 – 0.7978 – 0.5695 –
0.1432 – 0.1197 – 0.1106 – 0.0822 –
0.3447 0.5658 0.3592 0.6433 0.4282 1.0316 0.4215 1.2281
0.1677 0.1466 0.1676 0.1457 0.1686 0.1563 0.1704 0.1635
0.1657 – 0.1492 – 0.2317 – 0.1816 –
0.8015 – 0.6552 – 0.7541 – 0.7502 –
0.1201 – 0.0963 – 0.1561 – 0.1249 –
0.7751 0.7239 0.6673 1.0249 1.7875 1.2943 1.3039 1.4663
0.1812 – 0.1904 – 0.2747 – 0.4214 –
0.9981 – 0.6028 – 0.8007 – 0.5025 –
0.1370 – 0.1122 – 0.1734 – 0.2011 –
0.1550 –
0.7596 –
0.1137 –
0.13 0.16 0.19 Happy (048)
DJ
0.08 0.11 0.14 0.17
Happy (057)
SP
0.08 0.11 0.14 0.17
Happy (066)
PA
0.16 0.18 0.20 0.22
Angry (003)
FH
0.03 0.04 0.05 0.06
Angry (025)
SG
0.25 0.30 0.35 0.40
Angry (069)
JB
0.14 0.18 0.22 0.26
Angry (105)
AH
0.18 0.22 0.26 0.30
Fearful (024)
JG
0.20
yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc
– 0.5473 – 0.7246 – 0.9527 – 0.2679 – 0.5735 – 1.7146 – 2.0759 – 0.3542 – 0.3367 – 0.3873 – 0.3430 – 0.7024 – 0.5641 – 1.5831 – 1.1526 – 0.7015 – 1.3553 – 1.9729 – 2.5150 – 0.4821 –
⁄⁄⁄
– ⁄⁄⁄
– NS – NS – ⁄⁄⁄
– ⁄
– NS – NS – NS – ⁄⁄
– NS – ⁄⁄⁄
– ⁄⁄⁄
– ⁄⁄⁄
– NS – NS – NS – ⁄⁄
– NS – NS – NS – ⁄⁄⁄
– NS – ⁄⁄⁄
– NS – ⁄
– NS – ⁄⁄⁄
– NS – ⁄
– ⁄
–
Z
P (Z)c
P (Z)e
1.039
NS
NS
0.527
NS
NS
0.052
NS
NS
0.445
NS
NS
0.354
NS
NS
1.828
⁄
NS
0.361
NS
NS
1.055
NS
NS
0.091
NS
NS
0.803
NS
NS
0.087
NS
NS
0.234
NS
NS
1.110
NS
NS
0.779
NS
NS
0.209
NS
NS
0.608
NS
NS
0.078
NS
NS
0.836
NS
NS
0.798
NS
NS
0.332
NS
NS
0.992
NS
NS
1.279
NS
NS
2.625
⁄⁄
NS
3.415
⁄⁄⁄
NS
0.1742 0.1644 0.1724 0.1718 0.2050 0.1937 0.1849 0.1815
0.214
NS
NS
1.469
NS
NS
⁄
NS
0.627
NS
NS
0.7021 0.7610 1.6415 1.8565 2.1779 2.9626 3.1781 3.7929
0.1713 0.1519 0.1989 0.2041 0.2161 0.3410 0.3532 0.3554
0.257
NS
NS
0.754
NS
NS
1.944
⁄
NS
1.227
NS
NS
0.5429 0.5943
0.1684 0.1531
0.226
NS
NS
–1.748
73
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78 Table 1 (continued) Expression
Obs
Con
Task
0.25
yn 2afc yn 2afc yn 2afc
0.30 0.35 Fearful (037)
GK
0.18 0.21 0.24 0.27
Fearful (095)
IB
0.17 0.20 0.23 0.26
Fearful (104)
IM
0.25 0.30 0.35 0.40
yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc yn 2afc
A 0.5470 – 0.4104 – 0.6929 – 0.5533 – 0.5771 – 1.2982 – 1.2010 – 0.0253 – 0.3324 – 0.6516 – 1.0872 – 0.5666 – 1.0125 – 1.0372 – 1.5885 –
sA
B
sB
P (B)
0.1592 – 0.1509 – 0.1685 –
0.8387 – 0.7753 – 0.9099 –
0.1240 – 0.1141 – 0.1403 –
NS –
0.1611 – 0.1558 – 0.1924 – 0.1719 –
0.8694 – 0.8080 – 0.8133 – 0.5824 –
0.1166 – 0.1064 – 0.1298 – 0.0994 –
NS – NS – NS –
0.1578 – 0.1592 – 0.1528 – 0.1625 –
0.9212 – 0.9134 – 0.6916 – 0.5709 –
0.1174 – 0.1169 – 0.0976 – 0.0966 –
0.1743 – 0.1664 – 0.1682 – 0.2034 –
1.0207 – 0.7054 – 0.6997 – 0.4072 –
0.1357 – 0.1096 – 0.1101 – 0.1023 –
⁄
– NS –
⁄⁄⁄
– NS – NS – ⁄⁄
– ⁄⁄⁄
– NS – ⁄⁄
– ⁄⁄
– ⁄⁄⁄
–
da
sda
P (Z)c
P (Z)e
1.540
NS
NS
1.983
⁄
NS
1.744
NS
NS
0.1693 0.1459 0.1671 0.1507 0.1845 0.1695 0.1911 0.2057
0.133
NS
NS
0.978
NS
NS
0.018
NS
NS
1.858
⁄
NS
0.0263 0.1777 0.3471 0.2003 0.7579 0.6421 1.3353 0.9264
0.1642 0.1405 0.1652 0.1407 0.1719 0.1455 0.1833 0.1516
0.701
NS
NS
0.677
NS
NS
0.514
NS
NS
1.691
⁄
NS
0.5607 0.2452 1.1701 0.7219 1.2018 1.0692 2.0806 1.6779
0.1668 0.1410 0.1782 0.1475 0.1790 0.1558 0.2356 0.1833
1.445
NS
NS
1.938
⁄
NS
0.559
NS
NS
1.349
NS
NS
0.5928 0.9447 0.4587 0.9120 0.7248 1.1343
0.1670 0.1560 0.1660 0.1572 0.1685 0.1635
0.5905 0.5607 0.6349 0.8550 1.4243 1.4287 1.4677 1.9894
Z
Key: SOA – stimulus onset asynchrony; A – y intercept of ROC plotted in z coordinates, equal to d0 when the variances of the two underlying distributions are equal; sA – standard deviation of A; B – gradient of ROC plotted in z coordinates, equivalent to the ratio of the variances of the two underlying distributions; sB – standard deviation of B; P(B) – probability of B deviating from a value of 1.0 by chance; da – measure of sensitivity, taking into account inequality of variances of the underlying distributions; sda – standard deviation of da; Z – the difference between da(yn) and da(2afc) expressed as a normal deviate; P(Z)c – comparison-wise probability of the difference between da(yn) and da(2afc) arising by chance; P(Z)e – experiment-wise probability of the difference between da(yn) and da(2afc) arising by chance. For P(B), P(Z)c and P(Z)e: ⁄ P < .05, ⁄⁄ P < .01, ⁄⁄⁄ P < .001. Each set of parameters is derived from 160 trials.
ferroni correction was applied to give an experiment-wise level of Pe = .05. Therefore, across a range of expressions and contrasts, these results provide virtually no evidence of a significant dissociation between sensitivity to facial stimuli in subjective and objective tasks. 4. Discussion This experiment was designed to demonstrate unconscious processing of facial expression according to the conditions outlined by Cheesman and Merickle (1986) by comparing forced-choice discrimination with observers’ reports of what they could see when using near-threshold stimuli. However, for the vast majority of cases there was no significant difference between sensitivity in yes–no and forced-choice detection of faces when bias-free measures were used. This applied across the range of expressions (happy, angry, fearful) for which dissociations had been previously reported in the literature. These findings suggest that for faces, as for simpler stimuli (Balsdon & Azzopardi, 2015), dissociations between subjective and objective measure may arise trivially as a result of response bias rather than as a result of unconscious pressing producing enhanced sensitivity in the objective task. Only one previous study has conducted a comparison of performance in subjective and objective tasks using masked facial stimuli. As mentioned earlier, Esteves and Öhman (1993) reported for happy faces that the threshold SOA required for accurate performance in their objective task was significantly lower than that for their subjective measure. Although there was no significant difference between the two thresholds when angry faces were tested, Esteves & Öhman suggest that this may reflect their failure to test an appropriately narrow range of SOAs, and conclude generally in support of Cheesman and Merickle’s (1986) hypothesis. However, performance in Esteves & Öhman’s study was assessed by percentage correct scores which may be influenced by response bias as discussed: this is one possible explanation for the discrepancy between their findings and those of the present study, which found no such dissociation when using bias-free measures. There is, however, another possible explanation for this discrepancy which relates to the use of different masks in the two studies:
74
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Fig. 3. Example ROCs fitted to the responses obtained from a single observer (FI) during yes–no detection. Key: H = hit rate; F = false alarm rate; z is the inverse of the normal distribution function; da = estimate of d0 corrected for unequal variances of underlying signal distributions (Simpson & Fitter, 1973).
whilst Esteves & Öhman used a neutral face to mask target facial expressions, the present study used an abstract pattern mask. The decision to use a pattern mask rather than a neutral face mask was intended to avoid any potentially confounding interactions or artefacts such as apparent motion arising from similarity between target facial expressions and a neutral face mask. For example, Esteves and Öhman (1993) note a confusion between angry and neutral faces (Hansen & Hansen, 1988) which may underlie their observers’ tendency to identify the neutral mask faces as angry more often than happy. Interactions between target and mask faces might produce effects which are independent of the processing of the target itself, and in this sense were seen as confounding factors to be excluded in the present experiment. However, it is possible that such interactions might have specific effects on facilitating forced choice performance independently of awareness, and
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
75
Fig. 4. Sensitivity, measured as da for the detection of happy faces in yes–no () and 2afc tasks ( ) plotted as a function of stimulus contrast for 4 observers. Numbers in brackets refer to the Ekman face presented to each observer.
could therefore have contributed to the dissociation observed in Esteves & Öhman’s study. Whilst there are no obvious mechanism by which the masking stimulus might influence the relationship between objective and subjective performance, this remains a possible explanation for the discrepancy between the two studies, and the present study might be replicated using a neutral face mask in order to assess whether there are any masking conditions in which a dissociation can be demonstrated using bias-free measures. Whilst the present study demonstrated equal sensitivity in subjective and objective tasks relating to detection of facial stimulus (present or absent), Esteves & Öhman’s tasks related to identification of facial expressions (as happy or angry): it remains to be seen whether observers would similarly show equal sensitivity in subjective and objective identification tasks if assess using bias-free measures. Clearly, identifying the emotion expressed by a face requires more detailed visual analysis than is required for simple presence/absence decisions: it therefore remains possible that identification performance could be selectively facilitated by unconscious processing of the emotional significance of stimuli as postulated by LeDoux (1989) and others. This possibility requires explicit testing, using bias-free measures of sensitivity to compare yes–no and forcedchoice identification performance. In contrast to the ‘forced choice’ identification task used by Esteves and Öhman (1993), where observers showed a bias in labelling faces as angry more often than as happy, a truly objective test of identification might involve participants in deciding in which of two temporal intervals a happy face (or angry, or fearful, etc.) was presented, given that a neutral face was presented in the other interval. It is necessary to present a neutral face rather than a blank screen in the other interval in order to asses observers’ perception of facial expressions (i.e. stimulus content) in isolation from their ability to detect the presence of a facial stimulus at all. Similarly, a subjective test of identification might involve a yes–no response (with
76
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Fig. 5. Sensitivity, measured as da for the detection of angry faces in yes–no () and 2afc tasks ( ) plotted as a function of stimulus contrast for 4 observers. Numbers in brackets refer to the Ekman face presented to each observer.
confidence ratings) as to whether a happy face was presented, with either a happy or a neutral face presented in each trial. As in the present study, a range of facial expressions should be tested to ensure that findings can be generalized. Observers’ responses in these tasks could then be used to calculate the sensitivity (da) as described in previous sections in order to compare yes–no and forced choice identification of facial expressions independently of response bias. The present study has demonstrated that sensitivity to facial stimuli backwardly masked by an abstract pattern does not differ significantly in subjective and objective detection tasks when calculated independently of response criterion. It remains to be seen whether this result depends on the particular task or masking parameters used in the present study, or whether it reflects a more generalized finding of equal sensitivity in subjective and objective tasks as long as bias-free measures are used. In addition, the present findings raise doubts about brain imaging studies purporting to demonstrate selective activation of brain areas in response to facial stimuli which are unavailable for conscious report (e.g. Morris et al., 1999; Whalen et al., 1998). Since these claims were also based on potentially biased measures of awareness, observers’ sensitivity to such stimuli may have been concealed by their response criterion. Such studies should be replicated at stimulus energy levels where observers show no sensitivity in tests of awareness using bias-free measures, in order to asses whether there remains any evidence of selective activation for truly ‘unconscious’ facial stimuli. The wider implications of a failure to demonstrate a dissociation between perception and awareness with bias-free measures relate to the functional architecture of the visual system as discussed earlier. Previously, dissociations demonstrated by psychophysical experiments or imaging studies have been taken as evidence for separate pathways mediating performance and awareness, or perception and action, or some other functional distinction. However the majority of these studies have
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
77
Fig. 6. Sensitivity, measured as da for the detection of fearful faces in yes–no () and 2afc tasks ( ) plotted as a function of stimulus contrast for 4 observers. Numbers in brackets refer to the Ekman face presented to each observer.
ignored or dismissed the possibility that response bias can account for the difference between objective and subjective thresholds. In the relatively few studies designed to include sufficient trials to allow calculation of sensitivity using rigorous SDT methods, comparisons have consistently shown – as in the present case – equally sensitivity in objective and subjective tasks for normal observers. Whilst SDT analysis indicates that performance can be dissociated from awareness following neurological damage (Azzopardi & Cowey, 1997, 1998, 2001), this depends on severe, qualitative disruption to normal functions of the visual system. It appears that for healthy observers, the same information is available to influence reports on phenomenological experience as for generating forced-choice responses; on these grounds, there is no reason to postulate that performance and awareness are modulated by separate neural pathways. Acknowledgment This work was supported by the United Kingdom Medical Research Council, Grant No. G971/387/B. References Azzopardi, P., & Cowey, A. (1997). Is blindsight like normal, near-threshold vision? Proceedings of the National Academy of Science of the United States of America, 94, 14190–14194. Azzopardi, P., & Cowey, A. (1998). Blindsight and visual awareness. Consciousness and Cognition, 7, 292–311.
78
F. Heeks, P. Azzopardi / Consciousness and Cognition 32 (2015) 68–78
Azzopardi, P., & Cowey, A. (2001). Why blindsight is blind? In B. de Gelder, E. de Haan, & C. Heywood (Eds.), Varieties of unconscious processing: New findings and new models (pp. 3–19). Oxford: Oxford University Press. Balsdon, T., Azzopardi, P. (2015). Absolute and relative blindsight. Consciousness and Cognition, 32, 79–91. Cheesman, J., & Merickle, P. M. (1984). Priming with and without awareness. Perception & Psychophysics, 36, 387–395. Cheesman, J., & Merickle, P. M. (1986). Distinguishing conscious from unconscious perceptual processes. Canadian Journal of Psychology, 40, 343–367. Dixon, N. F. (1971). Subliminal perception: The nature of a controversy. London: McGraw-Hill. Dixon, N. F. (1981). Preconscious processing. Chichester: Wiley. Dorfman, D. D., & Alf, E. Jr., (1969). Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervalsrating-method data. Journal of Mathematical Psychology, 6, 487–496. Ekman, P., & Friesen, W. (1976). Pictures of Facial Affect. Palo Alto: Consulting Psychologists Press. Eriksen, C. W. (1960). Discrimination and learning without awareness: A methodological survey and evaluation. Psychological Review, 67, 279–300. Esteves, F., & Öhman, A. (1993). Masking the face: Recognitions of emotional facial expressions as a function of the paradigms of backward masking. Scandinavian Journal of Psychology, 34, 1–18. Green, D., & Swets, J. (1966). Signal detection theory and psychophysics. New York: Wiley. Hansen, C. H., & Hansen, R. D. (1988). Finding the face in the crowd: An anger superiority effect. Journal of Personality and Social Psychology, 54, 917–924. Holender, D. (1986). Semantic activation without conscious identification in dichotic listening, parafoveal vision, and visual masking: A survey and appraisal. Behavioural and Brain Sciences, 9, 1–66. LeDoux, J. (1989). Cognitive-emotional interactions in the brain. Cognition & Emotion, 3, 267–289. Macmillan, N., & Creelman, D. (1991). Signal detection theory: A user’s guide. Cambridge: Cambridge University Press. Marascuilo, L. (1970). Extensions of the significance test for one-parameter signal detection hypotheses. Psychometrika, 35, 237–243. Meeres, S. L., & Graves, R. (1990). Localisation of unseen visual stimuli by humans with normal vision. Neuropsychologia, 28, 1231–1237. Morris, J. S., Öhman, A., & Dolan, R. J. (1999). A subcortical pathway to the right amygdala mediating ‘‘unseen’’ fear. Proceedings of the National Academy of Science of the United States of America, 96, 1680–1685. Öhman, A. (1986). Face the beast and fear the face: Animal and social fears as prototypes for evolutionary analyses of emotion. Psychophysiology, 23, 123–145. Purcell, D. G., & Stewart, A. L. (1986). The face-detection effect. Bulletin of the Psychonomic Society, 24, 118–120. Purcell, D. G., & Stewart, A. L. (1988). The face-detection effect: configuration enhances detection. Perception and Psychophysics, 43, 355–366. Simpson, A., & Fitter, M. (1973). What is the best index of detectability? Psychological Bulletin, 80, 481–488. Taylor, M. M., & Creelman, C. D. (1967). PEST: Efficient estimates on probability functions. Journal of the Acoustical Society of America, 41, 782–787. Weiskrantz, L. (1986). Blindsight: A case study and its implications. Oxford: Oxford University Press. Reprinted with a new introduction, 1998. Whalen, P. J., Rauch, S. L., Etcoff, N. L., et al (1998). Masked presentations of emotional facial expressions modulated amygdala activity without explicit knowledge. Journal of Neuroscience, 18, 411–418. Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35, 151–175.