- Split View
-
Views
-
Cite
Cite
Luiz Pessoa, Shruti Japee, David Sturman, Leslie G. Ungerleider, Target Visibility and Visual Awareness Modulate Amygdala Responses to Fearful Faces, Cerebral Cortex, Volume 16, Issue 3, March 2006, Pages 366–375, https://doi.org/10.1093/cercor/bhi115
- Share Icon Share
Abstract
The goals of the present study were twofold. First, we wished to investigate the neural correlates of aware and unaware emotional face perception after characterizing each subject's behavioral performance via signal detection theory methods. Second, we wished to investigate the extent to which amygdala responses to fearful faces depend on the physical characteristics of the stimulus independently of the percept. We show that amygdala responses depend on visual awareness. Under conditions in which subjects were not aware of fearful faces flashed for 33 ms, no differential activation was observed in the amygdala. On the other hand, differential activation was observed for 67 ms fearful targets that the subjects could reliably detect. When trials were divided into hits, misses, correct rejects, and false alarms, we show that target visibility is an important factor in determining amygdala responses to fearful faces. Taken together, our results further challenge the view that amygdala responses occur automatically.
Introduction
It has been proposed that the processing of some emotional stimuli, such as fearful faces, is not only somewhat independent of attention, but that it can take place without reaching conscious awareness (Ohman, 2002; Dolan, 2003; Pessoa, 2005). Evidence for this view comes from studies showing both skin conductance and neuroimaging responses to briefly presented and masked stimuli that subjects were apparently unaware of (Ohman et al., 1995; Morris et al., 1998; Whalen et al., 1998). Thus, the view has emerged that emotional (especially negative) faces are processed in a largely automatic fashion, which is independent of attention and awareness.
Brief presentation and backward masking have been used to manipulate awareness during the viewing of emotional stimuli. A strategy devised by Ohman and colleagues is widely used for masking emotional faces (Esteves and Ohman, 1993). An initial emotional target face is presented for a brief duration, typically ∼30 ms, and is immediately followed by a neutral face that is shown for a slightly longer duration, typically 50 ms or more. Under such conditions, it is widely believed that emotional faces are effectively masked because subjects exhibit chance levels of performance at detecting the target face or because they report not having seen the stimuli on subsequent debriefing. However, in the past, chance performance has been assessed by determining percent correct values, which are known to be highly sensitive to response bias (Green and Swets, 1966; Macmillan and Creelman, 1991). In the face of weak, noisy signals, subjects may often indicate not detecting target stimuli and thus appear to be unable to reliably detect them. Thus, the assessment of chance performance by percent correct values can seriously distort results and lead to an assessment of objective awareness that is incorrect — participants may appear to be unaware of stimuli when in fact they can reliably detect them.
Visual awareness can be characterized by both objective and subjective criteria (Merikle et al., 2001; Snodgrass, 2004; Snodgrass et al., 2004). In a recent behavioral study, we assessed awareness according to objective criteria by having subjects perform a forced-choice fear detection task (Pessoa et al., 2005). Performance was evaluated according to standard signal detection theory (SDT) methods, which provide a measure of sensitivity that is independent of a subject's response bias (Green and Swets, 1966; Macmillan and Creelman, 1991). We varied the duration of an emotional target stimulus and characterized behavioral performance with receiver operating characteristic (ROC) curves to determine whether subjects could (aware) or could not (unaware) reliably detect briefly presented and masked fearful faces. Our results revealed large individual differences in subjects' ability to detect near-threshold fearful faces. In fact, ∼60% of the participants were able to reliably detect 33 ms masked fearful targets, a duration linked with unaware perception in the past.
The goals of the present study were twofold. First, we wished to investigate the neural correlates of aware and unaware emotional face perception after characterizing each subject's behavioral performance via SDT methods. Second, we wished to investigate the extent to which amygdala responses to fearful faces depend on the physical characteristics of the stimulus and are independent of the percept. The hypothesis that amygdala responses are largely automatic would predict that stimulus visibility would have little or no impact on amygdala responses. On the other hand, we reasoned that if amygdala responses are more closely tied to the perception of fear, stimulus visibility would be expected to modulate responses. To address these two questions, we employed functional magnetic resonance imaging (fMRI) while subjects performed a yes-no fear detection task. As in our prior behavioral study, performance was evaluated according to SDT methods for each individual (Pessoa et al., 2005). fMRI responses were evaluated for both aware and unaware conditions, as well as a function of the subject's report. In this manner, we assessed how amygdala responses depend on awareness and whether they are modulated by target visibility.
Materials and Methods
Subjects
Thirty-seven normal volunteers (18 females) aged 28.1 ± 5.9 (mean ± SD) years participated in the study, which was approved by the National Institute of Mental Health (NIMH) Institutional Review Board. All subjects were in good health with no past history of psychiatric and neurological disease and gave informed consent. Subjects had normal or corrected-to-normal vision.
Trial Structure
Each trial began with a white fixation cross shown for 300 ms, followed by a 50 ms blank screen, followed by a pair of faces presented consecutively (Fig. 1). Each pair of faces consisted of a fearful, happy or neutral face that was immediately followed by a neutral face, which served as a mask. The duration of the first face was either 33 or 67 ms. The total duration of the face pair was fixed at 150 ms. Subjects were instructed that the stimulus would always comprise two faces, but that sometimes the first would be very brief and the pair could appear as a single face. They were instructed to respond ‘fear’ if they perceived fear, however briefly. Following the presentation of each face pair, subjects indicated ‘fear’ or ‘no fear’ with a button press using a device that contained five buttons arranged in a ‘+’ shape. On each trial, subjects also rated the confidence in their response on a scale of 1–3, 1 corresponding to low confidence, 3 to high confidence. The total trial duration was 4.5 s. Each subject performed 324 trials, providing 54 trials per condition (six total conditions: two target durations × three face pair types). Faces subtended 4° of visual angle. Target presentation durations were confirmed by employing a photodiode and an oscilloscope.
Stimuli
Face stimuli were obtained from the Ekman set (Ekman and Friesen, 1976), a set recently developed by Ohman and colleagues (KDEF, D. Lundqvist, A. Flykt and A. Ohman, Karolinska Hospital, Stockholm, Sweden), as well as a set developed and validated by Alumit Ishai at NIMH. Fifty-four instances each of fearful, happy and neutral faces were employed. Happy faces were included to more closely match fearful faces in terms of low-level features, such as brightness around the mouth and eye regions, as both fearful and happy faces tend to be brighter than neutral ones in these regions. Thus, the inclusion of happy faces precluded subjects from utilizing a strategy of ‘detecting’ fearful faces by simply using such low-level cues. The inclusion of happy faces also precluded subjects from adopting a strategy of indicating ‘fear’ whenever facial features deviated from those of a neutral face. However, happy faces were not included in the fMRI analyses (see below).
Behavioral Data Analysis
Behavioral response data were analyzed using receiver operating characteristic (ROC) curves (Green and Swets, 1966; Macmillan and Creelman, 1991). ROC curves were obtained by determining the probability of reporting ‘fear’ given that the target was not a fearful face [P(‘Fear’|not Fear), i.e. false alarm] and the probability of reporting ‘fear’ given that the target was a fearful face [P(‘Fear’|Fear), i.e. hit] for every confidence rating (three levels for reporting ‘fear’ and three for reporting ‘no fear’). In this manner, confidence ratings were used to gauge a subject's criterion, and were associated with different proportions of hits and false alarms, allowing us to construct ROC curves. Sensitivity to fearful stimuli was determined using A′, the area under the ROC curve and tested for significance for each individual. Perception was considered aware when A′ values were significantly greater than 0.5 (Hanley and McNeil, 1982), the value of the area under the ROC curve associated with chance performance (y = x line, i.e. the same values for false alarms and hits); otherwise, perception was considered unaware. The P value adopted for statistical significance was 0.05. We discarded five subjects who were unable to detect stimuli at 67 ms. Two additional participants exhibited a trend towards being overachievers (P < 0.1) and were thus not included in either group (i.e. they were excluded).
fMRI Data Acquisition and Analysis
fMRI data were collected using a General Electric 1.5 Tesla scanner. Each scanning session began with the acquisition of a high resolution SPGR anatomical sequence. Each subject performed nine experimental runs, each lasting 5 min 53 s. During each functional scan, 157 gradient echo echo-planar volumes were acquired with a TE of 36 ms and a TR of 2.25 s. Each volume consisted of 27 axial slices with slice thickness of 4 mm and in-plane resolution of 3.75 × 3.75 mm.
fMRI data were analyzed using the General Linear Model in AFNI (Cox et al., 1996). Different trial types were randomly ordered to maximize their separation (Dale, 1999; Birn et al., 2002). The first four volumes of each run were discarded and slice-timing correction was performed on the remaining volumes. These volumes were then spatially registered to the volume acquired closest in time to the particular subject's high-resolution SPGR anatomy. Three subjects whose movement parameters thus obtained were larger than a single voxel dimension (4 mm), were discarded from further analysis. Next, each volume was spatially smoothed with a 6 mm Gaussian filter (full-width half-maximum). Each subject's data were analyzed in two ways. First, a multiple regression analysis was performed using the six trial types (two target durations × three face pair types) as regressors of interest. The goal of this analysis was to assess the degree to which evoked responses depended on the physical characteristics of the stimuli (e.g. did or did not contain a fearful target face). In a second analysis, to probe fMRI responses as a function of the subject's report, trials were grouped into five conditions (for each target duration) based on the participant's response: hits (correct fear-present trials), misses (incorrect fear-present trials), correct rejections (CRs, correct neutral-present trials), false alarms (FAs, incorrect neutral-present trials) and invalid trials containing late or no responses. Only results of 67 ms trials were analyzed in this manner since, for 33 ms trials, several trial types had an insufficient number of repetitions (<10). Although happy faces were included in the experiment, they were treated as ‘control’ stimuli and were not included in either of the two types of analysis (i.e. trials involving happy faces were included in a regressor of no interest).
fMRI analyses employed standard multiple regression methods (Friston et al., 1995). The linear models included a constant term and a linear term (for each run) that served as covariates of no interest (these terms controlled for drifts of MR signal across and within runs). F-maps of the contrasts of interest were generated for each individual. Fixed-effects statistical group maps were then obtained by converting each individual's F-map into a Z-map and then combining these into a composite final Z-map. For that purpose, each individual's brain was transformed with AFNI into the standard coordinate space of Talairach and Tournoux (1988). These transformed maps were then combined (averaged together and multiplied by the square root of the number of subjects). To assess the reliability of the results in terms of the population, we performed a mixed-effects analysis in which participant was a random factor and target-mask pair and target duration were fixed factors (see Tables 1 and 2). For this purpose, a standard two-stage analysis employed the regression coefficients (‘parameter estimates’) obtained from multiple linear regression for each experimental condition (first stage), which were then employed (second stage) in paired t-tests or repeated-measures analyses of variance.
Fear > neutral . | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Normals: 67 ms | ||||||
L. amygdala | −20 | 0 | −13 | 3.12a | ||
R. amygdala | 20 | −4 | −11 | 2.72b | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.13a | ||
Overachievers: 67 ms | ||||||
L. amygdala | −21 | −8 | −11 | 3.43a | ||
R. amygdala | 29 | −6 | −9 | 4.85a | ||
L. fusiform gyrus | −41 | −44 | −18 | 5.76a | ||
R. fusiform gyrus | 44 | −50 | −25 | 4.74a | ||
Overachievers: 33 ms | ||||||
L. amygdala | −16 | −5 | −5 | 3.14a | ||
L. fusiform gyrus | −33 | −61 | −19 | 3.82a | ||
R. fusiform gyrus | 26 | −60 | −20 | 3.69a |
Fear > neutral . | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Normals: 67 ms | ||||||
L. amygdala | −20 | 0 | −13 | 3.12a | ||
R. amygdala | 20 | −4 | −11 | 2.72b | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.13a | ||
Overachievers: 67 ms | ||||||
L. amygdala | −21 | −8 | −11 | 3.43a | ||
R. amygdala | 29 | −6 | −9 | 4.85a | ||
L. fusiform gyrus | −41 | −44 | −18 | 5.76a | ||
R. fusiform gyrus | 44 | −50 | −25 | 4.74a | ||
Overachievers: 33 ms | ||||||
L. amygdala | −16 | −5 | −5 | 3.14a | ||
L. fusiform gyrus | −33 | −61 | −19 | 3.82a | ||
R. fusiform gyrus | 26 | −60 | −20 | 3.69a |
Clusters significant in random-effects analysis at P < 0.05.
Clusters with a trend in random-effects analysis (P < 0.1).
Fear > neutral . | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Normals: 67 ms | ||||||
L. amygdala | −20 | 0 | −13 | 3.12a | ||
R. amygdala | 20 | −4 | −11 | 2.72b | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.13a | ||
Overachievers: 67 ms | ||||||
L. amygdala | −21 | −8 | −11 | 3.43a | ||
R. amygdala | 29 | −6 | −9 | 4.85a | ||
L. fusiform gyrus | −41 | −44 | −18 | 5.76a | ||
R. fusiform gyrus | 44 | −50 | −25 | 4.74a | ||
Overachievers: 33 ms | ||||||
L. amygdala | −16 | −5 | −5 | 3.14a | ||
L. fusiform gyrus | −33 | −61 | −19 | 3.82a | ||
R. fusiform gyrus | 26 | −60 | −20 | 3.69a |
Fear > neutral . | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Normals: 67 ms | ||||||
L. amygdala | −20 | 0 | −13 | 3.12a | ||
R. amygdala | 20 | −4 | −11 | 2.72b | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.13a | ||
Overachievers: 67 ms | ||||||
L. amygdala | −21 | −8 | −11 | 3.43a | ||
R. amygdala | 29 | −6 | −9 | 4.85a | ||
L. fusiform gyrus | −41 | −44 | −18 | 5.76a | ||
R. fusiform gyrus | 44 | −50 | −25 | 4.74a | ||
Overachievers: 33 ms | ||||||
L. amygdala | −16 | −5 | −5 | 3.14a | ||
L. fusiform gyrus | −33 | −61 | −19 | 3.82a | ||
R. fusiform gyrus | 26 | −60 | −20 | 3.69a |
Clusters significant in random-effects analysis at P < 0.05.
Clusters with a trend in random-effects analysis (P < 0.1).
. | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Hits > correct rejects | ||||||
L. amygdala | −17 | 1 | −9 | 2.87a | ||
R. amygdala | 18 | −4 | −7 | 3.23a | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.80a | ||
Hits > misses | ||||||
L. amygdala | −15 | −3 | −8 | 3.12a | ||
R. amygdala | 20 | −4 | −8 | 4.09a | ||
R. fusiform gyrus | 25 | −45 | −21 | 3.58a | ||
Hits > false alarms | ||||||
L. amygdala | −18 | −8 | −9 | 2.26a | ||
R. amygdala | 25 | −7 | −17 | 2.95a | ||
R. fusiform gyrus | 24 | −51 | −12 | 2.21a | ||
False alarms > misses | ||||||
R. amygdala | 18 | −8 | −7 | 4.12a | ||
R. fusiform gyrus | 43 | −53 | −25 | 4.23a |
. | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Hits > correct rejects | ||||||
L. amygdala | −17 | 1 | −9 | 2.87a | ||
R. amygdala | 18 | −4 | −7 | 3.23a | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.80a | ||
Hits > misses | ||||||
L. amygdala | −15 | −3 | −8 | 3.12a | ||
R. amygdala | 20 | −4 | −8 | 4.09a | ||
R. fusiform gyrus | 25 | −45 | −21 | 3.58a | ||
Hits > false alarms | ||||||
L. amygdala | −18 | −8 | −9 | 2.26a | ||
R. amygdala | 25 | −7 | −17 | 2.95a | ||
R. fusiform gyrus | 24 | −51 | −12 | 2.21a | ||
False alarms > misses | ||||||
R. amygdala | 18 | −8 | −7 | 4.12a | ||
R. fusiform gyrus | 43 | −53 | −25 | 4.23a |
Clusters significant in random-effects analysis at P < 0.05.
. | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Hits > correct rejects | ||||||
L. amygdala | −17 | 1 | −9 | 2.87a | ||
R. amygdala | 18 | −4 | −7 | 3.23a | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.80a | ||
Hits > misses | ||||||
L. amygdala | −15 | −3 | −8 | 3.12a | ||
R. amygdala | 20 | −4 | −8 | 4.09a | ||
R. fusiform gyrus | 25 | −45 | −21 | 3.58a | ||
Hits > false alarms | ||||||
L. amygdala | −18 | −8 | −9 | 2.26a | ||
R. amygdala | 25 | −7 | −17 | 2.95a | ||
R. fusiform gyrus | 24 | −51 | −12 | 2.21a | ||
False alarms > misses | ||||||
R. amygdala | 18 | −8 | −7 | 4.12a | ||
R. fusiform gyrus | 43 | −53 | −25 | 4.23a |
. | Talairach coordinates . | . | . | Z-score . | ||
---|---|---|---|---|---|---|
. | x . | y . | z . | . | ||
Hits > correct rejects | ||||||
L. amygdala | −17 | 1 | −9 | 2.87a | ||
R. amygdala | 18 | −4 | −7 | 3.23a | ||
R. fusiform gyrus | 26 | −44 | −19 | 3.80a | ||
Hits > misses | ||||||
L. amygdala | −15 | −3 | −8 | 3.12a | ||
R. amygdala | 20 | −4 | −8 | 4.09a | ||
R. fusiform gyrus | 25 | −45 | −21 | 3.58a | ||
Hits > false alarms | ||||||
L. amygdala | −18 | −8 | −9 | 2.26a | ||
R. amygdala | 25 | −7 | −17 | 2.95a | ||
R. fusiform gyrus | 24 | −51 | −12 | 2.21a | ||
False alarms > misses | ||||||
R. amygdala | 18 | −8 | −7 | 4.12a | ||
R. fusiform gyrus | 43 | −53 | −25 | 4.23a |
Clusters significant in random-effects analysis at P < 0.05.
The main goal of the present investigation was to test the hypothesis concerning the responses of the amygdala to briefly presented and masked fearful faces. Because of the importance of the fusiform gyrus in the processing of faces, we also investigated the responses evoked in this region. Responses evoked by very brief and masked stimuli presented in an event-related fashion are relatively weak. Thus, we initially determined the voxels with a significant task-related response, namely, a robust response to any stimulus type. Because, at the group level, these responses were very robust, we employed a threshold of 10−10 for significance. These significant voxels were then used to create a mask that constrained the search space for the specific contrasts outlined above. For these contrasts, we employed 0.05 as the threshold for significance.
To illustrate activation strength in the amygdala, we used parameter estimates from the fit to the data as a function of experimental condition. For the results shown in Figure 3, the peak coordinate from the contrast of masked fearful targets relative to neutral stimuli for both normal subjects and overachievers was used (see Table 1) to select parameter estimates in each individual. Similarly, for the results shown in Figure 5, the peak coordinate from the contrast of hits versus misses (see Table 2) was used.
Results
On each trial, a fearful, happy or neutral face stimulus was immediately followed by a neutral face that served as a mask. The subjects' task was to indicate whether they saw a fearful face or not, no matter how briefly. The duration of the first face stimulus was either 33 or 67 ms and the mask stimulus was shown for at least 83 ms. Subjects also rated the confidence in their response on a scale of 1–3, 1 corresponding to low confidence, 3 to high confidence (see Fig. 1 and Materials and Methods).
Behavioral Results
To characterize behavioral performance, for each subject, we determined the area under the ROC curve (called A′) for the 33 and 67 ms conditions. A total of 19/37 subjects reliably detected 67 ms target faces (A′ values significantly greater than 0.5), but could not detect 33 ms targets (A′ values not significantly greater than 0.5). Thus, these subjects behaved in a manner similar to what has been reported before, namely, they were aware of masked 67 ms targets but unaware of masked 33 ms targets (Fig. 2A). We also observed a group of 8/37 subjects who were able to reliably detect both 33 and 67 ms masked targets (Fig. 2B). We will refer to these subjects, who were aware of both 67 and 33 ms targets, as ‘overachievers’. Of the remaining 10 subjects, three were discarded from the fMRI analysis due to excessive head motion; two were discarded because although they did not meet the criterion for detecting 33 ms targets, they exhibited a strong trend toward detecting them; and five were discarded because they failed to reliably detect even 67 ms targets.
Eye Blinks
In a separate experimental session in the behavioral laboratory, we monitored eye blinks to test whether participants who were better at detecting fearful faces blinked less. A total of 8/17 subjects reliably detected only 67 ms targets (normals), while the remaining 9/17 detected both 33 and 67 ms targets (overachievers). We analyzed visual blinks within a 300 ms window starting 75 ms before the onset of the target face and terminating 75 ms after the offset of the mask face. Overall, subjects blinked very rarely during the 300 ms window (i.e. less than 3.5% of the trials for each subject). Critically, no significant difference in the mean number of blinks was observed between the two groups for the 33 ms condition (normals: 5.3 ± 4.1; overachievers: 3.8 ± 3.7; P > 0.4) or for the 67 ms condition (normals: 3.8 ± 3.3; overachievers: 4.8 ± 6.0; P > 0.6). We also regressed sensitivity to fearful faces (A′) on the number of blinks and found no linear relationship for either group (slope not significantly different from 0; in both cases, P > 0.5). These results show that sensitivity to fearful faces in our masking task was not explained by eye blinks. We also explicitly tested for a difference between eye blinks during hits and misses for both target durations, for both normals and overachievers. On average, during hit or miss trials, subjects blinked only once or less, and the only significant difference was observed for overachievers for 67 ms targets (hits: 1.2 ± 1.1; misses: 0.2 ± 0.4); however, in this case, a greater number of blinks was observed for hit trials (i.e. subjects did not blink more during miss trials).
Amygdala and Fusiform Responses Are Modulated by Awareness
Initially, we contrasted stimuli containing fearful targets (fear–neutral pairs) to those containing only neutral faces (neutral–neutral pairs) for the group of normal observers. As expected (Breiter et al., 1996), stronger responses were evoked in the amygdala for 67 ms masked fearful targets relative to neutral stimuli (Fig. 3A; parameter estimates are shown in Fig. 3C; see Table 1 for coordinates). Such differential activity was not observed, however, when normal subjects viewed stimuli containing 33 ms targets (Fig. 3B,C). As noted in the behavioral results, eight subjects were reliably able to detect both 67 and 33 ms masked fearful targets. If amygdala responses to fearful faces depend on awareness, differential activation would be expected for both stimulus parameters. Indeed, for this group of ‘overachiever’ subjects, the contrast of fear-containing stimuli and neutral-containing stimuli exhibited differential activation for both 67 and 33 ms (Fig. 3D,E; parameter estimates shown in Fig. 3F; Table 1. For 67 ms stimuli, differential activation in the right amygdala was stronger than that observed for normals, even though the group of overachievers contained only eight subjects, compared with 19 normal subjects (see also Fig. 3F). For 33 ms stimuli, differential activation for the overachievers was observed in the left amygdala only (see also Fig. 3F). Thus, in the amygdala, differential activations were observed only when subjects were aware of the fearful faces.
The fusiform gyrus (FG) plays an important role in the processing of faces (Puce et al., 1995; Kanwisher et al., 1997; Haxby et al., 2001). Therefore, we also investigated this region for effects of awareness and stimulus visibility (see below). Consistent with other reports, we observed differential activation in the FG when stimuli containing a fearful target (fearful–neutral pairs) were compared with neutral–neutral pairs (LaBar et al., 2003; Ishai et al., 2004). For the group of normal subjects, such activation was only evident for 67 ms masked targets, i.e. those that were reliably detected (Fig. 4A,B). At the same time, as in the amygdala, differential activation was observed for both 67 and 33 ms masked targets for the group of overachievers (Fig. 4C,D) who could detect fearful faces at both stimulus durations. Thus, the pattern of activity in the FG paralleled that found in the amygdala.
Amygdala and Fusiform Responses Are Modulated by Stimulus Visibility
Next, we investigated how the subjective report of seeing or not seeing a fearful target face modulated responses in the amygdala (Table 2). These analyses were performed for 67 ms trials, which provided a sufficient number of repetitions by trial type. Stronger responses were evoked during hits versus correct rejects (Fig. 5A), as expected, given that hit trials contained fearful target faces and correct reject trials contained only neutral faces (for parameter estimates associated with this and the following contrasts, see Fig. 5C,F). Critically, hit trials evoked stronger responses relative to miss trials (Fig. 5B). Since the two trial types are physically identical, and differ only in the subject's report, such differential activation must be related to the perception of a fearful target. Thus, target visibility modulated amygdala responses. Remarkably, false alarm trials evoked stronger responses relative to miss trials (Fig. 5D). Thus, reporting a fearful face when none was physically present produced stronger amygdala activation than viewing a stimulus that actually contained a fearful face of which the subject was unaware. Finally, we compared hit trials to false alarm trials (Fig. 5E). Little differential activity was observed in the left amygdala. Moreover, in the right amygdala, the peak of this contrast was situated more ventrally at z = −17 (7–10 mm inferior to the other activations; see Table 2). Thus, most of the left amygdala and portions of the right dorsal amygdala responded similarly to hit and false alarm trials, indicating that their responses were to a large extent linked to subjective report, such that even when no fearful face was physically present but was reportedly seen by the subject, robust responses were evoked.
Ambiguity has been shown to influence amygdala responses (Whalen, 1998). One concern, therefore, is that strong responses evoked during false alarm trials may reflect an increase in uncertainty during false alarm trials and not the subjective experience of reporting a fearful face. To address this question, mean confidence ratings during misses (2.14 ± 0.48) were compared with mean confidence ratings during false alarms (1.97 ± 0.52); these means were not significantly different (P > 0.1, t-test).
We also interrogated the FG for effects of stimulus visibility. The results were largely parallel to the ones observed in the amygdala (Fig. 6). As expected, hit trials (containing fearful faces) evoked stronger responses than correct reject trials (not containing fearful faces). At the same time, hit trials evoked stronger activation than miss trials, although the two conditions were matched in terms of physical stimulation, in that both contained fearful faces. False alarm trials (not containing fearful faces) evoked stronger responses than miss trials (containing fearful faces). Finally, hits and false alarms evoked similar responses in FG, as evidenced by the lack of differential activation in this region.
Discussion
In the present study we tested the hypothesis that amygdala responses to briefly flashed and masked faces are independent of visual awareness and stimulus visibility. We show, first, that amygdala responses can depend on visual awareness. Under conditions in which subjects were not aware of masked fearful faces flashed for 33 ms, no differential activation was observed in the amygdala. On the other hand, differential activation was observed for 67 ms masked fearful targets that the subjects could reliably detect. Critically, differential activation was observed for a group of ‘overachievers’ who were able to detect 33 ms masked targets. Note that this was the case even though the group of overachievers was much smaller than the group of normals (n = 8 versus n = 19). Second, when trials were divided into hits, misses, correct rejects and false alarms, we show that target visibility is an important factor in determining amygdala responses to fearful faces. Indeed, not only were responses during hit trials larger than those during physically identical miss trials; responses during false alarm trials were also larger than those during misses, although in the former no actual fearful face was present while in the latter it was. A similar pattern of results was also observed in the FG, where responses were also modulated by awareness and stimulus visibility.
Previous Masking Studies
The present results conflict with previous masking studies reporting that emotional stimuli evoke responses in the amygdala even when subjects are not aware of them. Using the backward masking paradigms developed by Ohman and colleagues and employed here, Whalen et al. (1998) reported that fMRI signals in the amygdala were significantly stronger during the viewing of masked, fearful faces than during the viewing of masked, happy faces. Awareness was assessed by debriefing subjects after the experiment and lack of awareness was suggested by the fact that subjects reported seeing only neutral faces. Because awareness was not formally assessed with SDT methods for each individual participant, it is unclear whether the stimuli were truly masked. Thus, one cannot be certain that the reported amygdala activations were obtained under conditions of unawareness. In another study, Morris et al. (1998) combined backward masking with classical conditioning to investigate responses to perceived (not masked) and non-perceived (masked) angry faces. Although the participants never reported seeing the masked, angry stimuli, the contrast of conditioned and non-conditioned masked, angry faces activated the right amygdala. During the experiment, subjects were asked to indicate when they perceived an angry face; 0% of the masked angry faces were detected and 100% of the unmasked faces were detected. During the experiment, angry faces appeared both in masked and unmasked conditions, and only two angry faces were used. Because detecting unmasked angry faces was rather straightforward, it seems likely that subjects never reported masked angry faces due to the very limited evidence indicating an angry face in this condition, relative to the unmasked case. Moreover, the task was not set up in a way that allowed for the assessment of sensitivity with SDT methods, as subjects reported angry faces that could occur either in a masked or unmasked fashion. Thus, it is again unclear whether the reported activations for masked faces truly reflected unaware perception. By contrast, in a recent masking study, no evidence for unaware activation in the amygdala was obtained (Phillips et al., 2004). Although these results are in concordance with the current study, because subjects passively viewed target–mask pairs, awareness was not assessed on an individual basis.
Awareness and Binocular Rivalry
How can the present results be reconciled with recent reports that the amygdala is activated in conditions of suppression during binocular rivalry? In one study, amygdala activation increased bilaterally in response to fearful versus neutral faces, regardless of whether the face was the dominant stimulus or suppressed during rivalry (Williams et al., 2004). Related results were also obtained by Pasley et al. (2004) when they contrasted suppressed fearful faces to suppressed chairs or houses.
Although binocular rivalry has been used to manipulate visual awareness, it is an empirical question whether the extent to which the state of suppression during rivalry is equivalent to lack of awareness in the context of backward masking (as assessed by SDT). In particular, if visual awareness is not an all-or-none phenomenon, the suppression state during rivalry could be associated with reduced or weakened awareness, while effective visual masking would be linked with a more complete elimination of awareness. In general, strong neural signals may be directly associated with visual awareness. For example, imaging studies of visuospatial neglect show that signals evoked by unseen faces are weak compared with those evoked by seen faces (Rees et al., 2000; Vuilleumier et al., 2001b). Moreover, activity in the fusiform gyrus correlates with the confidence with which a subject reports recognizing an object (Bar et al., 2001). Thus, the ‘degree’ of awareness may be correlated with the magnitude of activation.
It should also be noted that the boundary between suppression and dominance in binocular rivalry is subjective and that intermediate, ‘mixed’ perceptual states are often obtained. The extent to which the suppression state is ‘pure’ is especially difficult to quantify, given that a subject's response criterion is involved in indicating what stimulus is dominant at any given time.
Fusiform Gyrus
Previous studies have shown that FG responses are modulated by stimulus valence. Thus, viewing fearful faces or unpleasant scenes (Taylor et al., 2000; Mourão-Miranda et al., 2003) evokes stronger responses than viewing neutral faces or neutral scenes, respectively. The present results replicate such previous findings but also reveal that responses in the FG, like those in the amygdala, are modulated by awareness and stimulus visibility (see also Pessoa and Padmala, 2005). Previous studies have also addressed these questions, albeit in a more indirect fashion. In one study, activation was shown to be correlated with performance in an object naming task when exposure duration was varied from 20 to 500 ms (Grill-Spector et al., 2000). Such correlation was found in the lateral occipital complex, as well as the posterior FG. In a related study, activity in the FG was shown to be correlated with the confidence with which a subject reported recognizing an object (Bar et al., 2001). Our results are consistent with these previous findings. In our study, however, awareness was quantified objectively for each individual, allowing a direct assessment of its role in modulating FG activation.
Automaticity
A commonly held view states that emotional processing is automatic (Ohman, 2002; Dolan, 2003). Such a view is consistent with findings that amygdala responses are somewhat independent of the focus of attention. For instance, it has been reported that paying attention to houses does not have an effect on differential responses to fearful and neutral faces in the amygdala (Vuilleumier et al., 2001a; Anderson et al., 2003). Consistent with the view that emotional stimuli are processed ‘pre-attentively’, the detection of threat-related stimuli (e.g. a snake) is associated with relatively flat ‘search slopes’ in visual search tasks (Ohman et al., 2001). However, this view has recently been challenged. Both fMRI and event-related potential studies have recently demonstrated that emotional perception cannot proceed when a competing task is made sufficiently demanding, thereby depleting attentional resources (Pessoa et al., 2002a,b; Eimer et al., 2003; Holmes et al., 2003). Further evidence that the processing of emotional stimuli requires attention comes from a recent behavioral study showing that fearful faces are also subject to an ‘attentional blink’ (M. Eimer and R. Jones, in preparation).
As stated earlier, the hypothesis that amygdala responses are largely automatic would predict that visual awareness and stimulus visibility would have little impact on amygdala responses. On the other hand, if amygdala responses depend on awareness and are more closely tied to the perception of fear, then stimulus visibility would be expected to modulate responses. The present results show that differential activity in the amygdala strongly depends on awareness. Moreover, stimulus visibility was also found to play an important role in determining amygdala responses. Together, these results further challenge the view that amygdala responses occur automatically.
Assessing Awareness: Subjective and Objective Methods
What is the best way to characterize visual awareness? Much heated debate has surrounded this question (Merikle et al., 2001; Snodgrass et al., 2004). According to objective criteria, unaware perception occurs when a subject's performance in a forced-choice task is at chance level. Under such conditions, behavioral effects of unaware stimuli (e.g. faster reaction time for undetected fearful faces), as well as associated fMRI signals, constitute correlates of unaware perception. According to subjective criteria, unaware perception occurs when subjects themselves report that they are unable to perform the task better than chance (independent of their actual objective performance). At present, it is perhaps safest to assume that both methods will provide valuable information concerning awareness (Pessoa, 2005). On the one hand, studies that reveal amygdala responses during subjective unawareness indicate that although stimuli are task irrelevant and verbally inaccessible, they still evoke responses (Whalen et al., 1998). A central difficulty with subjective awareness thresholds is that they may be seriously contaminated by response bias (Eriksen, 1960). On the other hand, studies that reveal neural correlates during objective unawareness indicate that even though subjects cannot reliably detect or discriminate stimuli, these could still have an impact on brain responses (Wong et al., 2004). The present results do not support the notion of robust differential amygdala responses during objective unawareness.
Amygdala Responses and Target Visibility
Thus far, we have discussed our results as suggesting that (i) robust differential responses in the amygdala to fearful relative to neutral faces require visual awareness; and (ii) target visibility strongly modulates amygdala responses. An alternative, although not mutually exclusive, interpretation is that amygdala activity itself determines target visibility — rather than target visibility determining amygdala activity. In this view, subjects who exhibit amygdala responses in response to both 33 and 67 ms fearful faces are the ones who can reliably detect such briefly presented faces (i.e. are objectively aware of the faces). Thus, individual differences in amygdala response strengths would be linked to the ability to detect fearful faces in such challenging conditions.
The above interpretation is consistent with reports of so-called ‘affective blindsight’ (de Gelder et al., 2001). For example, blindsight patient GY (who has a right hemianopia following left occipital lobe damage) is able to discriminate emotional facial expressions presented in his blind hemifield (de Gelder et al., 1999, 2001). Moreover, in a recent fMRI study, amygdala responses were elicited in GY to the presentation of fearful and fear-conditioned faces in his blind hemifield (Morris et al., 2001), possibly suggesting that information reached his amygdala subcortically or via alternative cortical paths bypassing V1 (Sincich et al., 2004; Soares et al., 2004). Thus, in the context of the preceding discussion, blindsight patients may be able to detect emotional faces due to increased amygdala sensitivity to this class of stimuli. In a related fashion, recent studies suggest that variability in individual anxiety levels is an important variable determining the extent to which the amygdala responds to masked or unattended emotional stimuli (Bishop et al., 2004; Etkin et al., 2004); at a more basic level, increased amygdala reactivity has been shown to be linked to serotonin transporter genes (Hariri et al., 2002, 2005; Furmark et al., 2004; Heinz et al., 2005). However, not all variability in sensitivity to fearful faces appears to be linked to amygdala reactivity. For example, in a recent study, individual detection ability did not explain ventral amygdala responsivity to 17 ms fearful ‘eye-whites’ that were subsequently masked (Whalen et al., 2004).
Summary and Conclusions
Since the seminal work by Kunst-Wilson and Zajonc (1980) showing that ‘mere exposure’ can lead to preferences (i.e. unconscious processing), a great deal of work has investigated visual awareness. Unlike the vast majority of previous studies, our strategy was to characterize each individual's behavioral performance via SDT methods. This allowed us to separate participants into a group of ‘normal’ individuals who could reliably detect masked fearful faces presented for 67 ms but not for 33 ms, and a group of ‘overachievers’ who could detect both 33 and 67 ms masked targets. Our fMRI results show that differential responses in the amygdala to masked fearful faces relative to masked neutral ones was only observed when subjects were objectively aware of the faces. Thus, under unawareness conditions, no amygdala responses to fearful faces were observed. While our results speak directly to the debate of visual awareness of emotional faces, they have more general implications for the understanding of unconscious processing. Specifically, neuroimaging studies (of emotional processing or otherwise) that have utilized subjective awareness thresholds or forced-choice assessments vulnerable to response bias may have overestimated the extent of unconscious processing.
We thank Alumit Ishai for making available a set of emotional faces, Laura Crocker for assistance with figures and help with the eye-blink data, and Kara Lindstrom for help with scanning. This study was supported in part by the National Institute of Mental Health-Intramural Research Program. L.P. is supported in part by 1 R01 MH071589-01.
References
Anderson AK, Christoff K, Panitz D, De Rosa E., Gabrieli JD (
Bar M, Tootell RB, Schacter DL, Greve DN, Fischl B, Mendola JD, Rosen BR, Dale AM (
Birn RM, Cox RW, Bandettini PA (
Bishop SJ, Duncan J, Lawrence AD (
Breiter HC, Etcoff NL, Whalen PJ, Kennedy WA, Rauch SL, Buckner RL, Strauss MM, Hyman SE, Rosen BR (
Cox RW (
de Gelder B, Vroomen J, Pourtois G, Weiskrantz L (
de Gelder B, Pourtois G, van Raamsdonk M, Vroomen J, Weiskrantz L (
Eimer M, Holmes A, McGlone FP (
Ekman P, Friesen WV (
Eriksen CW (
Esteves F, Ohman A (
Etkin A, Klemenhagen KC, Dudman JT, Rogan MT, Hen R, Kandel ER, Hirsch J (
Friston KJ, Holmes AP, Worsley KJ, Poline J-P, Heather JD, Frackowiak RS (
Furmark T, Tillfors M, Garpenstrand H, Marteinsdottir I, Langstrom B, Oreland L, Fredrikson M (
Grill-Spector K, Kushnir T, Hendler T, Malach R (
Hanley JA, McNeil BJ (
Hariri AR, Mattay VS, Tessitore A, Kolachana B, Fera F, Goldman D, Egan MF, Weinberger DR (
Hariri AR, Drabant EM, Munoz KE, Kolachana BS, Mattay VS, Egan MF, Weinberger DR (
Haxby JV, Gobbini MI, Furey ML, Ishai A, Shouten JL, Pietrini P (
Heinz A, Braus DF, Smolka MN, Wrase J, Puls I, Hermann D, Klein S, Grusser SM, Flor H, Schumann G, Mann K, Buchel C (
Holmes A, Vuilleumier P, Eimer M (
Ishai A, Pessoa L, Bikle PC, Ungerleider LG (
Kanwisher N, McDermott J, Chun MM (
Kunst-Wilson WR, Zajonc RB (
LaBar KS, Crupain MJ, Voyvodic JT, McCarthy G (
Macmillan NA, Creelman CD (
Merikle PM, Smilek D, Eastwood JD (
Morris JS, Ohman A, Dolan RJ (
Morris JS, DeGelder B, Weiskrantz L, Dolan RJ (
Mourão-Miranda J, Volchan E, Moll J, de Oliveira-Souza R, Oliveira L, Bramati I, Gattass R, Pessoa L (
Ohman A (
Ohman A, Esteves F, Soares JJF (
Ohman A, Flykt A, Esteves F (
Pasley BN, Mayes LC, Schultz RT (
Pessoa L (
Pessoa L, Padmala S (
Pessoa L, Kastner S, Ungerleider LG (
Pessoa L, McKenna M, Gutierrez E, Ungerleider LG (
Pessoa L, Japee S, Ungerleider LG (
Phillips ML, Williams LM, Heining M, Herba CM, Russell T, Andrew C, Bullmore ET, Brammer MJ, Williams SC, Morgan M, Young AW, Gray JA (
Puce A, Allison T, Gore JC, Mccarthy G (
Rees G, Wojciulik E, Clarke K, Husain M, Frith C, Driver J (
Sincich LC, Park KF, Wohlgemuth MJ, Horton JC (
Snodgrass M (
Snodgrass M, Bernat E, Shevrin H (
Soares JG, Diogo AC, Fiorani M, Souza AP, Gattass R (
Talairach J, Tournoux P (
Taylor SF, Liberzon I, Koeppe RA (
Vuilleumier P, Armony JL, Driver J, Dolan RJ (
Vuilleumier P, Sagiv N, Hazeltine E, Poldrack RA, Swick D, Rafal RD, Galbrieli JD (
Whalen PJ (
Whalen PJ, Rauch SL, Etcoff NL, McInerney SC, Lee MB, Jenike MA (
Whalen PJ, Kagan J, Cook RG, Davis FC, Kim H, Polis S, McLaren DG, Somerville LH, McLean AA, Maxwell JS, Johnstone T (
Williams MA, Morris AP, McGlone F, Abbott DF, Mattingley JB (
Author notes
1Department of Psychology, Brown University, Providence, RI, USA and 2Laboratory of Brain and Cognition, NIMH, Bethesda, MD, USA