- Email: [email protected]
Individual differences in adaptive choice strategies
Research in Economics 57 (2003) 219–233 www.elsevier.com/locate/yreec
Individual differences in adaptive choice strategies Barbara Fasoloa,*, Raffaella Misuracab, Gary H. McClellandc a
Max-Planck Institute of Human Development, Lentzeallee 94 Berlin 14195, Germany b Universite´ de Bourgogne, France c University of Colorado at Boulder, USA
Abstract Individual differences in compensatory and non-compensatory choice processes remain an unresolved issue for decision process researchers. This study investigates the stability and nature of individual differences in choice processes when individuals adapt to changes in the structure of the choice environment, namely the correlation among the choice attributes. By means of process tracing techniques, between-subjects differences in choice processing (option-based or attribute based) were found to be stable across different tasks. Individuals with higher openness to experience and ability to solve reasoning tasks were found to be more adaptive, that is to switch more promptly their choice process in adaptive ways, by using more option-based search strategies when attributes were negatively related. These results suggest that insight into individual differences in choice processes can be gained when attention is given to the structure of the choice task and to how decision makers adapt to it in the course of the choice task. q 2003 University of Venice. Published by Elsevier Ltd. All rights reserved. Keywords: Individual differences; Adaptive strategies; Decision processes; Open-mindedness; Compensation strategies; Non-compensatory strategies; Interattribute correlations; Choice environment
1. Individual differences in adaptive choice strategies One of the unresolved issues in decision process research is the inability to find stable individual differences in choice process, despite striking between-subjects differences in search pattern that are typically found in the data of any process tracing experiment (see Ford et al., 1989, for a comprehensive review). Some people choose via a distinct compensatory process, others via a distinct non-compensatory process, yet predicting what
* Corresponding author. Tel.: þ 49-30-82406-339; fax: þ49-30-82406-394. E-mail address: [email protected] (B. Fasolo). 1090-9443/03/$ - see front matter q 2003 University of Venice. Published by Elsevier Ltd. All rights reserved. doi:10.1016/S1090-9443(03)00032-2
220
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
makes an individual follow a compensatory process or non-compensatory process is still a difficult research problem (Capon and Davis, 1984; Payne et al., 1993; Broeder, 2003). The difference between the two processes lies in the order in which information is searched, and the extent to which the search is thorough and systematic. For instance, faced with the decision of which digital camera to buy, a compensatory decision maker would consider one camera at a time, and examine one-by-one all the available attributes (zoom, resolution, etc.), to form an overall evaluation of how good all cameras are. This process is compensatory because bad attribute values can be compensated for by good values on other attributes. Faced with the same decision, a non-compensatory decision maker would consider one or two attributes at a time, for all or some of the cameras. Whereas the compensatory decision maker mainly focuses on options (option-based process), the noncompensatory decision maker mainly focuses on attributes (attribute-based process). This distinction has implications for the quality of the decision reached: When the attributes that describe the options are negatively correlated (i.e. there is no option available that is best on all attributes, and trade-offs need to be made) and decision quality is gauged against the economic principle of expected utility maximization, a compensatory process leads to higher quality decisions than a non-compensatory process (Johnson and Meyer, 1984; Widing and Talarzyk, 1993). This difference in extent to which the two processes adhere to normative principles motivated the first researchers to search for individual differences in factors like cognitive ability (Capon and Davis, 1984), prior knowledge (Payne et al., 1993), and experience with the choice (Bettman and Park, 1980)—to no avail. More successful were two recent empirical investigations. The first, by Levin et al. (2000), found that participants’ choice processes were reliably related to the personality variable of Need for Cognition (NFC, Cacioppo and Petty, 1982), a measure of cognitive style widely used in studies of persuasion and attitude change. In the study of Levin et al. (2000) participants chose a computer for a friend in two phases: first participants winnowed options from 16 to a smaller number, then they explored the few remaining options. Levin et al. found that on average participants used a less effortful attribute-based process in the first stage, and a more effortful option-based process in the second stage. This shift from attribute-based to optionbased process when option set size decreases is ‘adaptive’, because it is an efficient response to a choice problem that becomes more manageable, and is consistent with the view of decision makers as ‘adaptive’ to the choice at hand (Payne et al., 1993; Gigerenzer et al., 1999). Traditional information-processing measures (e.g. option-based or attribute-based search pattern and amount of information searched) did not show a reliable and general relationship with NFC, confirming the difficulty of capturing individual differences in the process used along the whole decision process (at both stages). The interesting result was that individual differences were found in the adaptive shift in choice process. Individuals with higher NFC switched to option-based processes in the second stage more than those with lower NFC. The second interesting development was by Broeder (2003). He found a correlation between intelligence and use of the non-compensatory strategy Take-The-Best (Gigerenzer and Goldstein, 1996) in environments where this strategy had the highest expected payoff. Intelligence was found to correlate with the ability to recognize
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
221
quickly the payoff structure of the task and consequently to adapt toward the strategy that had the highest payoff, when the environment was favoring a non-compensatory strategy. What these two investigations have in common is a joint consideration of person characteristics (intelligence, NFC) and choice environment characteristics (payoff structure, option set size), instead of a context-free exploration of individual characteristics. We adopt here the same perspective, with a special focus on individual differences in responses to changes in the choice environment, namely in correlation among choice attributes. In a prior investigation (Fasolo et al., 2001) we found that participants choosing among digital cameras displayed on average an attribute-based process. Although betweensubjects differences were large, participants’ choice process was steadily influenced by changes in inter-attribute correlation. When choice attributes were conflicting (there is negative correlation among them), decision makers adapted and switched toward less attribute-based search (or, more option-based), and the choice was perceived as more difficult and dissatisfactory than when attributes were positively related. This study uses the same process-tracing procedures as Fasolo et al. (2001) to investigate individual differences both in the choice process used throughout the task, regardless of correlation changes, and in the switch in choice process that occurs in response to negative correlations among attributes. Besides capturing choice process with process-tracing techniques, preferred choice style will also be elicited by means of self-reports. Self-reported choice style, as measured by a Compensatory Style (CS) questionnaire (Zakay, 1990), was found to correlate with actual choice outcomes by Shiloh et al. (2001). Interestingly, the CS questionnaire scores were reliably related to open-mindedness, such that more open-minded individuals reported to be more compensatory. Open-mindedness is the first person characteristic of interest in our study. Besides its relationship to self-reported compensatory choice style, other appealing reasons for choosing the open-mindedness construct are: (1) open-mindedness has both an emotional dimension— openness to experience—and a cognitive dimension—openness to culture (McCrae and Costa, 1997). The emotional dimension is important because the behavior of interest entails the emotional as well as the computational difficulty of facing trade-offs when attributes were negatively related. The cognitive dimension—reliably related to NFC scores (see McCrae, 2000, table 12.2)—is important because NFC was found to be a good predictor of switches in decision strategies (Levin et al., 2000). The second person characteristic of interest is reasoning ability, measured by performance on a battery of reasoning tasks. Decision researchers have shown that option-based and compensatory processes are more effortful than attribute-based ones (Payne et al., 1993), so one hypothesis is that individuals with higher reasoning abilities are more likely to engage in compensatory processing—meaning that they display a more option-based process, search more information and report being more compensatory. An alternative hypothesis—in line with Broeder (2003)—is that individuals with higher reasoning ability would be more likely to detect the need to shift strategy when needed, and therefore display a more pronounced shift toward an option-based process when confronted with negative correlations. The relationship
222
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
between negative correlations and higher cognitive ability is also in line with research showing that learning positive correlations is well within working memory capacity limits, whereas learning negative correlations is significantly more taxing (Chasseigne et al., 1997). In sum, our experiment aims to determine whether the way people search information for choosing in response to changes in the correlations among attributes is related to openmindedness, reasoning ability and self-reported choice style. Furthermore, our choice experiment considers relationships between these individual differences factors and perceptions of confidence, difficulty and satisfaction with the choice made.
2. Experiment overview The choice task and manipulation were adapted from Fasolo et al. (2001). Participants made choices among five different digital cameras, each described in terms of eight different attributes, described in detail below. Spontaneous individual strategies were of interest, so no instruction was given on how attributes should be weighted. The complexity of the choice task was varied by manipulating the average correlation among attributes. Participants’ choice processes were measured by means of a web-based process tracing technique, and their confidence, satisfaction and difficulty were recorded after each choice on a web-based questionnaire. To measure individual differences in participants’ choice process, an original questionnaire was developed (Questionnaire of Individual Differences, or QUIND), comprising three separate sections. The first section included questions about the participants’ openness to experience and to culture. Overall more open-minded decision makers were expected to be more compensatory in their self-reported style and to be more ‘adaptive’ in their traced search process, that is to display larger shifts toward option-based processing in response to negative correlations. The second section presented a battery of reasoning tasks. Reasoning capacity was operationalized as performance on the Wason Selection Task (Wason, 1966) and on syllogisms with different degrees of difficulty (Johnson-Laird, 1983). In their studies of individual differences in rational thought, Stanovich and West (1998) found that syllogisms and Wason selection tasks were more highly correlated with reasoning capacity (as measured by the Scholastic Assessment Test). If the ability to adapt to changes in the choice environment is positively related to cognitive capacity, individuals with higher reasoning ability should display larger shifts in choice process in response to changes in correlation. If higher reasoning ability is related to more compensatory behavior, regardless of the structure of the task, then individuals with higher reasoning ability should always have a more compensatory process (more option-based and more information sought) than those with lower reasoning ability. The third section presented questions about the participants’ preferred choice style, similar to the CS questionnaire of Zakay (1990). Based on the findings of Zakay (1990) and Shiloh et al. (2001), we expected self-reported choice style to be correlated with traced choice process.
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
223
3. Method 3.1. Participants One hundred twenty-three students enrolled in the course of General Psychology at the University of Colorado participated in the experiment and received course credit in exchange for their participation. 3.2. Materials and apparatus Before the choice task, a web-based questionnaire was presented. The questionnaire was composed of four parts. “About me”. Twenty-six Likert items, inspired by the Neo-Personality Inventory (NEOPI, Costa and McCrae, 1985) and by the Big Five Questionnaire (Caprara et al., 1993), measured participants’ open-mindedness. Specifically, 13 items measured openness to culture (e.g. “I usually read”) and 13 measured openness to experience (e.g. “In choosing a destination for my travels I always tend to prefer exotic places different from those, where I live”). For each item, participants indicated the degree to which the content matched their opinions and behaviors by clicking the appropriate number on a scale from 1 (“the content of the statement does not match my opinions and behaviors”) to 5 (“the content of the statement matches completely my opinions and behaviors”). The final score was the sum of the individual item scores. The higher the score, the higher the open-mindedness. “About my reasoning”. Four syllogisms. According to Johnson-Laird’s (1983) theory of mental models, two of these syllogisms were easy (their solution needed the construction of only one mental model), and the other two were difficult (their solution needed the construction of three mental models). Each syllogism was composed of a pair of premises followed by four possible conclusions, only one of which was valid. Participants selected the one response believed to follow logically from the first two premises. A score of 1 was given for each correct response; a score of 0 for each incorrect response. The final score was the sum of the individual item scores. The higher the score, the higher the syllogistic reasoning abilities. “How I check violations of rules”. Two original conditional tasks inspired by the Wason Selection Tasks and implemented in a web-based questionnaire format. Consistent with the most recent literature on the topic (Cosmides, 1989; Cardaci et al., 2002), the traditional paradigm of four cards was integrated with a brief situational frame. Each task was composed of a conditional rule having the general form if p then q, followed by four cases, respectively, corresponding to the formal cases p, non-p, q, non-q. For each case, participants were asked if they needed more information to determine whether the rule presented was violated. The normative correct responses corresponded to the request of further information to determine whether the rule was violated in the cases, where p and non-q were given. One point was assigned for each correct response; no points for each incorrect response. The final score was the sum of the individual item scores. The higher the score, the higher the conditional reasoning abilities. “About my choice style”. Four items describing self-reported choice style, adapted from the CS questionnaire of Zakay (1990). Each task described a hypothetical choice
224
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
situation, followed by two possible responses: one was consistent with a compensatory decision strategy; the other with a non-compensatory decision strategy. Participants were asked to select the response that better described how they would normally decide. One point was given for each compensatory response; no point for each non-compensatory response. The final score was the sum of the individual item scores. The higher the score, the higher the tendency to use a compensatory strategy. According to ergonomic criteria of on-line research (Cardaci, 2001), all items were designed to allow for easy reading and immediate comprehension. For each item, participants gave a point-and-click response. Each selection excluded all others, avoiding the disadvantages due to compilation errors of paper-and-pencil questionnaire. Data were automatically stored in a web-server database. Following the questionnaire, participants were presented with the choice task. Four different sets of five digital cameras were presented on a web-based comparison table, where options appeared along the columns and attributes along the rows. Each digital camera was described along the same eight attributes. Emulating what is commonly available on web-based comparison shopping sites, clicking on the attribute name revealed a detailed description of each attribute. For each attribute (‘delay between shots’), participants were provided with (a) a short and comprehensible description (‘The amount of time, measured in seconds, it takes the camera to process and store an image when shooting in normal mode (non-burst), at the camera’s maximum resolution setting. This feature is also known as lag time’); (b) the direction of preference (‘the less delay, the better’); (c) the range of the attribute (‘1 – 20 s’); and (d) the levels of the attributes (‘1– 2 – 5 –7 –20 s’). The eight attributes were: Delay Between Shots, Image Capacity, LCD Display, Lighting, Optical Zoom, Resolution, Size and Weight. To keep track of the pieces of information searched and their exact order, the content of each cell was only revealed when the participant pointed the mouse on the cell. The cell was closed again before the content of the next cell was revealed. The information acquisition process was recorded with WebIDB (McClelland, 2002), an original WWWbased version of the matrix schema originally devised for the MouseLab program to trace decision processes on a computer monitor (Johnson et al., 1991). WebIDB is a Java applet suitable for use with any Java-capable web browser. After each choice, a short web-based questionnaire asked participants to rate their satisfaction with and confidence in the choice just made. The entire experiment (accessible at: http://psych.colorado.edu/~bfasolo/idb/baraga) was conducted on Macintosh computers using Internet Explorer version 5.1. 3.3. Design Inter-attribute correlation was varied between positive and negative in an asymmetric between– within design. To construct tables with a predetermined average inter-attribute correlation, a Mathematica routine was used that generates random observations from a multivariate normal distribution with a specified variance –covariance structure. Values for the random observations were rounded and rescaled to match the appropriate range and number of attribute levels. ‘Positive’ tables were drawn from a population with average pair-wise attribute correlation of þ 0.5. ‘Negative’ tables were sampled from a population
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
225
with the average negative inter-attribute correlation of 2 0.14, the most extreme negative correlation possible for eight attributes [21=ðk 2 1Þ; with k ¼ 8]. Every participant in each condition saw the choice matrices in the same order. Fifty-six participants made choices from four sets of cameras with negative correlations (‘Neg – Neg’ condition), and 67 participants first made choices from two sets with positive correlations and then two sets with negative correlations (‘Pos –Neg’ condition). Four dependent measures will be reported: (a) search pattern, by option or by attribute; (b) within-subject consistency in search pattern; (c) self-reported measures of satisfaction, confidence and decision difficulty; (d) responses to the four sections of the individual questionnaire: open-mindedness scores, syllogistic reasoning scores, Wason Selection Task scores, and self-reported choice style. The following is a more detailed description of measures of (a) and (b). (a) Search pattern by option or by attribute, was measured as the Payne Index (PI, e.g. Payne et al., 1993)—the most common measure of information search pattern available in the literature. This index combines the number of times a participant looks consecutively at information about the same option across attributes (Option Salient transition) with the number of times a participant looks consecutively at information about the same attribute across options (Attribute Salient transition). Reinspections to the same cell are included in the computations, whereas diagonal moves are excluded, because they are not consistent with either a prevailing optionbased or attribute-based process. The PI is determined by the number of Option Salient transitions minus the number of Attribute Salient transitions, divided by their sum. It ranges from 2 1 to þ 1, where positive values signify a predominately optionbased search process (implying a compensatory process) and negative values signify a predominantly attribute-based search pattern (implying a non-compensatory process). (b) Within-subject consistency was summarized by Cronbach’s alpha computed over the four Payne Indices obtained from each participant. To compute alpha, a Principal Component Analysis was performed as recommended by Judd and McClelland (1998, p. 204). The resulting eigenvalue (sum of the squared factor loadings on the first unrotated principal component, estimating the correlation between each PI and the underlying choice processing style) was then used to compute the coefficient alpha {½k=ðk-1Þ* ð1-ð1=lÞÞ; where k; is the number of items, and l; is eigenvalue}. Coefficients larger than 0.80 are reported by social psychologists as indicating high within-subject consistency. This formula has the advantage of not assuming equal variance across items, and yields a convenient and meaningful coefficient for summarizing the within-subject consistency in Payne Indices. 3.4. Procedure Participants were tested individually or in small groups and sat in front of computers connected to the experimental website. First, participants were introduced to the webbased QUIND questionnaire, and were told that they would be presented with some questions concerning their reasoning and their daily choice behaviors. It was emphasized
226
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
that they needed to think about the best description of their natural choice behavior and reasoning, as opposed to the absolutely ‘best’ answer. Next, they completed the questionnaire. Instructions for the choice task were displayed on the website. They were told to imagine having to spend a gift certificate on a digital camera of their choice among those presented. Cameras would be displayed in four successive tables. Each table would display five cameras, described on eight features. Their task was to choose from each table the camera that was best for them. Following the instruction screen, participants were presented with a hypothetical choice problem among computers, so that they could practice using the WebIDB information acquisition system on the comparison table. The experimenter also reminded participants that they could take as much time as they wished to acquire information before making their choice and that the experiment would conclude with a short web-based questionnaire. 3.5. Analyses notes Analyses were conducted on the participants’ choices made, search logs and responses to the QUIND questionnaire and to questions about satisfaction, confidence, and difficulty after each choice made. For each section of the QUIND questionnaire (open-mindedness, syllogisms, Wason Selection Task, and Choice Style) a Cronbach’s alpha was computed. Unless specified otherwise in the results section, the analyses reported in this experiment are from a two-way between– within ANOVA conducted as a series of planned contrasts for main effects, interactions, and simple effects. This use of planned contrasts is recommended by Rosenthal and Rosnow (1985) and Judd and McClelland (1989), among others. By so doing, results can be presented in terms of substantive comparisons instead of in terms of statistical abstractions. More importantly, in the case of analyses of withinparticipant variables, one-degree-of-freedom contrasts avoid the troublesome and seldom satisfied sphericity assumption required in the usual repeated-measures ANOVA (see Chapter 14 of Judd and McClelland (1989)). Participants’ search logs were first screened for ‘pass-throughs’ in the following way. In addition to tracking cells visited, WebIDB also records the amount of time spent in each cell. These individual reaction times (RT) per cell were plotted on a histogram, and analyzed for each individual participant. RTs would typically have a bi-modal distribution with a clear demarcation between the two modes. This demarcation occurred at a time varying from participant to participant between 250 and 450 ms after they first moved the mouse on the table. For all participants this demarcation occurred too soon to accompany a deliberative search strategy. Therefore, the transitions associated with the first distribution (below the demarcation time) were considered mere ‘pass-throughs’, and were eliminated from further analysis. The transitions associated with the second distribution (above the demarcation time) were considered deliberative, and composed the search logs ultimately analyzed. Search logs were processed for each participant to yield a measure of total Option-Salient and Attribute-Salient transitions, and were then combined into a PI for each table.
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
227
Fig. 1. Payne Index as a function of correlation and table order.
4. Results 4.1. Search pattern Search pattern was indexed using the PI. As Fig. 1 shows, search was, on average more by attribute than by option (PI ¼ 20:262; tð122Þ ¼ 28:78; p , 0:001), showing that the participants’ dominant search behavior was to compare different options along a given attribute before moving to the next attribute. The correlation manipulation significantly affected the search strategy in the expected direction: negative correlations increased the focus on options both when correlation was manipulated within (comparing the search pattern on the first two and latter two tables for the Pos– Neg group, Fð1; 66Þ ¼ 22:36; PRE1 ¼ 0.25, p , 0:001) and between (comparing the search pattern on the first two tables for the Pos –Neg group and Neg – Neg group, Fð1; 121Þ ¼ 6:40; PRE ¼ 0:05; p , 0:02). The search pattern also changed over time. Over trials, the search became more option based for both groups (Fð1; 121Þ ¼ 27:25; PRE ¼ 0:19; p , 0:001), but the increase toward more option-based strategies when correlation switched from positive to negative from the second to third choice tended to be more pronounced in the Pos– Neg group than in the Neg – Neg group (Fð1; 121Þ ¼ 3:42; PRE ¼ 0:03; p ¼ 0:06). 4.2. Within subject consistency in search pattern Although the search pattern was on average attribute-based across the four choice tasks, it ranged from a minimum of 2 0.87 to a maximum of 0.92 for individual participants. The data exhibited the same degree of variability observed in Fasolo et al. (2001). Table 1 shows an example, representing the search pattern (operationalized as PI) across the four comparison tables for three participants in the Neg –Neg condition. A principal component analysis was performed on the four Payne Indices of all participants in the Neg – Neg 1 PRE is the proportional reduction of error between a model of the data that makes the given distinction and one that does not (Judd and McClelland, 1989). It is a standardized effect size measure.
228
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
Table 1 Payne Index across the four tables for three participants in the condition Neg– Neg
Participant 1 Participant 2 Participant 24
PI1
PI2
PI3
PI4
20.67 20.051 0.02
20.87 20.56 0.24
20.31 20.050 0.17
0.06 20.45 0.41
condition. The first factor had an eigenvalue equal to 2.73. The coefficient alpha was 0.85, suggesting the existence of a highly reliable individual difference in search pattern. Participants who started to search by attribute (as participants 1 and 2) continued to do so all along the task, and the same consistency characterized participants who started to search by option (as participant 24). 4.3. Self-reported measures of satisfaction, confidence, and decision difficulty As Fig. 2 shows, participants in the Pos –Neg group were more satisfied with their first two choices (made from tables with positive correlations) than participants in the Neg – Neg group, who first chose from two tables with negative correlations (M ¼ 4:35 vs. 3.99, Fð1; 121Þ ¼ 30:25; p , 0:001; PRE ¼ 0.20). Negative correlations also affected perceptions within-subjects, extending previous findings of Fasolo et al. (2001). Participants from the Pos– Neg group reported being more satisfied with the first two choices (with positive correlations) than with the last two choices (with negative correlations). Participants felt particularly dissatisfied when they experienced negative correlations after having experienced positive correlations (M ¼ 4:35 vs. 3.59, Fð1; 66Þ ¼ 106:69; p , 0:001; PRE ¼ 0.62). The same results apply for the confidence ratings, with correlation affecting confidence both when manipulated between (M ¼ 4:14 vs. 3.71, Fð1; 121Þ ¼ 19:27; p , 0:001; PRE ¼ 0.14), and within (M ¼ 4:14 vs. 3.52, Fð1; 66Þ ¼ 63:92; p , 0:001; PRE ¼ 0.49). As Fig. 3 shows, participants in the Pos –Neg group found the choice task easier on the first two trials than those in the Neg – Neg group (M ¼ 2:7 vs. 1.8, Fð1; 121Þ ¼ 36:12; p , 0:001; PRE ¼ 0.23) and than on the second two negative trials (M ¼ 2:7 vs. 2.1, Fð1; 66Þ ¼ 18:49; p , 0:001; PRE ¼ 0.22). Therefore, negative correlation always made it more difficult for participants to make choices, regardless of whether correlation was manipulated within or between. 4.4. Responses to the questionnaire of individual differences Table 2 summarizes the data, reporting the correlations between individual differences factors and choice process (adaptive shifts in choice process as well as stable choice process). The following is a detailed account of the results, for each section of the questionnaire. Open-mindedness. Participants’ open-mindedness scores were divided into openness to experience and openness to culture. The openness to culture items revealed higher reliability (coefficient alpha was 0.84) than the openness to experience items (coefficient alpha was 0.67). Despite the low level of alpha, the experiment had enough power to detect
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
229
Fig. 2. Mean satisfaction rating as a function of the correlation experienced in the first two and last two tables.
a significant correlation between openness to experience and strategy shift. As expected, participants’ (in the Pos– Neg group) shifting towards option-based strategies when correlations changed from positive to negative was stronger if they exhibited higher openness to experience [rð65Þ ¼ 0:27; p , 0:05]. Moreover, participants with higher open-mindedness scores (both culture and experience) tended to report higher satisfaction across all four choices [rð122Þ ¼ 0:16; p ¼ 0:087]. Performance on Wason Selection Task (WST). Participants’ performance on the WST was higher than in the original version of the task (10% in Wason, 1966) and in accord with the performance reported in the wide number of studies using facilitated versions of the task (Cosmides, 1989). This indicates that the particular web-based format used had a facilitating effect. Correct responses to the WST provided sufficient variability in response scores and were reliably related across the two tasks (a ¼ 0.765), indicating that this task
Fig. 3. Mean rating of ‘choice easy’ as a function of the correlation experienced in the first two and last two tables.
230
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
Table 2 Individual differences and choice process (correlation table) Correlates of
Adaptive shift (within-subject differences in Payne Index)
Stable choice process (average Payne Index)
Openness to experience Openness to culture Reasoning (wason) Reasoning (syllogisms)
0.27 (*) 0.12 (n.s.) 0.27 (*) 0.14 (n.s)
0.031 (n.s.) 0.035 (n.s.) 20.095 (n.s.) 20.05 (n.s.)
*Significant at p , 0:05:
was an appropriate indicator of a common reasoning capacity. Higher WST performance significantly correlated with larger amount of information searched on comparison tables [rð120Þ ¼ 0:38; p , 0:0001], usually associated with more compensatory search processes. Participants with higher WST scores did not exhibit a more option based process along the whole experiment. Rather they shifted processing to a greater extent, by using a more option-based search with negative correlations than with positive correlations, more so than participants with lower WST scores [rð120Þ ¼ 0:27; p , 0:001] did. Syllogistic reasoning performance. Participants’ performances on the four Syllogistic Reasoning Tasks were high on the first two ‘easy’ syllogisms (because they required one mental model for their solutions), and low on the second two ‘difficult’ syllogisms (because they required three mental models for their solutions). On average, this differential performance was in accord with the literature (Johnson-Laird, 1983). Correct responses to the four syllogisms were not reliably related (a ¼ 0.381) and were less variable overall than the responses to the WST (across syllogistic reasoning task, average SD ¼ 0.34; across Wason Selection Task, average SD ¼ 0.64). Both might be possible reasons why no reliable association was found between measures of decision processing and performance in the syllogistic task. Self-reported choice style. Participants reported, on average, having a compensatory choice style (M ¼ 4:06; SD ¼ 0.89) consistent with results obtained previously by Shiloh et al. (2001). Answers to the five Choice Style items were not reliably related with one another (a ¼ 0.324), so no meaningful correlations can be found with the traced choice process.
5. Discussion The goal of this experiment was to determine if choice processing is a stable individual difference, and if adaptive shifts in choice process have predictable person correlates. Choice processing, as measured by the PI, is indeed a reliable individual difference, indicating that participants have a preferred (option-based or attribute-based) processing style measurable from their actual behavior. Participants’ traced choice process was on average consistent with a non-compensatory strategy (mainly attribute-based) whereas
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
231
their reported choice style was compensatory, perhaps related to the social desirability of describing oneself as a rational information seeker. When the average choice process is considered for each person, evidence for individual difference is very weak, confirming the difficulty already noted in the literature. It must be emphasized that these weak effects are not likely due to a lack of power because correlations were found even among constructs that had low levels of reliability (e.g. openness to experience). The data are more promising when the behavior analyzed for each individual is not the average choice process, but rather the shifts in choice process in response to changes in the correlation structure. When shifts in choice process are considered, significant individual differences are found, along with correlations to both openness to experience and reasoning ability. First, participants with higher openness to experience switch toward option-based strategies to a greater extent when correlations become more negative. This result extends data from Shiloh et al. (2001) and Levin et al. (2000). Second, participants with higher reasoning ability also move toward option-based strategies to a greater extent when correlations become more negative. This result is consistent with Broeder (2003), providing further evidence that reasoning ability is more related to the ability to detect when changes of search strategy are necessary, rather than to the unilateral adoption of the most effortful strategy. The effect of open-mindedness and reasoning capacity was found when participants faced choice tasks that encouraged them to switch strategies. Individual differences pertained to the extent to which people were ‘adaptive’, or switched strategies: participants with a more open mind and better at reasoning are also better able to adapt to the choice task as demanded. These findings highlight an important methodological issue: when studying the effect of person characteristics (here, open-mindedness and reasoning ability) on choice process, properties of the choice environment (here, correlation among attributes) are as important as characteristics of the individuals being studied, in line with the ‘organism-environment model’ advocated as early as 1939 by Brunswik (1939).
6. Conclusions This study confirms that inter-attribute correlation is a powerful task characteristic that affects search processes and perception: All participants, regardless of different preferences over attributes, dispositions toward thinking, and stated choice styles were able to adapt to it, although they disliked doing so, in line with prior research (Fasolo et al., 2001). Choice processing emerged as a reliable individual difference. Decision makers exhibit a preferred choice strategy—mainly option-based or mainly attribute-based—which consistently guides their search process. When considered independently of changes in the choice environment, choice process had—at best—weak person correlates. When changes in the environment were considered, responses to them (switches toward more optionbased process when correlations were negative) showed a small but significant relationship with open-mindedness and reasoning ability.
232
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
Methodologically, these results show that although person correlates of a stable choice process are difficult to grasp, it is possible to capture individual differences in the way decision makers switch process in response to task complexity, like the inter-attribute correlation in our experiment. Rather than the traditional distinction between ‘compensatory’ and ‘non-compensatory’ processes, we suggest that it would be more useful to distinguish between ‘more adaptive’ and ‘less adaptive’ decision makers. Our study, along with Levin et al. (2000) and Broeder (2003), made a first move in this direction, showing that an open mind and good reasoning ability are two good predictors of more adaptive decision makers.
References Bettman, J.R., Park, C.W., 1980. Effects of prior knowledge and experience and phase of the choice process on consumer decision processes: a protocol analysis. Journal of Consumer Research 7, 234– 248. Broeder, A., 2003. Decision making with the adaptive toolbox: influence of environmental structure, intelligence and working memory load. Journal of Experimental Psychology: Learning, Memory and Cognition in press. Brunswik, E., 1939. The conceptual focus of some psychological systems. Journal of Unified Science (Erkenntnis) 8, 36–49. Cacioppo, J.T., Petty, R.E., 1982. The need for cognition. Journal of Personality and Social Psychology 42 (1), 116–131. Capon, N., Davis, R., 1984. Basic cognitive ability measures as predictors of consumer information processing strategies. Journal of Consumer Research 11, 551–563. Caprara, G.V., Barbaranelli, C., Borgogni, L., Perugini, M., 1993. The big five questionnaire: a new questionnaire to assess the five factor model. Personality and Individual Differences 15, 281– 288. Cardaci, M., 2001. Ciber-psicologia. In: Cardaci, M. (Ed.), Esplorazioni cognitive di Internet, Carocci, Roma. Cardaci, M., Misuraca, R., Walker, R., 2002. Effetto di codifica multipla nel ragionamento condizionale con regole sociali. Giornale Italiano di Psicologia 4, 809– 823. Chasseigne, G., Mullet, E., Stewart, T.R., 1997. Aging and multiple cue probability learning: the case of inverse relationships. Acta Psychologica 97, 235– 252. Cosmides, L., 1989. The logic of social exchange: has natural selection shaped how humans reason? Studies with the Wason selection task. Cognition 39, 85–105. Costa, P.T., McCrae, R.R., 1985. The NEO Personality Inventory Manual, Psychological Assessment Resources, Odessa, FL. Fasolo, B., McClelland, G.H., Lange, K.A. 2001. Interattribute correlations influence whether decision processing strategies are attribute-based or option-based. Accepted pending revision. Ford, J.K., Schmitt, N., Schechtman, S.L., Hults, B.M., Doherty, M.L., 1989. Process tracing methods: contribution, problems, and neglected research questions. Organizational Behavior and Human Decision Processes 43, 75–117. Gigerenzer, G., Todd, P.M., The ABC Research Group, 1999. Simple heuristic that make us smart, Oxford University Press, New York. Gigerenzer, G., Goldstein, D.G., 1996. Reasoning the fast and frugal way: models of bounded rationality. Psychological Review 103, 650–669. Johnson, E.J., Meyer, R.J., 1984. Compensatory choice models of noncompensatory processes: the effect of varying context. Journal of Consumer Research 11 (1), 528 –541. Johnson, E.J., Payne, J.W., Schkade, D.A., Bettman, J.R., 1991. Monitoring information processing and decisions: the MouseLab system, Center for Decision Studies, Fuqua School of Business, Duke University, Unpublished manuscript. Johnson-Laird, P.N., 1983. Mental Models, Towards a Cognitive Science of Language, Inference, and Consciousness, transl. It. Modelli Mentali. Bologna: Il Mulino, 1988.
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
233
Judd, C.M., McClelland, G.H., 1989. Data Analysis: a Model Comparison Approach, Harcourt Brace Jovanovich, San Diego. Judd, C.M., McClelland, G.H., 1998. Measurement. In: Gilbert, D.T., Fiske, S.T., Lindzey, G. (Eds.), The handbook of social psychology, McGraw-Hill, Boston, MA. Levin, I.P., Huneke, M.E., Jasper, J.D., 2000. Information processing at successive stages of decision making: need for cognition and inclusion–exclusion effects. Organizational Behavior and Human Decision Processes 82 (2), 171–193. McClelland, G.H. 2002. WebIDB. Retrieved November 30, 2002, from http://samiam.colorado.EDU/~mcclella/ webIDB/ McCrae, R.R., 2000. Emotional intelligence from the perspective of the five-factor model of personality. In: Bar-on, R., Parker, J. D. A.(Ed.), The Handbook of Emotional Intelligence, Jossey-Bass, San Francisco, pp. 263 –276. McCrae, R.R., Costa, P.T.J., 1997. Conceptions and correlates of openness to experience. In: Hogan, J.A.J.R., Briggs, S.R. (Eds.), Handbook of personality psychology, Academic Press, Orlando, FL, pp. 269– 290. Payne, J.W., Bettman, J.R., Johnson, E.J., 1993. The adaptive decision maker, Cambridge University Press, Cambridge, UK. Rosenthal, R., Rosnow, R.L., 1985. Contrast analysis: focused comparisons in the analysis of variance, Cambridge University Press, New York. Shiloh, S., Koren, S., Zakay, D., 2001. Individual differences in compensatory decision making style and need for closure as correlated of subjective decision complexity and difficulty. Personality and Individual Differences 30, 699 –710. Stanovich, K.E., West, R.F., 1998. Individual differences in rational thought. Journal of Experimental Psychology: General 127, 161–188. Wason, P.C., 1966. Reasoning. In: Foss, B., (Ed.), New Horizons in Psychology, Harmondsworth, Penguin. Widing, R.E., Talarzyk, W.W., 1993. Electronic information systems for consumers: an evaluation of computerassisted formats in multiple decision environments. Journal of Marketing Research 30 (2), 125 –141. Zakay, D., 1990. The role of personal tendencies in the selection of decision-making strategies. Psychological Record 40 (2), 207– 213.
Individual differences in adaptive choice strategies Barbara Fasoloa,*, Raffaella Misuracab, Gary H. McClellandc a
Max-Planck Institute of Human Development, Lentzeallee 94 Berlin 14195, Germany b Universite´ de Bourgogne, France c University of Colorado at Boulder, USA
Abstract Individual differences in compensatory and non-compensatory choice processes remain an unresolved issue for decision process researchers. This study investigates the stability and nature of individual differences in choice processes when individuals adapt to changes in the structure of the choice environment, namely the correlation among the choice attributes. By means of process tracing techniques, between-subjects differences in choice processing (option-based or attribute based) were found to be stable across different tasks. Individuals with higher openness to experience and ability to solve reasoning tasks were found to be more adaptive, that is to switch more promptly their choice process in adaptive ways, by using more option-based search strategies when attributes were negatively related. These results suggest that insight into individual differences in choice processes can be gained when attention is given to the structure of the choice task and to how decision makers adapt to it in the course of the choice task. q 2003 University of Venice. Published by Elsevier Ltd. All rights reserved. Keywords: Individual differences; Adaptive strategies; Decision processes; Open-mindedness; Compensation strategies; Non-compensatory strategies; Interattribute correlations; Choice environment
1. Individual differences in adaptive choice strategies One of the unresolved issues in decision process research is the inability to find stable individual differences in choice process, despite striking between-subjects differences in search pattern that are typically found in the data of any process tracing experiment (see Ford et al., 1989, for a comprehensive review). Some people choose via a distinct compensatory process, others via a distinct non-compensatory process, yet predicting what
* Corresponding author. Tel.: þ 49-30-82406-339; fax: þ49-30-82406-394. E-mail address: [email protected] (B. Fasolo). 1090-9443/03/$ - see front matter q 2003 University of Venice. Published by Elsevier Ltd. All rights reserved. doi:10.1016/S1090-9443(03)00032-2
220
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
makes an individual follow a compensatory process or non-compensatory process is still a difficult research problem (Capon and Davis, 1984; Payne et al., 1993; Broeder, 2003). The difference between the two processes lies in the order in which information is searched, and the extent to which the search is thorough and systematic. For instance, faced with the decision of which digital camera to buy, a compensatory decision maker would consider one camera at a time, and examine one-by-one all the available attributes (zoom, resolution, etc.), to form an overall evaluation of how good all cameras are. This process is compensatory because bad attribute values can be compensated for by good values on other attributes. Faced with the same decision, a non-compensatory decision maker would consider one or two attributes at a time, for all or some of the cameras. Whereas the compensatory decision maker mainly focuses on options (option-based process), the noncompensatory decision maker mainly focuses on attributes (attribute-based process). This distinction has implications for the quality of the decision reached: When the attributes that describe the options are negatively correlated (i.e. there is no option available that is best on all attributes, and trade-offs need to be made) and decision quality is gauged against the economic principle of expected utility maximization, a compensatory process leads to higher quality decisions than a non-compensatory process (Johnson and Meyer, 1984; Widing and Talarzyk, 1993). This difference in extent to which the two processes adhere to normative principles motivated the first researchers to search for individual differences in factors like cognitive ability (Capon and Davis, 1984), prior knowledge (Payne et al., 1993), and experience with the choice (Bettman and Park, 1980)—to no avail. More successful were two recent empirical investigations. The first, by Levin et al. (2000), found that participants’ choice processes were reliably related to the personality variable of Need for Cognition (NFC, Cacioppo and Petty, 1982), a measure of cognitive style widely used in studies of persuasion and attitude change. In the study of Levin et al. (2000) participants chose a computer for a friend in two phases: first participants winnowed options from 16 to a smaller number, then they explored the few remaining options. Levin et al. found that on average participants used a less effortful attribute-based process in the first stage, and a more effortful option-based process in the second stage. This shift from attribute-based to optionbased process when option set size decreases is ‘adaptive’, because it is an efficient response to a choice problem that becomes more manageable, and is consistent with the view of decision makers as ‘adaptive’ to the choice at hand (Payne et al., 1993; Gigerenzer et al., 1999). Traditional information-processing measures (e.g. option-based or attribute-based search pattern and amount of information searched) did not show a reliable and general relationship with NFC, confirming the difficulty of capturing individual differences in the process used along the whole decision process (at both stages). The interesting result was that individual differences were found in the adaptive shift in choice process. Individuals with higher NFC switched to option-based processes in the second stage more than those with lower NFC. The second interesting development was by Broeder (2003). He found a correlation between intelligence and use of the non-compensatory strategy Take-The-Best (Gigerenzer and Goldstein, 1996) in environments where this strategy had the highest expected payoff. Intelligence was found to correlate with the ability to recognize
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
221
quickly the payoff structure of the task and consequently to adapt toward the strategy that had the highest payoff, when the environment was favoring a non-compensatory strategy. What these two investigations have in common is a joint consideration of person characteristics (intelligence, NFC) and choice environment characteristics (payoff structure, option set size), instead of a context-free exploration of individual characteristics. We adopt here the same perspective, with a special focus on individual differences in responses to changes in the choice environment, namely in correlation among choice attributes. In a prior investigation (Fasolo et al., 2001) we found that participants choosing among digital cameras displayed on average an attribute-based process. Although betweensubjects differences were large, participants’ choice process was steadily influenced by changes in inter-attribute correlation. When choice attributes were conflicting (there is negative correlation among them), decision makers adapted and switched toward less attribute-based search (or, more option-based), and the choice was perceived as more difficult and dissatisfactory than when attributes were positively related. This study uses the same process-tracing procedures as Fasolo et al. (2001) to investigate individual differences both in the choice process used throughout the task, regardless of correlation changes, and in the switch in choice process that occurs in response to negative correlations among attributes. Besides capturing choice process with process-tracing techniques, preferred choice style will also be elicited by means of self-reports. Self-reported choice style, as measured by a Compensatory Style (CS) questionnaire (Zakay, 1990), was found to correlate with actual choice outcomes by Shiloh et al. (2001). Interestingly, the CS questionnaire scores were reliably related to open-mindedness, such that more open-minded individuals reported to be more compensatory. Open-mindedness is the first person characteristic of interest in our study. Besides its relationship to self-reported compensatory choice style, other appealing reasons for choosing the open-mindedness construct are: (1) open-mindedness has both an emotional dimension— openness to experience—and a cognitive dimension—openness to culture (McCrae and Costa, 1997). The emotional dimension is important because the behavior of interest entails the emotional as well as the computational difficulty of facing trade-offs when attributes were negatively related. The cognitive dimension—reliably related to NFC scores (see McCrae, 2000, table 12.2)—is important because NFC was found to be a good predictor of switches in decision strategies (Levin et al., 2000). The second person characteristic of interest is reasoning ability, measured by performance on a battery of reasoning tasks. Decision researchers have shown that option-based and compensatory processes are more effortful than attribute-based ones (Payne et al., 1993), so one hypothesis is that individuals with higher reasoning abilities are more likely to engage in compensatory processing—meaning that they display a more option-based process, search more information and report being more compensatory. An alternative hypothesis—in line with Broeder (2003)—is that individuals with higher reasoning ability would be more likely to detect the need to shift strategy when needed, and therefore display a more pronounced shift toward an option-based process when confronted with negative correlations. The relationship
222
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
between negative correlations and higher cognitive ability is also in line with research showing that learning positive correlations is well within working memory capacity limits, whereas learning negative correlations is significantly more taxing (Chasseigne et al., 1997). In sum, our experiment aims to determine whether the way people search information for choosing in response to changes in the correlations among attributes is related to openmindedness, reasoning ability and self-reported choice style. Furthermore, our choice experiment considers relationships between these individual differences factors and perceptions of confidence, difficulty and satisfaction with the choice made.
2. Experiment overview The choice task and manipulation were adapted from Fasolo et al. (2001). Participants made choices among five different digital cameras, each described in terms of eight different attributes, described in detail below. Spontaneous individual strategies were of interest, so no instruction was given on how attributes should be weighted. The complexity of the choice task was varied by manipulating the average correlation among attributes. Participants’ choice processes were measured by means of a web-based process tracing technique, and their confidence, satisfaction and difficulty were recorded after each choice on a web-based questionnaire. To measure individual differences in participants’ choice process, an original questionnaire was developed (Questionnaire of Individual Differences, or QUIND), comprising three separate sections. The first section included questions about the participants’ openness to experience and to culture. Overall more open-minded decision makers were expected to be more compensatory in their self-reported style and to be more ‘adaptive’ in their traced search process, that is to display larger shifts toward option-based processing in response to negative correlations. The second section presented a battery of reasoning tasks. Reasoning capacity was operationalized as performance on the Wason Selection Task (Wason, 1966) and on syllogisms with different degrees of difficulty (Johnson-Laird, 1983). In their studies of individual differences in rational thought, Stanovich and West (1998) found that syllogisms and Wason selection tasks were more highly correlated with reasoning capacity (as measured by the Scholastic Assessment Test). If the ability to adapt to changes in the choice environment is positively related to cognitive capacity, individuals with higher reasoning ability should display larger shifts in choice process in response to changes in correlation. If higher reasoning ability is related to more compensatory behavior, regardless of the structure of the task, then individuals with higher reasoning ability should always have a more compensatory process (more option-based and more information sought) than those with lower reasoning ability. The third section presented questions about the participants’ preferred choice style, similar to the CS questionnaire of Zakay (1990). Based on the findings of Zakay (1990) and Shiloh et al. (2001), we expected self-reported choice style to be correlated with traced choice process.
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
223
3. Method 3.1. Participants One hundred twenty-three students enrolled in the course of General Psychology at the University of Colorado participated in the experiment and received course credit in exchange for their participation. 3.2. Materials and apparatus Before the choice task, a web-based questionnaire was presented. The questionnaire was composed of four parts. “About me”. Twenty-six Likert items, inspired by the Neo-Personality Inventory (NEOPI, Costa and McCrae, 1985) and by the Big Five Questionnaire (Caprara et al., 1993), measured participants’ open-mindedness. Specifically, 13 items measured openness to culture (e.g. “I usually read”) and 13 measured openness to experience (e.g. “In choosing a destination for my travels I always tend to prefer exotic places different from those, where I live”). For each item, participants indicated the degree to which the content matched their opinions and behaviors by clicking the appropriate number on a scale from 1 (“the content of the statement does not match my opinions and behaviors”) to 5 (“the content of the statement matches completely my opinions and behaviors”). The final score was the sum of the individual item scores. The higher the score, the higher the open-mindedness. “About my reasoning”. Four syllogisms. According to Johnson-Laird’s (1983) theory of mental models, two of these syllogisms were easy (their solution needed the construction of only one mental model), and the other two were difficult (their solution needed the construction of three mental models). Each syllogism was composed of a pair of premises followed by four possible conclusions, only one of which was valid. Participants selected the one response believed to follow logically from the first two premises. A score of 1 was given for each correct response; a score of 0 for each incorrect response. The final score was the sum of the individual item scores. The higher the score, the higher the syllogistic reasoning abilities. “How I check violations of rules”. Two original conditional tasks inspired by the Wason Selection Tasks and implemented in a web-based questionnaire format. Consistent with the most recent literature on the topic (Cosmides, 1989; Cardaci et al., 2002), the traditional paradigm of four cards was integrated with a brief situational frame. Each task was composed of a conditional rule having the general form if p then q, followed by four cases, respectively, corresponding to the formal cases p, non-p, q, non-q. For each case, participants were asked if they needed more information to determine whether the rule presented was violated. The normative correct responses corresponded to the request of further information to determine whether the rule was violated in the cases, where p and non-q were given. One point was assigned for each correct response; no points for each incorrect response. The final score was the sum of the individual item scores. The higher the score, the higher the conditional reasoning abilities. “About my choice style”. Four items describing self-reported choice style, adapted from the CS questionnaire of Zakay (1990). Each task described a hypothetical choice
224
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
situation, followed by two possible responses: one was consistent with a compensatory decision strategy; the other with a non-compensatory decision strategy. Participants were asked to select the response that better described how they would normally decide. One point was given for each compensatory response; no point for each non-compensatory response. The final score was the sum of the individual item scores. The higher the score, the higher the tendency to use a compensatory strategy. According to ergonomic criteria of on-line research (Cardaci, 2001), all items were designed to allow for easy reading and immediate comprehension. For each item, participants gave a point-and-click response. Each selection excluded all others, avoiding the disadvantages due to compilation errors of paper-and-pencil questionnaire. Data were automatically stored in a web-server database. Following the questionnaire, participants were presented with the choice task. Four different sets of five digital cameras were presented on a web-based comparison table, where options appeared along the columns and attributes along the rows. Each digital camera was described along the same eight attributes. Emulating what is commonly available on web-based comparison shopping sites, clicking on the attribute name revealed a detailed description of each attribute. For each attribute (‘delay between shots’), participants were provided with (a) a short and comprehensible description (‘The amount of time, measured in seconds, it takes the camera to process and store an image when shooting in normal mode (non-burst), at the camera’s maximum resolution setting. This feature is also known as lag time’); (b) the direction of preference (‘the less delay, the better’); (c) the range of the attribute (‘1 – 20 s’); and (d) the levels of the attributes (‘1– 2 – 5 –7 –20 s’). The eight attributes were: Delay Between Shots, Image Capacity, LCD Display, Lighting, Optical Zoom, Resolution, Size and Weight. To keep track of the pieces of information searched and their exact order, the content of each cell was only revealed when the participant pointed the mouse on the cell. The cell was closed again before the content of the next cell was revealed. The information acquisition process was recorded with WebIDB (McClelland, 2002), an original WWWbased version of the matrix schema originally devised for the MouseLab program to trace decision processes on a computer monitor (Johnson et al., 1991). WebIDB is a Java applet suitable for use with any Java-capable web browser. After each choice, a short web-based questionnaire asked participants to rate their satisfaction with and confidence in the choice just made. The entire experiment (accessible at: http://psych.colorado.edu/~bfasolo/idb/baraga) was conducted on Macintosh computers using Internet Explorer version 5.1. 3.3. Design Inter-attribute correlation was varied between positive and negative in an asymmetric between– within design. To construct tables with a predetermined average inter-attribute correlation, a Mathematica routine was used that generates random observations from a multivariate normal distribution with a specified variance –covariance structure. Values for the random observations were rounded and rescaled to match the appropriate range and number of attribute levels. ‘Positive’ tables were drawn from a population with average pair-wise attribute correlation of þ 0.5. ‘Negative’ tables were sampled from a population
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
225
with the average negative inter-attribute correlation of 2 0.14, the most extreme negative correlation possible for eight attributes [21=ðk 2 1Þ; with k ¼ 8]. Every participant in each condition saw the choice matrices in the same order. Fifty-six participants made choices from four sets of cameras with negative correlations (‘Neg – Neg’ condition), and 67 participants first made choices from two sets with positive correlations and then two sets with negative correlations (‘Pos –Neg’ condition). Four dependent measures will be reported: (a) search pattern, by option or by attribute; (b) within-subject consistency in search pattern; (c) self-reported measures of satisfaction, confidence and decision difficulty; (d) responses to the four sections of the individual questionnaire: open-mindedness scores, syllogistic reasoning scores, Wason Selection Task scores, and self-reported choice style. The following is a more detailed description of measures of (a) and (b). (a) Search pattern by option or by attribute, was measured as the Payne Index (PI, e.g. Payne et al., 1993)—the most common measure of information search pattern available in the literature. This index combines the number of times a participant looks consecutively at information about the same option across attributes (Option Salient transition) with the number of times a participant looks consecutively at information about the same attribute across options (Attribute Salient transition). Reinspections to the same cell are included in the computations, whereas diagonal moves are excluded, because they are not consistent with either a prevailing optionbased or attribute-based process. The PI is determined by the number of Option Salient transitions minus the number of Attribute Salient transitions, divided by their sum. It ranges from 2 1 to þ 1, where positive values signify a predominately optionbased search process (implying a compensatory process) and negative values signify a predominantly attribute-based search pattern (implying a non-compensatory process). (b) Within-subject consistency was summarized by Cronbach’s alpha computed over the four Payne Indices obtained from each participant. To compute alpha, a Principal Component Analysis was performed as recommended by Judd and McClelland (1998, p. 204). The resulting eigenvalue (sum of the squared factor loadings on the first unrotated principal component, estimating the correlation between each PI and the underlying choice processing style) was then used to compute the coefficient alpha {½k=ðk-1Þ* ð1-ð1=lÞÞ; where k; is the number of items, and l; is eigenvalue}. Coefficients larger than 0.80 are reported by social psychologists as indicating high within-subject consistency. This formula has the advantage of not assuming equal variance across items, and yields a convenient and meaningful coefficient for summarizing the within-subject consistency in Payne Indices. 3.4. Procedure Participants were tested individually or in small groups and sat in front of computers connected to the experimental website. First, participants were introduced to the webbased QUIND questionnaire, and were told that they would be presented with some questions concerning their reasoning and their daily choice behaviors. It was emphasized
226
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
that they needed to think about the best description of their natural choice behavior and reasoning, as opposed to the absolutely ‘best’ answer. Next, they completed the questionnaire. Instructions for the choice task were displayed on the website. They were told to imagine having to spend a gift certificate on a digital camera of their choice among those presented. Cameras would be displayed in four successive tables. Each table would display five cameras, described on eight features. Their task was to choose from each table the camera that was best for them. Following the instruction screen, participants were presented with a hypothetical choice problem among computers, so that they could practice using the WebIDB information acquisition system on the comparison table. The experimenter also reminded participants that they could take as much time as they wished to acquire information before making their choice and that the experiment would conclude with a short web-based questionnaire. 3.5. Analyses notes Analyses were conducted on the participants’ choices made, search logs and responses to the QUIND questionnaire and to questions about satisfaction, confidence, and difficulty after each choice made. For each section of the QUIND questionnaire (open-mindedness, syllogisms, Wason Selection Task, and Choice Style) a Cronbach’s alpha was computed. Unless specified otherwise in the results section, the analyses reported in this experiment are from a two-way between– within ANOVA conducted as a series of planned contrasts for main effects, interactions, and simple effects. This use of planned contrasts is recommended by Rosenthal and Rosnow (1985) and Judd and McClelland (1989), among others. By so doing, results can be presented in terms of substantive comparisons instead of in terms of statistical abstractions. More importantly, in the case of analyses of withinparticipant variables, one-degree-of-freedom contrasts avoid the troublesome and seldom satisfied sphericity assumption required in the usual repeated-measures ANOVA (see Chapter 14 of Judd and McClelland (1989)). Participants’ search logs were first screened for ‘pass-throughs’ in the following way. In addition to tracking cells visited, WebIDB also records the amount of time spent in each cell. These individual reaction times (RT) per cell were plotted on a histogram, and analyzed for each individual participant. RTs would typically have a bi-modal distribution with a clear demarcation between the two modes. This demarcation occurred at a time varying from participant to participant between 250 and 450 ms after they first moved the mouse on the table. For all participants this demarcation occurred too soon to accompany a deliberative search strategy. Therefore, the transitions associated with the first distribution (below the demarcation time) were considered mere ‘pass-throughs’, and were eliminated from further analysis. The transitions associated with the second distribution (above the demarcation time) were considered deliberative, and composed the search logs ultimately analyzed. Search logs were processed for each participant to yield a measure of total Option-Salient and Attribute-Salient transitions, and were then combined into a PI for each table.
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
227
Fig. 1. Payne Index as a function of correlation and table order.
4. Results 4.1. Search pattern Search pattern was indexed using the PI. As Fig. 1 shows, search was, on average more by attribute than by option (PI ¼ 20:262; tð122Þ ¼ 28:78; p , 0:001), showing that the participants’ dominant search behavior was to compare different options along a given attribute before moving to the next attribute. The correlation manipulation significantly affected the search strategy in the expected direction: negative correlations increased the focus on options both when correlation was manipulated within (comparing the search pattern on the first two and latter two tables for the Pos– Neg group, Fð1; 66Þ ¼ 22:36; PRE1 ¼ 0.25, p , 0:001) and between (comparing the search pattern on the first two tables for the Pos –Neg group and Neg – Neg group, Fð1; 121Þ ¼ 6:40; PRE ¼ 0:05; p , 0:02). The search pattern also changed over time. Over trials, the search became more option based for both groups (Fð1; 121Þ ¼ 27:25; PRE ¼ 0:19; p , 0:001), but the increase toward more option-based strategies when correlation switched from positive to negative from the second to third choice tended to be more pronounced in the Pos– Neg group than in the Neg – Neg group (Fð1; 121Þ ¼ 3:42; PRE ¼ 0:03; p ¼ 0:06). 4.2. Within subject consistency in search pattern Although the search pattern was on average attribute-based across the four choice tasks, it ranged from a minimum of 2 0.87 to a maximum of 0.92 for individual participants. The data exhibited the same degree of variability observed in Fasolo et al. (2001). Table 1 shows an example, representing the search pattern (operationalized as PI) across the four comparison tables for three participants in the Neg –Neg condition. A principal component analysis was performed on the four Payne Indices of all participants in the Neg – Neg 1 PRE is the proportional reduction of error between a model of the data that makes the given distinction and one that does not (Judd and McClelland, 1989). It is a standardized effect size measure.
228
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
Table 1 Payne Index across the four tables for three participants in the condition Neg– Neg
Participant 1 Participant 2 Participant 24
PI1
PI2
PI3
PI4
20.67 20.051 0.02
20.87 20.56 0.24
20.31 20.050 0.17
0.06 20.45 0.41
condition. The first factor had an eigenvalue equal to 2.73. The coefficient alpha was 0.85, suggesting the existence of a highly reliable individual difference in search pattern. Participants who started to search by attribute (as participants 1 and 2) continued to do so all along the task, and the same consistency characterized participants who started to search by option (as participant 24). 4.3. Self-reported measures of satisfaction, confidence, and decision difficulty As Fig. 2 shows, participants in the Pos –Neg group were more satisfied with their first two choices (made from tables with positive correlations) than participants in the Neg – Neg group, who first chose from two tables with negative correlations (M ¼ 4:35 vs. 3.99, Fð1; 121Þ ¼ 30:25; p , 0:001; PRE ¼ 0.20). Negative correlations also affected perceptions within-subjects, extending previous findings of Fasolo et al. (2001). Participants from the Pos– Neg group reported being more satisfied with the first two choices (with positive correlations) than with the last two choices (with negative correlations). Participants felt particularly dissatisfied when they experienced negative correlations after having experienced positive correlations (M ¼ 4:35 vs. 3.59, Fð1; 66Þ ¼ 106:69; p , 0:001; PRE ¼ 0.62). The same results apply for the confidence ratings, with correlation affecting confidence both when manipulated between (M ¼ 4:14 vs. 3.71, Fð1; 121Þ ¼ 19:27; p , 0:001; PRE ¼ 0.14), and within (M ¼ 4:14 vs. 3.52, Fð1; 66Þ ¼ 63:92; p , 0:001; PRE ¼ 0.49). As Fig. 3 shows, participants in the Pos –Neg group found the choice task easier on the first two trials than those in the Neg – Neg group (M ¼ 2:7 vs. 1.8, Fð1; 121Þ ¼ 36:12; p , 0:001; PRE ¼ 0.23) and than on the second two negative trials (M ¼ 2:7 vs. 2.1, Fð1; 66Þ ¼ 18:49; p , 0:001; PRE ¼ 0.22). Therefore, negative correlation always made it more difficult for participants to make choices, regardless of whether correlation was manipulated within or between. 4.4. Responses to the questionnaire of individual differences Table 2 summarizes the data, reporting the correlations between individual differences factors and choice process (adaptive shifts in choice process as well as stable choice process). The following is a detailed account of the results, for each section of the questionnaire. Open-mindedness. Participants’ open-mindedness scores were divided into openness to experience and openness to culture. The openness to culture items revealed higher reliability (coefficient alpha was 0.84) than the openness to experience items (coefficient alpha was 0.67). Despite the low level of alpha, the experiment had enough power to detect
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
229
Fig. 2. Mean satisfaction rating as a function of the correlation experienced in the first two and last two tables.
a significant correlation between openness to experience and strategy shift. As expected, participants’ (in the Pos– Neg group) shifting towards option-based strategies when correlations changed from positive to negative was stronger if they exhibited higher openness to experience [rð65Þ ¼ 0:27; p , 0:05]. Moreover, participants with higher open-mindedness scores (both culture and experience) tended to report higher satisfaction across all four choices [rð122Þ ¼ 0:16; p ¼ 0:087]. Performance on Wason Selection Task (WST). Participants’ performance on the WST was higher than in the original version of the task (10% in Wason, 1966) and in accord with the performance reported in the wide number of studies using facilitated versions of the task (Cosmides, 1989). This indicates that the particular web-based format used had a facilitating effect. Correct responses to the WST provided sufficient variability in response scores and were reliably related across the two tasks (a ¼ 0.765), indicating that this task
Fig. 3. Mean rating of ‘choice easy’ as a function of the correlation experienced in the first two and last two tables.
230
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
Table 2 Individual differences and choice process (correlation table) Correlates of
Adaptive shift (within-subject differences in Payne Index)
Stable choice process (average Payne Index)
Openness to experience Openness to culture Reasoning (wason) Reasoning (syllogisms)
0.27 (*) 0.12 (n.s.) 0.27 (*) 0.14 (n.s)
0.031 (n.s.) 0.035 (n.s.) 20.095 (n.s.) 20.05 (n.s.)
*Significant at p , 0:05:
was an appropriate indicator of a common reasoning capacity. Higher WST performance significantly correlated with larger amount of information searched on comparison tables [rð120Þ ¼ 0:38; p , 0:0001], usually associated with more compensatory search processes. Participants with higher WST scores did not exhibit a more option based process along the whole experiment. Rather they shifted processing to a greater extent, by using a more option-based search with negative correlations than with positive correlations, more so than participants with lower WST scores [rð120Þ ¼ 0:27; p , 0:001] did. Syllogistic reasoning performance. Participants’ performances on the four Syllogistic Reasoning Tasks were high on the first two ‘easy’ syllogisms (because they required one mental model for their solutions), and low on the second two ‘difficult’ syllogisms (because they required three mental models for their solutions). On average, this differential performance was in accord with the literature (Johnson-Laird, 1983). Correct responses to the four syllogisms were not reliably related (a ¼ 0.381) and were less variable overall than the responses to the WST (across syllogistic reasoning task, average SD ¼ 0.34; across Wason Selection Task, average SD ¼ 0.64). Both might be possible reasons why no reliable association was found between measures of decision processing and performance in the syllogistic task. Self-reported choice style. Participants reported, on average, having a compensatory choice style (M ¼ 4:06; SD ¼ 0.89) consistent with results obtained previously by Shiloh et al. (2001). Answers to the five Choice Style items were not reliably related with one another (a ¼ 0.324), so no meaningful correlations can be found with the traced choice process.
5. Discussion The goal of this experiment was to determine if choice processing is a stable individual difference, and if adaptive shifts in choice process have predictable person correlates. Choice processing, as measured by the PI, is indeed a reliable individual difference, indicating that participants have a preferred (option-based or attribute-based) processing style measurable from their actual behavior. Participants’ traced choice process was on average consistent with a non-compensatory strategy (mainly attribute-based) whereas
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
231
their reported choice style was compensatory, perhaps related to the social desirability of describing oneself as a rational information seeker. When the average choice process is considered for each person, evidence for individual difference is very weak, confirming the difficulty already noted in the literature. It must be emphasized that these weak effects are not likely due to a lack of power because correlations were found even among constructs that had low levels of reliability (e.g. openness to experience). The data are more promising when the behavior analyzed for each individual is not the average choice process, but rather the shifts in choice process in response to changes in the correlation structure. When shifts in choice process are considered, significant individual differences are found, along with correlations to both openness to experience and reasoning ability. First, participants with higher openness to experience switch toward option-based strategies to a greater extent when correlations become more negative. This result extends data from Shiloh et al. (2001) and Levin et al. (2000). Second, participants with higher reasoning ability also move toward option-based strategies to a greater extent when correlations become more negative. This result is consistent with Broeder (2003), providing further evidence that reasoning ability is more related to the ability to detect when changes of search strategy are necessary, rather than to the unilateral adoption of the most effortful strategy. The effect of open-mindedness and reasoning capacity was found when participants faced choice tasks that encouraged them to switch strategies. Individual differences pertained to the extent to which people were ‘adaptive’, or switched strategies: participants with a more open mind and better at reasoning are also better able to adapt to the choice task as demanded. These findings highlight an important methodological issue: when studying the effect of person characteristics (here, open-mindedness and reasoning ability) on choice process, properties of the choice environment (here, correlation among attributes) are as important as characteristics of the individuals being studied, in line with the ‘organism-environment model’ advocated as early as 1939 by Brunswik (1939).
6. Conclusions This study confirms that inter-attribute correlation is a powerful task characteristic that affects search processes and perception: All participants, regardless of different preferences over attributes, dispositions toward thinking, and stated choice styles were able to adapt to it, although they disliked doing so, in line with prior research (Fasolo et al., 2001). Choice processing emerged as a reliable individual difference. Decision makers exhibit a preferred choice strategy—mainly option-based or mainly attribute-based—which consistently guides their search process. When considered independently of changes in the choice environment, choice process had—at best—weak person correlates. When changes in the environment were considered, responses to them (switches toward more optionbased process when correlations were negative) showed a small but significant relationship with open-mindedness and reasoning ability.
232
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
Methodologically, these results show that although person correlates of a stable choice process are difficult to grasp, it is possible to capture individual differences in the way decision makers switch process in response to task complexity, like the inter-attribute correlation in our experiment. Rather than the traditional distinction between ‘compensatory’ and ‘non-compensatory’ processes, we suggest that it would be more useful to distinguish between ‘more adaptive’ and ‘less adaptive’ decision makers. Our study, along with Levin et al. (2000) and Broeder (2003), made a first move in this direction, showing that an open mind and good reasoning ability are two good predictors of more adaptive decision makers.
References Bettman, J.R., Park, C.W., 1980. Effects of prior knowledge and experience and phase of the choice process on consumer decision processes: a protocol analysis. Journal of Consumer Research 7, 234– 248. Broeder, A., 2003. Decision making with the adaptive toolbox: influence of environmental structure, intelligence and working memory load. Journal of Experimental Psychology: Learning, Memory and Cognition in press. Brunswik, E., 1939. The conceptual focus of some psychological systems. Journal of Unified Science (Erkenntnis) 8, 36–49. Cacioppo, J.T., Petty, R.E., 1982. The need for cognition. Journal of Personality and Social Psychology 42 (1), 116–131. Capon, N., Davis, R., 1984. Basic cognitive ability measures as predictors of consumer information processing strategies. Journal of Consumer Research 11, 551–563. Caprara, G.V., Barbaranelli, C., Borgogni, L., Perugini, M., 1993. The big five questionnaire: a new questionnaire to assess the five factor model. Personality and Individual Differences 15, 281– 288. Cardaci, M., 2001. Ciber-psicologia. In: Cardaci, M. (Ed.), Esplorazioni cognitive di Internet, Carocci, Roma. Cardaci, M., Misuraca, R., Walker, R., 2002. Effetto di codifica multipla nel ragionamento condizionale con regole sociali. Giornale Italiano di Psicologia 4, 809– 823. Chasseigne, G., Mullet, E., Stewart, T.R., 1997. Aging and multiple cue probability learning: the case of inverse relationships. Acta Psychologica 97, 235– 252. Cosmides, L., 1989. The logic of social exchange: has natural selection shaped how humans reason? Studies with the Wason selection task. Cognition 39, 85–105. Costa, P.T., McCrae, R.R., 1985. The NEO Personality Inventory Manual, Psychological Assessment Resources, Odessa, FL. Fasolo, B., McClelland, G.H., Lange, K.A. 2001. Interattribute correlations influence whether decision processing strategies are attribute-based or option-based. Accepted pending revision. Ford, J.K., Schmitt, N., Schechtman, S.L., Hults, B.M., Doherty, M.L., 1989. Process tracing methods: contribution, problems, and neglected research questions. Organizational Behavior and Human Decision Processes 43, 75–117. Gigerenzer, G., Todd, P.M., The ABC Research Group, 1999. Simple heuristic that make us smart, Oxford University Press, New York. Gigerenzer, G., Goldstein, D.G., 1996. Reasoning the fast and frugal way: models of bounded rationality. Psychological Review 103, 650–669. Johnson, E.J., Meyer, R.J., 1984. Compensatory choice models of noncompensatory processes: the effect of varying context. Journal of Consumer Research 11 (1), 528 –541. Johnson, E.J., Payne, J.W., Schkade, D.A., Bettman, J.R., 1991. Monitoring information processing and decisions: the MouseLab system, Center for Decision Studies, Fuqua School of Business, Duke University, Unpublished manuscript. Johnson-Laird, P.N., 1983. Mental Models, Towards a Cognitive Science of Language, Inference, and Consciousness, transl. It. Modelli Mentali. Bologna: Il Mulino, 1988.
B. Fasolo et al. / Research in Economics 57 (2003) 219–233
233
Judd, C.M., McClelland, G.H., 1989. Data Analysis: a Model Comparison Approach, Harcourt Brace Jovanovich, San Diego. Judd, C.M., McClelland, G.H., 1998. Measurement. In: Gilbert, D.T., Fiske, S.T., Lindzey, G. (Eds.), The handbook of social psychology, McGraw-Hill, Boston, MA. Levin, I.P., Huneke, M.E., Jasper, J.D., 2000. Information processing at successive stages of decision making: need for cognition and inclusion–exclusion effects. Organizational Behavior and Human Decision Processes 82 (2), 171–193. McClelland, G.H. 2002. WebIDB. Retrieved November 30, 2002, from http://samiam.colorado.EDU/~mcclella/ webIDB/ McCrae, R.R., 2000. Emotional intelligence from the perspective of the five-factor model of personality. In: Bar-on, R., Parker, J. D. A.(Ed.), The Handbook of Emotional Intelligence, Jossey-Bass, San Francisco, pp. 263 –276. McCrae, R.R., Costa, P.T.J., 1997. Conceptions and correlates of openness to experience. In: Hogan, J.A.J.R., Briggs, S.R. (Eds.), Handbook of personality psychology, Academic Press, Orlando, FL, pp. 269– 290. Payne, J.W., Bettman, J.R., Johnson, E.J., 1993. The adaptive decision maker, Cambridge University Press, Cambridge, UK. Rosenthal, R., Rosnow, R.L., 1985. Contrast analysis: focused comparisons in the analysis of variance, Cambridge University Press, New York. Shiloh, S., Koren, S., Zakay, D., 2001. Individual differences in compensatory decision making style and need for closure as correlated of subjective decision complexity and difficulty. Personality and Individual Differences 30, 699 –710. Stanovich, K.E., West, R.F., 1998. Individual differences in rational thought. Journal of Experimental Psychology: General 127, 161–188. Wason, P.C., 1966. Reasoning. In: Foss, B., (Ed.), New Horizons in Psychology, Harmondsworth, Penguin. Widing, R.E., Talarzyk, W.W., 1993. Electronic information systems for consumers: an evaluation of computerassisted formats in multiple decision environments. Journal of Marketing Research 30 (2), 125 –141. Zakay, D., 1990. The role of personal tendencies in the selection of decision-making strategies. Psychological Record 40 (2), 207– 213.