Acquiescence in personality questionnaires: Relevance, domain specificity, and stability

Journal of Research in Personality 57 (2015) 119–130

Contents lists available at ScienceDirect

Journal of Research in Personality journal homepage: www.elsevier.com/locate/jrp

Acquiescence in personality questionnaires: Relevance, domain specificity, and stability q Daniel Danner a,⇑, Julian Aichholzer b, Beatrice Rammstedt a a b

GESIS – Leibniz Institute for the Social Sciences, Mannheim, Germany Department of Methods in the Social Sciences, University of Vienna, Austria

a r t i c l e

i n f o

Article history: Available online 28 May 2015 Keywords: Acquiescence Response style Latent state–trait theory Big Five

a b s t r a c t Acquiescence, which is defined as agreeing to items regardless of content, is a well-known bias in self-report instruments. This paper investigates the relevance, domain specificity, and the stability of acquiescence in personality questionnaires. Data from two large samples representative for the German (N = 1999) and for the Austrian adult population (N = 3266) were investigated with structural equation models. In both studies respondents answered, besides others, a short Big Five inventory. The three core findings are: (1) acquiescence systematically affects the variance of personality items and biases the association with other variables, (2) acquiescence is consistent across different question types, and (3) acquiescence in personality items is moderately stable over time. Implications for research and the application of personality questionnaires are discussed. Ó 2015 Elsevier Inc. All rights reserved.

1. Introduction An individual’s response to an item consists partly of content-related response and partly of non-content-related response styles, such as extreme responding or acquiescence. Such unwanted response tendencies can bias the results of surveys and questionnaires, especially when participants differ in their tendency for the corresponding response style. One of the most common response styles is acquiescence, defined as agreeing with an item or question regardless of the content (e.g., Cronbach, 1942; Ferrando, Condon, & Chico, 2004; Paulhus, 1991). Evidence for the distorting effect of acquiescence has been widely demonstrated. For example, Bentler, Jackson, and Messick (1971) demonstrated that acquiescent responding can bias the correlation between personality items. In several studies they found that the intended strong negative correlations between oppositely poled adjectives like ‘‘happy’’ and ‘‘sad’’ were only weak. After controlling for acquiescence, however, these coefficients markedly increased in strength, thus indicating that the adjective ratings are severely affected by acquiescence. In the same vein, Rammstedt and colleagues (Rammstedt & Farmer, 2013; Rammstedt, Goldberg, & Borg, 2010; Rammstedt & Kemper, 2011) and Aichholzer (2014)

q We thank Lena Raffetseder, Saul Berenson, and two anonymous reviewers for their helpful comments on an earlier draft of the manuscript. ⇑ Corresponding author at: GESIS – Leibniz Institute for the Social Sciences, P.O. Box 122155, D-68072 Mannheim, Germany. E-mail address: [email protected] (D. Danner).

http://dx.doi.org/10.1016/j.jrp.2015.05.004 0092-6566/Ó 2015 Elsevier Inc. All rights reserved.

showed that acquiescence can even affect the factor structure of established personality questionnaires. Based on large scale data, the authors demonstrated that the initial factor loadings were inconsistent with the expected Big Five factor structure. However, after controlling for acquiescence, the expected factor structure emerged in textbook-like clarity (see also McCrae, Herbst, & Costa, 2001). In addition, they showed that the amount of acquiescence bias is related to educational attainment which suggests that acquiescence can differently affect the validity of personality questionnaires in subpopulations. These results indicate that acquiescence can be a serious obstacle in empirical research and especially in personality research. During the last decades, acquiescence has been investigated from different perspectives. Several studies investigated how acquiescence can be measured (e.g., Billiet & McClendon, 2000; Paulhus, 1991; Winkler, Kanouse, & Ware, 1982), which situational or item characteristics affect acquiescence (e.g., Elliott, 1961; Krosnick & Presser, 2010; McBride & Moran, 1967; Trott & Jackson, 1967), and how acquiescence is related to personality or demographic variables (e.g., DiStefano, Morgan, & Motl, 2012; Knowles & Nathan, 1997; Weems, Onwuegbuzie, Schreiber, & Eggers, 2003). However, few studies have examined the structure of the construct acquiescence itself. Therefore, several crucial aspects of acquiescence remain unclear until now. First of all, it is still debated whether acquiescence is a broad, general, or a domain specific construct, whether persons who tend to acquiescent responding in attitude or knowledge items also tend to acquiescent responding in personality items. This is particularly

120

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

important because it has been suggested to use control scales for identifying acquiescent responding (e.g., Amelang & Borkenau, 1981; Paulhus, 1991; Rorer & Goldberg, 1965). Control scales consist of heterogeneous items (e.g., Amelang & Borkenau, 1981; Handel, Ben-Porath, Tellegen, & Archer, 2010; Weijters, Geuens, & Schillewaert, 2010) or semantically balanced scales (e.g., Billiet & McClendon, 2000; Ferrando et al., 2004; Rammstedt et al., 2010). Balanced scales contain an equal amount of positively poled items (e.g., ‘‘I see myself as someone who is sociable, outgoing’’) and negatively poled items (e.g., ‘‘I see myself as someone who is reserved’’). Using such items to control for acquiescent responding in different domains would only be admissible if acquiescence is consistent across different item contents. Furthermore, investigating the specificity of acquiescence allows gaining a better understanding of the causes of acquiescent responding. A multidimensional, domain specific construct would suggest a heterogeneous response style with different causes in different domains. A consistent, unidimensional construct in contrast would suggest a homogeneous response style which is affected by similar factors in different domains. Second, it is not clear whether acquiescence in personality questionnaires is stable over time, that is, whether persons who tend to acquiescent responding today also tend to acquiescent responding several months later. From the perspective of personality psychology, it is important to detect whether acquiescent responding is primarily determined by the person or by the measurement occasion. From an applied perspective, it is important to decide whether acquiescent responding at a first measurement occasion can be used to control for acquiescent responding at a following measurement occasion.

2. Previous research 2.1. Domain specificity of acquiescence Several studies suggest that there is a certain level of generalizability of acquiescence within domains like attitude scales or personality questionnaires. For example, Billiet and McClendon (2000) analyzed data from 986 respondents and reported a latent correlation of r = .44 between acquiescence in two attitude scales. Likewise, Ferrando et al. (2004) assessed 207 students and reported positive correlations (r = .14 to r = .54) between several personality acquiescence scales, and Hinz, Michalski, Schwarz, and Herzberg (2007) analyzed data from 2037 respondents and reported positive correlations (r = .15 to r = .40) between several personality acquiescence indicators. However, there have been only a few studies investigating the generalizability of acquiescence across domains. For example, Gage, Leavitt, and Stone (1956) assessed 97 teachers and reported a correlation of r = .19 between acquiescent responding in personality items and difficult knowledge items. Similarly, Ray (1983) analyzed data from several Australian samples (N = 83 to N = 377) and reported positive correlations between acquiescence in personality items (on average r = .30), between acquiescence in attitude items (on average r = .30), but not between acquiescence in personality and attitude items (on average r = .09). In turn, a study by Blumberg (1973) in a student sample suggests practically no convergence between acquiescence measures derived from personality and attitude scales whatsoever (see also Rorer & Goldberg, 1965). Summarizing, these results suggest that acquiescence is rather domain specific and persons who tend to acquiescent responding in attitude or knowledge items do not equally tend to acquiescent responding in personality items. However, these studies did not report reliability estimates for the acquiescent measures and hence, these small correlations may also be biased by a small

reliability of the manifest acquiescence variables. In the present study, we will investigate the relation between acquiescence in personality questionnaires and acquiescence in attitude scales using structural equation models. This method allows analyzing the relation between latent variables adjusted for unsystematic measurement error. 2.2. Stability of acquiescence The stability of acquiescence over time has also received little scrutiny. Billiet and Davidov (2008) analyzed data from the Belgian Election study (N = 1503) and reported a correlation of r = .56 between two latent acquiescence variables over a period of 4 years. In addition, Weijters, Geuens, and Schillewaert (2010b) analyzed data of 604 respondents with latent state–trait models and reported that 60% of acquiescence variance is stable over one year. Similarly, other authors have considered the response bias associated with wording effects as wording method factors and also find relatively high stability of these factors, for instance, in the Rosenberg Self-Esteem Scale (Gana et al., 2013; Marsh, Scalas, & Nagengast, 2010; Motl & DiStefano, 2002). These results do suggest that a substantial proportion of acquiescence is stable over time. However, Billiet and Davidov as well as Weijters and colleagues used only attitude items in their analyses. McCrae et al. (2001) investigated the stability of acquiescence in personality items using a sample of elder respondents (above 60 years, N = 255) and reported a correlation of r = .74 between two acquiescence indicators. Considering the rather low correlations between acquiescence in personality questionnaires and acquiescence in attitude scales (Ray, 1983), we do not know whether acquiescence is stable in general or only within the domain of attitude items. In the present study, we will close this gap and use a latent state–trait model to investigate the stability of acquiescence in personality items. 2.3. The present study The present research will investigate to what extent acquiescence is relevant in assessing personality. In particular, we will estimate the extent to which acquiescence affects the variance of single items and we will illustrate that acquiescence can bias the relation between personality items and criterion variables. Furthermore, this research investigates to what extent acquiescence is a general or a domain specific construct and to what extent acquiescence is stable over time. These research questions will be investigated based on two data sources, namely two representative population samples, the Austrian National Election Study (AUTNES; Kritzinger et al., 2013; Kritzinger et al., 2014) and the German Longitudinal Election Study (GLES; Rattinger, Roßteutscher, Schmitt-Beck, & Weßels, 2013). Subsequently we will discuss implications for future research and the application of personality questionnaires. 3. Method 3.1. Participants and procedure The present research is based on two samples. The AUTNES sample uses data from the Austrian National Election Study (Pre- and Post-Election Survey 2013, Kritzinger et al., 2013). The survey is based on a representative random sample of the Austrian voting age population living in private households (response rate 62%). The sample was drawn using an address-based stratified design (for details see Kritzinger et al., 2014). The resulting sample was only slightly biased with regard to some socio-demographic

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

characteristics, as is often the case for population surveys (Groves, 1989). The survey was administered as a 45-min face-to-face interview (CAPI) and contains various attitude scales and demographic questions as well as a short version of the Big Five Inventory (BFI-10; Rammstedt & John, 2007). The sample consists of N = 3266 participants (51% women, 49% men). The age of participants varies between 15 and 96 years (M = 45, SD = 20). The GLES sample uses data from the German Longitudinal Election Study 2009 (GLES Short-term Campaign Panel, Rattinger et al., 2013). The target population includes all German citizens who are eligible to vote. The participants were recruited to participate in an online survey (CAWI) and the frame population is restricted to members of an Online-Access-Panel provided by Respondi AG (approximately 65,000 active panelists). Quota sampling (by age, sex, and education) was used for sampling of the frame population. The GLES study consisted of seven measurement occasions. For the purpose of the present analysis, we analyzed data from the first and sixth measurement occasions where the participants completed the BFI-10. Initially N = 4552 respondents participated in the study of which N = 2269 participants completed the first and sixth measurement occasion (2 months interval on average). Participants with extreme short response times (the lowest 10%) were excluded because such extremely short completion times often entail idiosyncratic response patterns and poor data quality. Hence, the final sample analyzed consists of N = 1999 participants (49% women, 51% men). Their age varies between 18 and 80 years (M = 43, SD = 15). 3.2. Questionnaires 3.2.1. The BFI-10 The BFI-10 (Rammstedt & John, 2007) is an abbreviated version of the Big Five Inventory (BFI; John, Donahue, & Kentle, 1991; John & Srivastava, 1999; for the German version see Lang, Lüdtke, & Asendorpf, 2001; Rammstedt, 1997). The questionnaire consists of ten items measuring neuroticism, extraversion, openness, conscientiousness, and agreeableness. Each trait is measured with two items, one positively poled (e.g., ‘‘I see myself as someone who is outgoing, sociable’’) and one negatively poled (e.g., ‘‘I see myself as someone who is reserved’’). In the GLES sample, the items of the BFI-10 are to be answered on a five-point scale ranging from ‘‘fully disagree’’ to ‘‘fully agree’’. In the questionnaire of the AUTNES sample the BFI-10 was assessed with an inverted response scale ranging from ‘‘fully agree’’ to ‘‘fully disagree’’. As previous research has shown, this inversion does not change the quality of the resulting responses (Rammstedt & Krebs, 2007). All ten items were re-coded for this analysis so that ‘‘fully disagree’’ was scored as 1 and ‘‘fully agree’’ as 5. The items are shown in Appendix A. 3.2.2. Attitude scales We analyzed two attitude scales in the AUTNES sample. The rationale for selecting the attitude scales was that the scales were balanced (containing an equal amount of positively and negatively poled items). The first scale assesses right-wing authoritarian attitudes. Three items were positively poled (e.g., ‘‘Our society for once has to crack down harder on criminals’’), three items were negatively poled (e.g., ‘‘It is important to also protect the rights of criminals’’). The second scale measures attitudes toward immigrants and Muslims with four items, two positively poled, two negatively poled. Identical to the BFI-10 the scales were to be answered on a fully labeled five-point scale ranging from ‘‘fully agree’’ to ‘‘fully disagree’’. The items and their wording are shown in Appendix A. 3.2.3. Criterion variables In the AUTNES sample, we additionally analyzed whether acquiescence biases the relation between the BFI-10 items and

121

criterion variables. In particular, the questions ‘‘When you think about immigration, do you feel very, fairly, a little or not at all worried?’’ and ‘‘When you think about immigration, do you feel very, fairly, a little or not at all confident?’’ were used as a criteria for neuroticism, the questions ‘‘Have you participated in a citizens’ initiative?’’ and ‘‘Have you ever participated in a student or youth parliament?’’ were used as a criteria for extraversion, the questions ‘‘Austria’s cultural life is enriched by immigrants’’ and ‘‘Generally speaking, are you very, fairly, a little or not at all interested in politics?’’ were used as a criteria for openness, the questions ‘‘Did you cast a ballot at the last state election?’’ and ‘‘I feel guilty when I don’t vote in an election’’ were used as a criteria for conscientiousness, and the questions ‘‘How many politicians are honest with voters? Almost all, most of them, about half, only a few or almost none?’’ and ‘‘Some people are just more valuable to society than others’’ were used as a criteria for agreeableness. These criterion variables were selected based on their face validity to illustrate that the relation between personality items and criterion variables can be biased by acquiescence. 3.3. Analysis We analyzed the data with structural equation models. We modeled acquiescence as a latent variable (as suggested by Aichholzer 2014; Billiet & McClendon, 2000; Weijters, Geuens, & Schillewaert, 2010a; Weijters et al., 2010b). In particular, the covariance matrix of the manifest variables was constrained by one latent acquiescence variable which loaded positively (+1) on all variables as well as by several latent content variables (e.g., extraversion) for which the loadings were freely estimated. The latent acquiescence variable was specified to be uncorrelated with the content variables to ensure a content-free measurement of acquiescence. In addition, each manifest variable received a residual variable (or measurement error) which was specified to be uncorrelated with all other variables. The model parameters were estimated with the full information maximum-likelihood estimator implemented in Mplus (Muthén & Muthén, 1998-2010). In addition, we investigated the relation between the criterion variables and the BFI items with linear or logistic regression models. For each BFI item, we specified a model where the criterion variable (e.g., ‘‘Did you cast a ballot at the last state election?’’) was predicted by one BFI item (e.g. ‘‘I see myself as someone who tends to be lazy’’), acquiescence, and the interaction between the BFI item and acquiescence. The acquiescence score was computed as the mean score across the ten balanced BFI items. The BFI items and the acquiescence score were z-standardized before the analysis. 4. Results 4.1. Relevance of acquiescence in personality items The relevance of acquiescence was investigated with a structural equation model and quantified as the percentage of manifest item variance that can be explained by the latent acquiescence variable. Following the terminology of Steyer and colleagues (e.g. Steyer, Schmitt, & Eid, 1999), we call this parameter acquiescence specificity. The greater the acquiescence specificity is, the more an item is inflated by acquiescence. Acquiescence specificity was evaluated based on the model shown in Fig. 1. As can be seen, this model allows for the decomposition of the variance r2 of each manifest item r2(Yi) into construct-specific variance r2(gi) (e.g., extraversion), acquiescence-specific variance r2(xi), and residual measurement error r2(ei). In particular, the acquiescence specificity (AS) or proportion of acquiescence variance for each item

122

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

η1

λ1 λ2 λ3

η2 λ4 λ5 η3 λ6 λ7 η4

λ8 λ9

η5 λ10

ε1 ε6 ε2 ε7 ε3 ε8 ε4 ε9 ε5 ε10 ε11

λ11 λ12 ε12 λ13 η6

λ14 λ15

ε13 ε14

λ16 ε15 ε16 λ17 λ18 η7

ε17 ε18

λ19 λ20

ε19 ε20

Y1 Y6 Y2 Y7 Y3 ω1 Y8 Y4 Y9 Y5 Y10 Y11 Y12 Y13 ω2 Y14 Y15 Y16 Y17 Y18 ω3 Y19 Y20

Fig. 1. Structural equation model for the personality and the attitude items. x1 = personality acquiescence, x2–3 = attitude acquiescence, g1 = extraversion, g2 = neuroticism, g3 = openness, g4 = conscientiousness, g5 = agreeableness, g6 = right-wing authoritarian attitudes, g5 = attitudes toward immigrants and Muslims, e1–20 = residuals, Y1–10 = BFI-10 items, Y11–20 = attitude items, k1–20 = loadings, unlabeled paths were set to 1. The model additionally allowed correlations between the latent personality variables and the latent attitude variables.

can be computed by AS(Yi) = r2(xi)/r2(Yi) (which is equivalent to the item’s squared standardized loading on x). We investigated the acquiescence specificity of the manifest items in the AUTNES sample. We modeled one latent acquiescence variable x1 for the personality items and two latent acquiescence variables for the attitude items, namely one latent acquiescence variable for the authoritarian items (x2) and one latent acquiescence variable for the immigrant items (x3). The specified

model is shown in Fig. 1. As can be seen, the latent acquiescence variables are specified to load positively (+1) on all manifest variables. The loadings ki of all content variables gi were estimated, whereas the loadings on the positively poled items (e.g., ‘‘I see myself as someone who is outgoing, sociable’’) were constrained to be positive and the loadings on the negatively poled items (e.g., ‘‘I see myself as someone who is reserved’’) where constrained to be negative. The model allows for correlations between

123

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130 Table 1 Acquiescence specificity and estimated model parameters of the BFI-10 and the attitude scales in the AUTNES sample. Item

Construct

1 2 3 4 5 6 7 8 9 10

Extraversion Agreeableness Conscientiousness Neuroticism Openness Extraversion Agreeableness Conscientiousness Neuroticism Openness

11 12 13 14 15 16

Authoritarianism Authoritarianism Authoritarianism Authoritarianism Authoritarianism Authoritarianism

17 18 19 20

Immigrants Immigrants Immigrants Immigrants

Construct variance (g)

Acquiescence variance (x)

Residual variance (e)

Item reliability

Acquiescence-specificity

.50 .13a .20 .33b .42 .56 .13a .73 .33b .58

.04 .04 .04 .04 .04 .04 .04 .04 .04 .04

.78 .93 .42 .74 .57 .36 .93 .47 .62 1.01

.41 .15 .36 .33 .45 .63 .15 .62 .37 .38

.03 .03 .06 .03 .04 .04 .03 .03 .04 .02

.14 .39 .38 .41 .11 .37

.03 .03 .03 .03 .03 .03

1.24 1.08 .53 .99 1.05 .94

.12 .28 .43 .31 .12 .30

.02 .02 .03 .02 .03 .02

1.04 .51 .92 1.25

.07 .07 .07 .07

.82 .75 .54 .53

.58 .44 .65 .71

.03 .05 .04 .04

.39

.03

Average 2

2

Note. N = 3266, all parameters were significant at p < .01. The construct variance was computed as r (gi) = ki . The item reliability was computed as [r2(gi) + r2(xi)]/ [r2(gi) + r2(xi) + r2(ei)]. The acquiescence specificity was computed as r2(xi)/[r2(gi) + r2(xi) + r2(ei)]. a,b Parameters set equal for model convergence.

the latent personality variables and the latent attitude variables but not with the latent acquiescence variables. The correlations between the latent personality and latent attitude variables are shown in Appendix B. The specified model showed a moderate fit, RMSEA = .06, CFI = .86, v2(145) = 1866.93, and all estimated parameters were statistically significant. Since the CFI does not explicitly incorporate the parsimoniousness of the model and the RMSEA suggests an acceptable fit, we decided to accept this model. Table 1 shows the acquiescence specificities of the items as well as the estimated model parameters. As can be seen, the variance of the latent acquiescence variable was statistically significant on all scales and accounted for 3% of each items’ variance of the manifest variables on average, ranging from 2% to 6%. This demonstrates that there are individual differences in acquiescence that systematically affect personality as well as attitude items, though to a small extent.1 We also specified a model without acquiescence variables which significantly (and substantially) impaired the model fit, RMSEA = .07, CFI = .79, v2(151) = 2718.37, Dv2(6) = 851.44, p < .001. In addition, we investigated whether acquiescence can bias the association between the manifest BFI items and manifest criterion variables. We estimated linear/logistic regression models and predicted each criterion variable (e.g., ‘‘Did you cast a ballot at the last state election?’’) by the corresponding BFI item (e.g. ‘‘I see myself as someone who tends to be lazy’’), acquiescence, and the interaction between item and acquiescence. The rationale of these regression analyses is to assess to what extent the criterion variables are associated with the personality items (effect of personality item), to what extent the criterion variables are associated with

1

As suggested by Aichholzer (2014), we also ran the analysis using unrestricted factor loadings (EFA) for the BFI items with Exploratory Structural Equation Modeling (ESEM; Asparouhov & Muthén, 2009). Generally, the model fit of ESEM is better than the model fit of standard structural equation models because ESEM allows cross-loadings between manifest items and latent factors. In line with expectations, the model fit improved, RMSEA = .05, CFI = .92, v2(123) = 1139.78, but the acquiescence specificities and the correlations between acquiescence factors were almost identical.

acquiescence (effect of acquiescence), and to what extent the effect of the personality item depends on an acquiescence bias (effect of interaction). Table 2 shows the standardized regression coefficients. As can be seen, the criterion variables can be predicted by the BFI items as well as by acquiescent responding. More importantly, the (statistically significant) interaction terms reveal that the association between the BFI items and the criterion variables also depend on acquiescence. For example, the criterion variable ‘‘Did you cast a ballot at the last state election?’’ was predicted by the personality item ‘‘I see myself as someone who tends to be lazy’’ (b = .31, p < .001). Persons who described themselves to be lazy, tended not to vote in the last election. Voting was further predicted by acquiescence (b = .12, p = .021) showing that persons who were prone to acquiescent responding in personality items, also tended to agree to the criterion more often. More importantly, voting was predicted by the interaction between acquiescence and the personality item (b = .10, p = .010) showing that the predictive value of the personality item was moderated by the respondents’ acquiescence. These results demonstrate that a positive association between the personality item and the criterion can be inflated, whereas a negative association can be suppressed by acquiescence. Generally, the relations between personality items and criterion variables reflect small to medium effects (f2 < .05). Given the rather small reliability of the personality items (.15–.63), the unknown reliability of the criterion variables, and the asymmetry of the variables (cf. Epstein, 1983; Wittmann, 1988), the sizes of these associations appear to be within the expected range. 4.2. Domain specificity of acquiescence The domain specificity of acquiescence was evaluated by the association between acquiescence in personality items and acquiescence in attitude items. As can be seen in Table 3, the correlation between the latent attitude acquiescence variables (x2 and x3) was r = .71, p <.001 which suggests consistency within the attitude domain. The correlation between the latent personality acquiescence variable (x1) and the first latent attitude acquiescence variable (x2, authoritarian attitudes) was r = .86, p < .001 and the

124

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

Table 2 Standardized regression coefficients for criterion variables regressed on the manifest BFI items, acquiescence and the interaction between BFI items and acquiescence. Criterion

N

When you think about immigration, do you feel very, fairly, a little or not at all worried?

When you think about immigration, do you feel very, fairly, a little or not at all confident?

E

Have you participated in a citizens’ initiative?b

Have you ever participated in a student or youth parliament?b

O

Austria’s cultural life is enriched by immigrants

Generally speaking, are you very, fairly, a little or not at all interested in politics?

C

Did you cast a ballot at the last state election?b

I feel guilty when I don’t vote in an election

A

How many politicians are honest with voters? Almost all, most of them, about half, only a few or almost none?

Some people are just more valuable to society than others

Item

Acquiescencea

BFI item Without acquiescence

With acquiescence

I see myself as someone who is relaxed, handles stress well I see myself as someone who gets nervous easily

.09⁄⁄⁄

.06⁄⁄

.07⁄⁄⁄

I see myself as someone who is relaxed, handles stress well I see myself as someone who gets nervous easily

Interaction

R2 Without acquiescence

With acquiescence

.07⁄⁄⁄

.06⁄⁄⁄

.01

.01

.05⁄⁄

.04⁄

.02

.00

.01

.07⁄⁄⁄

.04⁄

.10⁄⁄⁄

.03⁄

.01

.02

.08⁄⁄⁄

.05⁄

.06⁄⁄

.02

.01

.01

.27⁄⁄⁄

.32⁄⁄⁄

.20⁄⁄⁄

.00

.03

.13⁄

.27⁄⁄⁄

.06

.01

.02

.54⁄⁄

.43⁄⁄

.25⁄

.01

.05

.40⁄

.27

.05

.06

I see myself as someone who is reserved I see myself as someone who is outgoing, sociable

.09⁄⁄⁄

I see myself as someone who is reserved I see myself as someone who is outgoing, sociable

.25

.67⁄⁄

.77⁄⁄

I see myself as someone who has few artistic interests I see myself as someone who has an active imagination

.19⁄⁄⁄

.26⁄⁄⁄

.19⁄⁄⁄

.05⁄⁄

.04

.07

.16⁄⁄⁄

.14⁄⁄⁄

.08⁄⁄⁄

.01

.00

.03

I see myself as someone who has few artistic interests I see myself as someone who has an active imagination

.13⁄⁄⁄

.17⁄⁄⁄

.12⁄⁄⁄

.06⁄⁄

.02

.04

.13⁄⁄⁄

.12⁄⁄⁄

.05⁄⁄

.00

.02

.02

I see myself as someone who tends to be lazy I see myself as someone who does a thorough job

.22⁄⁄⁄

.31⁄⁄⁄

.12⁄

.10⁄

.01

.02

.35⁄⁄⁄

.35⁄⁄⁄

.00

.01

.02

.02

I see myself as someone who tends to be lazy I see myself as someone who does a thorough job

.04⁄

I see myself as someone who is generally trusting I see myself as someone who tends to find fault with others I see myself as someone who is generally trusting I see myself as someone who tends to find fault with others

.20⁄⁄⁄

.13⁄⁄⁄

.18⁄⁄⁄

.04⁄⁄

.00

.03

.12⁄⁄⁄

.11⁄⁄⁄

.13⁄⁄⁄

.04⁄

.02

.03

.09⁄⁄⁄

.03

.16⁄⁄⁄

.02

.01

.03

.01

.10⁄⁄⁄

.15⁄⁄⁄

.16⁄⁄⁄

.00

.07

.01

.03

.06⁄⁄

.04⁄

.00

.01

.02

.04⁄⁄

.00

.01

.06⁄⁄

.04⁄

Note. AUTNES sample N = 3266, ⁄ p < .05, ⁄⁄ p < .01, ⁄⁄⁄ p < .001, N = Neuroticism, E = Extraversion, O = Openness, C = Conscientiousness, A = Agreeableness. a The composite reliability (Raykov & Shrout, 2002) of the manifest acquiescence score was .36. b Logistic regression with Pseudo R2.

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130 Table 3 Correlations between the manifest and the latent acquiescence variables. Correlation between acquiescence in

Correlation between manifest variables

Correlation between latent variables

BFI scale – authoritarianism scale Authoritarianism scale – immigrant scale Immigrant scale – BFI scale

.22⁄⁄⁄ .14⁄⁄⁄

.86⁄⁄⁄ .71⁄⁄⁄

.26⁄⁄⁄

.79⁄⁄⁄

⁄⁄⁄

Note. p < .001. The composite reliability (CR) of the authoritarianism acquiescence scale was .16. The composite reliability of the immigrants acquiescence scale was .33. The composite reliability of the BFI acquiescence scale was .36.

125

correlation between the latent personality acquiescence variable and the second latent attitude acquiescence variable (attitudes toward immigrants) was r = .79, p < .001 thus indicating a strong consistency across domains. Therefore we also specified a model with only one acquiescence variable, a general acquiescence factor, across all personality and attitude items. This model similarly fit the data, RMSEA = .06, CFI = .86, v2(150) = 1901.42. Even though the v2 difference test suggests that the original model with three acquiescence factors was slightly more appropriate for the available data, Dv2(5) = 34.49, p < .001, the difference in RMSEA and CFI suggest that both models fit the data equally well (Cheung & Rensvold,

Fig. 2. Structural equation model for the personality items at two measurement occasions. Y101–110 = BFI-10 items at measurement occasion 1, Y201–210 = BFI-10 items at measurement occasion 2, x1 = personality acquiescence at the first measurement occasion, x2 = personality acquiescence at the second measurement occasion, g1–g5 = Big Five trait factors, t1–5 = method factors, e101–210 = residuals, k1–10 = loadings, unlabeled paths were set to 1. The model additionally allowed correlations between the latent personality variables.

126

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

Table 4 Acquiescence specificity and estimated model parameters of the BFI-10 in the GLES sample. Item

Measurement occasion

Construct

Construct variance (g)

Acquiescence trait variance (n)

Acquiescence state residual variance (f)

Method variance (t)

Residual variance (e)

Item reliability

Acquiescence specificity

1 2 3 4 5 6 7 8 9 10

1 1 1 1 1 1 1 1 1 1

Extraversion Agreeableness Conscientiousness Neuroticism Openness Extraversion Agreeableness Conscientiousness Neuroticism Openness

.69 .02 .21 .29 .42 .45 .42 .25 .61 .33

.02 .02 .02 .02 .02 .02 .02 .02 .02 .02

.02 .02 .02 .02 .02 .02 .02 .02 .02 .02

.00 .60 .16 .28 .00 .26 .00 .00 .00 .60

.45 .55 .28 .49 .37 .33 .38 .42 .48 .55

.62 .54 .48 .55 .56 .69 .55 .66 .57 .64

.03 .03 .07 .03 .04 .03 .04 .03 .03 .02

1 2 3 4 5 6 7 8 9 10

2 2 2 2 2 2 2 2 2 2

Extraversion Agreeableness Conscientiousness Neuroticism Openness Extraversion Agreeableness Conscientiousness Neuroticism Openness

.69 .02 .21 .29 .42 .45 .42 .25 .61 .33

.02 .02 .02 .02 .02 .02 .02 .02 .02 .02

.02 .02 .02 .02 .02 .02 .02 .02 .02 .02

.00 .60 .16 .28 .00 .26 .00 .00 .00 .60

.42 .49 .28 .35 .33 .26 .31 .39 .41 .46

.64 .57 .47 .63 .59 .74 .59 .67 .61 .68

.03 .03 .06 .04 .04 .03 .04 .03 .03 .02

.60

.04

Average

Note. N = 1999, all parameters were significant at p < .01. The construct variance was computed as r2(gi) = ki2. The item reliability was computed as [r2(gi) + r2(ni) + r2(fi) + r2(ti)]/[r2(gi) + r2(ni) + r2(fi) + r2(ti) + r2(ei)]. The acquiescence specificity was computed as [r2(ni) + r2(fi)]/[r2(gi) + r2(ni) + r2(fi) + r2(ti) + r2(ei)].

2002). Since the v2 difference test depends on the sample size and a statistical significant difference is not synonymous with a meaningful difference, the present results suggest that acquiescence is very consistent across domains. We also computed the correlations between the manifest acquiescence variables (i.e., simple composites) and compared them with the correlations between the latent variables (see Table 3). As expected, the manifest correlations were generally smaller, given the low composite reliability estimates. 4.3. Stability of acquiescence The stability2 of acquiescence was investigated with a latent state–trait model (Steyer et al., 1999). The corresponding model is shown in Fig. 2. As can be seen, there were ten manifest items at the first measurement occasion (Y101–Y110) and ten items at the second measurement occasion (Y201–Y210). We modeled a latent acquiescence variable for the first measurement occasion (x1) and a latent acquiescence variable for the second measurement occasion (x2). The two latent acquiescence variables were specified to load positively (+1) on all manifest variables. The latent acquiescence variables were decomposed into an acquiescence trait variable (n), a state residual for the first measurement occasion (f1), and a state residual for the second measurement occasion (f2). The loadings on the acquiescence trait factor were set to +1. In addition, we modeled latent trait variables for extraversion (g1), agreeableness (g2), conscientiousness (g3), neuroticism (g4), and openness (g5) with variances set to 1. The loadings ki of the BFI-10 items were freely estimated whereas each item was specified to have the same factor loading at both measurement occasions. In addition, we specified residuals e101–210 for each manifest variable and five method factors (t1–5) that load positively (+1) on one item per trait (at both measurement 2 Steyer et al. (1999) call the variance proportion that is stable over time and consistent across situations consistency. To avoid confusion with the domain-specificity of acquiescence, we call the variance proportion that is stable over time and consistent across personality traits stability.

occasions) to account for item specific aspects that affected the first and the second measurement occasion (e.g., Eid, Lischetzke, Nussbeck, & Trierweiler, 2003). The method factors reflect individual differences of the particular items that are not shared by the latent trait.3 The acquiescence trait variable (n), the acquiescence state residuals (f1 and f2), the method factors (t1–5), and the item residuals e101–210 were specified to be uncorrelated with all other variables. The latent personality trait variables were allowed to correlate with each other. We investigated the stability of acquiescence in the GLES sample. The specified model fit the data acceptably, RMSEA = .05, CFI = .93, v2(162) = 996.19, and all estimated parameters were statistically significant. The model parameters are shown in Table 4. As can be seen, the variance of the latent acquiescence trait variable r2(n) was .02 and the variances of the latent state residuals r2(f1) and r2(f2) were .02. Using the proportion of trait to state variance, i.e., r2(n)/[r2(n) + r2(f)], this suggests that 43–44% of the total acquiescence variance is stable over time. In addition, we computed the reliability and the acquiescence specificity of the BFI-10 items. As can be seen, the acquiescence state specificities in the GLES sample were similar to those in the AUTNES sample (4% on average).

4.4. Relation to educational attainment There was no statistically significant relation between acquiescence in the BFI (manifest score) and educational group in the AUTNES sample (four educational groups ranging from ‘‘compulsory school, without leaving certificate’’ to ‘‘university(-related) degree’’), F = 1.07, p = .360, g2 = .001, or in the GLES sample (five educational groups ranging from ‘‘compulsory school, without leaving certificate’’ to ‘‘higher qualification, entrance qualification for universities’’), F = 1.26, p = .283, g2 = .003.

3 We first included 10 separate method factors for each indicator and then excluded those 5 method factors with non-significant variances.

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

5. Discussion The presented study aimed to investigate (a) to what extent acquiescence is relevant in assessing personality, (b) if acquiescence is a general or a domain specific construct, and (c) if acquiescence is stable over time. To evaluate the relevance of acquiescence in personality questionnaires we analyzed data of a short instrument assessing the Big Five in two large scale studies representative of the adult population of the two countries. The results suggest that acquiescence accounts for about 4% of the variance of personality items, which is similar to what previous research has found (e.g., Aichholzer, 2014; Billiet & McClendon, 2000; Rorer & Goldberg, 1965). At first glance, this number suggests only a minor influence of acquiescence on personality items. However, even such a minor variance component can impair model fit, the association between personality items and criterion variables, or can bias factor loading estimates. This complements the findings reported by Bentler et al. (1971), Rammstedt and colleagues (Rammstedt & Farmer, 2013; Rammstedt & Kemper, 2011; Rammstedt et al., 2010), and Aichholzer (2014) who have shown that acquiescence can substantially bias the correlations between items and the factorial structure of established personality inventories. More importantly, acquiescence can not only cause biased scores on an item level, but also on a scale level. This is of particular importance for scales containing only positively poled items, because the acquiescence specific variance proportion increases on scale level. For example, the acquiescence specificity of a scale containing ten positively poled items would increase from 5% to 7%.4 Taken together, the present findings tie in with previous research and suggest that acquiescence systematically biases personality questionnaires. The second aim of the study was to investigate the consistency of acquiescence within and across domains. For one thing, the present results demonstrate that acquiescence is consistent within domains. In line with previous research (Billiet & McClendon, 2000; Ferrando et al., 2004; Hinz et al., 2007) our findings indicate a high latent correlation (r = .71) between the attitude acquiescence variables and thus demonstrates that acquiescence is a response style that is not specific to certain items or scales. Furthermore, the present results suggest that acquiescence is also strongly consistent across domains. The correlation between acquiescence in personality items and acquiescence in attitude items was comparable in size to that within the domain of attitude items (r P .79). Previous studies (Gage et al., 1956; Ray, 1983) also reported positive but smaller correlation between acquiescence in different domains. However, those studies reported correlations between manifest variables which can be biased by their potential unreliability and, hence, the size of those correlations cannot be interpreted properly. Consistent with this conjecture, we found small reliabilities of the manifest acquiescence scores (.16–.36) which substantially decreased the association between the manifest acquiescence variables (cf. Table 3). This may be explained by the small number of items we used for this analysis, because more indicators will generally give a more accurate picture of

4 In the present example, there are ten positively poled items and the variance of each item r2(Y) can be decomposed into construct variance r2(g), acquiescence variance r2(x), and measurement error r2(e), i.e. r2(Y) = r2(g) + r2(x) + r2(e). Assuming that the construct variance of each items is r2(g) = 0.40, the acquiescence variance is r2(x) = 0.04, and the residual item variance is r2(e) = 0.36, the acquiescence specificity of each item is r 2 (x )/[r 2 (g ) + r 2(x ) + r 2( e)] = 0.04/ (0.40 + 0.36 + 0.04) = 0.05. The variance of the ten items’ sum sore can be decomposed in the sum of the variances (r2) and the sum of the covariances (q) of P P P P 2 these components, r2sum = 10 r2 ðg Þþ 10 r2 ðx Þ þ 10 r2 ðei Þ þ 2 10 i¼1 r ðgi ; gi Þþ P10 2 P10 2 i¼1 Pi 10 2 i¼1 Pi 10 2 i¼1 2 i¼1 r ðxi ; xi Þ ¼ 3 i¼1 r ðgi Þ þ 3 i¼1 r ðxi Þ þ i¼1 r ðei Þ = 3 ⁄ 10 ⁄ 0.40 + 3 ⁄ 10 ⁄ 0.04 + 10 ⁄ 0.36 = 16.80. Thus, the acquiescence specificity of the sum score is P (3 10 i¼1 xi /rsum = (3 ⁄ 10 ⁄ 0.04)/16.80 = 0.07.

127

behavioral tendencies (cf. Epstein, 1983), or by the rather small acquiescence specificity of the items. Therefore, we analyzed the association between latent variables which are adjusted for measurement error. In summary, our study extends previous research and suggests that acquiescence is a general response style across different domains such as personality questionnaires and attitude scales. This also coincides with the literature investigating the relation between acquiescence and cognitive ability which demonstrates that cognitive ability is negatively related with acquiescence in attitude items and with acquiescence in personality items (Gudjonsson, 1986; Gudjonsson & Clare, 1995; Gudjonsson & Young, 2001). The third aim of the present study was to investigate the stability of acquiescence in personality questionnaires. The results of the latent state–trait analysis suggest that acquiescence has trait-like, as well as state-like properties. Individual differences in acquiescence are triggered by the characteristics of the respondent as well as by the characteristics of the particular measurement occasion, a finding that concurs with the assumption that response styles can vary with the particular survey situation (see Aichholzer, 2013). This suggests that individual characteristics such as cognitive ability or personality, as well as situational characteristics such as fatigue or learning effects must be taken into account when the causes of acquiescence responding are investigated. Previous research has already demonstrated that acquiescence is negatively related with cognitive ability (e.g., Elliott, 1961; Gudjonsson, 1986; Gudjonsson & Clare, 1995; Gudjonsson & Young, 2001; Zhou & McClendon, 1999), educational attainment (e.g., Billiet & McClendon, 2000; Mirowsky & Ross, 1991; Weijters et al., 2010a), and personality variables such as rigid mental organization as well as lower tolerance and cognitive complexity (Knowles & Nathan, 1997). Given that these constructs represent stable characteristics of the respondents, these variables may in particular explain trait differences in acquiescence. In addition, the work of Krosnick and colleagues (e.g., Krosnick, 1991; Krosnick & Presser, 2010) suggests that acquiescence can also be triggered by situational characteristics like motivation or fatigue effects which may in particular explain the state differences in acquiescence. Clearly, in order to fully understand the causes of acquiescent responding, both perspectives have to be taken into account. A fertile approach for future research will be investigating whether these factors are causally or correlatively related and whether there are mediating or interactive relations between these factors. 5.1. Controlling acquiescence Personality questionnaires are biased by acquiescence. This can substantially bias correlations between items (Bentler et al., 1971), factor structures of personality inventories (Aichholzer, 2014; Rammstedt & Farmer, 2013), comparability of personality assessments across countries (Rammstedt, Kemper, & Borg, 2013) and (as shown in the present research) the suitability of underlying measurement models, and bias the correlation between personality items and criterion variables. Therefore, acquiescence should not be ignored (cf. Savalei & Falk, 2014). As indicated by this research, acquiescence bias can be controlled if balanced scales are used. Balanced scales contain an equal amount of positively poled items (e.g., ‘‘I see myself as someone who is sociable, outgoing’’) and negatively poled items (e.g., ‘‘I see myself as someone who is reserved’’). On item level, acquiescence in balanced scales can be controlled using structural equation modeling or ipsative transformation. Using structural equation modeling allows for decomposition of the variance of a manifest variable into construct variance (e.g., extraversion), acquiescence variance, and residual measurement error variance. This permits analyzing the relationship between latent variables that are adjusted for acquiescence

128

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

and measurement error. In particular, the structural equation modeling approach by Billiet and McClendon (2000) provides a promising way that has already been applied in other studies (e.g., Aichholzer, 2014; Billiet & Davidov, 2008). An alternative approach to control for acquiescence on item level can be ipsative transformation of the items’ raw scores. Ipsative transformation means subtracting the mean score of a balanced scale from each item’s raw score (e.g., the mean score of the BFI-10 from each BFI-10 item). The resulting scores are adjusted for acquiescence and thus can be interpreted as more valid indicators of the underlying construct (e.g., Brown & Maydeu-Olivares, 2011; Rammstedt et al., 2010; Soto, John, Gosling, & Potter, 2008). In samples that are too small for structural equation modeling (e.g., Bollen, 1989; Hoyle, 1995), ipsative transformation can be a very useful approach. However, the low reliabilities of the manifest acquiescence variables suggest that this approach is limited, especially if only a small number of items can be used for analyses. On scale level, acquiescent responding can be controlled for by recoding negatively poled items before computing the mean score across items. By definition, acquiescence would increase the scores of positively poled items and decrease the scores of negatively poled items. Thus the mean score across all items is adjusted for acquiescence. If non-balanced scales are used, it may be reasonable to adjust individuals’ test scores for acquiescent responding. This can be done by increasing the confidence interval of an individual’s test score. For example, a person may complete a personality questionnaire and obtain a T-value of 55 for extraversion. Taking a reliability estimate (R) of 0.80, the standard deviation (SD) of 10 for T-values, and an acquiescence specificity (AS) of 0.05 allows computing an acquiescence-adjusted confidence interval pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi (95%) by, CI95% = 55 ± 1.96 ⁄ SD ⁄ 1  ðR  ASÞ = 55 ± 1.96 ⁄ 10 ⁄ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1  ð0:80  0:05Þ = 55 ± 5.00. It has also been suggested to use control scales (e.g., Block, 1965; Handel et al., 2010; Tellegen, 1988) or person fit statistics (e.g., Ferrando & Lorenza-Seva, 2010; Tellegen, 1988) to identify highly acquiescent respondents. In individual assessments, this allows identifying persons whose responses are most likely biased by acquiescence. On large scale assessments, this permits the exclusion of highly acquiescent responders from the analysis. However, excluding data always leads to a loss of information. Thus, using balances scales and controlling acquiescence with structural equation modeling or ipsative transformation appears to be a more promising approach in practice.

across different domains, and (c) acquiescence in personality questionnaires is only moderately stable over time. The generalizability of these results, however, depends on the investigated samples and variables, of course. The AUTNES and GLES samples used for evaluating the relevance and the domain consistency of acquiescence are large samples representative for the Austrian and German population, respectively, which strengthens the generality of the present findings. The sample of variables on the other hand may be regarded as limited. Even though the BFI-10 was thoroughly developed, is frequently used, and has been cited over 500 times, it cannot be ruled out that the present findings are limited by the specific selection of variables. Likewise, analyzing data of the AUTNES and the GLES sample allowed us to investigate only two balanced attitude scales and no balanced scales of other domains (such as knowledge items) which is an additional limitation. We hope that the present study lays the foundation for future research that will extend the analyses to other scales and other item domains. 6. Conclusion The present findings suggest that acquiescence accounts for about 4% of the variance of personality items and can bias the association with other variables. Together with previous research this suggests that acquiescence distorts the psychometric properties of personality assessments. In addition, acquiescence appears to have trait-like as well as state-like properties. It can thus be assumed that the tendency for acquiescence is determined by the characteristics of the respondent as well as by occasion specific effects. This emphasizes the usefulness of identifying personality variables as well as situational effects that are associated with acquiescence. We hope that the present analyses will stimulate future research and help to gain a deeper understanding of this response style. Appendix A. See Tables A1 and A2. Appendix B. See Tables B1 and B2.

5.2. Limitations

Appendix C. Supplementary material

In sum, the present study showed that (a) acquiescence systematically affects personality items, (b) acquiescence is consistent

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jrp.2015.05.004.

Table A1 BFI-10 items. Item number

Construct

Item

1 2 3 4 5 6 7 8 9 10

Extraversion Agreeableness Conscientiousness Neuroticism Openness Extraversion Agreeableness Conscientiousness Neuroticism Openness

I I I I I I I I I I

see see see see see see see see see see

Polarity myself myself myself myself myself myself myself myself myself myself

as as as as as as as as as as

someone someone someone someone someone someone someone someone someone someone

who who who who who who who who who who

is reserved is generally trusting does a thorough job is relaxed, handles stress well has an active imagination is outgoing, sociable tends to find fault with others tends to be lazy gets nervous easily has few artistic interests

 + +  + +   + 

129

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130 Table A2 AUTNES 2013 attitude items. Item number

Construct

Item

Polarity

11 12 13 14 15 16 17 18 19 20

Authoritarian Authoritarian Authoritarian Authoritarian Authoritarian Authoritarian Anti-immigrant Anti-immigrant Anti-immigrant Anti-immigrant

We should be grateful for leaders, that tell us exactly what we shall do and how The age in which discipline and obedience for authority are some of the most important virtues should be over Our society for once has to crack down harder on criminals It is important to also protect the rights of criminals Our country needs people who oppose traditions and try out different ideas This country would flourish if young people paid more attention to traditions and values Due to many Muslims living in Austria I sometimes feel like a stranger in my own country European lifestyle and the lifestyle of Muslims are easily compatible Austria’s cultural life is enriched by immigrants Immigration to Austria must be stopped

+  +   + +   +

Table B1 Correlation between latent personality and attitude variables in the AUTNES sample. A E A C N O AU

C

N ⁄⁄⁄

.03

.18 .31⁄⁄⁄

O ⁄⁄⁄

.39 .15⁄⁄ .29⁄⁄⁄

AU ⁄⁄⁄

IM ⁄⁄

.32 .28⁄⁄⁄ .13⁄⁄⁄ .19⁄⁄⁄

.09 .21⁄⁄⁄ .24⁄⁄⁄ .07⁄ .46⁄⁄⁄

.06⁄ .27⁄⁄⁄ .06⁄ .05 .35⁄⁄⁄ .63⁄⁄⁄

Note. ⁄ p < .05, ⁄⁄⁄ p < .001, E = Extraversion, A = Agreeableness, C = Conscientiousness, N = Neuroticism, O = Openness, AU = Authoritarian attitudes, IM = Anti-immigrant attitudes. The correlations were estimated based on the model shown in Fig. 1, N = 3266.

Table B2 Correlation between latent personality variables in the GLES sample.

E A C N

A

C

N

O

.21⁄⁄⁄

.26⁄⁄⁄ .16⁄⁄⁄

.46⁄⁄⁄ .11⁄⁄⁄ .38⁄⁄⁄

.38⁄⁄⁄ .01 .25⁄⁄⁄ .31⁄⁄⁄

Note. ⁄⁄ p < .01, ⁄⁄⁄ p < .001, E = Extraversion, A = Agreeableness, C = Conscientiousness, N = Neuroticism, O = Openness, The correlations were estimated based on the model shown in Fig. 2. N = 1999.

References Aichholzer, J. (2013). Intra-individual variation of extreme response style in mixedmode panel studies. Social Science Research, 42(3), 957–970. http://dx.doi.org/ 10.1016/j.ssresearch.2013.01.002. Aichholzer, J. (2014). Random intercept EFA of personality scales. Journal of Research in Personality, 53, 1–4. http://dx.doi.org/10.1016/j.jrp.2014.07.001. Amelang, M., & Borkenau, P. (1981). Untersuchungen zur Validität von KontrollSkalen für Soziale Erwünschtheit und Akquieszenz. Diagnostica, 27(4), 295–312. Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16(3), 397–438. http://dx.doi.org/10.1080/ 10705510903008204. Bentler, P. M., Jackson, D. N., & Messick, S. (1971). Identification of content and style: A two-dimensional interpretation of acquiescence. Psychological Bulletin, 76(3), 186–204. http://dx.doi.org/10.1037/h0031474. Billiet, J. B., & Davidov, E. (2008). Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociological Methods & Research, 36(4), 542–562. http://dx.doi.org/10.1177/ 0049124107313901. Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7(4), 608–628. http://dx.doi.org/10.1207/S15328007SEM0704_5. Block, J. (1965). The challenge of response sets: Unconfounding meaning, acquiescence, and social desirability in the MMPI. East Norwalk, CT: Appleton-Century-Crofts. Blumberg, H. H. (1973). The specificity of acquiescence. Journal of Personality Assessment, 37(5), 479–485. http://dx.doi.org/10.1080/ 00223891.1973.10119907. Bollen, K. A. (1989). A new incremental fit index for general structural equation models. Sociological Methods & Research, 17(3), 303–316. http://dx.doi.org/ 10.1177/0049124189017003004. Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71(3), 460–502. http://dx.doi.org/10.1177/0013164410375112.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(29), 233–255. http://dx.doi.org/10.1207/S15328007SEM0902_5. Cronbach, L. J. (1942). Studies of acquiescence as a factor in the true-false test. Journal of Educational Psychology, 33(6), 401–415. http://dx.doi.org/10.1037/ h0054677. DiStefano, C., Morgan, G. B., & Motl, R. W. (2012). An examination of personality characteristics related to acquiescence. Journal of Applied Measurement, 13(1), 41–56. Eid, M., Lischetzke, T., Nussbeck, F. W., & Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M1) Model. Psychological Methods, 8(1), 38–60. http://dx.doi.org/10.1037/1082-989X.8.1.38. Elliott, L. L. (1961). Effects of item construction and respondent aptitude on response acquiescence. Educational and Psychological Measurement, 21, 405–415. Epstein, S. (1983). Aggregation and beyond: Some basic issues in the prediction of behavior. Journal of Personality, 51(3), 360–392. http://dx.doi.org/10.1111/ j.1467-6494.1983.tb00338.x. Ferrando, P. J., Condon, L., & Chico, E. (2004). The convergent validity of acquiescence: An empirical study relating balanced scales and separate acquiescence scales. Personality and Individual Differences, 37(7), 1331–1340. http://dx.doi.org/10.1016/j.paid.2004.01.003. Ferrando, P. J., & Lorenza-Seva, U. (2010). Acquiescence as a source of bias and model and person misfit: A theoretical and empirical analysis. British Journal of Mathematical and Statistical Psychology, 63(2), 427–448. http://dx.doi.org/ 10.1348/000711009X470740. Gage, N. L., Leavitt, G. S., & Stone, G. C. (1956). The intermediary key in the analysis of interpersonal perception. Psychological Bulletin, 53(3), 258–266. http:// dx.doi.org/10.1037/h0042729. Gana, K., Saada, Y., Bailly, N., Joulain, M., Hervé, C., & Alaphilippe, D. (2013). Longitudinal factorial invariance of the Rosenberg Self-Esteem Scale: Determining the nature of method effects due to item wording. Journal of

130

D. Danner et al. / Journal of Research in Personality 57 (2015) 119–130

Research in Personality, 47(4), 406–416. http://dx.doi.org/10.1016/ j.jrp.2013.03.011. Groves, R. M. (1989). Survey errors and survey costs. New York: John Wiley. Gudjonsson, G. H. (1986). The relationship between interrogative suggestibility and acquiescence: Empirical findings and theoretical implications. Personality and Individual Differences, 7(2), 195–199. http://dx.doi.org/10.1016/01918869(86)90055-3. Gudjonsson, G. H., & Clare, I. C. H. (1995). The relationship between confabulation and intellectual ability, memory, interrogative suggestibility, and acquiescence. Personality and Individual Differences, 19(3), 333–338. Gudjonsson, G. H., & Young, S. (2001). Personality and deception. Are suggestibility, compliance and acquiescence related to socially desirable responding? Personality and Individual Differences, 50(2), 192–195. http://dx.doi.org/ 10.1016/j.paid.2010.09.024. Handel, R. W., Ben-Porath, Y. S., Tellegen, A., & Archer, R. P. (2010). Psychometric functioning of the MMPI-2-RF VRIN-r and TRIN-r scales with varying degrees of randomness, acquiescence, and counter-acquiescence. Psychological Assessment, 22(1), 87–95. http://dx.doi.org/10.1037/a0017061. Hinz, A., Michalski, D., Schwarz, R., & Herzberg, P. Y. (2007). The acquiescence effect in responding to a questionnaire. GMS Psycho-Social-Medicine, 4, 1–9. Hoyle, R. H. (1995). Structural equation modeling: Concepts, issues, and applications. Thousand Oaks: Sage Publications. John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The big five inventory – versions 4a and 54. Berkeley, CA: Institute for Personality and Social Research. John, O. P., & Srivastava, S. (1999). The Big Five Trait taxonomy: History, measurement, and theoretical perspectives. In L. A. Pervin & O. P. John (Eds.), Handbook of personality: Theory and research (2 ed, pp. 102–138). New York: Guilford Press. Knowles, E. S., & Nathan, K. T. (1997). Acquiescent responding in self-reports: Cognitive style or social concern? Journal of Research in Personality, 31(2), 293–301. http://dx.doi.org/10.1006/jrpe.1997.2180. Kritzinger, S., Zeglovits, E., Aichholzer, J., Glantschnigg, C., Glinitzer, K., Johann, D., et al. (2013). AUTNES Pre- and Post-Election Survey 2013. GESIS Data Archive: Cologne (ZA5859 Data File Version 1.0.0). Kritzinger, S., Zeglovits, E., Aichholzer, J., Glantschnigg, C., Glinitzer, K., Johann, D., et al. (2014). AUTNES pre- and post-election survey 2013 – Documentation. Vienna: University of Vienna. Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213–236. http://dx.doi.org/10.1002/acp.2350050305. Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–313). Bingley Emerald. Lang, F. R., Lüdtke, O., & Asendorpf, J. B. (2001). Testgüte und psychometrische Äquivalenz der deutschen Version des Big Five Inventory (BFI) bei jungen, mittelalten und alten Erwachsenen. Diagnostica, 47(3), 111–121. http:// dx.doi.org/10.1026//0012-1924.47.3.111. Marsh, H. W., Scalas, L. F., & Nagengast, B. (2010). Longitudinal tests of competing factor structures for the Rosenberg self-esteem scale: Traits, ephemeral artefacts, and stable response styles. Psychological Assessment, 22(2), 366–381. http://dx.doi.org/10.1037/a0019225. McBride, L., & Moran, G. (1967). Double agreement as a function of item ambiguity and susceptibility to demand implications of the psychological situation. Journal of Personality and Social Psychology, 6(1), 115–118. http://dx.doi.org/10.1037/ h0024533. McCrae, R. R., Herbst, J. H., & Costa, P. T. J. (2001). Effects of acquiescence on personality factor structures. In F. O. R. Riemann & F. Spinath (Eds.), Personality and temperament: Genetics, evolution, and structure (pp. 217–231). Berlin: Pabst Science Publishers. Mirowsky, J., & Ross, C. E. (1991). Eliminating defense and agreement bias from measures of the sense of control: A 22 index. Social Psychology Quarterly, 54(2), 127–145. http://dx.doi.org/10.2307/2786931. Motl, R. W., & DiStefano, C. (2002). Longitudinal invariance of self-esteem and method effects associated with negatively worded items. Structural Equation Modeling, 9(4), 562–578. http://dx.doi.org/10.1207/s15328007sem0904_6. Muthén, L. K., & Muthén, B. O. (1998-2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). San Diego, CA US: Academic Press. Rammstedt, B. (1997). Die deutsche Version des Big Five Inventory (BFI): Übersetzung und Validierung eines Fragebogens zur Erfassung des FünfFaktoren-Modells der Persönlichkeit. Unveröffentlichte Diplomarbeit. Bielefeld, Germany: Universität Bielefeld.

Rammstedt, B., & Farmer, R. F. (2013). The impact of acquiescence on the evaluation of personality structure. Psychological Assessment. http://dx.doi.org/10.1037/ a0033323. Rammstedt, B., Goldberg, L. R., & Borg, I. (2010). The measurement equivalence of Big-Five factor markers for persons with different levels of education. Journal of Research in Personality, 44(1), 53–61. http://dx.doi.org/10.1016/ j.jrp.2009.10.005. Rammstedt, B., & John, O. P. (2007). Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. Journal of Research in Personality, 41(1), 203–212. http://dx.doi.org/10.1016/ j.jrp.2006.02.001. Rammstedt, B., & Kemper, C. J. (2011). Measurement equivalence of the Big Five: Shedding further light on potential causes of the educational bias. Journal of Research in Personality, 45(1), 121–125. http://dx.doi.org/10.1016/ j.jrp.2010.11.006. Rammstedt, B., Kemper, C., & Borg, I. (2013). Correcting big five measurements for acquiescence: An 18-country cross-cultural study. European Journal of Personality, 27(1), 71–81. http://dx.doi.org/10.1002/per.1894. Rammstedt, B., & Krebs, D. (2007). Does response scale format affect the answering of personality scales? Assessing the Big Five dimensions of personality with different response scales in a dependent sample. European Journal of Psychological Assessment, 23(1), 32–38. http://dx.doi.org/10.1027/10155759.23.1.32. Rattinger, H., Roßteutscher, S., Schmitt-Beck, R., & Weßels, B. (2013). Short-term campaign panel (GLES 2009). GESIS Data Archive: Cologne (ZA5305 Data file Version 4.0.0). Ray, J. J. (1983). Reviving the problem of acquiescent response bias. The Journal of Social Psychology, 121(1), 81–96. http://dx.doi.org/10.1080/ 00224545.1983.9924470. Raykov, T., & Shrout, P. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9(2), 195–212. http://dx.doi.org/10.1207/ S15328007SEM0902_3. Rorer, L. G., & Goldberg, L. R. (1965). Acquiescence in the MMPI? Educational and Psychological Measurement, 25(3), 801–817. http://dx.doi.org/10.1177/ 001316446502500311. Savalei, V., & Falk, C. F. (2014). Recovering substantive factor loadings in the presence of acquiescence bias: A comparison of three approaches. Multivariate Behavioral Research, 49(5), 407–424. http://dx.doi.org/10.1080/ 00273171.2014.931800. Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2008). The developmental psychometrics of big five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20. Journal of Personality and Social Psychology, 94(4), 718–737. http://dx.doi.org/10.1037/00223514.94.4.718. Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state–trait theory and research in personality and individual differences. European Journal of Personality, 13(5, Spec Issue), pp. 389–408, doi: 10.1002/(SICI)1099-0984(199909/ 10)13:5<389::AID-PER361>3.0.CO;2-A. Tellegen, A. (1988). The analysis of consistency in personality assessment. Journal of Personality, 56(3), 621–663. Trott, D. M., & Jackson, D. N. (1967). An experimental analysis of acquiescence. Journal of Experimental Research in Personality, 2(4), 278–288. Weems, G. H., Onwuegbuzie, A. J., Schreiber, J. B., & Eggers, S. J. (2003). Characteristics of respondents who respond differently to positively and negatively worded items on rating scales. Assessment & Evaluation in Higher Education, 28(6), 587–606. http://dx.doi.org/10.1080/0260293032000130234. Weijters, B., Geuens, M., & Schillewaert, N. (2010a). The individual consistency of acquiescence and extreme response style in self-report questionnaires. Applied Psychological Measurement, 34(2), 105–121. http://dx.doi.org/10.1177/ 0146621609338593. Weijters, B., Geuens, M., & Schillewaert, N. (2010b). The stability of individual response styles. Psychological Methods, 15(1), 96–110. http://dx.doi.org/ 10.1037/a0018721. Winkler, J. D., Kanouse, D. E., & Ware, J. E. (1982). Controlling for acquiescence response set in scale development. Journal of Applied Psychology, 67(5), 555–561. http://dx.doi.org/10.1037/0021-9010.67.5.555. Wittmann, W. W. (1988). Multivariate reliability theory: Principles of symmetry and successful validation. In J. R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (pp. 505–560). New York: Plenum Press. Zhou, B., & McClendon, M. J. (1999). Cognitive ability and acquiescence. In American Statistical Association (Eds.), Proceedings of the survey research methods section (pp. 1003–1012). Alexandria.