The Multidimensional Nature and Multicultural Validity of a New Measure of Social Anxiety: The Social Anxiety Questionnaire for Adults

Available online at www.sciencedirect.com

Behavior Therapy 43 (2012) 313 – 328

www.elsevier.com/locate/bt

The Multidimensional Nature and Multicultural Validity of a New Measure of Social Anxiety: The Social Anxiety Questionnaire for Adults Vicente E. Caballo Isabel C. Salazar University of Granada

María Jesús Irurtia Benito Arias University of Valladolid

Stefan G. Hofmann Boston University CISO-A Research Team This study was funded by a grant from Spain's Ministry of Science and Technology awarded to the research project with reference BSO2003-07029/PSCE and cofinanced by the European Regional Development Fund (ERDF). Financial assistance from the Foundation for the Advancement of Behavioral Clinical Psychology (Funveca) is also acknowledged. Stefan G. Hofmann is supported by a grant from the National Institute of Mental Health (MH0078308) and is a consultant for Organon. We gratefully appreciate the collaboration of all those participating in the studies from the various countries. The CISO-A Research Team, co-author of this article, comprises the following researchers: Argentina: M. Correche, L. Gómez, F. Rivarola, P. Robles, S. Savoini, M. Tapia; Bolivia: D. Pinelo; Brazil: B. Donato, R. Lopes, L. Lourenço, M. Oliveira, C. Neufeld, M. Wagner; Chile: C. Guerra, C. Soto; Colombia: L. Ariza, C. Benavidez, Ó. David, N. Estupiñán, M. Lemos, R. Mazo, M. Varela, D. Villa-Roel; Ecuador: Y. Dávila; El Salvador: Ó. Olmedo, O. Olmedo M., A. Zúñiga; Guatemala: G. Aguilar, A. Musso; Honduras: R. Ardón; Mexico: S. Anguiano, P. Balcázar, M. Bonilla, A. Camarena, I. Carrillo, R. del Pino, G. García, M. González, G. Gurrola, S. Hernández, M. Karam, R. Landero, J. Olvera, F. Páez, C. Reyes, M. Ríos, R. Robles, P. Vázquez; Paraguay: A. Caballero, R. Estigarribia, M. Silva; Peru: V. Barreda, A. Galli, M. Salazar, C. Segura; Portugal: R. Barroso, F. Cardoso, P. Carvalho, E. Fernandes, M. Loureiro; Spain: C. Antona, P. Bas, J. Delgado, M. Fernández, A. Goñi, M. Muñoz, C. Rausell, S. Torrecillas; United States: A. Pina, I. Villalta; Uruguay: M. Golberg, M. Lagos; Venezuela: L. Feldman, Z. Lugli, J. Pellicer, E. Vivas. The first author of the manuscript is the director of the research team and the person responsible for the CISO-A Research Team; he also holds all rights for this team. Address correspondence to Vicente E. Caballo, Faculty of Psychology, University of Granada, 18071 Granada, Spain; e-mail: [email protected]. 0005-7894/43/313-328/$1.00/0 © 2011 Association for Behavioral and Cognitive Therapies. Published by Elsevier Ltd. All rights reserved.

Much has been written about the situations most often feared by persons with social phobia, and several self-report measures are frequently used to assess such feared situations. However, it is not clear whether the situations feared by persons with social phobia form unidimensional or multidimensional factors. If these situations are multidimensional, reliance on a total score of feared situations would not reflect important differences between those dimensions. This research examined the multidimensional nature and multicultural validity of a newly developed instrument (the Social Anxiety Questionnaire for Adults [SAQ-A]) in two studies with a total of 539 patients diagnosed with social phobia and 15,753 nonpatients from 20 different countries. The structure (five clear and solid factors) and psychometric properties of the final instrument (the SAQ-A30) support the multidimensional nature of social anxiety and provide a new perspective in the assessment of social phobia.

Keywords: social anxiety; social phobia; SAQ-A30; multidimensionality; multicultural measure

SOCIAL PHOBIA (SP) IS one of the most common mental disorders in the United States (12.1% lifetime

314

caballo et al.

prevalence and 6.8% 12-month prevalence; Kessler et al., 2005, 2008). However, prevalence rates are much lower in other countries. According to data from the World Health Organization's latest World Mental Health Surveys (Kessler & Üstün, 2008), the United States exceeds the rates for Colombia (5.0% lifetime prevalence and 2.8% 12-month prevalence; Posada-Villa et al., 2008), France (4.3% lifetime prevalence; Arbabzadeh-Bouchez, Gasquet, KovessMasfety, Negre-Pages, & Lépine, 2008), South Africa (2.8% lifetime prevalence and 1.9% 12month prevalence; Herman et al., 2008), Germany (2.5% and 1.4%, respectively; Alonso & Kessler, 2008), and Spain (1.2% and 0.6%, respectively; Haro et al., 2008). Regarding prevalence by sex, the findings are mixed and somewhat controversial. In general, studies on SP suggest a higher prevalence of SP in women than in men, specifically in community samples with adults (e.g., Fehm, Beesdo, Jacobi, & Fiedler, 2008; Merikangas, Avenevoli, Acharyya, Zhang, & Angst, 2002; Stein, Walker, & Forde, 1994); in addition, reports of such gender differences also exist in clinical samples (e.g., Pollard & Henderson, 1988). However, other studies have not found significant differences between men and women in clinical (e.g., Yonkers, Dyck, & Keller, 2001) or nonclinical samples (e.g., Bourdon et al., 1988). SP is defined by the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; American Psychiatric Association, 2000) as a “marked and persistent fear of one or more social or performance situations in which the person is exposed to unfamiliar people or to possible scrutiny by others” (p. 456). The DSM-IV-TR further states that the specifier “Generalized” “can be used when the fears are related to most social situations” (e.g., initiating or maintaining conversations, participating in small groups, dating, speaking to authority figures, attending parties). Individuals whose clinical manifestations do not meet the definition of “Generalized” compose “a heterogeneous group that includes persons who fear a single performance situation as well as those who fear several, but not most, social situations” (pp. 451–452). The diagnostic criteria do not specify how many social situations are “several” or “most” and few selfreport instruments exist that assess the range of social situations. Although rates of SP differ across cultures, it is unclear whether the situations feared by persons with SP also differ across cultures. In order to assess these situations, researchers and clinicians have used semistructured interviews (e.g., the Composite International Diagnostic Interview [CIDI], World Health Organization, 1997; Anxiety Disorders

Interview Schedule for DSM-IV: Lifetime Version [ADIS-IV-L], Di Nardo, Brown, & Barlow, 1994) and self-report measures (e.g., Liebowitz Social Anxiety Scale–Self-Report [LSAS-SR], Liebowitz, 1987; Social Phobia and Anxiety Inventory [SPAI], Turner, Beidel, Dancu, & Stanley, 1989; Social Phobia Inventory [SPIN], Connor et al., 2000). Regarding this last type of assessment, field research reports a weak and inconsistent factorial structure in most studies. For instance, some studies have reported a bifactorial structure (social interaction situations and social performance situations) for the LSAS (e.g., Mennin et al., 2002), whereas a four-factor solution was found in other studies (Oakman, Van Ameringen, Mancini, & Farvolden, 2003; Safren et al., 1999). Baker, Heinrichs, Kim, and Hofmann (2002) state that fit indices did not consistently support the four-factor solution suggested by Safren et al. and that more than four factors may be needed. Regarding the SPAI, studies have identified one (García-López, Olivares, Hidalgo, Beidel, & Turner, 2001), four (Olivares, GarcíaLópez, Hidalgo, Turner, & Beidel, 1999), or five (Turner, Stanley, Beidel, & Bond, 1989) factors. Furthermore, factor analytic studies of the SPIN have identified three (Radomsky et al., 2006) or five (Connor et al., 2000) factors. A second issue concerning most self-report measures assessing SP is that the decision as to whether or not a person has the disorder is based on the overall score obtained on the specific measure. This could be a limitation given that people who are highly anxious in specific situations, but not in others, and have a low total score would not be identified with this assessment strategy. As a consequence, these people are less likely to be referred for additional evaluation or treatment. Alternatively, there are cases when each situation (item) is considered as a subtype of SP and accordingly the patient is said to have 8, 10, or even more types of SP (e.g., Heimberg, Hope, Dodge, & Becker, 1990; Hughes et al., 2006; Kessler, Stein, & Berglund, 1998; Pollard & Henderson, 1988; Turner, Beidel, & Townsley, 1992). Third, most field research results are based on data obtained with measures created within an English-speaking culture, primarily North America and Australia. Studies dealing with social anxiety in other cultures have usually simply translated questionnaires from English into their own language for application (e.g., Bobes et al., 1999; González et al., 1998; Levin, Marom, Gur, Wechter, & Hermesh, 2002; Olivares et al., 1999; Osório, Crippa, & Loureiro, 2007; Radomsky et al., 2006; Sosic, Gieler, & Stangier, 2008; Terra et al., 2006), with few measures of SP being developed in or for non–English-speaking countries (e.g.,

315

the multidimensionality of social anxiety Furmark et al., 1999). Consequently, the content validity of assessment instruments used in non– English-speaking countries to measure SP has not been evaluated. When used in different cultures, these instruments may include irrelevant situations and, more seriously, omit relevant ones, thereby diminishing their validity and clinical utility. Fourth, the samples used in research on SP with non–English-speaking persons, in both predominantly English and predominantly non–English-speaking countries, have usually been small (e.g., Heinrichs et al., 2006; van Dam-Baggen, Kraaimaat, & Elal, 2003). Finally, multicultural research on social anxiety has included only a very few countries (e.g., Cox, Clara, Sareen, & Stein, 2008). All the former issues with self-report measures led us to develop a new self-report measure of SP with our primary aim being to address these issues. This measure was initially called the Social Anxiety Questionnaire for Adults (SAQ-A) and 11 countries, 24,423 subjects, and more than 100 researchers participated in the development stages (see Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010, for a more in-depth description of this work). The overriding goal of this study was to improve the former questionnaire in order to provide a shorter and more useful version for application in clinical settings. The specific subgoals were the following: (a) revision by experts of the items composing the revised version of the questionnaire (SAQ-AR) so as not to leave out potentially important situations for subjects with SP; (b) use patients with SP to determine what situations most discriminated them from people without it; (c) confirm the dimensions of SP; (d) include more countries than the previous study (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010) for greater multicultural validity; (e) confirm social anxiety differences linked to gender; (f) reduce the number of items to ones more suitable for clinical use; (g) establish the new version's psychometric characteristics, including cutoff points to identify patients with SP; and (h) propose a more systematic way to differentiate patients with specific SP from those with the generalized form.

Study 1 method Participants The first group of participants involved 2,105 nonclinical individuals (M = 24.56 years, SD = 8.38; range = 16–68 years) from 14 countries (20.96% Bolivia, 18.10% Colombia, 10.24% Dominican Republic, 9.48% Panama, 9.34% Mexico, 7.00% Argentina, 6.29% Peru, 4.43% Portugal, 3.19% Guatemala, 2.43% Ecuador, 2.43% Paraguay, 2.43% Spain, 2.33% Costa Rica, and 1.33% El

Salvador). The sample included 1,362 women (M = 23.95 years, SD = 7.74) and 737 men (M = 25.68 years, SD = 9.34). The participants had varied levels of occupation at the time of the assessment: 29.97% were university psychology students, 26.54% were university students from other majors, 14.77% were high school students, 10.96% were workers with a university degree, 6.15% were workers with no university degree, and 7.00% could not be included in any of the former categories (e.g., retired or unemployed). No data on occupation were obtained for the remaining 4.62% of participants. The second group of participants consisted of 280 patients (M = 33.01 years, SD = 10.73; range = 16–66 years) from 10 countries (34.29% Spain, 19.64% Argentina, 16.79% Brazil, 13.21% Colombia, 10.71% Mexico, 2.14% Costa Rica, 1.07% Peru, 0.71% Chile, 0.71% Ecuador, and 0.71% Venezuela). The sample consisted of 164 women (M = 32.10 years, SD = 10.82) and 116 men (M = 34.28 years, SD = 10.52). For inclusion in this group, patients had to receive a primary diagnosis of SP according to DSM-IV-TR (American Psychiatric Association, 2000) or ICD-10 (World Health Organization, 1992) criteria. Each center made its own diagnosis of individual patients based on one of these two nosological systems. These patients were included even if they had other disorders in addition to SP (see Table 1), but invalid cases were removed

Table 1

Distribution of Patients by Psychiatric Disorders in Study 1 Psychiatric Disorder

Women Men Total

Social phobia Social phobia + other anxiety disorder Social phobia + mood disorder Social phobia + other anxiety disorder + mood disorder Social phobia + substance use disorder Social phobia + mood disorder + substance use disorder Social phobia + substance use disorder + personality disorder Social phobia + substance use disorder + other disorder Social phobia + personality disorder Social phobia + eating disorder Social phobia + one other nonpsychotic disorder Social phobia + two other nonpsychotic disorders Social phobia + three other nonpsychotic disorders Total

127 9 15 0

69 7 13 1

196 16 28 1

1 0

6 3

7 3

1

5

6

1

2

3

2 4 2

3 1 4

5 5 6

0

2

2

2

0

2

164

116

280

316

caballo et al.

for several reasons (e.g., incomplete data, presence of psychotic disorders, SP not the primary diagnosis). From a pool of 387 patients, 280 satisfied the former criteria. In reference to occupation, 33.93% were workers with a university degree, 24.29% were workers with no university degree, 16.43% were university students from different majors, 4.64% were university psychology students, 1.79% were high school students, and 10.36% could not be included in any of the former categories (e.g., retired or unemployed). No data on occupational status were obtained for the remaining 8.57% of participants. Measures A modified version of the SAQ-AR (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010) was used in this first study. This self-report measure was the result of several years of work by our team in nine Latin American countries, Spain, and Portugal with 22,262 participants, being composed of 72 items grouped under six factors (12 items per factor): (a) awkward behaviors in embarrassing situations; (b) interaction with the opposite sex; (c) interaction with strangers; (d) criticism and embarrassment; (e) assertive expression of annoyance, disgust, or displeasure; and (f) speaking/performing in public/talking with people in authority (see Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010, for the original development and psychometric characteristics of the SAQ-AR). However, some of the social situations usually included in most assessment measures for SP (questionnaires, semistructured interviews, diagnostic systems) were not included in SAQ-AR. Therefore, we decided that four experts in SP should add the most significant social situations that had not appeared in SAQ-AR, although these added situations were included in the initial exploratory 512-item questionnaire (SAQ-A; see Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010, for a description of this questionnaire). Procedure Forty-four new items (social situations) were incorporated into the modified version of the SAQ-AR for use in Study 1. To select the new items, four clinical psychologists with expertise in SP reviewed the initial pool of 512 items (distributed across the same six factors that later remained in the SAQ-AR) that served as the basis for the SAQ-AR (see Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010). By consensus, six to eight clinically significant social situations were added to each one of the six factors of the SAQ-AR. These situations were not included in the previous study by Caballo, Salazar, Irurtia, Arias, Hofmann, et al.

because these items did not satisfy specific statistical criteria for selection. However, we wanted to give a second chance to those social situations traditionally considered central to the assessment of SP (see American Psychiatric Association, 2000; World Health Organization, 1992), such as “working while being observed,” “drinking in public places,” “blushing in front of others,” “being the center of attention,” or “urinating in a public bathroom.” The new version of the SAQ-AR for use here consisted of 116 items plus two control items (“One of my parents getting seriously ill,” “Being mugged or robbed by an armed gang”). The two control items were not included in the statistical analysis but they did allow us to estimate how many subjects might be completing the questionnaire at random. Small language differences between countries were addressed in the same way as in the former study; that is, agreement was reached among the collaborating researchers from each country. The questionnaire was also translated from Spanish into Portuguese (Brazil and Portugal, respectively), then back translated to Spanish, and finally revised again in Portuguese and Spanish until agreement was reached between translators. The items were randomly ordered and each item was answered on a 7-point Likert scale to indicate the level of “unease, stress, or nervousness” in response to each situation: 1 (not at all), 2 (very slight), 3 (slight), 4 (moderate), 5 (high), 6 (very high), and 7 (extremely high). The reason for using words like “unease,” “stress,” or “nervousness” instead of the word “anxiety” was that the word “anxiety” is not clearly understood at many educational levels, whereas the former words are (they are synonymous with “anxiety,” in Spanish and Portuguese). The modified version of the SAQ-AR (118 items) was administered to the clinical and nonclinical samples. Application to the nonclinical sample was anonymous and in groups, whereas application to the clinical sample was done individually in the clinical settings.

results An exploratory factor analysis (EFA) with promax rotation was carried out separately with the clinical and nonclinical samples. Exactly the same five factors (scree-test) were obtained in both samples (although in a different order for some of them) and they were particularly clear. In the clinical and nonclinical samples these five factors explained 47.11% and 42.59% of the cumulative variance, respectively, as follows (in clinical and nonclinical samples, respectively): F1 and F4. Speaking in public/talking with people in authority (eigenvalues: 35.83 and 3.31; % total variance = 30.90 and 2.81,

the multidimensionality of social anxiety respectively): F2 and F2. Interactions with the opposite sex (eigenvalues: 6.20 and 7.64; % total variance = 5.35 and 6.47, respectively): F3 and F3. Assertive expression of annoyance, disgust, or displeasure (eigenvalues: 5.17 and 4.85; % total variance = 4.46 and 4.11, respectively): F4 and F5. Interactions with strangers (eigenvalues: 4.33 and 2.55; % total variance = 3.73 and 2.16, respectively): F5 and F1. Criticism and embarrassment (eigenvalues: 3.11 and 31.90; % total variance = 2.69 and 27.04, respectively). These factors were very similar to those found in previous research (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010). In fact, four of the five factors were exactly the same and the two remaining factors in the previous research converged into one in this study. Those two factors, with very similar items, were awkward behavior in embarrassing situations and criticism and embarrassment, now converged into a single factor, called criticism and embarrassment. Means and standard deviations were obtained for each of the 116 items with the clinical and nonclinical samples. Those items that loaded at least .40 in one (and only one) of the five factors (in both samples) and that produced the greatest differences between the mean score of clinical and nonclinical samples were selected for the next version of the questionnaire. For instance, in Factor 2, the item “telling someone that their behavior bothers me and asking him or her to stop” loaded .56 in its factor, and the difference between the mean score of patients and nonpatients was 1.61 (student t = 15.50), whereas the item “exchanging a defective item” loaded .53 in its factor, and the difference between the mean score of patients and nonpatients was .92 (student t = 8.17). Cohen's d was also calculated for these mean differences between patients and nonpatients. In the former example, Cohen's d for the first item was 1.06 (large effect size), whereas in the second one it was .54 (moderate effect size). So the first item was retained for the next version of the questionnaire, whereas the second one was not. There was one situation usually included in most self-report and diagnostic measures of SP (“using a public restroom”) that presented no difference at all between patient and nonpatient samples. Given that in the former study (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010) that item had one of the smallest item-total score correlations, it was omitted from the next version of the questionnaire. In this way, 16 items were selected within each of the five factors, resulting in a total of 80 items (plus the two control items), which formed a new version of the questionnaire, the SAQ-A82. In order to make

317

the questionnaire easier for respondents, particularly thinking of a final version for patients, as people are more accustomed to a 5-point scale than a 7-point scale in their daily lives and many selfreport measures on mental disorders (including SP) use a 5-point Likert scale, it was decided to alter the response format from a 7- to a 5-point scale. This 82-item version was the focus of Study 2.

Study 2 method Participants There were two groups of participants in Study 2. The first group of participants comprised 13,303 nonclinical participants (M = 25.58 years, SD = 10.28; range = 16–78 years) from 16 countries (21.97% Mexico, 20.35% Colombia, 9.39% Spain, 9.27% Peru, 7.48% Brazil, 4.81% Chile, 4.56% Bolivia, 3.93% Portugal, 3.16% Argentina, 3.10% Uruguay, 3.03% Venezuela, 3.01% El Salvador, 1.51% Guatemala, 1.50% Honduras, 1.50% Paraguay, and 1.42% Ecuador). The sample consisted of 7,544 women (M = 24.98 years, SD = 9.83) and 5,736 men (M = 26.36 years, SD = 10.80). The participants had varied levels of occupation: 37.10% were university students from other majors, 22.50% were university psychology students, 15.56% were workers with a university degree, 8.68% were high school students, 8.45% were workers with no university degree, and 7.13% could not be included in any of the former categories (e.g., retired or unemployed). No data were obtained on occupational status for the remaining 0.57% of participants. This group of nonclinical participants did not overlap with the one in Study 1. The second group of participants involved 259 patients (M = 31.35 years, SD = 11.44; range = 16– 71 years) from nine countries (38.61% Spain, 11.97% Peru, 10.81% Brazil, 8.49% Argentina, 7.34% Chile, 7.34% Colombia, 5.79% Mexico, 5.02% Uruguay, and 4.63% Portugal). The sample consisted of 170 women (M = 31.78 years, SD = 10.63) and 89 men (M = 30.54 years, SD = 11.85). This clinical sample was obtained after discarding invalid cases (e.g., incomplete or incorrect data, presence of psychotic disorders, a score lower than 30 in the LSAS-SR [see Measures section]) from an initial pool of 380 patients in the first stages of clinical assessment in public or private mental health centers, with a primary diagnosis of SP (with/without other disorders) according to DSM-IV-TR (American Psychiatric Association, 2000) or ICD-10 (World Health Organization, 1992) criteria. Table 2 presents the distribution of patients by psychiatric disorders. In reference to

318

caballo et al.

Table 2

Distribution of Patients by Psychiatric Disorders in Study 2 Psychiatric Disorder

Women Men Total

Social phobia Social phobia + other anxiety disorder Social phobia + mood disorder Social phobia + other anxiety disorder + mood disorder Social phobia + substance use disorder Social phobia + mood disorder + substance use disorder Social phobia + personality disorder Social phobia + eating disorder Social phobia + one other nonpsychotic disorder Social phobia + three other nonpsychotic disorders Total

64 33 45 2

26 19 16 4

90 52 61 6

2 1

11 3

13 4

12 6 5

6 0 3

18 6 8

0

1

1

170

89

259

occupational status, 30.50% were workers with a university degree, 22.39% were university students from other majors, 18.15% were workers with no university degree, 4.63% were university psychology students, 1.93% were high school students, and 14.67% could not be included in any of the former categories (e.g., retired or unemployed). No data on occupational status were obtained for the remaining 7.72% of participants. This group of patients is not related in any way to the one in Study 1. Measures The SAQ-A82 uses 80, plus two control, items from Study 1 that were randomly ordered to form the SAQ-A82. Each item was answered on a 5-point Likert scale to indicate the level of unease, stress, or nervousness in response to each social situation: 1 (not at all or very slight), 2 (slight), 3 (moderate), 4 (high), and 5 (very high or extremely high). The LSAS (Liebowitz, 1987) is a 24-item interviewer-rated instrument that assesses fear and avoidance of specific social situations. Respondents are asked to rate fear on a 4-point scale ranging from 0 (none) to 3 (severe) and avoidance on a 4point scale ranging from 0 (never) to 3 (usually). However, the LSAS-SR has also been used as a selfreport instrument in the literature (e.g., Baker et al., 2002; Fresco et al., 2001). Mennin et al. (2002) recommended a cutoff score on the LSAS-SR between 30 and 60 for nongeneralized SP and a score above 60 for generalized SP. Regarding certain psychometric characteristics of the Spanish version, González et al. (1998) found a four-factor structure of the LSAS-SR explaining 48.9% of the variance; the Cronbach's alpha for the LSAS– Anxiety was 0.87 and for the LSAS–Avoidance it was 0.88. These authors and Bobes et al. (1999)

concluded that the LSAS is valid for use in the clinical research and assessment of patients with SP in Spain. Regarding the Portuguese version of the LSAS-SR, Terra et al. (2006) found a five-factor structure of the LSAS-SR explaining 52.9% of the variance, and the Cronbach's alpha for the LSAS total was 0.95. Procedure Both questionnaires were administered jointly to the clinical and nonclinical samples. The procedure was similar to that of Study 1. Researchers in each country once again initially reviewed the items of the SAQ-A82 in order to verify that all the words and meanings were clearly understood in their countries.

results Exploratory Factor Analysis of the SAQ-A82 Given that the response format of the questionnaire was changed (from a 7- to a 5-point scale), that the number of items composing the questionnaire (80 + 2 control items) was high for use in clinical settings, and that we had a fairly large sample, it was decided to conduct a new EFA on the SAQ-A82 with promax rotation for the nonclinical sample. Given the ordinal nature of the data, a polychoric correlation matrix from the direct scores of the 80 items was computed. It was then verified that the items complied with the following conditions: (a) there were no items with extreme distributions (skewness from –.379 to .833 with SE of .030, kurtosis from −1.092 to −0.153 with SE = .060), (b) all the separate items within each cluster had high-corrected item-total correlations (homogeneity index; from .42 to .67), (c) all the proposed factors had more than four items, (d) the sample was large enough to avoid possible fluctuations of correlations, (e) most of the elements of the antiimage correlation matrix tended to zero, and (f) the Kaiser–Meyer–Olkin (KMO) index exceeded the recommended cutoff of .50. Half the subjects were used for the EFA with the Latin American, Spanish, and Portuguese sample. The Bartlett χ 2 test (3,160, n = 6,613) = 267,455.4, p = .000 and the KMO index of .985 support the adequacy of factorial data analysis. Matrix sampling adequacy (MSA) indices (from .958 to .992) confirm that the measure of sampling adequacy of the variables in all cases fits the structure of the rest of the variables. Finally, 35% of communalities were above .50 (from .26 to .65). In order to choose the optimal number of factors, a parallel analysis (Horn, 1965) was implemented using the Monte Carlo procedure with 200 replications. Parallel analysis compares the

the multidimensionality of social anxiety observed eigenvalues extracted from the observed correlation matrix to be analyzed with those obtained from uncorrelated normal variables (parallel components derived from random data). Results showed that the five-factor solution was the best fit to our data, given that only the eigenvalues of these five factors were greater than the randomly generated eigenvalues. This EFA identified five factors with eigenvalues higher than 1.00, explaining 50.97% of the cumulative variance. The first factor (eigenvalue = 28.92) explained 36.14% of the variance. The items loading highly in this factor describe speaking in public/talking with people in authority. The second factor (eigenvalue = 3.78) explained 4.73% of the total variance. The high-loading items describe situations of interactions with the opposite sex. Factor 3 (eigenvalue = 3.18) explained 3.98% of the variance. The items in this factor refer to situations of assertive expression of annoyance, disgust, or displeasure. Factor 4 (eigenvalue = 3.01) explained 3.76% of the variance, with the items referring to situations of criticism and embarrassment. Factor 5 (eigenvalue = 1.88) explained 2.35% of the variance and is related to interactions with strangers. An EFA was also conducted with the clinical sample. The best solution based on the scree-test was also a structure of five factors with eigenvalues higher than 1.00, explaining 40.80% of the cumulative variance. The first factor (eigenvalue = 17.26) explained 21.58% of the variance. The items loading highly in this factor describe speaking in public/talking with people in authority. The second factor (eigenvalue = 5.62) explained 7.02% of the total variance. The high-loading items describe situations of interactions with the opposite sex. Factor 3 (eigenvalue = 4.41) that explained 5.52% of the variance. The items in this factor refer to situations of assertive expression of annoyance, disgust, or displeasure. Factor 4 (eigenvalue = 2.95) explained 3.69% of the variance, with the items referring to situations of interactions with strangers. Factor 5 (eigenvalue = 2.39) explained 2.99% of the variance and is related to criticism and embarrassment. Analysis and Selection of Items In order to reduce the length of the questionnaire for simpler application in clinical settings, the best items for each factor in the patient and nonpatient samples were determined. Items were selected based on the following criteria: (a) a loading of at least 0.40 in the factor, (b) they must load in the same factor in both samples, and (c) items with the highest mean difference between patients and nonpatients. For example, in Factor 3, the item “speaking in public” loaded 0.74 in its factor

319

(“speaking in public/talking with people in authority”) and the difference between the mean score of patients and nonpatients was 1.43 (student t = 17.96), which was the highest difference for an item within this Factor 3, while the item “presenting in public a topic I don't know much about” loaded 0.51 in its factor and the difference between the mean score of patients and nonpatients was 1.00 (student t = 13.57), which was the lowest difference for an item within this factor. Given the large sample of nonpatients, all the differences between the two samples were statistically significant (p = 0.0000), so we also calculated Cohen's d. In the first item, Cohen's d was 1.24, whereas in the second it was 1.01. In both cases the effect size was high (d N 0.80), as it also was in most of the items in the SAQ-A82. On the basis of these data, the first item was selected for the final version, whereas the second one was not. Furthermore, if two or more items were very similar, that is, they were highly related within each factor, only the item with the highest loading and the highest patient/nonpatient difference was chosen (e.g., “performing in public” and “speaking in public” have a relationship of r = 0.66, but this latter item had a higher loading on its factor, both in patient and nonpatient samples, and a higher patient/nonpatient difference). Additionally, it was hoped to have items as varied as possible within each factor for a more comprehensive clinical application (DeVellis, 2003). Six items finally remained for each of the five factors, yielding a total of 30 items overall for the new questionnaire (the SAQ-A30; see Appendix A). The item loadings are presented in Table 3 together with their item-total correlations. Interfactor correlations ranged from low to moderate (.33 to .55). Table 4 shows these correlations. Construct Validity of the SAQ-A30 Confirmatory factor analysis (CFA; LISREL, v. 8.8.; Scientific Software International, 2006) and exploratory structural equation modeling (ESEM; MPlus, v. 6.0; Muthén & Muthén, 2010) were conducted to test the construct validity of the SAQ-A30. The ESEM models have recently been developed for solving the problems usually found in CFA models. In these latter models the necessity to fix to zero the saturations frequently leads to an important modification of the model in order to obtain a better fit. The ESEM models can solve this problem as they do not impose such restrictions (Asparouhov & Muthén, 2009; Marsh, 2007; Marsh, Hau, & Grayson, 2005). The CFA and ESEM were completed with the second subsample (see above) of Latin American, Spanish, and Portuguese participants (n = 6,613) using WLSMV as an estimation method. Table 5 presents fit indices of the two models tested.

320

caballo et al.

Table 3

Thirty Items Loadings of SAQ-A82 for Every Factor and Correlations Item-Total Score Factors and Items

Factor Loadings

F1. Speaking in public/talking with people in authority 26. Speaking in public 12. Having to speak in class, at work, or in a meeting 03. Participating in a meeting with people in authority 77. Talking to a superior or a person in authority 58. Being asked a question in class by the teacher or by a superior in a meeting 75. While having dinner with colleagues, classmates, or workmates, being asked to speak on behalf of the entire group F2. Interactions with the opposite sex 81. Telling someone I am attracted to that I would like to get to know them better 57. Being asked out by a person I am attracted to 73. Starting a conversation with someone of the opposite sex that I like 14. Asking someone attractive of the opposite sex for a date 37. Asking someone I find attractive to dance 67. Feeling watched by people of the opposite sex F3. Assertive expression of annoyance, disgust, or displeasure 69. Telling someone that their behavior bothers me and asking them to stop 78. Expressing my annoyance to someone that is picking on me 56. Having to ask a neighbor to stop making noise 45. Refusing when asked to do something I don't like doing 82. Telling someone that they have hurt my feelings 64. Complaining to the waiter about my food F4. Criticism and embarrassment 65. Talking to someone who isn't paying attention to what I am saying 47. Being teased in public 25. Greeting someone and being ignored 41. Being reprimanded about something I have done wrong 54. Making a mistake in front of other people 72. Being criticized F5. Interactions with strangers 68. Talking to people I don't know at a party or a meeting 28. Maintaining a conversation with someone I've just met 43. Attending a social event where I know only one person 49. Greeting each person at a social meeting when I don't know most of them 55. Making new friends 61. Looking into the eyes of someone I have just met while we are talking

Regarding CFA, the five first-order factors and one second-order factor model had a poorer fit than the five-factor model, and the ESEM of this latter model

Item-Total Correlation

F1

F2

F3

F4

F5

.83 .80 .68 .68 .59 .48

–.07 –.07 .10 .04 .08 .02

–.10 .06 –.02 .11 .12 -.02

.04 –.13 .01 –.01 .02 .26

.08 .01 –.07 .05 .06 .15

.557 .500 .485 .625 .641 .593

–.01 .02 .01 .06 .03 .08

.79 .78 .74 .71 .65 .50

.02 .09 –.01 –.09 –.05 .10

.10 –.14 –.02 .28 .13 –.03

–.01 .02 .19 -.13 .11 .21

.621 .579 .656 .512 .582 .629

.05 .12 .00 –.10 –.04 .03

.01 .02 .01 –.01 .26 .07

.72 .71 .67 .61 .54 .52

.09 .07 .08 .09 .04 .16

–.02 –.14 .01 .12 –.05 .00

.582 .536 .517 .494 .521 .510

–.18 –.01 –.19 .03 .17 .05

–.06 .05 –.14 .08 .01 –.04

.26 .15 21 .31 .21 .33

.60 .47 .45 .44 .44 .42

.17 .19 .40 –.01 .13 .08

.439 .510 .416 .520 .604 .519

.12 .06 .17 .16 .09 .18

.10 .18 .02 –.03 .20 .18

–.01 .08 .02 .13 .15 .19

–.02 –.25 .06 .07 –.41 –.27

.68 .63 .58 .51 .51 .42

.642 .540 .592 .585 .452 .547

presented a better fit than the CFA of this same model.

Factors SAQ-A30

F1

F2

F3

F4

F5

F1 F2 F3 F4 F5

1.00 .55 .53 .36 .45

1.00 .55 .36 .48

1.00 .30 .56

1.00 .29

1.00

Internal Consistency of the SAQ-A30 The internal consistency (Cronbach's α) estimates of the SAQ-A30 total score and five factors were very good, particularly taking into account that every factor possesses only six items. The Cronbach's α was F1. Speaking in public/talking with people in authority = .84, F2. Interactions with the opposite sex = .86, F3. Assertive expression of annoyance, disgust, or displeasure = .80, F4. Criticism and embarrassment = .78, and F5 = interactions with strangers = .82. Total SAQ-A30 = .93.

Note. F1 = speaking in public/talking with people in authority; F2 = interactions with the opposite sex; F3 = assertive expression of annoyance, disgust, or displeasure; F4 = criticism and embarrassment; F5 = interactions with strangers.

Convergent Validity of the SAQ-A30 Convergent validity of the SAQ-A30 was assessed via correlations with the LSAS-SR, which was

Table 4

Interfactor Correlations for Exploratory Factor Analysis of the SAQ-A30

321

the multidimensionality of social anxiety Table 5

Fit Indices of the Two Models Tested Analysis χ2

CFA1 CFA2 ESEM

p

DF

SRMR CFI

6551.13 .000 395 .041 10216.69 .000 400 .095 4297.61 .000 295 .020

TLI

RMSEA

.981 .979 .051 .967 .964 .061 .975 .963 .045

Note. CFA1 = confirmatory factor analysis, five-factor model; CFA2 = confirmatory factor analysis, second-order factor model; ESEM = exploratory structural equation modeling; SRMR (standardized root mean square residual) = values less than .10 are generally considered favorable; the smaller the SRMR, the better the model fit; CFI (Comparative Fit Index) and TLI (Tucker–Lewis Index) = values higher than .90 indicate good fit; RMSEA (root mean square error of approximation) = values less or equal to .05 indicate close approximate fit; values between .05 and .08 suggest reasonable error of approximation, and values higher or equal to .10 suggest poor fit (see Kline [2005] for a review of most of these indices).

administered (together with the SAQ-A82) to the clinical and nonclinical samples. The correlations between the total score on the SAQ-A30 and the LSAS-SR Anxiety subscale and LSAS-SR total score were moderate in both patient and nonpatient samples (from .57 to .65). However, as expected, the correlations between factors on the SAQ-A30 and the LSAS-SR scores were lower (see Table 6). Toward a Diagnosis of Social Phobia With the SAQ-A30 Receiver operating characteristics (ROC) analysis was used with the whole Latin American, Spanish, and Portuguese sample to examine cutoff values of the SAQ-A30 that corresponded to a diagnosis of SP by sex, distinguishing between patients with SP and comparison subjects (Cohen's d in the factors and total score of the SAQ-A30 ≥ 1.30). The differences between the cutoff points according to sex are due to the mean for women being higher than for men (p b .05) in three of the factors and in

the total score of the SAQ-A30 in the clinical, and in the five factors and the total score in the nonclinical sample. Based on the five correlated factors model, the invariance regarding sex was calculated for both samples. In the nonclinical sample, RMSEA values fell between .051 and .053, whereas in the clinical sample these values fell between .063 and .065. Using ΔCFI N .01 as criterion (Wu, Li, & Zumbo, 2007), the configural, weak, strong, and strict invariance was supported in both samples (see Table 7). On the other hand, in order to find the differences between men and women in the five latent variable means, a Structured Means Model analysis was computed. Table 8 shows that the means of the nonclinical female sample are higher than the means of the nonclinical male sample in all the cases (p b .001) and the contribution to χ 2 is also higher in women. Such differences are smaller in the clinical sample where Factor 5 does not record significant differences between men and women. In males, the ROC analysis produced a robust area under the curve (AUC = .931, SE = .018) with a 95% confidence interval between .924 and .937 (z = 23.016, p b .0001) for the classification of males in SP/non-SP groups. The SAQ-A30 total score of 92 provided the best balance between sensitivity (.921) and specificity (.798) and correctly classified 92.13% of the males diagnosed with SP (82 out of 89) and 79.83% (4,540 out of 5,736) without SP. Table 9 shows the results with ROC curves in males using the five factors (dimensions) and the total score of the SAQ-A30. Regarding females, the ROC analysis produced a robust AUC (.904, SE = .015) with a 95% confidence interval between .897 and .910 (z = 28.851, p b .0001) for the classification of females in SP/non-SP groups. The SAQ-A30 total score of 97 provided the best balance between sensitivity (.918)

Table 6

Correlations (Pearson) Between the SAQ-A30 and Its Factors and the LSAS-SR in Clinical and Nonclinical Samples SAQ-A30

F1. Speaking in public/talking with people in authority F2. Interactions with the opposite sex F3. Assertive expression of annoyance, disgust, or displeasure F4. Criticism and embarrassment F5. Interactions with strangers Total

Clinical Sample (N = 259)

Nonclinical Sample (N = 12,662)

LSAS Anx.

LSAS Av

LSAS Total

LSAS Anx.

LSAS Av

LSAS Total

.43 .26 .40

.35 .27 .37

.42 .29 .42

.57 .51 .48

.43 .38 .38

.54 .48 .46

.32 .53 .57

.27 .48 .52

.32 .55 .59

.46 .56 .65

.36 .45 .50

.44 .55 .62

Note. All correlations significant at p b .0001; LSAS-SR = Liebowitz Social Anxiety Scale–Self–Report; LSAS Anx = Liebowitz Social Anxiety, Anxiety subscale; LSAS Av = Liebowitz Social Anxiety, Avoidance subscale.

322

caballo et al.

Table 7

Fit Indices for Invariance Models Nonclinical Sample

DF

χ2 (p)

Δχ2 (p)

RMSEA

CFI

ΔCFI

SRMR

Configural invariance

800

—

.052

.979

—

.045

Weak invariance

830

.979

.000

.052

860

.053

.977

–.002

.050

Strict invariance

890

100.14 (.000) 866.47 (.000) 346.05 (.000)

.051

Strong invariance

7,913.28 (.000) 8,013.42 (.000) 8,879.89 (.000) 9,225.95 (.000)

.053

.976

–.001

.052

Clinical Sample Configural invariance

800

—

.065

.898

—

.077

Weak invariance

830

.065

.897

–.001

.088

Strong invariance

860

.065

.892

–.005

.082

Strict invariance

890

47.74 (.021) 47.40 (.023) 15.21 (.989)

.063

.890

–.002

.084

1269.45 (.000) 1317.19 (.000) 1364.59 (.000) 1379.80 (.000)

Table 8

Structured Means Model Nonclinical Group χ2 Sample

Males Females

% SRMR DF contribution to χ2

3,056.68 40.94 4,410.17 59.06

Clinical Sample Males 746.22 Females 711.87

51.18 48.82

Global χ2 RMSEA CFI (p)

TLI

Mean Vector of Independent Variables F1

F2

F3

F4

F5

(t)

(t)

(t)

(t)

(t)

0.040 0.044

840 7,466.85 .051 (.000)

.978 .978 0.42*** 0.04*** 0.18*** 0.25*** 0.19*** (17.54) (17.32) (7.90) (12.71) (8.70)

0.100 0.044

840 1,458.09 .065 (.000)

.896 .892 0.29*** 0.25*** 0.22** (2.94) (3.03) (2.22)

0.28*** 0.08 ns (3.34) (0.68)

* p b .05, ** p b .01, *** p b .001; ns = nonsignificant.

and specificity (.738) and correctly classified 91.76% of the females diagnosed with SP (156 out of 170) and 73.80% (5,478 out of 7,544) without SP (see Table 9). Figure 1 shows the ROC curve for diagnosing SP in males and females according to the SAQ-A30 total score. 1

General Discussion This study's main goal was to develop and validate a short but clinically useful version of the SAQ-AR (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010) for use in applied settings. The final short version (SAQ-A30) comprises 30 items grouped under five factors (or dimensions), with excellent psychometric properties. Although the dimension1 ROC plots of SAQ-A30 are available from the first author named in the manuscript.

based assessment of SP has been strongly supported in this work, further studies are needed to confirm not only these five dimensions but the approach itself. As already stated, the reason for developing a new self-report measure of SP was to address some of the problems with current questionnaires that measure social anxiety, particularly the instability and variability of their component factors, the types of situations that are assessed (or not assessed), the lack of research on a multicultural level, and the difficulties in properly identifying individuals with generalized and nongeneralized SP. The SAQ-A30 developed in this study has stable and solid factors maintained in different samples (clinical and nonclinical) with a large number of participants from many countries. Further research with a large sample of Spanish university students (N = 15,504 participants) using the new version of the

Results for the ROC Analysis of the Five-Factors and Total Score of the SAQ-A30 by Sex SAQ-A30

Men F1 F2 F3 F4 F5 Total Women F1 F2 F3 F4 F5 Total

Sensitivity

Specificity

Cutoff

AUC

SE

95% CI

z

p

True Positive

True Negative

False Positive

False Negative

N

%

N

%

N

%

N

%

86.5 69.7 73.0 89.9 74.2 92.1

77.1 86.5 89.8 68.3 81.0 79.8

19 20 21 19 17 92

.879 .843 .891 .843 .857 .931

.0237 .0261 .0227 .0262 .0253 .0187

.870–.887 .833–.852 .883–.899 .833–.852 .848–.866 .924–.937

15.962 13.116 17.263 13.104 14.132 23.016

.0001 .0001 .0001 .0001 .0001 .0001

77 62 65 80 66 82

86.52 69.66 73.03 88.89 74.16 92.13

4385 4919 5108 3885 4606 4540

77.11 86.50 89.82 68.31 80.99 79.83

1302 768 579 1802 1081 1147

22.89 13.50 10.18 31.69 19.01 20.17

12 27 24 9 23 7

13.48 30.34 26.97 10.11 25.84 7.87

70.6 69.4 92.4 81.8 68.2 91.8

83.0 79.9 65.7 69.6 80.8 73.8

23 20 19 21 18 97

.839 .821 .867 .823 .823 .904

.0191 .0198 .0178 .0197 .0197 .0156

.831–.848 .812–.829 .859–.874 .814–.832 .814–.832 .897–.910

17.777 16.183 20.565 16.365 16.372 28.851

.0001 .0001 .0001 .0001 .0001 .0001

120 118 157 139 116 156

70.59 69.41 92.35 81.76 68.24 91.76

6161 5931 4879 5169 5995 5478

83.00 79.90 65.73 69.63 80.76 73.80

1262 1492 2544 2254 1428 1945

17.00 20.10 34.27 30.37 19.24 26.20

50 52 13 31 54 14

29.41 30.59 7.65 18.24 31.77 8.24

Note. AUC = area under curve; F1 = speaking in public/talking with people in authority; F2 = interactions with the opposite sex; F3 = assertive expression of annoyance, disgust, or displeasure; F4 = criticism and embarrassment; F5 = interactions with strangers.

the multidimensionality of social anxiety

Table 9

323

324

caballo et al.

FIGURE 1 Sensitivity and specificity for classifying men and women with and without social phobia for different values of the SAQ-A30.

questionnaire developed in the present study (the SAQ-A30) shows exactly this same five-factor structure with all the items loading in the same factors (see Caballo, Salazar, Arias, et al., 2010). The initial and distinct generation of social situations for inclusion in the current assessment measure (see Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010; Caballo et al., 2008), plus the analyses carried out in previous and current studies, leads us to conclude that these five dimensions (factors) may form the basic structure of the SP construct. Although further research is needed, this could be a first step in that direction. The controversial issue of the uni- versus multidimensionality of the SP construct is closely related to the former findings. The results here clearly support the multidimensionality of the construct. This result is further in line with certain studies (e.g., Baker et al., 2002; Connor et al., 2000; Davidson et al., 1997; Ruscio et al., 2008; Safren et al., 1999) but inconsistent with others, which suggest a single higher-order factor to explain SP (e.g., Mattick & Clarke, 1998; Osman et al., 1996; Rodebaugh, Woods, & Heimberg, 2007). This confusing state of affairs in the field may be reflected in the work by Ruscio et al., who reported results supporting the multidimensionality of the SP construct, but also claimed support for unidimensionality. The 13 specific social fears examined in that study perfectly match the five dimensions found in our research here. Specifically, four fears would belong to the dimension of criticism and embarrassment, three to interactions with strangers, one to interactions with the opposite sex, one to assertive expression of annoyance, and four to speaking in public/talking with people in

authority. This study could help to clarify some of these matters, particularly the multidimensional nature of SP. There are further reasons supporting the significance of the current findings. First, if there are multiple dimensions to the situations feared by persons with SP, the identification of those dimensions can aid treatment, tailoring its content according to the types of feared situations. Second, treatment outcomes that might differ across the situations feared by persons with SP would be more accurately reflected by data from individual factors or scale scores than by data from a total score. Third, data from individual factors or scale scores might facilitate the identification of variables that trigger or maintain social anxiety in different situations. Another related question involves the social situations included in the assessment measures. This could clearly influence the multi- or unidimensional structure of the measure, yet it also involves the validity of the assessment measure in different cultures. For example, the situations “drinking in public places,” “urinating in a public bathroom,” “writing while being observed,” “working while being observed,” and “blushing in front of others” are all part of the LSAS-SR and were also initially included by our item-generation procedure for the SAQ-AR (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010). Interestingly, however, we observed that these items either did not produce much anxiety in our samples or showed poor item reliability and validity. The reasons for the scant utility of those items might be that they do not seem to apply to the countries included in our study. For instance, “drinking in public places” is a very common and almost unavoidable situation in many of these countries, as we discussed in a previous paper (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010). For an individual with SP in Latin America, Spain, or Portugal, the problem is not to “drink in public places” but rather to “have a friend or a group of friends” to drink with. Sex differences in social anxiety in both the patient and nonpatient samples were also addressed. We found small but significant differences (and Cohen's d above 0.20) between men and women in three dimensions and in the total SAQA30 in both samples (patients and nonpatients). Women scored higher than men in the dimensions of interactions with the opposite sex, speaking in public/talking with people in authority, and criticism and embarrassment, as well as in the total anxiety score. This is in line with earlier findings (Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010). Consistent with the earlier study, there was a lack of relevant differences linked to sex in the

the multidimensionality of social anxiety dimensions assertive expression of annoyance, disgust, or displeasure and interaction with strangers. Future studies will need to address whether those sex differences are a central characteristic of SP. Another limitation of existing self-report instruments for social anxiety is that they do not efficiently detect nongeneralized SP. For instance, Bhogal and Baldwin (2007) concluded that “the LSAS may not be as useful for individuals with very specific social fears, such as circumscribed fear of performing, with little anxiety in other situations, as the scores will be relatively low” (p. 218). Current self-report measures of SP are not structured well to map onto the dimensions of the SP construct. They only give a total score, and on the basis of this score people are said to have generalized or nongeneralized SP. With the SAQ-A30 it would be possible to have a different perspective regarding this question. People with SP are distributed along a continuum (as Carter & Wu, 2010, recently stated), ranging from people who fear all social situations (or better expressed, all types of situations) to people with fear of a situation (or, better expressed, situations of one type). As we have seen in this study, the mean score differences between people with SP being assessed with the SAQ-A30 in clinical settings and people from the community are large with regard to the five dimensions and total score of the questionnaire (Cohen's d N 1.30 in all the cases). When considering the new assessment approach to SP in this study, the more dimensions in which an individual scores high, the more generalized his or her SP will be. Cutoff scores for each dimension and, separately, for women and men would tell us how high a score is in a specific dimension. The SAQ-A30 would give us more specific information about the type of social fears a person has than any other self-report measure for SP used today (see Bhogal & Baldwin, 2007, for more comments about this matter). Accordingly, it is interesting to note that the LSAS-SR, and other widely used social anxiety questionnaires such as the SPIN, the SPS (Social Phobia Scale; Mattick & Clarke, 1998), and the SIAS (Social Interaction and Anxiety Scale; Mattick & Clarke, 1998), among others (see Caballo, Salazar, Irurtia, Arias, Hofmann, et al., 2010), rarely include items about interactions with the opposite sex or assertive expression of annoyance, disgust, or displeasure. These are two of the basic, stable dimensions of the SAQ-A30, and it is unclear why these two dimensions are not more consistently represented in the established instruments. Empirically developed measures, like the SAQ-A30, could offer a rather different perspective than traditional self-reported measures for SP.

325

Finally, certain limitations of the present study should be stated. The seven-item Likert scale format of the first version of the questionnaire in Study 1 was changed to a five-item Likert scale format in Study 2. Although this five-item format has received strong support in further studies (e.g., Caballo, Salazar, Arias, et al., 2010), the change could represent a limitation of differences between formats. A second question refers to the lack of measures for texting the discriminant validity of the questionnaire. Although we have found (see Caballo, Salazar, Irurtia, Arias, & Guillen, 2010) that the SAQ-A30 behaves in exactly the same way as other SP questionnaires, such as the LSAS, the SPIN, and the SPAI regarding personality disorders and other Axis I disorders measured by the Millon Clinical Multiaxial Inventory–III (MCMI-III; Millon, Davis, & Millon, 1994), we need to address this topic in greater depth in the future. A third question is that no procedures were undertaken to confirm the absence of social anxiety disorder in the nonclinical groups. Although the latest epidemiology studies on the prevalence of social anxiety in Spain and other Latin American countries show that it is as low as 0.6% (Spain; Haro et al., 2008), 2.8% (Colombia; Posada-Villa et al., 2008) or 1.4% men and 2.6% women (Mexico; Medina-Mora et al., 2008), this will probably not have any significant impact on the means of the community samples but it should be noted as a limitation of this work. A fourth question involves patient diagnosis. All the patients were recruited from community clinics rather than research centers. Although this maximized the external validity of the study, we cannot verify the reliability or validity of each clinic's diagnostic process. Although in Study 2 a cutoff score on the LSAS-SR (Liebowitz, 1987) was used as a prerequisite for the patient sample, it was very difficult to verify that all the clinical centers applied the same semistructured interviews. Although the diagnosis of SP was a requirement for selecting patient samples (DSMIV-TR or ICD-10 criteria), the differences in resources, people, and countries preclude us from addressing that question as we would have liked to do. A next step in the research with the SAQA30 will be validation with a semistructured diagnostic interview, albeit only in one or two countries. A further limitation of this research is that the questionnaire has yet to be used as a preor posttreatment measure. Although we have already begun to move in this direction, we believe that this study contributes to the field of social anxiety assessment and raises important issues about the construct of SP and its measurement.

326

caballo et al.

Appendix A. Social Anxiety Questionnaire for Adults (SAQ-A30) a Below are a series of social situations that may or may not cause you UNEASE, STRESS, or NERVOUSNESS. Please place an “X” on the number next to each social situation that best reflects your reaction, where "1" represents no unease, stress, or nervousness and "5" represents very high or extreme unease, stress, or nervousness. If you have never experienced the situation described, please imagine what your level of UNEASE, STRESS, or NERVOUSNESS might be if you were in that situation, and rate how you imagine you would feel by placing an “X” on the corresponding number. Level of Unease, Stress, or Nervousness Not at all or very slight 1

Slight

Moderate

High

2

3

4

Very high or extremely high 5

Please rate all the items and do so honestly; do not worry about your answer because there are no right or wrong ones. 1. Greeting someone and being ignored

2. Having to ask a neighbor to stop making noise 3. Speaking in public 4. Asking someone attractive of the opposite sex for a date 5. Complaining to the waiter about my food 6. Feeling watched by people of the opposite sex 7. Participating in a meeting with people in authority 8. Talking to someone who isn't paying attention to what I am saying 9. Refusing when asked to do something I don't like doing 10. Being mugged or robbed by an armed gang 11. Making new friends 12. Telling someone that they have hurt my feelings 13. Having to speak in class, at work, or in a meeting 14. Maintaining a conversation with someone I've just met 15. Expressing my annoyance to someone that is picking on me 16. Greeting each person at a social meeting when I don't know most of them 17. Being teased in public 18. Talking to people I don't know at a party or a meeting 19. Being asked a question in class by the teacher or by a superior in a meeting 20. Looking into the eyes of someone I have just met while we are talking 21. Being asked out by a person I am attracted to 22. Making a mistake in front of other people

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

23. Attending a social event where I know only one person 24. Starting a conversation with someone of the opposite sex that I like 25. Being reprimanded about something I have done wrong 26. While having dinner with colleagues, classmates or workmates, being asked to speak on behalf of the entire group 27. One of my parents getting seriously ill 28. Telling someone that their behavior bothers me and asking them to stop 29. Asking someone I find attractive to dance 30. Being criticized 31. Talking to a superior or a person in authority 32. Telling someone I am attracted to that I would like to get to know them better

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1 2 3 4 5 1 2 3 4 5 1 1 1 1

2 2 2 2

3 3 3 3

4 4 4 4

5 5 5 5

Note. Items 10 and 27 are control items and do not count at all for the dimensions score or total score of the questionnaire. A score of 1 or 2 on both items is suspicious that the questionnaire could have been answered at random. a Reproduced with permission from Caballo, Salazar, Arias, et al. (2010).

References Alonso, J., & Kessler, R. C. (2008). Prevalence and treatment of mental disorders in Germany: Results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) Survey. In R. C. Kessler & T.B. Üstün (Eds.), The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders (pp. 331–345). New York: Cambridge University Press. American Psychiatric Association (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.) Washington, DC: Author. Arbabzadeh-Bouchez, S., Gasquet, I., Kovess-Masfety, V., NegrePages, L., & Lépine, J. P. (2008). The prevalence of mental disorders and service use in France: Results from a National Survey 2001–2002. In R. C. Kessler, & T. B. Üstün (Eds.), The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders (pp. 305–330). New York: Cambridge University Press. Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397–438. Baker, S. L., Heinrichs, N., Kim, H., & Hofmann, S. G. (2002). The Liebowitz Social Anxiety Scale as a self-report instrument: A preliminary psychometric analysis. Behaviour Research and Therapy, 40, 701–715. Bhogal, K. S., & Baldwin, D. S. (2007). Pharmacological treatment of social phobia. Psychiatry, 6, 217–223. Bobes, J., Badía, X., Luque, A., García, M., González, M. P., & Dal-Ré, R. (1999). Validation of the Spanish version of the Liebowitz Social Anxiety Scale, Social Anxiety and Distress Scale and Sheehan Disability Inventory for the evaluation of social phobia. Medicina Clínica, 112, 530–538. Bourdon, K. H., Boyd, J. H., Rae, D. S., Burns, B. J., Thompson, J. W., & Locke, B. Z. (1988). Gender differences in phobias: Results of the ECA Community Survey. Journal of Anxiety Disorders, 2, 227–241. Caballo, V. E., Salazar, I. C., Arias, B., Irurtia, M. J., Calderero, M., & the CISO-A Research Team Spain. (2010). Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students:

the multidimensionality of social anxiety Similarities and differences among degree subjects and regions. Behavioral Psychology/Psicologia Conductual, 18, 5–34. Caballo, V. E., Salazar, I. C., Irurtia, M. J., Arias, B., & Guillen, J.L. (2010). Relaciones entre ansiedad social y rasgos, estilos y trastornos de la personalidad [Relationships between social anxiety and personality traits, styles, and disorders]. Behavioral Psychology/Psicologia Conductual, 18, 259–276. Caballo, V. E., Salazar, I. C., Irurtia, M. J., Arias, B., Hofmann, S. G., & the CISO-A Research Team. (2008). Social anxiety in 18 nations: Sex and age differences. Behavioral Psychology/ Psicologia Conductual, 16, 163–187. Caballo, V. E., Salazar, I. C., Irurtia, M. J., Arias, B., Hofmann, S. G., & the CISO-A Research Team. (2010). Measuring social anxiety in 11 countries: Development and validation of the Social Anxiety Questionnaire for Adults. European Journal of Psychological Assessment, 26, 95–107. Carter, S. A., & Wu, K. D. (2010). Relations among symptoms of social phobia subtypes, avoidant personality disorder, panic and depression. Behavior Therapy, 41, 2–13. Connor, K. M., Davidson, J. R. T., Churchill, L. E., Sherwood, A., Foa, E., & Weisler, R. H. (2000). Psychometric properties of the Social Phobia Inventory (SPIN): A new self-rating scale. British Journal of Psychiatry, 176, 379–386. Cox, B. J., Clara, I. P., Sareen, J., & Stein, M. B. (2008). The structure of feared social situations among individuals with a lifetime diagnosis of social anxiety disorder in two independent nationally representative mental health surveys. Behaviour Research and Therapy, 46, 477–486. Davidson, J. R., Miner, C. M., De Veaugh-Geiss, J., Tupler, L.A., Colket, J. T., & Potts, N. L. (1997). The Brief Social Phobia Scale: A psychometric evaluation. Psychological Medicine, 27, 161–166. DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage. Di Nardo, P. A., Brown, T. A., & Barlow, D. H. (1994). Anxiety Disorders Interview Schedule for DSM-IV: Lifetime version (ADIS-IV-L). Albany, NY: Graywind. Fehm, L., Beesdo, K., Jacobi, F., & Fiedler, A. (2008). Social anxiety disorder above and below the diagnostic threshold: Prevalence, comorbidity and impairment in the general population. Social Psychiatry and Psychiatric Epidemiology, 43, 257–265. Fresco, D. M., Coles, M. E., Heimberg, R. G., Liebowitz, M. R., Hami, S., Stein, M. B., & Goetz, D. (2001). The Liebowitz Social Anxiety Scale: A comparison of the psychometric properties of self-report and clinician-administered formats. Psychological Medicine, 31, 1025–1035. Furmark, T., Tillfors, M., Everz, P. -O., Marteinsdottir, I., Gefvert, O., & Fredrikson, M. (1999). Social phobia in the general population: Prevalence and sociodemographic profile. Social Psychiatry and Psychiatric Epidemiology, 34, 416–424. García-López, L. J., Olivares, J., Hidalgo, M. D., Beidel, D. C., & Turner, S. M. (2001). Psychometric properties of the Social Phobia and Anxiety Inventory, the Social Anxiety Scale for Adolescents, the Fear of Negative Evaluation Scale, and the Social Avoidance and Distress Scale in an adolescent Spanish-speaking sample. Journal of Psychopathology and Behavioral Assessment, 23, 51–59. González, M. P., Bobes, J., García, M., Badía, X., Luque, A., & Dal-Ré, R. (1998). Assessing social phobia. The Spanish validation of the “gold standard” clinical scales: The LSAS and the SADS. European Neuropsychopharmacology, 8(Suppl. 2), 259–260. Haro, J. M., Alonso, J., Pinto-Meza, A., Vilagut Saiz, G., Fernández, A., Codony, M., et al. (2008). The epidemiology

327

of mental disorders in the general population of Spain. In R.C. Kessler, & T. B. Üstün (Eds.), The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders (pp. 406–430). New York: Cambridge University Press. Heimberg, R. G., Hope, D. A., Dodge, C. S., & Becker, R. E. (1990). DSM-III-R subtypes of social phobia: Comparison of generalized social phobics and public speaking phobics. Journal of Nervous and Mental Disease, 173, 172–179. Heinrichs, N., Rapee, R. M., Alden, L. A., Bögels, S., Hofmann, S. G., Oh, K. J., & Sakano, Y. (2006). Cultural differences in perceived social norms and social anxiety. Behaviour Research and Therapy, 44, 1187–1197. Herman, A. A., Williams, D., Stein, D. J., Seedat, S., Heeringa, S. G., & Moomal, H. (2008). The South African Stress and Health Study (SASH): A foundation for improving mental health care in South Africa. In R. C. Kessler, & T. B. Üstün (Eds.), The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders (pp. 238–264). New York: Cambridge University Press. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185. Hughes, A. A., Heimberg, R. G., Coles, M. E., Gibb, B. E., Liebowitz, M. R., & Schneier, F. R. (2006). Relations of the factors of the tripartite model of anxiety and depression to types of social anxiety. Behaviour Research and Therapy, 44, 1629–1641. Kessler, R. C., Berglund, P. A., Chiu, W. -T., Demler, O., Glantz, M., Lane, M. C., et al. (2008). The National Comorbidity Survey Replication (NCS-R): Cornerstone in improving mental health and mental health care in the United States. In R. C. Kessler, & T. B. Üstün (Eds.), The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders (pp. 165–209). New York: Cambridge University Press. Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K.R., & Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 593–602. Kessler, R. C., Stein, M. B., & Berglund, P. (1998). Social phobia subtypes in the National Comorbidity Survey. American Journal of Psychiatry, 155, 613–619. Kessler, R. C., & Üstün, T. B. (Eds.). (2008). The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders New York: Cambridge University Press. Levin, J. B., Marom, S., Gur, S., Wechter, D., & Hermesh, H. (2002). Psychometric properties and three proposed subscales of a self-report version of the Liebowitz Social Anxiety Scale translated into Hebrew. Depression and Anxiety, 16, 143–151. Liebowitz, M. R. (1987). Social phobia. Modern Problems in Pharmacopsychiatry, 22, 141–173. Marsh, H. (2007). Application of confirmatory factor analysis and structural equation modeling in sport and exercise psychology. In G. Tenenbaum, & R. C. Eklung (Eds.), Handbook of sport psychology (pp. 774–798). (3rd ed.). Hoboken, NJ: Wiley. Marsh, H., Hau, K. T., & Grayson, D. (2005). Goodness of fit evaluation in structural equation modeling. In A. MaydeuOlivares, & J. McArdel (Eds.), Contemporary psychometrics: A festchrift for Roderick P. McDonald (pp. 275–340). Mahwah, NJ: Erlbaum. Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny and social interaction anxiety. Behaviour Research and Therapy, 36, 455–470. Medina-Mora, M. E., Borges, G., Lara, C., Benjet, C., Fleis, C., Rojas, et al. (2008). The Mexican National Comorbidity

328

caballo et al.

Survey (M-NCS): Overview and results. In R. C. Kessler, & T. B. Üstün (Eds.), The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders (pp. 144–164). New York: Cambridge University Press. Mennin, D. S., Fresco, D. M., Heimberg, R. G., Schneier, F. R., Davies, S. O., & Liebowitz, M. R. (2002). Screening for social anxiety disorder in the clinical setting: Using the Liebowitz Social Anxiety Scale. Journal of Anxiety Disorders, 16, 661–673. Merikangas, K. R., Avenevoli, S., Acharyya, S., Zhang, H., & Angst, J. (2002). The spectrum of social phobia in the Zurich cohort study of young adults. Society of Biological Psychiatry, 51, 81–91. Millon, T., Davis, R. D., & Millon, C. (1994). Manual for the Millon Clinical Multiaxial Inventory-III. Minneapolis, MN: National Computer Systems. Muthén, L. K., & Muthén, B. O. (2010). MPlus (Version 6.0) [Computer software]. Los Angeles: Author. Oakman, J., Van Ameringen, M., Mancini, C., & Farvolden, P. (2003). A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. Journal of Clinical Psychology, 59, 149–161. Olivares, J., García-López, L. J., Hidalgo, M. D., Turner, S. M., & Beidel, D. C. (1999). The Social Phobia and Anxiety Inventory: Reliability and validity in an adolescent Spanish population. Journal of Psychopathology and Behavioral Assessment, 21, 67–78. Osman, A., Barrios, F. X., Haupt, D., King, K., Osman, J. R., & Slavens, S. (1996). The Social Phobia and Anxiety Inventory: Further validation in two nonclinical samples. Journal of Psychopathology and Behavioral Assessment, 18, 35–47. Osório, F. L., Crippa, J. A., & Loureiro, S. R. (2007). A study of the discriminative validity of a screening tool (MINI-SPIN) for social anxiety disorder applied to Brazilian university students. European Psychiatry, 22, 239–243. Pollard, C. A., & Henderson, J. G. (1988). Four types of social phobia in a community sample. Journal of Nervous and Mental Disease, 176, 440–445. Posada-Villa, J., Rodríguez, M., Duque, P., Garzón, A., AguilarGaxiola, S., & Breslau, J. (2008). Mental disorders in Colombia: Results from the World Mental Health Survey. In R. C. Kessler, & T. B. Üstün (Eds.), The WHO Mental Health Surveys: Global perspectives on the epidemiology of mental disorders (pp. 131–143). New York: Cambridge University Press. Radomsky, A. S., Ashbaugh, A. R., Saxe, M. L., Ouimet, A. J., Golden, E. R., Lavoie, S. L., & O'Connor, K. (2006). Psychometric properties of the French and English versions of the Social Phobia Inventory. Canadian Journal of Behavioural Science, 38, 354–360. Rodebaugh, T. L., Woods, C. M., & Heimberg, R. G. (2007). The reverse of social anxiety is not always the opposite: The reverse-scored items of the Social Interaction Anxiety Scale do not belong. Behavior Therapy, 38, 192–206.

Ruscio, A. M., Brown, T. A., Chiu, W. T., Sareen, J., Stein, M. B., & Kessler, R. C. (2008). Social fears and social phobia in the USA: Results from the National Comorbidity Survey Replication. Psychological Medicine, 38, 15–28. Safren, S. A., Heimberg, R. G., Horner, K. J., Juster, H. R., Schneier, F. R., & Liebowitz, M. R. (1999). Factor structure of social fears: The Liebowitz Social Anxiety Scale. Journal of Anxiety Disorders, 13, 253–270. Scientific Software International (2006). LISREL (Version 8.8) [Computer software]. Lincolnwood, IL: Author. Sosic, Z., Gieler, U., & Stangier, U. (2008). Screening for social phobia in medical in- and outpatients with the German version of the Social Phobia Inventory (SPIN). Journal of Anxiety Disorders, 22, 849–859. Stein, M. B., Walker, J. R., & Forde, D. R. (1994). Setting diagnostic thresholds for social phobia: Considerations from a community survey of social anxiety. American Journal of Psychiatry, 151, 408–412. Terra, M. B., Barros, H. M. T., Stein, A. T., Figueira, I., Athayde, L. D., Gonçalves, M. S., . . ., & da Silveira, D. X. (2006). Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Revista Brasileira de Psiquiatria, 28, 265–269. Turner, S. M., Beidel, D. C., Dancu, C. V., & Stanley, M. A. (1989). An empirically derived inventory to measure social fears and anxiety: The Social Phobia and Anxiety Inventory. Psychological Assessment, 1, 35–40. Turner, S. M., Beidel, D. C., & Townsley, R. M. (1992). Social phobia: A comparison of specific and generalized subtypes and avoidant personality disorder. Journal of Abnormal Psychology, 101, 326–331. Turner, S. M., Stanley, M. A., Beidel, D. C., & Bond, L. (1989). The Social Phobia and Anxiety Inventory: Construct validity. Journal of Psychopathology and Behavioral Assessment, 11, 221–234. Van Dam-Baggen, R., Kraaimaat, F., & Elal, G. (2003). Social anxiety in three Western societies. Journal of Clinical Psychology, 59, 673–686. World Health Organization (1992). International classification of diseases (10th ed.). Geneva: Author. World Health Organization (1997). Composite International Diagnostic Interview (CIDI). Geneva: Author. Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research and Evaluation, 12, 1–26. Yonkers, K. A., Dyck, I. R., & Keller, M. B. (2001). An eightyear longitudinal comparison of clinical course and characteristics of social phobia among men and women. Psychiatric Services, 52, 637–643.

R E C E I V E D : October 8, 2010 A C C E P T E D : July 11, 2011 Available online 21 July 2011