The Effectiveness of Acceptance and Commitment Therapy Bibliotherapy for Enhancing the Psychological Health of Japanese College Students Living Abroad

Available online at www.sciencedirect.com

Behavior Therapy 42 (2011) 323 – 335

www.elsevier.com/locate/bt

The Effectiveness of Acceptance and Commitment Therapy Bibliotherapy for Enhancing the Psychological Health of Japanese College Students Living Abroad Takashi Muto Doshisha University

Steven C. Hayes Tami Jeffcoat University of Nevada, Reno

International students often experience significant psychological distress but empirically tested programs are few. Broadly distributed bibliotherapy may provide a costeffective approach. About half of the Japanese international students in a western university in the United States (N = 70) were randomly assigned to a wait-list or to receive a Japanese translation of a broadly focused acceptance and commitment therapy (ACT) self-help book. Although recruited without regard to health status, the sample was highly distressed with nearly 80% exceeding clinical cutoffs on one or more measures. After a 2-months period for the

As part of the editorial process, information about two process measures was removed to shorten the manuscript. Values consistency from the Personal Values Questionnaire (PVQ; Blackledge & Ciarrochi, 2006) and accepting without judgment from the Kentucky Inventory of Mindfulness Skills (KIMS; Baer, Smith, & Allen, 2004). Both were impacted by ACT and values consistency mediated GHQ outcomes. Values consistency was significantly worse at pre for those in the ACT condition. Adjusting for that difference did not impact outcomes. Change scores were used in meditational analysis for all process measures because of this predifference, however. Details are available from the authors. The authors would like to thank Matt McKay, Elizabeth Adamska, Mariko Mukaibo, Sayo Uyama, and the entire staff at New Harbinger Publications for their generous and ethical support of research focused on their self-help volumes. Volumes were supplied by the publisher at cost for evaluation. The authors of this study received no financial benefit from this study per se, however, it should be noted that the book evaluated generates royalties for Dr. Hayes. Address correspondence to Steven C. Hayes, Ph.D., Department of Psychology, University of Nevada, Reno, NV 89667-0062; e-mail: [email protected]. 0005-7894/11/323–335/$1.00/0 © 2011 Association for Behavioral and Cognitive Therapies. Published by Elsevier Ltd. All rights reserved.

first treatment group to read the book and a 2-month follow up, wait-list participants also received the book. Students receiving the book showed significantly better general mental health at post and follow up. Moderately depressed or stressed, and severely anxious students showed improvement compared to those not receiving the book. These patterns were repeated when the wait-list participants finally received the book. Improvements in primary outcomes were related to how much was learned about an ACT model from the book. Follow-up outcomes were statistically mediated by changes in psychological flexibility, but not vice versa and were moderated by level of initial flexibility. Overall, the data suggest that ACT bibliotherapy improved the mental health and psychological flexibility of Japanese international students.

ALTHOUGH SOCIETY TENDS TO view the college years as carefree, a significant segment of the college population is distressed (Kadison & DiGeronimo, 2004). International students provide a case example (Johnson & Sandhu, 2006). Thrust into a different cultural milieu, far from home, students studying abroad may experience culture shock (Pederson, 1995) and significant emotional distress (Singaravelu & Pope, 2006), including depression, anxiety, and difficulties in interpersonal relationships (Matsumoto et al., 2001). Cultural factors can prevent international students from seeking out mental health services. For

324

muto et al.

example, Japanese international students in the United States see psychological difficulties as being more shameful and stigmatizing than do either Caucasian students or non-Asian ethnic minority students (Masuda et al., 2009). Sharing negative information with others is more difficult (Shin, 2002); concerns over shaming one's family are greater (Uba, 1994); and as a result, mental health services of any kind are greatly underutilized (Brinson & Kottler, 1995; Sue, Fujino, Hu, Takeuchi, & Zane, 1991). Broad distribution of self-help books to the general international student population provides one possible solution to this conundrum. Bibliotherapy is widely used and cost-effective in many areas of mental health (see Den Boer, Wiersma, & Van Den Bosch, 2004, and Gregory, Schwer-Canning, Lee, & Wise, 2004, for recent meta-analyses). Books addressing the stress of international students need to deal with a broad spectrum of possible problems and cultural factors of importance. The present study was focused on Japanese international students. East-Asian students are known to adjust better if they have good emotion regulation skills, greater openness and flexibility, and good critical thinking skills (Matsumoto, LeRoux, Bernhard, & Gray, 2004); their motivation is self-determined (Chirkov, Vansteenkiste, Tao, & Lynch, 2007); and they are more accepting of discomforting thoughts (Ting & Hwang, 2009). Acceptance and commitment therapy (ACT; Hayes, Strosahl, & Wilson, 1999), a contextual or “thirdwave” (Hayes, 2002) form of behavioral and cognitive therapy (Hayes, Levin, Plumb, Boulanger, & Pistorello, in press), seems to fit this profile. ACT uses acceptance and mindfulness strategies, and commitment and behavioral activation strategies to increase psychological flexibility. ACT is broadly beneficial in several areas of behavioral and mental health, and appears to work because it fosters open, accepting, and flexible forms of emotional and cognitive regulation, and greater guidance by chosen values (Hayes, Luoma, Bond, Masuda, & Lillis, 2006). These are processes that seem to parallel the helpful adjustment strategies of East-Asian students just listed (Chirkov et al., 2007; Matsumoto et al., 2004; Ting & Hwang, 2009). ACT has also been used successfully with Asian populations in several problem areas (e.g., Lundgren, Dahl, Yardi, & Melin, 2008; Masuda, Muto, Hayes, & Lillis, 2008; Takahashi, Muto, Tada, & Sugiyama, 2002). There are several studies that have examined bibliotherapy with the targeted problems of college students (e.g., Register, Beckham, May, & Gustafson,

1991; Wenfeng, Meiling, & Lianyun, 2004) but none have been distributed on a population basis. The purpose of the present study was to provide preliminary evidence of the impact of an ACT self-help book on the mental health and psychological flexibility of Japanese international students recruited without regard to health status.

Method participants and recruitment The participants were 70 Japanese international students (44 females; average age 23.6, range 20–26) enrolled at the University of Nevada, Reno (UNR), during the 2008–2009 academic year. In accord with the population-based focus of the present study, participants constituted a majority of all 138 Japanese international students at the university at the time. Power analysis, conducted using G⁎Power (Faul, Erdfelder, Lang, & Buchner, 2007), showed that the sample had adequate power (N.9) to detect the effect sizes previous ACT studies have shown (using the effect size estimate provided by Öst [2008] for ACT vs. wait-list, d = .96). Each participant had to be a Japanese international student enrolled at UNR, at least 18 years of age, with access to the Internet, and have selfreported proficiency in reading both Japanese and English. In order to be admitted to the university, international students had to have an overall score on the Test of English as a Foreign Language (TOEFL) at the borderline of fair and intermediate (61 of 120 overall on the Internet-based test), a level that qualifies the student as an “independent user” in the Common European Framework of Reference for Languages (see Tannenbaum & Wylie, 2008). There were no exclusion criteria other than failure to complete baseline measures. Three of the participants were psychology majors (4%) and none had taken classes with the investigators, limiting obvious demand characteristics. Students were recruited through flyers on the billboards of all buildings on the UNR campus and via an e-mail sent from the UNR Office of International Students and Scholars to all Japanese students at the university. The e-mail and flyers directed possible participants to a Web site where the study was described. The Web site described the purpose of the study, which was to improve general mental health and overall functioning; to better cope with stress, anxiety, or depression; and to learn to behave more congruently with personal values. Participants were told that they would be reading a Japanese translation of Get Out of Your Mind and

act bibliotherapy for japanese international students

325

Into Your Life (Hayes & Smith, 2005). The Web site also described the assessments, incentives for participation, researchers, ethical issues, and signup procedures.

participants to complete questionnaires or quizzes. This enabled the researchers to know on an ongoing basis who had complete questionnaires and quizzes.

Workbook The Japanese translation (Hayes & Smith, 2008) was done by four native-speaking Japanese doctoral -level behavior analysts or behavior therapists well versed in ACT—Takashi Muto, Hiroaki Harai, Masako Yoshioka, and Miyo Okajima. The translation was not merely literal but instead modified exercises, concepts, examples, and metaphors to fit the Japanese language community. No modifications were felt to be necessary in the central concepts of ACT to fit a Japanese reader—the issue was the means of presenting those ideas. The modifications included such things as changing parts of the book related to etymology of English concepts to refer instead to the origin of Japanese terms in ancient Chinese characters that made a similar point, changing terms for which there was no parallel term in Japanese, and changing metaphors and idioms to those that made the same functional point but were well known in Japanese culture.

Outcome Measures The present study attempted to reach the majority of a demographic group, and thus measures were deliberately general. The primary outcome measure was the General Health Questionnaire–12 (GHQ; Goldberg et al., 1997). The GHQ is a widely used instrument that successfully screens for psychiatric problems in a general population by measuring behaviors, functioning, and distress that are predictive of nonpsychotic psychiatric disorders on a 4-point Likert rating scale. In this study we used the 12-item version and a Likert scoring style (0–1–2–3) for each item, resulting in scores that could range from 0 to 36 with higher scores indicating greater psychological difficulty. The GHQ has very good reliability and validity (Goldberg et al., 1997). Its psychometric properties are undiminished when delivered via the Internet (Vallejo, Jordan, Diaz, Comeche, & Ortega, 2006). The English-language version is valid when used with late adolescents and young adults (Banks, 1983; Tait, Hulse, & Robertson, 2002). Alpha levels ranged from .82 to .86 in its validation studies (Goldberg & Williams, 1988) and was .82 in the present study. GHQ was considered the primary outcome measure because it fit the broad focus of the intervention. Secondary outcome measures were the depression, anxiety, and stress subscales of the Depression Anxiety Stress Scales (DASS; Lovibond & Lovibond, 1995). The DASS is a widely used screening instrument that is especially appropriate for general populations. It was used in the present study to identify those individuals with particular difficulties in the areas of depression, anxiety, and stress. The DASS has good convergent and discriminant validity when compared with other validated measures of depression and anxiety (Crawford & Henry, 2003). In the original validation study Chronbach's alpha ranged from .82 to .90 across the three scales (Lovibond & Lovibond, 1995); in the present study alpha was .83 for depression, .74 for anxiety, and .79 for stress. Its psychometric properties are not significantly diminished when delivered via the Internet (Zlomke, 2009).

measurement procedures and devices Measurement Process All measures and quizzes were presented online in English using Survey Monkey (http://www. surveymonkey.com/) with SSL encryption of the links and response pages during transmission. English assessment devices were used because only one of the needed measures had been validated in Japanese and it did not seem wise to mix English and Japanese assessment devices. In addition, unlike the workbook that often presented innovative ideas, measures items were generally readily understandable to average readers as any items that were not would be weeded out by psychometric evaluation criteria. Given the English abilities of the participants it thus seemed best to use English measures and a Japanese workbook. The measures used were also Internet based. A number of studies have investigated the psychometric soundness of administering self-report measures online for assessing problems and in general the results are reassuring (e.g., Hedman et al., 2010; Read, Farrow, Jaanimägi, & Ouimette, 2009). The outcomes measures used in the present study have been shown to be valid when delivered in an Internetbased format as we document below. E-mail addresses of participants and their study ID numbers imported to our management account, and individual e-mails with coded links were sent to

Process Measure The Acceptance and Action Questionnaire (AAQ; Hayes & Strosahl, 2004) was used as a general measure of psychological flexibility—the ability to contact difficult emotions and thoughts

326

muto et al.

and still engage in values-based action. The most recent version (Bond et al., under submission) was used. It contains 10 items rated on a 7-point Likerttype scale; higher scores indicate greater psychological flexibility and lower scores indicate more experiential avoidance and inflexibility. The AAQ has good internal reliability with a mean alpha of .83 in its validation studies, and .70 in the current study, which is considered adequate (George & Mallery, 2003). Adherence Quizzes Six different online multiple-choice quizzes were administered. Each included 10 items relevant to either two or three chapters of the book, depending on the specific quiz. Quizzes were given to assess objective understanding of book content, and also to collect qualitative and quantitative data related to book exercises. These quizzes were completed at participants' own pace at any time during the 8 weeks they were scheduled to read the book.

procedure A randomized wait-list design was used, with one group receiving the workbook, having 8 weeks to read it, and then a 2-month follow-up. Three assessments were taken during this period (pre, post, and follow-up). Those randomized to the wait-list had the same three assessments, but then were also given the workbook and, after 8 weeks to read it, a final assessment was taken only for this group. The steps in the design were as follows. Week 0 All participants completed a battery of questionnaires via a link sent to the participant's e-mail. Those completing the baseline were entered into a lottery and the winner was paid $20.00. Treatment assignment was determined by the senior investigator using a random number table to assign conditions to participant numbers. At the time of randomization, the investigator did not know the identity of the participants or their baseline scores. The names of participants assigned to each condition were revealed and participants were notified using standard e-mails for each arm of the study. Weeks 1–8 Those in the workbook condition picked up the books and were given investigator-designated passwords for access to a message board on the study Web site (entitled in Japanese “Your Life 08”). The message board and all six quizzes became immediately available to participants in the workbook condition via links sent to the participant's e-mail.

The trial period lasted 8 weeks; during this time participants in the workbook condition were asked to read the workbook and complete its exercises at their own pace and to complete each of the quizzes over specific chapters. If participants had questions while reading and engaging in exercises during the trial period, they were free to post anonymous questions at any time and could get some answers and comments from investigators and other participants. In order to provide incentives for completion, each quiz had two $20 lottery prizes, awarded by a random number drawing conducted during week 8 of the study. Participants had to score 60% correct or higher on a given quiz to be eligible. The total time commitment differed for individuals but according to participant reports, it varied from 20 to 28 total hours including reading, six quizzes, and a maximum of four sets of study questionnaires. Week 9 The second assessment battery was administered online to all participants and a lottery prize was distributed as with the baseline battery. Weeks 10–18 The participants in the workbook condition continued to have access to the message board in order to communicate with each other. Week 19 A third assessment battery (the 2-month followup) was administered online to all participants and a lottery prize was distributed as with the baseline battery. Participation in the study was then concluded for those in the workbook condition. Weeks 20–27 Participants in the wait-list condition received the workbook and participated precisely as had those in the workbook condition during weeks 1–8 including availability of the quizzes, lottery opportunities, and message-board communication. Week 28 Participants in the wait-list condition completed a final assessment battery (the 4-month follow-up) with a lottery as before. This second follow-up was, in effect, the first postworkbook assessment for wait-list participants. All participants also received $20 for completion of the final battery.

Results A flow chart of participation is shown in Fig. 1. All totaled, 60 of the 70 participants (86%) had complete data in the main part of the study (pre,

act bibliotherapy for japanese international students

Target Population: Japanese International Students at the University of Nevada, Reno (N = 138)

Agreed to Participate, Completed Pre Assessment and were randomly assigned (N = 70; 51% of target population)

Declined Participation (N=2; 2.8% of those eligible)

Allocated to Receive Workbook (N = 35; all received intervention)

Allocation

5 of 35 (14%) not assessed “too busy with academic work”

Post Test (2 months later)

4 of 35 (11%) not assessed “too busy with academic work”

2 of 35 (6%) not assessed “too busy with academic work”

First Follow-Up (2 months later)

All assessed All then sent workbook

35 (100%) in analysis 0 excluded

Analysis

35 (100%) in analysis 0 excluded

Second Follow-Up (2 months later)

7 of 35 (20%) not assessed All “too busy” 0 excluded from supplementary analysis

FIGURE 1

Allocated to Wait List (N = 35)

Participant flow through the study.

327

328

muto et al.

Table 1 Mean scores and standard deviations of all measures for the two condition ACT Workbook Condition

N except as noted: Outcome Measures GHQ GHQ N 11 (Distressed) DASS-Depression DASS-Depression N 13 (moderate or more) DASS-Anxiety DASS-Anxiety N 9 (moderate or more) DASS-Anxiety N 14 (severe or more) DASS-Stress DASS-Stress N 18 (distressed) Process Measures AAQ

Waitlist Condition

Baseline

Post

Follow-up

Baseline

Post

Follow-up

Second Follow-up

n = 35

n = 30

n = 27

n = 35

n = 31

n = 32

n = 24

14.71 (5.68) 18.19 (4.26) n = 22 13.20 (7.90) 21.43 (4.80) n = 14 15.83 (8.06) 19.26 (5.38) n = 27 21.89 (4.03) n = 19 17.71 (7.69) 25.00 (4.49) n = 14

10.03 (5.12) 11.00 (5.05) n = 18 11.33 (7.56) 17.27 (5.68) n = 11 12.53 (8.20) 15.45 (7.49) n = 22 17.57 (7.37) n = 14 13.87 (6.21) 15.45 (5.66) n = 11

9.59 (3.68) 10.81 (3.89) n = 16 12.07 (7.55) 16.80 (6.34) n = 10 13.63 (6.95) 16.32 (6.30) n = 19 17.54 (6.54) n = 13 14.67 (7.25) 15.00 (4.14) n = 10

13.51 (5.36) 16.75 (4.77) n = 20 12.51 (9.32) 22.29 (6.60) n = 14 14.11 (7.79) 18.35 (6.02) n = 23 21.33 (5.33) n = 15 16.00 (8.43) 25.17 (4.47) n = 12

12.58 (5.41) 15.12 (5.40) n = 17 9.10 (7.00) 16.00 (4.08) n = 13 10.45 (6.75) 12.30 (5.96) n = 20 14.00 (6.66) n = 12 12.77 (7.51) 19.09 (4.13) n = 11

13.13 (7.31) 15.39 (7.95) n = 18 13.25 (10.51) 20.57 (6.77) n = 14 14.75 (9.00) 16.95 (9.50) n = 21 20.77 (8.31) n = 13 15.81 (8.51) 20.83 (7.55) n = 12

9.33 (4.59) 9.45 (4.95) n = 11 9.42 (7.35) 15.20 (5.43) n = 10 9.92 (7.66) 11.88 (7.39) n = 16 13.00 (7.09) n=8 11.67 (8.14) 16.50 (5.73) n=8

40.49 (6.95)

44.30 (6.67)

44.70 (8.85)

43.37 (8.53)

43.48 (8.63)

42.72 (9.96)

46.42 (8.59)

post, and the first follow-up). In the wait-list arm, eight participants failed to complete the fourth and final assessment after they too received the book. Including this final assessment, 84% of the participants provided at least one postbook assessment package, and 74% provided complete data. Across both arms of the study, during the 8 weeks the workbook was being read, 74.3% of the participants completed at least one quiz. On average, 4.97 quizzes were completed (83% of those possible) by those who took them, with an average percentage correct of 73.4% (SD = 14.91). Only eight messages or questions were posted on the message board throughout the study, suggesting that it was not a major part of the intervention. Table 1 shows the obtained means for all outcome measures for the two conditions for each measurement occasion. There were no prescore differences on any of these measures.

outcome analysis strategy Hierarchical linear modeling (HLM) or mixedmodel repeated measures (MMRM) were used to investigate primary outcomes and processes using an intent-to-treat sample (Raudenbush & Bryk, 2001). Mixed regression models are a particularly powerful way to conduct an intent-to-treat analysis. Unlike normal analysis of variance/analysis of covariance or repeated measures models, these models use all available data from all subjects. This approach takes into account the obtained outcome and missingness for participants with

missing data, somewhat reducing the analytic problem presented by missing data. For example, estimates of ultimate treatment impact will be reduced for conditions with dropouts among participants doing poorly before producing missing data. The primary analyses were conducted on the first three assessment occasions (pre, post, and follow-up) as the design was fully balanced to that point. Supplementary analyses were used to address the impact of the workbook in the waitlist treatment arm. HLM analyses were initially conducted assuming a random intercept and a random slope, allowing the two to be correlated. If the HLM analyses did not converge or the Hessian matrix was not positive definite, an MMRM was conducted. Simpler models were tested in each analytic approach: in HLM, assuming correlated slopes and intercepts, and then a random interceptonly model; in MMRM, models using compound symmetry, Toeplitz, and compound symmetry heterogeneous covariance assumptions. Simpler models were used only if there was no significant difference in the fit as determined by comparison of nested models through the restricted log likelihood. Denominator degrees of freedom for the fixedeffects test statistics were based on the Sattherthwaite approximation. The time value for initial assessments in all HLM analyses was set to zero and time was modeled by the sequential assessment occasion. Effects sizes (Cohen's d) for F values were based on the method suggested for

act bibliotherapy for japanese international students 16

whereas the effect for time assessed the strength of impact of the book.

Mean General Mental Health

15

Waitlist Condition

14

13

Also Received Book

12

11

10

9

329

Workbook Condition

Pre

Post

Two Month Four Month Follow-Up Follow-Up

Assessment Period FIGURE 2 GHQ-adjusted means showing levels of general mental health for all assessment time periods for the workbook and wait-list conditions. Vertical bars show standard errors. Values are taken from an MMRM analysis with unstructured covariance.

repeated measures and multilevel designs by Rosenthal and Rosnow (1991; see also Verbeke & Molenberghs, 2000); for slopes as suggested by Raudenbush and Liu (2001); for MMRM contrasts as specified by Wackerly, Mendenhall, and Scheaffer (2008, p. 271); and for HLM results by examining end-point outcomes using the method suggested by Feingold (2009). The specific effect sizes reported in a given analysis were selected so as to characterize effects adequately while avoiding redundancy. Contrasts and correlations that were based on study hypotheses were tested using one-tailed tests. Effect sizes were discussed using the cutoffs suggested by Cohen (1988). In order to save text space, only the values for the Time × Treatment interactions in the main analysis will be placed in the text. A supplementary analysis using an MMRM analysis with an unstructured covariance addressed the impact of the workbook in the wait-list by examining the baseline and last follow-up data for both arms. In these supplementary analyses, the lack of a significant Time × Condition interaction indicated a similar impact of the book whether it was presented at the beginning or end of the study,

Primary Outcome Results: General Mental Health The average GHQ score at preassessment was 14.11 (SD = 5.51; range: 4–32). Forty two of the 70 participants (60%) were above the clinical cutoff (11) suggested for the Likert-style scoring of the GHQ (Goldberg et al., 1997), showing that this was a relatively distressed population despite the fact that it was not selected based on health need. Results of the workbook are shown in Fig. 2. An HLM model assuming a random and correlated intercept and slope and with an unstructured covariance structure best fit the data and showed no effect for treatment condition, but there was a significant effect for time, and the interaction of condition and time, F(1, 64.14) = 6.41, p = .014. Analysis of the significance of slopes within each condition showed no improvement in the wait-list condition: slope estimate = – .21, SE = .64, t(61.67) =– .32, p = .75, effect size = .08; but there was a significant and large improvement for participants who were given the workbook: slope estimate =–2.54, SE = .67, t(66.42) = –3.81, p = .000, effect size = .98. Wait-list participants also improved significantly and to the same degree as the initial book arm after receiving the book. Pre to final follow-up changes combining both arms of the study, after the book was received by both, were large and significant, F(1, 56.72) = 23.86, p b .001, effect size = 1.33. By the end of the study the average GHQ score fell below clinical means for the GHQ (9.71; Donath, 2001) in both arms of the study (workbook arm: 9.63, SE = 1.05; wait-list arm: 9.41, SE = .99). Primary Outcome Results: Distressed Subgroup HLM analysis of the 42 participants above the clinical cutoff on the GHQ (N11 at baseline), conducted as with the overall group, showed the same pattern as the overall outcomes, although with slightly larger effect sizes. There was no improvement in the wait-list condition: slope estimate = –.69, SE = .95, t(37.00) = –.73, p = .47, effect size = .24; but there was a significant and large improvement for participants who were given the workbook: slope estimate = –3.88, SE = .97, t(40.45) = –4.02, p = .000, effect size = 1.33). The workbook impacted GHQ scores equally in both arms of the study; the baseline to final follow-up effect was large and significant, F(1, 38.91)= 39.40, p b .001, effect size = 2.01. The reliability of these general mental health improvements in the distressed subsample was examined using the Jacobson methodology for assessing reliable change (Jacobson, Follette, & Revenstorf, 1984; Jacobson & Truax, 1991). A

330

muto et al.

change of 6.92 on the GHQ was considered reliable. GHQ scores from the HLM analysis were used in this analysis, providing a full intentto-treat analysis. Examining the baseline to followup data, in the wait-list condition 3 of 20 (15%) of the participants improved reliably and 2 of 20 (10%) deteriorated reliably for a net gain of 1. In contrast, 14 of 22 (64%) in the workbook condition improved reliably and no one deteriorated reliably. Analyzing net gains showed a large and significant difference in favor of the ACT workbook, χ2 (1) =13.24, p b .0002, one-tailed, effect size = 1.36. Clinical significance using Method C from the Jacobson methodology required both reliable change and a follow-up score of less than 11. Of the 22 workbook participants 13 (59%) showed clinically significant change at follow-up versus 2 of 20 (10%) wait-list participants—a large and significant difference, χ2 (1) = 8.96, p b .0002, one-tailed, effect size = 1.04. Primary Outcome Results: Effects on Prevention A nonparametric examination of the primary outcome was undertaken with this subgroup to see whether the workbook had a preventive effect. In the wait-list condition at the initial follow-up, 5 of 15 (33%) of the previous nondistressed sample (GHQ b 12) were now in the distressed range (GHQ N 11), while none of the 13 nondistressed workbook participants were now distressed, a significant difference, Fisher's exact probability = .03, one-tailed, effect size= .72. Secondary Outcome Results: Stress, Depression, and Anxiety DASS subscales were analyzed using participants who at baseline met the cutoff criteria (Lovibond & Lovibond, 1995) for moderate or severe difficulties in the specified area. In some cases, the number of participants in these analyses fell below the level recommended to generate reliable covariance estimates in mixed models (Raudenbush & Bryk, 2001, pp. 280–284). In these instances, classical repeated measures analysis of covariance methods were used with the baseline score as the covariate and the post and first follow-up as repeated measures. Since baseline values are a covariate in this classical analysis, treatment effects are shown either by a significant condition effect applying to both post and follow-up, or a significant Time × Condition interaction. A repeated measures analysis of covariance for those with moderate and above levels of stress (baseline DASS-S score above 18; N = 21) showed a significant difference between conditions, F(1, 18) = 6.93, p = .017, effect size = .86, but no effect for

time (that is, post and follow-up were similarly improved), and no Time × Condition interaction (see Table 1). A similar analysis for those with moderate and above levels of depression (baseline DASS-D score above 13; N = 23) showed no significant difference between conditions, and no effect for time, but did reveal a significant and large Time × Condition interaction, F(1, 20) = 4.70, p = .042, effect size = .97. The interaction occurred because the conditions were not significantly different from each other at post but workbook participants had significantly lower depression scores at follow-up (see Table 1). There were sufficient participants with moderate levels of anxiety (DASS-A N 9, N = 50, or 71% of the sample) to conduct a mixed-model analysis. An MMRM with unstructured covariance showed only a significant time effect, but no effect for condition, nor the interaction of Time × Condition. A repeated measures analysis of covariance for those with severe levels of anxiety, however (baseline DASS-A score above 14; N = 25), showed no significant difference between conditions, no effect for time, and a significant and large Time × Condition interaction, F(1, 22) = 4.24, p = .042, effect size = .88. The interaction occurred because those in the workbook condition improved across post and follow-up whereas wait-list participants improved at post and then reverted to baseline levels at follow-up (see Table 1). Omnibus evaluations of the workbook from pre to final follow-up for both arms of the study found that in each of these subsamples there was no difference caused by whether the workbook was received first or second, and there was a large and significant improvement in relevant scores at the end-point outcome, depression: F(1, 19) = 8.97, p = .007, effect size = 1.37; stress: F(1, 16) = 36.03, p = .000, effect size = 3.00; anxiety: F(1, 33) = 6.47, p = .016, effect size = .89; severe anxiety: F(1, 19) = 8.93, p = .008, effect size = 1.37). Changes of 10.3 (depression), 10.4 (stress), and 12.2 (anxiety) were required to show reliable change. Baseline to follow-up reliable changes were examined nonparametrically, combining the three clinical scales (as numbers would otherwise be too small to analyze); missing data was counted as unimproved, providing a full intent-to-treat analysis. Six percent of the participants improved significantly and 2% deteriorated significantly in the wait-list condition, as compared to 15% significantly improved and 0% deteriorated in the workbook condition. Analyzing the net gain showed a small but significant difference in reliable change on the DASS scales, χ2 (1) = 3.27, p = .035, one-tailed, effect size = .36. After the workbook was

act bibliotherapy for japanese international students distributed in the wait-list condition the response was no longer different from the original workbook arm, χ2 (1) = .06, p = .80. Process Results: Psychological Flexibility An HLM model assuming a random and uncorrelated intercept and slope showed no effect for treatment condition and no effect for time but there was a significant and large interaction of condition and time, F(1, 77.67) = 4.28, p = .042. Analysis of the significance of slopes within each condition showed no improvement in the wait-list condition: slope estimate = – .25, SE = .79, t(77.02) = – .31, p = .75, effect size = .10; but there was a significant and large improvement for participants who were given the workbook: slope estimate = 2.13, SE = .84, t(78.25) = 2.54, p = .013, effect size = .87. Baseline to final follow-up analyses combining both arms showed no effect for when the book was received, p N .15, but a large difference between the baseline and final follow-up periods, F(1, 52.21) = 12.26, p = .001, effect size = .97. ACT Knowledge and Outcomes The percentage correct on quizzes taken during the book-reading period was correlated with changes in the primary outcome (general mental health) from baseline to the final follow-up, combining both arms of the study. Understanding of book content during the reading period summarizing across all six quizzes predicted changes in general mental health from baseline to the final follow-up, r(55) = .29, p = .016, one-tailed, suggesting that reading and understanding the book was helpful. It could also be evidence that brighter participants changed more even without reading the book, but that seems unlikely given that the percentage of correct items also correlated with selfreported frequency and completeness of book use, r(55) = .41, p = .001, one-tailed.

mediation and moderation analyses Mediation When process measures are taken after outcomes have already changed (as in this case) it is not possible to determine whether change processes are functionally important. The zero-order correlation at pretest between the AAQ and GHQ was itself significant, r(70) = .40, p = 001, further complicating the comparison. Nevertheless, statistical mediation can provide incremental evidence for model failure (as contrasted with model support) because with all the statistical advantages this violation provides if it is not is seen fairly consistently across studies in these circumstances, either

331

the process measures are inadequate or the model is likely to be incorrect. A nonparametric bootstrapping cross products of the coefficients approach (Preacher & Hayes, 2004, 2008) with 2,999 bootstrapped samples, showed that post-AAQ changes mediated follow-up GHQ changes in those distressed on the primary outcome variable (bootstrapped point estimate = –.98; 90% CI lower and upper = – 3.54, – .03) but not vice versa (bootstrapped point estimate = 2.14; 90% CI lower and upper = –.19, 7.07). Moderation Baseline to follow-up change scores in the GHQ were correlated with all baseline measures to examine possible moderators of change. Improvement was significantly greater in those who at baseline were less psychologically flexible as measured by the AAQ, r(55) = .31, p = .02. GHQ change scores did not correlate significantly with any of the prelevels of the DASS scales, suggesting that the pattern of possible moderation by the primary targeted process was probably not because this process measure was merely a marker of clinical distress per se.

Discussion Although college is often thought of as a carefree time, the evidence suggests otherwise. College students in general experience remarkably high rates of distress. The most recent report of the American College Health Association (2008) found that about 13% of college students reported having symptoms of anxiety and more than 18% reported depression symptoms; 43% said they felt so depressed at some point in the academic year that it was difficult to function; 10% had seriously considered suicide. The distress experienced by international students is even higher (Matsumoto et al., 2001). In addition to the difficulties of higher education, there can be large language barriers, notable cultural differences, and reduced social support felt due to diminished family contact. The baseline scores of the present study showed the problem. Sixty percent of the participants were above standard clinical cutoffs for the GHQ and a startling 79% (55 of 70) were above moderate clinical cutoffs on one or more specific scales of the DASS. Since a majority of all Japanese international students on campus participated, even if we assume that none of the nonparticipants were distressed, the levels of behavioral health issues in this population as a whole were extraordinarily high. Yet it is known that the mental health needs of this group are generally not met (Sue et al., 1991).

332

muto et al.

Bibliotherapy is an obvious avenue to explore but we could find no studies that had used bibliotherapy with this group, and few intervention interventions studies of any kind. The present study shows that distributing a Japanese translation (Hayes & Smith, 2008) of Get Out of Your Mind and Into Your Life and arranging conditions for it to be read was helpful in ameliorating and preventing mental health problems in international students. Based on the HLM slopes in an intent-to-treat sample, the workbook produced a difference between conditions at the 2-month follow-up of 6.4 points on the GHQ among distressed individuals, which is a substantial ameliorative effect (effect size = 1.41). For those experiencing moderately or severely elevated levels of depression or stress, or severely elevated levels of anxiety, the book produced significant gains as compared to a wait-list. It would be a mistake to substitute a broad self-help book with minimal personnel involvement for needed treatment in a specific area, however, and these results do not suggest that colleges can use self-help materials in lieu of professional behavioral health care providers. Among the initially nondistressed group, a third became distressed in the waitlist condition, but none did in the workbook condition. Gains in primary outcomes were associated with greater ACT knowledge and with positive changes in psychological flexibility, which reduces (but does not eliminate) the concern that these benefits were merely due to the broad demand characteristics of study participation. In general, between-group pre to first follow-up effects were larger than pre to post effects, but this may be due in part to the time of academic year. Post measures were taken as the spring semester ended and there may have been a relief effect. On some measures students in the wait-list condition showed improvement at this time (particularly in stress, anxiety, and depression). By follow-up (as the new semester got underway) that effect had passed and wait-list participants had returned to baseline levels whereas workbook participants maintained gains. Baseline to final follow-up effects were virtually identical in the two arms on all measures suggesting that once the book was distributed to the wait-list participants they caught up with the other participants. It is known that self-help materials can provide affordable psychological assistance for specific conditions (Johnson & Sandhu, 2006) with effect sizes that can approach that of professional intervention (Den Boer et al., 2004), provided readers are actively involved in behavior change (Redding, Herbert, Forman, & Gaudiano, 2008, p. 538). The present study appears to be one of the

first to show a broad ameliorative and preventative impact in a population-based sample rather than one defined by specific behavioral or psychological difficulties. This is meaningful for three reasons. Popular psychological trade books are distributed widely, without regard to identified need. A book that is helpful for the severely distressed could be harmful for others. That did not appear to be the case, which is reassuring given the realities of self-help book distribution in the marketplace. Second, these findings suggest that it may be practical for colleges to distribute and structure the reading of inexpensive behavioral and cognitive self-help materials as part of student orientation, or in first-year experience classes and the like. Broadly focused populationbased interventions may be easier to mount by colleges than multiple targeted programs, and thus transdiagnostic models targeting common core processes of psychopathology or psychological growth seem especially useful in that context. Finally, it is worth noting that such a populationbased approach could help avoid the stigma for seeking needed services because all students would receive the intervention, which may be especially important for subgroups or individuals for whom mental health stigma is particularly strong (Ting & Hwang, 2009). In addition to the practical benefits of bibliotherapy it is worth noting the potential scientific benefits. Difficult methodological issues such as adherence to manuals, therapist allegiance, therapist training, therapist personality factors, the strength of the therapeutic alliance, demand characteristics, and the like are reduced or eliminated in bibliotherapy studies. Bibliotherapy may be an ideal preparation to examine important theoretical issues such as differences in processes of change or in component analysis, and indeed studies have begun to appear on such thorny issues as whether cognitive challenging is helpful, inert, or hurtful to the impact of traditional cognitive behavioral therapy (CBT) packages (Haeffel, 2010). There are weaknesses in the current study that should be noted, however. At the time this study was conducted, Japan was the single most represented international student group at the university, comprising 22% of all the international students enrolled. Participants were relatively active, with nearly three quarters of the enrollees completing quizzes over book content. By enrolling a majority of Japanese students, an ACT model may have become relatively well known to the Japanese student population. Individual reading of the book without that social context could have had different effects.

act bibliotherapy for japanese international students Perhaps the biggest scientific weakness of the study is that a wait-list control was used, not an alternative volume, so general demand characteristics might partially account for the results. This is a general problem with all wait-list control conditions. The vast majority of bibliotherapy research is based on such designs (Den Boer et al., 2004), perhaps because the lack of personal therapist contact reduces concerns over nonspecific factors. It does not eliminate them, however, and taking advantage of the full scientific possibilities requires that bibliotherapy research begins using more complex comparison conditions. Nevertheless, there is a role for wait-list designs (e.g., Rounsaville, Carroll, & Onken, 2001). Further, the literature shows that merely presenting any book is no guarantee of positive benefits (Rosen, Glasgow, & Moore, 2003). The development strategy being followed by many ACT researchers has been termed a “contextual behavioral science” approach (Hayes et al., in press; Vilardaga et al., 2010). In this strategy, attention is given early on to issues of processes changes, mediation, moderation, and disseminability, in addition to such issues as the development of basic principles, and research on components. For example, in the present study the ability to move psychological flexibility in this population and to see whether it related to mental health improvement was a key focus. Japanese culture contains a considerable emphasis on acceptance and mindfulness, and it was by no means certain that this volume would move these processes, particularly in this cultural subgroup. Since there is no currently supported general self-help text for this population, and no known harm to participants from waiting for the treatment, the use of wait-list control also fits the ethical guidelines for the use of wait-lists (World Medical Association, 2000). Some of these process issues are better examined initially in wait-list studies. For example, a control such as reflective listening might not just provide broad social support, but might also model or even train acceptance skills. Without having a base of wait-list studies such effects would be unlikely to be detected. Finally, in practical environments it is often more important to see whether an effect exists before determining whether it is due to specific or nonspecific factors. Another weakness is that the assessment devices used were few and were all self-report. Given the broad focus of the study and limited incentive to participate, the assessment burden had to be kept very low. As a result, no data were taken on treatment seeking, use of medications, diagnoses as such, academic performance, whether participants

333

practiced meditation, and a variety of other potentially important variables. This limits what can be said when going beyond the measures used. Seasonal changes linked to the academic calendar also complicated the evaluation, and the follow-up period was not lengthy. Despite the generally positive results in the present study it is important not yet to assume that ACT self-help books will be helpful in general. Scholars in the area of bibliotherapy have long warned that self-help books should not merely be based on evidence-based procedures such as ACT but also should be evaluated themselves, because it is known that evidence-based psychosocial procedures can become inert or even harmful when removed from the context of psychotherapy and direct interaction with professionals (Rosen, 1993). Get Out of Your Mind and Into Your Life (Hayes & Smith, 2005) was the first and is still the most widely sold general ACT self-help book, and it has provided a basic template for many of the ACT selfhelp volumes that have followed. The present study and others (Johnston, Foster, Shennan, Starkey, & Johnson, 2010; Lazzarone et al., 2007) provide some reassurance that these volumes are likely to be helpful to the members of the public who are reading them, but more research is needed on ACT bibliotherapy methods before that can be fully assumed. References American College Health Association. (2008). National College Health Assessment, 2008. Retrieved May 16, 2009, from http://www.acha-ncha.org/index.html Baer, R. A., Smith, G. T., & Allen, K. B. (2004). Assessment of mindfulness by self-report: The Kentucky Inventory of Mindfulness Skills. Assessment, 11, 191–206. Banks, M. H. (1983). Validation of the General Health Questionnaire in a young community sample. Psychological Medicine, 13, 349–353. Blackledge, J. T., & Ciarrochi, J. (2006, July). Initial validation of the Personal Values Questionnaire. Paper presented at the Second ACBS World Conference. London, UK. Bond, F. W., Hayes, S. C., Baer, R. A., Carpenter, K. M., Orcutt, H. K., Waltz, T., & Zettle, R. D. (under submission). Preliminary psychometric properties of the Acceptance and Action Questionnaire–II: A revised measure of psychological flexibility and acceptance. Brinson, J. A., & Kottler, J. A. (1995). Minorities' underutilization of counseling centers' mental health services: A case for outreach and consultation. Journal of Mental Health Counseling, 17, 371–385. Chirkov, V., Vansteenkiste, M., Tao, R., & Lynch, M. (2007). The role of self-determined motivation and goals for study abroad in the adaptation of international students. International Journal of Intercultural Relations, 31, 199–222. Cohen, J. (1988). Statistical power for the behavioral sciences, (2nd ed.). Hillsdale, NJ: Erlbaum. Crawford, J. R., & Henry, J. D. (2003). The Depression Anxiety Stress Scales (DASS): Normative data and latent

334

muto et al.

structure in a large non-clinical sample. British Journal of Clinical Psychology, 42(Pt. 2), 111–131. Den Boer, P., Wiersma, D., & Van Den Bosch, R. J. (2004). Why is self-help neglected in the treatment of emotional disorders? A meta-analysis. Psychological Medicine, 34(6), 959–971. Donath, S. (2001). The validity of the 12-item General Health Questionnaire in Australia: A comparison between three scoring methods. Australian and New Zealand Journal of Psychiatry, 35, 231–235. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G⁎Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. Feingold, A. (2009). Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychological Methods, 14, 43–53. George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update, (4th ed.). Boston: Allyn & Bacon. Goldberg, D. P., Gater, R., Sartorius, N., Ustun, T., Picinelli, M., Gureje, O., & Rutter, C. (1997). The validity of two versions of the GHQ in the WHO study of mental illness in general health care. Psychological Medicine, 27, 191–197. Goldberg, D. P., & Williams, P. (1988). A user's guide to the GHQ. Windsor, UK: NFER-Nelson. Gregory, R. J., Schwer-Canning, S., Lee, T. W., & Wise, J. C. (2004). Cognitive bibliotherapy for depression: A metaanalysis. Professional Psychology Research and Practice, 35, 275–280. Haeffel, G. (2010). When self-help is no help: Traditional cognitive skills training does not prevent depressive symptoms in people who ruminate. Behaviour Research and Therapy, 48, 152–157. Hayes, S. C. (2002). Buddhism and acceptance and commitment therapy. Cognitive and Behavioral Practice, 9, 58–66. Hayes, S. C., Levin, M., Plumb, J., Boulanger, J., & Pistorello, J. (in press). Acceptance and commitment therapy and contextual behavioral science: Examining the progress of a distinctive model of behavioral and cognitive therapy. Behavior Therapy. Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J. (2006). Acceptance and commitment therapy: Model, processes and outcome. Behaviour Research and Therapy, 44, 1–25. Hayes, S. C., & Smith, S. (2005). Get out of your mind and into your life: The new acceptance and commitment therapy. Oakland, CA: New Harbinger. Hayes, S. C., & Smith, S. (2008). Anata no Jinsei wo Hajimeru tameno Workbook: Kokoro tono Atarashii Tsukiai kata, acceptance and commitment (T. Muto, H. Harai, M. Yoshioka, & M. Okajima, Trans.). Tokyo: Brain Shuppan. Hayes, S. C., & Strosahl, K. D. (2004). A practical guide to acceptance and commitment therapy. New York: Springer-Verlag. Hayes, S. C., Strosahl, K., & Wilson, K. G. (1999). Acceptance and commitment therapy: An experiential approach to behavior change. New York: Guilford Press. Hedman, E., Ljótsson, B., Rück, C., Furmark, T., Carlbring, P., Lindefors, N., & Andersson, G. (2010). Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation. Computers in Human Behavior, 26(4), 736–740. Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336–352.

Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. Johnson, L. R., & Sandhu, D. S. (2006). Isolation, adjustment, and acculturation issues of international students: Intervention strategies for counselors. In H. D. Singaravelu, & M. Pope (Eds.), A handbook for counseling international students in the United States (pp. 13–35). Alexandria, VA: American Counseling Association. Johnston, M., Foster, M., Shennan, J., Starkey, N. J., & Johnson, A. (2010). The effectiveness of an acceptance and commitment therapy self-help intervention for chronic pain. Clinical Journal of Pain, 26, 393–402. Kadison, R., & DiGeronimo, T. F. (2004). College of the overwhelmed: The campus mental health crisis and what to do about it. San Francisco: Jossey-Bass. Lazzarone, T., Hayes, S. C., Luoma, J., Kohlenberg, B., Pistorello, J., et al. (2007, November). The effectiveness of an ACT self-help manual: “Get Out of Your Mind and Into Your Life.” Paper presented at the Annual Meeting of the Association for Behavioral and Cognitive Therapies, Philadelphia, PA. Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the Depression Anxiety Stress Scales, (2nd ed.). Sydney: Psychology Foundation of Australia. Lundgren, T., Dahl, J., Yardi, N., & Melin, L. (2008). Acceptance and commitment therapy and yoga for drugrefractory epilepsy: A randomized controlled trial. Epilepsy & Behavior, 13, 102–108. Masuda, A., Hayes, S. C., Twohig, M. P., Lillis, J., Fletcher, L. B., & Gloster, A. T. (2009). Comparing Japanese international students and U.S. college students' mental health related stigmatizing attitudes. Journal of Multicultural Counseling and Development, 37, 178–189. Masuda, A., Muto, T., Hayes, S. C., & Lillis, J. (2008). Acceptance and commitment therapy: Application to a Japanese client. Japanese Journal of Behavior Therapy, 34, 137–148. Matsumoto, D., LeRoux, J. A., Bernhard, R., & Gray, H. (2004). Unraveling the psychological correlates of intercultural adjustment potential. International Journal of Intercultural Relations, 28, 281–309. Matsumoto, D., LeRoux, J. A., Ratzlaff, C., Tatani, H., Uchida, H., Kim, C., & Araki, S. (2001). Development and validation of a measure of intercultural adjustment potential in Japanese sojourners: The Intercultural Adjustment Potential Scale (ICAPS). International Journal of Intercultural Relations, 25, 483–510. Öst, L. (2008). Efficacy of the third wave of behavioral therapies: A systematic review and meta-analysis. Behaviour Research and Therapy, 46(3), 296–321. Pederson, P. (1995). The five stages of culture shock: Critical incidents around the world. Westport, CT: Greenwood Press. Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, 36, 717–731. Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879–891. Raudenbush, S., & Bryk, A. (2001). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage. Raudenbush, S. W., & Liu, X. F. (2001). Effects of study duration, frequency of observation, and sample size on

act bibliotherapy for japanese international students power in studies of group differences in polynomial change. Psychological Methods, 6, 387–401. Read, J., Farrow, S., Jaanimägi, U., & Ouimette, P. (2009). Assessing trauma and traumatic stress via the Internet: Measurement equivalence and participant reactions. Traumatology, 15, 94–102. Redding, R. E., Herbert, J. D., Forman, E. M., & Gaudiano, B. A. (2008). Popular self-help books for anxiety, depression, and trauma: How scientifically grounded and useful are they? Professional Psychology: Research and Practice, 39, 537–545. Register, A. C., Beckham, J. C., May, J. G., & Gustafson, D. J. (1991). Stress inoculation bibliotherapy in the treatment of test anxiety. Journal of Counseling Psychology, 38, 115–119. Rosen, G. M. (1993). Self-help or hype? Comments on psychology's failure to advance self-care. Professional Psychology: Research and Practice, 24, 340–345. Rosen, G. M., Glasgow, R., & Moore, T. E. (2003). Self-help or hype? The science and business of giving psychology away. In S. Lilienfeld (Ed.), Science and pseudoscience in contemporary clinical psychology New York: Guilford Press. Rosenthal, R., & Rosnow, R. L. (1991). Essentials of behavioral research: Methods and data analysis, (2nd ed.). New York: McGraw-Hill. Rounsaville, B., Carroll, K., & Onken, L. (2001). A stage model of behavioral therapies research: Getting started and moving on from stage I. Clinical Psychology: Science and Practice, 8, 133–142. Shin, J. K. (2002). Help-seeking behaviors by Korean immigrants for depression. Issues in Mental Health Nursing, 23, 461–476. Singaravelu, H. D., & Pope, M. (2006). A handbook for counseling international students in the United States. Alexandria, VA: American Counseling Association. Sue, S., Fujino, D. C., Hu, L., Takeuchi, D. T., & Zane, N. (1991). Community mental health services for ethnic minority groups: A test of the cultural responsiveness hypothesis. Journal of Consulting and Clinical Psychology, 59, 533–540. Tait, R. J., Hulse, G. K., & Robertson, S. I. (2002). A review of the validity of the General Health Questionnaire in adolescent populations. Australian and New Zealand Journal of Psychiatry, 36, 550–557.

335

Takahashi, M., Muto, T., Tada, M., & Sugiyama, M. (2002). Acceptance rationale and increasing pain tolerance: Acceptance-based and FEAR-based practice. Japanese Journal of Behavior Therapy, 28, 35–45. Tannenbaum, R. J., & Wylie, E. C. (2008). Linking Englishlanguage test scores onto the Common European Framework of Reference: An application of standard-setting methodology (Rep. No. TOEFLiBT-06). Princeton, NJ: Educational Testing Service. Ting, J. Y., & Hwang, W. (2009). Cultural influences on helpseeking attitudes in Asian American students. American Journal of Orthopsychiatry, 79, 125–132. Uba, L. (1994). Asian Americans: Personality patterns, identity, and mental health. New York: Guilford Press. Vallejo, M., Jordan, C., Diaz, M., Comeche, M., & Ortega, J. (2006). Psychological assessment via the internet: A reliability and validity study of online (vs paper-and-pencil) versions of the General Health Questionnaire-28 and the Symptoms Check-List-90-Revised (SCL-90-R). Journal of Medical Internet Research, 9, 1–10. Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. New York: Springer-Verlag. Wackerly, D., Mendenhall, W., & Scheaffer, R. L. (2008). Mathematical statistics with applications, (7th ed.). Pacific Grove, CA: Duxbury Press. Wenfeng, G., Meiling, G., & Lianyun, W. (2004). Comparative analysis of bibliotherapy for mental problems in college students. Chinese Mental Health Journal, 18, 418–420. World Medical Association. (2000). Declaration of Helsinki: Ethical principles for medical research involving human subjects. Retrieved from http://www.wma.net/e/policy/b3. htm. Zlomke, K. (2009). Psychometric properties of internet administered versions of Penn State Worry Questionnaire (PSWQ) and Depression, Anxiety, and Stress Scale (DASS). Computers in Human Behavior, 25, 841–843. R E C E I V E D : May 25, 2010 A C C E P T E D : August 12, 2010 Available online 17 February 2011