A randomised placebo-controlled trial of a self-help Internet-based intervention for test anxiety

ARTICLE IN PRESS

Behaviour Research and Therapy 45 (2007) 483–496 www.elsevier.com/locate/brat

A randomised placebo-controlled trial of a self-help Internet-based intervention for test anxiety Gili Orbacha, Stan Lindsayb,, Susan Greyc a

Child Psychiatry, St. Georges Team, Lanesborough Wing, St. Georges Hospital, Blackshaw Road, London SW17 0QT, UK b Psychology Department, Institute of Psychiatry, P.O. 78, De Crespigny Park, London SE5 8AF, UK c Maudsley Psychology Centre, The Maudsley Hospital, Denmark Hill, London SE5, UK Received 17 August 2005; received in revised form 14 April 2006; accepted 14 April 2006

Abstract Test anxiety is widespread and associated with poor performance in academic examinations. The Internet, not wellproven for the treatment of anxiety, should be able to deliver highly accessible Cognitive Behavior Therapy (CBT). This study sought to test the hypothesis that CBT, available on the Internet, could reduce test anxiety. Ninety university students were randomly allocated to CBT or a control program, both on the Internet. Before and after treatment, the participants completed the Test Anxiety Inventory (TAI), an Anxiety Hierarchy Questionnaire (AHQ), the Exam ProblemSolving Inventory (EPSI), the General Self-Efficacy Scale (GSES) and the Heim reasoning tests (AH) as a measure of test performance. Of the CBT and control groups 28% and 35%, respectively, withdrew. According to the TAI, 53% of the CBT group showed a reliable and clinically significant improvement with treatment but only 29% of the control group exhibited such a change. On the AHQ, 67% of the CBT group and 36% of the control group showed a clinically significant improvement, more than two standard deviations above the mean of the baseline, a change in favour of CBT. Both groups improved on the GSES, in state anxiety during exams retrospectively assessed, and on the AH tests. The improvement on the AH tests was probably a practice effect and not a reflection of a change in capacity for academic testing. This study thus supports use of CBT on the Internet for the treatment of test anxiety. r 2006 Elsevier Ltd. All rights reserved. Keywords: Test anxiety; Randomised controlled trial; Internet; CBT; Placebo

Introduction Test anxiety is experienced throughout the world and in all socio-economic groups; it is more common in females and is the most common and most persistent fear in young people. It affects between 25% and 30% of students including those with learning difficulties (McDonald, 2001; Wachelka & Katz, 1999; Zeidner, 1998). Avoidance by students of exams is rarely practicable (Zeidner, 1998). Those with high test anxiety perform more poorly than others (Hembree, 1988) but it is not clear if this is the cause or effect of test anxiety (Cassady & Johnson, 2002; Hembree, 1988; Seipp, 1991). If test anxiety does contribute to poor performance, because Corresponding author. Fax: +44 1252 812752.

E-mail address: [email protected] (S. Lindsay). 0005-7967/$ - see front matter r 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.brat.2006.04.002

ARTICLE IN PRESS 484

G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

many organisations employ tests, the outcome of testing could affect the life-long destiny of candidates. Nevertheless, whatever the causal relationship is, many students believe that anxiety does impair test-taking (Hong & Karstensson, 2002). Test anxiety has been regarded ubiquitously as a continuous variable rather than a discrete diagnostic category, present or absent (Zeidner, 1998). ‘‘Test anxiety’’ therefore applies to extreme anxiety on that dimension. Reviewed by Zeidner (1998), test anxiety was first seen as excessive arousal which interfered with the performance of tests although moderate anxiety could enhance effectiveness. The remedy for excessive arousal has been relaxation training. Test anxiety was then identified as a situation-specific trait in students evident in a variety of situations in which they were being evaluated. The corresponding treatment has been systematic desensitisation. Morris and Liebert (1969) distinguished worry, a cognitive component, from emotionality, evident as autonomic arousal, and concluded that only worry interfered with task performance. It does so by diverting attention to self-deprecating thoughts which provoke autonomic arousal. Corresponding treatment has consisted of attention training. Highly anxious students are also deficient in skills for studying and taking tests according to Kirkland and Hollandsworth (1979). Those students who believe this and who are aware of impaired performance become anxious accordingly. Self-efficacy theory (Jones & Petruzzi, 1995) has a similar perspective. Cognitive approaches have emphasised dysfunctional beliefs in students such as the need, suggested by rational emotive therapy, to strive for perfection which they find unattainable (Zeidner, 1998). Alternatively they believe that they compare unfavourably with their peers under whose scrutiny they think they fall (Beck, Emery, & Greenberg, 1996). Treatments corresponding to all these perspectives have been successful in reducing test anxiety but cognitive therapy plus skills training or emotion-focussed components have worked best (Ergene, 2003; Vagg & Spielberger, 1995). The simplest hypothesis would therefore suggest that the most effective treatment should include all these approaches as components in ‘‘multimodal treatment’’ (Zeidner, 1998). However, this would require a unifying theory to support the hypothesis that several such components would have an additive effect (Zeidner, 1998). There has been one attempt at such a theory, that by Spielberger and Vagg (1995). They proposed that differences in trait anxiety interact with situational factors such as the student’s perception of the test’s difficulty, skills in studying and taking tests and the importance of the exam to determine the extent to which an exam is seen as threatening. This will influence arousal, contribute to difficulties in information processing, cause task irrelevant behaviour, produce a decrement in performance and corresponding self-derogatory cognitions. However, a test of this theory would require structural path analysis (Zeidner, 1998) which has not been done and so corresponding treatments have not been tested. Moreover, there appear to have been no multimodal treatments, of more than two modules, which have included cognitive therapy. Therefore, the present study adopted the simplest and widespread assumption (Zeidner, 1998) that several treatments, shown separately to have treated test anxiety successfully (Ergene, 2003), would be more effective if all, rather than one or two, were included in a multimodal package of Cognitive Behaviour Therapy (CBT). So widespread is test anxiety, there is a considerable need for effective short treatments. However, the opportunity for treatment is severely limited by cost, the scarcity of therapists and inaccessibility for people who are in full-time learning, long hours of work, night-shift working or in distant communities dependent on tele-learning. Computer-based treatment may be one solution. CBT is well suited to this because of its welldefined procedures (Bloom, 1992). Computer-administered CBT on CDs has been tested for several anxiety disorders (Kaltenhaler, Parry, & Beverley, 2004) but the National Institute for Health and Clinical Excellence (NICE, 2002) concluded that the evidence (Kaltenhaler et al., 2002), although promising, was not strong enough to recommend this for clinical practice. However, NICE has this under review and they would consider the latest study by Marks, Kenwright, McDonough, Whittaker, and Mataix-Cols (2004). They have shown that self-exposure therapy presented mainly by a computer was no less successful in treating phobias and panic disorders than a similar programme conducted by a therapist alone. However, 43% of computeraided clients dropped out of treatment. That study is typical in that very few studies have included a credible procedure delivered by computer with which to compare the computerised treatment. They seek rather to show that the latter is no less effective than clinician-delivered therapy. The Internet is another means of delivering programs via a computer. The studies of computeraided treatment noted above would encourage research on the use of the Internet to treat anxiety

ARTICLE IN PRESS G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

485

disorders. Furthermore, that medium could provide advantages over CDs. Programs could be more accessible, more readily modified by authors. Access by users could be monitored; they could try before buying. Lange, Schrieken, van de Ven, Bredeweg, and Emmelkamp (2000) and Richards and Alvarenga (2002), with therapists aided by the Internet, found improvements in measures of anxiety but neither study had a control group. Klein and Richards (2001) provided 11 participants suffering from panic disorder (PD) with information which was delivered by the Internet and introduced by a therapist. All of several measures of anxiety and panic declined after a week’s treatment unlike those in a similar group who only monitored their symptoms. Carlbring, Westling, Ljungstrand, Ekselius, and Andersson (2001) compared a CBT Internet program of six modules, based on published self-help manuals, with a waiting list control. Twenty-six of 41 PD participants completed the trial. On all but one of eight self-report inventories and five of seven variables in a diary, they improved to a greater extent than the controls. Most of these improvements were clinically significant but no effect size or comparison with face-to-face therapy was reported. This program was repeated by Carlbring, Ekselius, and Andersson (2003) and compared with applied relaxation instructions both delivered by the Internet in a sample of 22 participants divided into two groups. Both improved on most of the self-report measures and diary records but there was no difference between treatments. The moderate effect sizes were similar in both groups. Kenardy, Mc-Cafferty, and Rose (2003) found that an Internet program based on published self-help manuals for panic reduced scores on self-report of anxiety in 43 participants selected to be high in anxiety sensitivity. Although this improvement was greater than in a waiting list control group, there was no significant effect on anxiety sensitivity, contrary to prediction. Schneider, Mataix-Cols, Marks, and Bachofen (2005) have recently reported that Internet-delivered CBT plus telephone support was effective in treating phobias with or without panic, whether or not the treatment included exposure. Most recently, Carlbring et al. (2005) compared Internet-delivered CBT with treatment provided by therapists in vivo to 49 participants with PD with or without agoraphobia. The only contact with a therapist in the Internet group was provided by e-mail. No behavioural data were reported but responses to questionnaires showed significant improvements with large effect sizes in both groups. The authors claim that the Internet group improved as much as the in vivo therapy participants but they provide few results to support this. Therefore, although anxiety has been reduced in several studies using the Internet in comparison with other approaches, none has compared treatment with a placebo program delivered by the Internet. The Internet can generate influential expectations which might spuriously inflate the effect of programs (Richards, Klein, & Calbring, 2003). So much trust in web sites can be placed by users that ethical debate has ensued (Briggs, Burford, De Angeli, & Lynch, 2002; Fisher & Fried, 2003). Placebo programs delivered on the Internet (Zeidner, 1998) should be included, therefore, to evaluate the efficacy of CBT in this medium. No study has done this. Moreover, as noted above, there has been insufficient use of placebo-controlled trials of computerised treatment to pre-empt such a comparison for Internet-delivered treatment. Computerised packages for test anxiety (Buglione, Devito, & Mulloy, 1990) have consisted mainly of systematic desensitisation and, although there are many relevant Internet sites (e.g. University of South Australia, 2001), these provide information, not treatment. Therefore, Internet programs cannot yet be recommended for test anxiety. The present study therefore compared two programs on the Internet: CBT and a placebo package presenting similar procedures. We were to measure test anxiety by the Test Anxiety Inventory (TAI) (Spielberger, 1980) and our own Anxiety Hierarchy Questionnaire (AHQ) before and after treatment in groups of participants allocated to each program. We expected that anxiety on both measures would be less after each intervention than before but the decline in anxiety would be greater in the group who had received CBT than in the controls. We expected that there would be corresponding changes in the participants’ confidence about succeeding in exams and about their ability to overcome exam-related problems. Test anxiety may impair performance of intellectual tasks. Therefore, we expected that a test of cognitive functioning, administered before and after the interventions, would show a greater improvement in the CBT group than in the controls. Previous research (Zeidner, 1998) using the TAI (Spielberger, 1980), the most widely used measure, suggested that, with a power of 0.80 and a ¼ 0:05, we would require 25 participants in each group to reject the null hypothesis.

ARTICLE IN PRESS 486

G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

Method Design Participants were allocated at random to the treatment or control group with repeated measures, before and after intervention (Fig. 1). Participants We circulated an e-mail to all students in Kings College London, London University, a long established college offering courses for masters, doctoral, medical and bachelor degrees. The e-mail invited students who experienced anxiety when preparing for, or taking, exams to take part in our study which would show them, by means of computer programs, different ways of managing their anxiety and thus improving their performance. The e-mail noted that they would need to have access to a computer connected to the Internet which could thus provide entry to our programs with restricted access to ensure confidentiality. Interested students were invited to contact the investigator (GO) by phone or e-mail for further details. Among the 90 students who thus volunteered some would be excluded if they were already receiving treatment for test anxiety. None had to be eliminated at this stage. As in previous research on test anxiety (Zeidner, 1998) they were not selected by further diagnostic criteria but were to be characterised by their scores on measures of test anxiety. To enrol and introduce the students to the programs, the investigator arranged to meet each one if they had sole access to a room with a computer on their campus. If they did not have such a facility ðn ¼ 35Þ, the

Initial assessment and enrolment to the programs: 1. 2. 3. 4. 5. 6.

Information sheet and consent form AH2 test for face to face enrolment group only Administration of Questionnaires, TAI, HQ, EPSI, GSES Completion of the demographic questionnaire Random allocation to CBT program or control program Introduction and registration to program for each group

Home-based treatment on allocated program: approximately 30 minutes a week for up to 6 weeks. On Excel spreadsheets participants report time on program and use of modules

Post-intervention assessment: 1. AH3 test for face to face enrolment group only 2. Administration of Questionnaires, TAI, HQ, EPSI, GSES 3. Qualitative debriefing including usefulness ratings of each program 4. Enrolment to second program

Post-exam follow-up fourmonths later by telephone: 1. STAI-short form to assess, retrospectively, anxiety in exams before and after our interventions 2. Brief interview

Fig. 1. Flow diagram of the procedure.

ARTICLE IN PRESS G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

487

investigator arranged to speak to them by telephone. Having elicited details of the participants’ history of test anxiety (Table 1) and having determined if English was their first or second language and that they would be able to read the programs, the investigator then described the experimental procedure, including the use of a placebo program. This was summarised on an information sheet for each student. All consented to participate and so the investigator allocated 47 to the CBT group and 43 to the control group by tossing a coin. During the trial, 13 (28%) from the treatment group and 15 (35%) controls withdrew, a non-significant difference ðw2 ¼ 0:55; p40:05Þ. Four in the CBT group were not assessed because of recording difficulties so that 30 in that group and 28 in the control group provided data (Table 1) according to the experimental protocol, not intention to treat.

Material The following were presented on paper except for participants contacted by phone who completed the questionnaires on their computers. TAI: The TAI (internal consistency a ¼ 0:92; Spielberger, 1980) is a 20-item self-report measure, providing multiple choice answers on a Likert scale from ‘‘almost never’’ scoring 1 to ‘‘almost always’’ scoring 4. It assesses subjects’ habitual anxiety and discomfort in test-taking. AHQ: This is a 16-item measure created for the present study (Orbach, 2002) to assess, more directly than the TAI, anxiety about a forthcoming exam. Each item outlines an exam-related situation, for which participants specify how anxious they feel on a scale of 0–100, with 100 indicating extreme anxiety. We divided the total scores by 100 to simplify computation. The items are ordered in proximity to an exam, from one week before to one immediately afterwards. For example, ‘‘You read the first question in the exam and do not know the answer’’. This questionnaire had high internal consistency ða ¼ 0:87Þ for our 58 participants. Exam Problem-Solving Inventory (EPSI). The Personal Problem-Solving Inventory (PPSI; Heppner & Petersen, 1985) has been designed to measure the dimensions of respondents’ beliefs about their capacity for solving problems in everyday life. Three sub-scales based on students’ answers have been identified by factor analysis, viz. problem-solving confidence, approach-avoidance style and personal control. Correlation coefficients, ranging from 0.24 to 0.44 ðpo0:001Þ between scores on the separate scales and scores on similar questionnaires, show moderate concurrent validity. Test–retest reliability coefficients for the scales of the PPSI have ranged from 0.83 to 0.88. The PPSI is not significantly correlated with tests of intelligence.

Table 1 Characteristics of the study participants and dropouts: means and frequencies, 95% confidence intervals and standard deviations in brackets

Sex Mean age Mean years at university Mean years test anxiety Failed exams Ill due to exams Medication for exams Psychological help for exams *

Treatment group ðN ¼ 30Þ

Control group ðN ¼ 28Þ

Treatment cf. controls*

Dropouts ðN ¼ 28Þ

Dropouts cf. the rest*

Female ¼ 18 (60%) 24.72 (22.36–27.09, 6.89) 3.18 (2.16–4.11, 2.53) 7.59 (5.08–9.95, 6.67) Yes ¼ 15 (50%) Yes ¼ 14 (47%) Yes ¼ 7 (23%)

Female ¼ 24 (86%) 22.54 (20.07–24.97, 5.71) 3.02 (2.01–4.07, 2.81) 6.12 (3.62–8.67, 6.39) Yes ¼ 14 (50%) Yes ¼ 11 (39%) Yes ¼ 8 (29%)

w2 ð1Þ ¼ 4:79** Z ¼ 0:62

w2 ð1Þ ¼ 0:19 tð80Þ ¼ 1:15

tð54Þ ¼ 0:78 w2 ð1Þ ¼ 0:10 w2 ð1Þ ¼ 0:35

Female ¼ 19 (68%) 25.51 (23.98–27.02, 7.90) 3.47 (3.00–3.81, 2.12) 7.96 (6.38–9.42, 6.04) Yes ¼ 15 (58%) Yes ¼ 9 (35%) Yes ¼ 3 (12%)

Yes ¼ 4 (13%)

Yes ¼ 4 (14%)

w2 ð1Þ ¼ 0:01

Yes ¼ 3 (12%)

p40:05 unless otherwise shown. p ¼ 0:03.

**

tð54Þ ¼ 1:30 tð55Þ ¼ 0:14

tð81Þ ¼ 0:50 w2 ð1Þ ¼ 0:43 w2 ð1Þ ¼ 0:57 w2 ð1Þ ¼ 2:41 w2 ð1Þ ¼ 2:41 w2 ð1Þ ¼ 0:08

ARTICLE IN PRESS 488

G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

To measure confidence about solving problems in exams, we used the 16 items measuring problem-solving confidence and personal control and added ‘‘in my next exam’’ to each item. For example, ‘‘When my first efforts to solve a problem in my next exam will fail, I will become uneasy about my ability to handle the situation’’. Each item had six-point multiple choice answers from 0, ‘‘strongly disagree’’ to 5, ‘‘strongly agree.’’ The two sub-scales as one questionnaire had high internal consistency ða ¼ 0:88Þ for our participants. The General Self-Efficacy Scale (GSES)—exam related: People can believe that intentional acts will lead to certain outcomes: outcome expectancies. They also have beliefs about whether or not they would be able to carry out those behaviours: self-efficacy expectancy. Sherer et al. (1982) have developed a measure of the latter, the GSES. The GSES has 30 items each answered on a five-point Likert scale, 0–4 measuring strength of agreement, to encompass all behaviours according to items such as, ‘‘Failure just makes me try harder’’. Factor analysis has identified a measure of general self-efficacy, based on 17 of the items, with internal consistency a ¼ 0:86, accounting for most of the explained variance. The remaining items constitute a social self-efficacy sub-scale. Correlations with other questionnaires measuring ego strength and personal control have indicated low to moderate concurrent validity (Sherer et al., 1982). We required only the measure of general self-efficacy, to assess the respondents’ belief in their capacity to carry out exam-related behaviour. However, because of copyright restrictions, we had to give the whole questionnaire with the instructions, ‘‘wherever possible, answer the questions in relation to how you feel about studying, revising and taking exams’’. Performance measure, the Alice Heim (AH) tests of general reasoning: The AH printed tests of intelligence (Heim, Watts, & Simmonds, 1974) are designed to be administered to groups of normal and above average students. The items were devised according to their capacity to discriminate among students of different academic abilities (Klein, 2000). The AH2 and AH3 are highly intercorrelated parallel forms of the same test. Both are comprised of verbal, numerical and perceptual questions. As estimates of concurrent validity, the total scores on the tests have been significantly correlated with scores on tests of intelligence such as the Ravens Progressive Matrices and tests of attainment in mathematics ðr ¼ 0:87Þ, reading ðr ¼ 0:69Þ and other skills. Because the AH tests are measures of intelligence they are used to aid the selection of students and so could provoke test anxiety with the result that performance on them could be impaired. In using the AH tests, therefore, as measures of performance we excluded the verbal sub-test because eight participants did not have English as their first language. The test–retest reliability of the numerical and perceptual sub-tests has been estimated as 0.83 and 0.76, respectively. The perceptual-numerical test with a time limit of 15 min provided scores ranging from 0 to 80. State-Trait Anxiety Inventory—Short Form (STAI-SF). The STAI-SF (internal consistency a ¼ 0:85; Marteau & Bekker, 1992) is a questionnaire of six items each rated on a Likert scale of 1–4, measuring state anxiety. The computer programs These were designed for the Internet, with access by passwords for the start of each program. The programs had animation with music and voiceover recorded by a professional narrator. Both programs had modules which could be selected from a menu. Participants could use any module without further passwords and as often as they liked. Each module was independent of the others. The treatment program had six, the control program four. Both programs started with an Introduction describing test anxiety, its symptoms and prevalence plus a summary of the aims of the other modules. A module to promote relaxation followed for both programs. In that module, the treatment participants were told that negative thoughts and symptoms of anxiety could exacerbate anxiety which could then increase worries and symptoms and so on. Instructions, mainly in progressive muscular relaxation (Jacobson, 1938) followed. The third treatment module, ‘‘Apply Rational Thinking’’, introducing cognitive therapy, presented examples of how negative thoughts can promote anxiety and how these may be appraised by, for instance, evidence which challenges them. Typical thinking errors were provided (Sanders & Wills, 2005) for which the participants could find more detail and challenges by clicking on a balloon. They were then shown a diary of eight columns per page and encouraged to record, among other matters, distressing emotions as they had arisen, the corresponding circumstances, negative

ARTICLE IN PRESS G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

489

thoughts, evidence for and against the thoughts and other constructive challenges. They could then print out the diary for later use. The fourth and fifth treatment modules gave, respectively, advice on efficient studying as ‘‘Study Skills’’, and instructions to imagine a hierarchy of exam situations to which they practised relaxation, ‘‘Control your Stress’’. In the fourth, advice included information about the study environment, avoiding massed practice and procrastination. In the final treatment module, ‘‘Your Own Strategies’’, the participants were asked to produce simple sentences describing their current stresses, to key words from which the computer responded with suggestions from a large library developed by the software company for a previous project. Participants in the control program were told in their second module that relaxation could counter the tension produced by exam anxiety and so they were then provided with music in adagio tempo to which they were instructed, without further details, to relax while breathing slowly. Because the music and accompanying visual presentation of pleasant scenes occurred in repeated cycles, this module could be continued indefinitely until the participants pressed the stop button. In their third module the control participants were told that writing down their thoughts was a recognised way of unburdening one of worries and allowing one to clarify them and see them from fresh perspectives. They were then provided with a diary to record thoughts and relaxation experiences especially those which had promoted a reduction in anxiety on that day. The final module in the control program, the fourth, ‘‘Brain Puzzles’’, directed participants to a web site which presented ‘‘thousands’’ (http:www.brainbashers.com) of numerical and verbal puzzles which they were asked to solve. Participants were told that practice on such tasks can improve attention, important for performance in exams, and could also operate to distract them from worries about exams. They could practise this for as long as they liked. Thus the programs differed mainly in the extent to which instructions were made explicit about relaxation, focussing attention and rational thinking about anxiety. In the treatment program, ‘‘Apply Rational Thinking’’ and ‘‘Your Own Strategies’’ would have been of indefinite length, depending on the amount of material which the participants produced. Therefore, the control program had also to be indefinitely long. Hence the relaxation and ‘‘Brain Puzzles’’ modules were chosen because they were as long as the participants chose to use them.

Procedure For this (Fig. 1) we had to meet the requirements of the Research Ethical Committee of the Institute of Psychiatry. Pretreatment: In the presence of the investigator (GO) or over the phone, participants read the information sheet. This described the procedure and ‘‘a new intervention for exam anxiety using ways of challenging negative thinking about exams which would be compared with a computer-based procedure using writing, relaxation and mind-exercising strategies’’ that was expected to be ‘‘effective in treating exam anxiety but not as effective as the thought-challenging program’’. Each participant was allocated at random by the toss of a coin to one of the two programs but was allowed to use the alternative program after 6 weeks. Having signed the consent form, they spent around 18 min on the AH2, then completed the TAI, AHQ, exam-related GSES and the EPSI. Program allocation by the investigator’s (GO) toss of a coin followed. Then participants were registered for the program with usernames, passwords and the web site address. They completed demographic details on the computer and the experimenter demonstrated the program. They were told that they could contact the experimenter via e-mail or telephone in case of technical difficulties. For those enrolled by telephone the procedure was identical, except that the AH2 test was omitted, the information sheet was sent via e-mail, the consent forms were sent and returned by post and participants completed the questionnaires on the computer. Treatment: Participants were encouraged to use the program for a minimum of 30 min per week regularly, using all modules, but they could decide how to use it. At the end of each session, they were required to complete an Excel file specifying what they used and how long they spent on the program. This was stored on the program website for downloading later by the investigator. For technical reasons we had not been able to design the programs to record this information automatically.

ARTICLE IN PRESS 490

G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

Post-treatment: Between 6 and 8 weeks after enrolment, participants were recalled for interview face to face or by telephone as at the outset. The face-to-face participants did the AH3. All completed the questionnaires as before. There followed a short interview which required them to estimate the usefulness of the programs on a ten-point scale. They were then enrolled in the second program and paid £5 for participation. Telephone follow-up: Around 4 months afterwards the participants completed two STAI-SFs read to them to estimate the anxiety recalled for the exams prior to and after our intervention. They then were sent £10. Results Because the data showed normal distributions, according to inspection, parametric tests were used (SPSS, 2003) except where noted. Characteristics of the participants Our students had high levels of test anxiety, their mean on the TAI being more than one standard deviation above the average for a random sample of students (Spielberger, 1980). Thirty-nine were above that range. Eight did not have English as their first language. The remaining characteristics are summarised in Table 1 and by the baseline measures in Table 2. The control group had a higher female to male ratio compared with the CBT group but there were no other significant differences between these groups or between the dropouts and participants on any of the demographic variables (Table 1). The CBT and control groups also did not differ on baseline measures (Table 2) for the TAI ðt ð55Þ ¼ 0:38; p ¼ 0:71Þ, the AHQ ðt ð52Þ ¼ 0:19; p ¼ 0:85Þ, the EPSI ðt ð53Þ ¼ 0:52; p ¼ 0:61Þ, the exam-related GSES ðt ð52Þ ¼ 1:46; p ¼ 0:15Þ or the AH2 ðt ð37Þ ¼ 1:12; p ¼ 0:25Þ. On these measures there were also no significant differences at baseline between the dropouts ðN ¼ 28Þ and the others according to t-tests, the largest value being 0.19 (84) for the TAI ðp ¼ 0:43Þ. Those enrolled by telephone did not differ on the outcome variables from those enrolled face to face, except for exam problem solving on which those enrolled by telephone (mean ðSDÞ ¼ 50:13ð4:31Þ) had higher scores than the others (mean ½SD ¼ 44:73½6:91, t½54 ¼ 2:82; p ¼ 0:007). The values for the other variables were no Table 2 Means, 95% confidence intervals and standard deviations (in brackets), for the dependent variables by group and time Measure

Test anxiety Inventory (TAI)

Anxiety Hierarchy Questionnaire (AHQ) Exam Problem Solving Inventory (EPSI) AH tests (perceptual and numerical)

Exam-related General Self-Efficacy Scale (GSES) *

CBT group ðN ¼ 30Þ Pretreatment

Post-treatment

58.14 (55.00–61.19, 8.43) 12.64 (1.67)

47.31 (44.16–51.31, 9.49) 10.38 (3.45)

45.72 (54.98–60.95, 6.83) 44.55 (38.32–50.78, 13.32) 57.97 (54.98–60.95, 7.84)

47.93 (57.32–63.16, 5.90) 46.60 (40.37–52.83, 13.30) 60.24 (57.33–63.16, 7.67)

Control group ðN ¼ 28Þ Effect size*

Pretreatment

Post-treatment

0.88

59.18 (54.45–63.91, 12.20) 12.73 (1.66)

54.25 (49.89–58.64, 11.31) 12.62 (2.04)

46.67 (44.01–49.32, 6.71) 48.74 (44.80–52.68, 8.18) 53.46 (48.18–58.75, 13.09)

47.11 (45.31–48.91, 4.54) 51.58 (46.85–56.31, 9.82) 54.69 (49.34–60.04, 7.67)

1.28 0.25

0.04

0.20

Effect size** 0.40

0.07 0.07

0.35

0.09

CBT cf controls by formula ððX 1cbt  X 2cbt Þ=S1cbt Þ  ððX 1c  X 2c Þ=S1c Þ where X 1cbt and X 2cbt are the means for times 1 and 2, respectively, for the CBT group; X 1c and X 2c are the means, respectively, for times 1 and 2 for the controls; S1cbt and S1c are the standard deviations for time 1 for the CBT and control groups, respectively. ** Controls’ treatment cf. baseline: Cohen’s d (Kirk, 1995).

ARTICLE IN PRESS G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

491

greater than recorded for the TAI ðtð54Þ ¼ 1:42; p ¼ 0:17Þ. Therefore, method of enrolment was introduced as an independent variable only in the analysis of exam problem solving. Test Anxiety Inventory (TAI) A repeated-measures analysis of variance (SPSS, 2003) for test anxiety revealed a significant main effect of time ðF ð1; 56Þ ¼ 40:12; po0:001Þ and a significant time  group interaction ðF ð1; 56Þ ¼ 5:03; p ¼ 0:03Þ but no main effect of group ðF ð1; 56Þ ¼ 2:36; p ¼ 0:12Þ. This confirmed that, although the participants in the control group improved significantly ðt ð27Þ ¼ 3:07; p ¼ 0:005Þ, there was a greater improvement in the treatment group than in the controls (Table 2). A difference measure was then computed for each participant by subtracting the baseline score from the post-intervention value. The product moment correlation between the baseline score and the difference score was significantly greater than zero for both the CBT group ðr ¼ 0:44; po0:02Þ and the control group ðr ¼ 0:45; po0:02Þ. Thus the higher the level of anxiety at the beginning, the greater was the change with the interventions. A statistic, ‘‘c’’ (Jacobson & Truax, 1991), a cutoff point to identify subjects who have made a clinically significant change, compared our participants with 1449 college undergraduates (Spielberger, 1980). This showed that 20 among the 30 of our CBT group and eight of the 28 controls had improved beyond that threshold. According to a Reliable Change Index (Jacobson & Truax, 1991), 19 of the CBT participants and nine of the control group showed a reliable improvement. Therefore after treatment, 16 of the 30 CBT participants and eight of the 28 controls had made a reliable and clinically significant improvement on the TAI. Anxiety Hierarchy Questionnaire (AHQ) The TAI and the AHQ were positively correlated (product moment r ¼ 0:38; p ¼ 0:005) confirming the validity of the latter. Analysis of variance of the AHQ revealed a significant effect of time ðF ð1; 56Þ ¼ 11:52; p ¼ 0:001Þ and a time by group interaction ðF ð1; 56Þ ¼ 9:49; p ¼ 0:003Þ but no significant effect of group ðF ð1; 56Þ ¼ 3:80; p ¼ 0:06Þ. Thus, participants in the treatment group showed a steeper decline in anxiety than the controls (Table 2). The correlations between difference scores and baselines were not significantly greater than zero for the CBT group ðr ¼ 0:05; p ¼ 0:79Þ or the controls ðr ¼ 0:25; p ¼ 0:22Þ. therefore, the extent of improvement in anxiety according to the AHQ was not influenced by the baseline levels. Twenty of the 30 in the CBT group and 10 of the 28 participants in the control group lay outside the dysfunctional range after treatment, according to Jacobson and Truax’s (1991) ‘‘a’’ index, a more frequent clinically significant improvement for participants in the CBT group on the AHQ. Exam Problem-Solving Inventory (EPSI) There was no effect of group ðF ð1; 52Þ ¼ 0:19; p ¼ 0:67Þ, of time ðF ð1; 52Þ ¼ 0:54; p ¼ 0:46Þ, time  group interaction ðF ð1; 52Þ ¼ 0:11; p ¼ 0:74Þ or time by enrolment ðF ð1; 52Þ ¼ 1:64; p ¼ 0:21Þ but, as shown earlier, those enrolled by phone had higher scores than those enrolled face to face ðF ð1; 52Þ ¼ 8:01; p ¼ 0:007Þ. However, the results show that there were no changes with time according to treatment (Table 2) or method of enrolment. General Self-Efficacy Sub-Scale (GSES)—exam related There was a significant effect of time ðF ð1:52Þ ¼ 5:58; p ¼ 0:02Þ, but not of group ðF ð1:52Þ ¼ 3:27; p ¼ 0:08Þ or time by group interaction ðF ð1; 52Þ ¼ 0:50; p ¼ 0:48Þ. Exam-related self-efficacy improved to a similar degree in both groups (Table 2). Performance on AH2 and AH3 tests of general reasoning There was a significant effect of time ðF ð1; 37Þ ¼ 5:34; p ¼ 0:03Þ but no significant group effect ðF ð1; 37Þ ¼ 1:50; p ¼ 0:05Þ or time by group interaction ðF ð1; 37Þ ¼ 0:34; p ¼ 0:28Þ. Thus, the treatment and control participants showed a similar improvement in performance on this task (Table 2).

ARTICLE IN PRESS 492

G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

Computer-recorded data and qualitative debriefing In several instances, data recorded by the participants on spreadsheets were incomplete because of technical difficulties or because participants logged off incorrectly. Nevertheless, data from the spreadsheets showed that they spent on average a total of 157 min ðSD ¼ 138Þ on the CBT program and 75 min ðSD ¼ 45Þ on the control program, a significant difference ðt ð56Þ ¼ 2:379; p ¼ 0:02Þ. All participants said that they had looked at all the modules in their allocated program. Their mean rating of usefulness for the treatment program was 8.00 ðSD ¼ 1:02Þ and 4.50 ðSD ¼ 1:87Þ for the control program, a significant advantage for the former (Wilcoxon Z ¼ 4:03; po0:001). However, there was considerable overlap in the distribution of these scores. Forty percent of the CBT group gave a mid-scale rating for their program while 21% of the control group gave that rating for their program. In the CBT program ‘‘Your Own Strategies’’ was used by 10 people, each once only. The remaining modules, ‘‘Relaxation’’ ðM ½SD ¼ 2:08 ½2:42Þ, ‘‘Rational Answers’’ ðM ½SD ¼ 1:96 ½2:26Þ, ‘‘Study Skills’’ ðM ½SD ¼ 1:57 ½1:39Þ and ‘‘Control Your Stress’’ ðM ½SD ¼ 0:92 ½1:37Þ differed in the number of times they were used according to a Friedman test ðw2 ½3 ¼ 27:72; po0:001Þ. Post-exam follow-up phone interview Because 10 participants had not completed exams since our interventions and 10 were out of the country, 38 completed the follow-up interview by which time all had opportunity to try both programs. Their mean STAISF score was 20.29 ðSD ¼ 2:57Þ for exams recalled before the interventions and 13.00 ðSD ¼ 2:97Þ for exams taken afterwards, a significant improvement in anxiety (paired t ð37Þ ¼ 15:63; p ¼ 0:000). All but one participant had improved. Twenty-seven said that only the treatment program was helpful. Discussion Our findings Our students had experienced high test anxiety. Because of exams 25 had been ill, 15 had used medication and eight had sought psychological help. According to the TAI and the AHQ, our CBT program produced a statistically significant greater reduction in test anxiety. That was greater than the change under the control procedure, both programs having the novelty and presumed authority of the Internet. These results were reflected in an improvement in anxiety recalled for subsequent exams according to the STAI-SF. Furthermore, the most anxious students improved most according to the TAI. The TAI measures apprehension about tests recalled from experience as, for example, ‘‘during tests I feel very tense’’. Therefore, it would not have been most sensitive to anxiety about forthcoming exams. The AHQ, scores on which were correlated with those on the TAI, was thus more valid in that respect. Nevertheless, the TAI has the advantage of published data from a normative sample, which allowed us to show reliable improvement according to the Reliable Change Index in 63% of the CBT group. Comparison with published norms also showed that 67% of this group had improved to lie within the normal range after treatment. Combining these findings showed that 53% had made a reliable improvement to lie within the normal functional range of the TAI. The statistical effect sizes (Table 2), another criterion, were 0.88 for the improvement in the TAI and 1.28 for the AHQ in the CBT group, compared with the controls. This is greater than for treatments delivered by clinicians, according to a meta-analysis (Ergene, 2003). Moreover, these effect sizes are large according to convention (Kirk, 1995). The control group showed a modest effect size on the TAI and this may partly reflect the authority of the Internet. That could be tested by comparing our control program with a no treatment procedure. Our interventions had no effect on confidence about solving problems associated with exams (EPSI) in spite of the improvement in test anxiety. The exam-related General Self-Efficacy Sub-Scale (GSES) was significantly correlated with the TAI before and after treatment, both coefficients being greater than 0:40 ðp ¼ 0:01Þ. However, with the AHQ this was significant only after treatment ðr ¼ 0:31; p ¼ 0:02Þ. Thus there was a weak relationship in that the more anxious the participants were, the less effective did they feel about exams. As

ARTICLE IN PRESS G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

493

would be expected, therefore, an improvement occurred in exam-related self-efficacy although our treatment was no better than the control program in this. The GSES, contains such items as, ‘‘I cannot get down to work as I should’’. Therefore, the improvement in GSES scores probably shows that the participants increased their resolution about preparing for exams. This could have determined their perseverance with the programs. The effect on the GSES might have been greater if we had been allowed to exclude the items on social self-efficacy which was not relevant to our study. On the AH2–AH3 tests both groups improved, probably a practice effect because the change resembled that reported for test–retest results (Cane & Heim, 1950). We chose these tests because our university students, being high achievers, were anxious to do well. However, the outcome of the tests would not have been sufficiently important to provoke much anxiety. Critique of our study Several aspects of our programs could be improved. The module ‘‘Your Strategies’’ was used much less often than the others in the CBT program and so could be omitted. Our control program might have offered a closer comparison with the CBT program and so would have offered a stricter test if it had a simulated desensitisation module corresponding to ‘‘Control Your Stress’’ in the CBT program. That treatment module might have been improved if participants could have signalled a reduction in anxiety with each step and so be allowed to proceed to the next. In our module, participants could move to next step when they liked as in a previous successful version of desensitisation (Meichenbaum, 1972). ‘‘Apply Rational Thinking’’ in the CBT program provided instruction in cognitive therapy. It did not present any response to the participants’ presentation of negative thoughts and challenges. It did not have a therapist who could respond empathically and guide clients to disclose thoughts more consonant with the emotions expressed or who could identify core beliefs, the key to lasting benefits of cognitive therapy (Sanders & Wills, 2005). Our procedure required our investigator to introduce the programs to the participants and that would have added to the expense of administration especially where the investigator visited the participants for a face-toface introduction. However, introducing the programs by phone for 39% of participants appears not to have compromised efficiency although our participants preferred face-to-face introduction. Therefore, the less expensive alternative of introducing all participants by telephone might not have compromised efficacy. There is no consensus about the extent to which a therapist should be involved in CBT delivered by the Internet (Carlbring et al., 2003; Richards et al., 2003). We were interested in Internet-delivered therapy for people who lack the opportunity for interviews with a therapist. Therapy by e-mail, telephone and televisual conferencing (Barnett & Scheetz, 2003) may be as effective as our Internet CBT program. Therefore, extent of therapist involvement and other means of delivering programs from a distance should be investigated. The instruction to the participants, that one program would be more effective than the other, required by our Ethical Committee, may have reduced improvement with the control program. However, the participants were not told which program would be the less effective. Therefore, they could not have perceived the difference, if at all, until they had extensive experience of the programs or until they had seen both programs after we had collected the data. Moreover, the distribution of scores given by the participants for the usefulness of the programs overlapped and so many participants would not have been clear, before the end of the study, about which program was the less effective. In addition, the number of dropouts was no greater for the control program than for CBT. However, on average, the participants probably did spend less time on the former probably because, as they reported, they had found it less useful. Thus there was probably a circular relationship for an indeterminate number of participants between growing pessimism about the placebo program as they used it and their perseverance with it. It is impracticable to exclude such expectations (NICE, 2002) even in pharmacological trials (Margraf et al., 1991). As in most other studies of computerised treatment (NICE, 2002), our control program therefore provided an imperfect comparison. Hence the CBT may have been more successful than the control program in part because the participants expected to benefit more. Nevertheless, participants on our placebo program did become less anxious according to the TAI ðt ð27Þ ¼ 3:07; p ¼ 0:005Þ although not according to the AHQ ðt ð27Þ ¼ 0:33; p ¼ 0:75Þ. In view of the success of our CBT it is disappointing that so many participants, 13 of the 47, dropped out of that program and so may have failed to benefit. On the other hand, the treatment program may have been

ARTICLE IN PRESS 494

G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

successful because the most unresponsive subjects withdrew. That is probably unlikely because there was no difference between the dropouts and the persevering participants on any of the dependent variables. Our study was not exceptional in the numbers of dropouts in computerised treatment which has been as high as 50% in other studies of anxiety (Kaltenhaler et al., 2004). Although attrition can be high in treatment delivered by the Internet (Andersson, Bergstrom, Carlbring, & Lindefors, 2005; Richards et al., 2003), most studies of anxiety have too small numbers for realistic estimates. They range from one among 11 (Klein & Richards, 2001), six of 49 (Carlbring et al., 2005) and three of nine (Richards & Alvarenga, 2002) to 15 of 41 (Carlbring et al., 2001). + Pettersson, & Andersson, 2000). In an Internet study of headache, 56% of 102 participants withdrew (Strom, Of our participants who gave a reason for withdrawing, only three, all in the control program, said that their program was not sufficiently effective. Other reasons were the pressure of course-work, understandable in anxious subjects, and difficulties in accessing the Internet. People who sign up by e-mail or web sites may do so + et al., 2000) and may abandon access to a web site if it takes longer than 10 sec (Nielsen, impulsively (Strom 2000). Hence the effect of faster access should be studied. The advantages of our treatment program are probably similar to those expressed for other therapy delivered by computer. Subjects can proceed at their own pace, they can use the programs when it is convenient for them, some feel less uncomfortable than if they have to disclose themselves to a therapist face to face (Bloom, 1992). Computer programs, especially those delivered by the Internet, may be more appealing to young people accustomed to using them extensively. It may be possible to monitor the performance of clients’ homework more accurately by the Internet than in face-to-face therapy. Electronic recording of clients’ entering or leaving programs and modules should be practicable. Our study would have been improved if we had been able to do that. However, it is not clear if it would be possible in Internet programs to detect clients’ browsing through programs rather than engaging with them thoroughly. No study of Internet programs for anxiety disorders has raised this issue and so it remains to be investigated. Our CBT program appears to have been unique for test anxiety in that it included cognitive therapy among several other components. Because there has been no study of the contribution of separate components to outcome in a multimodal treatment of test anxiety, even in face-to-face therapy (Zeidner, 1998), it would have been tempting to have used our programs to investigate this. However, the effects of the modules may have been influenced by the extent to which our participants engaged with, rather than browsed through, each one. Moreover, our modules were not matched for length. Participants could spend as long as they liked on several of them and could repeat them. Furthermore, there may also have been interactions among the modules in which their sequence played a part. For example, participants who preferred the relaxation module might have concentrated less on the cognitive module because that was presented second. To investigate the effect of each module, investigators should therefore vary their contents and sequence, where the participants spend the same time on each and where the participants’ engagement is closely monitored. As our CBT program stands, however, it is ecologically valid in that participants could use it as they liked, as with any other web site. Finally we note that our follow-up period of 4 months was brief and it would be instructive to determine the duration of the beneficial effects of the CBT program. However on the Internet, the CBT program could be very accessible and so could be used by students repeatedly as booster treatment. Nevertheless, this also remains to be investigated. Conclusions We have shown that our CBT program, introduced by a therapist but delivered by the Internet and used freely by University students, can reduce test anxiety substantially according to statistical effect sizes, can effect a reliable and clinical significant improvement in more than half a sample of test anxious students and as effectively as face-to-face therapy, according to effect sizes compared with other studies. It would be less expensive of therapists’ time and more accessible to students than therapist-delivered therapy. It would be important, therefore, to determine the effectiveness of our program for other groups including less able students for whom test anxiety is most disruptive. There are probably many Internet programs of questionable value in health care (Briggs et al., 2002). Therefore, we believe that evidence-based programs such as ours, with clear guidance about their use, should have the approval of a professional organisation before being made available on the Internet to the public.

ARTICLE IN PRESS G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

495

Acknowledgements The authors gratefully acknowledge the help of our statistician, Dr. Sabine Landau, and CPP who gave us permission to modify and use the Problem-Solving Inventory. The computer programs were designed and produced by MindLife. References Andersson, G., Bergstrom, J., Carlbring, P., & Lindefors, N. (2005). The use of the Internet in the treatment of anxiety disorders. Current Opinion in Psychiatry, 18, 73–77. Barnett, J. E., & Scheetz, K. (2003). Technological advances and telehealth: Ethics, law and the practice of psychotherapy. Psychotherapy: Theory, Research, Practice, Training, 40, 86–93. Beck, A. T., Emery, G., & Greenberg, R. L. (1996). Cognitive therapy for evaluation anxieties. In C. G. Lindemann (Ed.), Handbook of the treatment of the anxiety disorders (2nd ed., pp. 235–260). Northvale, NJ: Aronson Inc. Bloom, B. L. (1992). Computer-assisted psychological intervention: A review and commentary. Clinical Psychology Review, 12, 169–198. Briggs, P., Burford, B., De Angeli, A. G., & Lynch, P. (2002). Trust in online advice. Social Science Computer Review, 20, 321–332. Buglione, S. A., Devito, A. J., & Mulloy, J. M. (1990). Traditional group therapy and computer-administered treatment for test anxiety. Anxiety Research, 3, 33–39. Cane, V. R., & Heim, A. W. (1950). The effects of repeated retesting: III. Further experiments and general conclusions. Quarterly Journal of Experimental Psychology, 2, 182–195. Carlbring, P., Ekselius, L., & Andersson, G. (2003). Treatment of panic disorder via the Internet and a randomised trial of CBT vs. applied relaxation. Journal of Behavior Therapy and Experimental Psychiatry, 34, 129–140. Carlbring, P., Nilsson-Ihrfelt, E., Waara, J., Kollenstam, C., Buhrman, M., Kaldo, V., et al. (2005). Treatment of panic disorder: Live therapy vs. self-help via the Internet. Behaviour Research and Therapy, 43, 1321–1334. Carlbring, P., Westling, B. E., Ljungstrand, P., Ekselius, L., & Andersson, G. (2001). Treatment of panic disorder via the Internet: A randomised trial of a self-help program. Behaviour Therapy, 32, 751–764. Cassady, S. C., & Johnson, R. E. (2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychology, 27, 270–295. Ergene, T. (2003). Effective interventions on test anxiety reduction. School Psychology International, 24, 313–328. Fisher, C. B., & Fried, A. L. (2003). Internet-mediated psychological services and the American Psychological Association Ethics Code. Psychotherapy: Theory, Research and Training, 40, 103–111. Heim, A., Watts, K. P., & Simmonds, V. (1974). AH2/AH3 group test of general reasoning manual. Windsor: NEER-Nelson. Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 7–77. Heppner, P. P., & Petersen, C. H. (1985). The development and implications of a personal problem-solving inventory. Journal of Counseling Psychology, 29, 66–75. Hong, E., & Karstensson, L. (2002). Antecedents of state test anxiety. Contemporary Educational Psychology, 27, 348–367. Jacobson, E. (1938). Progressive relaxation. Chicago: University of Chicago Press. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. Jones, L., & Petruzzi, D. C. (1995). Test anxiety: A review of theory and current treatment. Journal of College Student Psychotherapy, 10, 3–15. Kaltenhaler, E., Parry, G., & Beverley, C. (2004). Computerised cognitive behaviour therapy: A systematic review. Behavioural and Cognitive Psychotherapy, 32, 31–56. Kaltenhaler, E., Shackleg, P., Stevens, K., Deverley, C., Parry, G., & Chilcott, J. (2002). Computerised cognitive behaviour therapy for depression and anxiety. London: National Institute for Clinical Excellence. hwww.nice.org.uk/i. Kenardy, J., Mc-Cafferty, K., & Rose, V. (2003). Internet-delivered indicated prevention for anxiety disorders: A randomised controlled trial. Behavioural and Cognitive Psychotherapy, 31, 279–289. Klein, P. (2000). Handbook of psychological testing (2nd ed.). London: Routledge. Kirk, R. (1995). Experimental design. Procedures for the behavioral sciences. Pacific Grove: Brooks Cole. Kirkland, K., & Hollandsworth, J. G. (1979). Test anxiety, study skills and academic performance. Journal of College Student Personnel, 20, 431–436. Klein, B., & Richards, J. C. (2001). A brief Internet-based treatment for panic disorder. Behavioural and Cognitive Psychotherapy, 29(29), 113–117. Lange, A., Schrieken, B., van de Ven, J. P., Bredeweg, B., & Emmelkamp, M. G. (2000). Intertherapy: The effects of a short protocolled treatment of post-traumatic stress and pathological grief through the Internet. Behavioural and Cognitive Psychotherapy, 28, 175–192. Margraf, J., Ehlers, A., Roth, W. T., Clark, D. B., Sheikh, J., Agras, W. S., et al. (1991). How blind are double-blind studies? Journal of Consulting and Clinical Psychology, 59, 184–187. Marks, I. M., Kenwright, M., McDonough, M., Whittaker, M., & Mataix-Cols, D. (2004). Saving clinicians’ time by delegating routine aspects of therapy to a computer: A randomised controlled trial in phobia/panic disorder. Psychological Medicine, 34, 9–17.

ARTICLE IN PRESS 496

G. Orbach et al. / Behaviour Research and Therapy 45 (2007) 483–496

Marteau, T. M., & Bekker, H. (1992). The development of a six-item short-form of the state scale of the Spielberger state-trait anxiety inventory (STAI). British Journal of Clinical Psychology, 31, 301–306. McDonald, A. S. (2001). The prevalence and effects of test anxiety in school children. Educational Psychology, 21(1), 89–101. Meichenbaum, D. H. (1972). Cognitive modification of test anxious college students. Journal of Consulting and Clinical Psychology, 39, 370–380. Morris, L. W., & Liebert, R. M. (1969). Effects of anxiety on timed and untimed intelligence tests: Another look. Journal of Consulting and Clinical Psychology, 33, 240–244. National Institute for Clinical Excellence. (2002) Guidance on the use of computerised cognitive behavioural therapy for anxiety and depression. NICE technology appraisal guidance no. 51. London: NICE. Nielsen, J. (2000). Designing web usability: The practice of simplicity. Indianapolis: New Riders. Orbach, G. (2002). The Development and evaluation of an Internet-based computerised intervention for exam anxiety. Thesis, Institute of Psychiatry, University of London, London, unpublished. Richards, J., Klein, B., & Calbring, P. (2003). Internet-based treatment for panic disorder. Cognitive Behaviour Therapy, 32(3), 125–135. Richards, J. C., & Alvarenga, M. E. (2002). Extension and replication of an internet-based treatment for panic disorder. Cognitive Behaviour Therapy, 31, 41–47. Sanders, D., & Wills, F. (2005). Cognitive therapy. An introduction (2nd ed.). London: Sage Publications. Schneider, A. J., Mataix-Cols, D., Marks, I. M., & Bachofen, M. (2005). Internet guided self-help with or without exposure therapy for phobia and panic disorders. Psychotherapy and Psychosomatics, 74, 154–164. Seipp, B. (1991). Anxiety and academic performance: A meta analysis of findings. Anxiety Research, 4, 27–41. Sherer, M., Maddux, J. E., Mercandente, B., Prentice-Dunn, S., Jacobs, B., & Rogers, R. W. (1982). The self-efficacy scale: Construction and validation. Psychological Reports, 51, 663–671. Spielberger, C. D. (1980). Test anxiety inventory: Preliminary professional manual. Palo Alto, CA: Consulting Psychologists Press. Spielberger, C. D., & Vagg, P. R. (1995). Test anxiety: A transactional process. In C. D. Spielberger, & P. R. Vagg (Eds.), Test anxiety: Theory, assessment and treatment (pp. 3–14). Washington, DC: Taylor and Francis. SPSS. (2003). SPSS 11.5 for Windows. Chicago: SPSS Inc. + Strom, L., Pettersson, R., & Andersson, G. (2000). A controlled trial of self-help treatment of recurrent headache conducted via the Internet. Journal of Consulting and Clinical Psychology, 68, 722–727. University of South Australia. (2001). Making exam anxiety work for you. URL: hhttp://www.unisanet.unisa.edu.au/controlexamanxietyi. Vagg, P. R., & Spielberger, C. D. (1995). Treatment of test anxiety: Application of the transactional process model. In C. D. Spielberger, & P. R. Vagg (Eds.), Test anxiety: Theory, assessment and treatment (pp. 195–215). Washington, DC: Taylor and Francis. Wachelka, D., & Katz, R. C. (1999). Reducing test anxiety and improving academic self-esteem in high school students and college students with learning disabilities. Journal of Behaviour Therapy and Experimental Psychology, 30, 191–198. Zeidner, M. (1998). Test anxiety. The state of the art. New York: Plenum Press.