Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members

Archives of Clinical Neuropsychology 20 (2005) 33–65

Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members夽 Laura A. Rabin a,∗ , William B. Barr b , Leslie A. Burton c a

Dartmouth Medical School, Department of Psychiatry, Dartmouth-Hitchcock Medical Center, One Medical Center Drive, Lebanon, NH 03756, USA b NYU Comprehensive Epilepsy Center, New York University Medical School of Medicine, New York, NY, USA c Fordham University, Department of Psychology, Bronx, NY, USA Accepted 26 February 2004

Abstract The present study surveyed assessment practices and test usage patterns among clinical neuropsychologists. Respondents were 747 North American, doctorate-level psychologists (40% usable response rate) affiliated with Division 40 of the American Psychological Association (APA), the National Academy of Neuropsychology (NAN), or the International Neuropsychological Society (INS). Respondents first provided basic demographic and practice-related information and reported their most frequently utilized instruments. Overall, the Wechsler Adult Intelligence Scales and Wechsler Memory Scales were most frequently used, followed by the Trail Making Test, California Verbal Learning Test, and Wechsler Intelligence Scale for Children. Respondents also reviewed a vignette about a traumatic brain injury patient, and then reported the instruments they would use to assess this patient’s specific cognitive symptomatology, general cognitive ability, and capacity to return to work. Particular attention was paid to the areas of memory, attention, and executive functioning. The current study represents the largest and most comprehensive test usage survey conducted to date within the field of clinical neuropsychology. Survey results update and greatly expand knowledge about neuropsychologists’ assessment practices. Following a review of findings, results are compared to those obtained in prior surveys and implications for the field of neuropsychology are discussed. © 2004 National Academy of Neuropsychology. Published by Elsevier Ltd. All rights reserved. Keywords: Test usage; Current assessment practices; Neuropsychology survey 夽

The study reported in this article was derived from the dissertation of Laura A. Rabin. Tel.: +1-603-650-5811. E-mail address: [email protected] (L.A. Rabin).

∗

0887-6177/$ – see front matter © 2004 National Academy of Neuropsychology. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.acn.2004.02.005

34

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

1. Introduction Survey research has tracked various aspects of psychological assessment since the 1930s. The first published survey of psychological test usage patterns in U.S. clinical settings, Report of Committee of Clinical Section of the APA, was printed in 1935. This list contained numerous measures of verbal and performance abilities, but few measures of personality. The next survey of this kind was conducted a decade later and exposed marked changes in psychological assessment practices (Louttit & Browne, 1947). Its findings revealed the increasing popularity of adult intelligence and reading tests, vocational interest scales, and projective measures of personality and emotional behavior. Following the publication of these initial studies, other researchers conducted similar test usage surveys, often comparing their results with preceding works. Darley and Marquis (1946) surveyed test practices in veterans’ guidance centers, Swenson and Lindgren (1952) reported on the use of psychological tests in industry, and Sundberg (1961) surveyed the testing practices of clinicians in various agencies and hospitals. Notably, Sundberg found, in the period from 1935 to 1961, a significant decrease in the use of intelligence tests and a rise in the use of projective tests. The Rorschach (Beck, 1944), a projective measure, was the highest-ranked instrument, surpassing all others in the number of settings that employed the test and the frequency of its usage. Another national survey of psychological test usage was published in 1971 (Lubin, Wallis, & Paine, 1971). In the 10 years between the Sundberg (1961) and the Lubin et al. study, the Wechsler Adult Intelligence Scale (WAIS; Wechsler, 1955), a measure of intelligence, had replaced the Rorschach as the highest-ranked instrument. Another finding was the continued popularity of projective tests relative to objective personality measures. These trends were supported in later surveys of psychologists in mental health agencies and hospitals (Brown & McGuire, 1976), the APA Division 12 (Clinical Psychology; Wade & Baker, 1977; Wade, Baker, Morton, & Baker, 1978), and outpatient mental health facilities (Piotrowski & Keller, 1978). Other surveys during the 1970s revealed that practitioners in specialty areas favored instruments within their areas of competence. For example, Sell and Torres-Henry (1979) found that counseling psychologists frequently used career assessment instruments. In addition, Wade, Baker, and Hartmann (1979) found that behavior therapists used behavioral interviews, observation, and surveys more frequently than intelligence tests, objective and projective personality measures, or neuropsychological tests. While the overall pattern of survey findings from the 1930s through the 1970s reveals that intelligence and projective tests dominated psychological testing, some early surveys included a small number of neuropsychological instruments. These tests and the survey years in which they appeared, chronologically ordered, were as follows: BRL Sorting (1947), Halstead Sorting Test (1947), Vygotsky Concept Formation (1947), Wells Memory (1947), Wechsler Memory Scale (1947, 1961, 1976; WMS; Wechsler, 1945, 1974), Concept Formation (1961), Benton Visual Retention Test (1971; BVRT; Benton, 1946), Graham–Kendell Memory for Designs Test (1971, 1978; MFD; Graham & Kendall, 1960), and the Halstead–Reitan Neuropsychological Battery (1971; HRNB; Reitan & Wolfson, 1985). Although listed infrequently and not ranked as highly as tests from other categories, their very presence suggests that neuropsychological tests were occasionally incorporated into assessment batteries of these times.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

35

Beginning in the 1980s, researchers attempted to investigate and characterize the vast growth occurring within the field of clinical neuropsychology. A series of mailed surveys were conducted to examine issues related to educational backgrounds and specialty training of clinical neuropsychology instructors (McCaffrey & Issac, 1984; McCaffrey & Lynch, 1996), graduate training programs in neuropsychology (McCaffrey, Malloy, & Brief, 1985), professional self-designation and putative professional characteristics (Slay & Valdivia, 1988), opinions regarding postconcussional syndrome (McMordie, 1988), salary ranges (Putnam, 1989; Putnam & DeLuca, 1991; Putnam, DeLuca, & Anderson, 1994), the use of neuropsychological technicians and other assistants (DeLuca, 1989; DeLuca & Putnam, 1993), practices and beliefs among neuropsychologists certified by the American Board of Professional Psychology (ABPP) and non-ABPP neuropsychologists (Sweet & Moberg, 1990; Sweet, Moberg, & Suchy, 2000; Sweet, Moberg, & Westergaard, 1996; Sweet, Peck, Abramowitz, & Etzweiler, 2002), qualifications of neuropsychology internship supervisors (Ryan & Paolo, 1990), neuropsychologists’ ethical beliefs and behaviors regarding practice issues (Brown, 1999), preference for the HRNB or Luria–Nebraska Neuropsychological Battery (LNNB; Golden, Purisch, & Hammeke, 1985; Guilmette & Faust, 1991), and neuropsychological test usage (Butler, Retzlaff, & Vanderploeg, 1991; Camara, Nathan, & Puente, 2000; Lees-Haley, Smith, Williams, & Dunn, 1996). Additional studies covered a broader range of topics, from sociodemographic characteristics, practice issues, assessment, and education and training, to views of professional matters (Guilmette, Faust, Hart, & Arkes, 1990; Hartlage & Telzrow, 1980; Putnam & DeLuca, 1990; Seretny, Dean, Gray, & Hartlage, 1986). Hartlage and Telzrow (1980) conducted the first published study to directly address assessment issues in the field of neuropsychology. The investigators mailed surveys to each full member of NAN, questioning issues related to work setting, professional practice, test usage, practice preparation, and important figures in the field. The majority of respondents identified hospitals as their primary work setting, while private practice and college or university settings were also fairly common. On average, 15 neuropsychological evaluations were performed monthly; including report writing, each averaged 8 h in length. The researchers provided usage data on 16 tests and batteries. The 10 most commonly used instruments were: age-appropriate Wechsler Intelligence Scales, partial HRNB, Wide Range Achievement Test (WRAT; Jastak & Bijou, 1941), Bender–Gestalt (Bender, 1946), full HRNB, BVRT, Luria (Christensen/Golden; Christensen, 1975), WMS, MFD, and the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1951). Over the next several years, the field of clinical neuropsychology underwent tremendous growth and change, necessitating a revised study of practice issues. Seretny et al. (1986) surveyed neuropsychologists about their work settings, practice activities, test usage, referral sources, fees, practice preparation, involvement in legal proceedings, and use of technicians. NAN and APA Division 40 (clinical neuropsychology) members completed the questionnaire, and findings were compared with those of Hartlage and Telzrow (1980) and an unpublished 1980 report of test usage patterns of APA Division 40 members (as cited in Seretny et al., 1986). Results indicated that both organizations had shifted the top-ranked work setting from hospitals to private practice. This change likely reflected an increased number of psychologists with training in other primary areas who introduced neuropsychological techniques to their clinical practices. In addition, the average number of monthly evaluations had decreased, from 15 to 11.

36

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

The researchers provided usage data on 11 tests and batteries. As before, the age-appropriate Wechsler Intelligence Scale was the highest-ranked test, followed by portions of the HRNB and LNNB. Other commonly employed instruments were the WRAT, BVRT, MMPI, Mental Status Exam (MSE; Strub & Black, 1977), WMS, clinical assessment of subcortical functions, and the Wisconsin Card Sorting Test (WCST; Heaton, Chelune, Talley, Kay, & Curtiss, 1993). Guilmette et al. (1990) conducted a follow-up to Hartlage and Telzrow’s (1980) survey. They gathered information concerning a range of practice issues, including professional affiliations, education and training, referral sources, clients served, work settings, use of assessment instruments, and views on professional issues. The researchers did not limit exploration to specialists in the field; rather, they selected psychologists of varying levels of training, experience, and involvement with neuropsychology. Results revealed great diversity within the field, with the modal practitioner minimally trained and involved in neuropsychological activities. In general, the more one’s professional activities were devoted to neuropsychology, the better one’s training. The three most frequent reasons reported for conducting neuropsychological evaluations were to rule in or out organic conditions, assess capacity to work, and develop an educational plan. Neuropsychologists reported more frequent employment in rehabilitation centers than in previous surveys. The researchers also provided usage data on 13 tests and batteries. Instruments were rank ordered in the following manner: WAIS, WMS, Trail Making Test (Lewis & Rennick, 1979), MMPI, WRAT, BVRT, partial HRNB, Rorschach, BVRT, full HRNB, WCST, LNNB, and Luria–Christianson Procedures. Putnam and DeLuca (1990) conducted a practice survey of APA Division 40 members. The questionnaire contained two independent sections, respectively addressing respondents’ primary employment settings and private practices. Investigators surveyed a variety of topics related to clinical practices, patient characteristics, and fee-setting. While this survey did not directly address test usage, several findings related to assessment issues. For example, APA Division 40 members reported spending approximately half their clinical time conducting neuropsychological evaluations, assessing an average of nine patients per month. Medical centers and private practices proved the most common employment settings. The activity that consumed the most time for respondents in both groups was neuropsychological assessment, followed, in rank order, by “other neuropsychological activities” (e.g., research or teaching), neuropsychological treatment, and consultation. Results also indicated that clinical neuropsychologists served a disproportionate number of young and middle-aged adults relative to children and older patients. Head trauma and psychiatric patients were seen most frequently, followed by patients with learning disabilities and strokes. Sweet and Moberg (1990) surveyed ABPP clinical neuropsychologists and APA Division 40 non-ABPP neuropsychologists on issues related to personal and practice demographics, professional background, referrals, training and assessment philosophies, allocation of clinical time, information gathering, and journal preferences. Although this study did not investigate test usage patterns, several findings related to assessment practices. First, both groups reported approximately 28 h per week spent in clinical practice, though ABPPs spent more time performing neuropsychological evaluation and treatment. The same five referral sources were found for both groups: neurologists, psychiatrists, rehabilitation specialists, attorneys, and neurosurgeons. Both ABPPs and non-ABPPs favored a “flexible battery” comprised of variable but routine groupings of tests for different types of patients. The distant second choice

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

37

for both groups was a “completely flexible” approach, based upon the needs of each individual case. The third choice for both groups was a “standardized battery” approach, utilizing a routine grouping of tests uniform across patient types. Relatively few respondents reported spending extensive clinical time with children, adolescents, or geriatric adults. In terms of data collection, both groups almost always reported gathering medical and psychological history and neuropsychological data, while projective personality testing commonly went ungathered. Butler et al. (1991) conducted the first extensive neuropsychological test usage survey. Respondents included 250 members of INS (56% response rate). The questionnaire listed 116 instruments abstracted from the literature and instructed respondents to endorse tests routinely selected or used with “most” of their patients. Results indicated that the most common theoretical orientation was eclectic, followed by a hypothesis-testing approach. Less common were the HRNB, Benton, and Lurian approaches. Most respondents indicated that they typically performed additional testing of memory, speech and language, visuospatial, psychomotor, executive, and personality functions. However, great variability existed in the choices of instruments used. The WAIS/Wechsler Adult Intelligence Scale-Revised (WAIS-R; Wechsler, 1981) was by far the most widely used instrument. The next 10 most commonly used instruments, in rank order, were: WCST, MMPI, Boston Naming Test (Kaplan, Goodglass, & Weintraub, 1978), Rey–Osterrieth Complex Figure Task (ROCFT; Rey, 1941), Trail Making Test, Grooved Pegboard Test (GPT; Matthews & Klove, 1964), Wechsler Memory Scale-Revised (WMS-R; Wechsler, 1987), Token Test (De Renzi & Vignolo, 1962), Controlled Oral Word Association Test (COWAT; Benton & Hamsher, 1989), and Rey Auditory Verbal Learning Test (RAVLT; Wiens, McMinn, & Crossen, 1988). Lees-Haley et al. (1996) examined the frequency with which various psychological tests were utilized in forensic evaluations. Rather than using a self-report format, they gathered data from reports of adult personal injury evaluations by 100 examiners identified as neuropsychological experts for cases in litigation. They then rank ordered the tests mentioned in the reports in terms of usage and functional category. Results indicated that the number of tests administered in an evaluation ranged from 1 to 32, with an average of 11.7. No two experts used identical batteries, except those few who used under five tests for evaluations. The most frequently assessed functions, in rank order, were intelligence, personality, memory, mental flexibility, motor, visuospatial, and abstract concept formation. The researchers also provided usage data on 44 tests and batteries. The most frequently used tests, in rank order, were the WAIS-R, MMPI/Minnesota Multiphasic Personality Inventory-Second Edition (MMPI-2; Graham, 1990), WMS/WMS-R, Trail Making Test, Finger Oscillation Test (Reitan & Wolfson, 1985), Bender–Gestalt, Halstead Category Test (Halstead, 1947) and the WCST. Sweet et al. (1996, 2000) reported 5- and 10-year follow-up studies to the original Sweet and Moberg (1990) study, with the hope of delineating areas of stability and change in the field of clinical neuropsychology. Results revealed increases in the number of neuropsychologists trained at the predoctoral level and in the number of younger neuropsychologists attaining ABPP Diplomate status. In fact, among all 11 practice areas in which the ABPP awarded board certification, clinical neuropsychology was proportionally the fastest growing area. Across the 10-year span, there continued to be relatively few neuropsychologists whose practices included a substantial number of children and adolescents; however, the percentage of neuropsychologists working with geriatric patients had increased. For both groups (i.e., ABPP diplomates and

38

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

non-ABPP neuropsychologists), the percentage of practitioners employed in private or group practice settings increased while the percentage employed in medical settings decreased. Psychiatrists gradually replaced neurologists as the top referral source for both groups. As in the original Sweet and Moberg (1990) survey, test usage issues were not explicitly investigated, though several findings related to assessment practices. For example, members of both groups reported an increase in the number of clinical practice hours related to neuropsychological activities and lessened activity teaching and conducting research. These changes may have related to healthcare trends, such as the proliferation of managed care, which created an economic need for greater clinical activity and administration. In addition, the overall number of neuropsychologists favoring a flexible battery approach steadily increased, while support for both the completely flexible and standardized approaches declined. Lastly, the number of respondents providing treatment activities (i.e., rehabilitation, remediation, or psychotherapy) increased substantially over the 10-year span. Thus, the view of clinical neuropsychologists as involved exclusively in assessment activities proved narrow and inaccurate (Sweet & Moberg, 1990; Sweet et al., 1996, 2000). Camara et al. (2000) investigated test usage by APA clinical psychologists and NAN neuropsychologists. Findings revealed that over 80% of clinical psychologists spent fewer than 5 h each week performing assessment activities. In contrast, 33% of neuropsychologists devoted in excess of 20 h per week to assessment, while another 35% devoted 10–20 h to this activity. Respondents also provided information about the utilization of specific assessment techniques by choosing instruments from a list of 120 individual tests. Only data from respondents who devoted more than 5 h each week to assessment activities (i.e., 179 clinical psychologists and 447 clinical neuropsychologists) were included in the analyses. Results indicated that clinical psychologists used an average of 13.4 separate tests in their assessment practice. The WAIS-R, MMPI/MMPI-2, Wechsler Intelligence Scale for Children-Third Edition (WISC-III; Wechsler, 1991), Rorschach, Bender–Gestalt, Thematic Apperception Test (TAT; Morgan & Murray, 1935), Wide Range Achievement Test-Revised/Third Edition (WRAT-R/WRAT-3; Jastak & Wilkinson, 1984; Wilkinson, 1993), House-Tree-Person (HTP; Van Hutton, 1993), WMS-R, and Beck Depression Inventory (BDI; Beck, 1987) were the 10 most frequently used instruments. Neuropsychologists indicated use of 17.6 different tests on average in their assessment practice, with 10% of respondents using over 30 different tests. The MMPI/MMPI-2, WAIS-R, WMS-R, Trail Making Test, COWAT, Finger Tapping Test (Reitan & Davison, 1974), HRNB, Boston Naming Test, Halstead Category Test, and WRAT-R/WRAT-3, were the 10 most frequently used instruments. Most recently, Sweet et al. (2002), surveyed members of APA Division 40 and NAN who reside in the United States regarding practitioner and practice characteristics, professional activities, and economic variables. Findings revealed that members of Division 40 and NAN were very similar with regard to employment setting and professional practices. Results also revealed a strong trend for younger, more recently licensed clinical neuropsychologists to be women. Though this study did not directly survey test usage practices, several findings related to assessment practices. Respondents reported devoting the greatest amount of their neuropsychological assessment time to test administration, followed by interpretation and report writing, test scoring, and clinical interview. Respondents endorsed private practice as the predominant employment setting, and private practitioners experienced more diverse

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

39

weekly clinical activities, were less likely to use assistants, and engaged in forensic activities more often than those who worked in institutional settings. The body of survey research in the field of clinical neuropsychology reveals several interesting trends. With respect to test usage, neuropsychologists have reported using the Wechsler Intelligence Scales more often than any other test or battery. This was true two decades ago and continues to be the case. Another finding was the increase in the number of clinicians who favored a flexible battery approach to test selection over completely flexible or standardized battery approaches. This trend may be related to the decline in popularity of certain instruments such as the HRNB and LNNB and the greater popularity of other tests, such as the WMS-R/Wechsler Memory Scale-Third Edition (WMS-III; Wechsler, 1997b), MMPI-2, WCST, Trail Making Test, and Boston Naming Test. There is also evidence in support of expanded roles and work settings for clinical neuropsychologists. While most clinical neuropsychologists reported working at medical centers and private practices, an increasing number were employed in rehabilitation facilities. Results also revealed great diversity in practitioners’ training and experience. Thus, the view of neuropsychologists as engaging only in assessment activities proves inaccurate; respondents reported ongoing involvement in research, teaching, consultation, and various forms of treatment delivery as well. While these surveys taken as a whole provide useful information about the field of clinical neuropsychology, they are limited in several ways. First, the data from most of the test usage studies were collected more than a decade ago, limiting present applicability. Second, none of the studies surveyed members from all three major neuropsychological organizations. Third, though investigators have focused on a broad range of topics, including practice characteristics and activities, education and training, and views on professional matters, basic test usage has received comparatively scant attention. Only two studies (Butler et al., 1991; Camara et al., 2000) provided detailed information about general test usage among neuropsychologists. Additionally, both of these studies employed a closed-ended format, in which participants were presented with a list of tests from which they endorsed those used in their practices. This format fails to capture the entire range of tests utilized by neuropsychologists. The current study addressed the above limitations through a contemporary, comprehensive survey of neuropsychologists’ assessment practices that investigated aspects of test usage not previously considered. This study employed an open-ended format, in which respondents reported their most frequently used assessment instruments as well as the instruments they would use to evaluate a specific patient’s memory, attention, and executive functioning. Respondents also reported any additional instruments they would use in predicting this patient’s ability to return to work. Through its exploration of test usage patterns, this study aimed to document and elucidate current trends in neuropsychological assessment and raise issues critical to the continued success of the field of clinical neuropsychology. 2. Method 2.1. Potential participants Potential participants were 2,004 randomly selected members of APA Division 40, NAN, and/or INS. In order to select the sample, the year 2000 membership directories of NAN, INS,

40

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

and APA Division 40 were combined and membership overlap eliminated. Only members in possession of doctoral degrees (i.e., Ph.D., Psy.D. or Ed.D.) and residing in the United States or Canada were selected for inclusion. The resulting population of 5,840 members likely encompassed most practitioners of neuropsychology in the United States and Canada. Random numbers were then assigned to each of the 5,840 cases, and 2,004 were drawn to form the sample, representing approximately one-third of the population. Questionnaires were mailed in January 2001; those returned within 4 months of the initial mailing were included in the analyses. Fifty-one surveys were returned undeliverable due to relocation, lack of proper forwarding address, or both reasons. 2.2. Questionnaire and procedure To assess test usage practices among clinical neuropsychologists, this study utilized a survey questionnaire1 divided into two main sections: demographic/practice-related information and a case study. In the first part of the questionnaire, respondents provided basic information including: (a) gender, (b) age, (c) degree type and field, (d) board certification status, (e) length of neuropsychological practice, (f) percentage of time devoted to various professional activities, (g) primary work settings, (h) referral sources, (i) battery approach, (j) assessment referral questions, (k) patient characteristics, (l) information gathering practices, and (m) most frequently used assessment instruments. In the second part of the questionnaire, respondents read a vignette about a traumatic brain injury (TBI) patient experiencing cognitive difficulties arising from a sports-related accident. Based on the information presented in the case study, respondents were asked to list the instruments they would use to evaluate the patient’s memory, attention, and executive functioning. Respondents also listed the instruments they would use to assess specific symptomatology in each of those cognitive domains. Subsequently, respondents reported additional instruments they would use to predict the patient’s ability to return to work. This question was included because research has indicated that assessing capacity to work is among the most frequent reasons for performing a neuropsychological evaluation (Guilmette et al., 1990). Despite the significance of this referral question to the field of neuropsychology, researchers previously have not surveyed neuropsychologists’ utilization of instruments for this purpose. All potential participants received a questionnaire packet consisting of an explanatory letter, questionnaire, stamped and self-addressed envelope, and two incentives (computer monitor wiper and book tabs, both with brain image). To enhance response rate, potential participants received a follow-up letter and a second incentive (bookmark with a neuron image) 4 weeks after the initial mailing. The follow-up mailing served as both a thank you to those who had already responded and a reminder to those who had not. After the questionnaire responses were recorded, respondents’ names were deleted from the mailing list and not linked to their responses. In addition, identifying information (e.g., comments written on letterhead) received 1

The data presented in this article were part of a larger study that examined a range of issues related to neuropsychological assessment. Aspects of the questionnaire not directly related to the test usage portion of the survey are not described in the current article. Additional data and analyses are being prepared for future publication.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

41

with the completed questionnaire was separated from the questionnaire in order to maintain anonymity of response.

3. Results 3.1. Response rate and organizational affiliations A total of 879 surveys were returned, representing a 44% response rate. However, 132 surveys were deemed unusable due to such factors as retirement, death, inactivity in the practice of clinical neuropsychology, or similarly being identified with APA Division 40, NAN, or INS on the basis of intellectual interest alone. Thus, the questionnaire was inappropriate for 15% of the sampled respondents who did not offer neuropsychological services. Subtracting these 132 from the original 2,004 questionnaires yielded an adjusted response rate of 40%, reflecting the opinions and practices of 747 clinical neuropsychologists. Study participants belonged to one of seven membership affiliation categories: members of APA Division 40 exclusively (Category A), members of NAN exclusively (Category B), members of INS exclusively (Category C), members of Division 40 and NAN (Category D), members of Division 40 and INS (Category E), members of NAN and INS (Category F), and members of Division 40, NAN, and INS (Category G). Table 1 presents information about participants’ organizational affiliations. Overall, the percentage of returned questionnaires per participant category was consistent with each category’s initial percentage breakdown. No pattern of bias in the characteristics of respondents versus nonrespondents was discerned. Notably, members of APA (Category A) and INS (Category C) returned the greatest proportion of unusable surveys (44 and 26%, respectively). 3.2. Questionnaire part I: demographic and practice-related information 3.2.1. Personal demographics and educational background The first group of survey results pertained to respondents’ personal demographics and educational backgrounds. The average age of the respondents was 48 (S.D. = 8.7, range 29–85 years). Table 2 presents information about respondents’ gender, educational background, and board certification status. Consistent with previous surveys of clinical neuropsychologists, women were noticeably underrepresented (40%) relative to male counterparts (60%). However, as noted by Sweet et al. (2002), the gap appears to have narrowed from the 3:1 and 2:1 male-to-female ratios previously reported (Guilmette et al., 1990; Putnam et al., 1994; Sweet et al., 1996, 2000). With respect to type of doctorate degree, the percentage of respondents holding Ph.D.s was highest (87%), followed by Psy.D.s (9%) and Ed.D.s (3%). The majority of respondents received their degrees in clinical psychology (62%), followed by counseling psychology (11%), clinical neuropsychology (11%), school psychology (5%), and neuroscience (2%). Ten percent of respondents received their degrees in areas of psychology not listed on the questionnaire, including: biological/physiological, experimental, developmental, health, educational,

42

A B C D E F G

Initial population (n = 5,840)

Potential participants (n = 2,004)

Returned surveys (n = 879)

Usable surveys (n = 747)

Unusable surveys (n = 132)

Membership affiliation categories

n

Percentage

n

Percentage

n

Percentage

n

Percentage

n

Percentage

Division 40 exclusively NAN exclusively INS exclusively Division 40 and NAN Division 40 and INS NAN and INS Division 40, NAN, and INS

1,604 851 704 840 627 313 901

27.5 14.6 12.1 14.4 10.7 5.4 15.4

526 302 222 308 218 111 317

26.2 15.1 11.1 15.4 10.9 5.5 15.8

196 132 97 141 95 59 159

22.3 15.0 11.0 16.0 10.8 6.7 18.1

138 117 63 131 86 57 155

18.5 15.7 8.4 17.5 11.5 7.6 20.7

58 15 34 10 9 2 4

43.9 11.4 25.8 7.6 6.8 1.5 3.0

Note. Table percentage values may not total to exactly 100% due to rounding errors; this applies to subsequent tables.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Table 1 Participants’ organizational affiliations

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

43

Table 2 Personal demographics and professional background Variable

n

Percentage of respondents

Gender (all questionnaires) Potential participants Male Female Total

1,173 831 2,004

58.5 41.5 100.0

Returned questionnaires Male Female Total

513 365 878

58.4 41.6 100.0

Usable questionnaires Male Female Total

446 301 747

59.7 40.3 100.0

653 70 24 747

87.4 9.4 3.2 100.0

464 80 37 84 11 71 747

62.1 10.7 5.0 11.2 1.5 9.5 100.0

123 41 164/747

16.3 5.5 21.8

Degree (usable questionnaires) Highest degree earned Ph.D. Psy.D. Ed.D. Total Field of highest degree earned Clinical psychology Clinical neuropsychology School psychology Counseling psychology Neuroscience Other Total Board certification (usable questionnaires) ABPP certified ABPN certified Total

cognitive/perceptual, rehabilitation, social/personality, and child. Only 22% of respondents reported being board certified in neuropsychology by ABPP or ABPN, and the percentage of female respondents (25 and 15%, respectively) was considerably lower than male respondents (75 and 85%, respectively). Other researchers have also reported this finding (e.g., Sweet and Moberg, 1990; Sweet et al., 1996, 2000), and the gender disparity among board certified neuropsychologists does not appear to be diminishing. As shown in Table 3, this gender disparity is not unique to clinical neuropsychology; it is also found among all other ABPP membership boards (American Board of Professional Psychology, 2001).

44

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Table 3 Percentage breakdown of ABPP specialties by gender Gender of members Specialties

Male (%)

Female (%)

Clinical neuropsychology Behavioral psychology Clinical psychology Counseling psychology Family psychology Forensic psychology Group psychology Health psychology Industrial/organizational psychology Psychoanalysis psychology School psychology Rehabilitation psychology

75 82 79 81 79 85 66 77 90 63 69 82

25 18 21 19 21 15 34 23 10 37 31 18

Mean

77

23

Note. Information for this table was obtained from the American Board of Professional Psychology (2001).

3.2.2. Practice demographics Additional survey questions pertained to general practice issues. Table 4 presents the percentage of time devoted to various professional activities. The overwhelming majority of respondents (96%) indicated that they were currently performing neuropsychological assessments; the average respondent spent 42% of his or her professional time conducting neuropsychological assessments. This was followed by psychotherapy, research/teaching, and rehabilitation/cognitive remediation. Respondents also reported engaging in a variety of other activities not listed on the questionnaire, including: administration/program development, psychological assessment, consultation, forensic assessment, supervision, educational evaluation, writing/editing, and disability evaluation. As shown in Table 5, respondents’ top five referral sources were neurologists, psychiatrists, psychologists, attorneys, and general medical practitioners. Notably, 40% of respondents reported additional referral sources not listed on the questionnaire. The most frequently listed “other” referral sources were: educators/school personnel, workers’ compensation specialists, Table 4 Percentage of time devoted to professional activities Activity

n

Percentage of respondents

Mean percentage of time

Neuropsychological assessment Psychotherapy Research and/or teaching Other Rehabilitation and/or cognitive remediation

709 469 454 309 284

95.7 63.3 61.3 41.7 38.3

42.0 19.2 16.7 13.9 8.0

Note. Percentage values do not total to 100% because many respondents endorsed more than one professional activity; this applies to subsequent tables.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

45

Table 5 Distribution of respondents’ referral sources Referral source

Percentage of respondents

Neurologists Psychiatrists Psychologists Attorneys General medical practitioners Physiatrists Other Neurosurgeons Social workers Pediatricians Vocational counselors Occupational therapists Cardiologists Psychiatric nurses

79.3 73.1 68.3 68.3 64.2 41.3 39.7 38.9 33.8 33.1 28.7 11.4 11.0 8.7

insurance companies, physician specialists not listed on the questionnaire, case managers, and speech and language pathologists. Tables 6 and 7 present information about patient characteristics. Respondents reported spending the greatest percentage of professional time with adults, followed by young adults and older adults. Respondents reported spending less than a third of their professional time serving child and adolescent populations, and approximately half of respondents spend no professional time with children. Respondents served a diversity of diagnostic groups, most frequently patients with depression or mild head trauma. In addition to those listed on the questionnaire, respondents worked with numerous other diagnostic groups, most frequently patients with autism and pervasive developmental disorders and Parkinson’s disease. 3.2.3. Neuropsychological activities The remainder of survey items dealt specifically with professional neuropsychological activities. Respondents reported professionally practicing neuropsychology for an average of 13.2 years (S.D. = 7.7, range 1–50 years). The majority of respondents (74%) performed 1–15 neuropsychological assessments per month, with an additional 16% performing 16–30 assessments per month. Respondents used an average of 12 different tests in a typical neuropsychological assessment battery, with approximately 10% using over 20 different tests. However, Table 6 Professional time spent with specific age groups Age group

Mean percentage of time

Percentage of respondents

Children (age < 12) Adolescents (ages 12–18) Young adults (ages 19–39) Adults (ages 40–65) Older adults (age > 65)

14.6 12.5 23.5 28.7 20.2

53.8 71.3 85.9 86.8 75.8

46

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Table 7 Frequency of clinical work with specific diagnostic groups Percentage of respondents Diagnostic group

Never (1)

Rarely (2)

Occasionally (3)

Frequently (4)

Mean rank

Depression Mild head trauma Learning disabilities Dementia Anxiety disorders ADHD Stroke/vascular Seizure disorders Severe head trauma Personality disorders Bipolar disorder Substance abuse Pain CNS tumor Toxic/metabolic Multiple sclerosis Mental retardation Schizophrenia AIDS

1.5 1.4 3.1 9.2 3.3 6.8 7.1 2.6 7.8 7.3 5.6 9.5 13.8 7.7 9.8 13.4 14.0 19.0 26.5

4.1 8.3 12.7 12.8 14.0 16.7 16.1 20.6 23.1 26.6 27.5 24.9 26.1 32.6 41.0 34.1 38.7 47.6 48.3

24.9 30.7 33.6 27.0 44.2 30.1 36.8 47.6 32.4 38.0 46.7 40.5 33.5 44.0 38.1 41.2 30.6 22.1 20.4

69.5 59.6 50.5 51.0 38.5 46.5 40.0 29.2 36.7 28.1 20.2 25.1 26.6 15.7 11.2 11.3 16.7 11.3 4.9

3.63 3.49 3.32 3.20 3.18 3.16 3.10 3.03 2.98 2.87 2.82 2.81 2.73 2.68 2.51 2.50 2.50 2.26 2.04

Note. Ratings based on a 4-point scale: 1 = never, 2 = rarely, 3 = occasionally, and 4 = frequently. A higher mean rank represents a more commonly assessed patient group.

this finding may represent an underestimate of the number of tests utilized since many instruments (e.g., WMS-III) comprise a variety of individual tests or subtests. Twenty-five percent of respondents reported spending over 20 h per week on neuropsychological assessment, with another 30% spending 10–20 h per week providing this service. The remaining 45% of respondents devoted 10 h per week or less to neuropsychological assessment. Table 8 presents the settings in which respondents performed their neuropsychological work. The majority of respondents (62%) identified private or group practice, with medical Table 8 Distribution of respondents’ work settings Work setting

n

Percentage of respondents

Private or group practice Medical hospital Rehabilitation facility Other (explained in text) Psychiatric hospital VA hospital College/university counseling center Community mental health center Business/industry

462 256 129 110 53 37 31 22 7

62.3 34.4 17.3 14.8 7.1 5.0 4.2 3.0 1.0

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

47

Table 9 Frequent assessment referral questions Referral question

n

Percentage of respondents

Determination of diagnosis Rehabilitation/treatment planning Forensic determination Educational planning Assess capacity to work Establish baseline of function for subsequent testing Assess capacity for independent living Pre- and post-medical intervention Other Localization of lesion

524 353 236 219 207 180 146 71 32 20

70.7 48.3 31.8 29.6 27.9 24.3 19.7 9.6 4.3 2.7

hospitals (34%) and rehabilitation facilities (17%) proving fairly common as well. Thirty-six percent of respondents worked in more than one setting. In addition to the locales listed on the questionnaire, 15% of respondents were employed in “other” settings, the most frequently mentioned of which were schools, correctional facilities, university research centers, university medical centers/clinics, and nursing homes. Of respondents who worked in medical, psychiatric, or VA hospitals, the most frequently specified departments were psychiatry, psychology/neuropsychology, neurology, and physical medicine and rehabilitation. As shown in Table 9, the most frequently endorsed assessment referral questions were determination of diagnosis, rehabilitation and/or treatment planning, and forensic determination. Table 10 presents the frequency with which respondents evaluated specific abilities during neuropsychological assessments. Overall, there was great consensus regarding the Table 10 Assessment of specific abilities during neuropsychological evaluations Percentage of respondents Abilities

Never (1)

Rarely (2)

Occasionally (3)

Frequently (4)

Mean rank

Attention Verbal memory Executive functions Visuospatial skills Nonverbal memory Intelligence Language Construction Motor skills Achievement Auditory perception Tactile perception

0.1 0.1 0.1 0.1 0.3 0.5 0.4 0.5 0.4 2.6 4.8 5.6

0.1 0.0 0.5 0.4 1.8 1.9 2.7 4.8 4.8 10.9 20.7 34.6

2.0 3.8 3.0 4.9 10.2 8.2 13.4 11.0 26.9 31.6 31.1 32.7

97.7 96.1 96.3 94.6 87.8 89.4 83.5 83.7 67.9 54.9 43.5 27.2

3.97 3.96 3.95 3.94 3.86 3.86 3.80 3.78 3.62 3.39 3.13 2.81

Note. Ratings based on a 4-point scale: 1 = never, 2 = rarely, 3 = occasionally, and 4 = frequently. A higher mean rank represents a more frequently assessed ability.

48

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

types of abilities evaluated. For example, attention, construction, executive functions, intelligence, language, motor skills, verbal and nonverbal memory, and visuospatial skills were endorsed as “frequently” assessed, while auditory and tactile perception reportedly were assessed “occasionally.” Consistent with the findings of other recent surveys (e.g., Sweet et al., 2000), the vast majority of respondents (68%) favored a flexible battery approach to test selection (i.e., variable but routine groupings of tests for different types of patients), with an additional 20% favoring a flexible approach (i.e., based upon the needs of an individual case), and 11% favoring a standardized battery (i.e., HRNB, LNNB). Respondents also reported the frequency with which they utilized certain types of information in their assessments. As shown in Table 11, the top five sources of information included medical/psychiatric history, neuropsychological test data, the referral source, psychosocial history, and measures of mood and affect. Information gained from work records and projective personality testing was the least used. The final question in part I of the survey asked respondents to list their three most frequently used assessment instruments. Table 12 presents a rank-ordered list of respondents’ top 40 instruments; a complete list of responses is available upon request. The WAIS-R/Wechsler Adult Intelligence Scale-Third Edition (WAIS-III; Wechsler, 1997a) were the most frequently used instruments, followed by the WMS-R/WMS-III, Trail Making Test, California Verbal Learning Test-Second Edition (CVLT/CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000; Delis, Kramer, Kaplan, Ober, & Fridlund, 1987), and the WISC-III. In general, there was great overlap between the most frequently cited instruments in the current study and those cited in previous neuropsychological assessment surveys (e.g., Camara et al., 2000). Table 11 Type of information used in neuropsychological assessments Percentage of respondents Information

Never (1)

Rarely (2)

Occasionally (3)

Frequently (4)

Mean rank

Medical/psychiatric history Neuropsychological test data Referral source Psychosocial history Mood and affect measures Developmental history Current social supports Objective personality tests Mental status exam Significant other interview Environmental demand characteristics Behavioral assessments School records Functional assessments Work records Projective personality tests

0.0 0.0 0.1 0.0 0.4 0.3 0.4 1.9 3.0 2.1 3.0 2.1 2.4 3.3 9.5 28.0

0.5 1.9 0.7 1.2 2.5 5.3 7.2 9.4 12.6 7.1 12.3 15.0 17.0 19.0 36.6 34.6

4.6 4.8 8.1 8.9 16.7 14.1 21.0 27.6 18.5 34.9 33.3 33.9 40.4 40.7 39.4 24.5

94.9 93.3 91.1 89.9 80.4 80.3 71.4 61.1 65.9 55.9 51.4 49.0 40.2 37.0 14.5 12.9

3.94 3.91 3.90 3.89 3.77 3.74 3.63 3.48 3.47 3.45 3.33 3.30 3.18 3.11 2.59 2.22

Note. Ratings based on a 4-point scale: 1 = never, 2 = rarely, 3 = occasionally, and 4 = frequently. A higher mean rank represents a more frequently used information source.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

49

Table 12 Top 40 neuropsychological assessment instruments Rank

Instrument

n

Percentage of responses

Percentage of respondents

1 2 3 4 5 6 7 8 9 10 11 11 13 13 15 16 16 18 19 20 20 22 23 23 25 25 27 27 29 30 30 32 32 34 35 35 35 35 35 40 40 40 40

WAIS-R/WAIS-III WMS-R/WMS-III Trail Making Test CVLT/CVLT-II WISC-III HRNB WCST ROCFT MMPI/MMPI-2 RAVLT NEPSY WJ-R/WJ-III Boston Naming Test WRAML DRS Halstead Category Test LNNB WRAT/WRAT-R/WRAT-3 COWAT Cognistat/NCSE WIAT/WIAT-II WASI Children’s Memory Scale RBANS Bender VMI CPT/CPT-II TOVA MMSE MAS Stroop Test HVLT/HVLT-R WMS-R/WMS-III Logical Memory Rorschach Aphasia Screening Test BVRT/BVRT-R CERAD Clock Drawing Test PASAT Booklet Category Test Gordon CPT SDMT WAIS-R NPI/WAIS-III NPI

460 311 128 126 116 113 84 76 43 40 32 32 29 29 25 24 24 23 19 18 17 16 15 15 13 13 12 12 11 10 10 9 9 8 7 7 7 7 7 6 6 6 6

21.4 14.5 6.0 5.9 5.4 5.3 3.9 3.5 2.0 1.9 1.5 1.5 1.3 1.3 1.2 1.1 1.1 1.1 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3

63.1 42.7 17.6 17.3 15.9 15.5 11.5 10.4 5.9 5.5 4.4 4.4 4.0 4.0 3.4 3.3 3.3 3.2 2.6 2.3 2.3 2.2 2.1 2.1 1.8 1.8 1.6 1.6 1.5 1.4 1.4 1.2 1.2 1.1 1.0 1.0 1.0 1.0 1.0 0.8 0.8 0.8 0.8

Note. Refer to appendix for meaning of abbreviations; a complete list of responses is available upon request.

50

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

3.3. Questionnaire part II: case study In the second part of the survey, respondents read a brief history regarding a TBI patient (“PB”) who was experiencing cognitive difficulties resultant from a sports-related injury. Respondents then answered questions based on this information. Two survey questions asked respondents to list the instruments they would administer to assess PB’s specific memory complaints (i.e., problems recalling people’s names and recent events) and general memory functioning. Table 13 presents a rank-ordered list of the top 40 instruments mentioned by respondents in response to these two questions; a complete list of responses is available upon request. In the area of memory assessment, the WMS-R/WMS-III, CVLT/CVLT-II, ROCFT, Boston Naming Test, and WAIS-R/WAIS-III were the most frequently reported instruments. Two additional survey questions asked respondents to list the instruments they would administer to assess PB’s specific attention complaints (i.e., problems with concentrating and maintaining train of thought) and general attentional functioning. Table 14 presents a rank-ordered list of the top 40 instruments mentioned by respondents; a complete list of responses is available upon request. In the area of attention, Trail Making Test, Digit Span (Wechsler 1997a,b), Paced Auditory Serial Addition Test (PASAT; Gronwall, 1977), Stroop Test (Jensen & Rohwer, 1966), and Continuous Performance Tests (CPT/CPT-II; Conners & Multi-Health Systems Staff, 1995) were the most frequently reported instruments. Two additional survey questions asked respondents to list the instruments they would administer to assess PB’s specific executive function complaints (i.e., problems with organization and planning) and general executive functioning. Table 15 presents a rank-ordered list of the top 40 reported instruments; a complete list of responses is available upon request. In the area of executive functioning, the WCST, ROCFT, Halstead Category Test, Trail Making Test, and COWAT were the most frequently reported instruments. A final survey question asked respondents to list any additional instruments that they would use to predict PB’s ability to return to work. Table 16 presents a rank-ordered list of the top 40 instruments mentioned by respondents; a complete list of responses is available upon request. In the area of return to work, the MMPI/MMPI-2, WAIS-R/WAIS-III, driving evaluation, BDI/BDI-II, and a clinical interview were the most frequently reported instruments.

4. Discussion 4.1. Review of study findings The present study updates and greatly expands knowledge about neuropsychologists’ assessment practices and test usage patterns. Assessment has defined the practice of clinical neuropsychology since the specialty’s inception. While research has surveyed a broad range of issues related to clinical neuropsychology, basic test usage has received relatively little attention. Results of this study extend previous test usage research in several ways. First, this is the only study to survey members from all three major neuropsychological organizations (i.e., APA Division 40, NAN, and INS). The aim was to include as many doctorate-level individuals involved with neuropsychological assessment as possible. Second, the usable

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

51

Table 13 Top 40 memory assessment instruments Rank

Instrument

n

Percentage of responses

Percentage of respondents

1 2 3 4 5 6 7 8 9 10 11 12 13 14 14 16 16 18 19 19 19 22 23 24 25 26 26 26 29 30 31 31 31 34 35 35 37 38 39 40

WMS-R/WMS-III CVLT/CVLT-II ROCFT Boston Naming Test WAIS-R/WAIS-III COWAT RAVLT WMS-R/WMS-III Logical Memory WAIS-III/WMS-III Digit Span WRMT Clinical Interview MMPI/MMPI-2 BSRT Tactual Performance Test WCST Trail Making Test WMS-R/WMS-III Visual Reproduction BVRT/BVRT-R MAS RBMT TOMM MSE Interview with Collaterals WMS-R/WMS-III Verbal Paired Associates PASAT Facial Recognition Test Halstead Category Test HRNB WMS-III Faces Subtest Semantic Fluency/Animal Naming Aphasia Screening Test BDI/BDI-II WAIS-R/WAIS-III Information CPT/CPT-II Stroop Test WMS-R Visual Paired Associates CVMT Sentence Repetition Test WJ-R/WJ-III Rey 15 Item Memory Test

488 374 312 231 173 150 147 140 76 72 68 65 53 51 51 50 50 46 44 44 44 37 34 33 32 30 30 30 29 28 27 27 27 26 24 24 23 22 19 18

12.1 9.3 7.7 5.7 4.3 3.7 3.6 3.5 1.9 1.8 1.7 1.6 1.3 1.3 1.3 1.2 1.2 1.1 1.1 1.1 1.1 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.4

70.8 54.3 45.3 33.5 25.1 21.8 21.3 20.3 11.0 10.4 9.9 9.4 7.7 7.4 7.4 7.3 7.3 6.7 6.4 6.4 6.4 5.4 4.9 4.8 4.6 4.4 4.4 4.4 4.2 4.1 3.9 3.9 3.9 3.8 3.5 3.5 3.3 3.2 2.8 2.6

Note. Refer to appendix for meaning of abbreviations; a complete list of responses is available upon request.

sample of 747 respondents represents the largest test usage survey conducted to date within the field of clinical neuropsychology. Third, the current study examined aspects of test usage never before considered. Previous studies have provided basic information about neuropsychological instruments, including

52

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Table 14 Top 40 attention assessment instruments Rank

Instrument

n

Percentage of responses

Percentage of respondents

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 19 21 22 23 24 25 26 26 28 29 29 31 32 33 34 35 35 35 35 35 40 40

Trail Making Test WAIS-III/WMS-III Digit Span PASAT Stroop Test CPT/CPT-II WCST WAIS-R/WAIS-III Arithmetic WAIS-III/WMS-III Letter-Number Sequencing WAIS/WAIS-R/WAIS-III Seashore Rhythm Test WAIS-R/WAIS-III Digit Symbol-Coding WMS-R/WMS-III Speech Sounds Perception Test WMS-III Spatial Span SDMT COWAT Cancellation Test WMS-R/WMS-III Mental Control Halstead Category Test TOVA Two and Seven Cancellation Test Digit Vigilance Test Clinical Interview VSAT Brief Test of Attention ACT CCC WAIS-III Symbol Search CVLT/CVLT-II IVA d2 VIGIL Gordon CPT Sentence Repetition Test Functional Assessment HRNB MSE TEA WAIS-R/WAIS-III Picture Completion MMPI/MMPI-2 Serial 7s/Serial Digits

421 318 288 208 204 189 140 139 113 111 108 97 92 91 60 59 58 52 50 50 46 44 34 30 28 27 27 26 24 24 19 18 17 16 14 14 14 14 14 12 12

11.2 8.4 7.6 5.5 5.4 5.0 3.7 3.7 3.0 2.9 2.9 2.6 2.4 2.4 1.6 1.6 1.5 1.4 1.3 1.3 1.2 1.2 0.9 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3

63.7 48.1 43.6 31.5 30.9 28.6 21.2 21.0 17.1 16.8 16.3 14.7 13.9 13.8 9.1 8.9 8.8 7.9 7.6 7.6 7.0 6.7 5.1 4.5 4.2 4.1 4.1 3.9 3.6 3.6 2.9 2.7 2.6 2.4 2.1 2.1 2.1 2.1 2.3 1.8 1.8

Note. Refer to appendix for meaning of abbreviations; complete list of responses is available upon request.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

53

Table 15 Top 40 executive functioning assessment instruments Rank

Instrument

n

Percentage of responses

Percentage of respondents

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 17 19 20 21 22 22 24 25 26 27 28 28 28 31 32 32 34 34 36 36 36 39 40

WCST ROCFT Halstead Category Test Trail Making Test COWAT WAIS-R/WAIS-III/WASI Block Design WAIS-R/WAIS-III Stroop Test WAIS-R/WAIS-III Picture Arrangement Porteus Maze Test Tactual Performance Test WAIS-R/WAIS-III Similarities Booklet Category Test RFFT Tower of London Test WAIS-R/WAIS-III Comprehension CVLT/CVLT-II WAIS-III/WASI Matrix Reasoning Clock Drawing Test WAIS-R/WAIS-III Object Assembly Tower of Hanoi Test Bender Raven’s Progressive Matrices Clinical Interview Tinkertoy Test Frontal Motor Series SB Categories Hooper VOT Interview with Collaterals Semantic Fluency/Animal Naming WISC-III Mazes WMS-R/WMS-III Functional Assessment Design Fluency Test PASAT D-KEFS Go-No-Go Test HRNB WAIS-R/WAIS-III Picture Completion BADS

494 268 262 260 154 114 111 110 78 76 62 53 49 48 45 39 38 38 37 34 32 30 30 29 28 26 25 23 23 23 21 19 19 18 18 16 16 16 15 13

15.1 8.2 8.0 7.9 4.7 3.5 3.4 3.4 2.4 2.3 1.9 1.6 1.5 1.5 1.4 1.2 1.2 1.2 1.1 1.0 1.0 0.9 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4

75.5 41.0 40.1 39.8 23.5 22.8 17.0 16.8 11.9 11.6 9.5 8.1 7.5 7.3 6.9 6.0 5.8 5.8 5.7 5.2 4.9 4.6 4.6 4.4 4.3 4.0 3.8 3.5 3.5 3.5 3.2 2.9 2.9 2.8 2.8 2.5 2.5 2.5 2.3 2.0

Note. Refer to appendix for meaning of abbreviations; a complete list of responses is available upon request.

54

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Table 16 Top 40 “return to work” assessment instruments Rank

Instrument

n

Percentage of responses

Percentage of respondents

1 2 3 4 5 6 7 8 8 10 11 11 13 13 15 16 17 17 19 19 21 21 21 21 25 26 27 28 28 30 30 30 30 34 34 36 36 36 36 40 40

MMPI/MMPI-2 WAIS-R/WAIS-III Driving Evaluation BDI/BDI-II Clinical Interview Finger Tapping Test Grooved Pegboard Test Functional Assessment Interview with Collaterals Structured Work Trial JLO PAI Grip Strength Rorschach MCMI-I/MCMI-II WRAT/WRAT-R/WRAT-3 Purdue Pegboard Test Review Work Record and Job Responsibilities Boston Naming Test SCL-90/SCL-90-R HRNB HRNB Sensory Perceptual Exam Tactual Performance Test Beck Anxiety Inventory Reaction Time Test Assessment of Job Demands Hooper VOT WMS-R/WMS-III WAIS-R/WAIS-III/WASI Block Design COWAT Thematic Apperception Test WAIS-R/WAIS-III Comprehension WCST Interview with Coworkers WAIS-R/WAIS-III Digit Symbol-Coding Clinical Interview with Functional Questions Reitan–Klove Sensory Perceptual Test ROCFT Trail Making Test BDAE Facial Recognition Test

213 85 72 67 64 55 47 43 43 37 29 29 27 27 24 22 19 19 18 18 17 17 17 17 16 15 13 12 12 11 11 11 11 10 10 9 9 9 9 9 9

13.2 5.3 4.5 4.1 4.0 3.4 2.9 2.7 2.7 2.3 1.8 1.8 1.7 1.7 1.5 1.4 1.2 1.2 1.1 1.1 1.1 1.1 1.1 1.1 1.0 0.9 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6

39.9 15.9 13.5 12.5 12.0 10.3 8.8 8.1 8.1 6.9 5.4 5.4 5.1 5.1 4.5 4.1 3.6 3.6 3.4 3.4 3.2 3.2 3.2 3.2 3.0 2.8 2.4 2.2 2.2 2.1 2.1 2.1 2.1 1.9 1.9 1.7 1.7 1.7 1.7 1.7 1.7

Note. Refer to appendix for meaning of abbreviations; a complete list of responses is available upon request.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

55

utilization rates and time required to administer, score, and interpret frequently used tests. The current study aimed to move beyond investigations of only the most commonly used instruments. In addition to reporting their three most frequently used assessment instruments, respondents listed the instruments they would use to assess specific symptoms and general functioning in the areas of memory, attention, and executive functioning. Respondents also listed any additional instruments they would use to help predict the ability to return to work. Respondents were 747 doctoral-level psychologists with membership affiliations in APA Division 40, NAN, and/or INS. The average respondent was 48 years old, held a Ph.D. in clinical psychology, and had been practicing neuropsychology for 13 years. Consistent with Sweet et al.’s most recent survey (2002), the representation of women in the current study exceeded previous findings (e.g., Sweet et al., 1996, 2000), suggesting a growing proportion of female neuropsychologists. Despite these gains, however, the observed percentage of women in neuropsychology (40%) still lags behind the percentage of women doctorates in the whole field of psychology (64%; National Science Foundation, 1999). Respondents were actively involved in neuropsychological assessment, as well as various other professional endeavors. Nearly all respondents regularly performed neuropsychological assessments; over half devoted the greatest percentage of their professional time to this activity. The majority of respondents conducted between 1 and 15 neuropsychological assessments per month, using an average of 12 instruments per battery. In terms of work setting, 62% of respondents performed their neuropsychological work in private or group practices. Other researchers have noted a matching trend toward increased employment in the private sector, perhaps in response to fewer opportunities in medical settings (Sweet et al., 2000, 2002). Respondents received referrals from a variety of medical and nonmedical professionals and served diverse diagnostic groups. Relatively few respondents spent substantial professional time with children and adolescents, with approximately half spending no professional time with children under the age of 12. Results of this study confirmed Sweet et al.’s (2000) finding of a steady decline in the use of both standardized batteries (e.g., HRNB and LNNB) and completely flexible approaches to test selection. These declines are likely related, in part, to changing reimbursement sources and policies. For example, the rise of managed care adversely impacts the time allotted for neuropsychological evaluations as well as the procedures allowed. Neuropsychologists currently utilize abbreviated, more focused instruments. The increased popularity of the flexible battery also may be the result of neuropsychologists’ attempts to standardize assessment procedures within diagnostic groups to ready clinical data for research purposes. With regard to test usage practices, this study and previous test usage surveys together reveal that neuropsychologists utilize a multiplicity of assessment procedures, varied in form (e.g., standardized and nonstandardized instruments, clinical interviews, etc.), that can address a full range of age groups. In addition, neuropsychologists assess a variety of domains (e.g., intelligence, language, memory, personality). Consistent with the results of test usage surveys from the early 1990s onward, instruments such as the WAIS-R/WAIS-III, WMS-R/WMS-III, and Trail Making Test were among those most frequently cited by respondents. In addition, several other instruments, such as the WCST, ROCFT, CVLT/CVLT-II, COWAT, Halstead Category Test, Stroop, and Boston Naming Test, also received frequent mention in response to this survey (see Table 17 for a list of the 25 most frequently reported instruments).

56

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Table 17 Top 25 neuropsychological assessment instruments Rank

Instrument

n

Percentage of responses

Percentage of respondents

1 2 3 4 5 6 7 8 9 10 11 11 13 13 15 16 16 18 19 20 20 22 23 23 25 25

WAIS-R/WAIS-III WMS-R/WMS-III Trail Making Test CVLT/CVLT-II WISC-III HRNB WCST ROCFT MMPI/MMPI-2 RAVLT NEPSY WJ-R/WJ-III Boston Naming Test WRAML DRS Halstead Category Test LNNB WRAT/WRAT-R/WRAT-3 COWAT Cognistat/NCSE WIAT/WIAT-II WASI Children’s Memory Scale RBANS Bender VMI

460 311 128 126 116 113 84 76 43 40 32 32 29 29 25 24 24 23 19 18 17 16 15 15 13 13

21.4 14.5 6.0 5.9 5.4 5.3 3.9 3.5 2.0 1.9 1.5 1.5 1.3 1.3 1.2 1.1 1.1 1.1 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6

63.1 42.7 17.6 17.3 15.9 15.5 11.5 10.4 5.9 5.5 4.4 4.4 4.0 4.0 3.4 3.3 3.3 3.2 2.6 2.3 2.3 2.2 2.1 2.1 1.8 1.8

Note. Refer to appendix for meaning of abbreviations; a complete list of responses is available upon request.

Close examination of the test usage data reveals several interesting details. First, neuropsychologists use a variety of instruments to assess patients’ functioning. In the areas of memory, attention, and executive functioning, respondents reported respective use of 273, 220, and 219 instruments, in total (Table 18 presents the top 10 instruments in each of these domains). Of these 712 instruments, 101 were unique to the domain of memory, 45 unique to attention, and 56 unique to executive functioning. In predicting return to work, respondents listed a total of 249 instruments, 80 of which were unique to this area; a complete list of these unique instruments is available upon request. These data reveal both extensive overlap and exclusivity with respect to the selection and utilization of instruments within specific functional domains. Another interesting finding is that the majority of respondents appeared to use the same small group of assessment instruments. Thus, while respondents reported using 273 different memory assessment instruments, the top three instruments (i.e., full WMS-R/WMSIII, CVLT/CVLT-II, and ROCFT) saw use by 71, 54, and 45% of respondents, respectively. Similar trends were found for the instruments used to assess attention and executive functioning and to predict return to work (refer to Tables 14–16).

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

57

Table 18 Top 10 memory, attention, and executive functioning instruments Rank

Instrument

n

Percentage of responses

Percentage of respondents

Memory 1 2 3 4 5 6 7 8 9 10

WMS-R/WMS-III CVLT/CVLT-II ROCFT Boston Naming Test WAIS-R/WAIS-III COWAT RAVLT WMS-R/WMS-III Logical Memory WAIS-III/WMS-III Digit Span WRMT

488 374 312 231 173 150 147 140 76 72

12.1 9.3 7.7 5.7 4.3 3.7 3.6 3.5 1.9 1.8

70.8 54.3 45.3 33.5 25.1 21.8 21.3 20.3 11.0 10.4

Attention 1 2 3 4 5 6 7 8 9 10

Trail Making Test WAIS-III/WMS-III Digit Span PASAT Stroop Test CPT/CPT-II WCST WAIS-R/WAIS-III Arithmetic WAIS-III/WMS-III Letter-Number Sequencing WAIS/WAIS-R/WAIS-III Seashore Rhythm Test

421 318 288 208 204 189 140 139 113 111

11.2 8.4 7.6 5.5 5.4 5.0 3.7 3.7 3.0 2.9

63.7 48.1 43.6 31.5 30.9 28.6 21.2 21.0 17.1 16.8

494 268 262 260 154 114 111 110 78 76

15.1 8.2 8.0 7.9 4.7 3.5 3.4 3.4 2.4 2.3

75.5 41.0 40.1 39.8 23.5 22.8 17.0 16.8 11.9 11.6

Executive functioning 1 WCST 2 ROCFT 3 Halstead Category Test 4 Trail Making Test 5 COWAT 6 WAIS-R/WAIS-III/WASI Block Design 7 WAIS-R/WAIS-III 8 Stroop Test 9 WAIS-R/WAIS-III Picture Arrangement 10 Porteus Maze Test Note. Refer to appendix for meaning of abbreviations.

Several factors may account for respondents’ tendency to use the same group of instruments despite the proliferation of assessment techniques in the field of neuropsychology. First, neuropsychologists actively may choose to employ instruments based on psychometric considerations; the observed subset of highly used instruments may actually possess the strongest psychometric properties (e.g., reliability, validity, norms, research base). Another possibility is that neuropsychologists simply prefer to use the instruments with which they were trained during graduate school, internship, or postdoctoral fellowship. This small group of highly utilized instruments (e.g., WMS-R/WMS-III, WCST, Trail Making Test) may have earned popularity, in part, by virtue of its long and rich history as a set of training tools. In

58

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

addition to being used in training and practice, these instruments also tend to be among those addressed most frequently and thoroughly in assessment texts, journals, and courses. Consequently, when designing assessment batteries, they are among the instruments that first come to mind for many neuropsychologists. 4.2. Study limitations It is important when considering the implications of this project to also consider its limitations. In the current study, the overall response rate was 44%. While this response rate falls within the range of 40–50% acceptable to behavioral research (Kerlinger, 1973), a higher return rate would have been desirable. While potential respondents in the current study received a follow-up mailing, further contact may have enhanced the response rate. In addition, only neuropsychologists who hold memberships in Division 40, NAN, or INS were sampled for inclusion in this study. This sample frame was utilized because the likely coverage errors were acceptable compared with the possible sampling alternatives. Nevertheless, this study likely overlooked a number of individuals who practice neuropsychology in the United States and Canada. A related problem involved respondents’ range of professional experience. In an effort to ensure representation of a high level of training in the study sample, only individuals possessing doctorate-level degrees qualified for participation. An unforeseen consequence of this requirement was the exclusion of many individuals who had recently completed doctoral training. These individuals were likely overlooked because membership affiliations did not yet reflect their change in professional status. The average study participant was 48 years old and professionally had practiced neuropsychology for 13 years. Future studies should attempt to survey a broader range of neuropsychologists. In doing so, researchers may better capture the recent shift in gender representation reported by Sweet et al. (2002). Finally, the current study utilized a questionnaire designed by the principal investigator. When completing the questionnaire, respondents were repeatedly asked to list the “instruments” they would use to assess or predict a patient’s functioning and behavior. In future studies, the word “instrument” should be replaced with, or supplemented by, broader, more inclusive words like “technique” or “method.” Several respondents found the word “instrument” inappropriate and limiting because it excludes such techniques as clinical interviews, direct observations, etc. Certainly, it was not the investigator’s intention to limit respondents’ test usage reporting in any way. Unfortunately, some respondents misinterpreted the meaning of the word instrument, thereby altering the test data reported in this study. 4.3. Implications for the practice of neuropsychology and directions for future research Several findings shed light on the field of clinical neuropsychology and its practitioners. First, the test usage data provides a comprehensive account of the range of tests utilized by neuropsychologists. In addition to illuminating the nature of neuropsychological activities, this information can guide clinical practice. Practitioners can compare their assessment instruments with those reported by respondents in this survey. They might discover regular

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

59

employment of an instrument utilized by few others, leading to an examination of the validity or appropriateness of that instrument. Second, the current data may generate important questions about the battery construction process. For example, do neuropsychologists modify test batteries based on specific patient characteristics such as occupational status? This question is particularly relevant given that assessing capacity to work is among neuropsychologists’ most common assessment referral questions. Indeed, some researchers (e.g., Guilmette & Kastner, 1996) have suggested that neuropsychologists should consider the specific cognitive demands of patients’ jobs when designing test batteries to assess and predict aspects of vocational functioning. Future research could address this important issue by surveying opinions about the utility and predictive power of specific neuropsychological techniques across members of varied occupational groups. More broadly, researchers could explore the varied factors neuropsychologists consider when they incorporate specific instruments into their assessment batteries. Third, study results raise questions about the ecological validity of neuropsychological assessments. Findings revealed that 70% of respondents endorsed at least one assessment referral question that focused on rehabilitation or real-world outcomes (i.e., rehabilitation/treatment planning, assessment of work capacity, assessment of independent living capacity). Indeed, a recent trend in clinical neuropsychology has been a withdrawal from purely descriptive, diagnostic endeavors toward prescriptive, treatment-oriented assessments to bridge the gap between diagnosis and rehabilitation (Boyd, 1988; Cubic & Gouvier, 1996; Heinrichs, 1990). Despite the significance of ecological validity to the field of neuropsychology, no study has surveyed opinions about the use of neuropsychological assessment instruments for ecological purposes. Future studies might assess the degree to which various instruments are considered useful in addressing ecological issues (e.g., assessing real-world functioning, predicting occupational competence). In addition, research could directly explore utilization rates of instruments designed with ecological concerns in mind (e.g., Rivermead Behavioral Memory Test (RBMT); Wilson, Cockburn, & Baddeley, 1985); Behavioral Assessment of the Dysexecutive Syndrome (BADS; Wilson, Alderman, Burgess, Emslie, & Evans, 1996). The test usage data collected in the present study could serve as a foundation for future research on these matters. Finally, the test usage data presented above highlight the value of going beyond investigation of only the most frequently used instruments. Surveys that explore a broader range of techniques are better able to capture the richness and diversity of current neuropsychological assessment practices. Since this is the first study to explore test usage in such detail, there exists no reliable baseline by which to judge the obtained results. These findings can ground future investigations, which should extend the current data to include instruments used to assess other cognitive domains, such as language and visuospatial skills. Ultimately, the value of this type of survey research lies in its ability to reveal current trends in assessment, inform future test selection decisions, and raise questions worthy of further exploration. Acknowledgments The authors would like to communicate their gratitude to the many INS, NAN, and APA Division 40 members who took time from their busy schedules to complete and return this questionnaire.

60

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Appendix A. Standardized instruments Abbreviation

Instrument

ACT BADS BDAE/BDAE-3 BDI/BDI-II Bender BSRT BVRT/BVRT-R CCC CERAD Cognistat/NCSE COWAT CPT/CPT-II

Auditory Consonant Trigram Test Behavioral Assessment of the Dysexecutive Syndrome Boston Diagnostic Aphasia Examination-Third Edition Beck Depression Inventory-Second Edition Bender Visual Motor Gestalt Test/Bender–Gestalt Buschke Selective Reminding Test Benton Visual Retention Test-Revised Edition Brown–Peterson Consonant Trigram Memory Task Consortium to Establish a Registry for Alzheimer’s Disease Neurobehavioral Cognitive Status Examination Controlled Oral Word Association Test/FAS Continuous Performance Task-Second Edition (type unspecified) California Verbal Learning Test-Second Edition California Verbal Learning Test, Children’s Version Continuous Visual Memory Test D2 Test: Concentration Endurance Test Delis–Kaplan Executive Function System Mattis Dementia Rating Scale Finger Tapping/Finger Oscillation Test Gordon Diagnostic System, Continuous Performance Test Hand Dynamometer/Grip Strength Hooper Visual Organization Test Halstead–Reitan Neuropsychological Battery Hopkins Verbal Learning Test-Revised Edition Integrated Visual and Auditory Continuous Performance Test Benton Judgment of Line Orientation Test Luria–Nebraska Neuropsychological Battery Memory Assessment Scales Millon Clinical Multiaxial Inventory-Second Edition Minnesota Multiphasic Personality Inventory-Second Edition Mini-Mental State Examination Mental Status Examination A Developmental Neuropsychological Assessment Personality Assessment Inventory Paced Auditory Serial Addition Task Rey Auditory Verbal Learning Test Repeatable Battery for the Assessment of Neuropsychological Status

CVLT/CVLT-II CVLT-C CVMT d2 D-KEFS DRS Finger Tapping Gordon CPT Grip Strength Hooper VOT HRNB HVLT/HVLT-R IVA JLO LNNB MAS MCMI/MCMI-II MMPI/MMPI-2 MMSE MSE NEPSY PAI PASAT RAVLT RBANS

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

61

Appendix A (Continued )

RBMT RBMT-C RFFT ROCFT SB Categories SB-R SCL-90/SCL-90-R SDMT Stroop Test TEA TOMM TOVA VIGIL VMI VSAT WAIS WAIS-R WAIS-R NPI WAIS-III WAIS-III NPI WASI WCST WIAT WIAT-II WISC-R WISC-III WJ-R WJ-III

WMS WMS-R WMS-III WRAML WRAT WRAT-R WRAT-3 WRMT

Instrument Rivermead Behavioral Memory Test Rivermead Behavioral Memory Test for Children Ruff Figural Fluency Test Rey–Osterrieth Complex Figure Task Stanford–Binet Category Test Stanford–Binet Intelligence Scale-Revised Symptom Checklist 90-Revised Symbol Digit Modalities Test Stroop Neuropsychological Screening Test Test of Everyday Attention Test of Memory Malingering Test of Variables of Attention Vigilance Continuous Performance Test Developmental Test of Visual-Motor Integration Visual Search and Attention Test Wechsler Adult Intelligence Scale Wechsler Adult Intelligence Scale-Revised WAIS-R Neuropsychological Instrument WAIS-III Neuropsychological Instrument Wechsler Adult Intelligence Scale-Third Edition Wechsler Abbreviated Scale of Intelligence Wisconsin Card Sorting Test Wechsler Individual Achievement Test Wechsler Individual Achievement Test-Second Edition Wechsler Intelligence Scale Children-Revised Wechsler Intelligence Scale Children-Third Edition Woodcock–Johnson Psychoeducational Battery-Revised Woodcock–Johnson Tests of Achievement-Third Edition; Woodcock–Johnson Tests of Cognitive Abilities Battery-Third Edition Wechsler Memory Scale Wechsler Memory Scale-Revised Wechsler Memory Scale-Third Edition Wide Range Assessment of Memory and Language Wide Range Achievement Test Wide Range Achievement Test-Revised Edition Wide Range Achievement Test-Third Edition Warrington Recognition Memory Test

62

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

Appendix B. Nonstandardized instruments Abbreviation

Instrument

Assessment of Job Demands

Assessment of Job Demands/Expectations/Current Job Requirements Assessment of Topic Maintenance/Lapses of thought during interview/Ability to have goal directed conversation Miscellaneous Cancellation Task Clinical/Diagnostic/Psychosocial Interview and History Competing Programs Test/Luria Executive Tests/Frontal Motor Series/Alternating Hand Movements/3-Step Command/Fist-Edge-Palm Functional Assessment/Direct Observation of Patient in Environment

Assessment of Topic Maintenance

Cancellation Test Clinical Interview Frontal Motor Series

Functional Assessment

References American Board of Professional Psychology. (2001). Clinical neuropsychology diplomate directory. Retrieved June 5, 2001, from http://www.abpp.org/. Beck, A. T. (1987). Beck Depression Inventory. San Antonio, TX: The Psychological Corporation. Beck, S. J. (1944). Rorschach’s Test I: Basic processes. New York: Grune & Stratton. Bender, L. (1946). Instructions for the use of the Visual Motor Gestalt Test. New York: American Orthopsychiatric Association. Benton, A. L. (1946). A visual retention test for clinical use. New York: Psychological Corporation. Benton, A. L., & Hamsher, K. deS. (1989). Multilingual Aphasia Examination. Iowa City, IA: AJA Associates. Boyd, T. A. (1988). Clinical assessment of memory in children: A developmental framework for practice. In M. G. Tramontana & S. R. Hooper (Eds.), Assessment issues in child neuropsychology (pp. 177–204). New York: Plenum Press. Brown, L. B. (1999). A survey of neuropsychologists’ ethical beliefs and practices (Doctoral dissertation, Saint Louis University, 1999). Dissertation Abstracts International, 60, AAG9942756. Brown, W. R., & McGuire, J. M. (1976). Current psychological assessment practices. Professional Psychology, 7, 475–484. Butler, M., Retzlaff, P., & Vanderploeg, R. (1991). Neuropsychological test usage. Professional Psychology: Research and Practice, 22, 510–512. Camara, W. J., Nathan, J. S., & Puente, A. E. (2000). Psychological test usage: Implications in professional psychology. Professional Psychology: Research and Practice, 31, 141–154. Christensen, A. L. (1975). Luria’s neuropsychological investigation. Copenhagen, Denmark: Munksgaard. Conners, C. K., & Multi-Health Systems Staff. (1995). Conners’ Continuous Performance Test. Toronto, Canada: MHS. Cubic, B. A., & Gouvier, W. D. (1996). The ecological validity of perceptual tests. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 203–224). Boca Raton, FL: St. Lucie Press. Darley, J. G., & Marquis, D. G. (1946). Veterans’ guidance centers: A survey of their problems and activities. Journal of Clinical Psychology, 2, 109–116.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

63

Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). The California Verbal Learning Test-Second Edition: Adult version manual. San Antonio, TX: The Psychological Corporation. Delis, D. C., Kramer, J. H., Kaplan, E., Ober, B. A., & Fridlund, A. (1987). The California Verbal Learning Test. New York: Psychological Corporation. DeLuca, J. W. (1989). Neuropsychology technicians in clinical practice: Precedents, rationale. The Clinical Neuropsychologist, 3, 3–21. DeLuca, J. W., & Putnam, S. H. (1993). The professional/technician model in clinical neuropsychology: Deployment characteristics and practice issues. Professional Psychology: Research and Practice, 24, 100–106. De Renzi, E., & Vignolo, L. (1962). The Token Test: A sensitive test to detect receptive language disturbances in aphasia. Brain, 85, 665–678. Golden, C. J., Purisch, A. D., & Hammeke, T. A. (1985). Luria–Nebraska Neuropsychological Battery: Forms I and II. Los Angeles: Western Psychological Services. Graham, F. K., & Kendall, B. S. (1960). Memory-for-Designs Test: Revised general manual. Perceptual and Motor Skills, 11, 147–188. Graham, J. R. (1990). MMPI-2: Assessing personality and psychopathology. New York: Oxford University Press. Gronwall, D. M. A. (1977). Paced auditory serial-addition task: A measure of recovery from concussion. Perceptual and Motor Skills, 44, 367–373. Guilmette, T. J., & Faust, D. (1991). Characteristics of neuropsychologists who prefer the Halstead–Reitan or the Luria–Nebraska Neuropsychological Battery. Professional Psychology: Research and Practice, 22, 80–83. Guilmette, T. J., Faust, D., Hart, K., & Arkes, H. R. (1990). A national survey of psychologists who offer neuropsychological services. Archives of Clinical Neuropsychology, 5, 373–392. Guilmette, T. J., & Kastner, M. P. (1996). The prediction of vocational functioning from neuropsychological data. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 387–411). Boca Raton, FL: St. Lucie Press. Halstead, W. C. (1947). Brain and intelligence. Chicago: University of Chicago Press. Hartlage, L. C., & Telzrow, C. F. (1980). The practice of clinical neuropsychology in the U.S. Clinical Neuropsychology, 2, 200–202. Hathaway, S. R., & McKinley, J. C. (1951). The Minnesota multiphasic personality interview. New York: The Psychological Corporation. Heaton, R. K., Chelune, G. J., Talley, J. L., Kay, C. G., & Curtiss, G. (1993). Wisconsin Card Sorting Test manual. Odessa, FL: Psychological Assessment Resources. Heinrichs, R. W. (1990). Current and emergent applications of neuropsychological assessment: Problems of validity and utility. Professional Psychology: Research and Practice, 21, 171–176. Jastak, S., & Bijou, S. (1941). Wide Range Achievement Test: Reading, spelling, arithmetic from kindergarten to college. Oxford, England: Chas L. Story Company. Jastak, S., & Wilkinson, G. (1984). The Wide Range Achievement Test-Revised. Wilmington, DE: Jastak Associates. Jensen, A. R., & Rohwer, W. D. (1966). The Stroop Color-Word Test: A review. Acta Psychologia, 25, 36–93. Kaplan, E. F., Goodglass, H., & Weintraub, S. (1978). The Boston Naming Test. Boston, MA: Kaplan & Goodglass. Kerlinger, F. (1973). Foundations of behavioral research (2nd ed.). New York: Holt, Rinehart, & Winston. Lees-Haley, P. R., Smith, H. H., Williams, C. W., & Dunn, J. T. (1996). Forensic neuropsychological test usage: An empirical study. Archives of Clinical Neuropsychology, 11, 45–51. Lewis, R. F., & Rennick, P. M. (1979). Manual for the repeatable cognitive-perceptual-motor battery. Grosse Pointe Park, MI: Axon Publishing. Louttit, C. M., & Browne, C. G. (1947). The use of psychometric instruments in psychological clinics. Journal of Consulting Psychology, 11, 49–54. Lubin, B., Wallis, R. R., & Paine, C. (1971). Patterns of psychological test usage in the United States: 1935–1969. Professional Psychology, 2, 70–74. Matthews, C. G., & Klove, H. (1964). Instruction manual for the Adult Neuropsychology Test Battery. Madison, WI: University of Wisconsin Medical School. McCaffrey, R. J., & Isaac, W. (1984). Survey of the educational backgrounds and specialty training of instructors of clinical neuropsychology in APA-approved graduate training programs. Professional Psychology: Research and Practice, 15, 26–33.

64

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

McCaffrey, R. J., & Lynch, J. K. (1996). Survey of the educational backgrounds and specialty training of instructors of clinical neuropsychology in APA-approved graduate training programs: A 10-year follow-up. Archives of Clinical Neuropsychology, 11, 11–19. McCaffrey, R. J., Malloy, P. F., & Brief, D. J. (1985). Internship opportunities in clinical neuropsychology emphasizing recent INS training guidelines. Professional Psychology: Research and Practice, 16, 236–252. McMordie, W. R. (1988). Twenty-year follow-up of the prevailing opinion on the posttraumatic or postconcussional syndrome. The Clinical Neuropsychologist, 2, 198–212. Morgan, C. D., & Murray, H. A. (1935). A method for investigating phantasies: The Thematic Apperception Test. Archives of Neurology and Psychiatry, 34, 289–306. National Science Foundation. (1999). Women, minorities, and persons with disabilities in science and engineering. Arlington, VA: Author. Piotrowski, C., & Keller, J. W. (1978). Psychological test usage in southeastern outpatient mental health facilities in 1975. Professional Psychology, 9, 63–67. Putnam, S. H. (1989). The TCN salary survey: A salary survey of neuropsychologists. The Clinical Neuropsychologist, 3, 97–115. Putnam, S. H., & DeLuca, J. W. (1990). The TCN professional practice survey: Part I: General practices of neuropsychologists in primary employment and private practice settings. The Clinical Neuropsychologist, 4, 199–244. Putnam, S. H., & DeLuca, J. W. (1991). The TCN professional practice survey: Part II: An analysis of the fees of neuropsychologists by practice demographics. The Clinical Neuropsychologist, 5, 103–124. Putnam, S. H., DeLuca, J. W., & Anderson, C. (1994). The second TCN salary survey: A survey of neuropsychologists part II. The Clinical Neuropsychologist, 8, 245–282. Reitan, R. M., & Davison, L. A. (1974). Clinical neuropsychology: Current status and applications. New York: Hemisphere. Reitan, R. M., & Wolfson, D. (1985). The Halstead Reitan neuropsychological test battery: Theory and clinical interpretation. Tucson, AZ: Neuropsychology Press. Report of Committee of Clinical Section of American Psychological Association. (1935). The Psychological Clinic, 23, 23–27. Rey, A. (1941). Psychological examination of traumatic encephalopathy. Archives de Psychologie, 28, 286–340 (sections translated by Corwin, J., & Bylsma, F. W. The Clinical Neuropsychologist, 7, 4–9). Ryan, J. J., & Paolo, A. M. (1990). Credentials and clinical activities of internship supervisors in neuropsychology: A comparison of VA and non-VA training sites. Archives of Clinical Neuropsychology, 5, 69–75. Sell, J. M., & Torres-Henry, R. (1979). Testing practices in university and college counseling centers in the United States. Professional Psychology, 10, 774–779. Seretny, M. L., Dean, R. S., Gray, J. W., & Hartlage, L. C. (1986). The practice of clinical neuropsychology in the United States. Archives of Clinical Neuropsychology, 1, 5–12. Slay, D. K., & Valdivia, L. (1988). Neuropsychology as a specialized health service listed in the National Register of Health Service Providers in Psychology. Professional Psychology: Research and Practice, 19, 323–329. Strub, R. L., & Black, F. W. (1977). The Mental Status Examination in neurology. Philadelphia: F.A. Davis. Sundberg, N. D. (1961). The practice of psychological testing in clinical services in the United States. American Psychologist, 16, 79–83. Sweet, J. J., & Moberg, P. J. (1990). A survey of practices and beliefs among ABPP and non-ABPP clinical neuropsychologists. The Clinical Neuropsychologist, 4, 101–120. Sweet, J. J., Moberg, P. J., & Suchy, Y. (2000). Ten-year follow-up survey of clinical neuropsychologists: Part I. Practices and beliefs. The Clinical Neuropsychologist, 14, 18–37. Sweet, J. J., Moberg, P. J., & Westergaard, C. K. (1996). Five-year follow-up survey of practices and beliefs of clinical neuropsychologists. The Clinical Neuropsychologist, 10, 202–221. Sweet, J. J., Peck, E. A., Abramowitz, C., & Etzweiler, S. (2002). National Academy of Neuropsychology/Division 40 of the American Psychological Association practice survey of clinical neuropsychology in the United States, Part I: Practitioner and practice characteristics, professional activities and time requirements. The Clinical Neuropsychologist, 16, 109–127.

L.A. Rabin et al. / Archives of Clinical Neuropsychology 20 (2005) 33–65

65

Swenson, W. M., & Lindgren, E. (1952). The use of psychological tests in industry. Personnel Psychology, 5, 19–23. Van Hutton, V. (1993). Quantitative scoring system for children’s projective drawings (House-Tree-Person and Draw-A-Person) with focus on personality emotional characteristics prevalent in abused children (Doctoral dissertation, University of California, Los Angeles, 1993). Dissertation Abstracts International, 52, 70944-001. Wade, T. C., & Baker, T. B. (1977). Opinions and use of psychological tests: A survey of clinical psychologists. American Psychologist, 32, 874–882. Wade, T. C., Baker, T. B., & Hartmann, D. P. (1979). Behavior therapists’ self-reported views and practices. The Behavior Therapist, 2, 3–6. Wade, T. C., Baker, T. B., Morton, T. L., & Baker, L. J. (1978). The status of psychological testing in clinical psychology: Relationships between test use and professional activities and orientations. Journal of Personality Assessment, 42, 3–10. Wechsler, D. (1945). A standardized memory scale for clinical use. Journal of Psychology, 19, 87–95. Wechsler, D. (1955). Wechsler Adult Intelligence Scale manual. New York: The Psychological Corporation. Wechsler, D. (1974). Wechsler Memory Scale manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1981). Wechsler Adult Intelligence Scale-Revised manual. New York: The Psychological Corporation. Wechsler, D. (1987). Wechsler Memory Scale-Revised manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1991). Wechsler Intelligence Scale for Children-Third Edition manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1997a). Wechsler Adult Intelligence Scale-Third Edition: Administration and scoring manual. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1997b). Wechsler Memory Scale-Third Edition: Administration and scoring manual. San Antonio, TX: The Psychological Corporation. Wiens, A. N., McMinn, M. R., & Crossen, J. R. (1988). Rey Auditory-Verbal Learning Test: Development of norms for healthy young adults. Clinical Neuropsychologist, 2, 67–87. Wilkinson, G. S. (1993). The Wide Range Achievement Test-3rd Edition (WRAT-3). Wilmington, DE: Wide Range. Wilson, B. A., Alderman, N., Burgess, P. W., Emslie, H., & Evans, J. J. (1996). Behavioral Assessment of the Dysexecutive Syndrome (BADS). Bury St. Edmonds, UK: Thames Valley Test Company. Wilson, B. A., Cockburn, J., & Baddeley, A. (1985). The Rivermead Behavioral Memory Test manual. Bury St. Edmonds, UK: Thames Valley Test Company.