Development of a Neurobehavioral Battery for Children Exposed to Neurotoxic Chemicals

NeuroToxicology 22 (2001) 657±665

Development of a Neurobehavioral Battery for Children Exposed to Neurotoxic Chemicals Diane S. Rohlman*, W. Kent Anger, Alys Tamulinas, Jacki Phillips, Steffani R. Bailey, Linda McCauley Center for Research on Occupational and Environmental Toxicology, CROET L606, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Portland, OR 97201, USA Received 16 January 2001; accepted 8 July 2001

Abstract In recent years there has been heightened concern over the potential impact of environmental exposures on neurological function in children. Children are thought to be especially vulnerable to neurotoxic effects due to a number of factors including play behavior, differences in metabolism, and the development state of the brain. Performance tests from the computerized Behavioral Assessment and Research System (BARS) and other non-computerized tests have been combined to develop a brief battery that assesses multiple neurobehavioral functions in preschool children aged 4±6. Tests were selected to assess a variety of cognitive functions including attention, memory, motor speed and coordination and other executive functions. The battery has also been translated into Spanish and developed for use with Latino populations. Four to six-year-old children are particularly challenging because of the shorter attention span and lower motivation to complete an extended test session. When testing this group it is important to maintain the motivation of the child throughout the entire session in order to obtain accurate performance measures. A series of sequential pilot studies were used to select and develop appropriate methods and parameters for the tests in the battery. Although English-speaking children were able to complete the initial battery with minimal dif®culties, several dif®culties were encountered when the tests were administered to a Latino population. Cultural differences made some material inappropriate for testing due to unfamiliarity with the material, and in some cases items in a test had more than one correct translation which made administration dif®cult. # 2001 Elsevier Science Inc. All rights reserved.

Keywords: Neurobehavioral tests; Computerized assessment; Children; Latino

INTRODUCTION By its very de®nition, occupational health research focuses on adult populations in a working environment (Anger et al., 1994; Johnson et al., 1994; Sizemore and Amler, 1996; Gamberale, 1993). Issues such as age and education may be considered in a study design, but the ages are clearly con®ned to a limited range, education is generally expected to include attainment of a high school degree or equivalent, and most of the research methods require the participant to read and speak English (Letz, 1994). In contrast to occupational health research, community health research includes a much * Corresponding author. Tel.: ‡1-503-494-2514; fax: ‡1-503-494-4278. E-mail address: [email protected] (D.S. Rohlman).

broader range of participants whose ages may range from that of infants to elderly adults, whose educational attainment may range from pre-kindergarten to advanced graduate training, and whose ®rst language may not be English (Amler et al., 1996; Sizemore and Amler, 1996). In recent years there has been heightened concern over the potential impact of environmental exposures on neurological function in children (Krasnegor et al., 1994; Olden and Guthrie, 2000; Schettler et al., 2000). Children are thought to be especially vulnerable to neurotoxic effects due to a number of factors including play behavior, frequent hand-to-mouth contact, and differences in metabolism (Mott, 1995; Zartarian et al., 1995; Reed et al., 1999; Cohen Hubal et al., 2000). Unlike adult neurobehavioral testing, pediatric neurobehavioral testing does not have an extensive existing

0161-813X/01/$ ± see front matter # 2001 Elsevier Science Inc. All rights reserved. PII: S 0 1 6 1 - 8 1 3 X ( 0 1 ) 0 0 0 4 9 - 3

658

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665

body of background information and experience (Amler et al., 1996; Krasnegor et al., 1994; Davidson et al., 2000). Pediatric neurotoxicology has focused primarily on children exposed to lead (e.g. Bellinger et al., 1990; Baghurst et al., 1992, 1995) and mercury (e.g. Davidson et al., 1998; Grandjean et al., 1997; Dahl et al., 1996; Choi, 1989; Shamlaye et al., 1995; McKeown et al., 1983). Typically, children are tested one-on-one by a health professional or psychologist trained to explain simply how to perform the test and to maintain motivation through often heroic measures (Otto et al., 1996). This approach succeeds in obtaining data but leaves open to question the consistency of methods from child to child. There is a need for reliable, easy-to-administer tests to assess neurotoxic exposure in children (Otto et al., 1996). The use of neurobehavioral tests to evaluate culturally diverse populations in the US poses unique challenges, in part because this topic has not been extensively studied (Helms, 1992). The majority of tests that have been developed to assess the neurotoxic potential of environmental chemicals have not been evaluated in ethnically and culturally diverse groups. Pesticides represent one category of environmental contaminants to which minorities and socioeconomically disadvantaged groups are inequitably exposed (Moses et al., 1993). All populations are exposed to pesticide exposures from non-agricultural use, and through residues on food (Hill et al., 1995; Kutz et al., 1992), but there exist subgroups of the population with higher risk of potential exposure and health effects (National Research Council, 1993). Latino workers comprise the majority of agricultural workers in the United States (Mines et al., 1997) and workers range in age from children in their teens to adults in their sixties (Arcury and Quandt, 1998). Young children who do not directly work in the ®elds may also be at risk for exposure due to housing located in close proximity to farm ®elds or residues from parents' work clothing or tools (Eshleman and Daidhizar, 1997; Fenske et al., 2000; Loewenherz et al., 1997). Few ef®cient tools are available for researchers who seek to gather information on the adverse health effects of exposures in this and similar population subgroups who are known to be exposed to pesticides and other potentially harmful compounds (Task Force of Environmental Cancer, 1990). Without test methods appropriate for non-English-speaking populations, the full extent of neurotoxic health effects from exposure to toxicants cannot be studied comprehensively (Williamson, 1990). When testing non-English-speaking populations it is obviously important to have an accurate translation of

tests. Two of the main concerns when translating test materials from English to Spanish are cultural differences of test items and the use of formal language by well-educated Latinos but not by those less educated (Bernal, 1990; Mahurin et al., 1992). Test items may have more than one correct translation, especially when different local dialects are taken into account. Also, items or scenarios described in the test materials may be unfamiliar to the Latino population found in agriculture. Formal language, often used in a word-forword translation, may also distort the original meaning of the material and lead to confusion, especially in a population with minimal education. A better approach is to use a functional translation, which uses terms that are easily comprehensible but convey the equivalent idea or concept. Development of a Neurobehavioral Test Battery This paper describes the methods used to develop a neurobehavioral test battery to be used in ®eld-based screening of children. Because of the large variability in the cognitive and attentional abilities of children, we chose to develop the battery by focusing on the most challenging subset of children, preschool children aged 4±6. A neurobehavioral test battery was assembled by combining performance tests from the computerized Behavioral Assessment and Research System (BARS, Anger et al., 1996) and other non-computerized tests to develop a brief battery that assesses multiple neurobehavioral functions. Computerized tests have the advantage of presenting information in a consistent and ef®cient manner across participants while minimizing the impact of the examiner. Non-computerized tests were added to supplement the neurobehavioral functions tested by the BARS. Tests were selected to assess a variety of cognitive functions including attention, memory, motor speed and coordination and other executive functions. BARS was initially developed for use with a broad range of working populations that have different education levels and cultural backgrounds (Anger et al., 1996; Rohlman et al., 1996). Features of BARS that enable this include: simple language instructions broken down into basic concepts (step-by-step training with competency testing at each instruction step); a ``smiling face'' used to reinforce performance and adjustable parameter settings. A durable response unit with nine buttons is placed over the keyboard (pictured in Anger et al., 1996) to minimize the adverse impact of working on an unfamiliar device such as a computer keyboard. Four to six-year-old children are particularly

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665

challenging because of the shorter attention span and lower motivation to complete an extended test session. When testing this population it is important to maintain the motivation of the child through out the entire session in order to obtain accurate performance measures. To help maintain motivation a token dispenser was added as a supplement to the ``smiling face'' feedback for the computerized BARS tests during the instructions and in some cases during the tests. Rohlman et al. (2000) describes the development process for the computerized BARS tests in the neurobehavioral test battery. Based on the performance of English-speaking preschool children, modi®cations were made to the original test battery and parameter settings. Five BARS tests were originally administered, Symbol-Digit, Digit Span, Finger Tapping, Continuous Performance, and Matching to Sample. The symboldigit test was dropped from the battery because the children had dif®culty ®nishing the assessment portion of the test. Although they understood how to perform the test, they would become restless and distracted during the assessment and fail to complete the test. A switch from a visual presentation to a spoken presentation during Digit Span was found to improve the number of children completing the test. The alternating version of Finger Tapping (tap sequentially with the left hand, then right, then left, etc.) was eliminated from the battery because the children were unable to respond fast enough to correctly meet the training criterion (®ve left±right taps in 10 s). Failure to meet the training criterion would cause the child to repeat the instructions and training. Unsuccessful performance and repeating the instructions multiple times often would decrease the motivation of the child to continue testing. The left and right hand trials remain in the battery. Parameter modi®cations were made to the Continuous Performance test. When the modi®ed battery of BARS tests were then administered to Spanishspeaking preschool children similar results were found. All of the children were able to complete Finger Tapping and the majority of children were able to complete Digit Span forward, Continuous Performance, and Matching to Sample. In addition to the computerized tests, four non-computerized tests were also administered. Three tests from the Pediatric Environmental Neurobehavioral Test Battery (PENTB; Amler and Gibertini, 1996) were selected: the Story Memory subtest from the Wide Range Assessment of Memory and Learning (WRAML), a test of verbal memory; the vocabulary and matrices subtests from the Kaufman Brief Intelligence Test (K-BIT) and the Purdue Pegboard test. A new test of recall and

659

recognition, the Object Memory test (Mahurin et al., 1992), was also administered. Three problems were found when the instructions and materials for the non-computerized tests were translated into Spanish. First, some of the material was culturally inappropriate for a Latino population. The scenarios described in the WRAML stories discussing birthday parties and going to the skating rink, were unfamiliar to this population. This test was excluded from the battery. Second, we found that for some items in the K-BIT there was more than one translation (i.e. feather, bridge, owl). Although we attempted to account for this problem by allowing multiple answers to be correct, we were unable to eliminate it entirely, which made scoring the test potentially unreliable. At times children would also name some test items in English instead of Spanish. This test was also excluded from the battery. The ®nal problem occurred when the children were unfamiliar with an item in a test. For example, many Latino children did not know what a whistle was; this was an item used in the Object Memory test. A solution to this problem is to replace the unfamiliar test items. However, the Object Memory test does not require children to know the correct name of an item, only to be able to give the item a name, therefore, the test was kept in the battery. None of the children had dif®culty with the Purdue Pegboard test. Based on these ®ndings, the tests in the battery were modi®ed. The Object Memory test and the Purdue Pegboard test remained in the battery. Two other non-computerized tests from the PENTB were added to develop a more comprehensive test battery, the Divided Attention test and the Visual Motor Integration test. The four computerized BARS tests also remained in the battery: Digit Span, Finger Tapping, Matching to Sample, and Continuous Performance. The battery was then administered to second groups of English- and Spanish-speaking preschool children and their performance was compared. Based on the previous modi®cations to the test battery minimum group performance differences were expected. METHODS Participants English-speaking children (n ˆ 31) between the age of 4 and 6 years were recruited from local childcare centers in Portland, OR and through advertisements in local newspapers. The Spanish-speaking children (n ˆ 17) were recruited from La Clinica de Buena Salud (a healthcare clinic located in Portland, OR),

660

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665

Table 1 Demographics of children administered the neurobehavioral test battery

Age (months) Percent females (%) Language tested

English-speaking (n ˆ 31)

Spanish-speaking (n ˆ 17)

62.2 (10.6) 52 English

59.9 (9.2) 24 Spanish

from advertisements posted at Oregon Health & Science University, and from Oregon Head Start Centers. Demographic characteristics of the children are shown in Table 1. Neurobehavioral Test Battery Table 2 shows the individual tests in the neurobehavioral test battery and provides a brief description of

each test and the function measured. Further information on these tests can be found in Rohlman et al. (2000) and Amler and Gibertini (1996). The BARS, originally developed in English for a working population, was translated into Spanish and developed for use with Latino populations. A certi®ed translator originally translated the test instructions into Spanish. However, because the primary application of the tests would be with a Latino farm worker population (both adults and children), they were then reviewed and modi®ed by personnel at the Oregon Child Development Coalition who were familiar with the vocabulary and sentence structure used by the farm worker population we planned to study. During the development period, the instructions were sequentially administered to small groups of Latino adolescents followed by modi®cations to test instructions and parameter settings. Each sequential administration was

Table 2 Description of tests and performance of English- and Spanish-speaking children on the neurobehavioral battery Test

Description

Performance (% completed)

BARS Digit Span

Memory and attention Spoken presentation of number sequences Forward and reverse recall Two chances at each span length

Forward English: 97% Spanish: 80% Reverse English: 62% Spanish: 56%

BARS Finger Tapping

Response speed and coordination Right and left hand tested Number of taps in 20 s duration

English: 100% Spanish: 100%

BARS Match to Sample

Visual memory 15 stimuli shown for 3 s Choose from three choices Delay between presentation and choice varies from 1 to 8 s

English: 93% Spanish: 93%

BARS Continuous Performance

Attention 75 shapes shown rapidly, 30 targets Increased number of incorrect responses allowed during practice

English: 67% Spanish: 40%

Object Memory Test

Recall and recognition memory Shown 16 objects and asked to name Immediate and delayed recall Recognition test

English: 100% Spanish: 100%

Purdue Pegboard

Dexterity Number of small pegs placed in holes during 30 s Right, left and both hand trials

English: 90% Spanish: 100%

Divided Attention

Working memory Recite nursery rhyme while tapping finger Right and left hands tested

English: 94% Spanish: 56% Children had difficulty learning nursery rhyme

Visual Motor Integration

Compare number of taps while reciting to number of taps while not reciting Visual motor integration Drawing test Replicate 24 figures

English: 100% Spanish: 94%

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665

intended to reveal needed modi®cations in the test instructions and test order, and did so. A similar translation and review process was conducted with the non-computerized tests. Methods The tests were administered individually to each child. An examiner was always present during the test session, to read instructions, answer questions, and reinforce responding when necessary. A second examiner was also present during testing to observe the test session and record the child's reaction to the tests (e.g. comments or questions from the child or inattention during the test). Tokens were earned for correct performance on the BARS tests and the tokens were exchanged for nickels at the end of the test session. A child would earn tokens for correct responding during the instructions for each test and also when a correct match was made during the match to sample and completing a trial during ®nger tapping. Although tokens were earned for correct responding, a child who has dif®culty completing the instructions and practice and subsequently repeats the instructions more than once, would have more opportunities to receive tokens than a child who correctly completes the training for the ®rst time. RESULTS During the initial stages of development of the battery, the primary goal is to determine if children are able to successfully complete the tests in the battery. Success is de®ned as completing a test in a timely manner as well as having meaningful performance data (e.g. performance within reasonable ranges and well above ¯oor values, since environmental exposures can be predicted to drive down performance on many tests). Table 2 presents the percentage of English- and Spanish-speaking children who were able to complete the tests in the neurobehavioral battery. The majority of English-speaking children were able to successfully complete all of the tests except for the Digit Span reverse. The number of children completing the Continuous Performance test was increased when a modi®cation was made to the practice criterion during pilot testing. The modi®cation made the success criterion required to move from the practice to assessment more lenient (i.e. more incorrect responses were allowed in practice). This change was made after an examination of the data and observation of the test

661

sessions revealed that children understood the task (to press the key when the target appeared), but would make several errors or false alarms during the practice. This would cause them to repeat the training and practice sessions, sometimes several times. Comments from the children and observations of their behavior indicated that repeating the practice session reduced their motivation for completing the test. Making the criterion more lenient allows them to proceed to the assessment more quickly. The percentage of Englishspeaking children successfully completing the Continuous Performance test increased from 56 to 87% after this change was implemented. The Spanish-speaking children were also able to complete all of the BARS tests in the battery, except for Digit Span reverse, as was the case for the Englishspeaking children. They also had no dif®culty with the non-computerized tests except for Divided Attention. The PENTB adaptation of this task requires a child to perform the ®nger tapping task while reciting the ``Jack and Jill'' nursery rhyme. Eleven of the English-speaking children were tested with the ``Jack and Jill'' nursery rhyme. However, because several of the English-speaking children were unfamiliar with the nursery rhyme and because there is no Spanish equivalent of this nursery rhyme, we changed the protocol to use the ``Itsy Bitsy Spider'' nursery rhyme or ``La AranÄa PequenÄita'' for our Spanish-speaking participants. Several of the Spanish preschool children were unfamiliar with this nursery rhyme. They had dif®culty learning the rhyme and subsequently, dif®culty performing the task while reciting it. Successfully completing a test also involves having meaningful data. Because limited data with preschool children exists on these tests there are no normative data to use as a reference. Fig. 1 shows the performance data for both groups on the computerized BARS tests from the neurobehavioral battery. Performance across groups is similar. The English-speaking group performed slightly better on the Digit Span test, although this difference is not signi®cant when the signi®cance level is corrected for multiple comparisons. Performance data from the non-computerized tests is shown in Fig. 2. No signi®cant difference was found between the groups on Purdue Pegboard or Visual Motor Integration although the English-speaking group performed slightly better. A signi®cant difference was found for the number of items named on the Object Memory test (t ˆ 3:2, P ˆ 0:002). An examination of the naming errors made by both groups showed that both groups had dif®culty with similar objects. For example, many children had trouble naming

662

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665

Fig. 1. Performance data (mean  S.D.) for English- and Spanish-speaking preschool children on computerized BARS tests from the neurobehavioral battery.

``envelope'' they would call it a ``letter'', ``paper'', or ``card''. More Spanish-speaking children were unable to give the object a name when it was handed to them. There was also a tendency for the English-speaking children to give a more speci®c name for the object, e.g. rose for ¯ower or baseball for ball, although these were not counted as errors. Although an item may be named incorrectly, this did not impact their recall or recognition performance.

There were signi®cant differences in performance between groups on the number of words recited both while tapping with the right hand (t ˆ 10:2, P < 0:0001) and while not tapping (t ˆ 9:1, P < 0:0001) during the Divided Attention test. The Spanish-speaking children recited fewer words in both situations. Table 2 shows that fewer Spanish-speaking children were able to successfully complete this task. Several of these children had dif®culty with the rhyme that was

Fig. 2. Performance data (mean  S.D.) for English- and Spanish-speaking preschool children on non-computerized tests from the neurobehavioral battery.

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665

chosen, they were unfamiliar with it or had dif®culty reciting the rhyme while tapping simultaneously. This was also seen in some English-speaking children. An examination of the types of responses and pattern of responding indicates that in the majority of tests the children were correctly performing the tests. For example, both Finger Tapping and Purdue Pegboard tests show a range of responses and variability in the data. The Object Memory test demonstrates the expected pattern of results for a recall and recognition task, i.e. higher performance on recognition than recall. The performance data from the Continuous Performance test indicates children were responding using a strategy that led to correct performance. This is shown by the high number of hits and correct rejections and by the low number of misses and false alarms. A similar pattern is seen with the English-speaking children on the Match to Sample test. DISCUSSION Although English-speaking children were able to complete the initial battery with minimal dif®culties, several dif®culties were encountered when the tests were administered to the Latino population. Cultural differences made some material inappropriate for testing due to unfamiliarity with the material, and in some cases items in a test had more than one correct translation which made administration dif®cult. The rhyme chosen for the Divided Attention task may not be a culturally appropriate rhyme. Although the PENTB instructs the examiner to teach the rhyme to the child if they are unfamiliar with it this tends to add to the testing burden and causes the child not to be successful on the task. We found that it was necessary for the examiner to repeatedly remind the child what the rhyme was during the test, which had an impact on performance. The child was also more likely to fail to complete all trials of the test. It is also possible that the English and Spanish versions of the ``Itsy Bitsy Spider'' nursery rhyme are not equivalent. The English version contains more words (38) than the Spanish version (27). However, previous research examining recall from short-term memory has shown that the average number of syllables in a list of words in¯uences pronunciation time (Naveh-Benjamin and Ayres, 1986). For example, most of the English words for the numbers between 1 and 10 are one-syllable words. However, the Spanish words have on an average a greater number of syllables (e.g. cuatro, cinco, siete). This effect may contribute to the lower number of words recited from the Spanish-

663

speaking children. Selecting a different rhyme that is familiar to more children and written originally in Spanish may improve the performance of the Spanish-speaking children. An alternative solution is to develop a novel rhyme for both English and Spanish children that is equivalent in length. Both groups would then be required to learn the rhyme to a correct criterion before performing the Divided Attention task. This would eliminate any previous knowledge or experience the children would have with other nursery rhymes. Overall, it was found that lower performance in Spanish-speaking children compared to English-speaking children was slight, though consistent with adults in this population according to prior reports (Anger et al., 1997, 2000). Due to the small number of children in the Latino group, statistical comparisons are not appropriate. Statistical comparisons indicate that the magnitude of the differences appear to be slight and less than that of the adults. This provides support for the relative importance of the impact of education on performance and suggests that the impact of cultural differences apparent in adults may create less impact on children. More testing is currently underway to continue the evaluation of this neurobehavioral test battery in a Latino population. ACKNOWLEDGEMENTS This publication was supported by a grant from the National Institute of Environmental Health Science (NIEHS, NIH R21ES08707-01) and the Center for Research on Occupational and Environmental Toxicology at Oregon Health & Science University.

REFERENCES Amler RW, Gibertini M, editors. Pediatric environmental neurobehavioral test battery. Atlanta, GA: US Department of Health and Human Services, Public Heath Service, Agency for Toxic Substances and Disease Registry, 1996. Amler RW, Gibertini M, Lybarger JA, Hall A, Kakolewski K, Phifer BL, Olsen KL. Selective approaches to basic neurobehavioral testing of children in environmental health studies. Neurotoxicol Teratol 1996;18:429±34. Anger WK, Letz RE, Chrislip DW, Frumkin H, Hudnell K, Russo JM, Chappell W, Hutchinson L. Neurobehavioral test methods for environmental health studies of adults. Neurotoxicol Teratol 1994;16:489±97. Anger WK, Rohlman DS, Sizemore OJ, Kovera CA, Gibertini M, Ger J. Human behavioral assessment in neurotoxicology: producing appropriate test performance with written and shaping instructions. Neurotoxicol Teratol 1996;18:371±9.

664

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665

Anger WK, Sizemore OJ, Grossmann SJ, Glasser JA, Letz R, Bowler R. Human neurobehavioral research methods: impact of subject variables. Environ Res 1997;73:18±41. Anger WK, Liang Y-X, Nell V, Kang S-K, Cole D, BazylewiczWalczak B, Rohlman DS, Sizemore OJ. Lessons learned: 15 years of the WHO-NCTB. Neurotoxicology 2000;21:837±46. Arcury TA, Quandt SA. Chronic agricultural chemical exposure among migrant and seasonal frame-workers. Soc Natl Res 1998;11:829±43. Baghurst PA, McMichael AJ, Wigg NR, Vimpani GV, Robertson EF, Roberts RJ, Tong SL. Environmental exposure to lead and children's intelligence at the age of 7 years: the Port Pirie cohort study. N Engl J Med 1992;327:1279±84. Baghurst PA, McMichael AJ, Tong SL, Wigg NR, Vimpani GV, Robertson EF. Exposure to environmental lead and visualmotor integration at age of 7 years: the Port Pirie cohort study. Epidemiology 1995;6:104±9. Bellinger D, Leviton A, Sloman J. Antecedents and correlates of improved cognitive performance in children exposed in utero to low levels of lead. Environ Health Perspect 1990;89: 5±11. Bernal EM. Increasing the interpretative validity and diagnostic utility of Hispanic children's scores on tests of achievement and intelligence. In: Seragica FC, Schwebel AI, editors. Mental health of ethnic minorities. New York: Praeger, 1990. pp. 108±38. Choi BH. The effects of methylmercury on the developing brain. Prog Neurobiol 1989;32:447±70. Cohen Hubal EA, Sheldon LS, Burke JM, McCurdy TR, Berry MR, Rigas ML, Zartarian VG, Freeman NCG. Children's exposure assessment: a review of factors influencing children's exposure and the data available to characterize and assess that exposure. Environ Health Perspect 2000;108:475±86. Dahl R, White RF, Weihe P, Sorensen N, Letz R, Hudnell HK, Otto DA, Grandjean P. Feasibility and validity of three computer-assisted neurobehavioral tests in 7-year-old children. Neurotoxicol Teratol 1996;18:413±9. Davidson PW, Myers GJ, Cos C, Axtell C, Shamlaye C, SloanReeves J, Cernichiari E, Needham L, Choi A, Wang Y, Berlin M, Clarkson TW. Effects of prenatal and postnatal methylmercury exposure from fish consumption on neurodevelopment: outcomes at 66 months of age in the Seychelles child development study. J Am Med Assoc 1998;280:701±7. Davidson PW, Weiss B, Myers GJ, Cory-Slechta DA, Brockel BJ, Young EC, Orlando M, Loiselle D, Palumbo D, Pittelli R, Sloan-Reeves J. Evaluation of techniques for assessing neurobehavioral development in children. Neurotoxicology 2000;21:957±72. Eshleman J, Daidhizar R. Life in a migrant camps for children: a hazard to health. J Cultural Diversity 1997;4:13±7. Fenske RA, Kissel JC, Lu C, Kalman DA, Simcox NJ, Allen EH, Keifer MC. Biologically-based pesticide dose estimates for children in an agricultural community. Environ Health Perspect 2000;108:515±20. Gamberale F. The use of behavioral and psychophysiological methods in the monitoring of health at the worksite. Environ Res 1993;60:87±97. Grandjean P, Weihe P, White RF, Debes F, Araki S, Yokoyama K, Murata K, Sùrensen N, Dahl R, Jùrgensen PJ. Cognitive deficit in 7-year-old children with prenatal exposure to methylmercury. Neurotoxicol Teratol 1997;19:417±28.

Helms JE. Why is there no study of cultural equivalence in standardized cognitive ability testing? Am Psychol 1992;47: 1083±101. Hill Jr, RH, Head SL, Baker S, Gregg M, Shealy DB, Bailey SL, Williams CC, Sampson EJ, Needham LL. Pesticide residues in urine in adults living in the United States: reference range concentrations. Environ Res 1995;71:99±108. Johnson BL, Grandjean P, Amler RW. Neurobehavioral testing and hazardous chemical sites. Neurotoxicol Teratol 1994; 16:485±7. Krasnegor NA, Otto DA, Bernstein JA, Burke R, Chappell W, Eckerman DA, Needleman HL, Oakley G, Rogan W, Terracciano G, Hutchinson L. Neurobehavioral test strategies for environmental exposures in pediatric populations. Neurotoxicol Teratol 1994;16:499±509. Kutz FW, Cook BT, Carter-Pokras OD, Brody D, Murphy RS, Selected pesticide residues and metabolites in urine from a survey of the US general population. J Toxicol Environ Health 1992;37:277±91. Letz R. NES2 User's Manual (Version 4.6). Winchester, MA: Neurobehavioral Systems, 1994. Loewenherz C, Fenske RA, Simcox NJ, Bellamy G, Kalman D. Biological monitoring of organophosphorus pesticide exposure among children of agricultural workers in central Washington state. Environ Health Perspect 1997;105:1344±53. Mahurin RK, Espino DV, Holifield EB. Mental status testing in elderly Hispanic populations: special concerns. Psychopharmacol Bull 1992;4:391±6. McKeown GE, Ruedy J, Neims A. Methylmercury exposure in northern Quebec. II. Neurologic findings in children. Am J Epidemiol 1983;118:470±9. Mines R, Gabbard S, Steirman A. A profile of US farm workers: demographic, household composition, income and use of services. Based on data from the National Agricultural Workers Survey (NAWS). Washington, DC: US Department of Labor, Office of the Assistant Secretary for Policy, prepared for the Commission on Immigration Reform, 1997. Moses M, Johnson ES, Anger WK, Burse VW, Horstman SW, Jackson RJ, Lewis RG, Maddy KT, McConnell R, Meggs WJ, Zahm SH. Environmental equity and pesticide exposure. Toxicol Indus Health 1993;9:913±59. Mott L. The disproportionate impact of environmental health threats on children of color. Environ Health Perspect 1995;(Suppl 6):33±5. National Research Council. Pesticides in the diets of infants and children. Washington, DC: National Academy Press, 1993. Naveh-Benjamin M, Ayres TJ. Digit span, reading rate, and linguistic relativity. Quarterly J Exp Psychol 1986;38: 739±51. Olden K, Guthrie J. Children's health: a mixed review. Environ Health Perspect 2000;108:250±1. Otto DA, Skalik I, House DE, Hudnell HK. Neurobehavioral evaluation system (NES): comparative performance of second-, fourth-, and eighth-grade Czech children. Neurotoxicol Teratol 1996;18:421±8. Rohlman DS, Sizemore OJ, Anger WK, Kovera CA. Computerized neurobehavioral testing: techniques for improving test instructions. Neurotoxicol Teratol 1996;18:407±12. Rohlman DS, Gimenes LS, Ebbert C, Anger WK, Bailey SR, McCauley L. Smiling faces and other rewards: using the

D.S. Rohlman et al. / NeuroToxicology 22 (2001) 657±665 behavioral assessment and research system (BARS) with unique populations. Neurotoxicology 2000;21:973±8. Reed KJ, Jimenez M, Freeman NCG, Lioy PJ. Quantification of children's hand and mouthing activities through a videotaping methodology. J Exposure Anal Environ Epidemiol 1999;9: 513±20. Schettler T, Stein J, Reich F, Valenti M, Wallinga D. Harm's way: toxic threats to child development. Cambridge, MA: Greater Boston Physicians for Social Responsibility, 2000. Shamlaye CF, Marsh DO, Myers GJ, Cox C, Davidson PW, Choisy O, Cernichiari E, Choi A, Tanner MA, Clarkson TW. The Seychelles child development study on neurodevelopmental outcomes in children following in utero exposure to methylmercury from a maternal fish diet: background and demographics. Neurotoxicology 1995;16:597±612.

665

Sizemore OJ, Amler RW. Characteristics of ATSDRs adult and pediatric environmental neurobehavioral test batteries. Neurotoxicology 1996;17:229±36. Task Force of Environmental Cancer. Baker SR, Wilkinson CF, editors. Task Force of Environmental Cancer and Heart and Lung Disease. The effects of pesticides on human health. Princeton: Princeton Scientific Publishing Co., Inc., 1990. Williamson A. Regional issues in neurobehavioral testing: an overview. In: Russell RW, Flattau RE, Pope AM, editors. Behavioral measures of neurotoxicity. Washington, DC: National Academy Press, 1990. pp. 337±43. Zartarian VG, Streicker J, Rivera A, Cornejo CS, Molina S, Valadez OF, Leckie JO. A pilot study to collect micro-activity data of 2- to 4-year-old farm labor children in Salinas Valley, California. J Exposure Anal Environ Epidemiol 1995;5:21±34.