Do experimental measures of word learning predict vocabulary development over time? A study of children from grade 3 to 4

Learning and Individual Differences 26 (2013) 1–8

Contents lists available at SciVerse ScienceDirect

Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif

Do experimental measures of word learning predict vocabulary development over time? A study of children from grade 3 to 4 Anna S. Gellert ⁎, Carsten Elbro Department of Scandinavian Studies and Linguistics, University of Copenhagen, Denmark

a r t i c l e

i n f o

Article history: Received 22 May 2012 Received in revised form 11 March 2013 Accepted 18 April 2013 Available online xxxx Keywords: Vocabulary Word learning Testing Prediction Validity

a b s t r a c t Experimental measures have been used in several studies to simulate real word learning over time. The main purpose of the present study was to investigate whether such measures actually predict vocabulary development over time. Ninety grade 3 children were given experimental measures designed to assess phonological and semantic aspects of learning of novel words together with traditional measures of vocabulary and general cognitive ability. The vocabulary measures were administered nine months later to the same children in grade 4. Experimental measures of phonological aspects of word learning were found to contribute to the prediction of the children's vocabulary in grade 4 over and above their vocabulary and general cognitive ability in grade 3. A measure of semantic aspects of word learning did not contribute further to the prediction of vocabulary growth. The study provides initial evidence of the predictive validity of phonological measures of word learning. © 2013 Elsevier Inc. All rights reserved.

1. Introduction Vocabulary size is causally determined by word learning, and a measure of word learning can be assumed to gauge the operation of processes that lead to increases in vocabulary size (Gupta & Tisdale, 2009a). In naturalistic learning contexts, individual differences in vocabulary development are likely to be influenced by variable amounts of exposure to new words in a given language, and this may override genuine differences in word learning abilities (Masoura & Gathercole, 2005). However, experimental measures of novel word learning are assumed to minimise such differences in learning opportunities (e.g., Horton-Ikard & Weismer, 2007; Masoura & Gathercole, 2005). Accordingly, several researchers have developed experimental simulations or analogues of natural vocabulary acquisition to assess individual differences in word learning abilities (e.g., Gathercole, Hitch, Service, & Martin, 1997; Jarrold, Thorn, & Stephens, 2009) or word learning potential (e.g., Burton & Watkins, 2007; Kapantzoglou, Restrepo, & Thompson, 2012). If experimental measures are in fact adequate simulations or analogues of real word learning and do tap individual differences in the ability to learn new words, such measures can be assumed at least to contribute to the prediction of individual differences in vocabulary development over time. However, to the best of our knowledge, this assumption has not previously been tested, although evidence of the predictive validity of experimental measures of word learning would strengthen the rationale ⁎ Corresponding author at: Department of Scandinavian Studies and Linguistics, 120 Njalsgade, DK-2300 Copenhagen S, Denmark. Tel.: +45 304 83240. E-mail address: [email protected] (A.S. Gellert). 1041-6080/$ – see front matter © 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.lindif.2013.04.006

for their use in research into language acquisition and their inclusion in educational and clinical assessments. If children's performance on measures of word learning does indeed predict the natural process of vocabulary development over time, such measures could be used to identify children with difficulties in acquiring vocabulary despite normal exposure to language. Consequently, the main purpose of the present study was to test the predictive validity of an example of an experimental word learning paradigm. The chosen paradigm comprised a number of aspects of word learning which are introduced below. Learning a new word involves forming an internal representation of the sequence of sounds that comprise the word (its phonology), an internal representation of the meaning (its semantics), and a link between the two representations (Gupta & Tisdale, 2009b). However, the more specific relationship between phonological and semantic aspects of word learning is a source of controversy in research into children's language development and the nature of word learning difficulties. Thus, some researchers have argued that word learning difficulties are primarily due to phonological deficits, such as limited phonological storage capacity in working memory (e.g., Gathercole, 2006; Gathercole & Baddeley, 1990). Other researchers, however, have proposed that deficits in semantic learning skills may contribute directly and independently to word learning difficulties (Nash & Donaldson, 2005; Nation, Snowling, & Clarke, 2007). Some experimental studies have only included measures that focused on phonological aspects of new word learning, typically in the shape of tasks involving visual–verbal paired-associate learning (e.g., de Jong, Seveke, & van Veen, 2000; Morra & Camba, 2009). In these tasks, novel words are presented orally along with pictures.

2

A.S. Gellert, C. Elbro / Learning and Individual Differences 26 (2013) 1–8

The presentation is followed by a number of learning trials in which the participant is asked to provide the novel words when presented with the pictures and is given corrective feedback if he is unable to provide the words. A phonological retrieval cue such as the first syllable of the word may be provided (Mayringer & Wimmer, 2000). Such tasks do not directly assess the acquisition of semantic aspects or the linking of these aspects with the phonological forms of new words. However, other groups of researchers have devised paired associate learning tasks to assess learning of semantic information in association with phonological forms. For example, Aguiar and Brady (1991) and Nation et al. orally presented eight to ten-year-old children not only with novel words and pictures, but also with corresponding definitions. Following learning trials, the children were asked to recall the novel words when presented with the definitions of these words and later to recall the definitions when provided with the novel words. The most direct measure of semantic learning, in which the children had to recall definitions of the novel words, did not correlate significantly with the measures of phonological learning. The independence between phonological and semantic measures of word learning found in these studies allows for the hypothesis that semantic measures may add independent variance to the prediction of children's vocabulary development over time beyond the contribution from phonological measures of word learning. This hypothesis was tested in the present study. Delayed measures of recognition and recall of newly learned words have been included in some of the previous studies to investigate whether children can still remember these words some days after the initial learning session. Thus, between one and three weeks after the initial presentation of the new items, Aguiar and Brady gave their participants a recognition task similar to a task administered in the first test session. The correlation between the immediate and delayed measures of recognition was fairly high (.6), especially when considering that strong ceiling effects on the delayed measure were reported. However, Nation et al. found that children with poor reading comprehension performed worse than non-impaired children on a delayed test of recall of the novel words one week after the training session, although there was no significant difference between these groups on the measure of immediate recall. Similarly, Ricketts, Bishop, and Nation (2008) found that while poor comprehenders showed equivalent semantic learning to controls immediately after exposure to the meaning of novel words, this knowledge was not well retained over time. These results indicate that delayed measures may provide relevant information about children's word learning abilities in addition to the information obtained from the immediate measures. Thus, the degree to which children can still remember new words some days after the initial learning may add to the prediction of long-term vocabulary development. Still, the actual long-term vocabulary development was not investigated in the previous studies. Therefore, it remains to be seen whether delayed measures will add independent variance to the prediction of children's vocabulary development over time beyond the contribution from immediate measures of word learning. We address this question in the present study. Children's abilities to learn new words may to some degree depend on their present level of language knowledge. Indeed, measures of both phonological and semantic aspects of word learning have been found to correlate moderately to strongly with concurrent measures of vocabulary (Aguiar & Brady, 1991; Ricketts, Bishop, Pimperton, & Nation, 2011) and with a broader composite measure of language skills (Nation et al., 2007). Part of the explanation for this concurrent relationship may be that individuals employ existing vocabulary knowledge to support their learning of new words. Similarly, children's abilities to learn new words may to some extent depend on their general cognitive abilities. Thus, moderate correlations have been found between measures of word learning and concurrent measures of non-verbal reasoning (Gathercole et al., 1997; Nation et al., 2007). Therefore, the predictive power of word learning measures may be considerably

reduced when the children's initial level of general cognitive ability is taken into account. We also investigate this possibility in the present study. Finally, the predictive power of word learning measures may depend on the nature of the vocabulary measures used to assess the development over time. Concurrent correlations around .6–.7 between measures of expressive and receptive vocabulary have been reported from previous studies of children (Ouellette, 2006; Sideridis & Simos, 2010) — indicating that these aspects of vocabulary are related but not identical. The present study aimed to assess development in receptive as well as expressive vocabulary and therefore included both types of measures. In sum, the main purpose of the study was to investigate whether experimental measures of word learning contribute to the prediction of children's vocabulary development over time. More specifically, the research questions were the following: 1. Do phonological aspects of word learning contribute to the prediction of vocabulary development over time over and above general cognitive ability? 2. Do semantic aspects of word learning contribute over and above phonological aspects of word learning to the prediction of vocabulary development? In addition, do phonological aspects of word learning contribute over and above semantic aspects? 3. Do delayed measures of word learning contribute over and above immediate measures of word learning to the prediction of vocabulary development? 2. Method 2.1. Design Experimental measures of word learning were administered to children in grade 3 to investigate whether these measures contributed to the prediction of vocabulary development from grade 3 to 4. To control for general cognitive ability, a test of non-verbal reasoning was also administered in grade 3. 2.2. Participants Ninety-nine children from six Danish schools located in a suburb of Copenhagen were given parental permission and took part in the study in the middle of grade 3. One boy was later excluded from the study because he was diagnosed with an autism spectrum disorder. According to their teachers, none of the remaining 98 children had diagnoses which could affect their performance on the measures included in the study. Nine months later, 95 of these children were tested again. By then two children had moved to other school districts, and one student refused to participate. For five of the remaining 95 children, part of data were not collected or lost. Analyses were based on the 90 children (57 girls and 33 boys) from whom full data sets were available. Their mean age was 9 years, 7 months (SD = 4 months) at the first time point. Eleven of these 90 children spoke Danish as a second language. The regression analyses reported later in this paper were performed with and without these 11 children. As no significant differences between the results were found, the children with Danish as a second language were kept in the sample. 2.3. Materials 2.3.1. Experimental word learning The measures of word learning were based on a paradigm used in two previous studies (Aguiar & Brady, 1991; Nation et al., 2007). Six novel words were constructed for this study. Each novel word was semantically defined by four attributes (a noun and three adjectives),

A.S. Gellert, C. Elbro / Learning and Individual Differences 26 (2013) 1–8

for example Mafyk — a green, dotted, poisonous snake. Coloured illustrations for each word were mounted onto cards. The novel words and their definitions along with one of the illustrations are provided in Appendix A. 2.3.1.1. Initial presentation. In each of two blocks (A + B), each child was initially presented with three pictured animals and their names and definitions (similarly to the procedure in the original study by Aguiar and Brady). The child was shown the coloured drawings of three animals one at a time and asked to repeat and remember the names and definitions of these animals. The child only received one exposure of each of the animals along with one presentation of the corresponding name and definition. To increase sensitivity to differences across the whole range of the children's abilities, the selected novel words in block B were phonologically more complex than in block A. After the initial presentation, the following measures were administered: Vocabulary training, Definition knowledge, Immediate recall and Immediate recognition. Approximately one week later, measures of Delayed recall and Delayed recognition were administered. Each of these measures is described below. Fig. 1 provides an overview of the procedure. 2.3.1.2. Vocabulary training. The purpose of this task was to measure ease of acquisition of phonological forms of novel words. Similarly to Nation et al., we used vocabulary training as the primary measure of phonological learning. After the initial presentation of one block with three pictured animals and their names and definitions, the child was shown the animals one at a time in a randomised order and asked to provide the name of each of them (i.e., the novel word). If the child was unable to name the animal when presented, corrective feedback was given. In both blocks, the examiner thus provided the full correct response if the child did not name the animal correctly on his own. In block B with the phonologically more complex novel words, the examiner also gave the first syllable as a prompt before providing the child with the full correct answer. In each block, the child was given a maximum of 14 trials to learn the three novel words. The criterion was correct naming of all three animals on two consecutive trials without prompting. A score of one was given each time the child provided an animal's name correctly without prompting. Subsequent correct

responses were assumed once the criterion was met. Maximum score for the two blocks together was 84. 2.3.1.3. Definition knowledge. Following Nation et al., we used definition knowledge as the primary measure of semantic learning. After the vocabulary training phase of each block, the examiner pronounced each of the three novel words (e.g., Mafyk) and asked the child to tell which animal it was and to say as much as he remembered about the animal. If the child answered, “It is a green snake”, the examiner prompted him to mention more attributes by saying, “Yes, and do you remember more about Mafyk?”. The number of correct attributes supplied for each of the novel words formed the score. Thus, the child was given a score of four if he provided the correct animal and the three associated adjectives. Maximum score for the two blocks together was 24. 2.3.1.4. Immediate recall. The purpose of this task was to measure immediate recall of phonological forms of novel words. Following the measure of definition knowledge, the examiner read each of the definitions of the three words aloud and asked the child to provide the name for each one. The score was the number of correct names (novel words). 2.3.1.5. Immediate recognition. This task was included to measure immediate recognition of phonological forms of new words. After the measure of immediate recall, the child was presented with pictures of the three target animals and three distracters (listed in Appendix A). The examiner asked the child to point to a particular target (e.g., Mafyk). No corrective feedback was provided. The pictures were shuffled and the procedure repeated for the other novel words. The score was the number of animals correctly chosen. 2.3.1.6. Delayed recall. A test of delayed recall was administered approximately one week after the initial presentation of the novel words. The examiner read each of the definitions of the six words aloud and asked the child to provide the name for each one. The score was the number of correct names. 2.3.1.7. Delayed recognition. The child was presented with the six target animals and three distracters. The examiner asked the child to point to a particular target. No corrective feedback was provided.

Phase

Hear/see

Response

Initial presentation

Picture of animal + “novel

“Novel word” +

word” + “definition”

“definition”

Picture of animal

“Novel word” (with

Vocabulary training

corrective feedback) Definition knowledge

3

“Novel word”

“Attributes” (max of 4)

Immediate recall

“Definition”

“Novel word”

Immediate recognition

Target pictures +

Select targets

distracters 1 week between immediate and delayed tasks Delayed recall

“Definition”

“Novel word”

Delayed recognition

Target pictures +

Select targets

distracters Fig. 1. Procedure for word learning tasks.

4

A.S. Gellert, C. Elbro / Learning and Individual Differences 26 (2013) 1–8

The pictures were shuffled and the procedure repeated for the other novel words. The score was the number of animals correctly chosen. 2.3.2. Vocabulary Two vocabulary tests were administered individually in grades 3 and 4 to assess vocabulary development over time. 2.3.2.1. Receptive vocabulary. A Danish translation of the Peabody Picture Vocabulary Test (Dunn & Dunn, 1981) has previously been used with children (Elbro, Borstrøm, & Petersen, 1998; Poulsen, 2011), but has not been standardised. On the basis of previous results, the sequence of the items was revised (Nielsen, 2008). This revised version was used for the present study. The procedure described in the manual was followed. The score was the number of correct responses. 2.3.2.2. Expressive vocabulary. Since no suitable Danish test of expressive vocabulary for the age group existed, the first author constructed a new picture naming test for this study. Words for this test were selected from school books about science, history and social studies. The test had 54 items (see Appendix B). The stimuli were colour photos of objects presented on six sheets including common objects like skeleton and rarer ones like geyser. The child was required to name each picture with one word and encouraged to guess if he was not certain. The score was the number of correct responses. 2.3.3. General cognitive ability To test non-verbal reasoning, Raven's Coloured Progressive Matrices A, AB and B (Raven, Court, & Raven, 1990) were administered in group settings. The procedure described in the manual was followed. The score was the number of correct responses. 2.4. Procedure The measures of vocabulary and word learning were administered individually in quiet parts of the schools by the first author and trained test assistants. The individual assessments were recorded. In grade 3, the individual testing was conducted over two sessions. In the first test session, the two blocks of word learning measures (A + B) were separated by the measure of expressive vocabulary. The second test session included the measure of receptive vocabulary and the delayed measures of phonological recall and recognition. The delayed measures were to be administered approximately one week after the first test session. However, in several cases it was not possible to test the children at the planned time. Therefore, results on the delayed measures were included only for the children to whom these measures were administered between one and two weeks after the first test session (n = 68). The extent of this lag between the testing points (one to two weeks) did not impact significantly on performance. 3. Results Table 1 provides descriptive statistics. Most of the score distributions were approximately normal. However, a ceiling effect appeared on the word learning measure of immediate recognition, while a floor effect was found on the measure of delayed recall. As shown in Table 1, the measures of vocabulary training, immediate recall, vocabulary and general cognitive ability all had a high internal consistency (Cronbach's α > .7). The internal consistency of the remaining measures of word learning (definition knowledge, immediate recognition, delayed recall and delayed recognition) was low to moderate (Cronbach's α = .49–.64). The average number of trials given in the vocabulary training phase was 8.62 (SD = 4.66; range = 2–14) for block A and 12.71 (SD = 2.50; range = 4–14) for block B.

Table 1 Descriptive statistics for the measures of experimental word learning, vocabulary, and general cognitive ability. Measure

Grade

Max. score

Mean

SD

Range

α

Vocabulary training Definition knowledge Immediate recall Immediate recognition Delayed recalla Delayed recognitiona Receptive vocabulary Receptive vocabulary Expressive vocabulary Expressive vocabulary General cognitive ability

3 3 3 3 3 3 3 4 3 4 3

84 24 6 6 6 6 112 112 54 54 36

52.3 18.5 3.6 5.7 1.2 4.5 45.3 48.3 29.1 33.2 30.1

15.8 3.3 1.6 0.7 1.3 1.7 9.3 9.5 8.0 7.8 5.1

14–82 7–24 0–6 2–6 0–5 0–6 17–68 16–74 6–44 11–47 13–36

.95 .64 .57 .49 .56 .75 .71 .89 .87

Note: a Descriptive statistics for the delayed measures of recall and recognition are based on data from 68 participants only. For the other measures, descriptive statistics are based on data from 90 participants.

On average, the children performed significantly higher on the measure of receptive vocabulary in grade 4 (M = 48.26, SE = 1.00) than in grade 3 (M = 45.29, SE = .98), t(89) = −4.0, p b .001. Similarly, their average expressive vocabulary was significantly higher in grade 4 (M = 33.23, SE = .82) than in grade 3 (M = 29.13, SE = .84), t(89) = −10.39, p b .001. Table 2 shows the intercorrelations among the measures. Long-term test–retest reliability of the vocabulary measures was good as indicated by the grade 3 to 4 correlations (.72 and .89 for receptive and expressive vocabulary, respectively). The measures of receptive and expressive vocabulary correlated .64 in grade 3 and .69 in grade 4. The strength of these correlations is similar to the ones reported in previous studies with measures of children's receptive and expressive vocabulary (Ouellette, 2006; Sideridis & Simos, 2010) and indicates a fairly high level of concurrent validity. As shown in Table 2, the intercorrelations between the measures of word learning ranged from non-significant to high. The size of some of these correlations is probably restricted due to the reported ceiling and floor effects on the measure of immediate recognition and delayed recall. However, the measures of vocabulary training and immediate recall correlated .71. To simplify the following analyses, the scores of these two measures were normalised, and a composite score was calculated as the average of the two normalised scores. The same was done for the scores of the measures of delayed recall and recognition which correlated .60. In grade 3, most of the correlations between the measures of vocabulary and word learning were low to moderate. These correlations indicate some independence between the children's performance on concurrent measures of word learning and vocabulary. Hierarchical regression analyses were conducted to examine whether the experimental measures of word learning contributed to the prediction of vocabulary development. In each of these analyses, one of the grade 4 measures of vocabulary served as the dependent variable. Only word learning measures which correlated significantly with the relevant vocabulary measure in grade 4 (cf. Table 2) were included as independent variables in the regression analyses. Table 3 provides a summary of the results from a series of hierarchical regression analyses with the grade 4 measure of receptive vocabulary as the dependent variable. In model 1 (Table 3), receptive vocabulary in grade 3 was entered in the first step of the regression analysis. To investigate whether phonological measures of word learning contribute to the prediction of vocabulary development, the composite measure of vocabulary training and recall was entered in the second step and was found to account for an additional significant 6% of the variance in receptive vocabulary in grade 4. The measure of definition knowledge was entered in the third step to investigate whether this primary measure

A.S. Gellert, C. Elbro / Learning and Individual Differences 26 (2013) 1–8

5

Table 2 Intercorrelations among measures. 1 1. Vocabulary training G3 2. Definition knowledge G3 3. Immediate recall G3 4. Immediate recognition G3a 5. Delayed recall G3a,b 6. Delayed recognition G3b 7. Vocabulary training and recall (composite) 8. Delayed recall and recognition (composite)b 9. Receptive vocabulary G3 10. Receptive vocabulary G4 11. Expressive vocabulary G3 12. Expressive vocabulary G4 13. General cognitive ability G3

2

.31⁎⁎ .71⁎⁎ .37⁎⁎ .29⁎ .34⁎⁎ .93⁎⁎ .32⁎⁎

.33⁎⁎ .17 .31⁎ .40⁎⁎ .34⁎⁎ .46⁎⁎ .27⁎ .37⁎⁎

.12 .31⁎⁎ .20 .28⁎⁎ .25⁎

.20 .19 .05

3

4

.36⁎⁎ .29⁎ .25⁎ .93⁎⁎ .31⁎

.31⁎ .47⁎⁎ .44 .43⁎⁎

.05 .26⁎ .14 .19 .19

.05 .15 .14 .13 .29⁎⁎

5

6

.60⁎⁎ .27⁎ .87⁎⁎ .38⁎⁎ .41⁎⁎ .32⁎⁎ .38⁎⁎

.35⁎⁎ .43⁎⁎ .48⁎⁎ .38⁎⁎ .48⁎⁎ .49⁎⁎

.08

.08

7

8

9

10

11

12

.34⁎⁎ .09 .31⁎⁎ .18 .25⁎ .30⁎⁎

.43⁎⁎ .41⁎⁎ .43⁎⁎ .41⁎⁎ .18

.72⁎⁎ .64⁎⁎ .64⁎⁎ .30⁎⁎

.66⁎⁎ .69⁎⁎ .37⁎⁎

.89⁎⁎ .38⁎⁎

.35⁎⁎

Note: Data are excluded pairwise to maintain statistical power. a Spearman's ρ are computed for the correlations coefficients with measures of immediate recognition and delayed recall because of ceiling and floor effects. The remaining correlation coefficients are Pearson's r. b Correlations with the delayed measures of recall and recognition are based on data from 68 participants only. The remaining correlations are based on data from 90 participants. ⁎ p b .05 (2-tailed). ⁎⁎ p b .01 (2-tailed).

of semantic learning contributed over and above the phonological measures of word learning. This was not found to be the case, as the additional contribution of definition knowledge was non-significant. Neither did definition knowledge add significantly to the prediction when only one of the separate phonological measures (vocabulary training or recall) was controlled for. On the other hand, when definition knowledge was entered before vocabulary training and recall (model 2), the composite measure accounted for an additional significant 4% of the variance in receptive vocabulary in grade 4. The analysis in model 3 (Table 3) was conducted to examine whether the delayed measures of word learning contributed to vocabulary development beyond immediate measures. This analysis was based on data from the 68 participants to whom the delayed measures of word learning were administered between one and two weeks after the first test session. Thus, the statistical power was reduced for this analysis compared to the other analyses reported in Table 3. Inclusion of the composite measure of delayed recall and recognition did not add significantly to the prediction of receptive vocabulary in grade 4. Neither did any of the separate delayed measures (recall or recognition) contribute

Table 3 Prediction of grade 4 receptive vocabulary from grade 3 vocabulary, word learning, and general cognitive ability. Standardised regression coefficients (β's) are provided for the final models. Model

Step

G3 measures

R2

ΔR2

F change

β

1a

1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 4

Receptive vocabulary Vocabulary training and recall Definition knowledge Receptive vocabulary Definition knowledge Vocabulary training and recall Receptive vocabulary Vocabulary training and recall Delayed recall and recognition General cognitive ability Receptive vocabulary Vocabulary training and recall General cognitive ability Receptive vocabulary Expressive vocabulary Vocabulary training and recall

.51 .57 .58 .51 .55 .58 .50 .59 .59 .13 .54 .58 .13 .54 .59 .63

.51 .06 .01 .51 .03 .04 .50 .09 .00 .13 .40 .05 .13 .40 .05 .04

92.16⁎⁎ 12.33⁎⁎ 2.37 92.16⁎⁎ 6.37⁎ 8.01⁎⁎ 65.12⁎⁎ 13.05⁎⁎

.67⁎⁎ .21⁎⁎ .12 .67⁎⁎ .12 .21⁎⁎ .64⁎⁎ .29⁎⁎

0.23 13.51⁎⁎ 75.79⁎⁎ 9.68⁎⁎ 13.51⁎⁎ 75.79⁎⁎ 11.34⁎⁎ 8.38⁎⁎

.05 .12 .66⁎⁎ .22⁎⁎

2

3b

4a

5

Note: a n = 90. b n = 68. ⁎ p b .05. ⁎⁎ p b .01.

.06 .50⁎⁎ .29⁎⁎ .20⁎⁎

additionally. Therefore, the delayed measures of word learning were not included in the subsequent analyses. The analysis in model 4 (Table 3) was performed to examine whether the children's performance on word learning measures contributed to the prediction of receptive vocabulary development after general cognitive ability was accounted for. The analysis with the measure of general cognitive ability entered in the first step showed that this was the case, as vocabulary training and recall still accounted for an additional significant 5% of the variance in receptive vocabulary in grade 4. However, if the measure of receptive vocabulary was not perfectly reliable, any measure related to vocabulary might add to the prediction of the vocabulary development over time. To control for this possibility, an even stronger control for vocabulary was made by including both expressive and receptive vocabulary in grade 3 into the model before the composite measure of vocabulary training and recall (model 5 in Table 3). Expressive vocabulary did account for significant variance in receptive vocabulary in grade 4 beyond receptive vocabulary in grade 3. However, even after controlling for grade 3 expressive vocabulary and general cognitive ability, word learning remained a significant predictor adding 4% to the variance in receptive vocabulary in grade 4. Similar analyses with grade 4 expressive vocabulary as the dependent variable were carried out. However, the correlation between the grade 3 and the grade 4 measure of expressive vocabulary was nearly perfect (r = .89, cf. Table 2). This high stability made further prediction of changes in expressive vocabulary difficult and use of hierarchical regression analyses problematic. Still, tentative regression analyses were conducted and indicated that the composite measure of vocabulary training and recall added a significant 1% to the prediction of the development in expressive vocabulary after control for the strong autoregressive effect (cf. Table 4). The measure of definition knowledge did not add further to the prediction. The small additional contribution from the composite measure of vocabulary training and recall to the prediction of the development in expressive vocabulary remained significant even after controlling for general cognitive ability and receptive vocabulary. The delayed measures of word learning did not add further to the prediction. 4. Discussion The main purpose of the study was to investigate whether children's performance on experimental measures of word learning, as employed by Aguiar and Brady and Nation et al., contributes to the prediction of

6

A.S. Gellert, C. Elbro / Learning and Individual Differences 26 (2013) 1–8

Table 4 Prediction of grade 4 expressive vocabulary from grade 3 vocabulary, word learning, and general cognitive ability. Standardised regression coefficients (β's) are provided for the final models. Model 1

a

2b

3a

4a

Step 1 2 1 2 3 1 2 3 1 2 3 4

G3 measures Expressive vocabulary Vocabulary training and recall Expressive vocabulary Vocabulary training and recall Delayed recall and recognition General cognitive ability Expressive vocabulary Vocabulary training and recall General cognitive ability Expressive vocabulary Receptive vocabulary Vocabulary training and recall

R2 .79 .80 .79 .80 .80 .12 .79 .80 .12 .79 .80 .81

ΔR2

F change

β

.79 .01 .79 .01 .00 .12 .67 .01 .12 .67 .01 .01

328.06⁎⁎ 4.89⁎ 236.94⁎⁎

.87⁎⁎ .11⁎ .86⁎⁎ .08 .02 −.01 .87⁎⁎ .11⁎

2.32 .13 12.45⁎⁎ 273.69⁎⁎ 4.77⁎ 12.45⁎⁎ 273.69⁎⁎ 3.75 5.20⁎

−.02 .79⁎⁎ .13⁎ .11⁎

Note: a n = 90. b n = 68. ⁎ p b .05. ⁎⁎ p b .01.

development in vocabulary over time. The findings were positive. The first research question was whether measures of phonological aspects of word learning contribute to the prediction of vocabulary development over and above general cognitive ability. This was found to be the case. Thus, the children's abilities to repeat and recall phonological forms of novel words contributed to the prediction of vocabulary development over time even after control for general cognitive ability. In response to the second research question, a measure designed to assess semantic aspects of word learning did not contribute significantly over and above measures of abilities to repeat and recall phonological forms. However, these phonological measures of word learning contributed uniquely to the prediction of vocabulary development even after control for the semantic measure. Finally, in response to the third research question, delayed measures of word learning did not add to the prediction of vocabulary development over time after the contribution from the immediate measures of word learning was controlled for. The vocabulary training and recall tasks required the children to repeat and recall the phonological forms of novel words. Evidence of the predictive validity of these measures was provided by the demonstration of their contribution to the prediction of vocabulary development. The finding that they contributed to the prediction even after control for general cognitive ability supports the notion that they do tap specific abilities related to natural word learning. The unique contribution was small. It is not possible to say whether the modest size of the effect is a real finding or it should be attributed to the strong autoregressive effects of the vocabulary measures (see the limitations below). Surprisingly, the task which was assumed to be the most direct measure of semantic aspects of word learning did not contribute to the prediction of vocabulary development over and above the tasks requiring the children to repeat and recall the phonological forms of the novel words. In the semantic task, children were asked to recall the attributes by which the examiner initially had defined the novel words. Although the task was designed to be a measure of semantic learning, it probably also tapped phonological aspects of learning to some extent as the children had to remember the links between the phonological forms of the novel words and the definitions of the words. Similarly, the vocabulary training and recall tasks probably also tapped children's knowledge of links between phonology and semantics to some extent rather than just learning of phonological aspects of novel words. This overlap between semantics and phonology in the tasks may provide part of the explanation why the primary semantic task of definition knowledge did not contribute independently to the prediction of vocabulary development.

The delayed measures of word learning also did not add to the prediction of vocabulary development over time. However, the range in scores was limited because of a small number of items as well as floor and ceiling effects. The delayed measures were adapted from previous studies of children of approximately the same age (Aguiar & Brady, 1991; Nation et al., 2007), and descriptive statistics from these studies provide evidence of a similar problem. In any case, the lack of an additional contribution from the delayed measures should be interpreted with caution, as the statistical power was limited for the analyses including these measures due to problems with inclusion of the participants at the planned time for the second test session. Unfortunately, such problems appear to be a common challenge for researchers collecting data in schools. Furthermore, similarly to the studies by Aguiar and Brady and Nation et al., we did not include any delayed measures specifically designed to assess semantic aspects of word learning in the present study. Possibly, performance on such delayed measures might have added uniquely to the prediction of vocabulary development over time. An obvious limitation of the study was the relatively short period (nine months) for the assessment of the children's vocabulary development. This short period made the stability of the vocabulary scores at the two points in time very high. Thus, the autoregressive effects were very strong, especially for the expressive measure of vocabulary, and made it difficult for the experimental measures of word learning to contribute any further to the prediction of the vocabulary development. Actually, the finding that phonological measures of word learning contributed significantly to the development of vocabulary within this limited period is remarkable. To address some of the other limitations, we suggest that the word learning measures should be revised or replaced in future studies. Thus, some of the participants in the present study became tired because of the high number of trials given when the measure of vocabulary training was administered in the two blocks. Higher levels of motivation and more reliable results may be obtained if participants were instead given several parallel blocks with fewer trials. Moreover, computerising the tasks may improve the motivation of children. This would also make it possible to examine the reaction times which would be useful when analysing performance on measures with ceiling effects. Furthermore, future studies could include other measures of semantic aspects of word learning (e.g., semantic analysis or inference) to investigate whether such measures would contribute to the prediction of vocabulary development over and above primarily phonological measures of word learning. However, it may not be possible to separate semantic aspects of word learning entirely from phonological aspects. This reflects natural word learning as it is unusual for children to know phonological strings without any related semantic information whatsoever. Similarly, children are unlikely to know semantic word information without any phonological representation. Furthermore, the characteristics of the tests which were used to assess vocabulary development over time should be considered. First, none of these tests were norm-referenced, as standardised language tests unfortunately are scarce in Denmark. Second, both of the vocabulary tests assessed ‘breadth’ of vocabulary (how many words individuals know) rather that ‘depth’ (how well individuals know those words) (cf. Ouellette, 2006; Tannenbaum, Torgesen, & Wagner, 2006). It is possible that children's performance on the primary semantic measure of word learning would have been more closely linked to their performance on a vocabulary test which required more detailed semantic information about the words. In future research, it would be relevant to investigate the predictive validity of word learning measures with children of different ages and abilities. For example, phonological measures of word learning could be administered to typically developing pre-school children and to children who were suspected of having language learning disabilities. The children's actual language development should be followed for some years to investigate whether such word learning

A.S. Gellert, C. Elbro / Learning and Individual Differences 26 (2013) 1–8

measures would add to the early classification of children with and without language disabilities over and above traditional language measures. In conclusion, the present study was conducted to investigate the predictive validity of an experimental word learning paradigm which has previously been used to examine individual differences in children's word learning abilities. Within this paradigm, children's abilities to repeat and recall the phonological forms of novel words predicted a significant part of the vocabulary development over a period of nine months. Thus, initial evidence of the predictive validity of some experimental measures of word learning was provided. However, a measure which was assumed to assess semantic aspects of word learning, did not contribute additional variance. We suggest that future studies should include other measures of semantic aspects of word learning and investigate the predictive validity of such measures while taking care of other limitations of the present study. Acknowledgements This study was supported by a grant from The Danish Research Council to the first author (273-08-0438). The authors are also grateful to ten MA students for their help with data collection, and to the children and their schools for their participation. Appendix A Experimental word learning The novel words and their definitions Block

Novel word (name)

Definition

A

Goni Salu Fybe Targeli Pimut Mafyk

Sick, green, red-haired fish Big, brown, dangerous dog Orange, striped, sleeping cat Fat, white, spotted cow Old, grey, singing bird Green, dotted, poisonous snake

B

Example of illustration Salu — a big, brown, dangerous dog:

Distracters Illustrations of three animals (sheep, monkey, and horse). Appendix B Expressive vocabulary Expected answers 1. racket (ketsjer) 2. wheelbarrow (trillebør)

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53. 54.

7

globe (globus) shield (skjold) magnifying glass (lup/forstørrelsesglas) skeleton (skelet) zimmer frame (rollator) gun/shotgun/rifle (gevær/riffel) whisk (piskeris) greenhouse/hothouse (drivhus) well (brønd) shell (konkylie) basket (kurv) gallows (galge) skateboard (skateboard) daddy-longlegs/crane fly (stankelben) mole (muldvarp) skull/cranium (kranium) pyramid (pyramide) parachute (faldskærm) volcano (vulkan) stethoscope (stetoskop) maze/labyrinth (labyrint) compass (kompas) avocado (avocado) windmill (vindmølle) thermometer (termometer) anchor (anker) plough (plov) guillotine (guillotine) diamond (diamant) handcuff (håndjern) motor cycle (motorcykel) microscope (mikroskop) escalator (rulletrappe) cone (kogle) bat (flagermus) saxophone (saxofon) compasses/dividers (passer) waterfall (vandfald) igloo (iglo) medal (medalje) tambourine (tamburin) fossil (fossil) syringe (sprøjte) geyser (gejser) magnet (magnet) panther (panter) anteater (myresluger) battery (batteri) tandem (tandem) mortar (morter) chest of drawers (kommode) tweezers (pincet)

References Aguiar, L., & Brady, S. (1991). Vocabulary acquisition and reading ability. Reading and Writing, 3(3), 413–425. Burton, V. J., & Watkins, R. V. (2007). Measuring word learning: Dynamic versus static assessment of kindergarten vocabulary. Journal of Communication Disorders, 40(5), 335–356. de Jong, P. F., Seveke, M. J., & van Veen, M. (2000). Phonological sensitivity and the acquisition of new words in children. Journal of Experimental Child Psychology, 76(4), 275–301. Dunn, L. M., & Dunn, L. M. (1981). Peabody picture vocabulary test — Revised. Circle Pines, MN: American Guidance Service. Elbro, C., Borstrøm, I., & Petersen, D. K. (1998). Predicting dyslexia from kindergarten: The importance of distinctness of phonological representations of lexical items. Reading Research Quarterly, 33(1), 36–60.

8

A.S. Gellert, C. Elbro / Learning and Individual Differences 26 (2013) 1–8

Gathercole, S. E. (2006). Nonword repetition and word learning: The nature of the relationship. Applied PsychoLinguistics, 27(4), 513–543. Gathercole, S. E., & Baddeley, A. D. (1990). Phonological memory deficits in language disordered children: Is there a causal connection? Journal of Memory and Language, 29(3), 336–360. Gathercole, S. E., Hitch, G. J., Service, E., & Martin, A. J. (1997). Phonological short-term memory and new word learning in children. Developmental Psychology, 33(6), 966–979. Gupta, P., & Tisdale, J. (2009a). Does phonological short-term memory causally determine vocabulary learning? Toward a computational resolution of the debate. Journal of Memory and Language, 61(4), 481–502. Gupta, P., & Tisdale, J. (2009b). Word learning, phonological short-term memory, phonotactic probability and long-term memory: Towards an integrated framework. Philosophical Transactions of the Royal Society B — Biological Sciences, 364(1536), 3755–3771. Horton-Ikard, R., & Weismer, S. E. (2007). A preliminary examination of vocabulary and word learning in African American toddlers from middle and low socioeconomic status homes. American Journal of Speech-Language Pathology, 16(4), 381–392. Jarrold, C., Thorn, A. S. C., & Stephens, E. (2009). The relationships among verbal short-term memory, phonological awareness, and new word learning: Evidence from typical development and Down syndrome. Journal of Experimental Child Psychology, 102(2), 196–218. Kapantzoglou, M., Restrepo, M. A., & Thompson, M. S. (2012). Dynamic assessment of word learning skills: Identifying language impairment in bilingual children. Language, Speech, and Hearing Services in Schools, 43(1), 81–96. Masoura, E. V., & Gathercole, S. E. (2005). Contrasting contributions of phonological short-term memory and long-term knowledge to vocabulary learning in a foreign language. Memory, 13(3–4), 422–429. Mayringer, H., & Wimmer, H. (2000). Pseudoname learning by German-speaking children with dyslexia: Evidence for a phonological learning deficit. Journal of Experimental Child Psychology, 75(2), 116–133.

Morra, S., & Camba, R. (2009). Vocabulary learning in primary school children: Working memory and long-term memory components. Journal of Experimental Child Psychology, 104(2), 156–178. Nash, M., & Donaldson, M. L. (2005). Word learning in children with vocabulary deficits. Journal of Speech, Language, and Hearing Research, 48(2), 439–458. Nation, K., Snowling, M. J., & Clarke, P. (2007). Dissecting the relationship between language skills and learning to read: Semantic and phonological contributions to new vocabulary learning in children with poor reading comprehension. International Journal of Speech-Language Pathology, 9(2), 131–139. Nielsen, J. B. (2008). Det mentale leksikon og testning af receptivt ordforråd: ændring af Peabody-testen [The mental lexicon and assessment of receptive vocabulary: A revision of the Peabody Test]. Copenhagen: University of Copenhagen. Ouellette, G. P. (2006). What's meaning got to do with it: The role of vocabulary in word reading and reading comprehension. Journal of Educational Psychology, 98(3), 554–566. Poulsen, M. (2011). Do dyslexics have auditory input processing difficulties? Applied PsychoLinguistics, 32(2), 245–261. Raven, J. C., Court, J. H., & Raven, J. (1990). Coloured progressive matrices. Oxford: Oxford Psychologists Press. Ricketts, J., Bishop, D. V. M., & Nation, K. (2008). Investigating orthographic and semantic aspects of word learning in poor comprehenders. Journal of Research in Reading, 31(1), 117–135. Ricketts, J., Bishop, D. V. M., Pimperton, H., & Nation, K. (2011). The role of self-teaching in learning orthographic and semantic aspects of new words. Scientific Studies of Reading, 15(1), 47–70. Sideridis, G., & Simos, P. (2010). What is the actual correlation between expressive and receptive measures of vocabulary? Approximating the sampling distribution of the correlation coefficient using the bootstrapping method. The International Journal of Educational and Psychological Assessment, 5, 117–133. Tannenbaum, K. R., Torgesen, J. K., & Wagner, R. K. (2006). Relationships between word knowledge and reading comprehension in third-grade children. Scientific Studies of Reading, 10(4), 381–398.