Regularization of languages by adults and children: A mathematical framework

x2 Form 2

s

s

p

p

Source: Form 2

p

x1

x1

x1

Form 1

Form 1

Form 1

Fig. A.9. A schematic illustration of the reinforcement learning algorithm proposed here, in the case of two forms of the rule. The vertical bar represents the state of the learner: the small circle splits the bar into two parts, x1 and x2 , with x1 þ x2 ¼ 1, such that x1 is the probability of the learner to use form 1, and x2 is the probability of the learner to use form 2. In this example, x1 > x2 , that is, form 1 is the preferred form of the learner. The black arrows show the change in the state of the learner following an utterance of the source. If the source produces form 1, the value of x1 increases by amount s. If the source produces form 2, the value of x1 decreases by amount p. Three cases are presented: (a) s ¼ p (the symmetric learner), (b) s > p, and (c) s < p.

s>p

Form 2

x1

s
Form 2

x2

x1 Form 2

Form 1

Form 1

x2

x2

x1 Form 2

x1

Form 1

Form 1

x2

Fig. A.10. Boosting tendency occurs when s > p. The value of x1 is the learner’s propensity for form 1. The arrows on the diagram show the direction and the relative size of the update following the source’s utterance. The black arrows are the updates if the source utters form 1, and the light green arrows are the updates if the source utters form 2. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

We can see that if s > p, then the arrows pushing the dot towards the edges (zero and one) are larger than the ones pushing it toward the middle (for s < p, this is reversed). This means that if the learner is characterized by a stronger response to the source when the utterance coincides with its preferred form, then the boosting tendency is observed. A.3. Fitting of the piecewise-linear model Applying the symmetric reinforcement model (9) to describe the data of Hudson Kam and Newport (2009) we found that no parameter s gives a satisfactory fit. The best fit was obtained for the value s ¼ 1 and is depicted by the dashed line in Fig. A.11(b). Although at the first glance, this fit is not too different from the data, this model has a very serious conceptual drawback. The best fitting

20

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

0.6

s: the "preferred increment"

0.9 0.8

0.5

0.7 0.4

0.6 0.5

0.3 0.4 0.2

0.3 0.2

0.1 0.1 0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

p: the "non−preferred increment"

0.9

1.0

Mean Production of Main Determiner Forms

Adults 1.0

100% 90% 80% 70% 60% 50% 40% 30% 20%

Control

2ND

4ND

8ND

16ND

Input Group

Fig. A.11. Fitting the piecewise linear model, (9). (a) Heat plot of least squares error of the asymmetric model compared to the results of Hudson Kam and Newport (2009). s and p vary between 0.01 and 1 with a step size of 0.01. The dark blue areas indicate where the smallest error occurs. (b) Comparison of the results from Hudson Kam and Newport (2009) (the vertical intervals) with the results from our models (the lines), adult learners only. The dashed line is for the best fitting symmetric reinforcement model with s ¼ 1. The solid line is the best-fitting asymmetric model corresponding to s ¼ :19 and p ¼ :50. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

parameter value for the symmetric model is s ¼ 1. This means that the behavior of the learner predicted by the model is extremely unstable and unrealistic. The state of the learner is characterized by only two points, 0 and 1, with frequent switches between the two. This forces us to reject this model and seek a better fit in the generalized, asymmetric algorithms (Eq. (14)). Fig. A.11 gives a heat plot of the least square error computed for all pairs ðs; pÞ. The dark blue areas give the best overall error. We found that the parameters s ¼ 0:19 and p ¼ 0:50 give the best match. This is represented by the solid line in Fig. A.11. Therefore, for the adults, the best match comes when the ‘‘nonpreferred increment” is larger than the ‘‘preferred increment.” A similar result was obtained when we used the data of Hudson Kam and Newport (2005), see Appendix B. Appendix B. Application of the model to the data of Hudson Kam and Newport (2005) In Hudson Kam and Newport (2005), Newport and her colleagues conduct experiments similar to those from Hudson Kam and Newport (2009). In the first experiment, they create an artificial language that is divided into two groups. The ‘‘count/mass nouns” group has the nouns of the language divided into two classes on basis of meaning. The ‘‘gender condition” has the nouns divided into two classes in an arbitrary fashion. Each of these is then divided into four groups, where the determiner is used k% of the time (with k ¼ 45 (the low group), k ¼ 60 (the mid group), k ¼ 75 (the high group), and k ¼ 100 (the perfect group)), and one incorrect form is used otherwise. Altogether, there are eight groups of adults, with five learners in each group. In this experiment, the adults were not able to regularize the language. Their mean determiner production was almost always less than the input. We applied model (12) and (13) to fit the data presented in Hudson Kam and Newport (2005). As seen in Fig. B.12, for the count/mass groups, the best fit occurs when s ¼ 0:17 and p ¼ 0:50. For the gender groups, the best fit occurs when s ¼ 0:05, and p ¼ 0:50. These results are similar to what we found in experiment 1 from Hudson Kam and Newport (2009) (see Fig. 3). We again find that s < p. In the second experiment of Hudson Kam and Newport (2005), both children and adults were taught a simpler artificial language. In this experiment, there were 15 children and 8 adults. The children and adults were each divided into two groups. In one group, the determiner was used 100% of the time. In the other, it was used 60% of the time. These groups correspond to the 100% and 60% þ 0ND groups in the second experiment of Hudson Kam and Newport (2009), and their results were the same in both. The adults performed better than the children in the 100% group, and the adults and children performed about the same in the 60% group. However, the children were much more likely to be

21

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30 100

Mean Determiner Production (%)

90 80 70

Count/Mass−Paper Gender−Paper Input Count/Mass−Model Gender−Model

60 50 40 30 20 10 0 Low

Mid

High

Perfect

Input Group Fig. B.12. The best fit of the model compared to the results of Hudson Kam and Newport (2005). The solid lines give the frequency of production observed in the paper (blue with circles for the count/mass groups, and red with squares for the gender groups). The dashed lines show the best fit for each group by using model (12) and (13). The dashed black line gives the frequency of the input for each group. For the count/mass groups, the best fit occurs when s ¼ 0:17 and p ¼ 0:50. For the gender groups, the best fit occurs when s ¼ 0:05, and p ¼ 0:50. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

systematic learners. Although it is not possible to fit these data because there are only two data points for each group, the experimental results reported in Hudson Kam and Newport (2005) correspond to our findings in Section 3.2, where we modeled the second experiment of Hudson Kam and Newport (2009). Appendix C. Application of the model to the data of Fedzechkina et al. (2012) C.1. The language structure of the experiments in Fedzechkina et al. (2012) In Fedzechkina et al. (2012), adult learners were taught an artificial language with inefficient case marking. The artificial language consisted of simple sentences with a subject, an object, and a verb, with objects case-marked in a certain subset of sentences. All learners were native English speakers and had not previously been exposed to case-marking. The objective was to assess the learner’s efficiency of case-marking in different types of sentences, as explained below. C.1.1. Experiment 1 In the first experiment, the subjects were always animate, and the objects could be animate or inanimate. The order of the sentences could be Subject–Object–Verb (SOV) or Object–Subject–Verb (OSV). The sentences were considered harder to understand when both the subject and the object were animate. They were also considered harder to understand when the OSV form was used. The language input was broken down in two ways: (i) 50% of the sentences had an animate object and 50% of the sentences had an inanimate object and (ii) 60% of the sentences were SOV and 40% of the sentences were OSV. The objects were case-marked 60% of the time, with both animate and inanimate objects being equally likely to be case-marked. Fig. C.13 presents diagrams that show the input structure of the language. It follows that the language had four types of sentence: (i) animate and SOV, (ii) animate and OSV, (iii) inanimate and SOV, and (iv) inanimate and OSV. In each of these sentences, the object was either case-marked or not. The way the language was presented, the case marking was not efficient. A sentence that was harder to understand, such as an animate, OSV sentence might not be case-marked, while an easier to understand sentence (inanimate, SOV, for example) might be casemarked.

22

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

Fig. C.13. The breakdown of the sentences in the experiments of Fedzechkina et al. (2012). Top: breakdown in terms of their object: either animate or inanimate. Bottom: breakdown in terms of their word order: either Subject–Object–Verb (SOV) or Object–Subject–Verb (OSV).

The experiment went on for 4 days and had 20 participants. Each day, participants were taught 80 sentences, and then tested, with the exception that they were not tested at the end of the first day. The percentage of case-marked objects produced by each participant was recorded. The paper only presents data on the proportion of object case-marking from days 2 through 4. Fig. C.14 shows the results of experiment 1 (the solid lines). The following trends are immediately observed. When the objects are animate, the sentences are harder to understand, so one expects the learners to case-mark the animate objects more than the inanimate objects, which they do (see Fig. C.14(a)). Also, since the OSV sentences are harder to understand, one expects that the learners will case-mark the objects more often in these than they do in the SOV sentences. As we can see in Fig. C.14(b), they do. C.1.2. Experiment 2 In the second experiment, the input language was in a sense the complement of the language in the first experiment. The objects were always inanimate, and the subjects could be either animate or inanimate (50% of the subjects are animate and 50% are inanimate). The other aspects of the language were all the same. The authors were testing two hypotheses in the second experiment. First, they wanted to know if, in the first experiment, the preference to case-mark animate objects came from a bias to case-mark

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

23

Fig. C.14. Experiment 1. The solid lines represent the outcome of the experiments, with the standard deviation shown by  vertical bars. The dashed lines show the best fit of the model, see Appendix C.4. It occurs when Dþ AA ¼ 0:080; DAA ¼ 0:089; DþUU ¼ 0:001; DUU ¼ 0:017; d1 ¼ 0:01, and d2 ¼ 0:97. (a) Animate versus inanimate. (b) SOV versus OSV.

the atypical. If this was the case, then in the second experiment, they would expect to see the inanimate subjects being case-marked more often. The second possibility that the authors were testing was that the preference to mark animate objects came from a bias toward animate things. If this was the case, then they would expect to see the animate sentences case-marked the most in both experiments. Interestingly, it was found that both biases were at play. At first, the learners case-marked animate sentences more often, but by day three, they were case-marking the inanimate sentences more (see Fig. C.15(a), the solid lines). With regards to the SOV and OSV sentences, again two hypotheses were being tested. Firstly, the authors wanted to see if the bias was to case-mark to avoid miscommunication. If this was the case, then we should see OSV sentences case-marked more often again. Alternatively, the bias to case-mark could be ‘‘a bias to mention disambiguating information as early as possible in the sentence”. If this was the case, then SOV sentences should be case-marked more often. Again, it was found that both biases seemed to be at play. SOV sentences are case-marked more often than OSV, but by day 4, the rate of case-marking SOV sentences was decreasing, while the rate of case-marking OSV sentences was increasing (see Fig. C.15(b), the solid lines). C.2. Parameterizing the learning algorithm In order to adopt the algorithm to this particular setting, we notice that the variations in the input only concerned whether or not the sentence’s object was case-marked. This means that there were only two variants present: to case mark or not to case-mark. Therefore, we have n ¼ 2. The update rule (14) simplifies for n ¼ 2. Each binary rule can be simply characterized by the probability to case-mark, P; 0 6 P 6 1. Then, if a case-marked sentence of the appropriate kind is received, the following update applies:

(

P!

½P þ Dþ 1 ; P > 1=2; ½P þ D 1 ; P < 1=2:

ðC:1Þ

Similarly, if a non-case-marked sentence of this kind is received, then



P!

½P  D 1 ; P > 1=2; ½P  Dþ 1 ; P < 1=2:

Above, we use the following operation denoted by the square brackets with the subscript 1:



½Z1 ¼

Z; Z 6 1 1; Z > 1:

ðC:2Þ

24

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

Fig. C.15. Experiment 2. The solid lines represent the outcome of the experiments, with the standard deviation shown by  vertical bars. The dashed lines show the best fit of the model, see Appendix C.4. It occurs when Dþ AA ¼ 0:013; DAA ¼ 0:049; DþUU ¼ 0:100; DUU ¼ 0:200; d1 ¼ 0:10, and d2 ¼ 1:00. (a) Animate versus inanimate. (b) SOV versus OSV.

In the first case (Eq. (C.1)), the input sentence case-marks the object. Then, if the learner is more likely than not to case-mark this type of sentence, then her tendency to case-mark it increases by quantity Dþ (or becomes one); otherwise it increases by the quantity D (or becomes one). Eq. (C.2) describes the situation when the source’s sentence does not mark the object. In this case, if the learner is more likely to not case-mark this type of a sentence, then her tendency to not case-mark it increases by quantity Dþ (or becomes one); otherwise it increases by the quantity D (or becomes one). In the experiments, each sentence was both animate or inanimate (we will shorten these to an and in), and SOV or OSV (which are shortened to so and os). This means that the learners could hear four types of sentences: ðan; soÞ; ðan; osÞ; ðin; soÞ, and ðin; osÞ. Therefore, not one, but four different rules were learned, each corresponding to its sentence type. In the setting of the experiments, each individual can be characterized by probabilities

Pan;so ;

Pan;os ;

Pin;so ;

P in;os ;

where Pan;so is the probability of the learner to case-mark an animate, SOV sentence; P an;os is the probability of the learner to case-mark an animate, OSV sentence; P in;so is the probability of the learner to case-mark an inanimate, SOV sentence; and Pin;os is the probability of the learner to case-mark an inanimate, OSV sentence. The rules described above apply to all four sentence types. Therefore, we have eight update param eters, which we denote by Dþ m and Dm , where m 2 fðan; soÞ; ðan; osÞ; ðin; soÞ; ðin; osÞg. For each rule, we have two different increments that are used depending on whether or not the learner’s internal hypothesis matches the source. For example, if the source gives a case-marked ðan; soÞ sentence, and the learner prefers to case-mark ðP an;so > 1=2Þ, we update with the ‘‘+” increment. If the sentence from the source does not match the learner’s hypothesis, we use the ‘‘” increment. Upon receiving an input sentence, say an an; so sentence, the learner is assumed to only update the probability corresponding to that type of sentence; in other words, the four different rules are assumed to be independent. This is a simplification, and in reality, it is possible that the probabilities to case-mark other types of sentences can also be updated. For example, if the source’s sentence used case-marking in an an; so sentence, the quantities Pan;os and Pin;so could also increase, although perhaps to a lesser extent than the quantity Pan;so . Including such interactions among the four rules would increase the number of parameters significantly and make any parameter fitting impossible. Therefore, here we use the simplifying assumption of rule independence. Even in this simplest case however, the model contains eight independent increment parameters. In the next section we show how we can reduce the number of increment parameters to four by making further assumptions.

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

25

C.3. Ambiguity and reduction rules for the increments The sentences in the experiment differ by the degree of ambiguity. Denoting ‘‘ambiguous” by A and ‘‘unambiguous” by U, we can classify each sentence as type AA; AU; UA, or UU. In the first experiment, animate and OSV are ambiguous. In the second experiment, inanimate and OSV are ambiguous. Therefore, all sentences can be classified into four classes, as summarized in Table C.1. In this way, we can consider four increment types: DAA ; DAU ; DUA and DUU (with each of the types having a ‘‘+” and ‘‘” increment). Based on ambiguity considerations, there are several theoretical possibilities for reductions, where we assume that some of the variables are equal to each other. Possible hypotheses that allow us to reduce the number of parameters to four are:  All increments plus variables are equal to all minus variables. This gives 4 total independent increments and corresponds to a symmetric learning algorithm which was introduced in Mandelshtam and Komarova (2014).  We can further group the increments by ambiguity classes. For example, the increments for AA; AU; UA could be the same, and the most unambiguous sentences UU could have different increments. The plus and minus increments are kept different, and therefore we have 4 independent increments.  Similarly, we could use one type of increments for the most ambiguous sentences, AA, and another type of increments for AU; UA; UU. Again, this leads to 4 independent increments.  We can assume that the increments of ‘‘mixed” ambiguity are given by the arithmetic means of the AA and UU types:

DþUA ¼ DþAU ¼

DþAA þ DþUU 2

and DUA ¼ DAU ¼

DAA þ DUU : 2

ðC:3Þ

 þ  Again, this corresponds to 4 independent increment parameters, Dþ AA ; DAA ; DUU , and DUU .

It turns out that none of these rules produce a reasonable fit except for the last rule. We were not successful in reducing the number of independent increments further, so four such increments was the smallest number of increment parameters that allowed for a successful fit. In our model, we keep track of the learners’ frequencies to case-mark the four types of sentences as they learn each day. We would like to include discounted learning into our model to simulate the learners not remembering everything that they learned from the day before. At the beginning of day 2, the learners’ frequencies to case-mark the four types of sentences will be multiplied by d1 , where d1 will represent the percent that they remember from their first day of learning. Starting with day 3, we will instead multiply their frequencies by d2 , where d2 represents the amount that they remember from the day before (after they have already been learning for at least 2 days). The reason for this modeling choice is the reported trend that learning without testing has a lower retention rate than learning in the presence of testing, see e.g. Rohrer and Pashler (2007), Karpicke and Roediger (2007), Larsen, Butler, and Roediger (2009), Butler and Roediger (2007), and Roediger and Butler (2011). The learners in the work of Fedzechkina et al. (2012) were not tested at the end of day 1, and they were tested at the end of each subsequent day. It has been suggested in the literature that immediate testing of the material improves the percentage of information retained by the learners.

Table C.1 Sentence types classified by their ambiguity, for the two experiments.

AA UU AU UA

Experiment 1

Experiment 2

ðan; osÞ ðin; soÞ ðan; soÞ ðin; osÞ

ðin; osÞ ðan; soÞ ðin; soÞ ðan; osÞ

26

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

C.4. Fitting procedure and the results We used the following procedure to perform parameter fitting of the model against data of Fedzechkina et al. (2012). We simulated the learning process of 100 subjects over four days. In the simulation of the first experiment, on each day, the learners were presented with 8 sentences of 4 types, according to the linguistic breakdown as in Fig. C.13. The learners updated their probabilities Pm with m 2 fðan; soÞ; ðan; osÞ; ðin; soÞ; ðin; osÞg according to the learning algorithm described above, with fixed  parameters Dþ m and Dm (subjects to the constrains in Eq. (C.3), and with fixed memory discounting parameters d1 and d2 . All these parameters were assumed to be the same for all the learners. Then, at the end of days 2, 3, and 4, their probabilities Pm with m 2 fðan; soÞ; ðan; osÞ; ðin; soÞ; ðin; osÞg were recorded. In the data of Fedzechkina et al. (2012), these probabilities were not measured directly. Instead, the quantities P an ; P in ; P so , and P os were presented, which are the probabilities to case-mark animate, inanimate, SOV, and OSV sentences, respectively. Given the breakdown of the sentences (see Fig. C.13), they are connected with our variables as follows:

Pan ¼ 3=5Pan;so þ 2=5Pan;os ;

Pin ¼ 3=5Pin;so þ 2=5Pin;os ;

ðC:4Þ

Pso ¼ 1=2Pan;so þ 1=2Pin;so ;

Pos ¼ 1=2Pan;os þ 1=2Pin;os :

ðC:5Þ

Note that not all of these quantities are independent:

1=2Pan þ 1=2Pin ¼ 3=5Pso þ 2=5Pos : For each trial, we calculated quantities Pan ; Pin , etc from our probability values, and compared the mean values (over the population of learner) with the data of Fedzechkina et al. (2012) by calculating the mean square difference. Then all the parameters (that is, the increments and the memory parameters) were changed and the procedure repeated. In the end, we found the parameter combinations that corresponded to the least mean square error. Those were considered the best fitting parameters. A similar procedure was performed for experiment 2. The results are presented below.  þ For experiment 1, the best fit occurs when Dþ AA ¼ 0:080; DAA ¼ 0:089; DUU ¼ 0:001; DUU ¼ 0:017; d1 ¼ 0:01, and d2 ¼ 0:97 (see Fig. C.14).  þ  For experiment 2, the best fit occurs when Dþ AA ¼ 0:013; DAA ¼ 0:049; DUU ¼ 0:100; DUU ¼ 0:200; d1 ¼ 0:10, and d2 ¼ 1:00 (see Fig. C.15). C.5. Summary We have adapted the asymmetric reinforcement-type learning Algorithm (11) to describe the experiments of Fedzechkina et al. (2012). We were able to parameterize the model and obtain good fits with the quantitative data of Fedzechkina et al. (2012). For both experiments, the best fitting parameter values for the learning increments satisfy Dþm < Dm , which is consistent with the model in the main part of this paper. Recall that Dþm is the learning increment for rule m, which is implemented following a source’s string that is compatible with the learner’s internal hypothesis. On the other hand, D m is the learning increment used after a source’s string that does not coincide with the learner’s internal hypothesis. Thus, adults appear to react stronger when the evidence is inconsistent with their internal hypothesis. Further, in both experiments, the best fitting memory discounting parameters satisfy d2 > d1 , where d1 is the proportion retained after the first day of learning, and d2 is the proportion retained after any subsequent day of learning. This inequality agrees with the findings in the literature (Butler & Roediger, 2007; Karpicke & Roediger, 2007; Larsen et al., 2009; Roediger & Butler, 2011; Rohrer & Pashler, 2007), who showed that learners remember more of what they learn if they are tested on it right away. References Alós-Ferrer, C., & Netzer, N. (2010). The logit-response dynamics. Games and Economic Behavior, 68, 413–427. Andersen, R. W. (1983). Pidginization and creolization as language acquisition. ERIC.

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

27

Bendor, J., Diermeier, D., & Ting, M. (2003). A behavioral model of turnout. American Political Science Review, 97, 261–280. Bendor, J., Mookherjee, D., & Ray, D. (2001). Aspiration-based reinforcement learning in repeated interaction games: An overview. International Game Theory Review, 3, 159–174. Bever, T. G. (1982). Regression in the service of development. In Regression in mental development: Basic properties and mechanisms (pp. 153–88). Bickerton, D. (1984). The language bioprogram hypothesis. Behavioral and Brain Sciences, 7, 173–222. Birdsong, D. (1989). Metalinguistic performance and interlinguistic competence. Berlin: Springer-Verlag. Blume, L. E. (1993). The statistical mechanics of strategic interaction. Games and Economic Behavior, 5, 387–424. Botvinick, M. M., Niv, Y., & Barto, A. C. (2009). Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition, 113, 262–280. Bröder, A., Herwig, A., Teipel, S., & Fast, K. (2008). Different storage and retrieval deficits in normal aging and mild cognitive impairment: A multinomial modeling analysis. Psychology and Aging, 23, 353. Busemeyer, J. R., & Pleskac, T. J. (2009). Theoretical tools for understanding and aiding dynamic decision making. Journal of Mathematical Psychology, 53, 126–138. Bush, R. R., & Mosteller, F. (1955). Stochastic models for learning. John Wiley & Sons, Inc. Butler, A. C., & Roediger, H. L. III, (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19, 514–527. Bybee, J. L., & Slobin, D. I. (1982). Why small children cannot change language on their own: Suggestions from the english past tense. In Papers from the 5th international conference on historical linguistics (Vol. 21). Camerer, C. (2003). Behavioral game theory: Experiments in strategic interaction. Princeton University Press. Carroll, S., & Swain, M. (1993). Explicit and implicit negative feedback. Studies in Second Language Acquisition, 15, 357–386. Chang, F., Dell, G. S., & Bock, K. (2006). Becoming syntactic. Psychological Review, 113, 234. Chouinard, M. M., & Clark, E. V. (2003). Adult reformulations of child errors as negative evidence. Journal of Child Language, 30, 637–670. Cochran, B. P., McDonald, J. L., & Parault, S. J. (1999). Too smart for their own good: The disadvantage of a superior processing capacity for adult language learners. Journal of Memory and Language, 41, 30–58. Coppola, M., & Newport, E. L. (2005). Grammatical subjects in home sign: Abstract linguistic structure in adult primary gesture systems without linguistic input. Proceedings of the National Academy of Sciences of the United States of America, 102, 19249–19253. Craig, G. J., & Myers, J. L. (1963). A developmental study of sequential two-choice decision making. Child Development, 34, 483–493. Crone, E. A., Richard Ridderinkhof, K., Worm, M., Somsen, R. J., & Van Der Molen, M. W. (2004). Switching between spatial stimulus–response mappings: A developmental study of cognitive flexibility. Developmental Science, 7, 443–455. Crone, E. A., Zanolie, K., Van Leijenhorst, L., Westenberg, P. M., & Rombouts, S. A. (2008). Neural mechanisms supporting flexible performance adjustment during development. Cognitive, Affective, & Behavioral Neuroscience, 8, 165–177. Danks, D. (2003). Equilibria of the Rescorla–Wagner model. Journal of Mathematical Psychology, 47, 109–121. DeGraff, M. (2001). Language creation and language change: Creolization, diachrony, and development. The MIT Press. Derks, P. L., & Paclisanu, M. I. (1967). Simple strategies in binary prediction by children and adults. Journal of Experimental Psychology, 73, 278. Duffy, J. (2006). Agent-based models and human subject experiments. Handbook of Computational Economics, 2, 949–1011. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179–211. Erev, I., & Roth, A. E. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review, 848–881. Fedzechkina, M., Jaeger, T. F., & Newport, E. L. (2012). Language learners restructure their input to facilitate efficient communication. Proceedings of the National Academy of Sciences, 109, 17897–17902. Feldman, J., & Newell, A. (1961). A note on a class of probability matching models. Psychometrika, 26, 333–337. Ferdinand, N. K., Mecklinger, A., & Kray, J. (2008). Error and deviance processing in implicit and explicit sequence learning. Journal of Cognitive Neuroscience, 20, 629–642. Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1898–1902. Flache, A., & Macy, M. W. (2002). Stochastic collusion and the power law of learning a general reinforcement learning model of cooperation. Journal of Conflict Resolution, 46, 629–653. Fowler, J. H. (2006). Habitual voting and behavioral turnout. Journal of Politics, 68, 335–344. Frank, M. C., & Tenenbaum, J. B. (2011). Three ideal observer models for rule learning in simple languages. Cognition, 120, 360–371. Goldin-Meadow, S. (2005). The resilience of language: What gesture creation in deaf children can tell us about how all children learn language. Psychology Press. Goldin-Meadow, S., Mylander, C., de Villiers, J., Bates, E., & Volterra, V. (1984). Gestural communication in deaf children: The effects and noneffects of parental input on early language development. Monographs of the Society for Research in Child Development, 1–151. Gómez, R. L., & Gerken, L. (2000). Infant artificial language learning and language acquisition. Trends in Cognitive Sciences, 4, 178–186. Griffiths, T. L., Chater, N., Kemp, C., Perfors, A., & Tenenbaum, J. B. (2010). Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences, 14, 357–364. Guitart-Masip, M., Huys, Q. J., Fuentemilla, L., Dayan, P., Duzel, E., & Dolan, R. J. (2012). Go and no-go learning in reward and punishment: Interactions between affect and effect. Neuroimage, 62, 154–166. Gureckis, T. M., & Love, B. C. (2009). Short-term gains, long-term pains: How cues about state aid learning in dynamic environments. Cognition, 113, 293–313. Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2007). It’s worse than you thought: The feedback negativity and violations of reward prediction in gambling tasks. Psychophysiology, 44, 905–912.

28

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

Hämmerer, D., Li, S.-C., Müller, V., & Lindenberger, U. (2011). Life span differences in electrophysiological correlates of monitoring gains and losses during probabilistic reinforcement learning. Journal of Cognitive Neuroscience, 23, 579–592. Holyoak, K. J., Koh, K., & Nisbett, R. E. (1989). A theory of conditioning: Inductive learning within rule-based default hierarchies. Psychological Review, 96, 315. Hsu, A. S., Chater, N., & Vitányi, P. (2013). Language learning from positive evidence, reconsidered: A simplicity-based approach. Topics in Cognitive Science, 5, 35–55. Hudson Kam, C. L., & Newport, E. L. (2005). Regularizing unpredictable variation: The roles of adult and child learners in language formation and change. Language Learning and Development, 1, 151–195. Hudson Kam, C. L., & Newport, E. L. (2009). Getting it right by getting it wrong: When learners change languages. Cognitive Psychology, 59, 30. Huizinga, M. t., Dolan, C. V., & van der Molen, M. W. (2006). Age-related change in executive function: Developmental trends and a latent variable analysis. Neuropsychologia, 44, 2017–2036. Izquierdo, L. R., Izquierdo, S. S., Gotts, N. M., & Polhill, J. G. (2007). Transient and asymptotic dynamics of reinforcement learning in games. Games and Economic Behavior, 61, 259–276. Johnson, J. S., Shenkman, K. D., Newport, E. L., & Medin, D. L. (1996). Indeterminacy in the grammar of adult language learners. Journal of Memory and Language, 35, 335–352. Karpicke, J. D., & Roediger, H. L. III, (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57, 151–162. Klein, W., & Perdue, C. (1993). Utterance structure. In Adult language acquisition: Cross-linguistic perspectives (Vol. 2). (The results). Larsen, D. P., Butler, A. C., & Roediger, H. L. III, (2009). Repeated testing improves long-term retention relative to repeated study: A randomised controlled trial. Medical Education, 43, 1174–1181. Leeman, J. (2003). Recasts and second language development. Studies in Second Language Acquisition, 25, 37–63. Lee, D., Seo, H., & Jung, M. W. (2012). Neural basis of reinforcement learning and decision making. Annual Review of Neuroscience, 35, 287–308. Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106, 1126–1177. Lieberman, E., Michel, J.-B., Jackson, J., Tang, T., & Nowak, M. A. (2007). Quantifying the evolutionary dynamics of language. Nature, 449, 713–716. Luce, R. D. (1977). The choice axiom after twenty years. Journal of Mathematical Psychology, 15, 215–233. Ma, T., & Komarova, N. (in preparation). Stochastic nonlinear models of learning. (2015). Maia, T. V. (2009). Reinforcement learning, conditioning, and the brain: Successes and challenges. Cognitive, Affective, & Behavioral Neuroscience, 9, 343–364. Mandelshtam, Y., & Komarova, N. (2014). When learners surpass their sources: Mathematical modeling of learning from an inconsistent source. Available from arXiv preprint arxiv:1402.4678. Marchman, V. A., Plunkett, K., & Goodman, J. (1997). Overregularization in english plural and past tense inflectional morphology: A response to marcus. Journal of Child Language, 24, 767–779. Marcotte, J. (2004). Language acquisition, linguistic evidence, the baby, and the bathwater. Journal of Linguistics, 1–61. Marcus, G. F. (1993). Negative evidence in language acquisition. Cognition, 46, 53–85. Marcus, G. F. (1995). Children’s overregularization of english plurals: A quantitative analysis. Journal of Child Language, 22, 447–459. Marcus, G. F., Pinker, S., Ullman, M., Hollander, M., Rosen, T. J., Xu, F., & Clahsen, H. (1992). Overregularization in language acquisition. Monographs of the Society for Research in Child Development, i-178. McClure, S. M. (2003). Reward prediction errors in human brain. Ph.D. Thesis, Baylor College of Medicine. McLaren, I., & Mackintosh, N. (2000). An elemental model of associative learning: I. Latent inhibition and perceptual learning. Animal Learning & Behavior, 28, 211–246. Miller, R. R., Barnet, R. C., & Grahame, N. J. (1995). Assessment of the Rescorla–Wagner model. Psychological Bulletin, 117, 363. Monaghan, P., White, L., & Merkx, M. M. (2013). Disambiguating durational cues for speech segmentation. The Journal of the Acoustical Society of America, 134, EL45–EL51. Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. Nature, 431, 760–767. Mookherjee, D., & Sopher, B. (1994). Learning behavior in an experimental matching pennies game. Games and Economic Behavior, 7, 62–91. Mookherjee, D., & Sopher, B. (1997). Learning and decision costs in experimental constant sum games. Games and Economic Behavior, 19, 97–132. Mühlenbernd, R., & Nick, J. D. (2014). Language change and the force of innovation. In Pristine perspectives on logic, language, and computation (pp. 194–213). Springer. Narendra, K. S., & Thathachar, M. A. (2012). Learning automata: An introduction. Courier Dover Publications. Newport, E. L. (1990). Maturational constraints on language learning. Cognitive Science, 14, 11–28. Niv, Y. (2009). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53, 139–154. Niv, Y., & Montague, P. R. (2008). Theoretical and empirical studies of learning. In Neuroeconomics: Decision making and the brain (pp. 329–350). Niyogi, P. (2006). The computational nature of language learning and evolution. Cambridge: MIT Press. Norman, M. (1972). Markov processes and learning models. New York: Academic Press. Nowak, M. A., Komarova, N. L., & Niyogi, P. (2001). Evolution of universal grammar. Science, 291, 114–118. Pearce, J. M., & Hall, G. (1980). A model for pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532. Perruchet, P., & Pacton, S. (2006). Implicit learning and statistical learning: One phenomenon, two approaches. Trends in Cognitive Sciences, 10, 233–238. Pinker, S. (1989). Learnability and cognition: The acquisition of argument structure.

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

29

Plunkett, K., & Juola, P. (1999). A connectionist model of english past tense and plural morphology. Cognitive Science, 23, 463–490. Ramscar, M. (2015). Personal communication. Ramscar, M., Dye, M., Gustafson, J. W., & Klein, J. (2013a). Dual routes to cognitive flexibility: Learning and response-conflict resolution in the dimensional change card sort task. Child Development, 84, 1308–1323. Ramscar, M., Dye, M., & Klein, J. (2013b). Children value informativity over logic in word learning. Psychological Science, 24, 1017–1023. Ramscar, M., Dye, M., & McCauley, S. M. (2013c). Error and expectation in language learning: The curious absence of mouses in adult speech. Language, 89, 760–793. Ramscar, M., Dye, M., Popick, H. M., & O’Donnell-McCarthy, F. (2011). The enigma of number: Why children find the meanings of even small number words hard to learn and how we can help them do better. PloS One, 6, e22501. Ramscar, M., & Gitcho, N. (2007). Developmental change and the nature of learning in childhood. Trends in Cognitive Sciences, 11, 274–279. Ramscar, M., Hendrix, P., Love, B., & Baayen, R. (2013d). Learning is not decline: The mental lexicon as a window into cognition across the lifespan. The Mental Lexicon, 8, 450–481. Ramscar, M., Yarlett, D., Dye, M., Denny, K., & Thorpe, K. (2010). The effects of feature-label-order and their implications for symbolic learning. Cognitive Science, 34, 909–957. Reali, F., & Griffiths, T. L. (2009). The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning. Cognition, 111, 317–328. Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of Comparative and Physiological Psychology, 66, 1. Rescorla, R. A. (1988). Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43, 151. Rescorla, R.A., & Wagner, A.R. (1972b). A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical conditioning: Current research and theory. Rescorla, R., & Wagner, A. (1972a). A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. Black & W. Prokasy (Eds.), Classical conditioning II: Current Research and theory. New York: AppletonCentury-Crofts. Ribas-Fernandes, J. J., Solway, A., Diuk, C., McGuire, J. T., Barto, A. G., Niv, Y., & Botvinick, M. M. (2011). A neural signature of hierarchical reinforcement learning. Neuron, 71, 370–379. Roediger, H. L., III, & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15, 20–27. Rohde, D. L., & Plaut, D. C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109. Rohrer, D., & Pashler, H. (2007). Increasing retention without increasing study time. Current Directions in Psychological Science, 16, 183–186. Roth, A. E., & Erev, I. (1995). Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior, 8, 164–212. Roy, D. K., & Pentland, A. P. (2002). Learning words from sights and sounds: A computational model. Cognitive Science, 26, 113–146. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1, pp. 318–362). Cambridge. Saffran, J. R. (2003). Statistical language learning mechanisms and constraints. Current Directions in Psychological Science, 12, 110–114. Santesso, D. L., Segalowitz, S. J., & Schmidt, L. A. (2006). Error-related electrocortical responses in 10-year-old children and young adults. Developmental Science, 9, 473–481. Schultz, W. (2006). Behavioral theories and the neurophysiology of reward. Annual Review of Psychology, 57, 87–115. Schultz, W. (2007). Behavioral dopamine signals. Trends in Neurosciences, 30, 203–210. Sebba, M. (1997). Contact languages: Pidgins and creoles. London: Macmillan. Seidenberg, M. S. (1997). Language acquisition and use: Learning and applying probabilistic constraints. Science, 275, 1599–1603. Seidenberg, M. S., MacDonald, M. C., & Saffran, J. R. (2002). Does grammar start where statistics stop? Science, 298, 553–554. Seidl, A., & Johnson, E. K. (2006). Infant word segmentation revisited: Edge alignment facilitates target extraction. Developmental Science, 9, 565–573. Senghas, A. (1995). The development of nicaraguan sign language via the language acquisition process. In Proceedings of the 19th annual Boston University conference on language development (pp. 543–552). Senghas, A., & Coppola, M. (2001). Children creating language: How nicaraguan sign language acquired a spatial grammar. Psychological Science, 12, 323–328. Senghas, A., Coppola, M., Newport, E. L., & Supalla, T. (1997). Argument structure in nicaraguan sign language: The emergence of grammatical devices. In Proceedings of the 21st annual Boston University conference on language development (Vol. 2, pp. 550– 561). Shephard, E., Jackson, G., & Groom, M. (2014). Learning and altering behaviours by reinforcement: Neurocognitive differences between children and adults. Developmental Cognitive Neuroscience, 7, 94–105. Singleton, J. L., & Newport, E. L. (2004). When learners surpass their models: The acquisition of american sign language from inconsistent input. Cognitive Psychology, 49, 370–407. Smith, K., & Wonnacott, E. (2010). Eliminating unpredictable variation through iterated learning. Cognition, 116, 444–449. Steels, L. (2000). Language as a complex adaptive system. In Parallel problem solving from nature PPSN VI (pp. 17–26). Springer. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). Cambridge Univ Press. Thomason, S. G., & Kaufman, T. (1991). Language contact, creolization, and genetic linguistics. Univ of California Press. Thompson-Schill, S. L., Ramscar, M., & Chrysikou, E. G. (2009). Cognition without control when a little frontal lobe goes a long way. Current Directions in Psychological Science, 18, 259–263.

30

J.L. Rische, N.L. Komarova / Cognitive Psychology 84 (2016) 1–30

Van de Pol, M., & Cockburn, A. (2011). Identifying the critical climatic time window that affects trait expression. The American Naturalist, 177, 698–707. Van Duijvenvoorde, A. C., Zanolie, K., Rombouts, S. A., Raijmakers, M. E., & Crone, E. A. (2008). Evaluating the negative or valuing the positive? Neural mechanisms supporting feedback-based learning across development. The Journal of Neuroscience, 28, 9495–9503. Vulkan, N. (2000). An economistÕs perspective on probability matching. Journal of Economic Surveys, 14, 101–118. Wonnacott, E. (2011). Balancing generalization and lexical conservatism: An artificial language study with child learners. Journal of Memory and Language, 65, 1–14. Wonnacott, E., Newport, E. L., & Tanenhaus, M. K. (2008). Acquiring and processing verb argument structure: Distributional learning in a miniature language. Cognitive Psychology, 56, 165–209.