- Email: [email protected]
Relations between metastrategic knowledge and strategic performance
Cognitive
Development,
13,227-247
0 1998 Ablex Publishing
(1998)
All rights of reproduction
lS!SN 0885-2014
reserved.
RELATIONS BETWEEN METASTRATECIC KNOWLEDGE AND STRATEGIC PERFORMANCE
Deanna Columbia
Kuhn
University
Susan Pearsall Columbia
University
Two components
of metastrategic knowledge-knowledge of task objectives and knowledge of strategies-are distinguished and assessed in the context of an inductive causal reasoning task administered to fifth graders in a microgenetic design. Both metastrategic knowledge and strategic performance improved over a seven-week period of engagement with the task. However,
children were unlikely to attain strategic mastery until they had reached particular levels of metastrategic understanding. Discussion focuses on (a) the likelihood of multiple directions of influence between strategic and metastrategic knowledge systems, (b) coordination of task and strategy components of metastrategic knowledge as a central task of the metastrategic knowledge system, and (c) the generality of the present findings.
Initial research on children’s metacognitive or metastrategic knowledge focused almost entirely on memory skills (Brown, 1975, 1978; for current reviews see Kuhn, in press, or Schneider & Bjorkhmd, 1998). More recent, broader conceptions of metastrategic knowledge encompass awareness, understanding, monitoring, and management of one’s strategic performance of many kinds of cognitive tasks. The question of whether and how metastrategic knowledge relates to strategic performance is one worthy of investigation within this broader framework, especially as it relates to a topic receiving increasing attention from educatorsthe connection between performance and understanding (Gardner, 1991; Perkins, The research reported here was supported by a grant from the National Science Foundation to the first author. Direct all correspondence New York, NY 10027
to: Deanna
Kuhn, Box 119, Teachers
College, Columbia
University,
.
Manuscript received January 6,1997;
revision accepted July 2,1997
227
228
Kuhn and Pearsall
1992). As has by now been widely observed in both research and applied settings, skill in executing the component parts of a complex cognitive strategy is insufficient to ensure that the strategy will be performed in appropriate contexts in the absence of specific instruction (Brown, 1997; Kuhn, 1982). Some further, executive function is implicated (Kuhn, 1982), one that increasingly has been labeled as “metastrategic.” Reasoning is a domain in which we would expect metastrategic understanding to be particularly important (Moshman, 1998). It would seem desirable that people be aware of the inferences they make and the strategies or rules that allow them to be made, and we might assume that those who have such awareness will reason better than those who do not. The present work is addressed to the role of metastrategic knowledge in the domain of inductive reasoning, an area where there has not yet been a great deal of exploration of metacognitive or metastrategic phenomena. The theoretical conceptualization of the relation between metastrategic and strategic knowledge that guides the present work builds on a model of strategic development involving the coexistence of multiple strategies over time within an individual, with overlapping increases and decreases in frequencies of usage (Kuhn & Phelps, 1982; Kuhn, 1995; Siegler, 1996). The relation is conceived of as a bidirectional one in which “metastrategic understanding may both guide (in implicit form) and follow (in more explicit form) strategy development” (Kuhn, Garcia-Mila, Zohar, & Andersen, 1995, p. 114). This bidirectional conception of the relation between metastrategic understanding and strategic performance bears a similarity to Sophian’s (1997) conception of competence and its relation to performance: Competence guides performance and in turn is shaped by it. This bidirectional conceptualization suggests dual aspects of metastrategic understanding. In order for a strategy to be used, metastrategic understanding must be sufficient to enable the individual to recognize its applicability in relation to goals defined by the objectives of the task. An outcome of this use should be increasing awareness of the strategy and understanding of its power, again in relation to task objectives. We propose, then, two components of metastrategic understanding. One is the understanding and awareness of the nature and requirements of the task. This aspect of metastrategic knowledge is closely related to what Siegler (1996) has termed a goal sketch. A second component of metastrategic knowledge is awareness and understanding of the strategies available in one’s repertory that are potentially applicable to the task. The challenge of effective metastrategic functioning can be conceived of as one of coordination of the two components, understanding of the task and understanding of potential strategies, so that the ideal intersection of the two is realized in task performance. In the present study, we undertake separate measures of these two components of metastrategic knowledge-task understanding and strategy understandingand examine their relation to strategic performance. Because we believe a
Metastrategic
Knowledge
and Strategic
Performance
229
dynamic perspective offers more insight than a static one, we conducted the study microgenetically (Kuhn, 1995), examining performance in successive sessions with the same task over a period of seven weeks. Metastrategic knowledge we assessed twice, once at the second weekly session and once at the final (seventh) session. The design thus permits an assessment of change over time in both metastrategic knowledge and strategic performance. Finally, to enhance the generality of findings, we replicated the design across two kinds of task content, physical and social. METHOD Participants Participants in the study were lo- and 1 l-year-old fifth-grade students. This age level was chosen as one at which the strategic competencies being assessed are just beginning to emerge, based on findings from previous research (Kuhn, Amsel, & O’Loughlin, 1988; Kuhn et al., 1995; Kuhn, Schauble, & Garcia-Mila, 1992). Participants came from an urban public school serving a predominantly lower-income minority population. Of the 47 participants, 23 were female and 24 male. Procedure Strategic Task and Assessment. The strategic task presented to 23 of the 47 participants was one of the two physical science problems (the boat and car problems) used in research by Kuhn et al. (1995). The remaining 24 participants worked on one of the two social science problems (the TV and school problems) used by Kuhn et al. Full details of these tasks and the procedures involved in their repeated administration in a microgenetic design are available in the monograph by Kuhn et al. (1995). Here the boat problem is briefly described as an example. Because children were asked to engage the task repeatedly, it was important that it be situated within a pragmatic context that would provide the motivation and rationale for continuing to work on the task. We achieved this goal by asking children to investigate the causal structure (to find out “what makes a difference” and “what doesn’t make a difference”) characteristic of a particular problem domain and to use this developing knowledge to predict outcomes. The interviewer asked a range of questions (What are you going to find out? What do you think the outcome will be? What have you found out?) but provided no direction or feedback regarding the child’s activities. The purpose of the questions was to encourage the child to engage in cognitive activity, without trying to influence the specific direction it took. In designing the problems, the goal was to include two features that a child was likely to believe to be noncausal, with the effect in fact noncausal (and hence the belief confirmed) for one and the effect in fact causal (and the belief hence subject
230
Kuhn and Pearsall
to disconlirmation) for the other. Similarly, it was aimed to include at least two features the child would believe to be causal, with the effect in fact causal (and hence the belief confirmed) for one and the effect in fact noncausal (and the belief hence subject to disconfirmation) for the other. Complex (curvilinear and interactive) effects provided conditions for partial confirmation and partial disconfirrnation of beliefs (depending on which levels of the features were examined). Since children’s beliefs were not entirely predictable, without custom-designing problems for each participant, these goals could only be approximated. However, extensive pilot testing made it possible to identify features for which these conditions were met in a majority of cases. In the boat problem, a set of manipulable features influenced the speed with which model boats were pulled down a towing tank by a weight-and-pulley system. The task was to ascertain the effects these factors had on the boats’ speed. The problem was adapted from one developed by Schauble, Klopfer, and Raghavan (1991). The variable features were boat size, sail size, sail color, weight, and water depth. Two of the features, sail color and sail size, have no effect on outcome. Boat size has a simple causal effect, weight has a causal effect in interaction with size (it has an influence only with small boats), and depth is a three-level feature having a partial, curvilinear effect (the deep and medium-deep levels do not differ from one another but yield a faster outcome than the shallow level). At each session, the child was free to select boats and water levels for investigation and to record any information desired in a notebook provided for this purpose. Prior to running the boat, the child was asked to predict the outcome and explain the basis for this prediction. Following each observation, the child was questioned as to what inferences could be drawn, and at each session, a final judgment was elicited for that session as to whether a feature did or did not make a difference. The number of instances (boats, in this case) a child observed ranged between two and six per session, depending on time and the child’s wishes, but with the mode being five once the procedure became familiar (beginning with the second session). As part of a separate study of peer collaboration, children also participated in another seven sessions, one per week during the same seven-week period, during which they collaborated with a peer on a different, but parallel, task within the same domain, i.e., children working on a physical task worked on another physical task with a peer and those working on a social task worked with a peer on another social task. This additional task experience most likely contributed to strategic progress. Beyond this effect, however, the peer condition and its comparison to the solitary condition are not relevant to the questions addressed in the present study and are not examined further. Assessment of metastrategic understanding was confined to the task on which the child worked individually. understanding was initially Metastrategic Assessment. Metastrategic assessed in a separate interview that occurred on the same day as the second stra-
Metastrategic
Knowledge
and Strategic
Performance
231
tegic task session. The interview was readministered on the same day as the final (seventh) task session. The interview employed to assess metastrategic understanding was designed to provide separate indices of the task and strategy components identified earlier. The basic concept underlying the metastrategic assessment method was to have participants externalize their metastrategic understanding by having to communicate it to another child. Another child (not participating in the study) was brought to the interview room and it was explained that this new child would like to know what the work was that the participating child had been doing here each week. The interview began with the assessment of the task component. Specifically, the purpose was to assess whether children understood what the objective of the task was, i.e., to analyze the effects of the various features on outcome, and, if not, to assess what the child did construe the task objective to be. The interviewer instructed the child, “Please tell (the nonparticipating child) what you are supposed to do here.” The interviewer provided nondirective prompts to elaborate, if appropriate, but no feedback or other direction was given. The nonparticipating child had been previously instructed not to intervene or ask questions. The second segment of the metastrategic interview was addressed to the strategy component. Its purpose was to assess the understanding a child has of the strategies needed to accomplish the task. Do children understand the need for strategic behavior at all, and, if so, what awareness do they have of strategies that would be applicable and effective? In particular, do they understand the need to strategically choose for examination instances that provide informative comparisons? Because of the critical role of this strategic aspect of the task, it was the focus of the assessment of this second metastrategic component. Unless the child had already provided an answer in the first segment of the interview, the interviewer asked, “Can you explain to how you decide which ones to investigate?’
RESULTS Metastrategic
Performance
The two portions of the metastrategic interview, pertaining to task objective and strategy components, were coded separately. The respective levels of attainment for each component, however, can be roughly equated conceptually across the two components, as reflected in Table 1, which summarizes the successive levels of understanding we identified with respect to each of the components. Each interview was coded independently by two coders for each of the two metastrategic components. Percentage agreement between the two coders was .83 for the task objective component and .88 for the strategy component. Children at the lowest level (0) in Table 1 (who by definition score no higher than level 0 on either component), exhibit no insight regarding the task objective
Kuhn and Pearsall
232
Table 1.
Levels of Metastrategic Understanding Task objective
Strategy
level 0 level 1
Procedural only Attain a good outcome (“Find out which goes fastest”)
None (“Just choose anything”) Choose instances believed to yield a good outcome (“Choose the fastest”)
level 2
Analysis at instance level (“Find out how fast different boats will go”)
Try different instances and observe outcomes (‘Try different ones to see how fast they go”)
level 3
Analysis at feature level without discriminating features (“Find out what things are making a difference”)
level 4
Analysis at feature level with reference to multiple features (“See whether the size and weight make a difference”)
Compare instances in which multiple features are varied (“Compare a big heavy one and small light one to see if it makes a difference”)
level 5
Analysis at feature level with focus on single feature at a time (“Find out whether each of the features makes a difference”)
Compare instances in which a single feature is varied and other features are not mentioned (“See if the big or small one goes faster”)
level 6
Compare instances in which a single feature is varied and non-variation of other features is indicated (“Change just the weight and see if it makes a difference”)
and, when asked about it, simply describe the procedures involved in the task. Likewise, they exhibit no knowledge of strategies nor awareness of the need for them. The following is an example of one child’s initial metastrategic interview: what you are supposed to do here.) First you have to pick what items you (Please tell want for the car. And then what things that you are going to figure about. So you put a marker in each one you think you’re going to figure out. And then, you could guess how fast the car would go-number one, two, three, or four. And then you have to put a marker on which one you think is going to.. .on which one is going to go. And then after we ran it, and after that you see what things that you figured out about. And you put them.. .put a marker on each one you think you found out about, and then you answer questions.. .does it matter, and all that sort of questions. And then you’re done and then you got a couple more and that’s about it. how you decide which ones to investigate?) There are some (Can you explain to things here that you could pick. (How do you decide?) Ah, you just guess. To see what items you want to see in the car.
Children progressing beyond Level 0 conceive of some purpose of their activity beyond simply executing the procedural routines involved. At level 1, this purpose is focused on obtaining a good outcome, either as a task objective or a strategy (see Table 1). At levels 2 and above, the focus shifts from outcome to analysis of the effects operating to produce outcomes, again both as a task objective and strategy. At level 4, this analysis begins to focus on determining the effects of features as a task objective and on strategies for achieving this objective, with levels 5 and
Metastrategic Knowledge and Strategic Performance
233
6 incorporating the critical features, respectively, of focusing on individual features and controlling the influence of nonfocal features. (As reflected in Table 1, level 3 is not defined in the strategy domain and level 6 is not defined in the task domain.) These levels are discussed in further detail in the next section, in conjunction with examination of the strategic performance of children exhibiting the successive levels of metastrategic understanding. Children working with social task content overall exhibited lower levels of metastrategic understanding than did those working with physical task content (see Table 2). This outcome is consistent with earlier findings (Kuhn et al., 1995) of superiority of strategic performance for physical task content (also replicated in the strategic performance data obtained here). The physical materials and representations of outcomes evidently made the physical task more understandable to children, reflected in both higher metastrategic levels and higher levels of strategic performance-an outcome that in itself constitutes one form of support for a connection between strategic and metastrategic functioning. In looking more closely at this connection, however, the two groups (those who worked on a physical task and those who worked on a social task) are combined, since the connection should make itself apparent irrespective of task content. Children in both the social and physical task groups most often increased in their understanding from the first assessment, when they had had little experience with the task, to the final assessment when they had been working with the task for seven weeks. Of the 47 participants, only three showed any decline in metastrategic level from first to second assessment (one with respect to task objective and one with respect to both components, while one showed a mixed pattern of decline with respect to strategy but an increase with respect to task objective). Seven children showed no change from first to second metastrategic assessment. The remaining 37 (79%) showed an increase in level (on one or both components). Because the metastrategic assessment method is a verbal one and therefore subject to false negatives (i.e., the child may have a higher level of understanding than he or she communicates verbally), for purposes of examining the connection to strategic performance a child was classified as exhibiting the level of metastrategic understanding that was the highest exhibited in either the task objective or the strategy portion of the interview. A child thus in effect had two chances to exhibit metastrategic understanding, in communicating either the objective of the task or the strategies used to achieve it. Overall, of the 47 participants, 20 achieved their highest level in their characterizations of the task objective, 16 achieved their highest level in their characterizations of strategies, and 11 exhibited an equivalent level in the two portions of the interview. These patterns by individual level are informative and are summarized in Table 2. Children who progress beyond a procedural focus and attain level 1 (focused on obtaining a good outcome), for example, are more likely to do so with respect to their strategy knowledge than their task objective knowledge. Rarely
234
Kuhn and Pearsall
Table 2.
Patterns
for the Two Components of Metastrategic
of Performance
Understanding
Highest metastrategic
Highest level on strategy
I 1, 1 6,2 I 3.5
6 2
Equal levels
(total)
level
0 (procedural) 1 (outcome) 2 (instances) 3 (undistinguished features) 4 (multiple features) 5 (single features) 6 (single features with control) (total) Note.
Highest level on task objective
20
Entries are numbers of participants in italics.
4 0 4 16
2, 1 6 I 0 0
1 0 11
(3) (13) (5) (8) (5) (9) (4) (47)
showing each pattern. Those in the social task group appear
does a child cite achieving a good outcome as the task objective without also identifying as a strategy the choice of instances likely to produce a good outcome. Almost half exhibit both task objective and corresponding strategy knowledge. Others, however, may mention the strategy of choosing instances predicted to be positive, while their task descriptions remain at the procedural level. These children may see the good-outcome strategy as desirable but they do not link it to a task objective. We examine further the patterns across task and strategy components represented in Table 2 in turning now to the connections between metastrategic and strategic performance. Strategic Performance as a Function of Level of Metastrategic Understanding Analyses of strategic performance followed those used by Kuhn et al. (1995). Strategic skill can be assessed by a number of different indicators, including correct prediction of outcomes, attention to relevant evidence (i.e., comparison across two or more instances), and valid inference. Our earlier work has shown prediction and inference analyses to yield roughly equivalent results. For the sake of brevity, and since it can be regarded as a key indicator of the quality of strategic performance, we limit the present analysis of strategic performance to the indicator of valid inference. The task procedure does not require children to draw inferences at any point (they are always free to say they don’t know or haven’t yet found out), but of those inferences a child does choose to make, how many are valid, i.e., based on available evidence sufficient to support the inference? A valid inference regarding the effect of sail size, for example, is based on a comparison of two instances in which the feature of interest varies (e.g., large and small sails) but other features do not and the outcomes are compared. Invalid inferences took a variety of forms (see Kuhn et al., 1992, and Kuhn et al., 1995, for extended
Metastrategic
Knowledge
and Strategic
Performance
235
description). For example, inferences were based on a single instance or on a comparison of instances in which multiple features varied. Since a child’s investigation was continuous and accumulative across sessions, rather than use the session as an arbitrary starting and stopping point, we followed the analytic procedure adopted by Kuhn et al. (1995) and divided each child’s entire sequence of inferences into initial, middle, and final thirds. We then computed, for each child, the percentage of inferences that were valid for each third, and these percentages are employed as the dependent measure of strategic performance. Similar data were obtained for other indicators such as the percentage of evidence-based inferences and degree of prediction error, but, as noted above, only results for percentage of valid inference are reported here and related to metastrategic performance. Consonant with earlier findings by Kuhn et al. (1992) and Kuhn et al. (1995), a child’s percentages of valid inference in some cases stayed constant but most often increased, and infrequently decreased, across the seven-week period. Figures l-6 portray strategic and metastrategic performance over time. Children are grouped by highest level of metastrategic understanding attained, with each figure devoted to children of a particular metastrategic level. (As reported previously, in only three cases was this an initial level that was not sustained.) In each figure, the main graph portrays percentage of valid inferences (Y-axis) during each of the three segments (X-axis), for each individual child. As indicated in the figure legend, light dotted lines represent a single child. When more than one child showed identical patterns, they are represented by thicker lines of various types. Superimposed on each graph is a box depicting metastrategic performance at the two occasions (sessions 2 and 7) for each of the children represented in this graph. In the large majority of cases, as reflected in Figures 1-6, both metastrategic understanding and strategic performance are initially low, with percentage of valid inference in many cases at zero. Over time, many (though not all) children show strategic improvement, and almost all, as reported previously, show metastrategic advance, the most frequent pattern being an increase of one level, although many children show greater advances. Table 3 summarizes the relations between strategic performance and metastrategic level that are reflected in Figures l-6. The indicator of strategic performance in Table 3 is whether the child attained significant strategic mastery of the task as reflected in a percentage usage of valid inference greater than 50%. Children at levels 0 and 1, Figure 1 shows, never achieve significant strategic success. In fact, their percentages of valid inference most often remain at zero. Only five of the 16 children at these levels show any valid inference at all, and the highest percentage shown by any of the five for any third is 23%. In contrast, children at levels 2 and above begin to exhibit some strategic mastery. Achievement of strategic success, however, remains limited to a minority of children at levels 2 through 5, not increasing greatly even when the focus of analysis shifts from instances to the more appropriate one of features (level 2 to 3).
236
Kuhn and Pearsall
80
70
60
50
40
30
_
n=2
EC=
n=3
=
n=lO (Total Ne16)
Figure 1. Strategic and metastrategic attain metastrategic levels 0 or 1.
performance
over time among children who
Metastrategic Knowledge and Strategic Performance
237
100
90
80
70
60
SO
,
40
c
30
20
10
0
!
2
1
-___
3
n-l
-
n=3
-
n=4
(Total N=5)
Figure 2. Strategic and metastrategic attain metastrategie level 2.
performance
over time among children who
Kuhn and Pearsall
238
Metastrategic Level
3 -““F_‘_y_~_y_~ 2 90
, __--------_ -O,.-’
I
- - -_:_y---a _--=~-~=;-_c_~_
___-
‘.J__’
---mm__.
_*L
80
70
__a.
60
________----
,___--
/
I /
/
/
, /
I
50
/
,
/.
,
10
’
’
’
/ /
’ /
/
/
/
,
, ,
,
/
/
/
/ /
/
/
’
/
’
/ /
/
/
20
/
, ,
/
,
/
/
/ /
30
------,--_-___
/
/
40
/
c---e-.._
/
/
/
,
/
/
,
/
,
,
/
_--
0 ----
n=l
_-
w3
3E3i:
n=4
(Total N=8)
Figure 3. Strategic and metastrategic attain metastrategic level 3.
performance
over time among children who
Metastrategic Knowledge and Strategic Performance
239
100 90
80
70 --
, I / I
60 --
// /
50 --
40 --
/
30 --
20 -/ 10 --
/
----___ ---____ o Z______________~=~=r._&___,-,_____ ____________________------~111~11-----1 ____ n=l _ _ n=2 -
I
/
/
,
/
I
/
,
I
I
/
,
/
/
,
/
I
,
/
___---
L--
__I__-___ 2
3
n=3
(Total N=S)
Figure 4. Strategic and metastrategic attain metastrategic level 4.
performance
over time among children who
Kuhn and Pearsall
240
100
Y-
Metastrategic Level
5 90
_____--
4 _______-----3 2 1 O-
80
70
60
50
,
1
____
n=l
-
n=3
-
-
n=4
B
n+j
/
/
,
2
,
/
/-
,
/
/
,
,
/
,
/
,
,
/
,
,
/
3
(Total N=9)
Figure 5. Strategic and metastrategic attain metastrategic level 5.
performance
over time among children who
241
Metastrategic Knowledge and Strategic Performance
Metastrategic tevel L-C-* ‘;--2~t”- -_*----____ 22 - - - ,.-L ___---__-:, -$-----A_,___
*_-----__ --a_
100 90 .
90 ,
70 .
,
,
,
/
,
,
,
,
60 -
50 . 40 30 20 lo-
,
Figure 6. Strategic and m&strategic attain me~strat~ic Ievel6.
performance
over time among children who
242
Kuhn and Pearsall
Table 3.
Summary of Strategic Performance as a Function of Metastrategic 50% or less valid inference
Highest metastrategic level 0 (procedural) 1 (outcome) 2 (instances) 3 (undistinguished features) 4 (multiple features) 5 (single features) 6 (single features w/control) (Total) Note.
3 13 4 6 4 8 1 (39)
Level
Greater than 50% valid inference 0 (0%) 0 (0%) 1 (20%) 2 (25%) 1 (20%) l(ll%) 3 (75%) (8)
Entries are numbers of participants showing each pattern. N = 47. Percentages in parentheses are the percentages of participants at each metastrategic level who attained strategic mastery.
Not until level 6 does strategic success become the norm, and even then it is not universal. The relatively small proportion of children who achieve strategic success confines statistical analysis to a comparison of children at metastrategic levels 0 or 1 and those at higher levels. Strategic success is confined to children in the latter group, and the contrast between the two groups is statistically significant (x2 (1, N= 47) = 4.91, p < .OS). Further examination of Table 2 sheds some additional light on these patterns. All participants at level 3 by definition achieve this level on the task objective component (since a corresponding level did not emerge for the strategy component). Similarly, level 6 is attained only for the strategy component, since the concept of controlled comparison is expressed only as a strategy and not as a task objective. Levels 4 and 5, however, where children may achieve the level with respect to either task objective, strategy, or both, are informative. Level 4, recall, marks the first clear focus on features as the object of analysis. Most level 4 children achieve the level with respect to strategy, suggesting that attention may turn to the features of the instances being compared before analysis of the features’ effects becomes explicitly recognized as the task objective. Most informative, however, is the pattern at level 5, at which the need to focus on single features is recognized. All but one of the nine level-5 children achieves the level with respect to task objective but not strategy. The one exception is a child who achieves the level with respect to both task objective and strategy and is the only one of this group to achieve strategic success (Table 3). This pattern is consistent with the claim that both metastrategic components are necessary for strategic success. The eight of nine level-5 children who displayed level 5 with respect to task objective but not strategy understood the task was one of analyzing the effects of individual features but they apparently did not know of a strategy for doing so. Only the child who displayed level 5 on both components was successful strategically. Results for the final level, however, suggest that even a high level of understanding of both task objective and strategy does not insure strategic success and other
Metastrategic
Knowledge
and Strategic
Performance
243
factors may be contributory. Of the four children at this level, three show valid inference percentages of 60, 75, and lOO%, but the percentage of one of the four remained at zero throughout the task period. Finally, we can examine the metastrategic-strategic relationship in the other direction, with comparable results. Of the eight strategically successful children (Table 3), the majority show metastrategic competence at least at what appears to be a critical level of achieving an analytic focus (level 2). All but one achieves this level with respect to task objective, and all but one achieves it with respect to strategy. All of the eight achieve it with respect to at least one of the two.
DISCUSSION In the domain in which we have investigated it-inductive causal reasoning in a multivariable context-our results support the thesis that metastrategic knowledge is both assessable as an entity distinct from strategic performance and related to it. Although the method of assessment of metastrategic knowledge that we employed is dependent on verbal facility and therefore vulnerable to underestimation (participants may understand more than they are able or willing to verbalize), our assessment nonetheless indicated a coherent pattern of connections between strategic and metastrategic functioning. Although initial competence was minimal in both the strategic and metastrategic realms for almost all participants, the microgenetic method allowed observation of their joint evolution. Individual case studies of each participant’s pattern of change across the different indicators showed that some did exhibit what appeared to be linked gains in the two realms (strategic and metastrategic) across time. Others, however, showed gains in one but not the other. Theoretically, there is no reason to predict a tight trial-by-trial linkage of functioning in the two realms. Alternatively, a model that the data support is a necessary-but-not-sufficient relation, with both of the two metastrategic components necessary (but not sufficient) for mastery in strategic performance. Other factors, including simply time to consolidate the gain in metastrategic understanding, presumably play a role in exactly when and in what form this metastrategic gain is realized in strategic performance. The finding of changes in the strategic and metastrategic realms over the same time period, however, does not establish direction of causality, and other possibilities (than a necessary-but-not-sufficient “gatekeeper” relation of one to the other) should be considered. Perhaps strategic success leads to greater metastrategic awareness, rather than the reverse. Or perhaps it is strategic failure that leads to a metastrategic search for new approaches. And even if the direction of causality is from metastrategic to strategic, multiple possibilities exist. Failure to attain what the individual construes as the task objective is a possible impetus for strategic variation and hence improvement, but so might newly achieved metastrategic insight serve as the impetus for strategic change.
244
Kuhn and Pearsall
Rather than one of these possibilities being correct to the exclusion of the others, more likely, in our view, is the existence of multiple directions of influence between the strategic and metastrategic-the kind of bootstrapping feedback relation that Case (1998) and others have proposed in contemplating the relations between more general and more specific cognitive structures. In other words, advances at the general level feed down to support the more local structures, while at the same time gains in the local structures feed up to strengthen and consolidate the general ones (what Case calls a “hierarchical learning loop”). In the present case, the likelihood of such a bidirectional model is enhanced by the fact that initial competence was minimal in both strategic and metastrategic realms. Metastrategic understanding was able to emerge only in the context of strategic engagement with the task. Yet, as it developed in the course of this strategic engagement, it set limits on strategic progress (but still, as we saw, was not sufficient to guarantee strategic success). Also relevant in the present case, and implicative of the role of metastrategic competence, is the fact that most children continued to use a combination of less successful strategies-those in the process of being discarded-as well as more successful ones in the process of emerging. Use of successful strategies may “feed up” to the metastrategic level, strengthening awareness of the strategy and understanding of its value. But the use of unsuccessful strategies may also feed up, strengthening awareness of the strategy’s ineffectiveness and leading to recognition of the need for better strategies. Similarly, both components of metastrategic knowledge may feed down to the strategic level. Gains in understanding of the task objective feed down to guide the application of strategies. Awareness and understanding of available strategies likewise supports their effective implementation. In contrast, the paths of influence between the two components of metastrategic knowledge are not as apparent. Knowledge of a task objective cannot supply strategies to achieve it when they are absent. Nor can availability and awareness of strategies provide a task goal. We propose, then, that coordination of these two components of metastrategic knowledge may be a central task of the metastrategic knowledge system, and we would target how this coordination is achieved as a worthy topic of further investigation. In the case of the present task, recall, at the lower levels of metastrategic understanding, strategic understanding often exceeded task understanding (when children’s attention turned to outcomes or to features before they verbalized these as relevant to task objectives); at the higher levels, however, it was most often lack of awareness of a strategy to implement task objectives that impeded progress. The present work examines metastrategic and strategic functions within the context of a certain kind of cognitive task. How widely do these findings apply? Inclusion of two kinds of task content (physical and social) in our design allows us to conclude that that they apply across a range of content within this task domain. Do they apply as well, however, to different kinds of tasks?
Metastrategic
Knowledge
and Strategic
Performance
245
Siegler (1996) has questioned the extent to which metacognitive or metastrategic factors are influential in task performance, turning instead to associative strength as a more influential factor in strategy choice and hence developmental change in strategic performance. It is worthwhile, therefore, to compare the kinds of cognitive tasks and strategies he and his colleagues have studied with those we have examined here, to see if the differing content might account for the differing importance attributed to metacognitive factors. We believe that it does. Recent reviews of microgenetic methodology in the study of cognitive development (Kuhn, 1995; Siegler, 1996) have emphasized the commonality of findings across very different kinds of cognitive tasks. As numerous investigators have now reported, the same child is likely to display multiple strategies, even when the task remains constant, and development entails gradual shifts in the distribution of their usage. But these striking similarities in findings across disparate tasks may have led to an overlooking of important differences, ones that become important when we begin to look beyond these now well-replicated findings to consider more subtle process questions, such as the interaction of strategic and metastrategic factors. Siegler and colleagues have focused their studies on children’s mastery of the basic arithmetic “number facts” of single-digit addition and subtraction. One might anticipate a central role for associative learning mechanisms in the mastery of number facts, since associative learning is the ultimate task goal. By middle childhood, we want and expect children to have learned these facts as associative links in their knowledge base of mathematics, such that the problem statement 7+8 produces the response 15 rapidly and automatically, without the need for intervening mental operations. The “back-up” strategies that Siegler and his colleagues have studied, involving procedures such as counting on fingers or the “min” strategy (counting up from the larger addend), are temporary aids that are expected to drop out once the necessary associative learning has been achieved. The desired performance end state here does not involve strategy use except in the limited sense that we might call retrieval (i.e., asking oneself, “Do I know an answer to 7+8?“) a strategy. In this domain, then, associative strengths between the various number fact problems and corresponding responses we would expect to be the most direct and strongest predictor of performance, certainly stronger than less direct measures that have been investigated, such as the individual’s judgments of problem difficulty (regarded as a metastrategic indicator). In the case of the competencies we have examined, in contrast, the ultimate developmental goal is replacement of primitive strategies not with associative learning but with more complex and demanding strategies, ones that continue to require exacting cognitive processing each time that they are applied. This is an important difference, for it says a great deal about the anticipated role of metastrategic factors. If more effortful and exacting strategies are to be applied in lieu of more primitive ones that continue to exist in the repertory, some explanatory governing mechanism needs to be invoked to account for such a choice. It is here that
246
Kuhn and Pearsall
we have invoked metastrategic understanding as playing an important role. Also likely to be influential is the other component of metastrategic understanding examined here-understanding of task objective-which, as we saw, is variable and undergoes change, again in contrast to what is the case in a domain like mastery of number facts. In fact, as we suggested at the outset, inductive reasoning is a domain in which metastrategic understanding is of significance in its own right, rather than attaining its significance only from an alleged effect on strategic performance. We want to know whether an individual is consciously aware of and understands the investigative strategies and associated inference rules that allow certain conclusions to be drawn and render others invalid and whether the individual understands the connection between these strategies and rules and the task objective of identifying the causal structure of the domain. Equally important, does the individual recognize invalid strategies and why they fail to achieve the task objective? This understanding is an intrinsic aspect of the competence being investigated and of importance in its own right. We need to investigate the connections between this understanding and strategic performance, but one does not warrant conceptual priority over the other. Strategic performance is more visible, but it is metastrategic understanding that is likely to dictate whether and when strategies are applied. At least within the range of cognitive tasks that entail complex, effortful cognitive processing across all degrees of mastery, we advocate as most promising a model of multiple, bidirectional paths of influence between performance and understanding.
REFERENCES Brown, A. (1975). The development of memory: Knowing, knowing about knowing, and knowing how to know. In H. Reese (Ed.), Advances in child development and behavior (Vol. 10) (pp. 103-152) New York: Academic Press. Brown, A. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.), Advances in instructionalpsychology (Vol. 1) (pp. 77-165) Hillsdale, NJ: Erlbaum. Brown, A. (1997). Transforming schools into communities of thinking and learning about serious matters. American Psychologist, 52 , 399-413. Case, R. (1998). The development of conceptual structures. In W. Damon (Series Ed.), D. Kuhn & R. Siegler (Vol. Eds.), Handbook of child psychology: Vol. 2. Cognition, language, and perception (5th ed.) (pp. 745-800) New York: Wiley. Gardner, H. (1991). The unschooled mind: How children think and how schools should teach. New York: Basic Books. Kuhn, D. (1982). On the dual executive and its significance in the development of developmental psychology. In D. Kuhn & J. Meacham (Eds.), On the development of developmental psychology. Contributions to human development (Vol. 8) (pp. 81-110) Basel: Karger.
Metastrategic Knowledge and Strategic Performance
Kuhn, D. (1995). Microgenetic
247
study of change: What has it told us? Psychological Science, 6, 133-139. Kuhn, D. (in press). Metacognitive development. In C. Tamis-LeMonda (Ed.), Child psychology: A handbook of contemporary issues. New York: Garland. Kuhn, D., Amsel, E., & O’Loughlin, M. (1988). The development of scientific thinking skills. Orlando FL: Academic Press. Kuhn, D., Garcia-Mila, M.. Zohar, A., & Andersen, C. (1995). Strategies of knowledge acquisition. Societyfor Research in Child Development Monographs, 60, (4), (Serial no. 245). Kuhn, D., & Phelps, E. (1982). The development of problem-solving strategies. In H. Reese (Ed.), Advances in child development and behaviol; Vol. 17. (pp. l-44) New York: Academic Press. Kuhn, D., Schauble, L., & Garcia-Mila, M. (1992). Cross-domain development of scientific reasoning. Cognition and Instruction, 9, 285-327. Moshman, D. (1998). Cognitive development beyond childhood. In W. Damon (Series Ed.), D. Kuhn & R. Siegler (Vol. Eds.), Handbook ofchildpsychology: Vol. 2. Cognition, language, andperception (5th ed.) (pp. 947-978) New York: Wiley. Perkins, D. (1992). Smart schools: Better thinking and learning for every child. New York: Free Press. Schauble, L., Klopfer, L., & Raghavan, K. (1991). Students’ transition from an engineering model to a science model of experimentation. Journal ofResearch in Science Teaching, 28, 859-882. Schneider, W., & Bjorklund, D. (1998). Memory. In W. Damon (Series Ed.), D. Kuhn & R. Siegler (Vol. Eds.), Handbook of child psychology: Vol. 2. Cognition, language, and perception (5th ed.) (pp. 467-521) New York: Wiley. Siegler, R. (1996). Emerging minds: The process of change in children’s thinking. New York: Oxford University Press. Sophian, C. (1997). Beyond competence: The significance of performance for conceptual development. Cognitive Development, 12, 281-303.
Development,
13,227-247
0 1998 Ablex Publishing
(1998)
All rights of reproduction
lS!SN 0885-2014
reserved.
RELATIONS BETWEEN METASTRATECIC KNOWLEDGE AND STRATEGIC PERFORMANCE
Deanna Columbia
Kuhn
University
Susan Pearsall Columbia
University
Two components
of metastrategic knowledge-knowledge of task objectives and knowledge of strategies-are distinguished and assessed in the context of an inductive causal reasoning task administered to fifth graders in a microgenetic design. Both metastrategic knowledge and strategic performance improved over a seven-week period of engagement with the task. However,
children were unlikely to attain strategic mastery until they had reached particular levels of metastrategic understanding. Discussion focuses on (a) the likelihood of multiple directions of influence between strategic and metastrategic knowledge systems, (b) coordination of task and strategy components of metastrategic knowledge as a central task of the metastrategic knowledge system, and (c) the generality of the present findings.
Initial research on children’s metacognitive or metastrategic knowledge focused almost entirely on memory skills (Brown, 1975, 1978; for current reviews see Kuhn, in press, or Schneider & Bjorkhmd, 1998). More recent, broader conceptions of metastrategic knowledge encompass awareness, understanding, monitoring, and management of one’s strategic performance of many kinds of cognitive tasks. The question of whether and how metastrategic knowledge relates to strategic performance is one worthy of investigation within this broader framework, especially as it relates to a topic receiving increasing attention from educatorsthe connection between performance and understanding (Gardner, 1991; Perkins, The research reported here was supported by a grant from the National Science Foundation to the first author. Direct all correspondence New York, NY 10027
to: Deanna
Kuhn, Box 119, Teachers
College, Columbia
University,
Manuscript received January 6,1997;
revision accepted July 2,1997
227
228
Kuhn and Pearsall
1992). As has by now been widely observed in both research and applied settings, skill in executing the component parts of a complex cognitive strategy is insufficient to ensure that the strategy will be performed in appropriate contexts in the absence of specific instruction (Brown, 1997; Kuhn, 1982). Some further, executive function is implicated (Kuhn, 1982), one that increasingly has been labeled as “metastrategic.” Reasoning is a domain in which we would expect metastrategic understanding to be particularly important (Moshman, 1998). It would seem desirable that people be aware of the inferences they make and the strategies or rules that allow them to be made, and we might assume that those who have such awareness will reason better than those who do not. The present work is addressed to the role of metastrategic knowledge in the domain of inductive reasoning, an area where there has not yet been a great deal of exploration of metacognitive or metastrategic phenomena. The theoretical conceptualization of the relation between metastrategic and strategic knowledge that guides the present work builds on a model of strategic development involving the coexistence of multiple strategies over time within an individual, with overlapping increases and decreases in frequencies of usage (Kuhn & Phelps, 1982; Kuhn, 1995; Siegler, 1996). The relation is conceived of as a bidirectional one in which “metastrategic understanding may both guide (in implicit form) and follow (in more explicit form) strategy development” (Kuhn, Garcia-Mila, Zohar, & Andersen, 1995, p. 114). This bidirectional conception of the relation between metastrategic understanding and strategic performance bears a similarity to Sophian’s (1997) conception of competence and its relation to performance: Competence guides performance and in turn is shaped by it. This bidirectional conceptualization suggests dual aspects of metastrategic understanding. In order for a strategy to be used, metastrategic understanding must be sufficient to enable the individual to recognize its applicability in relation to goals defined by the objectives of the task. An outcome of this use should be increasing awareness of the strategy and understanding of its power, again in relation to task objectives. We propose, then, two components of metastrategic understanding. One is the understanding and awareness of the nature and requirements of the task. This aspect of metastrategic knowledge is closely related to what Siegler (1996) has termed a goal sketch. A second component of metastrategic knowledge is awareness and understanding of the strategies available in one’s repertory that are potentially applicable to the task. The challenge of effective metastrategic functioning can be conceived of as one of coordination of the two components, understanding of the task and understanding of potential strategies, so that the ideal intersection of the two is realized in task performance. In the present study, we undertake separate measures of these two components of metastrategic knowledge-task understanding and strategy understandingand examine their relation to strategic performance. Because we believe a
Metastrategic
Knowledge
and Strategic
Performance
229
dynamic perspective offers more insight than a static one, we conducted the study microgenetically (Kuhn, 1995), examining performance in successive sessions with the same task over a period of seven weeks. Metastrategic knowledge we assessed twice, once at the second weekly session and once at the final (seventh) session. The design thus permits an assessment of change over time in both metastrategic knowledge and strategic performance. Finally, to enhance the generality of findings, we replicated the design across two kinds of task content, physical and social. METHOD Participants Participants in the study were lo- and 1 l-year-old fifth-grade students. This age level was chosen as one at which the strategic competencies being assessed are just beginning to emerge, based on findings from previous research (Kuhn, Amsel, & O’Loughlin, 1988; Kuhn et al., 1995; Kuhn, Schauble, & Garcia-Mila, 1992). Participants came from an urban public school serving a predominantly lower-income minority population. Of the 47 participants, 23 were female and 24 male. Procedure Strategic Task and Assessment. The strategic task presented to 23 of the 47 participants was one of the two physical science problems (the boat and car problems) used in research by Kuhn et al. (1995). The remaining 24 participants worked on one of the two social science problems (the TV and school problems) used by Kuhn et al. Full details of these tasks and the procedures involved in their repeated administration in a microgenetic design are available in the monograph by Kuhn et al. (1995). Here the boat problem is briefly described as an example. Because children were asked to engage the task repeatedly, it was important that it be situated within a pragmatic context that would provide the motivation and rationale for continuing to work on the task. We achieved this goal by asking children to investigate the causal structure (to find out “what makes a difference” and “what doesn’t make a difference”) characteristic of a particular problem domain and to use this developing knowledge to predict outcomes. The interviewer asked a range of questions (What are you going to find out? What do you think the outcome will be? What have you found out?) but provided no direction or feedback regarding the child’s activities. The purpose of the questions was to encourage the child to engage in cognitive activity, without trying to influence the specific direction it took. In designing the problems, the goal was to include two features that a child was likely to believe to be noncausal, with the effect in fact noncausal (and hence the belief confirmed) for one and the effect in fact causal (and the belief hence subject
230
Kuhn and Pearsall
to disconlirmation) for the other. Similarly, it was aimed to include at least two features the child would believe to be causal, with the effect in fact causal (and hence the belief confirmed) for one and the effect in fact noncausal (and the belief hence subject to disconfirmation) for the other. Complex (curvilinear and interactive) effects provided conditions for partial confirmation and partial disconfirrnation of beliefs (depending on which levels of the features were examined). Since children’s beliefs were not entirely predictable, without custom-designing problems for each participant, these goals could only be approximated. However, extensive pilot testing made it possible to identify features for which these conditions were met in a majority of cases. In the boat problem, a set of manipulable features influenced the speed with which model boats were pulled down a towing tank by a weight-and-pulley system. The task was to ascertain the effects these factors had on the boats’ speed. The problem was adapted from one developed by Schauble, Klopfer, and Raghavan (1991). The variable features were boat size, sail size, sail color, weight, and water depth. Two of the features, sail color and sail size, have no effect on outcome. Boat size has a simple causal effect, weight has a causal effect in interaction with size (it has an influence only with small boats), and depth is a three-level feature having a partial, curvilinear effect (the deep and medium-deep levels do not differ from one another but yield a faster outcome than the shallow level). At each session, the child was free to select boats and water levels for investigation and to record any information desired in a notebook provided for this purpose. Prior to running the boat, the child was asked to predict the outcome and explain the basis for this prediction. Following each observation, the child was questioned as to what inferences could be drawn, and at each session, a final judgment was elicited for that session as to whether a feature did or did not make a difference. The number of instances (boats, in this case) a child observed ranged between two and six per session, depending on time and the child’s wishes, but with the mode being five once the procedure became familiar (beginning with the second session). As part of a separate study of peer collaboration, children also participated in another seven sessions, one per week during the same seven-week period, during which they collaborated with a peer on a different, but parallel, task within the same domain, i.e., children working on a physical task worked on another physical task with a peer and those working on a social task worked with a peer on another social task. This additional task experience most likely contributed to strategic progress. Beyond this effect, however, the peer condition and its comparison to the solitary condition are not relevant to the questions addressed in the present study and are not examined further. Assessment of metastrategic understanding was confined to the task on which the child worked individually. understanding was initially Metastrategic Assessment. Metastrategic assessed in a separate interview that occurred on the same day as the second stra-
Metastrategic
Knowledge
and Strategic
Performance
231
tegic task session. The interview was readministered on the same day as the final (seventh) task session. The interview employed to assess metastrategic understanding was designed to provide separate indices of the task and strategy components identified earlier. The basic concept underlying the metastrategic assessment method was to have participants externalize their metastrategic understanding by having to communicate it to another child. Another child (not participating in the study) was brought to the interview room and it was explained that this new child would like to know what the work was that the participating child had been doing here each week. The interview began with the assessment of the task component. Specifically, the purpose was to assess whether children understood what the objective of the task was, i.e., to analyze the effects of the various features on outcome, and, if not, to assess what the child did construe the task objective to be. The interviewer instructed the child, “Please tell (the nonparticipating child) what you are supposed to do here.” The interviewer provided nondirective prompts to elaborate, if appropriate, but no feedback or other direction was given. The nonparticipating child had been previously instructed not to intervene or ask questions. The second segment of the metastrategic interview was addressed to the strategy component. Its purpose was to assess the understanding a child has of the strategies needed to accomplish the task. Do children understand the need for strategic behavior at all, and, if so, what awareness do they have of strategies that would be applicable and effective? In particular, do they understand the need to strategically choose for examination instances that provide informative comparisons? Because of the critical role of this strategic aspect of the task, it was the focus of the assessment of this second metastrategic component. Unless the child had already provided an answer in the first segment of the interview, the interviewer asked, “Can you explain to how you decide which ones to investigate?’
RESULTS Metastrategic
Performance
The two portions of the metastrategic interview, pertaining to task objective and strategy components, were coded separately. The respective levels of attainment for each component, however, can be roughly equated conceptually across the two components, as reflected in Table 1, which summarizes the successive levels of understanding we identified with respect to each of the components. Each interview was coded independently by two coders for each of the two metastrategic components. Percentage agreement between the two coders was .83 for the task objective component and .88 for the strategy component. Children at the lowest level (0) in Table 1 (who by definition score no higher than level 0 on either component), exhibit no insight regarding the task objective
Kuhn and Pearsall
232
Table 1.
Levels of Metastrategic Understanding Task objective
Strategy
level 0 level 1
Procedural only Attain a good outcome (“Find out which goes fastest”)
None (“Just choose anything”) Choose instances believed to yield a good outcome (“Choose the fastest”)
level 2
Analysis at instance level (“Find out how fast different boats will go”)
Try different instances and observe outcomes (‘Try different ones to see how fast they go”)
level 3
Analysis at feature level without discriminating features (“Find out what things are making a difference”)
level 4
Analysis at feature level with reference to multiple features (“See whether the size and weight make a difference”)
Compare instances in which multiple features are varied (“Compare a big heavy one and small light one to see if it makes a difference”)
level 5
Analysis at feature level with focus on single feature at a time (“Find out whether each of the features makes a difference”)
Compare instances in which a single feature is varied and other features are not mentioned (“See if the big or small one goes faster”)
level 6
Compare instances in which a single feature is varied and non-variation of other features is indicated (“Change just the weight and see if it makes a difference”)
and, when asked about it, simply describe the procedures involved in the task. Likewise, they exhibit no knowledge of strategies nor awareness of the need for them. The following is an example of one child’s initial metastrategic interview: what you are supposed to do here.) First you have to pick what items you (Please tell want for the car. And then what things that you are going to figure about. So you put a marker in each one you think you’re going to figure out. And then, you could guess how fast the car would go-number one, two, three, or four. And then you have to put a marker on which one you think is going to.. .on which one is going to go. And then after we ran it, and after that you see what things that you figured out about. And you put them.. .put a marker on each one you think you found out about, and then you answer questions.. .does it matter, and all that sort of questions. And then you’re done and then you got a couple more and that’s about it. how you decide which ones to investigate?) There are some (Can you explain to things here that you could pick. (How do you decide?) Ah, you just guess. To see what items you want to see in the car.
Children progressing beyond Level 0 conceive of some purpose of their activity beyond simply executing the procedural routines involved. At level 1, this purpose is focused on obtaining a good outcome, either as a task objective or a strategy (see Table 1). At levels 2 and above, the focus shifts from outcome to analysis of the effects operating to produce outcomes, again both as a task objective and strategy. At level 4, this analysis begins to focus on determining the effects of features as a task objective and on strategies for achieving this objective, with levels 5 and
Metastrategic Knowledge and Strategic Performance
233
6 incorporating the critical features, respectively, of focusing on individual features and controlling the influence of nonfocal features. (As reflected in Table 1, level 3 is not defined in the strategy domain and level 6 is not defined in the task domain.) These levels are discussed in further detail in the next section, in conjunction with examination of the strategic performance of children exhibiting the successive levels of metastrategic understanding. Children working with social task content overall exhibited lower levels of metastrategic understanding than did those working with physical task content (see Table 2). This outcome is consistent with earlier findings (Kuhn et al., 1995) of superiority of strategic performance for physical task content (also replicated in the strategic performance data obtained here). The physical materials and representations of outcomes evidently made the physical task more understandable to children, reflected in both higher metastrategic levels and higher levels of strategic performance-an outcome that in itself constitutes one form of support for a connection between strategic and metastrategic functioning. In looking more closely at this connection, however, the two groups (those who worked on a physical task and those who worked on a social task) are combined, since the connection should make itself apparent irrespective of task content. Children in both the social and physical task groups most often increased in their understanding from the first assessment, when they had had little experience with the task, to the final assessment when they had been working with the task for seven weeks. Of the 47 participants, only three showed any decline in metastrategic level from first to second assessment (one with respect to task objective and one with respect to both components, while one showed a mixed pattern of decline with respect to strategy but an increase with respect to task objective). Seven children showed no change from first to second metastrategic assessment. The remaining 37 (79%) showed an increase in level (on one or both components). Because the metastrategic assessment method is a verbal one and therefore subject to false negatives (i.e., the child may have a higher level of understanding than he or she communicates verbally), for purposes of examining the connection to strategic performance a child was classified as exhibiting the level of metastrategic understanding that was the highest exhibited in either the task objective or the strategy portion of the interview. A child thus in effect had two chances to exhibit metastrategic understanding, in communicating either the objective of the task or the strategies used to achieve it. Overall, of the 47 participants, 20 achieved their highest level in their characterizations of the task objective, 16 achieved their highest level in their characterizations of strategies, and 11 exhibited an equivalent level in the two portions of the interview. These patterns by individual level are informative and are summarized in Table 2. Children who progress beyond a procedural focus and attain level 1 (focused on obtaining a good outcome), for example, are more likely to do so with respect to their strategy knowledge than their task objective knowledge. Rarely
234
Kuhn and Pearsall
Table 2.
Patterns
for the Two Components of Metastrategic
of Performance
Understanding
Highest metastrategic
Highest level on strategy
I 1, 1 6,2 I 3.5
6 2
Equal levels
(total)
level
0 (procedural) 1 (outcome) 2 (instances) 3 (undistinguished features) 4 (multiple features) 5 (single features) 6 (single features with control) (total) Note.
Highest level on task objective
20
Entries are numbers of participants in italics.
4 0 4 16
2, 1 6 I 0 0
1 0 11
(3) (13) (5) (8) (5) (9) (4) (47)
showing each pattern. Those in the social task group appear
does a child cite achieving a good outcome as the task objective without also identifying as a strategy the choice of instances likely to produce a good outcome. Almost half exhibit both task objective and corresponding strategy knowledge. Others, however, may mention the strategy of choosing instances predicted to be positive, while their task descriptions remain at the procedural level. These children may see the good-outcome strategy as desirable but they do not link it to a task objective. We examine further the patterns across task and strategy components represented in Table 2 in turning now to the connections between metastrategic and strategic performance. Strategic Performance as a Function of Level of Metastrategic Understanding Analyses of strategic performance followed those used by Kuhn et al. (1995). Strategic skill can be assessed by a number of different indicators, including correct prediction of outcomes, attention to relevant evidence (i.e., comparison across two or more instances), and valid inference. Our earlier work has shown prediction and inference analyses to yield roughly equivalent results. For the sake of brevity, and since it can be regarded as a key indicator of the quality of strategic performance, we limit the present analysis of strategic performance to the indicator of valid inference. The task procedure does not require children to draw inferences at any point (they are always free to say they don’t know or haven’t yet found out), but of those inferences a child does choose to make, how many are valid, i.e., based on available evidence sufficient to support the inference? A valid inference regarding the effect of sail size, for example, is based on a comparison of two instances in which the feature of interest varies (e.g., large and small sails) but other features do not and the outcomes are compared. Invalid inferences took a variety of forms (see Kuhn et al., 1992, and Kuhn et al., 1995, for extended
Metastrategic
Knowledge
and Strategic
Performance
235
description). For example, inferences were based on a single instance or on a comparison of instances in which multiple features varied. Since a child’s investigation was continuous and accumulative across sessions, rather than use the session as an arbitrary starting and stopping point, we followed the analytic procedure adopted by Kuhn et al. (1995) and divided each child’s entire sequence of inferences into initial, middle, and final thirds. We then computed, for each child, the percentage of inferences that were valid for each third, and these percentages are employed as the dependent measure of strategic performance. Similar data were obtained for other indicators such as the percentage of evidence-based inferences and degree of prediction error, but, as noted above, only results for percentage of valid inference are reported here and related to metastrategic performance. Consonant with earlier findings by Kuhn et al. (1992) and Kuhn et al. (1995), a child’s percentages of valid inference in some cases stayed constant but most often increased, and infrequently decreased, across the seven-week period. Figures l-6 portray strategic and metastrategic performance over time. Children are grouped by highest level of metastrategic understanding attained, with each figure devoted to children of a particular metastrategic level. (As reported previously, in only three cases was this an initial level that was not sustained.) In each figure, the main graph portrays percentage of valid inferences (Y-axis) during each of the three segments (X-axis), for each individual child. As indicated in the figure legend, light dotted lines represent a single child. When more than one child showed identical patterns, they are represented by thicker lines of various types. Superimposed on each graph is a box depicting metastrategic performance at the two occasions (sessions 2 and 7) for each of the children represented in this graph. In the large majority of cases, as reflected in Figures 1-6, both metastrategic understanding and strategic performance are initially low, with percentage of valid inference in many cases at zero. Over time, many (though not all) children show strategic improvement, and almost all, as reported previously, show metastrategic advance, the most frequent pattern being an increase of one level, although many children show greater advances. Table 3 summarizes the relations between strategic performance and metastrategic level that are reflected in Figures l-6. The indicator of strategic performance in Table 3 is whether the child attained significant strategic mastery of the task as reflected in a percentage usage of valid inference greater than 50%. Children at levels 0 and 1, Figure 1 shows, never achieve significant strategic success. In fact, their percentages of valid inference most often remain at zero. Only five of the 16 children at these levels show any valid inference at all, and the highest percentage shown by any of the five for any third is 23%. In contrast, children at levels 2 and above begin to exhibit some strategic mastery. Achievement of strategic success, however, remains limited to a minority of children at levels 2 through 5, not increasing greatly even when the focus of analysis shifts from instances to the more appropriate one of features (level 2 to 3).
236
Kuhn and Pearsall
80
70
60
50
40
30
_
n=2
EC=
n=3
=
n=lO (Total Ne16)
Figure 1. Strategic and metastrategic attain metastrategic levels 0 or 1.
performance
over time among children who
Metastrategic Knowledge and Strategic Performance
237
100
90
80
70
60
SO
,
40
c
30
20
10
0
!
2
1
-___
3
n-l
-
n=3
-
n=4
(Total N=5)
Figure 2. Strategic and metastrategic attain metastrategie level 2.
performance
over time among children who
Kuhn and Pearsall
238
Metastrategic Level
3 -““F_‘_y_~_y_~ 2 90
, __--------_ -O,.-’
I
- - -_:_y---a _--=~-~=;-_c_~_
___-
‘.J__’
---mm__.
_*L
80
70
__a.
60
________----
,___--
/
I /
/
/
, /
I
50
/
,
/.
,
10
’
’
’
/ /
’ /
/
/
/
,
, ,
,
/
/
/
/ /
/
/
’
/
’
/ /
/
/
20
/
, ,
/
,
/
/
/ /
30
------,--_-___
/
/
40
/
c---e-.._
/
/
/
,
/
/
,
/
,
,
/
_--
0 ----
n=l
_-
w3
3E3i:
n=4
(Total N=8)
Figure 3. Strategic and metastrategic attain metastrategic level 3.
performance
over time among children who
Metastrategic Knowledge and Strategic Performance
239
100 90
80
70 --
, I / I
60 --
// /
50 --
40 --
/
30 --
20 -/ 10 --
/
----___ ---____ o Z______________~=~=r._&___,-,_____ ____________________------~111~11-----1 ____ n=l _ _ n=2 -
I
/
/
,
/
I
/
,
I
I
/
,
/
/
,
/
I
,
/
___---
L--
__I__-___ 2
3
n=3
(Total N=S)
Figure 4. Strategic and metastrategic attain metastrategic level 4.
performance
over time among children who
Kuhn and Pearsall
240
100
Y-
Metastrategic Level
5 90
_____--
4 _______-----3 2 1 O-
80
70
60
50
,
1
____
n=l
-
n=3
-
-
n=4
B
n+j
/
/
,
2
,
/
/-
,
/
/
,
,
/
,
/
,
,
/
,
,
/
3
(Total N=9)
Figure 5. Strategic and metastrategic attain metastrategic level 5.
performance
over time among children who
241
Metastrategic Knowledge and Strategic Performance
Metastrategic tevel L-C-* ‘;--2~t”- -_*----____ 22 - - - ,.-L ___---__-:, -$-----A_,___
*_-----__ --a_
100 90 .
90 ,
70 .
,
,
,
/
,
,
,
,
60 -
50 . 40 30 20 lo-
,
Figure 6. Strategic and m&strategic attain me~strat~ic Ievel6.
performance
over time among children who
242
Kuhn and Pearsall
Table 3.
Summary of Strategic Performance as a Function of Metastrategic 50% or less valid inference
Highest metastrategic level 0 (procedural) 1 (outcome) 2 (instances) 3 (undistinguished features) 4 (multiple features) 5 (single features) 6 (single features w/control) (Total) Note.
3 13 4 6 4 8 1 (39)
Level
Greater than 50% valid inference 0 (0%) 0 (0%) 1 (20%) 2 (25%) 1 (20%) l(ll%) 3 (75%) (8)
Entries are numbers of participants showing each pattern. N = 47. Percentages in parentheses are the percentages of participants at each metastrategic level who attained strategic mastery.
Not until level 6 does strategic success become the norm, and even then it is not universal. The relatively small proportion of children who achieve strategic success confines statistical analysis to a comparison of children at metastrategic levels 0 or 1 and those at higher levels. Strategic success is confined to children in the latter group, and the contrast between the two groups is statistically significant (x2 (1, N= 47) = 4.91, p < .OS). Further examination of Table 2 sheds some additional light on these patterns. All participants at level 3 by definition achieve this level on the task objective component (since a corresponding level did not emerge for the strategy component). Similarly, level 6 is attained only for the strategy component, since the concept of controlled comparison is expressed only as a strategy and not as a task objective. Levels 4 and 5, however, where children may achieve the level with respect to either task objective, strategy, or both, are informative. Level 4, recall, marks the first clear focus on features as the object of analysis. Most level 4 children achieve the level with respect to strategy, suggesting that attention may turn to the features of the instances being compared before analysis of the features’ effects becomes explicitly recognized as the task objective. Most informative, however, is the pattern at level 5, at which the need to focus on single features is recognized. All but one of the nine level-5 children achieves the level with respect to task objective but not strategy. The one exception is a child who achieves the level with respect to both task objective and strategy and is the only one of this group to achieve strategic success (Table 3). This pattern is consistent with the claim that both metastrategic components are necessary for strategic success. The eight of nine level-5 children who displayed level 5 with respect to task objective but not strategy understood the task was one of analyzing the effects of individual features but they apparently did not know of a strategy for doing so. Only the child who displayed level 5 on both components was successful strategically. Results for the final level, however, suggest that even a high level of understanding of both task objective and strategy does not insure strategic success and other
Metastrategic
Knowledge
and Strategic
Performance
243
factors may be contributory. Of the four children at this level, three show valid inference percentages of 60, 75, and lOO%, but the percentage of one of the four remained at zero throughout the task period. Finally, we can examine the metastrategic-strategic relationship in the other direction, with comparable results. Of the eight strategically successful children (Table 3), the majority show metastrategic competence at least at what appears to be a critical level of achieving an analytic focus (level 2). All but one achieves this level with respect to task objective, and all but one achieves it with respect to strategy. All of the eight achieve it with respect to at least one of the two.
DISCUSSION In the domain in which we have investigated it-inductive causal reasoning in a multivariable context-our results support the thesis that metastrategic knowledge is both assessable as an entity distinct from strategic performance and related to it. Although the method of assessment of metastrategic knowledge that we employed is dependent on verbal facility and therefore vulnerable to underestimation (participants may understand more than they are able or willing to verbalize), our assessment nonetheless indicated a coherent pattern of connections between strategic and metastrategic functioning. Although initial competence was minimal in both the strategic and metastrategic realms for almost all participants, the microgenetic method allowed observation of their joint evolution. Individual case studies of each participant’s pattern of change across the different indicators showed that some did exhibit what appeared to be linked gains in the two realms (strategic and metastrategic) across time. Others, however, showed gains in one but not the other. Theoretically, there is no reason to predict a tight trial-by-trial linkage of functioning in the two realms. Alternatively, a model that the data support is a necessary-but-not-sufficient relation, with both of the two metastrategic components necessary (but not sufficient) for mastery in strategic performance. Other factors, including simply time to consolidate the gain in metastrategic understanding, presumably play a role in exactly when and in what form this metastrategic gain is realized in strategic performance. The finding of changes in the strategic and metastrategic realms over the same time period, however, does not establish direction of causality, and other possibilities (than a necessary-but-not-sufficient “gatekeeper” relation of one to the other) should be considered. Perhaps strategic success leads to greater metastrategic awareness, rather than the reverse. Or perhaps it is strategic failure that leads to a metastrategic search for new approaches. And even if the direction of causality is from metastrategic to strategic, multiple possibilities exist. Failure to attain what the individual construes as the task objective is a possible impetus for strategic variation and hence improvement, but so might newly achieved metastrategic insight serve as the impetus for strategic change.
244
Kuhn and Pearsall
Rather than one of these possibilities being correct to the exclusion of the others, more likely, in our view, is the existence of multiple directions of influence between the strategic and metastrategic-the kind of bootstrapping feedback relation that Case (1998) and others have proposed in contemplating the relations between more general and more specific cognitive structures. In other words, advances at the general level feed down to support the more local structures, while at the same time gains in the local structures feed up to strengthen and consolidate the general ones (what Case calls a “hierarchical learning loop”). In the present case, the likelihood of such a bidirectional model is enhanced by the fact that initial competence was minimal in both strategic and metastrategic realms. Metastrategic understanding was able to emerge only in the context of strategic engagement with the task. Yet, as it developed in the course of this strategic engagement, it set limits on strategic progress (but still, as we saw, was not sufficient to guarantee strategic success). Also relevant in the present case, and implicative of the role of metastrategic competence, is the fact that most children continued to use a combination of less successful strategies-those in the process of being discarded-as well as more successful ones in the process of emerging. Use of successful strategies may “feed up” to the metastrategic level, strengthening awareness of the strategy and understanding of its value. But the use of unsuccessful strategies may also feed up, strengthening awareness of the strategy’s ineffectiveness and leading to recognition of the need for better strategies. Similarly, both components of metastrategic knowledge may feed down to the strategic level. Gains in understanding of the task objective feed down to guide the application of strategies. Awareness and understanding of available strategies likewise supports their effective implementation. In contrast, the paths of influence between the two components of metastrategic knowledge are not as apparent. Knowledge of a task objective cannot supply strategies to achieve it when they are absent. Nor can availability and awareness of strategies provide a task goal. We propose, then, that coordination of these two components of metastrategic knowledge may be a central task of the metastrategic knowledge system, and we would target how this coordination is achieved as a worthy topic of further investigation. In the case of the present task, recall, at the lower levels of metastrategic understanding, strategic understanding often exceeded task understanding (when children’s attention turned to outcomes or to features before they verbalized these as relevant to task objectives); at the higher levels, however, it was most often lack of awareness of a strategy to implement task objectives that impeded progress. The present work examines metastrategic and strategic functions within the context of a certain kind of cognitive task. How widely do these findings apply? Inclusion of two kinds of task content (physical and social) in our design allows us to conclude that that they apply across a range of content within this task domain. Do they apply as well, however, to different kinds of tasks?
Metastrategic
Knowledge
and Strategic
Performance
245
Siegler (1996) has questioned the extent to which metacognitive or metastrategic factors are influential in task performance, turning instead to associative strength as a more influential factor in strategy choice and hence developmental change in strategic performance. It is worthwhile, therefore, to compare the kinds of cognitive tasks and strategies he and his colleagues have studied with those we have examined here, to see if the differing content might account for the differing importance attributed to metacognitive factors. We believe that it does. Recent reviews of microgenetic methodology in the study of cognitive development (Kuhn, 1995; Siegler, 1996) have emphasized the commonality of findings across very different kinds of cognitive tasks. As numerous investigators have now reported, the same child is likely to display multiple strategies, even when the task remains constant, and development entails gradual shifts in the distribution of their usage. But these striking similarities in findings across disparate tasks may have led to an overlooking of important differences, ones that become important when we begin to look beyond these now well-replicated findings to consider more subtle process questions, such as the interaction of strategic and metastrategic factors. Siegler and colleagues have focused their studies on children’s mastery of the basic arithmetic “number facts” of single-digit addition and subtraction. One might anticipate a central role for associative learning mechanisms in the mastery of number facts, since associative learning is the ultimate task goal. By middle childhood, we want and expect children to have learned these facts as associative links in their knowledge base of mathematics, such that the problem statement 7+8 produces the response 15 rapidly and automatically, without the need for intervening mental operations. The “back-up” strategies that Siegler and his colleagues have studied, involving procedures such as counting on fingers or the “min” strategy (counting up from the larger addend), are temporary aids that are expected to drop out once the necessary associative learning has been achieved. The desired performance end state here does not involve strategy use except in the limited sense that we might call retrieval (i.e., asking oneself, “Do I know an answer to 7+8?“) a strategy. In this domain, then, associative strengths between the various number fact problems and corresponding responses we would expect to be the most direct and strongest predictor of performance, certainly stronger than less direct measures that have been investigated, such as the individual’s judgments of problem difficulty (regarded as a metastrategic indicator). In the case of the competencies we have examined, in contrast, the ultimate developmental goal is replacement of primitive strategies not with associative learning but with more complex and demanding strategies, ones that continue to require exacting cognitive processing each time that they are applied. This is an important difference, for it says a great deal about the anticipated role of metastrategic factors. If more effortful and exacting strategies are to be applied in lieu of more primitive ones that continue to exist in the repertory, some explanatory governing mechanism needs to be invoked to account for such a choice. It is here that
246
Kuhn and Pearsall
we have invoked metastrategic understanding as playing an important role. Also likely to be influential is the other component of metastrategic understanding examined here-understanding of task objective-which, as we saw, is variable and undergoes change, again in contrast to what is the case in a domain like mastery of number facts. In fact, as we suggested at the outset, inductive reasoning is a domain in which metastrategic understanding is of significance in its own right, rather than attaining its significance only from an alleged effect on strategic performance. We want to know whether an individual is consciously aware of and understands the investigative strategies and associated inference rules that allow certain conclusions to be drawn and render others invalid and whether the individual understands the connection between these strategies and rules and the task objective of identifying the causal structure of the domain. Equally important, does the individual recognize invalid strategies and why they fail to achieve the task objective? This understanding is an intrinsic aspect of the competence being investigated and of importance in its own right. We need to investigate the connections between this understanding and strategic performance, but one does not warrant conceptual priority over the other. Strategic performance is more visible, but it is metastrategic understanding that is likely to dictate whether and when strategies are applied. At least within the range of cognitive tasks that entail complex, effortful cognitive processing across all degrees of mastery, we advocate as most promising a model of multiple, bidirectional paths of influence between performance and understanding.
REFERENCES Brown, A. (1975). The development of memory: Knowing, knowing about knowing, and knowing how to know. In H. Reese (Ed.), Advances in child development and behavior (Vol. 10) (pp. 103-152) New York: Academic Press. Brown, A. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.), Advances in instructionalpsychology (Vol. 1) (pp. 77-165) Hillsdale, NJ: Erlbaum. Brown, A. (1997). Transforming schools into communities of thinking and learning about serious matters. American Psychologist, 52 , 399-413. Case, R. (1998). The development of conceptual structures. In W. Damon (Series Ed.), D. Kuhn & R. Siegler (Vol. Eds.), Handbook of child psychology: Vol. 2. Cognition, language, and perception (5th ed.) (pp. 745-800) New York: Wiley. Gardner, H. (1991). The unschooled mind: How children think and how schools should teach. New York: Basic Books. Kuhn, D. (1982). On the dual executive and its significance in the development of developmental psychology. In D. Kuhn & J. Meacham (Eds.), On the development of developmental psychology. Contributions to human development (Vol. 8) (pp. 81-110) Basel: Karger.
Metastrategic Knowledge and Strategic Performance
Kuhn, D. (1995). Microgenetic
247
study of change: What has it told us? Psychological Science, 6, 133-139. Kuhn, D. (in press). Metacognitive development. In C. Tamis-LeMonda (Ed.), Child psychology: A handbook of contemporary issues. New York: Garland. Kuhn, D., Amsel, E., & O’Loughlin, M. (1988). The development of scientific thinking skills. Orlando FL: Academic Press. Kuhn, D., Garcia-Mila, M.. Zohar, A., & Andersen, C. (1995). Strategies of knowledge acquisition. Societyfor Research in Child Development Monographs, 60, (4), (Serial no. 245). Kuhn, D., & Phelps, E. (1982). The development of problem-solving strategies. In H. Reese (Ed.), Advances in child development and behaviol; Vol. 17. (pp. l-44) New York: Academic Press. Kuhn, D., Schauble, L., & Garcia-Mila, M. (1992). Cross-domain development of scientific reasoning. Cognition and Instruction, 9, 285-327. Moshman, D. (1998). Cognitive development beyond childhood. In W. Damon (Series Ed.), D. Kuhn & R. Siegler (Vol. Eds.), Handbook ofchildpsychology: Vol. 2. Cognition, language, andperception (5th ed.) (pp. 947-978) New York: Wiley. Perkins, D. (1992). Smart schools: Better thinking and learning for every child. New York: Free Press. Schauble, L., Klopfer, L., & Raghavan, K. (1991). Students’ transition from an engineering model to a science model of experimentation. Journal ofResearch in Science Teaching, 28, 859-882. Schneider, W., & Bjorklund, D. (1998). Memory. In W. Damon (Series Ed.), D. Kuhn & R. Siegler (Vol. Eds.), Handbook of child psychology: Vol. 2. Cognition, language, and perception (5th ed.) (pp. 467-521) New York: Wiley. Siegler, R. (1996). Emerging minds: The process of change in children’s thinking. New York: Oxford University Press. Sophian, C. (1997). Beyond competence: The significance of performance for conceptual development. Cognitive Development, 12, 281-303.