Exploring the relationship between process and outcome in young children’s learning

International Journal of Educational Research 29 (1998) 51 — 67

Chapter 4

Exploring the relationship between process and outcome in young children’s learning: stage one of a longitudinal study Christine Pascal*, Tony Bertram, Claire Mould, Richard Hall University College Worcester, Henwick Grove, Worcester WR2 6AJ, UK

Abstract The first phase of a small-scale, longitudinal study which is part of the Effective Early Learning (EEL) Project, a UK based, national, early childhood evaluation and improvement initiative is described. A key developmental proposition is that children who operate in a rich and stimulating learning environment and experience high levels of involvement and engagement in their learning will achieve enhanced learning outcomes. The EEL Project aims to investigate the validity of this proposition. In particular, it explores the relationship between one of the key process measures in the EEL Project, the Child Involvement Scale (CIS), and the outcome measures being used by early childhood settings to monitor academic progress in the UK, namely Baseline Assessment in English and Mathematics (BAEM) at 4 years of age, and Standard Assessment Tasks (SAT) at 7 years of age. ( 1998 Published by Elsevier Science Ltd. All rights reserved.

1. Introduction Children’s early educational experiences have a long-term and significant impact on their subsequent achievement and social behaviour (Leseman et al., 1992; Sylva & Wiltshire, 1993; Ball, 1994; Andersson, 1994; Plaisance, 1994; Brown, 1994; Bruner, 1996). There are also important economic and social issues which are increasing the pressure to expand early childhood services in order to develop human resources (Moss, 1994; Moss & Penn, 1996). Globalization, the increasing need for accountability, and other political pressures have meant that policy makers across the world have begun to accept the evidence for investment in early education as a critical element in the pursuit of economic and social development (Dahlberg & As se´n, 1994; Young, *Corresponding author 0278-4343/98/$19.00 ( 1998 Published by Elsevier Science Ltd. All rights reserved. PII: S 02 7 8-4 3 43 ( 9 8) 0 0 01 3 - 5

52

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

1995). In addition, there is a growing body of evidence which shows that it is important for early educational provision to be of quality, as not all forms of early educational provision will necessarily bring the benefits desired. In fact, if children are placed in settings which do not meet their developmental needs appropriately, their early educational experiences could actually be detrimental to subsequent progress (Schweinhart, Weikart & Larner, 1986; Schweinhart & Weikart, 1993). Thus, both the expansion and improvement of early childhood education are high on the political agenda internationally.

2. The Effective Early Learning Project The Effective Early Learning (EEL) Project was initiated in the UK in 1993 (Pascal, Bertram, Ramsden, Georgeson, Saunders & Mould, 1996; Pascal & Bertram, 1997). The EEL Project is a national research and development initiative which aims to improve the quality of early learning in a wide range of education and care settings. It grew out of an increasingly articulated need for procedures to evaluate and improve young children’s early educational experiences in the diverse mix of settings which make up pre-school provision in the UK (Ball, 1994). Policy here, as in other countries, has in recent years moved towards an expansion of pre-school provision and a lowering of the age of admission to school. This expansion has been coupled with a drive for quality in those settings which admit young children for educational purposes, many of which are not providing adequate or appropriate early learning experiences (DES, 1990; Pascal, 1990; National Commission, 1993; Ball, 1994; Moss, 1994; Moss & Penn, 1996, Audit Commission, 1996). The EEL Project provides a targeted strategy for improvement in early education. It attempts to build upon the existing skills and expertise of those who work with young children in a range of education and care settings, including the public, private and voluntary sectors (Pascal & Bertram, 1997a). Its approach is inclusionary (Moss & Pence, 1994), democratic (Pfeffer & Coote, 1991, Pascal & Bertram, 1997a) and developmental. It aims to empower and strengthen practitioners and providers through an improvement process (Freire, 1985; Dahlberg & As se´n, 1994, Pascal & Bertram, 1997a), rather than to threaten or judge them. A conceptual framework has been developed for evaluating and improving the effectiveness of early learning in early childhood settings (Bertram, 1996; Pascal et al., 1996). This framework, laid out in Fig. 1, has three major dimensions: Context, Process, and Outcome. Context focuses on those aspects of a setting which define the environment in which early learning takes place. This includes such aspects as curriculum, learning and teaching strategies, staffing and the physical environment. Process focuses on the educative interactions which occur between adults and pupils within a setting. In particular the levels of children’s involvement (Laevers, 1994) in the learning process and the levels of adult engagement (Bertram, 1997) with children are scrutinized. Outcome focuses on the products and results of learning. These include academic achievements, emotional well-being, respect for self and others, and educative dispositions. The EEL Project aims to improve the first two dimensions of effectiveness in order to enhance the third.

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

53

Fig. 1.

2.1. The relationship between process and outcomes in children+s learning The EEL Project chose to focus on assessing learning processes rather than learning outcomes because it was believed that outcome measures had a number of problems. Examples of these problems are as follows: f long-term, dormant effects are often not captured by immediate measures; f current outcome instruments are insensitive, and questions have been raised over their validity and reliability; f the social and affective context affect the results of outcome testing; f dispositions to learn may be as important as knowledge and skills, but are not being assessed;

54

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

f f f

there maybe a cultural bias in outcome measurements; outcome measurement is too late to effect change; the idiosyncratic nature of young children’s conceptualization and expression impedes accurate interpretation of outcome results; and, f outcome measures are often of a disembedded nature and not meaningful for young children. Despite the inadequacy of many outcome measures in early childhood, outcomes are very much part of the current agenda in educational policy making and practice. Providers and practitioners have to be confident that the early learning experiences being offered are productive and effective, leading to real and identifiable long-term educational progress. In an attempt to improve the quality of early learning, the challenge was to find ways to improve those aspects of educational settings which were capable of being rigorously and systematically assessed and altered. Practitioners tended to give greatest credence to context assessment; their concerns were often focused, for example, on resourcing and staffing issues. On the other hand, policy makers tended to be overly focused on outcome measures. Consequently, the EEL Project focused on ways of systematically improving the learning processes. The strength of this strategy, however, rests on the developmental proposition that children who operate in a rich and stimulating learning environment, and experience high levels of involvement and engagement in their learning will achieve enhanced learning outcomes. Although there is evidence to support this proposition from other studies (Skinner and Belmont, 1993), it requires further scrutiny in order to strengthen the case being made for the EEL improvement strategy. A small-scale study was initiated within the larger EEL Project. This paper describes the first phase of this small scale, longitudinal study which attempts to demonstrate the validity of our developmental proposition. In particular, the study explores the relationship between one of the key process measures used in the EEL Project, the Child Involvement Scale (CIS), and the outcome measures being used by early childhood settings to monitor children’s academic progress in the UK (e.g., Baseline Assessments in English and Mathematics (BAEM) at four years of age, and Standard Assessment Tasks (SAT) in English and Mathematics at seven years of age).

2.2. Aims of the study This small scale, longitudinal study aims to: 1. document and assess the quality of the children’s early learning processes, using the Leuven Involvement Scale, at the ages of four years and seven years; 2. document and assess the outcomes of early learning as displayed in the children’s level of academic attainment on a Baseline Assessment Scheme, at the ages of four years; and, seven years and, 3. explore the relationship between the involvement levels of the children and their attainment levels in English and Mathematics as they progress through the school system.

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

55

3. Methodology The sample included Early Years settings selected from a large authority in the West Midlands, UK. The sample was selected from those settings which had taken part in the authority’s Baseline Assessment Scheme 1994/95 (Birmingham LEA, 1997), and either the EEL Research Project 1994/5 (Pascal et al., 1996), or Mould’s Project 1994/5 (Mould, 1997). These latter two projects had systematically measured the levels of involvement of a cohort of the authority’s children at the age of four years. This cohort provided a final sample of 118 children from ten settings; four infant schools, three junior and infant schools, and three nurseries. All the professionals who worked in these settings were women. While the study does not claim to be based on a scientifically representative sample of children and settings, it did cover all geographic regions of this large authority. Furthermore, the sample of children characterized its cultural and socio-economic diversity. 3.1. Instruments The EEL Longitudinal Study focuses on two assessment instruments. The Child Involvement Scale focuses on learning processes while the Baseline Assessment Scheme assesses learning outcomes. 3.1.1. The Child Involvement Scale (CIS) Throughout the study Laevers’ (1993) definition of involvement has been respected. In his words, involvement is A quality of human activity, characterised by concentration and persistence, a high level of motivation, intense perceptions and experience of meaning, a strong flow of energy, a high degree of satisfaction and based on the exploratory drive and basic developmental schemes (p. 61).

Involvement, then, is a measure of quality, observable in a variety of situations at all ages. An involved child narrows his or her attention to a specific focus and is rarely, if ever, distracted. Involvement does not occur when the activities are too easy or when the task is too demanding (Pascal et al., 1996). An involved child will be situated at the edge of his or her capabilities; they will be at the edge of their zone of proximal development (Vygotsky, 1978). Support for the view that an involved child gains a deep, motivating, long-term educative experience which is likely to give rise to long-term learning comes from a number of studies (Csikszentmihayli, 1979; Holt, 1994; Laevers, 1994; Skinner and Belmont, 1993). Laevers (1994) argues that the level of involvement that a child displays is a key indicator of the quality and effectiveness of the child’s learning experience. Involvement levels are inferred from the presence or absence of ‘‘Involvement signals’’, which are physical signs that can be observed in the child. These include: f concentration; f energy; f creativity;

56 f f f f f f

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

facial expression and posture; persistence; precision; reaction time; language; and, satisfaction. The concept of involvement has become a key aspect of the research methodology employed by both the EXE Project (Laevers, 1993, 1994) and the EEL Project (Pascal et al., 1996). The CIS is the basis for an attractive observation instrument because it attempts to measure the process of learning rather than concentrating upon the outcomes. The CIS is grounded in common sense and is highly accessible to practitioners; it also has a strong theoretical underpinning (Pascal et al., 1995, 1996; Vejleskov, 1995; Bro¨strom, HanniKainen, Berg-de-Jong, Rubenstein Reich & Thyssen, 1995). The level of children’s involvement is graded on a scale of one to five; level one being given when a child displays ‘No involvement’ and level five when a child displays ‘intense involvement’. Six observations of involvement, each of two minutes duration, were made for each of the 118 study children. These were completed on different days at varying times (1,416 minutes of observations in total). The mean level of involvement was computed within each setting by averaging the scores recorded in that setting. However, the issues of observer consistency over time, and inter-observer agreement were addressed. A single observer can be idiosyncratic in his or her observations, but may be consistent over time. To address the issue of consistency, both over time and between observers, checks were built into the process. The observers were trained in various observation and evaluation techniques over a period of three days. On the last day of the training session they were given a blind test. This was made up of five video cameos each of two minutes duration. The set of video clips contained a randomly sequenced mix of the levels of child involvement and the observers were required to rate them independently with no conferring. The video cameos were previously rated by the core EEL team and the degree of agreement was determined. More than 90% of all the observations for any particular cameo were covered by two levels on the 5-point scale and were within one level of the rating given to each by the core team. For example, if the core team rated an interaction at level three, more than 90% of the practitioners would rate it either at levels three or four or at levels two or three. It is important to realise that these ratings were video clips of real situations. Adults in situ have many more contextual clues to draw on than is possible through a video, so their judgments would likely be more accurate, especially on the Leuven Involvement Scale. In addition to this training test the observers were also involved in moderating network meetings, which took place twice a term. These meetings involved an EEL Project external adviser in monitoring the observations within the study settings. This helped minimize ‘‘observer drift’’. The main EEL Project also carried out a check on 25% of the study settings involved in Phase 2 of the larger project, in which some of these settings were involved. A member of the EEL team revisited a quarter of the participating settings to record

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

57

Table 1 Comparison of observer rated and EEL research team rated mean involvement levels in EEL Project study settings Statistic

Study practitioners

EEL Project research team

Mean Standard Deviation Kurtois Skewness

3.348 1.027 !0.540 !0.198

3.293 1.087 !0.782 !0.057

Number of study settings visited"23. Number of valid observations by study practitioners"1,447. Number of valid observations by EEL Project Research Team"1,104.

his or her own involvement ratings to see how they equated with the aggregate scores recorded by the setting practitioners themselves. Each of the settings was visited by two members of the EEL team on two separate occasions to achieve approximately the same number of timed observations as had been completed by the observers. These visits took place during the same term that the initial assessment had taken place. The comparative figures for these mean scores, as shown in Table 1, show that agreement between the observers and the EEL monitoring team was very high. Other statistical confirmation of the scales comes from Laevers (1993) whose LIS-YC (Involvement) Scale is the root of the scale in the present study (Maes & Nijsmans 1988). The Child Involvement Scale has its origins in studies which had tested for inter-observer reliability (Spearman rho"0.9) in clinical and real classroom situations (Maes & Nijsmans, 1988; Laevers, 1994). Further, in its original form, it has been successfully used by adults other than professional researchers. Two studies are important in considering the use of this scale for research and development. First, Laevers (1994) was able to show that the measured quality of learning remained fairly stable within a setting over time in the absence of intervention. This demonstrates that the concept of involvement is a substantive, concrete characteristic of a setting which may be subject to assessment. Second, Theunissen (1992) found that measurable, positive change occurred in a setting when staff were trained in using the observation scale. This study confirms that the Leuven Involvement Scale provides a credible and effective means of assessing children’s involvement. The CIS is a frequency scale which looks at sequential occurrences of observed behaviour during a defined time. The scale is not based on consecutive timed frequencies but on timed interval frequency observations spread out over the course of a day and over a number of sessions. As such, it could claim to be more representative of the reality of the setting and an amalgamation of scores in a setting can be used to indicate a setting’s overall rating on child involvement. The CIS is conceived as being an ‘equal appearing interval scale’ (Thurstone and Chave 1929). The scale has been used with large numbers of early childhood educators who have made independent judgments in rating video cameos of interactions. Thurstone and Chave (1929) suggest that between 50 and 100 ‘judges’ are needed to

58

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

establish an interval scale based on their methodology of rating attitudes. The EEL Project has thus far involved more than 3,000 adults working with young children. Analysis of their observational responses have confirmed the reliability of the five levels of the CIS. 3.2. Baseline assessment scheme Early childhood practitioners now operate within a professional context in which measurement, ‘baseline assessment’, and ‘value added’ strategies are dominant features (Wolfendale, 1993; Fisher, 1995; Tymms, 1996). ‘Value added’ is the term used for the concept of the educative gains made between one test and a subsequent one. Early childhood professionals have to respond proactively to the demands placed upon them for quantitative evidence and to use this opportunity to promote what they believe ‘counts’ in early learning. National assessment requirements have been shown to shape and direct educational practice (Blenkin, 1995) and as these requirements are now being developed for younger children (DfEE, 1997) it is important that early childhood educators are informed by the knowledge base about what constitutes effective and long-term early learning. The development of measurement procedures in the key domains of young children’s learning is currently a priority for practitioners and providers. The recent requirements for local authorities in the UK to submit schemes for Baseline Assessment (DfEE, 1997) for approval have added impetus to this work. While we believe it is imperative that such schemes reflect the broader development of the child and include the assessment of attitudes, inclinations, and dispositions, most schemes at present focus on the more academic areas of development, such as language, literacy, and mathematics (Tymms, 1996). Therefore, for the purposes of this study, the existing Baseline Assessments, with their emphasis on English and Mathematics, have been accepted. If Baseline Assessment is to be universal, it is argued that some items within this domain should capture broader curriculum objectives and include areas of learning other than literacy and numeracy. For example, Gardner’s (1989) ideas on ‘multiple intelligence’ conflict with notions of a universal ‘core curriculum’ and suggest that other areas of learning may be as important. The evidence of Tymms (1996) suggests that very few providers who are currently developing assessment schemes for young children are focusing on these other areas of learning. Indeed, even the ‘personal and social development’ of young children is neglected. The cluster of attributes which are associated with a broad range of academic learning need to be identified and explored in more depth (Pascal & Bertram, 1997b). Despite these concerns, the current Baseline Assessments (1994—95) used within the study authority (Birmingham LEA, 1997) provides the only opportunity to inaugurate a longitudinal study and investigate the concept of value added assessment. The children’s levels of English and Mathematics were individually assessed on a fourpoint scale which ranged from no observable evidence (1) to an above average competence (4). This was done during the first half of the Autumn Term 1994—95 by the Early Years’ professionals working in each of the ten study settings. All observers had been trained in the use of this scale.

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

59

Neither the Child Involvement Scale nor the Baseline Assessments are psychometric instruments of children’s ability. Rather, the intention of both is to provide rigorous and systematic evidence of learning processes and outcomes for practitioners, policymakers and researchers as they occur in the ‘real world’ (Robson, 1993) of school practice. 3.3. Procedures Robson (1993) suggests that when assessment systems are used by practitioners they should be straightforward and reliable. He recommends seven criteria to which coding should conform. It should: 1. be focused; 2. be as objective as possible; 3. be non-context dependent; 4. contain explicitly defined categories; 5. contain exhaustive categories; 6. contain mutually exclusive categories, and 7. be easy to record. The Child Involvement Scale and Baseline Assessments were both focused scales looking at carefully selected aspects of the learning process and outcomes. They were based on defined levels of children’s observed behaviour that could be applied at any time. Specific categories and levels, coupled with a detailed methodology, were given to the observers during the training programs. In addition, the video cameos of the children’s levels of involvement gave explicit examples of appropriate scoring and the EEL Project Manual (Pascal et al. 1996) was a further written source of information for those conducting the observations. The proformas for recording the data were tested and altered in the light of feedback. Finally, each observation lasted two minutes so it could take place in the normal practice of the practitioner-observer. Both scales complied with Robson’s (1993) criteria. In five of the settings, data on the children’s involvement levels were gathered during the Autumn Term 1994—95 by Early Years’ professionals who had participated in a three day EEL Project training program. The use of the CIS was then externally validated by two core members of the EEL Project research team in three of these settings. In the other five settings the involvement data were gathered by a core EEL Project researcher; the observation scores were then discussed with the professionals in situ. The Baseline Assessment data were collected by the Early Years’ professionals during the Autumn Term 1994-95, collated by the Local Authority Assessment Unit, and accessed by a core researcher on the EEL project for use in this study. 3.4. Data analysis The data gathered were analysed utilizing computer software which supported a more complex and comprehensive analysis than previously possible (Dey, 1995). Dey (1995, p. 68) reinforces how the advent of fast, efficient, and eminently manageable techniques for handling data facilitates the achievement of traditional ideals. The

60

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

statistics package FASTAT (Bjerknes, 1988) is an interactive statistics and graphics package for Apple Macintosh computers which allows for a varied statistical representation. FASTAT was utilized to record each observation onto an in-depth statistical database. The subsequent statistical correlations established a wealth of evidence. The NUD*IST (Non-numerical Unstructured Data Indexing Searching and Theorizing) (Richards & Richards, 1993) package is designed to support researchers in the rigorous analysis of rich and complex qualitative research information. This software is particularly advantageous with regard to coding and theory building, alongside its strong search, retrieval, and matrix building characteristics. The framework of this software package provided rigour and structure to the process of analysis, combined with a flexibility that allowed the analysis to fit the aims of the study and not vice versa. The qualitative data additionally collected from each setting substantiated the nature of the quantitative evidence. This qualitative evidence consisted of interviews with children, governors, managers, parents and staff; professional biographies of the staff; context proformas; and physical environment schedules. These data allowed the exploration of the importance of causal networks.

4. Preliminary results The initial analysis of the individual children’s data enabled an exploration of simple correlations between their level of involvement and their English and Mathematics scores. The mean was calculated for each variable and this was subtracted from the individual child’s average score for each of the three variables. The result gave either a positive or negative score for each child on each variable and allowed a comparison of settings (see Table 2). It became clear that for eight of the settings there was a linear relationship between mean involvement and mean English and Mathematics scores. Thus, all three variable values were either positive or negative. Further investigation using the child as the

Table 2 Mean involvement, English and Mathematics scores for each setting Setting

Mean involvement

Mean English

Mean Mathematics

Sch1 Sch2 Sch3 Sch4 Sch5 Sch6 Sch7 Sch8 Sch9 Sch10

0.997 0.037 !0.262 !0.003 0.849 0.499 !0.003 0.714 !0.719 !0.707

0.506 0.172 !0.342 !0.143 0.266 0.117 !0.254 0.178 !0.357 0.080

!0.488 0.356 !0.544 !0.087 0.267 !0.053 !0.128 0.253 !0.088 !0.350

61

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

unit of analysis showed that there was the same linear relationship between the levels of involvement, English and mathematics for fifty-nine of the 118 children. In other words, in these fifty-nine cases children who had positive involvement scores also had positive English and mathematics scores. Similarly children with negative involvement scores also had negative English and mathematics scores. Of these children, thirty-five (twenty-one girls and fourteen boys) had completely positive scores, while twenty-four children (eleven girls and thirteen boys) were shown to have completely negative scores. Thus, the scores for these children were polarized at the positive or negative end of the range of scores. Since in only 50% of the total cases was there a linear relationship between the process and outcome measures, the evidence does not demonstrate a substantial relationship between the process measures of learning quality used in the EEL Project study and the outcome measures of learning quality used in the study schools. This replicates the findings of Schweinhart et al. (1986) that there was a latent period before differences in achievement became apparent. However, further analysis of the data reveals some interesting features within this initial evidence. We have already indicated slight gender differences in the thirty-five positive and twenty-four negative relationships. It was also notable that the date of birth of the child appeared to be a factor in the fifty-nine correlating cases. (See Table 3.) Of the thirty-five positive cases, thirteen were five years old in the Autumn Term, eleven were five in the Spring Term, and eleven had their fifth birthday in the Summer Term. In contrast, only one child from the negative case group had their fifth birthday in the Autumn Term, while five were five in the Spring Term, and eighteen in the Summer Term. Thus, the younger four year olds predominated in this latter group. Further investigation highlighted the fact that there was a slightly lower polarization between the children’s level of involvement and mathematics scores than existed between the children’s involvement and English scores. Hence, fewer children had a wholly positive or negative relationship between involvement and mathematics than did so for involvement and English. In summary, the correlation between the children’s levels of involvement and their English and mathematics scores was 0.33 for involvement-English and 0.40 for involvement-mathematics. While both correlations are statistically significant (p(0.001), their magnitude is rather low. Only sixteen percent or less of the variation in test scores can be attributed to differences in involvement. Thus, it is too early to make

Table 3 Comparison of polarised scores and term of birth Scores

Autumn

Spring

Summer

Frequency

Positive Boys Positive Girls Negative Boys Negative Girls

6 7 0 1

4 7 3 2

4 7 10 8

14 21 11 13

62

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

any definitive statements about the relationship between the process and outcome measurements used. In order to explore this relatively weak relationship, the individual children and settings were investigated in more depth. To facilitate this exploration the qualitative data which had been collected for each setting were utilized. These data focused on a number of aspects of the children’s personal circumstances, They included: f socio-economic status; f one parent families; f special educational needs; f English as a second language; and, f ethnicity. Other data collected on aspects of the child’s educative circumstances, included: f date of birth; f gender; f type of setting attended; f adult—child ratios.

5. Children’s personal circumstances The 118 children came from a diverse range of socio-economic groups. For example, data were available on the proportion of the children’s parents in each setting who were unemployed, as well as those whose employment was manual, semi-skilled, skilled, or professional. The range across the settings was from 90% unemployment in one to over 90% professional in another. Similar differences were found in other personal circumstances. For example, the percent of students in ranged one-parent families ranged from 70% in one setting to 2% in another. The proportion of the children with special educational needs ranged from 36% in one setting to 1% in another. The proportion of the children who had English as a second language ranged from 85% in one setting to another where it was stated that all of the children spoke English as their first language. Finally, the percentage of the children who were from ethnic minority families ranged across the settings from 85% to 1%. The cross-section of data on children’s personal circumstances illustrated a wide range of circumstances across the ten settings. However, they also highlight the danger of making assumptions about the personal circumstances of the children and the quality of learning within a particular setting. For example, one of the settings which had negative mean levels of involvement, English and mathematics (Sch4 in Table 2) was attended by children from predominantly professional, two-parent families who spoke English as a first language. Of the other settings with wholly negative scores, one had approximately 10% of children from one-parent families, with 10% from ethnic minorities, and 10% having special educational needs. The other two such settings were located in inner city, multicultural areas; one had high levels of unemployment (around 40%) and one parent families (approximately 40%), with

63

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

two-thirds of children from an ethnic minority; the other had a large majority of children from an ethnic minority background (around 85%), whose first language was not English. By contrast, one of the settings with totally positive mean scores (Sch5 in Table 2) had high ethnicity (70%) and a relatively large percentage of children for whom English was a second language (20%).

6. Children’s educative circumstances As mentioned earlier, the sample of ten settings consisted of four infant schools, three junior and infant schools, and three nurseries. Preliminary analysis indicated that there are links between the type of setting that the children attended and their involvement, English and mathematics scores. Relative to those children in infant and infant and junior schools the 32 children who attended the nursery settings were shown to have better scores on all three variables (see Table 4). Table 5 illustrates that children from the nurseries were relatively older than those from the other two types of setting. This may account for the higher proportion of children who scored positively on all three variables. The data suggest that the child’s date of birth, the type of setting, and, as mentioned earlier, gender were significant factors in the child’s scores on each variable. These factors, then, should be taken into consideration when examining the involvementachievement relationship in individual students.

7. Emerging issues At this first stage in the EEL longitudinal study, no substantial relationship between the learning process and learning outcome measures was found. However, Table 4 Relationship between type of setting and children’s scores Setting

Children

All Positive

All Negative

Mixed

Infant Junior infant Nursery

58 28 32

13 4 18

19 5 0

26 19 14

Table 5 Relationship between term of birth of children and type of setting Scores

Autumn

Spring

Summer

Total

Infant Junior infant Nursery

13 0 15

15 14 14

30 14 3

58 28 32

64

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

these initial results have pointed to the significance of a number of factors which appear to influence a child’s scores on both measures, and which need to be taken into account in any further analysis and reflection. These factors include: 1. The child’s date of birth (the younger the child when assessed, the lower he or she tended to score on both measures); 2. The child’s gender (boys appear to score lower than girls on all measures); and 3. The type of setting (the children in nursery settings scored more highly than the children in infant or junior and infant schools). To complicate matters further these factors may themselves be interrelated. For example, older children were found in nursery settings. In summary, these preliminary data have suggested that attempting to explore the relationship between process and outcome measures in early learning requires a complex and multifaceted approach. Relying on simple quantitative measures of process and outcome alone does not provide sufficient evidence of the educative phenomenon being studied. This early evidence indicates that a more complete understanding of the complex nature of the learning process and the identification of how learning might be enhanced requires a consideration of a number of personal and contextual factors which surround and affect the child in their ‘real world’. These factors include the child’s age, gender, and type of setting attended. Each of these factors appears to exert an influence on the effectiveness of the learning for each individual child, as demonstrated in both the process measure of ‘involvement’ and the outcome measures of English and mathematics. It is therefore clear that this longitudinal study of process and outcome in early learning will need to broaden its focus to allow a consideration of these factors in both its methodology and its reflections. We are in the fortunate position of having access to such broad data through the methodology of the larger scale EEL Project in which all of the study settings are participating. This broader methodology will be followed through as the longitudinal study progresses.

8. The way forward This preliminary study has provided a rich and illuminative portrait of educative processes in action. It has also led to a reconsideration of focus and a broadening of the methodology and analysis to include contextual and personal data on the children and the school settings. The need for an extended time-scale in studies of this nature has also been validated. The 118 study children will be tracked until they are seven years of age and possibly beyond. Data will be gathered annually on the children’s involvement levels and their progress on national assessment schemes in both English and mathematics. Collecting data on the children’s emotional well-being and their attitudes to learning as they progress through the school system is also being considered. The skills of the educators with whom they are engaged throughout this period will also be monitored using the Adult Engagement Scale (Bertram, 1996; Pascal et al., 1996). Finally, the school context in which the children are operating each year will be described systematically.

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

65

This study is an example of a ‘real world’ developmental, professional inquiry. It is responsive to both the changing educational contexts in which the children are operating and the ongoing analysis and reflection of the data gathered at each stage. In this way, it is hoped that the study will be authentic, relevant, and credible, providing practitioners and policy makers with meaningful evidence to strengthen professional practice in early childhood settings.

References Andersson, B. E. (1994). Public policies and early childhood education. European Early Childhood Education Research Journal, 2, 19—32. Audit Commission (1996). Counting to five, Education of children under five, National Report. London: HMSO. Ball, Sir C. (1994). Start right:¹he importance of early learning, Report by Sir Christopher Ball. London: Royal Society for the Encouragement of Arts, Manufactures & Commerce, RSA. Bertram A. D. (1996). Effective educators of young children: Developing a methodology for improvement. Doctoral Thesis, Coventry University. Bertram A. D., Laevers F., & Pascal C. (1996). E¨tude de la qualite´ de l’interaction adulte-enfant dans le pre´scolaire:le sche´ma d’observation du style de l’adulte. In S. Rayna, F. Laevers, & M. Deleau (Eds.), ¸’Education prescolaire: quels objectifs pe& dagogiques Paris: Nathan. Birmingham Local Education Authority (1997). Baseline assessment scheme. Birmingham: LEA. FASTAT (1988). FAS¹A¹, Illinois: Systat Inc. Blenkin, G. (1995). Improving the quality of children’s learning, Childcare Now, 15, 14—15. Bro¨strom, S., Hannikainen, M., Berg-de-Jong, M., Rubenstein R. L., & Thyssen S. (1995). The child’s perspective — An extended view of what constitutes quality. In F. Laevers (Ed.), An exploration of the concept of involvement as an indicator of quality in early childhood education. Dundee: CIDREE/Scottish Consultative Council on the Curriculum. Brown, A. (1994). The advancement of learning. Educational Researcher, 23, 4—12. Bruner, J. S. (1996). What we have learned about early learning. European Early Childhood Education Research Journal, 4, 5—16. Csikszentmihayli, M (1992). Flow: ¹he psychology of happiness. London: Rider. Dahlberg, G., & As se´n, G. (1994). Evaluation and regulation: A question of empowerment. In P. Moss, & A. Pence, (Eds.), »aluing quality in early childhood services: new approaches to defining quality. London: Chapman and Hall. Department of Education and Science (1990). Starting with quality (¹he Rumbold Report of the Committee of Inquiry into the quality of the educational experience offered to 3- and 4-year-olds). London: HMSO. Department of Education and Employment (1997). Baseline Assessment Schemes: Submission Guidelines. London: DfEE. Dey, I. (1995). Qualitative data analysis: A user friendly guide for social scientists. London: Routledge. Fisher, D. (1995). Baseline Assessment. EMIE. Freire, P. (1985) ¹he politics of education: Culture, power and liberation. London: Berg and Garvey. Gardner, H. (1989). Frames of mind: ¹he theory of multiple intelligences. London: Fontana Press. Holt, J. (1994). How children learn and fail. In A. Pollard, & J. Bourne (Eds), ¹eaching and learning in the primary school. London: Routledge. Laevers, F. (1993). Deep level learning — An exemplary application on the area of physical knowledge. European Early Childhood Education Research Journal, 1, 53—68. Laevers, F. (1994). ¹he ¸euven Involvement Scale for ½oung Children ¸IS-½C, Manual and »ideo ¹ape, Experiential Education Series No. 1. Leuven, Belgium: Centre for Experiential Education. Laevers, F. (Ed.) (1996). An exploration of the concept of involvement as an indicator for quality in early childhood education. Dundee: Scottish Consultative Council on the Curriculum.

66

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

Leseman, P., Vergeer, M. M., Sijsling, F. F., Japa-Joe, S. R. and Sahin, S. (1992). Informal education in the home and the development of language and cognition. A one year follow up paper presented at the Second European Conference on the Quality of Early Childhood Education, Worcester, UK. Maes, K., & Nijsmans, I. (1988) Observatie in de contest von het ervaringsgericht kleuteronderwijs (EGKO): Ontwikkeling van een instrumentarium, Niet-gepubliceerde licentiaatsverhandeling. Leuven: KU Leuven. Moss P. (1994). The early childhood League in Europe: Problems and possibilities in cross-national comparisons of levels of provision, European Early Childhood Education Research Journal, 2, 19—32. Moss, P., & Pence, A., (Eds), (1994). »aluing quality in early childhood services: New approaches to defining quality. London: Chapman and Hall. Moss, P., & Penn, H. (1996). ¹ransforming nursery education. London: Chapman and Hall. Mould, C. (1997). ¹he influence of researcher-teacher collaboration on the effectiveness of the early learning of four year olds in schools. Doctoral Thesis, Coventry University. National Commission on Education (1993). ¸earning to succeed: A radical look at education today and a strategy for the future, Report of the Paul Hamlyn Foundation, National Commission on Education. London: Heinemann. Olsen, P., & Zigler, E. (1989). An assessment of the all day kindergarten movement, Early Childhood Research Quarterly, 4, 167—87. Pascal, C. (1990). ºnder fives in infant classrooms. Stoke-on-Trent: Trentham Press. Pascal, C., & Bertram, A. D. (1995) Involvement and the effective early learning project: A collaborative venture. In F. Laevers (Ed.), An exploration of the concept of involvement as an indicator of quality in early childhood education. Dundee: CIDREE/Scottish Consultative Council on the Curriculum. Pascal, C., & Bertram, A. D., Ramsden F., Georgeson J., Saunders M., & Mould C. (1996). Evaluating and developing quality in early childhood settings A professional development programme. Worcester: Amber Publications. Pascal, C., & Bertram, A. D. (1997a). Effective early learning: Case studies in improvement. London: Hodder and Stoughton. Pascal, C., & Bertram, A. D. (1997b). An emotional plea, ¹imes Educational Supplement, May 30, 1997. Pfeffer, N., & Coote, A. (1991). Is quality good For you? A critical review of quality assurance in welfare services. London: Institute for Public Policy Research. Plaisance, E. (1994). Pre-school education and institutions for small children in France: Biological debates and facts concerning child minding and educational institutions for children from 0—6 years of age. Paper presented at the 14th Conference of the Deutsche Gesselschaft fu¨r Erziehungwissenschaft, Dortmund University, Germany. Richards, L., & Richards, T. (1993) Hard results fron soft data: Computing and qualitative analysis. Melbourne: La Trobe University. Schweinhart, L. J., Weikart, D. P., & Larner, M. (1986). Consequences of three pre-school curriculum models through age 15, Early Childhood Research Quarterly, 1, 15—45. Schweinhart, L. J., & Weikart, D. P., (1993). A summary of significant benefits: ¹he High/Scope Perry Pre-school Study through age 27. Ypsilanti, Michigan: High/Scope Press. Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of teacher behaviour and student engagement across the school year, Journal of Educational Psychology, 85, 571—81. Sylva, K., & Wiltshire, J. (1993). The impact of early learning on children’s later development. European Early Childhood Education Research Journal, 1, 17—40. Theunissen, J. (1992). Het adoptie-en implementatie onderzoek: Evaringsgericht Onderwijs in de Onderbouw, S. Hertogenbosch: Katholiek Pedagogisch Centrum. Thurstone, L. L., & Chave, E. J. (1929). ¹he measurement of attitude. Chicago: University of Chicago Press. Tymms, P. (1996). Baseline assessment and »alue-added. A report to the school curriculum and assessment authority. Middlesex: SCAA. Vejleskov, H. (1995). A pilot study of an application of LIS-YC. In F. Laevers (Ed.), An exploration of the concept of involvement as an indicator of quality in early childhood education. Dundee: CIDREE/Scottish Consultative Council on the Curriculum. Vygotsky, L. S. (1978). Mind in society. Cambridge, Massachusetts: Harvard University Press.

C. Pascal et al./Int. J. Educ. Res. 29 (1998) 51—67

67

Wolfendale, S. (1993). Baseline assessment: A review of current practice, issues and strategies for effective implementation. London: OMEP. Young, M. E. (1995). Investing in young children. World Bank Discussion Paper 275. Washington, D.C.: The World Bank.

Professor Christine Pascal is Director of the Centre for Research in Early Childhood at University College of Worcester, Director of a national research and improvement initiative entitled ‘Effective Early Learning: An Action Plan for Change’. From 1991—94 she was National President of the British Association of Early Childhood Education and is now Vice-President. She taught in infant schools in Birmingham for 10 yrs prior to working in Higher Education. Dr Tony Bertram is Senior Research Fellow and Deputy Director of the Centre for Research in Early Childhood at University College of Worcester. He is also Director of a major national research project entitled ‘Effective Early Learning: An Action Plan for Change’ and President of the European Early Childhood Education Research Association, which he co-founded. He taught in first schools and primary schools for 13 yr, and was a headteacher for 7 yrs prior to working in Higher Education. His research interests remain very much with young children and those who work with them. Dr Claire Mould is currently employed as Research Fellow at the Centre for Research in Early Childhood at University College of Worcester. Having been an early years teacher, she successfully completed her doctoral studies in August 1997. Her doctoral thesis focused on the influence of researcher—teacher collaboration on the effective nature of the early learning experiences of four year olds in schools. Dr Richard Hall completed his History Doctorate in August 1997, and since then has worked as a Research Assistant at the Centre for Research in Early Childhood at University College of Worcester. His main responsibilities are for the technological development of the Centre and the management and analysis of the research project data.