- Email: [email protected]
Technology enriched school project—The impact of information technology on children's learning
Compurers Educ. Vol. 21. No. l/2, pp. 4149, 1993 Printed in Great Britain. All rightsreserved
Copyright
0360-1315/93 $6.00 + 0.00 % 1993 Pergamon Press Ltd
TECHNOLOGY ENRICHED SCHOOL PROJECT-THE IMPACT OF INFORMATION TECHNOLOGY ON CHILDREN’S LEARNING MARGARET J. Cox Centre
for Educational
Studies,
King’s College London, Cornwall London SE1 8TX, England
House
Annexe.
Waterloo
Road,
Abstract-The purpose of this paper is to consider what level of technology enrichment in schools is required to achieve an enhancement of pupils’ learning. This question relates to one aspect of the 3-yr ImpacT project[l] commissioned by the U.K. Department for Education to evaluate the impact of information technology on children’s achievements in English, mathematics, science and geography, for pupils aged 8-10, 12-14 and 14-16 years. The focus of this paper is on the level and use of Information Technology (IT) in 70 of the 87 ImpacT classes which returned data from both the class teachers and each of the individual 2300 pupils, providing termly records of the nature and use of IT in their lessons. The summarized quantitative pre- and post-tests for all four subjects are discussed and the primary results are related to the level of IT use analysed through a rating scheme for frequency of pupils’ IT use, developed by the project, together with the analysed data from the teachers’ own records. The results show that those classes of pupils which had the highest regular use of information technology in a subject, experienced an improvement in learning compared with low or non users. Significantly, this use was possible within the current level of resourcing available in U.K. schools, although it is clear from the evidence that how the software was used and integrated within the subject was entirely dependent upon the enthusiasm and skills of the teacher and was an important contributory factor to the pupils’ learning.
INTRODUCTION
This paper addresses the question: what can be considered as a technology enriched environment in which pupils’ learning might be enhanced? This question relates to one aspect of the 3-yr ImpacT project[l] commissioned by the U.K. Department for Education, (previously: Department of Education and Science) to evaluate the impact of information technology on children’s achievements. Over the last 20 years large scale surveys have been carried out in many countries [2,3] to ascertain the level of IT resource and use in schools. Over 250 larger scale studies of the effects of IT on pupils’ learning of particular skills and concepts have also been reported in the literature [4]. These numerous research findings provide some evidence as to the positive effects of the use of information technology on pupils’ learning, but very little evidence has been gathered from the pupils themselves about the extent and nature of their IT use in the classroom. Most surveys have depended upon teacher questionnaires and school visits and have not included pupils’ records, while in smaller focussed studies, researchers have recorded pupils’ IT experiences through pupils’ interviews, classroom observations, and teachers’ interviews, mostly focussing on specific IT uses within a particular subject domain. One of the aims of the ImpacT project was to monitor the pupils’ IT use for the duration of the project by collecting some of that information from the pupils themselves, while assessing the impact on their learning. The full details of the methodology and research aims of the project are provided in the final report [5]. The aspects of the research methodology relevant to this paper are described below.
PROJECT
DESIGN
AND
METHODOLOGY
Over 2300 pupils in three age cohorts 8-10, 12-14 and 14-16, consisting of matched pairs of classes (total 87) were included in the study, which focussed on the use of IT in the U.K. National Curriculum foundation subjects: English, mathematics, science, and geography. In each subject age cohort there were a minimum of three matched pairs of classes recruited, shown in Table 1, to allow for a possible drop out of some during the 2-yr period. CAE 2,/,-2-D
41
42
MARGARET J. Table I. ImpacT
Cox
sample of classes No. of classes
Subject
Age range
HiIT
LoIT
English
S-10 12-14 l&l6 8-10 12-14 l&l6 12-14 I+16 S-10 12-14 l&l6
S 4 4 4 4 4 5 4 4 4 3
4 4 4 4 4 4 5 4 3 3 3
Maths
Geography’ sxnce
Total sample *Secondary
87 age groups only.
According to a review by Johnstone[6], of earlier studies of pupils’ learning achievements when using IT, it has often been difficult to identify what achievements can be attributed to the impact of IT and what is the result of good innovative teaching, teacher intervention, level of IT resource, and many other factors. In order to identify the specific effects of IT (aside from these other effects), pairs of classes were chosen, with the help of local education advisers and other educationalists, where (a) good teaching and curriculum delivery, including the use of IT (high IT), were taking place, and (b) where good teaching and curriculum delivery, but without IT use (low IT) were taking place. The high IT classes were those where the teachers intended to make regular use of computers in their subject teaching across a 2-yr period; and where there was a sufficient number of computers available to enable them to achieve this. At the secondary level, this meant having either access to at least a networked room of eight or more computers on a regular basis, or the location of a number of computers permanently in the subject classroom, access to a range of appropriate software, together with a curriculum plan which would include an integrated use of IT in their lessons. At the primary level, the project required plans for the regular use of IT in the curriculum, with software considered relevant to the curriculum areas under study, and regular access to at least one computer in the classroom. The low IT classes were those where the teachers had been recommended for their curriculum innovation and delivery, using a range of educational resources, and having a reputation for good teaching practice and motivating their pupils. Each school was visited at the beginning of the project to discuss with the class teachers their curriculum plans, class groupings over the 2 years, and for the high IT classes, the hardware and software they planned to use during the project study period. Figure 1 shows the types of IT uses we expected to find in the high IT classes, linked with the different intellectual and practical activities which they might promote. In this figure general purpose software includes word-processing, spreadsheets and databases. Four types of assessments were used, linked to the different types of intellectual activities shown. Learning outcomes were assessed through the application of (pre- and post-) (paper and pencil) tests, observations, and specific tasks for topic specific reasoning (i.e. learning specific to a particular concept or area of the curriculum within each subject). The project plan also included a number of in-depth case studies in several high IT (technology enriched) classes[7], in which observations and interviews were used to explore aspects of the learning activities, pedagogical strategies and variations in organizational patterns, both in the classroom and in the school as a whole. To monitor the IT resourcing, use, and general curriculum practices in the ImpacT sample, five instruments were designed as shown in Table 2: three questionnaires, a teachers’ record book, and a pupil’s record sheet.
ANALYSIS
OF THE
SUBJECT
ASSESSMENTS
AND
USE
OF
THE
TECHNOLOGY
A full analysis of the quantitative assessments, case studies, and IT resourcing and use is given in the final report[l]. Details of the most significant assessment results are given in this paper to
Children’s
Programming General-purpose Modelling Simulations Tutorials
software
Fig.
IT concepts and skills + + -I+ f
1. Uses of IT linked
relate them to the level of technology of pupils.
learning
43
General
Reasoning in subjects + + + +
ability + + +
to different
enrichment
types of learning
(IT resourcing
Topic-specific reasoning + + + +
outcomes.
and use) of the individual
classes
Subject assessments The reasoning in subject assessments were administered near the beginning of the study and again towards the end, except for English, for which there was an additional data collection of essays during the middle of the project for some of the classes. The assessment data for subject groups were collected for individual pupils, and then treated as class results in the analysis within each age cohort. The performance of the pupils in the general ability test provided an estimate of the range and ability of the different classes of pupils enabling the assessment data to be adjusted for any initial differences in intellectual ability between pupils in the corresponding subject/age cohorts, and experimental and control pairs. The quantitative results were therefore analysed and presented in the form of the difference between pre- and post-test pupils’ performances for each age and subject cohort, e.g. mathematics, 12-14. IT resourcing and use The analysis of the IT resourcing and use data involved several modes. The level of resourcing in the schools, and the types of software used by the ImpacT classes was compared with data provided by other sources, to provide measures of the relative IT provision of the ImpacT classes compared with national averages. The time spent using IT, as reported termly, by each class teacher, was used to assess the amount of access each pupil had to the use of IT. The curriculum and subject topics listed by each teacher were used to inform us of the similarities and differences between the experimental and control classes in each pair, and across cohorts, and to check the relevance of the software to the topic being taught. The analysis of the pupils’ IT resourcing and use data provided by the pupils themselves and their teachers, included the use of three independent variables: (1) Use of IT in the focus subject; (2) Use of IT across all subjects in the curriculum; (3) Extra-mural use of different types of IT.
Table 2. Type of IT information Information
schools and classes
Data
School and class demography
Schemes, syllabuses
and other data collected from ImpacT
and (where appropriate)
IT and other resources (school level) Record of teacher’s curriculum and IT use
Record of pupil’s IT use
IT access
School role Number of pupils in class Groupings of pupils Pupils’ ages Topics Access to computers Distribution of computers in department (secondary level) Number and distribution of computers in the school Curriculum topics Names of software programs used Frequency of use Class groupings for IT use Supporting materials (for IT) Frequency of use in each subject Working with others/alone Use outside the classroom Home use of IT
Frequency OlK%
Once/term
Once/yr Once/tcml
Once/term
44
MARGARET
Table Level Subject
3. The
No
Using
use of
computers
computers
Cross-curricular
use
No
of levels of ouoils‘
I
0 use
definitions
J. Cox
4
Using
Using
Using
computers three
to five
a term
times
a term
use of
use 3
once or twice
computers
IT
2
computers once a week
5
computers several
times
a week
Using
UWlg
Using
Using
Using
computers
computers
computers
computers
computers
once OI twice
three
to five
once a week
a term
times
a term
in any
Ill any
subject
in any subject
subject
several a week
times in any
subject(s)
several a week or more
tunes in two subjects
The first two variables were given frequency of use levels ranging from 0 to 4, and 0 to 5 as shown in Table 3 and the third variable was given four categories A-D which represented types of software reported being used by the pupils, either at home or in clubs, but not the frequency of their use. The definitions shown in Table 3 were used to code every pupil’s data and to estimate the frequency of IT use for each pupil per term. The percentage of pupils with each IT level per class was then calculated to provide a class maximum and minimum median (the level reached by at least 50% of the pupils), taken from the term in which there was the greatest use reported and the term for which the lowest use was reported. For example, Table 4 shows the percentage of pupils from one class with each IT use level for two different terms; e.g. 8.7% of pupils had no IT use in the maximum use term, and 97.1% had no IT use in the minimum use term. The resultant maximum median for the highest use term is therefore 2, and for the lowest use term is 0. All of the 70 classes, out of the total sample of 87, returning pupils’ IT use data were analysed using this technique. As mentioned earlier, the pupils were also asked to indicate their extra-mural experiences of IT, both in computer clubs and at home. The four categories of use, A-D, are shown below: A-no extra-mural use; B-one type of use (games, application or programming); C-two types of use (out of games, application or programming); D-games, an application and programming. These data varied across a class of pupils and from term to term and were therefore grouped as a percentage of pupils with each category of use, showing two terms’ data to illustrate the variation in range of uses. As the data provided by the pupils does not indicate the frequency of extra-mural use no median scores were used. RESULTS The results presented here are those selected from the wider range of ImpacT results[l] to show the significance of the level of technology enrichment which enabled a positive impact of IT on children’s learning to occur. The number of computers in each school and available to the ImpacT IT class or classes in the school was recorded from the teachers’ data. From this we calculated the pupil/computer ratio per school and per ImpacT class. IT resourcing
in primary
schools
The average number of computers in the ImpacT primary schools was 7.6, compared with the national average of 4.3, and the average number of pupils per computer, at the school level, was 40.7, compared with the national average for primary schools of 40. Therefore the ImpacT primary schools had similar levels of IT resourcing to the national average, which, apart from one class, was equivalent to a maximum of only one computer per class. The primary software used by the ImpacT classes included both topic specific software, such as MICROSMILE, COGS and CIRCLE used in mathematics, to more open ended software such as database software (OUR FACTS and GRASS) in science, and word-processing and desk-top publishing software (CAXTON, WRITER and F/P EXTRA ) used in English. The greatest range of software used was by the originally designated high IT classes El, Ml, M5, M7, Sl and S9,
Children’s learning
45
with one low IT science class, S2. At the primary level the most frequently used software was for mathematics and English with very little use reported specifically for science. IT resourcing in secondary schools
The average pupil/computer ratio for the high IT and low IT ImpacT secondary schools (mostly 11 years and upwards) was 21.2 and 18.0 respectively, with an overall average of 19.9 for all the secondary ImpacT schools compared with the national average of 18. Although the provision of computers per pupil was much higher than for the primary schools, and similar to the national average, access to the schools’ computers in some classes was limited by the school timetable giving preference to business studies and information technology classes. Access to computers in the secondary schools also depended upon timetabling of bookable rooms, distribution of computers across departments, mobility of the computers, and the class teacher’s other commitments. 10% of the classes had no access to computers, while 33%, with access to networks, had a pupil/computer ratio for the class of two or less. Those classes which had computers located in their department tended to use only those and not book the computer rooms. There was therefore unequal pupils’ access to computers at both the school and the class level biased against pupils who had chosen science, humanities and languages as their subject options in favour of pupils choosing business studies and information technology options, particularly for the 14-16 pupils. At the secondary level there was greater variety of software used than at the primary level, with more simulations, subject specific software, and a combination of both topic specific and open-ended software used in many of the high IT classes. The greatest use of simulations was in the geography classes, and the lowest use in science. 62% of the low IT secondary classes reported no software being used in their classes during the ImpacT study. lime spent
usingIT
in lessor
Although the data collected from the teachers about their IT use does not provide detailed information about the educational quality of the IT experiences, it does enable us to estimate the average pupil IT exposure in each class during a term. By considering the number of pupils and computers in the class, the average number of pupils in a group working at the computer, and the number of half hours during the term in which the maximum use of IT was reported, an average use time per pupil per class can be estimated. The results for the primary class are shown in Table 5. The designated high IT classes have odd numbers and the low IT classes, even numbers. For example, on average, each pupil in mathematics high IT class 1, spent 15.5 h during a single term working at the computer, while pupils in mathematics low IT class 8 only spent an average of 6 min, even though class 1 had 41 pupils~mi~ro compared with class 8 which had 18 pupils/mi~ro. These results also show that in 67% of the ImpacT primary classes, pupils had less than 1 h on average working at the computer, for the highest IT use term reported. Of the remaining 33% of the classes who had more than an hour in a term, 57% of those were mathematics classes I, 5, 6 and 7. According to the pupils’ own IT use records, some of them used computers more than others in the same class. Therefore variation in use amongst pupils in the same class will mean that some pupils made greater use, and some less use than the time shown in Table 5.
Table 5. Maximum
Table 4. Single class example: percentage of pupils in the class with corresoondinn IT use levels Term with maximum IT use level Percent
0
1
7
26
2 65
Maximum median
IT use 3 0
4 0
2
Minimum Term with minimum IT use level Percent
0 97
1 3
2 0
IT use 3 0
median 4 0
0
average orimarv
Mathematics Class 1 2 5 6 I 8
txmil IT use in one term
Science
English
Pupil use (h)
Class
Pupil use (h)
Class
Pupil use (h)
15.5 0.4 5.5 3.0 1.4 0.1
1 2 3 4 5 6 9
0.1 0.2 0.1 0.8 1.0 0.9 4.3
I 3 4 5 6 7 8 10
0.4 0.3 0.5 0.9 0.1 0.0 2.9 0.0
MARGARETJ. Cox
46
Variation
in IT use in the class
The data have been analysed to estimate the percentage of pupils in each class with each IT level for each term for which data were collected. For example, Fig. 2 shows the results for the primary mathematics class Ml for the Autumn term, 1990 (black bars, 90.3) and the Spring term, 1991 (white bars, 91 .l). From the figure we can see that in the Autumn term more than 50% of the pupils reached a level 3, which meant that they used IT in mathematics once a week for the term. However, it also shows that more than 30% of the pupils in the same class did not use IT at all in mathematics. This class had one computer between 28 pupils. In the following term. (P91.1). the figure shows that no pupils used IT in their mathematics lessons. This example shows the wide variation in IT use between pupils in a single class and for two different terms. In some classes there was a more equal use of IT between pupils, as for example in S2, shown in Fig. 3, which had one computer between two classes (64 pupils/micro). Where the primary pupils’ returned identical data for their class use (i.e. all the same level), there was the possibility that the teacher helped them complete the record sheets, even though, in all cases, the extra-mural use was evidently provided by the pupils themselves. During these two terms no pupils in science class 2 used IT in science more than two or three times a term (level 2). In 50% of the primary classes there was an unequal use of IT between the pupils in a class. Pupil’s
extra-mural
use
The primary pupils’ extra-mural use showed a significant number of pupils across the sample with no extra-mural use, which varied from school to school. This indicates that the extra-mural use may depend upon the backgrounds of the pupils and the types of communities from which they come. Of the nine primary classes with more than 25% of the pupils having no extra-mural use of IT, eight were from inner city schools. IT use for mathematics
primary
classes
Tables 6 and 7 show an example of the overall summary results for the high and low IT classes for the primary mathematics sample, giving the maximum and minimum medians for their pupils’ IT use, the number of terms for which data were collected, the number of pupils/micro in the class, the number grouped together when working with the micro and the software used in their lessons. These results show that all three high IT classes were using computers three to five times a term in at least one term in mathematics lessons, using a range of mathematics software, compared with two of the three low IT classes reporting no IT use in mathematics at all, even though the M6’s class teacher reported using some software. The hardware resources were similar for five of the six classes, with a significantly lower level of hardware for High IT class M7. This information, gathered from over 80% of the classes, has been used to extend the analysis of the assessment results.
Mathematics
0
1
2
Class
1
3
4
Subject IT Level Fig. 2. Frequency
of pupils’
IT use for primary
mathematics
class MI
Children’s learning
sdence class 2 100 80
60
%cd
40 20 0i
PuPb
0
1
2
3
4
Subject IT Level Fig. 3. Frequency of pupils’ IT use for primary science class S2.
Comparison of IT use with subject assessment results Full statistical details of all the assessment results are given in the final report showing that there was a significant difference in pupils’ subject reasoning test performances in favour of the originally designated high IT classes compared with the low IT classes for some age and subject groups. There was a significant difference for the mathematics 8-10 and 14-16; geography, 14-16 and English 8-10 age groups, and for the topic specific mini-studies (specific assessments relating to the use of a particular piece of software) for mathematics 12-14, science 14-16, geography 14-16, English 8-10 and 12-14 age groups. However, according to the original experimental and control designations some of the other groups produced either inconclusive or even negative results, i.e. there was no significant difference or the originally designed ‘low IT’ group of classes actually performed better than its ‘high IT’ pair. However, in comparing the IT results shown in Table 5, with the assessment results (in favour of the low IT classes) for the science 8-10 age group, it is clear that apart from class S9 (a ‘high IT’ class) there was very little difference in the time spent using IT between any of the other experimental or control science 8-10 classes. These results are also supported by the pupils’ reported IT use data in Table 8, where the most frequent use reported was for the low IT class 2 and high IT class 9, which both had a median IT use value of 2, equivalent to using computers three to five times in one term only. Science primary class 2, also had the highest achievement in the assessment score for the science 8-10 group. In fact when the median IT use scores for science are compared with those for mathematics, for the whole primary sample, as shown in Table 8, the
Table 6. IT use and resourcing Median of IT in subject use HilT class I 5 7
max
min
3 2 2
0
I 0
in the primary
No. of terms (max 6)
Pupils per micro
Hours per pupil
Pupil group
4 4 6
28 33 54
2.6 13.0 I.4
45 24 2
Table 7. IT in the primary Median of IT in subject use LoIT Class 2 6 8
high IT mathematics
max
min
No. of terms (max 6)
3 0’ 0*
0 0 0
5 5 2
low IT mathematics
Pupils per micro
Hours per pupil
Pupil group
37 33 23
0.4 3.0 0.1
65 2-3 1-2
*Less than 30% of pupils used the software
throughout
the term.
Examples
classes
of software
used
Jugs, Our Facts, Microsmile, Angles Star Seeker, Dart, Writer, Turtle Hanmnan. Dart. Turtle. Ourselves
classes
Examples
of software
used
Smile, Take Fletcher’s Castle, Writer Folio. Fun Phonics
48
MARGARETJ. Cox Table 8. Mathematics
and science 8-10 pupils’ maximum
and minimum
IT use in subject
Maximum median
Minimum median
SCORS
Maths ChSS
Maximum median
Minimum median
Science class
I
3
0
I
I
0
2
3 2 0 2 0
0 I 0 0 0
2 3 4 5 6 9
2 I
0 I I 0
5 6 7 8
I I I 2
I 0
IT use is greater for the science control group (classes 2, 4 and 6) than for the experimental group (classes 1, 3, 5 and 9). Whereas for the mathematics classes, which had a significantly higher score in the assessment for the ‘high IT’ group, all three of the high IT classes have at least an IT use score of 2, equivalent to three to five times per week, with one class scoring 3, at least once a week, and for the ‘low IT’ group, two out of the three classes, 6 and 8 recorded no use of IT in any of the six terms. In conclusion, for those classes where there was a measurable difference in IT use between the originally designated high and low IT class pairs, e.g. mathematics 8-10, there was a significant difference between their subject test results in favour of the high IT classes. Whereas where there was little or no difference between the ‘experimental’ and ‘control’ classes’ IT use, e.g. science 8-l 0 (in this case, higher use of IT in the control classes) there was no result in favour of the originally designated experimental (high IT) classes. Similar results have been obtained for the secondary cohorts where the IT use recorded has been shown to be greatest for those groups of classes recording the highest subject assessment achievements.
CONCLUSIONS
The purpose of this paper was to consider the level and use of IT technology required to facilitate a positive impact of using information technology on children’s learning. There are, of course many other factors which will influence the positive learning experiences of children, many of which have been addressed by the project and reported elsewhere[l,lO]. In terms of technology enrichment in the primary and secondary classes, although only one of the 21 ImpacT primary classes had more than one computer available to the class, the primary cohort results here, selected from the ImpacT sample, show that where there is integrated use of software in the curriculum (mathematics and English) even at the level of using IT only once a week, the technology has a positive impact on pupils’ learning. In terms of IT use at the secondary level, the evidence given in the full report shows that the pupil/computer ratio for the school was not a key factor in the level of use of IT in the ImpacT classes, since pupils in some classes reported using IT at least once a week, compared with others who did not use them at all even though they had the same level of resources. There was, however, a difference in the flexibility of hardware resources which indicates a restriction on IT use imposed by the limited flexibility and the school’s time-tabling of the hardware resources and possibly lack of enthusiasm of the teacher. This was a significant constraint on some teachers, preventing them from sustaining an IT use level of even two to three times a term in their lessons. In conclusion, the assessment and IT resourcing and use results, which are described more fully in the final report[l] show that only those classes of pupils, who had regular and sustained use of information technology in the subject, showed evidence of an improvement in learning compared with non users. Significantly, this use was possible within the current level of resourcing available in U.K. schools, even in the primary mathematics classes with only one micro between 25 pupils. It is therefore clear from the evidence that how the software is used and integrated within the subject was therefore entirely dependent upon the enthusiasm and skills of the teacher and was an important contributory factor to the pupils’ learning.
Children’s
learning
49
REFERENCES 1. Watson D. M. (Ed.), The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools. King’s College London (1993). 2. Pelgrum W. J. and Plomp T., The Use of Computers in Education Worldwide-Results from the IEA Computers in Education Survey in 19 Education Systems. Pergamon Press, Oxford (1991). 3. DES, Results of the survey of Information Technology in Schools Statistical Bulletin 6/93 London. Department for Education (1993). 4. Niemiec R. P. and Walburg H. J., The effects of computers on learning. Inf. J. Educl Rex 17, 999107 (1992). 5. Cox M. J., The project Design and Method. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 7-25. King’s College London (1993). 6. Johnstone V., The evaluation of microcomputer programmes: an area for debate. J. Comput. Assisted Learning 3,40-50 (1987). 7. Watson D. M., Moore V. and Rhodes V., Case studies. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 61-96. King’s College London (1993). 8. Johnson D. C. J. and Trushell J., Primary pupils’ achievements using IT. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 97-l 14. King’s College London (1993). 9. Johnson D. C. J., Secondary pupils’ achievements using IT. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 115-150. King’s College London (1993). 10. Kibby M. R. (Ed.), Computer Assisted Learning-Selected Proceedings from the CAL ‘89 Symposium. Computers Educ. 15, l-288 (1990).
Copyright
0360-1315/93 $6.00 + 0.00 % 1993 Pergamon Press Ltd
TECHNOLOGY ENRICHED SCHOOL PROJECT-THE IMPACT OF INFORMATION TECHNOLOGY ON CHILDREN’S LEARNING MARGARET J. Cox Centre
for Educational
Studies,
King’s College London, Cornwall London SE1 8TX, England
House
Annexe.
Waterloo
Road,
Abstract-The purpose of this paper is to consider what level of technology enrichment in schools is required to achieve an enhancement of pupils’ learning. This question relates to one aspect of the 3-yr ImpacT project[l] commissioned by the U.K. Department for Education to evaluate the impact of information technology on children’s achievements in English, mathematics, science and geography, for pupils aged 8-10, 12-14 and 14-16 years. The focus of this paper is on the level and use of Information Technology (IT) in 70 of the 87 ImpacT classes which returned data from both the class teachers and each of the individual 2300 pupils, providing termly records of the nature and use of IT in their lessons. The summarized quantitative pre- and post-tests for all four subjects are discussed and the primary results are related to the level of IT use analysed through a rating scheme for frequency of pupils’ IT use, developed by the project, together with the analysed data from the teachers’ own records. The results show that those classes of pupils which had the highest regular use of information technology in a subject, experienced an improvement in learning compared with low or non users. Significantly, this use was possible within the current level of resourcing available in U.K. schools, although it is clear from the evidence that how the software was used and integrated within the subject was entirely dependent upon the enthusiasm and skills of the teacher and was an important contributory factor to the pupils’ learning.
INTRODUCTION
This paper addresses the question: what can be considered as a technology enriched environment in which pupils’ learning might be enhanced? This question relates to one aspect of the 3-yr ImpacT project[l] commissioned by the U.K. Department for Education, (previously: Department of Education and Science) to evaluate the impact of information technology on children’s achievements. Over the last 20 years large scale surveys have been carried out in many countries [2,3] to ascertain the level of IT resource and use in schools. Over 250 larger scale studies of the effects of IT on pupils’ learning of particular skills and concepts have also been reported in the literature [4]. These numerous research findings provide some evidence as to the positive effects of the use of information technology on pupils’ learning, but very little evidence has been gathered from the pupils themselves about the extent and nature of their IT use in the classroom. Most surveys have depended upon teacher questionnaires and school visits and have not included pupils’ records, while in smaller focussed studies, researchers have recorded pupils’ IT experiences through pupils’ interviews, classroom observations, and teachers’ interviews, mostly focussing on specific IT uses within a particular subject domain. One of the aims of the ImpacT project was to monitor the pupils’ IT use for the duration of the project by collecting some of that information from the pupils themselves, while assessing the impact on their learning. The full details of the methodology and research aims of the project are provided in the final report [5]. The aspects of the research methodology relevant to this paper are described below.
PROJECT
DESIGN
AND
METHODOLOGY
Over 2300 pupils in three age cohorts 8-10, 12-14 and 14-16, consisting of matched pairs of classes (total 87) were included in the study, which focussed on the use of IT in the U.K. National Curriculum foundation subjects: English, mathematics, science, and geography. In each subject age cohort there were a minimum of three matched pairs of classes recruited, shown in Table 1, to allow for a possible drop out of some during the 2-yr period. CAE 2,/,-2-D
41
42
MARGARET J. Table I. ImpacT
Cox
sample of classes No. of classes
Subject
Age range
HiIT
LoIT
English
S-10 12-14 l&l6 8-10 12-14 l&l6 12-14 I+16 S-10 12-14 l&l6
S 4 4 4 4 4 5 4 4 4 3
4 4 4 4 4 4 5 4 3 3 3
Maths
Geography’ sxnce
Total sample *Secondary
87 age groups only.
According to a review by Johnstone[6], of earlier studies of pupils’ learning achievements when using IT, it has often been difficult to identify what achievements can be attributed to the impact of IT and what is the result of good innovative teaching, teacher intervention, level of IT resource, and many other factors. In order to identify the specific effects of IT (aside from these other effects), pairs of classes were chosen, with the help of local education advisers and other educationalists, where (a) good teaching and curriculum delivery, including the use of IT (high IT), were taking place, and (b) where good teaching and curriculum delivery, but without IT use (low IT) were taking place. The high IT classes were those where the teachers intended to make regular use of computers in their subject teaching across a 2-yr period; and where there was a sufficient number of computers available to enable them to achieve this. At the secondary level, this meant having either access to at least a networked room of eight or more computers on a regular basis, or the location of a number of computers permanently in the subject classroom, access to a range of appropriate software, together with a curriculum plan which would include an integrated use of IT in their lessons. At the primary level, the project required plans for the regular use of IT in the curriculum, with software considered relevant to the curriculum areas under study, and regular access to at least one computer in the classroom. The low IT classes were those where the teachers had been recommended for their curriculum innovation and delivery, using a range of educational resources, and having a reputation for good teaching practice and motivating their pupils. Each school was visited at the beginning of the project to discuss with the class teachers their curriculum plans, class groupings over the 2 years, and for the high IT classes, the hardware and software they planned to use during the project study period. Figure 1 shows the types of IT uses we expected to find in the high IT classes, linked with the different intellectual and practical activities which they might promote. In this figure general purpose software includes word-processing, spreadsheets and databases. Four types of assessments were used, linked to the different types of intellectual activities shown. Learning outcomes were assessed through the application of (pre- and post-) (paper and pencil) tests, observations, and specific tasks for topic specific reasoning (i.e. learning specific to a particular concept or area of the curriculum within each subject). The project plan also included a number of in-depth case studies in several high IT (technology enriched) classes[7], in which observations and interviews were used to explore aspects of the learning activities, pedagogical strategies and variations in organizational patterns, both in the classroom and in the school as a whole. To monitor the IT resourcing, use, and general curriculum practices in the ImpacT sample, five instruments were designed as shown in Table 2: three questionnaires, a teachers’ record book, and a pupil’s record sheet.
ANALYSIS
OF THE
SUBJECT
ASSESSMENTS
AND
USE
OF
THE
TECHNOLOGY
A full analysis of the quantitative assessments, case studies, and IT resourcing and use is given in the final report[l]. Details of the most significant assessment results are given in this paper to
Children’s
Programming General-purpose Modelling Simulations Tutorials
software
Fig.
IT concepts and skills + + -I+ f
1. Uses of IT linked
relate them to the level of technology of pupils.
learning
43
General
Reasoning in subjects + + + +
ability + + +
to different
enrichment
types of learning
(IT resourcing
Topic-specific reasoning + + + +
outcomes.
and use) of the individual
classes
Subject assessments The reasoning in subject assessments were administered near the beginning of the study and again towards the end, except for English, for which there was an additional data collection of essays during the middle of the project for some of the classes. The assessment data for subject groups were collected for individual pupils, and then treated as class results in the analysis within each age cohort. The performance of the pupils in the general ability test provided an estimate of the range and ability of the different classes of pupils enabling the assessment data to be adjusted for any initial differences in intellectual ability between pupils in the corresponding subject/age cohorts, and experimental and control pairs. The quantitative results were therefore analysed and presented in the form of the difference between pre- and post-test pupils’ performances for each age and subject cohort, e.g. mathematics, 12-14. IT resourcing and use The analysis of the IT resourcing and use data involved several modes. The level of resourcing in the schools, and the types of software used by the ImpacT classes was compared with data provided by other sources, to provide measures of the relative IT provision of the ImpacT classes compared with national averages. The time spent using IT, as reported termly, by each class teacher, was used to assess the amount of access each pupil had to the use of IT. The curriculum and subject topics listed by each teacher were used to inform us of the similarities and differences between the experimental and control classes in each pair, and across cohorts, and to check the relevance of the software to the topic being taught. The analysis of the pupils’ IT resourcing and use data provided by the pupils themselves and their teachers, included the use of three independent variables: (1) Use of IT in the focus subject; (2) Use of IT across all subjects in the curriculum; (3) Extra-mural use of different types of IT.
Table 2. Type of IT information Information
schools and classes
Data
School and class demography
Schemes, syllabuses
and other data collected from ImpacT
and (where appropriate)
IT and other resources (school level) Record of teacher’s curriculum and IT use
Record of pupil’s IT use
IT access
School role Number of pupils in class Groupings of pupils Pupils’ ages Topics Access to computers Distribution of computers in department (secondary level) Number and distribution of computers in the school Curriculum topics Names of software programs used Frequency of use Class groupings for IT use Supporting materials (for IT) Frequency of use in each subject Working with others/alone Use outside the classroom Home use of IT
Frequency OlK%
Once/term
Once/yr Once/tcml
Once/term
44
MARGARET
Table Level Subject
3. The
No
Using
use of
computers
computers
Cross-curricular
use
No
of levels of ouoils‘
I
0 use
definitions
J. Cox
4
Using
Using
Using
computers three
to five
a term
times
a term
use of
use 3
once or twice
computers
IT
2
computers once a week
5
computers several
times
a week
Using
UWlg
Using
Using
Using
computers
computers
computers
computers
computers
once OI twice
three
to five
once a week
a term
times
a term
in any
Ill any
subject
in any subject
subject
several a week
times in any
subject(s)
several a week or more
tunes in two subjects
The first two variables were given frequency of use levels ranging from 0 to 4, and 0 to 5 as shown in Table 3 and the third variable was given four categories A-D which represented types of software reported being used by the pupils, either at home or in clubs, but not the frequency of their use. The definitions shown in Table 3 were used to code every pupil’s data and to estimate the frequency of IT use for each pupil per term. The percentage of pupils with each IT level per class was then calculated to provide a class maximum and minimum median (the level reached by at least 50% of the pupils), taken from the term in which there was the greatest use reported and the term for which the lowest use was reported. For example, Table 4 shows the percentage of pupils from one class with each IT use level for two different terms; e.g. 8.7% of pupils had no IT use in the maximum use term, and 97.1% had no IT use in the minimum use term. The resultant maximum median for the highest use term is therefore 2, and for the lowest use term is 0. All of the 70 classes, out of the total sample of 87, returning pupils’ IT use data were analysed using this technique. As mentioned earlier, the pupils were also asked to indicate their extra-mural experiences of IT, both in computer clubs and at home. The four categories of use, A-D, are shown below: A-no extra-mural use; B-one type of use (games, application or programming); C-two types of use (out of games, application or programming); D-games, an application and programming. These data varied across a class of pupils and from term to term and were therefore grouped as a percentage of pupils with each category of use, showing two terms’ data to illustrate the variation in range of uses. As the data provided by the pupils does not indicate the frequency of extra-mural use no median scores were used. RESULTS The results presented here are those selected from the wider range of ImpacT results[l] to show the significance of the level of technology enrichment which enabled a positive impact of IT on children’s learning to occur. The number of computers in each school and available to the ImpacT IT class or classes in the school was recorded from the teachers’ data. From this we calculated the pupil/computer ratio per school and per ImpacT class. IT resourcing
in primary
schools
The average number of computers in the ImpacT primary schools was 7.6, compared with the national average of 4.3, and the average number of pupils per computer, at the school level, was 40.7, compared with the national average for primary schools of 40. Therefore the ImpacT primary schools had similar levels of IT resourcing to the national average, which, apart from one class, was equivalent to a maximum of only one computer per class. The primary software used by the ImpacT classes included both topic specific software, such as MICROSMILE, COGS and CIRCLE used in mathematics, to more open ended software such as database software (OUR FACTS and GRASS) in science, and word-processing and desk-top publishing software (CAXTON, WRITER and F/P EXTRA ) used in English. The greatest range of software used was by the originally designated high IT classes El, Ml, M5, M7, Sl and S9,
Children’s learning
45
with one low IT science class, S2. At the primary level the most frequently used software was for mathematics and English with very little use reported specifically for science. IT resourcing in secondary schools
The average pupil/computer ratio for the high IT and low IT ImpacT secondary schools (mostly 11 years and upwards) was 21.2 and 18.0 respectively, with an overall average of 19.9 for all the secondary ImpacT schools compared with the national average of 18. Although the provision of computers per pupil was much higher than for the primary schools, and similar to the national average, access to the schools’ computers in some classes was limited by the school timetable giving preference to business studies and information technology classes. Access to computers in the secondary schools also depended upon timetabling of bookable rooms, distribution of computers across departments, mobility of the computers, and the class teacher’s other commitments. 10% of the classes had no access to computers, while 33%, with access to networks, had a pupil/computer ratio for the class of two or less. Those classes which had computers located in their department tended to use only those and not book the computer rooms. There was therefore unequal pupils’ access to computers at both the school and the class level biased against pupils who had chosen science, humanities and languages as their subject options in favour of pupils choosing business studies and information technology options, particularly for the 14-16 pupils. At the secondary level there was greater variety of software used than at the primary level, with more simulations, subject specific software, and a combination of both topic specific and open-ended software used in many of the high IT classes. The greatest use of simulations was in the geography classes, and the lowest use in science. 62% of the low IT secondary classes reported no software being used in their classes during the ImpacT study. lime spent
usingIT
in lessor
Although the data collected from the teachers about their IT use does not provide detailed information about the educational quality of the IT experiences, it does enable us to estimate the average pupil IT exposure in each class during a term. By considering the number of pupils and computers in the class, the average number of pupils in a group working at the computer, and the number of half hours during the term in which the maximum use of IT was reported, an average use time per pupil per class can be estimated. The results for the primary class are shown in Table 5. The designated high IT classes have odd numbers and the low IT classes, even numbers. For example, on average, each pupil in mathematics high IT class 1, spent 15.5 h during a single term working at the computer, while pupils in mathematics low IT class 8 only spent an average of 6 min, even though class 1 had 41 pupils~mi~ro compared with class 8 which had 18 pupils/mi~ro. These results also show that in 67% of the ImpacT primary classes, pupils had less than 1 h on average working at the computer, for the highest IT use term reported. Of the remaining 33% of the classes who had more than an hour in a term, 57% of those were mathematics classes I, 5, 6 and 7. According to the pupils’ own IT use records, some of them used computers more than others in the same class. Therefore variation in use amongst pupils in the same class will mean that some pupils made greater use, and some less use than the time shown in Table 5.
Table 5. Maximum
Table 4. Single class example: percentage of pupils in the class with corresoondinn IT use levels Term with maximum IT use level Percent
0
1
7
26
2 65
Maximum median
IT use 3 0
4 0
2
Minimum Term with minimum IT use level Percent
0 97
1 3
2 0
IT use 3 0
median 4 0
0
average orimarv
Mathematics Class 1 2 5 6 I 8
txmil IT use in one term
Science
English
Pupil use (h)
Class
Pupil use (h)
Class
Pupil use (h)
15.5 0.4 5.5 3.0 1.4 0.1
1 2 3 4 5 6 9
0.1 0.2 0.1 0.8 1.0 0.9 4.3
I 3 4 5 6 7 8 10
0.4 0.3 0.5 0.9 0.1 0.0 2.9 0.0
MARGARETJ. Cox
46
Variation
in IT use in the class
The data have been analysed to estimate the percentage of pupils in each class with each IT level for each term for which data were collected. For example, Fig. 2 shows the results for the primary mathematics class Ml for the Autumn term, 1990 (black bars, 90.3) and the Spring term, 1991 (white bars, 91 .l). From the figure we can see that in the Autumn term more than 50% of the pupils reached a level 3, which meant that they used IT in mathematics once a week for the term. However, it also shows that more than 30% of the pupils in the same class did not use IT at all in mathematics. This class had one computer between 28 pupils. In the following term. (P91.1). the figure shows that no pupils used IT in their mathematics lessons. This example shows the wide variation in IT use between pupils in a single class and for two different terms. In some classes there was a more equal use of IT between pupils, as for example in S2, shown in Fig. 3, which had one computer between two classes (64 pupils/micro). Where the primary pupils’ returned identical data for their class use (i.e. all the same level), there was the possibility that the teacher helped them complete the record sheets, even though, in all cases, the extra-mural use was evidently provided by the pupils themselves. During these two terms no pupils in science class 2 used IT in science more than two or three times a term (level 2). In 50% of the primary classes there was an unequal use of IT between the pupils in a class. Pupil’s
extra-mural
use
The primary pupils’ extra-mural use showed a significant number of pupils across the sample with no extra-mural use, which varied from school to school. This indicates that the extra-mural use may depend upon the backgrounds of the pupils and the types of communities from which they come. Of the nine primary classes with more than 25% of the pupils having no extra-mural use of IT, eight were from inner city schools. IT use for mathematics
primary
classes
Tables 6 and 7 show an example of the overall summary results for the high and low IT classes for the primary mathematics sample, giving the maximum and minimum medians for their pupils’ IT use, the number of terms for which data were collected, the number of pupils/micro in the class, the number grouped together when working with the micro and the software used in their lessons. These results show that all three high IT classes were using computers three to five times a term in at least one term in mathematics lessons, using a range of mathematics software, compared with two of the three low IT classes reporting no IT use in mathematics at all, even though the M6’s class teacher reported using some software. The hardware resources were similar for five of the six classes, with a significantly lower level of hardware for High IT class M7. This information, gathered from over 80% of the classes, has been used to extend the analysis of the assessment results.
Mathematics
0
1
2
Class
1
3
4
Subject IT Level Fig. 2. Frequency
of pupils’
IT use for primary
mathematics
class MI
Children’s learning
sdence class 2 100 80
60
%cd
40 20 0i
PuPb
0
1
2
3
4
Subject IT Level Fig. 3. Frequency of pupils’ IT use for primary science class S2.
Comparison of IT use with subject assessment results Full statistical details of all the assessment results are given in the final report showing that there was a significant difference in pupils’ subject reasoning test performances in favour of the originally designated high IT classes compared with the low IT classes for some age and subject groups. There was a significant difference for the mathematics 8-10 and 14-16; geography, 14-16 and English 8-10 age groups, and for the topic specific mini-studies (specific assessments relating to the use of a particular piece of software) for mathematics 12-14, science 14-16, geography 14-16, English 8-10 and 12-14 age groups. However, according to the original experimental and control designations some of the other groups produced either inconclusive or even negative results, i.e. there was no significant difference or the originally designed ‘low IT’ group of classes actually performed better than its ‘high IT’ pair. However, in comparing the IT results shown in Table 5, with the assessment results (in favour of the low IT classes) for the science 8-10 age group, it is clear that apart from class S9 (a ‘high IT’ class) there was very little difference in the time spent using IT between any of the other experimental or control science 8-10 classes. These results are also supported by the pupils’ reported IT use data in Table 8, where the most frequent use reported was for the low IT class 2 and high IT class 9, which both had a median IT use value of 2, equivalent to using computers three to five times in one term only. Science primary class 2, also had the highest achievement in the assessment score for the science 8-10 group. In fact when the median IT use scores for science are compared with those for mathematics, for the whole primary sample, as shown in Table 8, the
Table 6. IT use and resourcing Median of IT in subject use HilT class I 5 7
max
min
3 2 2
0
I 0
in the primary
No. of terms (max 6)
Pupils per micro
Hours per pupil
Pupil group
4 4 6
28 33 54
2.6 13.0 I.4
45 24 2
Table 7. IT in the primary Median of IT in subject use LoIT Class 2 6 8
high IT mathematics
max
min
No. of terms (max 6)
3 0’ 0*
0 0 0
5 5 2
low IT mathematics
Pupils per micro
Hours per pupil
Pupil group
37 33 23
0.4 3.0 0.1
65 2-3 1-2
*Less than 30% of pupils used the software
throughout
the term.
Examples
classes
of software
used
Jugs, Our Facts, Microsmile, Angles Star Seeker, Dart, Writer, Turtle Hanmnan. Dart. Turtle. Ourselves
classes
Examples
of software
used
Smile, Take Fletcher’s Castle, Writer Folio. Fun Phonics
48
MARGARETJ. Cox Table 8. Mathematics
and science 8-10 pupils’ maximum
and minimum
IT use in subject
Maximum median
Minimum median
SCORS
Maths ChSS
Maximum median
Minimum median
Science class
I
3
0
I
I
0
2
3 2 0 2 0
0 I 0 0 0
2 3 4 5 6 9
2 I
0 I I 0
5 6 7 8
I I I 2
I 0
IT use is greater for the science control group (classes 2, 4 and 6) than for the experimental group (classes 1, 3, 5 and 9). Whereas for the mathematics classes, which had a significantly higher score in the assessment for the ‘high IT’ group, all three of the high IT classes have at least an IT use score of 2, equivalent to three to five times per week, with one class scoring 3, at least once a week, and for the ‘low IT’ group, two out of the three classes, 6 and 8 recorded no use of IT in any of the six terms. In conclusion, for those classes where there was a measurable difference in IT use between the originally designated high and low IT class pairs, e.g. mathematics 8-10, there was a significant difference between their subject test results in favour of the high IT classes. Whereas where there was little or no difference between the ‘experimental’ and ‘control’ classes’ IT use, e.g. science 8-l 0 (in this case, higher use of IT in the control classes) there was no result in favour of the originally designated experimental (high IT) classes. Similar results have been obtained for the secondary cohorts where the IT use recorded has been shown to be greatest for those groups of classes recording the highest subject assessment achievements.
CONCLUSIONS
The purpose of this paper was to consider the level and use of IT technology required to facilitate a positive impact of using information technology on children’s learning. There are, of course many other factors which will influence the positive learning experiences of children, many of which have been addressed by the project and reported elsewhere[l,lO]. In terms of technology enrichment in the primary and secondary classes, although only one of the 21 ImpacT primary classes had more than one computer available to the class, the primary cohort results here, selected from the ImpacT sample, show that where there is integrated use of software in the curriculum (mathematics and English) even at the level of using IT only once a week, the technology has a positive impact on pupils’ learning. In terms of IT use at the secondary level, the evidence given in the full report shows that the pupil/computer ratio for the school was not a key factor in the level of use of IT in the ImpacT classes, since pupils in some classes reported using IT at least once a week, compared with others who did not use them at all even though they had the same level of resources. There was, however, a difference in the flexibility of hardware resources which indicates a restriction on IT use imposed by the limited flexibility and the school’s time-tabling of the hardware resources and possibly lack of enthusiasm of the teacher. This was a significant constraint on some teachers, preventing them from sustaining an IT use level of even two to three times a term in their lessons. In conclusion, the assessment and IT resourcing and use results, which are described more fully in the final report[l] show that only those classes of pupils, who had regular and sustained use of information technology in the subject, showed evidence of an improvement in learning compared with non users. Significantly, this use was possible within the current level of resourcing available in U.K. schools, even in the primary mathematics classes with only one micro between 25 pupils. It is therefore clear from the evidence that how the software is used and integrated within the subject was therefore entirely dependent upon the enthusiasm and skills of the teacher and was an important contributory factor to the pupils’ learning.
Children’s
learning
49
REFERENCES 1. Watson D. M. (Ed.), The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools. King’s College London (1993). 2. Pelgrum W. J. and Plomp T., The Use of Computers in Education Worldwide-Results from the IEA Computers in Education Survey in 19 Education Systems. Pergamon Press, Oxford (1991). 3. DES, Results of the survey of Information Technology in Schools Statistical Bulletin 6/93 London. Department for Education (1993). 4. Niemiec R. P. and Walburg H. J., The effects of computers on learning. Inf. J. Educl Rex 17, 999107 (1992). 5. Cox M. J., The project Design and Method. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 7-25. King’s College London (1993). 6. Johnstone V., The evaluation of microcomputer programmes: an area for debate. J. Comput. Assisted Learning 3,40-50 (1987). 7. Watson D. M., Moore V. and Rhodes V., Case studies. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 61-96. King’s College London (1993). 8. Johnson D. C. J. and Trushell J., Primary pupils’ achievements using IT. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 97-l 14. King’s College London (1993). 9. Johnson D. C. J., Secondary pupils’ achievements using IT. In The ImpacT Report-an evaluation of the impact of Information Technology on children’s achievements in primary and secondary schools (Edited by Watson D. M.), pp. 115-150. King’s College London (1993). 10. Kibby M. R. (Ed.), Computer Assisted Learning-Selected Proceedings from the CAL ‘89 Symposium. Computers Educ. 15, l-288 (1990).