- Email: [email protected]
Improving teaching at a distance within the University of the South Pacific
07380.593188 u.Oo+ .Oo Pergamon Press plc
In;. 1. Edwuional Development. Vol. 8, No. 3, pp. 245-270.1988 Printed in Great Britain
IMPROVING TEACHING AT A DISTANCE WITHIN THE UNIVERSITY OF THE SOUTH PACIFIC* F. G. LOCKWOOD,
A. I. WILLIAMS
and D. W. ROBERTS
Open University, United Kingdom and University of the South Pacific, Fiji Abstract-A preliminary to improving the teaching materiai that is studied at the University of the South Pacific involves the identification and consideration of those factors that may detract from their teaching effectiveness. The relatively high withdrawal, dropout and failure rates on a number of courses indicated possible weaknesses in the teaching methods. The investigation of three emergent concerns, judged to be directly related to the teaching problem, is currently contributing to the improvement of distance teaching material.
The development and growth of distance education at the University of the South Pacific The offering of tertiary educational opportunities, at a distance, to the peoples of Cook Islands, Fiji, Kiribati, Nauru, Niue, Solomon Islands, Tokelau, Tonga, Tuvalu, Vanuatu and Western Samoa was part of the early concept of the University of the South Pacific (USP). The first students were admitted to oncampus studies in 1968 and the University was formally established in 1970 in accordance with the wishes and to serve the needs of these countries. In that same year, 1970, Extension Services began its operations within and as part of the School of Education. It became a separate, semi-autonomous unit of the University in 1974. The University of the South Pacific is thus a dual-mode institution in that it offers face-to-face teaching programmes to on campus students and distance study programmes to off-campus students. Extension Services has two responsibilities: (a) to prepare and deliver to distant learners throughout the South Pacific region an increasing number of University programmes, and (b) to encourage and organise all manner of other, non-credit learning opportunities. *The authors would like to thank the British Council for funding the visit by Fred Lockwood to the University of the South Pacific and supporting the work of Extension Services.
These two aspects of the work of Extension Services are called Extension Studies and Continuing Education respectively. The initial task of Extension Studies was to produce ‘Prelimin~y’ courses (the equivalent of New Zealand University Entrance Examination) to save students from countries without sixth forms spending that year on campus. Subsequently the whole of the ‘Foundation’ year could be completed through Extension. First year degree courses followed making it possible to complete the eight courses they needed for the first year of a Bachelor of Arts and Bachelor of Education degrees without having to leave their families and jobs. By 1987 a number of these people could also complete their second year courses and make a start on their final year for the degree. Over the past ten years the University has added courses which make possible the completion of 14 vocational programmes. In 1987 some 3500 students completed 8000 course enrolments. Whereas the development and growth of distance education courses at the University has been impressive it has not been without some pedagogical problems. Many academic staff experience difficulties in adapting to the demands of teaching at a distance. In this paper we shall examine three specific teaching problems that have been identified and various ways and means whereby these problems are being addressed. The three problems concern the effectiveness of pre-registration counselling, the ‘readability’ of course materials and effect of student workload.
265
266
F. G. LOCKWOOD
Course development and support services Most of the courses through Extension Studies to date, are replicas of the same courses taught on campus; they have the same examination papers as their on-campus equivalents. However, a number of courses and programmes are now being offered through Extension Studies only, and are not available to on-campus students. The Certificates in Librarianship and in the Teaching of English as a Second Language are examples of programmes offered only through distance study. University staff of the teaching departments are usually the course writers responsible for the origination and revision of the content of the Extension courses. They are part of the basic course development (and revision) team, the other members of which are instructional designers, editors and media experts. Some courses are built around a commercial text, some consist entirely of materials written by USP staff; most have an anthology of readings. Increasing numbers include one or more audiotapes and some have video materials. Textbooks are bought in bulk through the University Book Centre, tapes are produced by the University Media Unit, and USP written materials are printed commercially in Suva. Everything a learner needs to succeed in a course has to be able to be delivered by post. The materials needed by a regional Centre to meet its estimate of the enrolments for a coming semester are despatched to the Centre. The Centre counsels and enrols the students of the regional country it serves, distributes course materials, arranges tutorial support and supervises examinations. Most courses have several methods of allowing students to monitor their progress and check their understanding. Self-assessment questions with answers/follow-up comments are integrated into the materials. Courses also have short exercises or quizzes which are marked by a local tutor. All courses have two or more major assignments which are sent to the USP Campus in Suva for marking. The staff of the USP Centres together with local tutors handle most of the students’ academic, administrative and study-skills enquiries, and provide much professional and personal advice. Each Centre provides library and other study facilities. Centres also offer many students the opportunity of participating in satellite tutorial sessions led by their on-
ec al.
campus course tutors using the USPNet satellite network. These satellite tutorials are one way in which the self-inst~ctional study materials are supplemented for those who can and choose to attend these weekly or fortnightly sessions. Some course tutors also travel to regional countries to collaborate with local tutors and to meet their students face-toface. EMERGENT
CONCERNS
Student performance within individual courses and across the whole University is monitored each semester. This is one means whereby individual staff members can assess the teaching effectiveness of their materials and, where necessary, modify their future teaching techniques so as to achieve the stated goals. It allows members of a Faculty to compare and contrast student performance across the range of courses offered-to modify or develop these courses to meet South Pacific, regional and student needs. It allows, within the University as a whole, the identification of any emergent trends, irregularities or inconsistencies and their subsequent exploration. Within the USP the monitoring of student performance has focused on two major areas - academic performance on continuous and final assessment, and rates of withdrawal and drop out. Whilst there is obvious concern over the academic standards achieved by students in progressing towards their degree there is equal concern over those students who register for courses, commit time, money and effort, but who withdraw and drop out in large numbers. The pool of potential students within the South Pacific is finite. South Pacific regional needs for an educated workforce are great and cannot afford the rates of attrition experienced on many courses. (The average non-completion rate for all. courses offered through Extension studies by the USP has been 40-42% for a number of years. The range for courses offered in any one semester can be great-from 15% to So%.) Furthermore, the effect upon students of withdrawal, drop out . . 1 self imposed failure, may be considerable. Discussion within Extension Services identified a number of factors that appear to be associated with unusually high student withdrawal and non-completion on some USP courses:
IMPROVING TEACHING
inadequate pre-registration counselling and lack of preparatory materials; readability levels of teaching materials that are beyond those of many students; excessive student workloads. PRE-REGISTRATION COUNSELLING AND PROVISION OF PREPARATORY MATERIALS Effective pre-registration counselling can prevent students enrolling on a course for which they may have insufficient prerequisite knowledge or skills, inadequate time available for its study, or insufficient funds to pay for it, and in a course that does not fit into their programme of study or which is simply not relevant to their specific needs. In addition, the provision of preparatory materials, particularly for those enrolling for the first time, can be invaluable in helping students to address the academic demands of a subject, to explore different study techniques, to organise their study time, plan assignments and prepare for the examinations. The USP offers pre-enrolment counselling at each of its ten regional Centres. Centre staff have a sincere interest in the academic welfare of potential students but their advice is dependent upon the quality of the information at their disposal and their skill in counselling. Ensuring the accuracy of advice is not easy when new courses and programmes are being introduced with unusual rapidity to meet changing regional demands. Nevertheless, better informed and more highly skilled academic counsellors should help to reduce the .number of unnecessary student withdrawals. To this end workshops for Centre staff, designed to refine their counselling skills and share expertise, have been mounted and future workshops are envisaged. These workshops have concentrated on making the academic counsellors more perceptive of individual students needs, more aware of the demands of each course and the prerequisite knowledge required to be able to succeed in each course. The workshops also emphasised methods of improving the study skills of students. Many USP Extension students have never had an opportunity to study at tertiary level. They may have misconceptions about the study skills needed for undergraduate study, they
267
AT A DISTANCE
may have poorly developed study skills or, because of lack of practice, require an opportunity to practise them again. Indeed, evidence collected from students studying a post-foundation course within the British Open University-and prior to the provision of preparatory materials to all new undergraduates (Lockwood, 1986) revealed that large proportions of students maintained they had never received advice on different reading strategies they could adopt, different note taking methods, ways of organising their time, planning and writing assignments. In anticipation of this need amongst USP Extension students a study skills booklet, Studying by Extension, was assembled (Livingston, 1980) and is currently being revised for recirculation. Although a helpful resource for new students it was often not made available at the right. time and was seldom followed-up by face-to-face discussion sessions. READABILITY OF TEACHING MATERIALS Concern that the readability level of the teaching materials given to USP students may be above their level of comprehension, and thus counterproductive, is not new. Research conducted with on-campus students several years ago noted that Foundation level students at USP performed significantly below the required levels of reading, vocabulary and comprehension as measured by Progressive Achievement Tests (Elley, 1984). It was reported that 97% of New Zealand (native English speakers) were within the top 2 criterion levels of 9 and 10 on the Progressive Achievement Tests (Elley, 1984). It was reported that 97% of New Zealand students (native English speakers) were within the top 2 Yet most of the texts and reading materials for Foundation courses require a comprehension of at least Level 9 . . . Subsequent studies of first and third year students by the writer revealed Cloze Test scores, based on typical selections from their prescribed textbooks, well below the levels of apparent intelligibility. (Elley, 1984)
In a similar study of 172 USP students studying six Extension courses in four countries of the South Pacific,, .a Cloze Test was used to assess the readabthty of the distance teaching materials (Tuimaleali’ifano, 1985). The appropriateness of the Cloze Test (which
268
F. G. LOCKWOOD
is essentially the presentation of selected passages of teaching material within which every fifth, seventh or tenth word is deleted and which the student is asked to fill in with the missing word) has been repeatedly demonstrated as a valid indicator of the readability of teaching materials (Gilmore and Wagner, 1985). The major strength of the Cloze Test is that it is possible to assess the readability level of materials students will actually study. The Cloze Test . . can demonstrate directly whether a particular prose passage is suitable, in terms of difficulty level for each individual student. (Elley, 1977)
Student scores on Cloze Tests of between 40-45% are widely regarded as the minimum for the comprehension of teaching materials (Harrison, 1980). The findings from the survey conducted by Tuimaleali’ifano revealed that in only one of the six courses did students perform at a level equivalent to the minimum required. The mean score of the eleven students in the one course was 43%. However, Tuimaleali’ifano reported that six of the eleven students had completed the Cloze Test, based on selected passages, at a point late in their course of study and had thus already been exposed to the material constituting the Cloze Test. This exposure may have inflated the results and the remainder of Tuimaleali’ifano’s findings give great cause for concern. The student mean scores in the other five courses were 15%, 20%, 26%, 26% and 35% respectively; markedly below the accepted minimum of 40-45% required for comprehension. . . the readability level of the Extension courses under survey leaves much to be desired . . . only 6 of the 172 students scored over 50% in their respective (Cloze) test, and of these, only one scored over 57%. When these statistics are supplemented by the high occurence of zero test scores across the courses, it becomes easy to picture withdrawal from courses and failure to attend exams as a natural reaction of students highly discouraged by materials they are either unable to read with meaning or find so difficult to plough through that the only option remaining is to drop out and/or not turn up for the exams: they are going to fail anyway. (Tuimaleali’ifano, 1985)
Further investigations into the readability of USP materials are currently being conducted. However, current findings appear to have
et al.
major implications for teachers at USP. The USP Extension Studies course materials often include commercial textbooks. Nearly every course also has a ‘Reader’ which contains an anthology of selected articles and extracts from various sources. There is some evidence which shows that it is these materials with which students have most readability problems, and that the prose written by University staff with their students in mind is more easily read and understood. However, having students use original or seminal readings for degree courses is considered essential and so the readability problem remains. One way of alleviating this problem may be to provide glossaries and commentaries on the required readings. Another could be to persuade the student to grapple with an important reading by incorporating self assessment questions with model answers into the study materials. Much more attention is being paid to these and other ways of helping students’ reading now that investigations have revealed the nature and significance of the problem. In a region dominated (until very recently) by oral communication, Extension Services is encouraging the formation of peer tutorial groups and the greater use of local tutors. EXCESSIVE
STUDENT
WORKLOAD
Distance teaching institutions typically specify the amount of student study time an individual course and a whole programme of study is likely to take. Guidelines provided by the USP (University of the South Pacific, 1986) indicate that a typical semester course is thought to require about 10 hours of study per week for the 16 weeks of the semester. Unfortunately, there is often a mismatch between institutional claims as to what constitutes one week of study time and the demands the materials make (Roberts, 1984). The task is often one of reconciling a teacher’s enthusiasm and desire to produce a balanced, comprehensive, detailed set of teaching material with the need to package it for specified periods of study. Producing overlength material creates problems across the whole institution, which may be reflected in high dropout rates in individual courses and low student throughput. For the administration it may be the logistical and financial costs of producing, storing and disseminating extra
IMPROVING TEACHING AT A DISTANCE
material. For the Faculty and its members it may result in a reduction of takeup rates, course popularity and the increased time it takes to generate overlength material. For the student, the impact of facing overlength material results in a variety of reactions. Often survival strategies are employed which may not be the most approprate for effective learning or, faced by cumulative demands, students merely drop out of the course. The potential link between overlength courses and high student dropout rate was illustrated during the consideration of a Foundation Course offered, by Extension study, by USP.’ One of the authors examined the materials and study demands of this course in particular because of the unusually high student attrition rates and severe criticisms of the course levelled by various USP staff members. The investigation revealed that in 1986, 242 students registered for the course. There were seven official withdrawals but 55% failed to submit regular assignments and to sit the examinationthey dropped out of the course. Of the 107 who sat the examination only 38 passed; 15% of those who started the course. The course was described as requiring a modest amount of study time. Indeed, the figure of six hours per week was suggested. Estimates of student study-time were made for constituent elements in each weekly unit - 1 hour per week for ‘required’ tutorials, 2 hours per week for each weekly assignment. Estimates for textual materials were based on a ‘rule of thumb’ developed within the British Open University (Open University, 1985). The student study time (study time loot reading time) for textual material regarded as ‘easy’ was calculated at a rate of 100 words per minute (wpm), moderately difficult material at 70 wpm and difficult material at 40 wpm. From the discussion of readability above it is likely that much of the material would be regarded as, at least, ‘moderately difficult’ if not ‘difficult’ (an assumption confirmed by an inspection of the textual material provided in the course.) If the study time estimates are calculated for the first half of the course, with textual material assessed as of ‘moderate difficulty’ and then again as ‘difficult’, the implications for student study time can be seen in Fig. 1. In the first half of the course only two units corresponded to a workload of about 10 hours per week (both in excess of the workload
269
30
20 2 B 10,
0
1
2
3
4 UNIT
5
6
7
KEY
q Assignment
Rdings
e.ioderate)
Tutorial
lemi@sjudgedtobeof
‘moderate’ difficulty1
Fig. 1. Estimates of student workload.
“: UNIT
Fig. 2. Submission of weekly assignments.
specified in the course introduction) with other units of 20-30 hours in length. Reliable measures of the actual time of student drop out are notoriously difficult to obtain. However, if the submission of weekly assignments is taken as an indicator of students’ continuing study then the record of the submission of weekly assignments for marking in Suva for the first half of the course denotes a depressing trend (see Fig. 2). At the
F. G. LOCKWOOD
270
half way point in the course only 14 students were submitting regular assignments. The identification of excessive workload and of large fluctuations in the weekly demands of a course is possible and could contribute to a reduction in student withdrawal and drop out rates. The report to Extension Services by Lockwood at the end of his short investigation of student workloads was sent to all course writers, developers and Heads of Departments. His major findings were referred to academic meetings at the University. Appropriate action to extend his investigation using his methodology is being taken by the Course Development officers, and should result in more realistic workloads in new courses and when existing courses are revised. CONCLUDING
COMMENTS
The evaluation of the Extension Studies courses within USP is a continuous process with students’ comments sought during and at the end of each course. Part-time tutors and script markers are also asked to provide evaluatory information. Focussing on individual elements of distance education for occasional intensive investigation within an institution can provide most valuable and salutary data and the justification for urgent action to improve both courses and systems. The three investigations described above are an attempt to improve teaching at a distance within the University of the South Pacific. In due course it is hoped that clear evidence will
et al.
be available to confirm that the steps taken to improve teaching have been worthwhile.
NOTE 1. The investigation also revealed the role played by regular, weekly assignments with excessively long turn round times, requirements to attend tutorials and the need to purchase expensive texts.
REFERENCES Elley, W. B. (1977) A Close Look at the Cloze Test. NZ Council for Educational Research. Elley, W. B. (1984) Exploring the readability difficulties of second language learners in Fiji. In Reading in a Foreign Language (edited by Alderson, J. C. and Urquhart, H.). Longman, London. Gilmore, A. and Wagner, G. (1985) The Readability of Trade Examinations. NZ Council for Educational Research. Harrison, C. (1980) Readability in the Classroom. Cambridge University Press, Cambridge. Livingston, K. T. (1980) Studying by Extension. University of the South Pacific. Lockwood, F. G. (1986) Preparing students for distance learning. In Open Learning, pp. 44-45. Open University, Milton Keynes. Open University (1985) Making Self-Instructional Material for Adults: the Distant Teacher’s Handbook. The Open University Press, Milton Keynes. Roberts, D. W. (1984) An analysis of distance education patterns at R.C.A.E. Unpublished Graduate Diploma Project. Tuimaleali’ifano, E. (1985) A readability survey of USP Extension Course materials. Extension Services, the University of the South Pacific (mimeo). University of the South Pacific (1986) Extension Services Handbook 1987. University of the South Pacific, Suva.
In;. 1. Edwuional Development. Vol. 8, No. 3, pp. 245-270.1988 Printed in Great Britain
IMPROVING TEACHING AT A DISTANCE WITHIN THE UNIVERSITY OF THE SOUTH PACIFIC* F. G. LOCKWOOD,
A. I. WILLIAMS
and D. W. ROBERTS
Open University, United Kingdom and University of the South Pacific, Fiji Abstract-A preliminary to improving the teaching materiai that is studied at the University of the South Pacific involves the identification and consideration of those factors that may detract from their teaching effectiveness. The relatively high withdrawal, dropout and failure rates on a number of courses indicated possible weaknesses in the teaching methods. The investigation of three emergent concerns, judged to be directly related to the teaching problem, is currently contributing to the improvement of distance teaching material.
The development and growth of distance education at the University of the South Pacific The offering of tertiary educational opportunities, at a distance, to the peoples of Cook Islands, Fiji, Kiribati, Nauru, Niue, Solomon Islands, Tokelau, Tonga, Tuvalu, Vanuatu and Western Samoa was part of the early concept of the University of the South Pacific (USP). The first students were admitted to oncampus studies in 1968 and the University was formally established in 1970 in accordance with the wishes and to serve the needs of these countries. In that same year, 1970, Extension Services began its operations within and as part of the School of Education. It became a separate, semi-autonomous unit of the University in 1974. The University of the South Pacific is thus a dual-mode institution in that it offers face-to-face teaching programmes to on campus students and distance study programmes to off-campus students. Extension Services has two responsibilities: (a) to prepare and deliver to distant learners throughout the South Pacific region an increasing number of University programmes, and (b) to encourage and organise all manner of other, non-credit learning opportunities. *The authors would like to thank the British Council for funding the visit by Fred Lockwood to the University of the South Pacific and supporting the work of Extension Services.
These two aspects of the work of Extension Services are called Extension Studies and Continuing Education respectively. The initial task of Extension Studies was to produce ‘Prelimin~y’ courses (the equivalent of New Zealand University Entrance Examination) to save students from countries without sixth forms spending that year on campus. Subsequently the whole of the ‘Foundation’ year could be completed through Extension. First year degree courses followed making it possible to complete the eight courses they needed for the first year of a Bachelor of Arts and Bachelor of Education degrees without having to leave their families and jobs. By 1987 a number of these people could also complete their second year courses and make a start on their final year for the degree. Over the past ten years the University has added courses which make possible the completion of 14 vocational programmes. In 1987 some 3500 students completed 8000 course enrolments. Whereas the development and growth of distance education courses at the University has been impressive it has not been without some pedagogical problems. Many academic staff experience difficulties in adapting to the demands of teaching at a distance. In this paper we shall examine three specific teaching problems that have been identified and various ways and means whereby these problems are being addressed. The three problems concern the effectiveness of pre-registration counselling, the ‘readability’ of course materials and effect of student workload.
265
266
F. G. LOCKWOOD
Course development and support services Most of the courses through Extension Studies to date, are replicas of the same courses taught on campus; they have the same examination papers as their on-campus equivalents. However, a number of courses and programmes are now being offered through Extension Studies only, and are not available to on-campus students. The Certificates in Librarianship and in the Teaching of English as a Second Language are examples of programmes offered only through distance study. University staff of the teaching departments are usually the course writers responsible for the origination and revision of the content of the Extension courses. They are part of the basic course development (and revision) team, the other members of which are instructional designers, editors and media experts. Some courses are built around a commercial text, some consist entirely of materials written by USP staff; most have an anthology of readings. Increasing numbers include one or more audiotapes and some have video materials. Textbooks are bought in bulk through the University Book Centre, tapes are produced by the University Media Unit, and USP written materials are printed commercially in Suva. Everything a learner needs to succeed in a course has to be able to be delivered by post. The materials needed by a regional Centre to meet its estimate of the enrolments for a coming semester are despatched to the Centre. The Centre counsels and enrols the students of the regional country it serves, distributes course materials, arranges tutorial support and supervises examinations. Most courses have several methods of allowing students to monitor their progress and check their understanding. Self-assessment questions with answers/follow-up comments are integrated into the materials. Courses also have short exercises or quizzes which are marked by a local tutor. All courses have two or more major assignments which are sent to the USP Campus in Suva for marking. The staff of the USP Centres together with local tutors handle most of the students’ academic, administrative and study-skills enquiries, and provide much professional and personal advice. Each Centre provides library and other study facilities. Centres also offer many students the opportunity of participating in satellite tutorial sessions led by their on-
ec al.
campus course tutors using the USPNet satellite network. These satellite tutorials are one way in which the self-inst~ctional study materials are supplemented for those who can and choose to attend these weekly or fortnightly sessions. Some course tutors also travel to regional countries to collaborate with local tutors and to meet their students face-toface. EMERGENT
CONCERNS
Student performance within individual courses and across the whole University is monitored each semester. This is one means whereby individual staff members can assess the teaching effectiveness of their materials and, where necessary, modify their future teaching techniques so as to achieve the stated goals. It allows members of a Faculty to compare and contrast student performance across the range of courses offered-to modify or develop these courses to meet South Pacific, regional and student needs. It allows, within the University as a whole, the identification of any emergent trends, irregularities or inconsistencies and their subsequent exploration. Within the USP the monitoring of student performance has focused on two major areas - academic performance on continuous and final assessment, and rates of withdrawal and drop out. Whilst there is obvious concern over the academic standards achieved by students in progressing towards their degree there is equal concern over those students who register for courses, commit time, money and effort, but who withdraw and drop out in large numbers. The pool of potential students within the South Pacific is finite. South Pacific regional needs for an educated workforce are great and cannot afford the rates of attrition experienced on many courses. (The average non-completion rate for all. courses offered through Extension studies by the USP has been 40-42% for a number of years. The range for courses offered in any one semester can be great-from 15% to So%.) Furthermore, the effect upon students of withdrawal, drop out . . 1 self imposed failure, may be considerable. Discussion within Extension Services identified a number of factors that appear to be associated with unusually high student withdrawal and non-completion on some USP courses:
IMPROVING TEACHING
inadequate pre-registration counselling and lack of preparatory materials; readability levels of teaching materials that are beyond those of many students; excessive student workloads. PRE-REGISTRATION COUNSELLING AND PROVISION OF PREPARATORY MATERIALS Effective pre-registration counselling can prevent students enrolling on a course for which they may have insufficient prerequisite knowledge or skills, inadequate time available for its study, or insufficient funds to pay for it, and in a course that does not fit into their programme of study or which is simply not relevant to their specific needs. In addition, the provision of preparatory materials, particularly for those enrolling for the first time, can be invaluable in helping students to address the academic demands of a subject, to explore different study techniques, to organise their study time, plan assignments and prepare for the examinations. The USP offers pre-enrolment counselling at each of its ten regional Centres. Centre staff have a sincere interest in the academic welfare of potential students but their advice is dependent upon the quality of the information at their disposal and their skill in counselling. Ensuring the accuracy of advice is not easy when new courses and programmes are being introduced with unusual rapidity to meet changing regional demands. Nevertheless, better informed and more highly skilled academic counsellors should help to reduce the .number of unnecessary student withdrawals. To this end workshops for Centre staff, designed to refine their counselling skills and share expertise, have been mounted and future workshops are envisaged. These workshops have concentrated on making the academic counsellors more perceptive of individual students needs, more aware of the demands of each course and the prerequisite knowledge required to be able to succeed in each course. The workshops also emphasised methods of improving the study skills of students. Many USP Extension students have never had an opportunity to study at tertiary level. They may have misconceptions about the study skills needed for undergraduate study, they
267
AT A DISTANCE
may have poorly developed study skills or, because of lack of practice, require an opportunity to practise them again. Indeed, evidence collected from students studying a post-foundation course within the British Open University-and prior to the provision of preparatory materials to all new undergraduates (Lockwood, 1986) revealed that large proportions of students maintained they had never received advice on different reading strategies they could adopt, different note taking methods, ways of organising their time, planning and writing assignments. In anticipation of this need amongst USP Extension students a study skills booklet, Studying by Extension, was assembled (Livingston, 1980) and is currently being revised for recirculation. Although a helpful resource for new students it was often not made available at the right. time and was seldom followed-up by face-to-face discussion sessions. READABILITY OF TEACHING MATERIALS Concern that the readability level of the teaching materials given to USP students may be above their level of comprehension, and thus counterproductive, is not new. Research conducted with on-campus students several years ago noted that Foundation level students at USP performed significantly below the required levels of reading, vocabulary and comprehension as measured by Progressive Achievement Tests (Elley, 1984). It was reported that 97% of New Zealand (native English speakers) were within the top 2 criterion levels of 9 and 10 on the Progressive Achievement Tests (Elley, 1984). It was reported that 97% of New Zealand students (native English speakers) were within the top 2 Yet most of the texts and reading materials for Foundation courses require a comprehension of at least Level 9 . . . Subsequent studies of first and third year students by the writer revealed Cloze Test scores, based on typical selections from their prescribed textbooks, well below the levels of apparent intelligibility. (Elley, 1984)
In a similar study of 172 USP students studying six Extension courses in four countries of the South Pacific,, .a Cloze Test was used to assess the readabthty of the distance teaching materials (Tuimaleali’ifano, 1985). The appropriateness of the Cloze Test (which
268
F. G. LOCKWOOD
is essentially the presentation of selected passages of teaching material within which every fifth, seventh or tenth word is deleted and which the student is asked to fill in with the missing word) has been repeatedly demonstrated as a valid indicator of the readability of teaching materials (Gilmore and Wagner, 1985). The major strength of the Cloze Test is that it is possible to assess the readability level of materials students will actually study. The Cloze Test . . can demonstrate directly whether a particular prose passage is suitable, in terms of difficulty level for each individual student. (Elley, 1977)
Student scores on Cloze Tests of between 40-45% are widely regarded as the minimum for the comprehension of teaching materials (Harrison, 1980). The findings from the survey conducted by Tuimaleali’ifano revealed that in only one of the six courses did students perform at a level equivalent to the minimum required. The mean score of the eleven students in the one course was 43%. However, Tuimaleali’ifano reported that six of the eleven students had completed the Cloze Test, based on selected passages, at a point late in their course of study and had thus already been exposed to the material constituting the Cloze Test. This exposure may have inflated the results and the remainder of Tuimaleali’ifano’s findings give great cause for concern. The student mean scores in the other five courses were 15%, 20%, 26%, 26% and 35% respectively; markedly below the accepted minimum of 40-45% required for comprehension. . . the readability level of the Extension courses under survey leaves much to be desired . . . only 6 of the 172 students scored over 50% in their respective (Cloze) test, and of these, only one scored over 57%. When these statistics are supplemented by the high occurence of zero test scores across the courses, it becomes easy to picture withdrawal from courses and failure to attend exams as a natural reaction of students highly discouraged by materials they are either unable to read with meaning or find so difficult to plough through that the only option remaining is to drop out and/or not turn up for the exams: they are going to fail anyway. (Tuimaleali’ifano, 1985)
Further investigations into the readability of USP materials are currently being conducted. However, current findings appear to have
et al.
major implications for teachers at USP. The USP Extension Studies course materials often include commercial textbooks. Nearly every course also has a ‘Reader’ which contains an anthology of selected articles and extracts from various sources. There is some evidence which shows that it is these materials with which students have most readability problems, and that the prose written by University staff with their students in mind is more easily read and understood. However, having students use original or seminal readings for degree courses is considered essential and so the readability problem remains. One way of alleviating this problem may be to provide glossaries and commentaries on the required readings. Another could be to persuade the student to grapple with an important reading by incorporating self assessment questions with model answers into the study materials. Much more attention is being paid to these and other ways of helping students’ reading now that investigations have revealed the nature and significance of the problem. In a region dominated (until very recently) by oral communication, Extension Services is encouraging the formation of peer tutorial groups and the greater use of local tutors. EXCESSIVE
STUDENT
WORKLOAD
Distance teaching institutions typically specify the amount of student study time an individual course and a whole programme of study is likely to take. Guidelines provided by the USP (University of the South Pacific, 1986) indicate that a typical semester course is thought to require about 10 hours of study per week for the 16 weeks of the semester. Unfortunately, there is often a mismatch between institutional claims as to what constitutes one week of study time and the demands the materials make (Roberts, 1984). The task is often one of reconciling a teacher’s enthusiasm and desire to produce a balanced, comprehensive, detailed set of teaching material with the need to package it for specified periods of study. Producing overlength material creates problems across the whole institution, which may be reflected in high dropout rates in individual courses and low student throughput. For the administration it may be the logistical and financial costs of producing, storing and disseminating extra
IMPROVING TEACHING AT A DISTANCE
material. For the Faculty and its members it may result in a reduction of takeup rates, course popularity and the increased time it takes to generate overlength material. For the student, the impact of facing overlength material results in a variety of reactions. Often survival strategies are employed which may not be the most approprate for effective learning or, faced by cumulative demands, students merely drop out of the course. The potential link between overlength courses and high student dropout rate was illustrated during the consideration of a Foundation Course offered, by Extension study, by USP.’ One of the authors examined the materials and study demands of this course in particular because of the unusually high student attrition rates and severe criticisms of the course levelled by various USP staff members. The investigation revealed that in 1986, 242 students registered for the course. There were seven official withdrawals but 55% failed to submit regular assignments and to sit the examinationthey dropped out of the course. Of the 107 who sat the examination only 38 passed; 15% of those who started the course. The course was described as requiring a modest amount of study time. Indeed, the figure of six hours per week was suggested. Estimates of student study-time were made for constituent elements in each weekly unit - 1 hour per week for ‘required’ tutorials, 2 hours per week for each weekly assignment. Estimates for textual materials were based on a ‘rule of thumb’ developed within the British Open University (Open University, 1985). The student study time (study time loot reading time) for textual material regarded as ‘easy’ was calculated at a rate of 100 words per minute (wpm), moderately difficult material at 70 wpm and difficult material at 40 wpm. From the discussion of readability above it is likely that much of the material would be regarded as, at least, ‘moderately difficult’ if not ‘difficult’ (an assumption confirmed by an inspection of the textual material provided in the course.) If the study time estimates are calculated for the first half of the course, with textual material assessed as of ‘moderate difficulty’ and then again as ‘difficult’, the implications for student study time can be seen in Fig. 1. In the first half of the course only two units corresponded to a workload of about 10 hours per week (both in excess of the workload
269
30
20 2 B 10,
0
1
2
3
4 UNIT
5
6
7
KEY
q Assignment
Rdings
e.ioderate)
Tutorial
lemi@sjudgedtobeof
‘moderate’ difficulty1
Fig. 1. Estimates of student workload.
“: UNIT
Fig. 2. Submission of weekly assignments.
specified in the course introduction) with other units of 20-30 hours in length. Reliable measures of the actual time of student drop out are notoriously difficult to obtain. However, if the submission of weekly assignments is taken as an indicator of students’ continuing study then the record of the submission of weekly assignments for marking in Suva for the first half of the course denotes a depressing trend (see Fig. 2). At the
F. G. LOCKWOOD
270
half way point in the course only 14 students were submitting regular assignments. The identification of excessive workload and of large fluctuations in the weekly demands of a course is possible and could contribute to a reduction in student withdrawal and drop out rates. The report to Extension Services by Lockwood at the end of his short investigation of student workloads was sent to all course writers, developers and Heads of Departments. His major findings were referred to academic meetings at the University. Appropriate action to extend his investigation using his methodology is being taken by the Course Development officers, and should result in more realistic workloads in new courses and when existing courses are revised. CONCLUDING
COMMENTS
The evaluation of the Extension Studies courses within USP is a continuous process with students’ comments sought during and at the end of each course. Part-time tutors and script markers are also asked to provide evaluatory information. Focussing on individual elements of distance education for occasional intensive investigation within an institution can provide most valuable and salutary data and the justification for urgent action to improve both courses and systems. The three investigations described above are an attempt to improve teaching at a distance within the University of the South Pacific. In due course it is hoped that clear evidence will
et al.
be available to confirm that the steps taken to improve teaching have been worthwhile.
NOTE 1. The investigation also revealed the role played by regular, weekly assignments with excessively long turn round times, requirements to attend tutorials and the need to purchase expensive texts.
REFERENCES Elley, W. B. (1977) A Close Look at the Cloze Test. NZ Council for Educational Research. Elley, W. B. (1984) Exploring the readability difficulties of second language learners in Fiji. In Reading in a Foreign Language (edited by Alderson, J. C. and Urquhart, H.). Longman, London. Gilmore, A. and Wagner, G. (1985) The Readability of Trade Examinations. NZ Council for Educational Research. Harrison, C. (1980) Readability in the Classroom. Cambridge University Press, Cambridge. Livingston, K. T. (1980) Studying by Extension. University of the South Pacific. Lockwood, F. G. (1986) Preparing students for distance learning. In Open Learning, pp. 44-45. Open University, Milton Keynes. Open University (1985) Making Self-Instructional Material for Adults: the Distant Teacher’s Handbook. The Open University Press, Milton Keynes. Roberts, D. W. (1984) An analysis of distance education patterns at R.C.A.E. Unpublished Graduate Diploma Project. Tuimaleali’ifano, E. (1985) A readability survey of USP Extension Course materials. Extension Services, the University of the South Pacific (mimeo). University of the South Pacific (1986) Extension Services Handbook 1987. University of the South Pacific, Suva.