- Email: [email protected]
Using email for teaching
Computers & Education 33 (1999) 15±25
www.elsevier.com/locate/compedu
Using email for teaching C.D. Smith*, H.E. Whiteley, S. Smith Department of Psychology, University of Central Lancashire, Corporation Street, Preston, Lancashire, PR1 2HE UK Received 6 February 1998; accepted 9 June 1999
Abstract An account is given of a three-year study of the use of email for teaching purposes within two courses forming part of a psychology degree, in order to compare course delivery via email with delivery via `traditional' lectures. The courses were a ®rst year `Psychology and IT' course, which was taught entirely via email and a second year `Cognitive Psychology' course, parts of which were taught by email and part by lectures, some of which were supported by emailed summaries. Although there were drawbacks to the use of email for teaching, the email method was felt to have been successful, as was con®rmed by generally positive student feedback. The examination performance of two successive student cohorts on an multiple-choice question (MCQ) paper was studied. In one cohort the poorest part-time students performed better on material taught by email than by lectures, while in the other cohort there was some evidence for lectures supported by emailed 1000-word summaries producing better performance. However, in this second cohort MCQ performance deriving from emailed lectures was worse than that deriving from other delivery formats. # 1999 Elsevier Science Ltd. All rights reserved. Keywords: Email; Psychology teaching; Learning styles
1. Introduction 1.1. Background This paper reports some measurable outcomes of using email to replace or supplement lectures in two psychology courses. It describes one course in which email replaced all lectures * Corresponding author. Tel.: +1772-893420, +1772-893436; fax: +1772-892925. E-mail address: [email protected] (C.D. Smith) 0360-1315/99/$ - see front matter # 1999 Elsevier Science Ltd. All rights reserved. PII: S 0 3 6 0 - 1 3 1 5 ( 9 9 ) 0 0 0 1 3 - 5
16
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
and face-to-face contact and another in which email replaced some lectures and supplemented others. The use of email is now widespread within higher education and, according to Gilbert (1996), ``course-related use of email is becoming the single most powerful force for integrating information technology into teaching and learning''. The general advantages it oers for teaching purposes have been outlined together with some disadvantages (Hu, 1994). Among the advantages identi®ed by Hu and others are: . . . .
improved and asynchronous communication group dynamics become more equal (Markus, 1994; Sproull & Kiesler, 1991) synchronous and fast transmission of information to multiple recipients facilitation of remote collaboration (Wild & Winniford, 1993)
while the disadvantages include . the need for students to be willing and able to operate the software . most information is text based . loss of non-verbal communication. The last two points generally also apply to books and to correspondence courses. While McDonnell and Achterberg (1997) report favourable reactions and high student performance on a course with an email component, Rudy (1996) considers there to be too few studies of this nature, even though the use of email for teaching is becoming more widespread. To some extent this is unsurprising, because of the ethical considerations which might arise from manipulating teaching media. However, it proved possible to conduct such a study in the Department of Psychology at the University of Central Lancashire in the 1994±1995, 1995± 1996 and 1996±1997 academic years, which have allowed us to study some eects on performance of using email for teaching large groups of students. What follows is an account of the use of email for teaching purposes in two courses, which was based on such aspects of the `Integrative Evaluation' approach (Draper, Brown, Henderson & McAteer, 1996) as pre- and post-course questionnaires and examination performance. One course was a ®rst year `Psychology and Information Technology' course, which was taught entirely by email in the 1994±1995 academic year. The second was a secondyear `Cognitive Psychology' course parts of which were taught by email in 1995±1996 and 1996±1997. Because the study extended over three academic years, it is described in three parts after an account of a preliminary investigation. 1.2. Using email for teaching From the viewpoint of the tutors involved in the courses email oered many speci®c potential advantages, namely: . there could be fewer or no timetabled sessions; . delivery and `reading' of emailed course material was recorded (`reading' means only that the recipient had opened the email message) . submission of coursework could be direct to the tutor with delivery and reading recorded; . coursework needed to involve little or no paper;
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
17
. once prepared, course material could be delivered in seconds and reused in subsequent years Ð amended as necessary. There was, therefore, the potential to produce an overall reduction in teaching time and administration. From the students' viewpoint also email oered advantages, namely: . . . . . . . .
there was no attendance requirement students could work in their own time; students could work at their own pace; course material was received well in advance of its scheduled use. all course material could be stored for future use; access to the tutor could be gained via email; delivery (and non-delivery) and `reading' of coursework was recorded rapid, secure feedback could be given by email;
1.3. Preliminary investigation As part of the evaluation of a third year `Health Promotion' course, Smith (1994) gathered data anonymously which indicated generally positive attitudes towards the introduction of IT to supplement course delivery. These attitudes were independent of the students' self-rated computer literacy and competence. The 45 students reported that they were likely to use a computer to access the main text of lectures, but were relatively unlikely not to attend lectures, if they did so. They were likely to use computer-based learning, if it were available on every PC in the university. Only three students wanted the `Health Promotion' course to be lectures only, although none wanted it to be delivered entirely via computers. The overall results of this investigation encouraged the authors to think that the use of email to deliver course material would be regarded positively by most students and would not necessarily disadvantage any, not least because it has been found that user perceptions of a medium's usefulness will aect use (Golden, Beauclair & Sussman, 1992). Accordingly, it was decided to introduce an element of email delivery into the `Psychology and IT' and `Cognitive Psychology' courses and to study reactions to it and its eects on assessment. 2. Study 1: 1994±1995 2.1. Background `Psychology and IT' was an optional part of a B.Sc. Psychology course. Prior to it students took an `Information Technology for Psychology and Related Disciplines' course, which was a general, skills-based IT course. Seventy-two students took `Psychology and IT' in 1994±1995. The course consisted of four lectures and eight hands-on sessions. It was assessed by the best six of seven pieces of coursework. The aim of the course was to focus on those aspects of IT which are speci®cally related to psychology. The content of the course thus consisted of a
18
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
range of topics where psychology and IT make contact, such as methodology, AI, CAL and neural networks. These topics were covered by a mixture of `lectures' and `hands-on' sessions, in which students worked in small groups. The email delivery of `Psychology and IT' diered from conventional delivery in the following ways: . . . .
no classes took place at timetabled hours; there was no face-to-face contact between tutor and students; all course materials, `lectures' and details of hands-on sessions, were delivered by email; most course materials were delivered before the start date for the course; all materials were delivered well before their scheduled use; . no paper was used for course materials; . no paper was used for the handing in of coursework (although it was found to be easier to print copies of some coursework in order to be able to return it with written comments on it).
2.2. Results and discussion `Psychology and IT' was assessed by the marks for the best six of seven pieces of coursework, which was delivered to the tutor by email. With 72 students there were thus maximally 504 pieces of coursework and minimally 432 to meet the course requirements Ð although fewer than six pieces of coursework could still result in a pass. In the event 417 pieces of coursework were actually submitted, with the eight students who failed the course responsible for 32 of the 87 missing pieces of coursework. Thus, most failures resulted from insucient rather than poor quality coursework. Every student submitted two or more pieces of coursework, indicating that no student lacked the ability to manage the email system. A request for feedback produced 10 responses, all of which were positive. These ®ndings are taken as demonstrating that email can be used successfully to deliver course materials, receive coursework and give feedback. They suggest that a reasonable level of satisfaction can also be achieved. For a course which has no absolute requirement for a tutor to be present and which is not based on F2F contact, email would thus seem to be an ecient and eective medium for course delivery.
3. Study 2: 1995±1996 3.1. Using email within `cognitive psychology' One hundred and twenty-nine students took the course in 1995±1996, consisting of 82 fulltime and 30 part-time B.Sc. Psychology students, seven Combined Honours students and 10 Neuroscience students. `Cognitive Psychology' consists of 24 lectures, three seminars and one laboratory class. In 1995±1996, of the 24 lectures:
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
19
. 15 were given in the normal manner; . three were given normally and each was accompanied by a full transcript delivered via email; . three were given normally and each was accompanied by a summary delivered via email; the full transcripts of the remaining three lectures were prepared specially for email delivery only and were so delivered: these three lectures were carefully chosen to be particularly suitable for email delivery: i.e. two were on aspects of language and one on memory Ð topics which were mostly text based. In addition, email was used for general communication of course-related matters. The course is assessed by a laboratory report, a 2-from-6-question essay exam and a multiple-choice question (MCQ) paper. The latter consists of 50 questions (two per lecture, plus two general questions), which enabled sucient data to be generated to compare the performance of students on each method of lecture delivery. 3.2. Results and discussion This study was not designed to be a formal evaluation of the eectiveness of using email for teaching purposes, but, nevertheless, feedback was obtained which allows estimation of student reactions to email and of the general eectiveness of the medium in a teaching situation. Whilst it was hoped that email could be demonstrated to be a superior method of delivery Ð because students had a full set of lecture notes to refer to whenever and how they wished Ð the aims of introducing it into the teaching programme would have been satis®ed by ®nding that it did not lead to consistently negative feedback from students or to reduced performance. The ®ndings from the study consist of comments from formal Module Evaluation Questionnaires (MEQs) and MCQ examination results for Cognitive Psychology'. At the end of the course students were asked to complete a standard general MEQ, to which tutors are allowed to add module-speci®c questions. Two items were added to the MEQ about the use of email in the course. Respondents were asked to indicate their level of satisfaction. The two items and the results for them from 50 respondents are shown in Table 1 below. Responses to the ®rst item indicate greater overall satisfaction (66%) than dissatisfaction (34%) with the inclusion of email lectures. However, Table 1 also shows that 20% were `very dissatis®ed' with the inclusion of email lectures. When asked what the best feature of the module was and which was liked least about the module, email was the most nominated feature in each category with 21 and 38% respectively. This apparent polarization of views may re¯ect dierent attitudes towards email between the full- and part-time students. Although there was no certain way of separating the anonymous Table 1 MEQ statements and percentages of responses in each rating category to each question (1995±1996 ®gures) from 50 respondents
The inclusion of email lectures The use of email for general communication
1 (Very satis®ed)
2 (Satis®ed)
3 (Dissatis®ed)
4 (Very dissatis®ed)
40 34
26 28
14 20
20 14
20
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
responses to the MEQ for full- and part-time students, there were strong indications from general comments of a clear divide between them, with full-time students greatly favouring email and part-time students largely dissatis®ed with it. The full-time students tended to use email as a resource which they could access at any time, whereas the part-time students were concerned to print any material, because most of them had no access to email at times other than their limited attendance. The problems of printer congestion which arose from a large number of part-time students seeking to print a large amount of material in a comparatively short time seem likely to account for many of the `very dissatis®ed' responses and re¯ect dierent uses to which emailed material is put. The second item in Table 1 relates to the use of email for general communication of modulerelated information. This use of email was also generally regarded positively (62%) and seems perhaps to have attracted fewer `very dissatis®ed' responses (14%), because the material was secondary in importance to that of the lectures. Only 96% of the respondents to the ®rst item responded to the second item. The overall levels of satisfaction expressed in responses to the two questions about email which were added to the MEQs are rather disappointing, especially the ®gures of 20 and 14% in the `very dissatis®ed' category. The examination results allowed comparisons to be made between: . performance on areas of the course taught without email, with a short or a long email summary of lectures and with email lectures only; these comparisons could be made for MCQ scores, essay question scores and a combination of these; . the performance of all students and of full-time and part-time students, and subgroups thereof. Comparisons between the various student groups and for the whole group in terms of performance on either the essay or MCQ parts of the examination or both for questions deriving from `normal' lectures, email lectures or email-supplemented lectures produced no ®ndings favouring one method of delivery over another Ð with one exception. This concerned the performance of the high and low scorers among the part-time students on the MCQ questions. Statistical signi®cance was obtained when the bottom 10% of scorers were compared on MCQ scores of 54 and 31% for email and `normal' lectures respectively (U 9:5; n 7; p<0:05). Thus, the poorest part-time students produced signi®cantly better performance on those MCQ questions which derived from email lectures than on those questions which did not derive from email lectures. The explanation for this result is unclear, but by having lectures emailed to them students gain control over the use of the material and avoid the pitfalls of the process of notetaking in lectures.
4. Study 3: 1996±1997 One hundred and ®fteen students took the `Cognitive Psychology' course in 1996±1997, consisting of 92 full-time B.Sc. Psychology students, 13 Combined Honours students and 10 Neuroscience students. The course did not run in the evening in 1996±1997 and so was not
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
21
taken by any part-time students. The methods, formats and pattern of the delivery of the course lectures were identical to that of the previous year. 4.1. Results The MEQ scores to the same questions were obtained from 80 respondents and are shown in Table 2 below. The most notable dierence between 1995±1996 and 1996±1997 ®gures is that the general level of satisfaction had risen from 66 to 82.5% Ð perhaps because in 1996±1997 there were no part-time students on the course, for whom the emailed lectures seemed to cause particular problems. The ®gure for the question on the general communication use of email also increased, but only to 66% from 64%. The `best'/`worst feature' item was not included in 1996±1997 MEQ, but to a free-response question 27 students said that they did not like email lectures, of whom nine said explicitly that the email lectures were too long and that brief summaries to support. lectures would be preferable. Four students liked email lectures, especially the summaries, while 17 students expressed a preference for traditional lectures. Thus, while the increase in the overall levels of satisfaction is gratifying Ð with a reduction from 20 to 6.3% in the `very dissatis®ed' category Ð there was still a sizeable amount of opposition to email lectures. As in 1995±1996 the scores on the MCQ component of the examination paper were divisible into the three formats used to deliver the lectures. The results are shown in Table 3 below for the three methods of delivery both overall and separately for each of the two tutors (HEW and CDS). A two-way mixed measures ANOVA for overall scores found a signi®cant eect of delivery format (F 13:68; df 2,228; p<0:001), while Tukey tests found dierences between each of the overall scores for the three types of lecture (all p<0:01). A signi®cant dierence was found between the tutors, with scores on HEW's questions being higher than scores on CDS's questions (F 79:69: df 1,114; p<0:001). In addition, a signi®cant interaction was found between the tutor and the delivery format (F 5:01; df 2,228; p<0:01) with Tukey tests ®nding that the eects of delivery format were con®ned to HEW. Thus the results showed that for HEW there were more correct answers to questions derived from lectures with email summaries than for the two other types of lecture and there were fewer correct answers from email lectures than to `normal' lectures, while for CDS there were no dierences between any of the delivery formats. Dierences between the tutors and between lecture type prompted a comparison of the word length of email lectures and email summaries for the two tutors. The results are shown in Table 4. Table 2 MEQ statements and percentages of responses in each rating category to each question (1996±1997 ®gures) 1 (Very satis®ed) The inclusion of email lectures The use of email for general communication
47.5 26.7
2 (Satis®ed) 35 37.3
3 (Dissatis®ed)
4 (Very dissatis®ed)
11.3 21.3
6.3 14.7
22
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
Table 3 Overall and tutor MCQ scores in percentages for dierent formats of lecture Delivery format
Overall
HEW
CDS
Lectures Lectures + email summary Email lectures
55.34 61.40 50.58
62 71 55
49 51 47
Thus, although the two tutors' email lectures were almost identical in length, HEW's summaries were more than twice as long as those of CDS, suggesting that there is a critical length for informativeness of lecture summaries and that summaries of that critical length can amount to a suciently reinforcing form of material to produce higher performance on MCQ components of assessment. The results are, however, confounded by both author and lecture content. A sample of high and low scorers was selected consisting of the 37 highest scorers (with scores from 60 to 78%) and 37 low scorers (with scores from 40 to 50%), who had obtained or exceeded the pass mark of 40%. The lowest scorers from the overall sample were not selected, because some had very low scores across all delivery formats and at chance or near chance levels on the MCQ paper. The data from the chosen sample are shown in Table 5. A two-way ANOVA again found a main eect of lecture type (F 8:60; df 2,144; p<0:001) and also found an expected dierence between the high and low scorers (F 306:56; df 1,72; p<0:001). A marginally signi®cant interaction was found between lecture type and high/low scoring (F 2:49; df 2,2). Tukey tests found no dierences between lecture type for high scorers, while for low scorers lectures with email summaries produced higher scores than either `normal' or email lectures, which did not themselves dier. Although marginal, this interaction and the Tukey tests suggest that high scorers are unaected by lecture type, while low scorers may bene®t from having lectures supported by email summaries. 5. General discussion This was not an experimental study and the data are limited in many respects. The results are consistent in that across the three studies they showed that students generally managed to access and use the emailed material and were generally satis®ed with that method of delivery. Table 4 Average lengths in words of email lectures and email summaries Tutor
Lectures
HEW CDS
4386 4349
Summaries 1047 451
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
23
Table 5 MCQ scores in percentages for 37 high and 37 low scorers for dierent formats of lecture Delivery format
High scorers
Low scorers
Dierence
Lectures Lectures + email summary Email lectures
68 69 64
45 55 41
23 14 23
In both the 1995±1996 and 1996±1997 studies, however, worrying levels of dissatisfaction were expressed. The performance of these two cohorts was mixed, in that some weaker students seemed to gain from using emailed lectures and the use of emailed summaries seemed to enhance performance sometimes, while in the 1996±1997 cohort the use of emailed lectures produced a signi®cantly worse performance than other methods of delivery. Thus, the studies ®ts the ®ndings of McDonnell and Achterberg (1997) and of meta-analyses of the eects of computer-assisted learning, which are that the eects of CAL are either neutral or bene®cial (Kulik & Kulik, 1991). It should be noted that few of the students were naive email users and all were oered instruction in the use of the email system - and all the students in Study 1 submitted at least 2 pieces of coursework by email. Almost all the students in each study would have reached at least the third stage (understanding and application) of the six stages, which university students pass through when learning to use email, according to Russell (1995). The ®nding that the poorest students may actually bene®t from improved performance on the assessment of material delivered by email Ð a result which has been found before (e.g. Haddon et al. (1995), who used multimedia material) Ð is believed to arise largely from the opportunity email provides for these students to record material and work at their own (slower) pace. It thus seems reasonable to assume that email has the potential to be particularly advantageous not only for the poorer students, but also for some of the increasing number of students who are diagnosed as having special needs. For the better students, however, email oers no particular opportunity to perform better, nor does multimedia, according to Haddon et al. (1995), who assumed that such students were already performing at or near ceiling. From the 1996±1997 study, however, there was con¯icting evidence of the eects of email delivery on student performance. On the one hand, for one tutor no dierences were found between methods of delivery, while for another tutor there were signi®cant gains from emailing summaries of normally delivered lectures and signi®cantly lower performance from emailing lectures rather than delivering them normally. The signi®cant eects again seemed to be con®ned to the poorer students. The length of the summary seems to be important for the level of detail it provides appropriate for the MCQ questions derived from the lecture. One aspect of the general use of IT in teaching was not addressed by this study Ð the need for face-to-face contact. In the case of `Psychology and IT' the content did not really lend itself to discussion. Email was not replacing seminars or tutorials, but only lectures and group sessions with PCs. Although group discussions of a sort could be set up via email, a group of individuals would need to be on-line together and each member would need a distribution list consisting of all the other participants. Even with a chairperson to control the discussion and
24
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
some rules about responding, the discussion would lose the spontaneity and non-verbal components of normal group interaction Ð and the exchanges would be much slower. The ineectiveness of email as a discussive teaching medium, as opposed to an informationtransmitting teaching medium, has been outlined elsewhere (Kiesler, Siegel & McGuire, 1984; Smith, 1988). In the case of `Cognitive Psychology', of course, the reduction in F2F contact was minimal, involving only the nominal F2F contact occurring in lectures to large groups of students. Each of the studies involved a reduction in teaching time and administration through the use of email. In Study 1 none of the four lectures and eight hands-on sessions took place Ð and almost no paper in the form of handouts and coursework changed hands. In Studies 2 and 3 the reductions in lectures and administration were much lower, but were still not inconsiderable given the large number of students involved Ð and were roughly proportionate to the extent to which email was used. Email oers scope for, but does not guarantee, increased eciency. In summary, the three studies have established email as a viable alternative means of course delivery. There appear to be possible advantages for poorer part-time students and logistical disadvantages for part-time students generally, which showed up somewhat in the level of dissatisfaction felt by students towards email delivery, but did not account for all of it. The ®nding of improved student performance on lectures accompanied by emailed summaries from one tutor and the ®nding of worse performance on emailed lectures from the same tutor suggest more subtle eects of emailed delivery are possible. Summary length appears to matter, while lecture length does not appear to matter, but more research is needed to untangle the mechanisms at work here.
References Draper, S. W., Brown, M. I., Henderson, F. P., & McAteer, E. (1996). Integrative evaluation: an emerging role for classroom studies of CAL. Computers and Education, 26(1), 17±32. Gilbert, S. AAHESGIT: new thread email, Elect. Comm., 1996, personal communication. Golden, P. A., Beauclair, R., & Sussman, L. (1992). Factors aecting electronic mail use. Computers in Human Behavior, 8, 297±311. Haddon, K. A., Smith, C. D., Bratttan, D., & Smith, E. H. (1995). Can learning via multimedia bene®t weaker students? Active Learning, 3, 22±27. Hu C. Using email in psychology classes, 1994, personal communication. Kiesler, S., Siegel, J., & McGuire, T. W. (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39(10), 1123±1134. Kulik, C. L. C., & Kulik, J. A. (1991). Eectiveness of computer based instruction: an updated analysis. Computers in Human Behaviour, 7(1-2), 17±94. Markus, M. L. (1994). Finding a happy medium: explaining the negative eects of electronic communication on social life at work. ACM Transactions on Information Systems, 12(2), 119±149. McDonnell, E., & Achterberg, C. (1997). Development and delivery of a nutrition education course with and electronic mail component. Journal of Nutrition Education, 29(4), 210±214. Rudy, I. A. (1996). A critical review of research on electronic mail. European Journal of Information Systems, 4(4), 198±213. Russell, A. L. (1995). Stages in learning new technology: naive adult email users. Computers and Education, 25(4), 173±178.
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
25
Smith, C. D. (1988). The educational value of computer-mediated communications. Media in Education and Development, 12, 169±171. Smith, S. Attitudes to methods of delivery of `Health Promotion'. Department of Psychology Internal Report, University of Central Lancashire, 1994. Sproull, L., & Kiesler, S. (1991). Connections: new ways of working in the networked organization. Cambridge, MA: MIT Press. Wild, R. H., & Winniford, M. (1993). Remote collaboration among students using electronic mail. Computers and Education, 21(3), 193±203.
www.elsevier.com/locate/compedu
Using email for teaching C.D. Smith*, H.E. Whiteley, S. Smith Department of Psychology, University of Central Lancashire, Corporation Street, Preston, Lancashire, PR1 2HE UK Received 6 February 1998; accepted 9 June 1999
Abstract An account is given of a three-year study of the use of email for teaching purposes within two courses forming part of a psychology degree, in order to compare course delivery via email with delivery via `traditional' lectures. The courses were a ®rst year `Psychology and IT' course, which was taught entirely via email and a second year `Cognitive Psychology' course, parts of which were taught by email and part by lectures, some of which were supported by emailed summaries. Although there were drawbacks to the use of email for teaching, the email method was felt to have been successful, as was con®rmed by generally positive student feedback. The examination performance of two successive student cohorts on an multiple-choice question (MCQ) paper was studied. In one cohort the poorest part-time students performed better on material taught by email than by lectures, while in the other cohort there was some evidence for lectures supported by emailed 1000-word summaries producing better performance. However, in this second cohort MCQ performance deriving from emailed lectures was worse than that deriving from other delivery formats. # 1999 Elsevier Science Ltd. All rights reserved. Keywords: Email; Psychology teaching; Learning styles
1. Introduction 1.1. Background This paper reports some measurable outcomes of using email to replace or supplement lectures in two psychology courses. It describes one course in which email replaced all lectures * Corresponding author. Tel.: +1772-893420, +1772-893436; fax: +1772-892925. E-mail address: [email protected] (C.D. Smith) 0360-1315/99/$ - see front matter # 1999 Elsevier Science Ltd. All rights reserved. PII: S 0 3 6 0 - 1 3 1 5 ( 9 9 ) 0 0 0 1 3 - 5
16
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
and face-to-face contact and another in which email replaced some lectures and supplemented others. The use of email is now widespread within higher education and, according to Gilbert (1996), ``course-related use of email is becoming the single most powerful force for integrating information technology into teaching and learning''. The general advantages it oers for teaching purposes have been outlined together with some disadvantages (Hu, 1994). Among the advantages identi®ed by Hu and others are: . . . .
improved and asynchronous communication group dynamics become more equal (Markus, 1994; Sproull & Kiesler, 1991) synchronous and fast transmission of information to multiple recipients facilitation of remote collaboration (Wild & Winniford, 1993)
while the disadvantages include . the need for students to be willing and able to operate the software . most information is text based . loss of non-verbal communication. The last two points generally also apply to books and to correspondence courses. While McDonnell and Achterberg (1997) report favourable reactions and high student performance on a course with an email component, Rudy (1996) considers there to be too few studies of this nature, even though the use of email for teaching is becoming more widespread. To some extent this is unsurprising, because of the ethical considerations which might arise from manipulating teaching media. However, it proved possible to conduct such a study in the Department of Psychology at the University of Central Lancashire in the 1994±1995, 1995± 1996 and 1996±1997 academic years, which have allowed us to study some eects on performance of using email for teaching large groups of students. What follows is an account of the use of email for teaching purposes in two courses, which was based on such aspects of the `Integrative Evaluation' approach (Draper, Brown, Henderson & McAteer, 1996) as pre- and post-course questionnaires and examination performance. One course was a ®rst year `Psychology and Information Technology' course, which was taught entirely by email in the 1994±1995 academic year. The second was a secondyear `Cognitive Psychology' course parts of which were taught by email in 1995±1996 and 1996±1997. Because the study extended over three academic years, it is described in three parts after an account of a preliminary investigation. 1.2. Using email for teaching From the viewpoint of the tutors involved in the courses email oered many speci®c potential advantages, namely: . there could be fewer or no timetabled sessions; . delivery and `reading' of emailed course material was recorded (`reading' means only that the recipient had opened the email message) . submission of coursework could be direct to the tutor with delivery and reading recorded; . coursework needed to involve little or no paper;
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
17
. once prepared, course material could be delivered in seconds and reused in subsequent years Ð amended as necessary. There was, therefore, the potential to produce an overall reduction in teaching time and administration. From the students' viewpoint also email oered advantages, namely: . . . . . . . .
there was no attendance requirement students could work in their own time; students could work at their own pace; course material was received well in advance of its scheduled use. all course material could be stored for future use; access to the tutor could be gained via email; delivery (and non-delivery) and `reading' of coursework was recorded rapid, secure feedback could be given by email;
1.3. Preliminary investigation As part of the evaluation of a third year `Health Promotion' course, Smith (1994) gathered data anonymously which indicated generally positive attitudes towards the introduction of IT to supplement course delivery. These attitudes were independent of the students' self-rated computer literacy and competence. The 45 students reported that they were likely to use a computer to access the main text of lectures, but were relatively unlikely not to attend lectures, if they did so. They were likely to use computer-based learning, if it were available on every PC in the university. Only three students wanted the `Health Promotion' course to be lectures only, although none wanted it to be delivered entirely via computers. The overall results of this investigation encouraged the authors to think that the use of email to deliver course material would be regarded positively by most students and would not necessarily disadvantage any, not least because it has been found that user perceptions of a medium's usefulness will aect use (Golden, Beauclair & Sussman, 1992). Accordingly, it was decided to introduce an element of email delivery into the `Psychology and IT' and `Cognitive Psychology' courses and to study reactions to it and its eects on assessment. 2. Study 1: 1994±1995 2.1. Background `Psychology and IT' was an optional part of a B.Sc. Psychology course. Prior to it students took an `Information Technology for Psychology and Related Disciplines' course, which was a general, skills-based IT course. Seventy-two students took `Psychology and IT' in 1994±1995. The course consisted of four lectures and eight hands-on sessions. It was assessed by the best six of seven pieces of coursework. The aim of the course was to focus on those aspects of IT which are speci®cally related to psychology. The content of the course thus consisted of a
18
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
range of topics where psychology and IT make contact, such as methodology, AI, CAL and neural networks. These topics were covered by a mixture of `lectures' and `hands-on' sessions, in which students worked in small groups. The email delivery of `Psychology and IT' diered from conventional delivery in the following ways: . . . .
no classes took place at timetabled hours; there was no face-to-face contact between tutor and students; all course materials, `lectures' and details of hands-on sessions, were delivered by email; most course materials were delivered before the start date for the course; all materials were delivered well before their scheduled use; . no paper was used for course materials; . no paper was used for the handing in of coursework (although it was found to be easier to print copies of some coursework in order to be able to return it with written comments on it).
2.2. Results and discussion `Psychology and IT' was assessed by the marks for the best six of seven pieces of coursework, which was delivered to the tutor by email. With 72 students there were thus maximally 504 pieces of coursework and minimally 432 to meet the course requirements Ð although fewer than six pieces of coursework could still result in a pass. In the event 417 pieces of coursework were actually submitted, with the eight students who failed the course responsible for 32 of the 87 missing pieces of coursework. Thus, most failures resulted from insucient rather than poor quality coursework. Every student submitted two or more pieces of coursework, indicating that no student lacked the ability to manage the email system. A request for feedback produced 10 responses, all of which were positive. These ®ndings are taken as demonstrating that email can be used successfully to deliver course materials, receive coursework and give feedback. They suggest that a reasonable level of satisfaction can also be achieved. For a course which has no absolute requirement for a tutor to be present and which is not based on F2F contact, email would thus seem to be an ecient and eective medium for course delivery.
3. Study 2: 1995±1996 3.1. Using email within `cognitive psychology' One hundred and twenty-nine students took the course in 1995±1996, consisting of 82 fulltime and 30 part-time B.Sc. Psychology students, seven Combined Honours students and 10 Neuroscience students. `Cognitive Psychology' consists of 24 lectures, three seminars and one laboratory class. In 1995±1996, of the 24 lectures:
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
19
. 15 were given in the normal manner; . three were given normally and each was accompanied by a full transcript delivered via email; . three were given normally and each was accompanied by a summary delivered via email; the full transcripts of the remaining three lectures were prepared specially for email delivery only and were so delivered: these three lectures were carefully chosen to be particularly suitable for email delivery: i.e. two were on aspects of language and one on memory Ð topics which were mostly text based. In addition, email was used for general communication of course-related matters. The course is assessed by a laboratory report, a 2-from-6-question essay exam and a multiple-choice question (MCQ) paper. The latter consists of 50 questions (two per lecture, plus two general questions), which enabled sucient data to be generated to compare the performance of students on each method of lecture delivery. 3.2. Results and discussion This study was not designed to be a formal evaluation of the eectiveness of using email for teaching purposes, but, nevertheless, feedback was obtained which allows estimation of student reactions to email and of the general eectiveness of the medium in a teaching situation. Whilst it was hoped that email could be demonstrated to be a superior method of delivery Ð because students had a full set of lecture notes to refer to whenever and how they wished Ð the aims of introducing it into the teaching programme would have been satis®ed by ®nding that it did not lead to consistently negative feedback from students or to reduced performance. The ®ndings from the study consist of comments from formal Module Evaluation Questionnaires (MEQs) and MCQ examination results for Cognitive Psychology'. At the end of the course students were asked to complete a standard general MEQ, to which tutors are allowed to add module-speci®c questions. Two items were added to the MEQ about the use of email in the course. Respondents were asked to indicate their level of satisfaction. The two items and the results for them from 50 respondents are shown in Table 1 below. Responses to the ®rst item indicate greater overall satisfaction (66%) than dissatisfaction (34%) with the inclusion of email lectures. However, Table 1 also shows that 20% were `very dissatis®ed' with the inclusion of email lectures. When asked what the best feature of the module was and which was liked least about the module, email was the most nominated feature in each category with 21 and 38% respectively. This apparent polarization of views may re¯ect dierent attitudes towards email between the full- and part-time students. Although there was no certain way of separating the anonymous Table 1 MEQ statements and percentages of responses in each rating category to each question (1995±1996 ®gures) from 50 respondents
The inclusion of email lectures The use of email for general communication
1 (Very satis®ed)
2 (Satis®ed)
3 (Dissatis®ed)
4 (Very dissatis®ed)
40 34
26 28
14 20
20 14
20
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
responses to the MEQ for full- and part-time students, there were strong indications from general comments of a clear divide between them, with full-time students greatly favouring email and part-time students largely dissatis®ed with it. The full-time students tended to use email as a resource which they could access at any time, whereas the part-time students were concerned to print any material, because most of them had no access to email at times other than their limited attendance. The problems of printer congestion which arose from a large number of part-time students seeking to print a large amount of material in a comparatively short time seem likely to account for many of the `very dissatis®ed' responses and re¯ect dierent uses to which emailed material is put. The second item in Table 1 relates to the use of email for general communication of modulerelated information. This use of email was also generally regarded positively (62%) and seems perhaps to have attracted fewer `very dissatis®ed' responses (14%), because the material was secondary in importance to that of the lectures. Only 96% of the respondents to the ®rst item responded to the second item. The overall levels of satisfaction expressed in responses to the two questions about email which were added to the MEQs are rather disappointing, especially the ®gures of 20 and 14% in the `very dissatis®ed' category. The examination results allowed comparisons to be made between: . performance on areas of the course taught without email, with a short or a long email summary of lectures and with email lectures only; these comparisons could be made for MCQ scores, essay question scores and a combination of these; . the performance of all students and of full-time and part-time students, and subgroups thereof. Comparisons between the various student groups and for the whole group in terms of performance on either the essay or MCQ parts of the examination or both for questions deriving from `normal' lectures, email lectures or email-supplemented lectures produced no ®ndings favouring one method of delivery over another Ð with one exception. This concerned the performance of the high and low scorers among the part-time students on the MCQ questions. Statistical signi®cance was obtained when the bottom 10% of scorers were compared on MCQ scores of 54 and 31% for email and `normal' lectures respectively (U 9:5; n 7; p<0:05). Thus, the poorest part-time students produced signi®cantly better performance on those MCQ questions which derived from email lectures than on those questions which did not derive from email lectures. The explanation for this result is unclear, but by having lectures emailed to them students gain control over the use of the material and avoid the pitfalls of the process of notetaking in lectures.
4. Study 3: 1996±1997 One hundred and ®fteen students took the `Cognitive Psychology' course in 1996±1997, consisting of 92 full-time B.Sc. Psychology students, 13 Combined Honours students and 10 Neuroscience students. The course did not run in the evening in 1996±1997 and so was not
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
21
taken by any part-time students. The methods, formats and pattern of the delivery of the course lectures were identical to that of the previous year. 4.1. Results The MEQ scores to the same questions were obtained from 80 respondents and are shown in Table 2 below. The most notable dierence between 1995±1996 and 1996±1997 ®gures is that the general level of satisfaction had risen from 66 to 82.5% Ð perhaps because in 1996±1997 there were no part-time students on the course, for whom the emailed lectures seemed to cause particular problems. The ®gure for the question on the general communication use of email also increased, but only to 66% from 64%. The `best'/`worst feature' item was not included in 1996±1997 MEQ, but to a free-response question 27 students said that they did not like email lectures, of whom nine said explicitly that the email lectures were too long and that brief summaries to support. lectures would be preferable. Four students liked email lectures, especially the summaries, while 17 students expressed a preference for traditional lectures. Thus, while the increase in the overall levels of satisfaction is gratifying Ð with a reduction from 20 to 6.3% in the `very dissatis®ed' category Ð there was still a sizeable amount of opposition to email lectures. As in 1995±1996 the scores on the MCQ component of the examination paper were divisible into the three formats used to deliver the lectures. The results are shown in Table 3 below for the three methods of delivery both overall and separately for each of the two tutors (HEW and CDS). A two-way mixed measures ANOVA for overall scores found a signi®cant eect of delivery format (F 13:68; df 2,228; p<0:001), while Tukey tests found dierences between each of the overall scores for the three types of lecture (all p<0:01). A signi®cant dierence was found between the tutors, with scores on HEW's questions being higher than scores on CDS's questions (F 79:69: df 1,114; p<0:001). In addition, a signi®cant interaction was found between the tutor and the delivery format (F 5:01; df 2,228; p<0:01) with Tukey tests ®nding that the eects of delivery format were con®ned to HEW. Thus the results showed that for HEW there were more correct answers to questions derived from lectures with email summaries than for the two other types of lecture and there were fewer correct answers from email lectures than to `normal' lectures, while for CDS there were no dierences between any of the delivery formats. Dierences between the tutors and between lecture type prompted a comparison of the word length of email lectures and email summaries for the two tutors. The results are shown in Table 4. Table 2 MEQ statements and percentages of responses in each rating category to each question (1996±1997 ®gures) 1 (Very satis®ed) The inclusion of email lectures The use of email for general communication
47.5 26.7
2 (Satis®ed) 35 37.3
3 (Dissatis®ed)
4 (Very dissatis®ed)
11.3 21.3
6.3 14.7
22
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
Table 3 Overall and tutor MCQ scores in percentages for dierent formats of lecture Delivery format
Overall
HEW
CDS
Lectures Lectures + email summary Email lectures
55.34 61.40 50.58
62 71 55
49 51 47
Thus, although the two tutors' email lectures were almost identical in length, HEW's summaries were more than twice as long as those of CDS, suggesting that there is a critical length for informativeness of lecture summaries and that summaries of that critical length can amount to a suciently reinforcing form of material to produce higher performance on MCQ components of assessment. The results are, however, confounded by both author and lecture content. A sample of high and low scorers was selected consisting of the 37 highest scorers (with scores from 60 to 78%) and 37 low scorers (with scores from 40 to 50%), who had obtained or exceeded the pass mark of 40%. The lowest scorers from the overall sample were not selected, because some had very low scores across all delivery formats and at chance or near chance levels on the MCQ paper. The data from the chosen sample are shown in Table 5. A two-way ANOVA again found a main eect of lecture type (F 8:60; df 2,144; p<0:001) and also found an expected dierence between the high and low scorers (F 306:56; df 1,72; p<0:001). A marginally signi®cant interaction was found between lecture type and high/low scoring (F 2:49; df 2,2). Tukey tests found no dierences between lecture type for high scorers, while for low scorers lectures with email summaries produced higher scores than either `normal' or email lectures, which did not themselves dier. Although marginal, this interaction and the Tukey tests suggest that high scorers are unaected by lecture type, while low scorers may bene®t from having lectures supported by email summaries. 5. General discussion This was not an experimental study and the data are limited in many respects. The results are consistent in that across the three studies they showed that students generally managed to access and use the emailed material and were generally satis®ed with that method of delivery. Table 4 Average lengths in words of email lectures and email summaries Tutor
Lectures
HEW CDS
4386 4349
Summaries 1047 451
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
23
Table 5 MCQ scores in percentages for 37 high and 37 low scorers for dierent formats of lecture Delivery format
High scorers
Low scorers
Dierence
Lectures Lectures + email summary Email lectures
68 69 64
45 55 41
23 14 23
In both the 1995±1996 and 1996±1997 studies, however, worrying levels of dissatisfaction were expressed. The performance of these two cohorts was mixed, in that some weaker students seemed to gain from using emailed lectures and the use of emailed summaries seemed to enhance performance sometimes, while in the 1996±1997 cohort the use of emailed lectures produced a signi®cantly worse performance than other methods of delivery. Thus, the studies ®ts the ®ndings of McDonnell and Achterberg (1997) and of meta-analyses of the eects of computer-assisted learning, which are that the eects of CAL are either neutral or bene®cial (Kulik & Kulik, 1991). It should be noted that few of the students were naive email users and all were oered instruction in the use of the email system - and all the students in Study 1 submitted at least 2 pieces of coursework by email. Almost all the students in each study would have reached at least the third stage (understanding and application) of the six stages, which university students pass through when learning to use email, according to Russell (1995). The ®nding that the poorest students may actually bene®t from improved performance on the assessment of material delivered by email Ð a result which has been found before (e.g. Haddon et al. (1995), who used multimedia material) Ð is believed to arise largely from the opportunity email provides for these students to record material and work at their own (slower) pace. It thus seems reasonable to assume that email has the potential to be particularly advantageous not only for the poorer students, but also for some of the increasing number of students who are diagnosed as having special needs. For the better students, however, email oers no particular opportunity to perform better, nor does multimedia, according to Haddon et al. (1995), who assumed that such students were already performing at or near ceiling. From the 1996±1997 study, however, there was con¯icting evidence of the eects of email delivery on student performance. On the one hand, for one tutor no dierences were found between methods of delivery, while for another tutor there were signi®cant gains from emailing summaries of normally delivered lectures and signi®cantly lower performance from emailing lectures rather than delivering them normally. The signi®cant eects again seemed to be con®ned to the poorer students. The length of the summary seems to be important for the level of detail it provides appropriate for the MCQ questions derived from the lecture. One aspect of the general use of IT in teaching was not addressed by this study Ð the need for face-to-face contact. In the case of `Psychology and IT' the content did not really lend itself to discussion. Email was not replacing seminars or tutorials, but only lectures and group sessions with PCs. Although group discussions of a sort could be set up via email, a group of individuals would need to be on-line together and each member would need a distribution list consisting of all the other participants. Even with a chairperson to control the discussion and
24
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
some rules about responding, the discussion would lose the spontaneity and non-verbal components of normal group interaction Ð and the exchanges would be much slower. The ineectiveness of email as a discussive teaching medium, as opposed to an informationtransmitting teaching medium, has been outlined elsewhere (Kiesler, Siegel & McGuire, 1984; Smith, 1988). In the case of `Cognitive Psychology', of course, the reduction in F2F contact was minimal, involving only the nominal F2F contact occurring in lectures to large groups of students. Each of the studies involved a reduction in teaching time and administration through the use of email. In Study 1 none of the four lectures and eight hands-on sessions took place Ð and almost no paper in the form of handouts and coursework changed hands. In Studies 2 and 3 the reductions in lectures and administration were much lower, but were still not inconsiderable given the large number of students involved Ð and were roughly proportionate to the extent to which email was used. Email oers scope for, but does not guarantee, increased eciency. In summary, the three studies have established email as a viable alternative means of course delivery. There appear to be possible advantages for poorer part-time students and logistical disadvantages for part-time students generally, which showed up somewhat in the level of dissatisfaction felt by students towards email delivery, but did not account for all of it. The ®nding of improved student performance on lectures accompanied by emailed summaries from one tutor and the ®nding of worse performance on emailed lectures from the same tutor suggest more subtle eects of emailed delivery are possible. Summary length appears to matter, while lecture length does not appear to matter, but more research is needed to untangle the mechanisms at work here.
References Draper, S. W., Brown, M. I., Henderson, F. P., & McAteer, E. (1996). Integrative evaluation: an emerging role for classroom studies of CAL. Computers and Education, 26(1), 17±32. Gilbert, S. AAHESGIT: new thread email, Elect. Comm., 1996, personal communication. Golden, P. A., Beauclair, R., & Sussman, L. (1992). Factors aecting electronic mail use. Computers in Human Behavior, 8, 297±311. Haddon, K. A., Smith, C. D., Bratttan, D., & Smith, E. H. (1995). Can learning via multimedia bene®t weaker students? Active Learning, 3, 22±27. Hu C. Using email in psychology classes, 1994, personal communication. Kiesler, S., Siegel, J., & McGuire, T. W. (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39(10), 1123±1134. Kulik, C. L. C., & Kulik, J. A. (1991). Eectiveness of computer based instruction: an updated analysis. Computers in Human Behaviour, 7(1-2), 17±94. Markus, M. L. (1994). Finding a happy medium: explaining the negative eects of electronic communication on social life at work. ACM Transactions on Information Systems, 12(2), 119±149. McDonnell, E., & Achterberg, C. (1997). Development and delivery of a nutrition education course with and electronic mail component. Journal of Nutrition Education, 29(4), 210±214. Rudy, I. A. (1996). A critical review of research on electronic mail. European Journal of Information Systems, 4(4), 198±213. Russell, A. L. (1995). Stages in learning new technology: naive adult email users. Computers and Education, 25(4), 173±178.
C.D. Smith et al. / Computers & Education 33 (1999) 15±25
25
Smith, C. D. (1988). The educational value of computer-mediated communications. Media in Education and Development, 12, 169±171. Smith, S. Attitudes to methods of delivery of `Health Promotion'. Department of Psychology Internal Report, University of Central Lancashire, 1994. Sproull, L., & Kiesler, S. (1991). Connections: new ways of working in the networked organization. Cambridge, MA: MIT Press. Wild, R. H., & Winniford, M. (1993). Remote collaboration among students using electronic mail. Computers and Education, 21(3), 193±203.