- Email: [email protected]
Formative assessment workshops: Feedback sessions for large classes
65 0307-4412(95)00001-1
Formative Assessment Workshops: Sessions for Large Classes
Feedback
MARK CARROLL
Biochemistry Department Queen Mary & Westfield College London E1 4NS, UK Introduction The problems associated with providing meaningful educational feedback to students in large classes have been widely recognised (see, for example, ref 1). We have encountered these same issues over the past 4 years with our first-year intake of nearly 300 medical, dental and biomedical science students. 2 Although our students are academically very able, they nevertheless show heterogeneity within the cohort: for example, they may or may not have studied biology in their final two years at high school; they may have taken a year out between high school and university; or they may be mature students with a degree in a non-science subject. We provide them with a programme of two tutorials each week, in which problems with lectures, practicals, clinical demonstrations or 'self-directed learning' exercises may be discussed. However, a group size of 19 or 20 students per tutorial is a long way from the ideal of individual tuition! In order to overcome some of the educational problems posed by our large classes, we have instituted a programme of remedial/revision sessions, which we call 'formative assessment workshops'. These have some of the characteristics of the computer-based support offered at Leeds University.1
Formative Assessment Workshops The name we use for our revision sessions has three informative components: (1) Formative The purpose of the sessions is to provide each student with some feedback on his or her understanding, as the teaching proceeds. Although the structure of the workshops incorporates questions which could in principle generate a summative mark for each student (as at Leeds), we do not think this is appropriate in the first few weeks of the first-year course. At this early stage, students are concerned to know if they are coping with the intellectual level of the medical/dental course, and with its associated workload. They do take a summative in-course assessment at the end of the first module (in week 7), for which we want them to be adequately prepared. The workshops are not compulsory, but attendance is nearly always 100% - - one can only assume that at this early stage of the course they are still keen! (2) Assessment It is a well known fact that students are motivated to a large extent by the assessment procedures. In our examinations we use a mixture of multiple-choice questions (MCQs, of the T R U E / F A L S E format, with negative marking applied) and short-answer questions
BIOCHEMICAL EDUCATION 23(2) 1995
(SAQs, each typically of 10 minutes duration and with several parts). Students need to become familiar with the style of the questions they will encounter in future assessments and hence we base the workshops largely on MCQs taken from previous examination papers and with known characteristics (eg facility indices, discrimination ratios). Even though their mark will not "count" towards an assessment, students take the workshops seriously. (3) Workshops Typically, the entire first-year class of nearly 300 students assembles in the Teaching Laboratory, together with two tutors and a technician. The topics for the revision sessions encompass a group of related lectures. For example, in the first module (Molecules, Cells and Tissues), there is one workshop per week covering the following topics over a 5-week period: biomolecules, enzymes, molecular biology, intercellular communication, and the autonomic nervous system. Each session is remedial only, and is based on the learning objectives for the relevant taught components (largely lectures); no new information is introduced. At the end of the workshops, the tutors make themselves available to answer queries from students. Thus, feedback is provided at intervals of no more than one week.
Content and Delivery of the Workshops Our multi-disciplinary teaching laboratory of 700m 2 is equipped with an amplified sound system linked to a radio microphone, two rows each of ten TV monitors, with links to two CCTV cameras (one for black/white, one for colour), and a link to a microcomputer. Students sit at moveable tables, about 15 students per TV monitor (Figure 1). They know in advance the time and the topic of each workshop, and so they can prepare in advance if they wish. Having two tutors per class has advantages: the change of voice and tempo helps to maintain students' interest, and it also takes away some of the pressure associated with running these large classes. A technician familiar with the use of audio-visual aids is required to display the images on the TV monitors at the appropriate time (Figure 2). Typically each workshop starts with a brief introduction from one tutor as to its purpose and content (for example, a review of Molecular Biology). He or she will then launch into one section of that topic (for example, DNA and Replication), by summarising in a few sentences the main principles and concepts involved (already presented in detail in the preceding lectures). Where possible, we try to illustrate the summary with an appropriate image on the T V screens (for example, a suitable figure taken from a textbook). Then a MCQ is displayed on the monitors, such as: QI In a typical DNA m o l e c u l e . . . A the two strands are parallel to each other. B the base sequences of the two strands are identical. C the total number of (adenine + thymine) bases
66
Figure 1 The multi-disciplinary teaching laboratory in the Faculty of Basic Medical Sciences at QMWC. Dr Carroll is shown conducting a formative assessment workshop
Figure 2 The media technician at the control console. The two C C T V cameras are on the right. The micro-computer (not shown) is on his left
equals the total number of (guanine + cytosine) bases. D phosphodiester bonds link the 2'- and 5'carbons of adjacent deoxyribose residues. E purines are always hydrogen-bonded to pyrimidines.
The quality of our C C T V cameras limits the detail which can be displayed on the TV monitors; colour images in particular need to be simple and bold. A typed M C Q can be easily read by the students at a distance if it fills the T V screen and is prepared in large bold type. However, better resolution can be achieved if the M C Q is imported from a computer program. Our system allows us to take M C Q s and SAQs written in Word and display them directly on the T V monitors without the need for an intervening camera: there is thus no loss of resolution. This particular program has the facility to highlight sections of text with colour, or to use a variety of fonts, so as to enhance the visual presentation of the question. At the end of the workshop, students compute their total scores. They are told what is an 'acceptable' score, so they can get some idea of their own level of understanding of the topic under review. They have the opportunity to clarify any areas of confusion surrounding the questions just attempted by consulting the tutors at the end of the session. Alternatively, they can take up any points of misunderstanding in one of the regular tutorials. Sometimes we bring along to the workshops appropriate visual material: for example, we have two large-scale models of the D N A double helix, where students can readily see anti-parallel strands, base-pairing, grooves, etc.
Students have two minutes in which to answer the question. The tutor then runs through each part of the question giving the correct answers, with a brief explanation as to why each F A L S E answer was incorrect (Fig 1). Students mark their own answers and compute their score (with negative marking applied). The second tutor then takes over, and moves on to the next section (say, R N A and Transcription). He or she may feel that it is appropriate to use a question of the shortanswer format; see for example Table 1. H e r e , students have to complete the table; the time allowance (typically, 3 minutes) depends on the complexity of the question. The tutor runs through the answers, together with any necessary brief explanation. Then the first tutor takes over, focusing on a third section (say, Protein Synthesis). The second tutor might round off the session with a section on (say) Control of Gene
Expression. Table 1 Short-answer format question Q2 Compare and contrast the processes of replication (DNA (a) (b) (c) (d) (e) (f) (g) (h) (i)
synthesis) and transcription (RNA synthesis): Replication Transcription Where in cell? Nucleotides used? Direction of synthesis? Primer needed? (type?) Template needed? (type?) Semi-conservative? Stage of cell cycle'? Proof-reading mechanism? Enzyme needed? (names?)
B I O C H E M I C A L E D U C A T I O N 23(2) 1995
Conclusions In a one-hour session only a limited amount of ground can be covered, but the use of MCQs (of the T R U E / F A L S E type) maximises the number of aspects which the students have to address. The workshops themselves are highly structured, and close liaison is needed between the two tutors concerned (particularly as regards content), as well as each of them and the media technician (particularly as regards presentation). A 'dry-run' of one section is advisable to get some feel for the timing and delivery of the various components. In the post-Module questionnaire, students regularly express their satisfaction with the formative assessment
67 workshops in Molecules, Cells and Tissues: typically, 95% agree that the workshops are a useful form of revision; 90% are satisfied with the format of the sessions; 85% would like more such workshops. The sessions achieve the dual aims of familiarising the students with the type of questions used in our assessment procedure, and of providing individual students with some feedback on their understanding of the previous week's work. With such large classes, it is not an ideal arrangement, but we nevertheless feel that formative assessment workshops are educationally worthwhile, as well as being cost-effective. Of course, the onus is on weak students to remedy their own deficiencies, but that too is part of the experience of becoming an independent learner.
Acknowledgement I wish to thank John Puddefoot for expert technical assistance.
References ~Booth, A G (1994) Biochem Educ 22, 199-200 -'Carroll. M (1994) Biochem Educ 22, 75-78
0307-4412(94)00143-X
An Alternative Method for Teaching the Complement System MARIA LI]CIA SCROFERNEKER, FLAVIA HELENA C SORIA and AKEMI SCARLET SHIBA Department of Microbiology Federal University of Rio Grande do Sul, 90046-900 Porto Alegre Brazil
a kind of logic sequence game. The DMD method has been well accepted and produces a spontaneous engagement of the students in the study of metabolism far beyond the time required for the subject. Guided by the work of Siqueira et al a game was prepared using a set of cardboard pieces, in an attempt to find new methods to improve and facilitate learning process of the complement system.
Material and Methods The Complement Game consists of a board showing the sequential structure of the classical, alternative and common effector pathways (Fig 1). On this board 16 squares are drawn which are to be filled by the cardboard pieces (Fig 2). The game was applied by trained demonstrators, to 74 fourth semester students on the Medical Immunology Course at the Federal University of Rio Grande do Sul. The students worked in groups of 5 to 7; the pieces were randomly distributed, and the theoretical contents reviewed at the start. The game is initiated by the student who had the first piece beginning the classical pathway, placing it in the appropriate square on the board and explaining the corresponding complement fraction. The second space was then filled in by the next in sequence, and so on. The same was done with the Alternative, and Common Effector pathways. After the game, a standard questionnaire was applied, with closedend questions.
Results To 66 (89.1%) of the 74 students who participated, the game method was clear, not so to the other 6 (8.1%), CLASSIC PATHWAY
ALTERNATIVE PATHWAY
Introduction The complement system is a set of plasma proteins and glycoproteins involved in inflammatory responses and ultimately the lysis of cells, bacteria, viruses and pathogenic fungi. This system has two activation pathways: the so-called Classical and Alternative routes, along with a final pathway, the Common Effector (generation of the membrane attack complex). 1-3 When teaching the complement system to medical students it is difficult to make them understand the pathway and fragment sequences involved in the protein cascade. The subject is somewhat arid and requires a lot of attention and abstract thought. The students are only too easily distracted from the matter at hand! Siqueira et al 4 created a sort of game, the "Metabologram", to teach the Krebs Cycle. In this game there are pieces containing elements of the Cycle for the purpose of recognizing the metabolic pathways and their interconnections. The results obtained showed that this method was helpful to students and produced a high level of learning. Siqueira et al 5 also created a new method called 'Dynamic Metabolic Diagrams' (DMD) to stimulate the students individually or in groups to participate in
B I O C H E M I C A L E D U C A T I O N 23(2) 1995
Ca 2~
P-..~ C2b
-- "
Cga
-
C3
COMMON EFFECTOR
Figure 1 Board for the sequential structure of the classic alternative and common effector pathways the complement system
Formative Assessment Workshops: Sessions for Large Classes
Feedback
MARK CARROLL
Biochemistry Department Queen Mary & Westfield College London E1 4NS, UK Introduction The problems associated with providing meaningful educational feedback to students in large classes have been widely recognised (see, for example, ref 1). We have encountered these same issues over the past 4 years with our first-year intake of nearly 300 medical, dental and biomedical science students. 2 Although our students are academically very able, they nevertheless show heterogeneity within the cohort: for example, they may or may not have studied biology in their final two years at high school; they may have taken a year out between high school and university; or they may be mature students with a degree in a non-science subject. We provide them with a programme of two tutorials each week, in which problems with lectures, practicals, clinical demonstrations or 'self-directed learning' exercises may be discussed. However, a group size of 19 or 20 students per tutorial is a long way from the ideal of individual tuition! In order to overcome some of the educational problems posed by our large classes, we have instituted a programme of remedial/revision sessions, which we call 'formative assessment workshops'. These have some of the characteristics of the computer-based support offered at Leeds University.1
Formative Assessment Workshops The name we use for our revision sessions has three informative components: (1) Formative The purpose of the sessions is to provide each student with some feedback on his or her understanding, as the teaching proceeds. Although the structure of the workshops incorporates questions which could in principle generate a summative mark for each student (as at Leeds), we do not think this is appropriate in the first few weeks of the first-year course. At this early stage, students are concerned to know if they are coping with the intellectual level of the medical/dental course, and with its associated workload. They do take a summative in-course assessment at the end of the first module (in week 7), for which we want them to be adequately prepared. The workshops are not compulsory, but attendance is nearly always 100% - - one can only assume that at this early stage of the course they are still keen! (2) Assessment It is a well known fact that students are motivated to a large extent by the assessment procedures. In our examinations we use a mixture of multiple-choice questions (MCQs, of the T R U E / F A L S E format, with negative marking applied) and short-answer questions
BIOCHEMICAL EDUCATION 23(2) 1995
(SAQs, each typically of 10 minutes duration and with several parts). Students need to become familiar with the style of the questions they will encounter in future assessments and hence we base the workshops largely on MCQs taken from previous examination papers and with known characteristics (eg facility indices, discrimination ratios). Even though their mark will not "count" towards an assessment, students take the workshops seriously. (3) Workshops Typically, the entire first-year class of nearly 300 students assembles in the Teaching Laboratory, together with two tutors and a technician. The topics for the revision sessions encompass a group of related lectures. For example, in the first module (Molecules, Cells and Tissues), there is one workshop per week covering the following topics over a 5-week period: biomolecules, enzymes, molecular biology, intercellular communication, and the autonomic nervous system. Each session is remedial only, and is based on the learning objectives for the relevant taught components (largely lectures); no new information is introduced. At the end of the workshops, the tutors make themselves available to answer queries from students. Thus, feedback is provided at intervals of no more than one week.
Content and Delivery of the Workshops Our multi-disciplinary teaching laboratory of 700m 2 is equipped with an amplified sound system linked to a radio microphone, two rows each of ten TV monitors, with links to two CCTV cameras (one for black/white, one for colour), and a link to a microcomputer. Students sit at moveable tables, about 15 students per TV monitor (Figure 1). They know in advance the time and the topic of each workshop, and so they can prepare in advance if they wish. Having two tutors per class has advantages: the change of voice and tempo helps to maintain students' interest, and it also takes away some of the pressure associated with running these large classes. A technician familiar with the use of audio-visual aids is required to display the images on the TV monitors at the appropriate time (Figure 2). Typically each workshop starts with a brief introduction from one tutor as to its purpose and content (for example, a review of Molecular Biology). He or she will then launch into one section of that topic (for example, DNA and Replication), by summarising in a few sentences the main principles and concepts involved (already presented in detail in the preceding lectures). Where possible, we try to illustrate the summary with an appropriate image on the T V screens (for example, a suitable figure taken from a textbook). Then a MCQ is displayed on the monitors, such as: QI In a typical DNA m o l e c u l e . . . A the two strands are parallel to each other. B the base sequences of the two strands are identical. C the total number of (adenine + thymine) bases
66
Figure 1 The multi-disciplinary teaching laboratory in the Faculty of Basic Medical Sciences at QMWC. Dr Carroll is shown conducting a formative assessment workshop
Figure 2 The media technician at the control console. The two C C T V cameras are on the right. The micro-computer (not shown) is on his left
equals the total number of (guanine + cytosine) bases. D phosphodiester bonds link the 2'- and 5'carbons of adjacent deoxyribose residues. E purines are always hydrogen-bonded to pyrimidines.
The quality of our C C T V cameras limits the detail which can be displayed on the TV monitors; colour images in particular need to be simple and bold. A typed M C Q can be easily read by the students at a distance if it fills the T V screen and is prepared in large bold type. However, better resolution can be achieved if the M C Q is imported from a computer program. Our system allows us to take M C Q s and SAQs written in Word and display them directly on the T V monitors without the need for an intervening camera: there is thus no loss of resolution. This particular program has the facility to highlight sections of text with colour, or to use a variety of fonts, so as to enhance the visual presentation of the question. At the end of the workshop, students compute their total scores. They are told what is an 'acceptable' score, so they can get some idea of their own level of understanding of the topic under review. They have the opportunity to clarify any areas of confusion surrounding the questions just attempted by consulting the tutors at the end of the session. Alternatively, they can take up any points of misunderstanding in one of the regular tutorials. Sometimes we bring along to the workshops appropriate visual material: for example, we have two large-scale models of the D N A double helix, where students can readily see anti-parallel strands, base-pairing, grooves, etc.
Students have two minutes in which to answer the question. The tutor then runs through each part of the question giving the correct answers, with a brief explanation as to why each F A L S E answer was incorrect (Fig 1). Students mark their own answers and compute their score (with negative marking applied). The second tutor then takes over, and moves on to the next section (say, R N A and Transcription). He or she may feel that it is appropriate to use a question of the shortanswer format; see for example Table 1. H e r e , students have to complete the table; the time allowance (typically, 3 minutes) depends on the complexity of the question. The tutor runs through the answers, together with any necessary brief explanation. Then the first tutor takes over, focusing on a third section (say, Protein Synthesis). The second tutor might round off the session with a section on (say) Control of Gene
Expression. Table 1 Short-answer format question Q2 Compare and contrast the processes of replication (DNA (a) (b) (c) (d) (e) (f) (g) (h) (i)
synthesis) and transcription (RNA synthesis): Replication Transcription Where in cell? Nucleotides used? Direction of synthesis? Primer needed? (type?) Template needed? (type?) Semi-conservative? Stage of cell cycle'? Proof-reading mechanism? Enzyme needed? (names?)
B I O C H E M I C A L E D U C A T I O N 23(2) 1995
Conclusions In a one-hour session only a limited amount of ground can be covered, but the use of MCQs (of the T R U E / F A L S E type) maximises the number of aspects which the students have to address. The workshops themselves are highly structured, and close liaison is needed between the two tutors concerned (particularly as regards content), as well as each of them and the media technician (particularly as regards presentation). A 'dry-run' of one section is advisable to get some feel for the timing and delivery of the various components. In the post-Module questionnaire, students regularly express their satisfaction with the formative assessment
67 workshops in Molecules, Cells and Tissues: typically, 95% agree that the workshops are a useful form of revision; 90% are satisfied with the format of the sessions; 85% would like more such workshops. The sessions achieve the dual aims of familiarising the students with the type of questions used in our assessment procedure, and of providing individual students with some feedback on their understanding of the previous week's work. With such large classes, it is not an ideal arrangement, but we nevertheless feel that formative assessment workshops are educationally worthwhile, as well as being cost-effective. Of course, the onus is on weak students to remedy their own deficiencies, but that too is part of the experience of becoming an independent learner.
Acknowledgement I wish to thank John Puddefoot for expert technical assistance.
References ~Booth, A G (1994) Biochem Educ 22, 199-200 -'Carroll. M (1994) Biochem Educ 22, 75-78
0307-4412(94)00143-X
An Alternative Method for Teaching the Complement System MARIA LI]CIA SCROFERNEKER, FLAVIA HELENA C SORIA and AKEMI SCARLET SHIBA Department of Microbiology Federal University of Rio Grande do Sul, 90046-900 Porto Alegre Brazil
a kind of logic sequence game. The DMD method has been well accepted and produces a spontaneous engagement of the students in the study of metabolism far beyond the time required for the subject. Guided by the work of Siqueira et al a game was prepared using a set of cardboard pieces, in an attempt to find new methods to improve and facilitate learning process of the complement system.
Material and Methods The Complement Game consists of a board showing the sequential structure of the classical, alternative and common effector pathways (Fig 1). On this board 16 squares are drawn which are to be filled by the cardboard pieces (Fig 2). The game was applied by trained demonstrators, to 74 fourth semester students on the Medical Immunology Course at the Federal University of Rio Grande do Sul. The students worked in groups of 5 to 7; the pieces were randomly distributed, and the theoretical contents reviewed at the start. The game is initiated by the student who had the first piece beginning the classical pathway, placing it in the appropriate square on the board and explaining the corresponding complement fraction. The second space was then filled in by the next in sequence, and so on. The same was done with the Alternative, and Common Effector pathways. After the game, a standard questionnaire was applied, with closedend questions.
Results To 66 (89.1%) of the 74 students who participated, the game method was clear, not so to the other 6 (8.1%), CLASSIC PATHWAY
ALTERNATIVE PATHWAY
Introduction The complement system is a set of plasma proteins and glycoproteins involved in inflammatory responses and ultimately the lysis of cells, bacteria, viruses and pathogenic fungi. This system has two activation pathways: the so-called Classical and Alternative routes, along with a final pathway, the Common Effector (generation of the membrane attack complex). 1-3 When teaching the complement system to medical students it is difficult to make them understand the pathway and fragment sequences involved in the protein cascade. The subject is somewhat arid and requires a lot of attention and abstract thought. The students are only too easily distracted from the matter at hand! Siqueira et al 4 created a sort of game, the "Metabologram", to teach the Krebs Cycle. In this game there are pieces containing elements of the Cycle for the purpose of recognizing the metabolic pathways and their interconnections. The results obtained showed that this method was helpful to students and produced a high level of learning. Siqueira et al 5 also created a new method called 'Dynamic Metabolic Diagrams' (DMD) to stimulate the students individually or in groups to participate in
B I O C H E M I C A L E D U C A T I O N 23(2) 1995
Ca 2~
P-..~ C2b
-- "
Cga
-
C3
COMMON EFFECTOR
Figure 1 Board for the sequential structure of the classic alternative and common effector pathways the complement system