- Email: [email protected]
Formative evaluation of a neuroanatomy course
STUDIES IN EDUCATIONAL EVALUATION
Volume 2, No. 2
Summer 1976
F O R M A T I V E E V A L U A T I O N OF A N E U R O A N A T O M Y COURSE P A T R I C K R. S T E R R E T T
a n d J O H N H. L I T T L E F I E L D
Texas Tech University School of Medicine University of Texas Health Science Center at San Antonio INTRODUCTION This is the initial r e p o r t of an ongoing instructional d e v e l o p m e n t p r o g r a m designed to clarify the a b s t r a c t n a t u r e of basic n e u r o a n a t o m i c a l science a n d demo n s t r a t e clinical a p p l i c a t i o n s of neuroa n a t o m y to freshmen medical students. The p r o g r a m was conceived as an approach to solving t h r e e f u n d a m e n t a l problems in teaching n e u r o a n a t o m y : the d e v e l o p m e n t of useful instructional objectives, a p p r o p r i a t e use of instructional media, a n d t h e i n t r o d u c t i o n of clinical experiences r e l e v a n t to basic neuroanatomical concepts. I n addition, we hoped to instill a positive s t u d e n t a t t i t u d e t o w a r d Neuroscience as a p p l i e d to clinical p r o b l e m solving.
INSTRUCTIONAL DESIGN The e m p h a s i s in designing the Neuroa n a t o m y course was c o n c e n t r a t e d in three k e y areas - i n s t r u c t i o n a l objectives, supplementary instructional media and clinical d e m o n s t r a t i o n s . F i r s t , a set of i n s t r u c t i o n a l o b j e c t i v e s representing each m a j o r topic area of n e u r o a n a t o m y was established a n d lectures were developed. Second, a v a r i e t y of i n s t r u c t i o n a l materials were developed to a u g m e n t the lecture series. This included v i d e o t a p e s with scripts (self-paced), slide carousels, prosected materials, models, self-teaching packages, dissection exercises a n d problemsolving exercises (case studies) presented b y t h e students. F i n a l l y , neurological e x a m i n a t i o n s (physical diagnosis)
and p a t i e n t p r e s e n t a t i o n s were used in several i n s t r u c t i o n a l units. F o r example, p a t i e n t s manifesting spinal cord deficits were p r e s e n t e d i m m e d i a t e l y following the lecture p r e s e n t a t i o n s in n e u r o a n a t o m y of the spinal cord. These p a t i e n t prese n t a t i o n s were disigned to incorporate active s t u d e n t p a r t i c i p a t i o n in meaningful a p p l i c a t i o n s of n e u r o a n a t o m i c a l concepts. Research in complex learning indicates t h a t instruction is most effective when s t u d e n t s p l a y an active role (1) and " m e a n i n g f u l n e s s " of the concepts is e m p h a s i z e d (2).
EVALUATION METHODS AND RESULTS A f o r m a t i v e e v a l u a t i o n (3) of the curriculum was p e r f o r m e d in t e r m s of cognitive a c h i e v e m e n t a n d s t u d e n t ratings of the course. To measure cognitive achievement, the final e x a m i n a t i o n test items were developed from the instructional objectives a n d s t u d e n t performance was a n a l y z e d in t e r m s of the mean percentage of correct responses to the items associated with each objective. To assess s t u d e n t viewpoints, a course e v a l u a t i o n form was used to rate each instructional topic area in t e r m s of lecture quality, effectiveness of instructional media, a n d a d e q u a c y of the p a t i e n t presentations. T a b l e 1 provides a s u m m a r y analysis of the f o r m a t i v e e v a l u a t i o n d a t a organized b y instructional topic areas. Cognitive 147
148
ABSTRACTS
Table I. Summary Student Performance Topic Areas
Cognitive Performance
Affective Reactions *
% Slide ~/ Items Correct Responses Lectures Carousels
Videotapes
Patient Presentation
1. Microanatomy
6
89.7
3.59
3.a5
3.a2
N.A.
2. Cranial Nerves
59
87.2
4.10
N.A.
N.A.
N.A.
8
84.8
3.76
N.A.
3.05
3.29
60
80.4
3.67
N.A.
3.40
4.16
131
79.7
4.09
3.73
3.50
4.30
43
79.3
3.93
N.A.
3.68
3.59
104
79.1
3.48
N.A.
3.15
3.13
7
62.6
2.96
3.79
3.33
3.80
3. Neurophysiology 4. Clinical Correlations 5. Spinal Cord 6. Motor and Sensory 7. Special Systems 8. Cerebral Hemispheres
* Expressed as mean scores on a 1-5 scale where 1 was defined as "inadequate", 3 was defined as "Adequate", and 5 was defined as "Outstanding".
p e r f o r m a n c e has b e e n s u m m a r i z e d b y g r o u p i n g t h e o b j e c t i v e s for e a c h i n s t r u c t i o n a l t o p i c a r e a a n d c o m p u t i n g an o v e r a l l m e a n p e r c e n t a g e of c o r r e c t t e s t i t e m responses. S t u d e n t r a t i n g s were m a d e on a 1 to 5 scale w h e r e 1 was defined as " I n a d e q u a t e " , 3 was defined as " A d e q u a t e " , a n d 5 was defined as "Outstanding". The Affeetive Reactions e n t r i e s in T a b l e I are m e a n scores b a s e d u p o n t h e scale a b o v e . I n r e v i e w i n g T a b l e I, n o t e t h a t t h e t o p i c a r e a C e r e b r a l H e m i s p h e r e s was marked by poor cognitive performance a n d low r a t i n g s of l e c t u r e q u a l i t y . I n a d d i t i o n t h e v i d e o t a p e s d e s i g n e d to augment Neurophysiology and Special S y s t e m s also r e c e i v e d r e l a t i v e l y low ratings. DISCUSSION Student cognitive performance data a n d a f f e c t i v e r e a c t i o n s as s u m m a r i z e d
in T a b l e I h a v e p r o v i d e d t h e basis for a f o r m a t i v e e v a l u a t i o n of this n e u r o a n a t o m y c u r r i c u l u m . P r i o r to t e a c h i n g a n e w g r o u p of" s t u d e n t s , t h e lectures r e l a t e d to C e r e b r a l H e m i s p h e r e s will he s u b s t a n t i a l l y revised. I n a d d i t i o n , t h e v i d e o t a p e s a n d p a t i e n t p r e s e n t a t i o n s rel a t e d to N e u r o p h y s i o l o g y a n d Special S y s t e m s will be c a r e f u l l y r e v i e w e d for possible revision. F i n a l l y , s t u d e n t comm e n t s i n d i c a t e d t h a t t h e P a t i e n t Prese n t a t i o n s did s e r v e to e m p h a s i z e t h e clinical a p p l i c a t i o n s of basic n e u r o a n a t o m i c a l concepts. To s u m m a r i z e , t h e results of" a first cycle f o r m a t i v e e v a l u a t i o n h a v e p r o v i d e d v a l u a b l e f e e d b a c k for s y s t e m a t i c a l l y imp r o v i n g a new" N e u r o a n a t o m y curricul u m . T h e d a t a c o l l e c t e d w e r e n o t sophist i c a t e d ; h o w e v e r , based u p o n t h e results we are c o n f i d e n t t h a t t h e c u r r i c u l u m can be s u b s t a n t i a l l y i m p r o v e d p r i o r to t e a c h i n g it a second time.
ABSTRACTS
149
REFERENCES ROTHKOPF, E. Z. The Concept of Mathemagenic Activities. Review of Educational Research, 1970, 40, 32~336. JOHNSON, R. E. Meaning in Complex Learning. Review of Educational Research, 1970, 45, 32~336. SCRIVEN, M. The Methodology of Evaluation. A E R A Monograph Series on Curriculum Evaluation. 1. Chicago: Rand McNally, 1967.
Volume 2, No. 2
Summer 1976
F O R M A T I V E E V A L U A T I O N OF A N E U R O A N A T O M Y COURSE P A T R I C K R. S T E R R E T T
a n d J O H N H. L I T T L E F I E L D
Texas Tech University School of Medicine University of Texas Health Science Center at San Antonio INTRODUCTION This is the initial r e p o r t of an ongoing instructional d e v e l o p m e n t p r o g r a m designed to clarify the a b s t r a c t n a t u r e of basic n e u r o a n a t o m i c a l science a n d demo n s t r a t e clinical a p p l i c a t i o n s of neuroa n a t o m y to freshmen medical students. The p r o g r a m was conceived as an approach to solving t h r e e f u n d a m e n t a l problems in teaching n e u r o a n a t o m y : the d e v e l o p m e n t of useful instructional objectives, a p p r o p r i a t e use of instructional media, a n d t h e i n t r o d u c t i o n of clinical experiences r e l e v a n t to basic neuroanatomical concepts. I n addition, we hoped to instill a positive s t u d e n t a t t i t u d e t o w a r d Neuroscience as a p p l i e d to clinical p r o b l e m solving.
INSTRUCTIONAL DESIGN The e m p h a s i s in designing the Neuroa n a t o m y course was c o n c e n t r a t e d in three k e y areas - i n s t r u c t i o n a l objectives, supplementary instructional media and clinical d e m o n s t r a t i o n s . F i r s t , a set of i n s t r u c t i o n a l o b j e c t i v e s representing each m a j o r topic area of n e u r o a n a t o m y was established a n d lectures were developed. Second, a v a r i e t y of i n s t r u c t i o n a l materials were developed to a u g m e n t the lecture series. This included v i d e o t a p e s with scripts (self-paced), slide carousels, prosected materials, models, self-teaching packages, dissection exercises a n d problemsolving exercises (case studies) presented b y t h e students. F i n a l l y , neurological e x a m i n a t i o n s (physical diagnosis)
and p a t i e n t p r e s e n t a t i o n s were used in several i n s t r u c t i o n a l units. F o r example, p a t i e n t s manifesting spinal cord deficits were p r e s e n t e d i m m e d i a t e l y following the lecture p r e s e n t a t i o n s in n e u r o a n a t o m y of the spinal cord. These p a t i e n t prese n t a t i o n s were disigned to incorporate active s t u d e n t p a r t i c i p a t i o n in meaningful a p p l i c a t i o n s of n e u r o a n a t o m i c a l concepts. Research in complex learning indicates t h a t instruction is most effective when s t u d e n t s p l a y an active role (1) and " m e a n i n g f u l n e s s " of the concepts is e m p h a s i z e d (2).
EVALUATION METHODS AND RESULTS A f o r m a t i v e e v a l u a t i o n (3) of the curriculum was p e r f o r m e d in t e r m s of cognitive a c h i e v e m e n t a n d s t u d e n t ratings of the course. To measure cognitive achievement, the final e x a m i n a t i o n test items were developed from the instructional objectives a n d s t u d e n t performance was a n a l y z e d in t e r m s of the mean percentage of correct responses to the items associated with each objective. To assess s t u d e n t viewpoints, a course e v a l u a t i o n form was used to rate each instructional topic area in t e r m s of lecture quality, effectiveness of instructional media, a n d a d e q u a c y of the p a t i e n t presentations. T a b l e 1 provides a s u m m a r y analysis of the f o r m a t i v e e v a l u a t i o n d a t a organized b y instructional topic areas. Cognitive 147
148
ABSTRACTS
Table I. Summary Student Performance Topic Areas
Cognitive Performance
Affective Reactions *
% Slide ~/ Items Correct Responses Lectures Carousels
Videotapes
Patient Presentation
1. Microanatomy
6
89.7
3.59
3.a5
3.a2
N.A.
2. Cranial Nerves
59
87.2
4.10
N.A.
N.A.
N.A.
8
84.8
3.76
N.A.
3.05
3.29
60
80.4
3.67
N.A.
3.40
4.16
131
79.7
4.09
3.73
3.50
4.30
43
79.3
3.93
N.A.
3.68
3.59
104
79.1
3.48
N.A.
3.15
3.13
7
62.6
2.96
3.79
3.33
3.80
3. Neurophysiology 4. Clinical Correlations 5. Spinal Cord 6. Motor and Sensory 7. Special Systems 8. Cerebral Hemispheres
* Expressed as mean scores on a 1-5 scale where 1 was defined as "inadequate", 3 was defined as "Adequate", and 5 was defined as "Outstanding".
p e r f o r m a n c e has b e e n s u m m a r i z e d b y g r o u p i n g t h e o b j e c t i v e s for e a c h i n s t r u c t i o n a l t o p i c a r e a a n d c o m p u t i n g an o v e r a l l m e a n p e r c e n t a g e of c o r r e c t t e s t i t e m responses. S t u d e n t r a t i n g s were m a d e on a 1 to 5 scale w h e r e 1 was defined as " I n a d e q u a t e " , 3 was defined as " A d e q u a t e " , a n d 5 was defined as "Outstanding". The Affeetive Reactions e n t r i e s in T a b l e I are m e a n scores b a s e d u p o n t h e scale a b o v e . I n r e v i e w i n g T a b l e I, n o t e t h a t t h e t o p i c a r e a C e r e b r a l H e m i s p h e r e s was marked by poor cognitive performance a n d low r a t i n g s of l e c t u r e q u a l i t y . I n a d d i t i o n t h e v i d e o t a p e s d e s i g n e d to augment Neurophysiology and Special S y s t e m s also r e c e i v e d r e l a t i v e l y low ratings. DISCUSSION Student cognitive performance data a n d a f f e c t i v e r e a c t i o n s as s u m m a r i z e d
in T a b l e I h a v e p r o v i d e d t h e basis for a f o r m a t i v e e v a l u a t i o n of this n e u r o a n a t o m y c u r r i c u l u m . P r i o r to t e a c h i n g a n e w g r o u p of" s t u d e n t s , t h e lectures r e l a t e d to C e r e b r a l H e m i s p h e r e s will he s u b s t a n t i a l l y revised. I n a d d i t i o n , t h e v i d e o t a p e s a n d p a t i e n t p r e s e n t a t i o n s rel a t e d to N e u r o p h y s i o l o g y a n d Special S y s t e m s will be c a r e f u l l y r e v i e w e d for possible revision. F i n a l l y , s t u d e n t comm e n t s i n d i c a t e d t h a t t h e P a t i e n t Prese n t a t i o n s did s e r v e to e m p h a s i z e t h e clinical a p p l i c a t i o n s of basic n e u r o a n a t o m i c a l concepts. To s u m m a r i z e , t h e results of" a first cycle f o r m a t i v e e v a l u a t i o n h a v e p r o v i d e d v a l u a b l e f e e d b a c k for s y s t e m a t i c a l l y imp r o v i n g a new" N e u r o a n a t o m y curricul u m . T h e d a t a c o l l e c t e d w e r e n o t sophist i c a t e d ; h o w e v e r , based u p o n t h e results we are c o n f i d e n t t h a t t h e c u r r i c u l u m can be s u b s t a n t i a l l y i m p r o v e d p r i o r to t e a c h i n g it a second time.
ABSTRACTS
149
REFERENCES ROTHKOPF, E. Z. The Concept of Mathemagenic Activities. Review of Educational Research, 1970, 40, 32~336. JOHNSON, R. E. Meaning in Complex Learning. Review of Educational Research, 1970, 45, 32~336. SCRIVEN, M. The Methodology of Evaluation. A E R A Monograph Series on Curriculum Evaluation. 1. Chicago: Rand McNally, 1967.