Student-controlled evaluation — a pilot study

Name Education T&y (1991) 11,220-224 0 Longman Group UK Ltd 1991

WORK Student-controlled

evaluation - a pilot study

Kader Parahoo

There is general agreement now that evaluation is an essential part of the education process. However there are methodological, political and ethical issues which still need to be addressed. In this paper a pilot study is described in which students carry out their own evaluation of parts of a curriculum using an approach which allowed them to use their own words to express their views on areas of concern to them.

data and report the findings is so long that the reasons, motivation and momentum for change

INTRODUCTION Evaluation is now recognised as an essential part of the education process. contention which remain evaluation

The main areas are the purpose

and the methodological

to the collection

of evaluation

of of

approaches

data. Although

many would argue that the main purpose of evaluation is to improve the curriculum, others would question how the data are used and to what purpose; and when politics and economics are the determining factors in decision making, humanistic considerations tend to take second place. While evaluation

may increase

teachers’

respect and may even help to identify

self

potential

areas of professional development, it may also be seen as a threat to professional autonomy, if not, in some cases, to one’s job. Evaluation is time consuming

and often the time taken to analyse

Kader Psrahoo RMN BA (Hans) PhD AdvDipEd Senior

Course Tutor, BSc (Hens) Nursing/FIG/U, Department of Nursing and Health Visiting, University of Ulster, Coleraine, Co Londonderry BT52 ISA, Northern Ireland (Requests for offprints to KP) Manuscript accepted 20 January 1991

220

have disappeared. The

quality

assurance

movement

has high-

lighted the need to take into account consumers’ views. What remains unresolved is how best these

views should

process,

whether

remains These

be sought. in health

under the control are

discussed

some

of the

The

evaluation

care or education, of the professionals.

issues

which

will be

in this paper.

THE METHODOLOGICAL APPROACH Evaluation

shares

with research

the

need

to

produce valid and reliable data. Nisbet (1987) notes that ‘curriculum evaluation is an extension of educational research, sharing its roots, using its methods and skills’. The difference between the two according to Cronbach (1987) is that ‘evaluation fits into a different institutional and political context’. Various research instruments such as questionnaires, interviews, observations and case studies have been used in curriculum evaluation. The questionnaire itself consists of a variety of question formats such as closed- and open-ended questions, rating scales, semantic

NURSE EDUCATION

differential sentence

scales,

multiple

completion

choice

questions,

most student

and so on.

autonomy

centred

An important question in the positivist approach to curriculum evaluation is that of who

educational

controls

truly liberalised

the

evaluation.

tionnaire

and/or

that

evaluator

the

covered,

The

structured

use of a quesinterview

controls

the

means

areas

to be

and decides in advance the structure

the questions.

The question

wording

of

is decided

by the evaluator. Even with semi and unstructured interviews, of an individual or a group, the

further

courses)

in controlling

22 1

TODAY

achieved

whether

relative

or not their

needs and desires are met’. He asks

if ‘we are witnessing

ing adults,

the emergence

and liberating

or if the ‘bottom

institutional

and

of a

system of educatline’ still is that

organisation

come first?’ If nurse education

requirements is a form of social

control as argued by Holloway and Penson ( 1987) then the evaluation process could be seen, at best,

as a containment

and

patching

up

presence of teaching (and sometimes management) staff may inhibit the full and frank flow of

exercise and, at worst, a cover up for the glaring deficiencies of the curriculum. Such a view may

information,

seem

thereby

throwing

doubt

on the

validity of the data. Sometimes this atmosphere is so intimidating that only superficial views are expressed.

Teachers

vis-a-vis students articulate positions.

are in an unequal

position

and are by and large

and

better

Criticising

able

to

defend

one’s teacher

towards members context) presence evaluator)

(1972)

of the investigator has an effect

progress

of

studying’

come

According

departments.

the

scheme

he

is

that it does.

and the right to of opinions

and reprisals,

should be the underlying

free real or

principle

the evaluation degree

third year students. was the assumption

of of a

of units in one

course by second and

Underlying the evaluation that it should be, as far as

possible, student controlled and that it should reflect issues of concern to them and expressed in words of their own choice.

on the other

conceptual

hand,

system’

(Cronbach

also points out that ‘great value

is placed on the observations

and interpretations

However, a qualitative approach

to data collection

over it as

any evaluation process. What follows is an account and discussion term of a nursing

and

allows ‘the observer to pick out events he considers significant’ and to ‘filter what he observes

of participants’.

perceived,

expression

constraints

(in this case the

innovatory

approach,

1987). Cronbach

The

small ‘experiment’:

to them any form of data collection

his

control involves free expression from influence,

creates disturbance.

through

them to exert control

the

to the conclusion

A qualitative

ations. The emancipation of students requires more than participation in the evaluation pro-

information.

Parlett

whether

on the ‘conduct

the

evalu-

they are at the receiving end of the service. Such

in his or her

questioning

to illustrate controlled

cess. It requires

of staff who are not present at

come from other

it serves

of teacher

their

or those who (in the university

and Hamilton

but

power

more

presence is a daunting task to most students. Often many of the criticisms are directed the evaluation,

extreme

potential

is not without its problems.

Not

METHOD This project was part of the formative process

of

the

nursing

methods of evaluation

evaluation

curriculum;

other

are also used throughout

the course. Second and third year students (20 in

least is the amount

and type of data generated

each year) were asked to assemble and carry out

and the reporting.

problems

an evaluation of the four study units in the first term. The instructions on how to tarry out the

ensuing

Such an approach to a student

centred

of

analysis

and

would seem to suit the move curriculum

in nurse edu-

procedure guidelines

were kept to a minimum. The were designed to avoid the produc-

cation. The concern with the issue of who controls the curriculum is an important one and

tion of large amounts of information which would be time consuming to analyse. The evalu-

is well articulated questions whether

ation coordinator resentative and

NET

E

by Morral (1989) when he ‘students (even those of the

suggested that the class repher assistant conduct the

222

NURSE EDUCATION

TODAY

evaluation. The class was asked to produce up to five statements on each of the units undertaken. Making use of the overhead projector, the class representative wrote down statements from students, and if these numbered more than five for one unit they were asked to vote openly for the ‘top’ five. If they felt strongly that they would need more than five statements to express their views, they could do so. No areas of evaluation were specified in advance. For the evaluation to take place at least two thirds of the unit members had to be present. The suggested time for each evaluation exercise was not more than 1 hour. However they were free to take as long as they wished. No member of staff was present at these sessions.

Table

Area

Examoles of comments

Content -

-

-

Course organisation

NURSE EDUCATION

fear that this type of evaluation

could degener-

ate into a form of teacher appraisal. adopted

was a variation

The method

of the Nominal

Group

may also affect the data. Perhaps once the statements are generated each student can vote privately for his or her ‘top five’ views. One of the

Method (NGM) as described by Wells (1987). According to Wells one of the disadvantages df

reasons

the

sus while taking

NGM

is that

‘student

responses

tend

to

ignore curriculum content’. However, in this case students made comments about topics which do not relate failure

of some

adequately

to nursing

and about

of the units to prepare

for their clinical

the

them

placements.

They

did not always question the amount of material to be covered but the time in which this was possible.

They

made suggestions

as to how this

could be improved. Another

by Wells (1987) coverage

of the NGM as pointed

is that ‘it can produced

of the curriculum

‘inventory’,

‘patchy’ i.e. high-

profile/impact

items overshadow

profile/impact

items’. This tended to be the case

in this evaluation

the different

but for students

concerned

or exclude low-

as well. However

was not to examine curriculum

its purpose

aspects of the

to focus on what

them most. Some of the statements

such as ‘no problem’ and ‘too complex’ were clearly unhelpful. Perhaps the term ‘statement’ was misleading should

in that they did not feel they

give explanations

Feedback the

second

proceeded

of what they meant.

from students revealed that the evalu-

ation guidelines

were too prescriptive.

year

in this experiment

asked to vote collectively

the students

into account

Most, if not all, students

students

ignored

Indeed

them

to produce whole paragraphs

and

instead

of statements.

were

was to achieve consenindividual

views.

will work in teams later

and as such would be required

to carry out team

evaluation of various aspects of their practice. The comments and the language in which they are phrased did

not

seem to indicate

feel

constrained

that the students

by the

straitjacket

imposed by most teacher controlled evaluation. Although a qualitative approach was used, one must bear in mind that the students

disadvantage

223

TODAY

vious experiences

would have influenced carried

out

the

nomeneology

the way in which they

present

is useful

respondents

perceive

understanding

own pre-

with other forms of evaluation evaluation.

Phe-

in understanding

how

their world. However

this

is limited if we do not take into

account the symbolic roles of all the actors in this particular

event. ‘Tell it as it is’ can be influenced

by ‘to whom it is told’ and ‘for what purpose’.

We

can ask how we know if people are telling us what they really think. If students perceive the evaluation of units as a cosmetic exercise or a mandatory chore

on the part of the staff, would they

put all their effort Phenomenology

in telling us how they feel? without

symbolic

interac-

tionism tells us only part of the story. They must go hand in hand in the pursuit of valid and reliable data.

The size of the group can also have an impact on the evaluation.

Feedback

from students

sug-

What happens

to the data raises ethical and

political issues. According

to MacDonald

(1974)

gested that more than 25 students would have made the exercise difficult. As the evaluation was

‘when evaluation data influences power relationships, the evaluator is compelled to weigh

not based on any discussion among students it lasted less than 1 hour. Doubts can be raised

carefully

about the concentration

were specified in advance the students chose the

and whether

span of the participants

items evaluated

towards the latter

end of the session would have the same attention

fication’.

the consequences

of his task speci-

In this case as there were few tasks that

areas they wanted to comment

upon. One such

no

area was the performance of some of the lecturers. Comments varied from ‘work sessions’

indication as to whether or not this was the case. One serious criticism of this approach is that

being ‘beneficial and excellent’ to lectures being ‘poorly presented’ and ‘disappointing’. The lat-

as the previous

ones. There

was, however,

the proceedings could be dominated by articulate and influential students. The skills of the class representative

in conducting

can offset this limitation.

the evaluation

However his or her role

ter statement

would not only be threatening

to

the lecturer(s) concerned but may also be rejected by them. Such data could play into the hands

of those

in a position

to exploit

them.

224

NURSE EDUCATION

TODAY

Clearly one must decide in advance who should

themselves

have access to these data and how conflicting

(1985)

views between

acknowledgment

resolved.

students

Therefore

and lecturers

the purpose

could be

of the evalu-

ation and the role of the evaluator

should

be

student

carry out evaluations

states,

‘course

evaluation

is a formal

of the student perspective

views are the basis upon

course changes

if, as Lomax

which

and most

occur’.

seriously considered. MacDonald (1974) suggests that ‘the evaluator acts as a broker in

According to Morral (1989) ‘if the quality of nurse education is going to improve then the

exchanges

social context in which it operates requires structural and ideological alterations’. This

parties’.

of

information

between

different

One must also ask, in the case of the

evaluator being a member of staff, to whom does

would involve a distribution

he or she owe allegiance:

lecturers

and students.

whether

lecturers

both? Adelman evaluation the

and

Alexander

programmes

status

to students,

quo.

teachers or

(1982)

question

which do not threaten

What

is

often

however is the fact that students also have a stake see

a

successful there

between

lecturers

could

outcome may

well

and

be achieved.

Those

to

to

their

be

disagreement

students

on

course, how this

who believe

that a

student controlled evaluation is an exercise in chaos should reflect on their own prejudices regarding students and the degree of control which teachers have on the evaluation planning, process and outcome.

CONCLUSIONS Curriculum evaluation must different approaches in order According gained

to Cronbach

when

objective,

an

more

becomes

flexible.

broader

‘something becomes

reproducible,

trated. Something ation

make use of to be effective.

(1987)

evaluation

more

is

more concen-

else is gained when the evalu-

more

phenomenological,

in its coverage’.

question

more

are prepared whether fundamentally,

reform

of nurse education

remains

for this and, the current

will bring it about.

overlooked

in the status quo. They are just as concerned although

of power between

The

Students

more can

References Adelman C. Alexander R 1982 The self evaluating institution. Merhuen, London Cronbach L J 1987 Issues in planning evaluations. In: Murphy R, Torrance H (eds) Evaluating education: issues and methods. Harper and Row, London Holloway 1, Penson J 1987 Nurse education as social control. Nurse Education Today 7.5: 235-241 Lomax P 1985 Evaluating for course improvement: a case study. Assessment and Evaluation in Higher Education 10, 3: 254-264 MacDonald B 1974 Evaluation and the control of education. In: MacDonald B, Walker R (eds) Innovation, evaluation. research and the problem of control (SAFARI). Centre for Applied Research in Education, Norwich Morral P A 1989 Quality assurance in nurse education the social context of learning. Nurse Education Today 9,4: 236-24 1 Nisbet J 1987 The role of evaluators in accountability systems. In: Murphy R, Torrance H (eds) Evaluating education: issues and methods. Harper and Row, London Parlett M, Hamilton D 1972 Evaluation as illumination: a new approach to the study of innovatory programmes. Occasional Paper No 9. Centre for Research in Educational Sciences, Edinburgh Wells J 1987 Curriculum evaluation. In: Allan P, and Jolley M (eds) The curriculum in nursing education. Croom Helm. London