The impact of inservice training on teacher behaviour

Teaching & Teacher Educarion. Vol. IO, No. 3. pp. 303-317. 1994 Copyright @ 1994 Elscvier Science Ltd Printed in Great Britain. All rights reserved 0742-051x/94 s7.00+0.00

Pergamon 0742-051X(93)30001-8

THE

IMPACT

OF INSERVICE

S. VEENMAN,

TRAINING

M. VAN TULDER,

ON TEACHER

BEHAVIOUR

and M . VOETEN

University of Nijmegen, The Netherlands Abstract-Nowadays, a considerable amount of time, money, and energy is invested in inservice education or training (INSET). Yet, our knowledge ofdesign featuresand implementation conditions in schools that make INSET effective,is still limited. To examine the outcomes of inservice activities the impact ratings of a sample of teachers and principals who participated in individual-based and school-focused INSET were regressed on school characteristics, features of inservice programs, implementation characteristics, and types of inservice activity. Findings from this study suggest that some school characteristics, features of inservice programs, and implementation characteristics contribute to the impact of inservice training. Variables like external support, hours spent on INSET activities, degree of participation,, teacher educators’competence, implementation conditions, and the use of principal’s steering functions are subject to alteration to increase inservice training effectiveness.

Inservice education or inservice training (commonly abbreviated to INSET) is considered to be a key aspect of school improvement efforts (Sparks 8~ Loucks-Horsley, 1990). Inservice training servesthree main purposes: (1) to stimulate the professional competence and development of teachers; (2) to improve school practice; and (3) to implement political agreed-upon innovations in schools. Inservice training in the Netherlands - organized on a voluntary basis - is conveniently defined as a coherent set of activities intended to deepenand broaden knowledge, attitudes, and skills that are directly connectedwith the profession of teaching to improve teachers’professional competenceand the effectiveness of their schools. This coherent set of activities should be based on the entry-level capabilities of teachers acquired in preservice education. In spite of the acknowledged importance of inservice training for enhancing the quality of teaching and for implementing educational innovations, much remains to be learned about the process by which inservice training occurs (Sparks & Loucks-Horsley, 1990). Until now, most attention in the Netherlands is given to conceptual difficulties, and procedural/political governance and control issues.Teacher educa-

tion institutions are responsible for initial or preserviceeducation, but also for inservice education and training. Most of these inservice activities take the form of coursesor workshops. At the sametime, school advisory services,which are institutions working at local and regional level, concern themselveswith guidance,support and development activities, advising, providing information and evaluations for primary schools, and several types of special schools. The given guidance, support or advice is mostly job-embedded or job-related, and directed at the school as a whole or at groups of teachers at school. With the recent emphasis on school-focused or school-centred INSET the boundaries between the agencies for organizing inservice activities have become somewhat obscure. Coordination is emphasized,but not easily realized becauseof differences in legislation, financing, denomination, responsibilities, and competition due to vested interests. Inservice activities have often been criticized:

303

They are by their very nature compelled to offer information and experiencewhich is very general and is therefore insufficiently related to the specific needs and concerns of the participants; they tend to offer ‘theory’ which is unrelated to practice; they tend to over-use lectures and discussion methods; in conse-

304

S. VEENMAN

quence, they are ineffective in influencing teacher performance and school improvement. (Bolam, 1987)

Much of the criticism is directed at individually based inservice activities. In this type of inservice activity individual teachersfrom several schools are grouped together; it is focused on concerns and needs of individual teachers.If an individual teacher encounters problems while attempting to put newly acquired ideas or skills into practice, there are no convenient resources available for help or sharing by colleagues.It is partly for such reasons that nowadays more emphasisis placed on school-focusedINSET. To underline this shift in emphasis the term “inservice training” is replaced by the concept “staff development” to expressthat activities designed to advance knowledge, skills, and understandings of teachers encompass much more than a single teacher acting as an individual. Modern staff development is an enterprise of groups of teachers (Fenstermacher & Berliner, 1985; Sparks & Loucks-Horsley, 1990).Both types of inservice activities will be included in this study. Individual-based (or course-based)INSET in this study is defined as inservice training activities (programs) organized in severalprespecified sessionswithin one school year, in which individual teachers from several schools were grouped together. School-focusedINSET is defined as inservice training activities (programs) organized in several sessionswithin one or two school years, in which all teachers or a group of teachers of a specific school participate. Teams or subteamsfrom severalschools might or might not be grouped together, depending on the number of teachers.(An averagenumber of about 20 teachers per course is required by the subsidizing Dutch government.) Transfer of Training

Nowadays, a considerable amount of time, money, and energy is invested in inservice training. However, our knowledge of design features and implementation conditions in schools that make inservice education effective,i,sstill limited. Reviews of the literature on trammg indicated that little empirical attention has been devoted to the issueof training transfer (Baldwin & Ford, 1988; Gist, Bavetta, & Stevens, 1990).Transfer of training is defined as the degree to which knowledge and skills acquired by training are

et al.

effectively applied in the workplace of the school or classroom. For transfer to occur, trained behaviour must be generalizedto the job context and maintained over a period of time on the job. There is a growing recognition of a transfer problem in inservice education today. Teachers often complain that inservice courses are too theoretical, too far removed from the daily working experiences of teachers, that teacher educators are unable to recognize and use the expertise of the teachers attending inservice courses,that the training activities do not result in changesin teachers’instructional behaviours (Van Tulder, 1992). A widely accepted model to measure the effectiveness of training outcomes is Kirkpatrick’s model of training evaluation criteria (Kirkpatrick, 1987).This model proposesfour “levels” of training evaluation criteria: reactions, learning, behaviour, and results. Meta-analytic reviews conducted by Joslin (1980),Lawrence and Harrison (1980),and Wade (1984, 1985) reveal that only a few studies systematically determine the effectivenessof inservice training programs at the level of the pupils with whom the teachersinteract. Most inservice evaluation reports are simply statements of participants’ satisfaction or learning outcomes (levels 1 and 2: reactions and learning), which are used to determine the successof a program. The lack of research on the effects of inservice training programs with regard to teachers’behaviour and pupils’ behaviour and achievement (levels 3 and 4: behaviour and results) may be explained by the difficulty to set up methodologically sound studies that may explain casual relationships between inservice activities and pupils’ progress. When the existing data are grouped according to Kirkpatrick’s training evaluation criteria, Wade’s findings suggestthat attempts to increase teachers’learning through inservice teacher training are highly effective (.90 mean effect size); attempts to change teachers’ behaviour and to elicit positive reactions to the training are moderately effective (.60 and .42 mean effect size, respectively),while attempts to demonstrate results by looking at the pupils of the trained teachersare only mildly effective (.37 mean effect size) (Wade, 1984, 1985; see also Sparks & Loucks-Horsley, 1990;Walberg, 1991). The results from the meta-analytic reviews conducted by Joslin (1980),Lawrence and Har-

Impact of Inservice Training

rison (1980), and Wade (1985) suggest that inservice training m ight contribute to increased pupils’ progress becauseteachers are using the skills that were taught. This is not to say that inservice training has caused pupils’ progress to rise. Many other factors can affect increasesin pupils’ progress, including, for example, the instructional and administrative support of the school principal with respect to the skills that were taught, or a new educational policy adopted by the school board. To be able to reveal a training’s contribution, one must demonstrate casual links between pupils’ progress and the skills and knowledge being offered in the training, provide effective training in those skills and knowledge, and ascertain that the teachers do use on the job what they learned during training (Robinson & Robinson, 1989). Carrying out these four activities is expensive and time-consuming, and therefore seldom put into practice. In this study, the impact of inservice training will be examined on the level of behavioural results by self-reports from trainees (level 3: behaviour): Do teachers confirm that they do use on the job what they learned during inservice training? To answer this question interviews and questionnaireswere used.Observations were not included because this study is directed at a large-scaleevaluation of inservice training activities in the Netherlands. The topics of the inservice activities, their contents, goals, and designs differed too much to allow observations to determine whether teachersare using on the job what they have been taught. Observations and achievementdata were used in a study to evaluate the impact of a well defined inservice training program for teachers in m ixed-age classes (Veenman, Lem, & Roelofs, 1989; Roelofs, Raemaekers,& Veenman, 1991; Roelofs, 1993).

on Teacher Behaviour

305

of INSET, implementation includes the activities of teachersand school principals to use the new insights or skills acquired from inservice activities. Innovating is often painful, or as Sir Francis Bacon put it in his essay “Of innovations” published in 1625:“new things piecenot so well”. Changing instructional and management practices involves confusions, self-doubts, temporary setbacks, new procedures for daily work, and other uncertainty-arousing events.Successfulimplementation usually requires measures to reduce the intensity or duration of these events (Huberman & M iles, 1984). Provisions for support, encouragement,and assistanceare important early steps in the process of mastering the innovation. Van der Vegt and Knip (1988) state that arrangementsdesignedto facilitate implementation have to be situated close to the educational day-to-day practice of the school. Situating it close to the daily practice facilitates direct transfer pertaining to the innovative concept. To have impact, inservice activities must have a central position in the implementation process. Implementation also needs a specific design or work structure. Innovating frequently fails at the implementation stagebecausethe existing structure is not a good fit for implementation activities. Making implementation work meansreassigning tasks; investments in time, resources,and support structures; arrangements for planning and coordination. Implementation of educational programs by inservice training is more successfulin schools with norms of collegiality and continuous improvement (experimentation),in which a greater range of professional interaction with fellow teachersor administrators is pursued, including talks about instruction, structured observation, and shared planning or preparation (Little, 1982; Rosenholtz, 1991).Active involvement and supImplementatio~t of Educational Programs port of principals are of crucial importance to The impact of inservice activities dependsnot enhance implementation (Leithwood & Montonly upon the qualities of the training supplied gomery, 1982). In designing inservice activities, but also on the features of the implementation one must take into account factors regarding process to put the newly acquired knowledge, effective implementation. insights, and skills into practice and the organization of the school in which they are used. 7he Objects of the Study Implementation is the process by which the The overall purpose of the researchproject on school attempts a change in state; it comprises the activities of users attempting to use an “Inservice education and educational change in innovative idea (Berman, 1981). In the context the Netherlands” is to determine which charac-

S. VEENMAN

et al.

ticipated in school-focused INSET and their prinicipals (interviews and questionnaires), for inservice teacher educator/trainers (interviews), and for school counsellors who participated in school-focusedINSET (interviews). Becausethis study is restricted to the data on teachers and principals, attention will only be paid to the data-gathering techniques used for this group of respondents.Findings from INSET trainers and school counsellors are reported in Van Tulder and Veenman (1991). For the individual-based inservice activities the questionnaire method was used in order to include a large amount of different inservice programs. For the school-focused inservice activities the interview method was used to obtain not only information about INSET characteristics and appraisals,but also about the embedding of the inservice activities in the daily life of the schools. Besidesinterviews, questionnaires were also used for school-focused INSET to enlarge the group of respondentsfor this type of inservice training. Draft versions of the questionnaires and interview schedules were prepared and piloted. Feedback receivedfrom this pilot study enabled several revisions to be made before the final versions were produced. Among the data-gathering techniques the questionnaires and interview schedulesdirected at the teacherswho had participated in INSET, were the most extensive. These questionnaires and interview schedules were developed to measure features of inservice programs, and to determine the implementation of characteristics of effective INSET, judged in the Delphi-procedure as important for educational change (implementation at program level). Teachers were also asked to indicate the impact of INSET on teaching behaviours (implementation at classroom level). The questionnaire or interview schedulefor the principal was primarily directed at the impact of INSET at the organization of the school (implementation at school level). Most questions required a tick, or the selection Method of a forced-choice category on a four-point or five-point Likert scale.However, provisions were Questionnaires and Interview Schedules also made for open-ended responsesto enable Questionnaires and interview scheduleswere respondents to make comments of importance developed for teacherswho participated in indi- or suggestions on matters not covered by the vidually based INSET and for their principals forced-choiceitems. Concerning implementation (questionnaires), for school teams who par- and impact characteristics respondents only

teristics inservice activities - focused on the implementation of educational innovations must have to be effective, and to what extent designersof inservice activities take these characteristicsinto account. The study is also directed at the relationship between these characteristics of inservice activities and their effects on the implementation of newly acquired knowledge and skills by teachers.Effective characteristics of inservice activities were determined by a Delphiprocedure (Van Tulder, Veenman, & Sieben, 1988).The Delphi technique was usedto identify salient features of effective inservice practices, and their programming and design. It was based on a review of educational literature relevant to inservice education and training (INSET), staff development, and educational change. After reviewing this literature a list of effective features of inservice activities was compiled and rated by a panel of experts. Based on the findings of the Delphi-study different questionnaires and interview scheduleswere designedfor teachers,principals, INSET trainers, and school counsellors. By surveying a sample of teachersand principals features of inservice activities were determined, and teachers’, principals’, trainers’, and school counsellors’ involvement in inservice activities and their perceptions of inservice activities’ usefulness and impact were assessed.Results of this survey are reported by Van Tulder and Veenman (1991).An indepth case study was conducted in six schools to collect qualitative data regarding the effects of the inservice program on the way these schools handled the innovation process that was initiated by the inservice activities. The results of this multi-case study are reported in Van Tulder, Van der Vegt, and Veenman (1993). The present study is directed at the relationship between effective design and implementation characteristics of inservice programs and their effects on the instructional behaviours of teachers.

Impact of InserviceTraining on TeacherBehaviour answered those questions that related to the content of their programs, because goals and applications varied greatly between INSET programs. Item wordings for questionnaires and interview scheduleswere identical. The interview form asked almost all the same questions as the questionnaire to make comparisons of respondents’ answers possible.

307

ary for contacting the teachers who participated in their inservice programs. In the school year 1986-1987, a total of 1140 teachers were contacted by mail, 830 teachers who participated in individual-based INSET and 310 teachers who participated in school-focused INSET. The questionnaire was returned by 355 teachers who participated in individual-based INSET (response rate 44%) and by 55 teachers who participated in school-focused INSET (reSmple sponse rate 18%). When a teacher returned the To identify the users of INSET activities the questionnaire the principal of that teacher was following procedure was adopted for selecting also sent a questionnaire (response rate 63%, appropriate samples. Twenty colleges of educa- 169 principals who participated in individualtion for primary school teachers (PABO) were based INSET, and 35 who participated in schoolcontacted. They all agreed to participate in the focused INSET). study (about one-third of the colleges in the In addition, teachers and principals of 82 Netherlands). These colleges were located in the schools were contacted who participated in middle, east, and south of the country. The school-focused INSET. Of these schools the co-ordinator for INSET activities was asked to whole teaching staff or a group of teachers of supply information about their inservice pro- the same school participated in school-focused grams. From this pool of inservice programs only INSET. Two or three teachers of these schools those programs that were directed at educational were asked to participate as a group in an change processes in the primary school were interview study. The principal would be interselected.According to a new Education Act, put viewed separately. Of these schools 53 agreed to into operation in 1985, kindergartens and pri- participate (response rate 65 %). mary schools in the Netherlands were merged. The total sample of the questionnaire part of All programs selected were directed at aspects the study comprised 355 teachers and 169 prinof the implementation of the legally prescribed cipals who participated in individual-based INnew conception of primary education for SET, and 55 teachers and 35 principals who children between the ages of 4 and 12. This participated in school-focused INSET. The total selection criterion resulted in 18 topics for indi- sample of the interview part of the study comvidual-based INSET, and 13 for school-focused prised 53 school teams with their principals who INSET. Nine of these topics were given in both participated in school-focused INSET. For a types of inservice training. Because some topics complete description of the design, the instruwere offered by several colleges of education, the mentation, and data collection procedures see number of actually conducted programs or Van Tulder (1992). courses included in the survey amounted to 103 Although the response rates in the questionfor individual-based INSET and to 119 for naire part of the study were not very impressive, school-focused INSET. The most frequently oc- they are in line with the response rates that are curing topics were: English language, school normally found in Dutch surveys in these days. management for school leaders, dealing with Schools and teachers are not very eager to individual differencesin reading instruction, edu- cooperate in surveys becauseof the high teaching cational technology, and kindergarten educa- load and financial cuts. The teachers’ unions tion. The first two topics and the last one were have asked teachers to concentrate primarily on specially related to the new conception of the their teaching tasks, and not on administrative duties as a protest against the financial cuts by primary school. After the selection of the courses, the co- the government. The reasons for non-response ordinator for INSET activities was asked per- given by the teachers reflect this attitude. The mission to contact the teacher educators charged most frequently mentioned reasons for nonwith inservice activities. After this permission response were: too busy, refusals expressedas a was granted, these educators acted as intermedi- protest against government policies, not working

S. VEENMAN

at the school that participated in INSET, team decision not to cooperate in research projects. When schools were approached in a personal way, as in the interview part of the study, the response rates were higher. Direct contact between researchers and schools appeared to be important to gain the cooperation of schools.

The analysis of the data is based on an adaptation of the model for explaining educational staff use of professional knowledge as developed by Walberg and Genova (1982). In this model, use of knowledge - broadly defined to include ideas, information, craft skills, materials, and experience - is influenced by five sets of variables: background and psychological traits of inservice workshop participants; school characteristics and climate; and inservice workshop features. School climate and workshop featuresare consideredpotentially alterable, they can be considered policy variables subject to modification by those within and outside schools to improve knowledge use.Individual and school characteristics are considered as less alterable, more stable variables. Moreover, these characteristics can be considered causally prior to the alterable variables. Thus, the assumptions are that the less alterable variables may influence the alterable ones and the impact variables, whereas the more alterable variables do not influence the less alterable, but may influence inservice workshop impact. In general, the Walberg and Genova model resemblesthe model of the transfer processas developedby Baldwin and Ford (1988).In this model of the transfer process the following five sets of variables are distinguished: trainee characteristics, training design, work environment, learning and retention, and generalization and maintenance. For our purposes the Walberg and Genova model was adapted as follows. Staff background characteristics and staff psychological traits were not included in our analysis becausethe sampled data from two or three staff members could not be considered as a reliable index for the school staff as .a whole. For the same reason school climate was omitted. In the model of Walberg and Genova characteristics of the inservice teachereducator are not considered,in our study they are included in the factor “inservice pro-

et al.

gram” features.Inspired by the literature on the implementation of innovations (Fullan, 1982; Huberman & Miles, 1984;Van der Vegt & Knip, 1988) we assumed that, besides school characteristics and inservice program features, implementation characteristics also may influence the impact of inservice training. Implementation characteristics are those features of the school as a work place, that support the utilization impact. These characteristics are defined as: “conditions for implementation” (including items like: provision of support materials, specification of an agenda-for-change,encouragement provided in school, flexibility for personal discretion, support and assistance by colleagues),“information exchange”(including items like: information asked by colleagues,information given by the trainees, convincing colleagues to use ideas or products, exchange of implementation ideas). Principal’s steering functions were also included in the implementation characteristics. Based on the work of Van der Vegt and Knip (1988) four steering functions were distinguished: direction/concept clarification, directional pressure, latitude definition, and assistance/support (see also Huberman & Miles, 1984). These steering functions appeared to be critical leadership dimensions for the outcomes of implementation work. From a management perspective, these functions must be installed in the dynamics of school life. The function of direction/concept clarification deals with the extent to which the principal invests in providing teachers with a clear vision of what will be achieved by implementation. Directional pressure refers to operational mastery of implementation together with pressure to achieve (e.g., formulating expectations for actions and investments, setting deadlines). Latitude definition is specifying the range of acceptable paths to the goal region (e.g., distinguishing betweenlegitimate individual discretion, and attempts to subvert or abandon components of the innovation concept). Assistance/support relates to the principal’s capacity to mobilize resources and expertise for renewal work (e.g., delivering technical assistance, providing social-emotional support). The sets of variables thought important of explaining the effects of inservice activities are presented in Figure 1. To examine the influence of several design features of inservice programs and implementa-

Impact of Inservice Training

309

on Teacher Behaviour

Levels

of

impact

* classroom impact * school impact * knowledge utilization impact

Figure 1. Model used for explaining possible influences on impact levels in educational improvement efforts (adapted from Walberg & Genova, 1982).

tion characteristics on the outcome of inservice activities, several multiple regression analyses were performed. The dependentand independent variables and the statistical procedures are described below. The dependent variables comprise three levels of impact: classroom-level impact, school-level impact, and knowledge use. These impact measureswere basedon teachers’and subteams’ ratings on 10 impact items that were subjected to a principal components analysis with varimax rotation. This analysis yielded three factors both for individual-based and school-focused INSET - that collectively accounted for, respectively, 61% and 63% of the total variance in teacher impact ratings. Factor 1, interpreted as classroom-levelimpact, accounted for 30% of the rating variance for individual-based INSET, and 36% for school-focused INSET. Factor 2, interpreted as impact on school-level’s working conditions, accounted for, respectively,21% and 16% of the variance; and factor 3, interpreted as use of new practices for, respectively, 10% and 11%. Based on the results of the principal components analysis, the lo-item impact scale was broken into three subscales:(1) classroomlevel impact (changes in learning materials, teaching methods, extra support for at-risk children, relationships with pupils); (2) schoollevel impact (changes in productivity of school meetings, working relationships among teachers);and (3) actual use of new knowledge, skills and attitudes in the staff’s instructional repertory during or immediately after the program (discussionsof improvement efforts, adop-

tion by colleaguesof new ideas or products, use of new ideas in the classroom) (seeTable 1). To explain impact differences multiple regression analyses were performed. Five sets of independent variables were considered as potentially influencing impact scores: (1) school characteristics, (2) features of inservice activities, (3) implementation characteristics, (4) type of inservice activity (individual-based versus schoolfocused), (5) interactions between the type of inservice activity and the variables from the first three sets. School characteristics were considered as relatively stable and unalterable variables. Included were: the number of pupils in the school, former experiencewith educational innovation projects, and the frequency of support by the school advisory service. Features of inservice programs included simple and complex variables. Simple variables were: duration of the inservice program, extra hours spent on INSET activities, number of participants, and the recommendability of the inservice program to other teachers. For complex variables scale scores were computed: involvement of participants in planning, being well informed in advance, degree of participation, degree of satisfaction with course content, degreeof satisfaction with the instructional skills of the INSET trainers, and degree of satisfaction with the practicality of INSET trainers’ recommendations. To be most effective, inservice training should include theory, demonstration, practice, feedback,and coaching (Joyce & Showers, 1980, 1988). Without coaching transfer of new skills

S. VEENMAN et al.

310 Table 1 Overview of rhe Depemierlr and hdependenr

Variables

min

max

N

1.0 1.0 1.0

3.0 3.0 3.0

457 455 458

22 1

680 2

395 396

.94

1

4

402

2.21 2.09

.69 .78

1 1

4 4

455 459

18.71 280

5.32 103

6 45

50 765

455 453

8.4

0

98

441

4 5 2

459 462 449

Variables

k

II

a

M

Level of impact Classroom impact School impact Knowledge use impact

5 2 3

281 281 281

.71

1.27 1.29 1.83

.37 .50 .60

197.5 1.38

108.4 .55

2.49

.84 .14

School characteristics Number of pupils Former experience with innovation projects Frequency of support by the school advisory service Features of inservice programs Being well informed in advance Involvement of participants in planning Number of participants Duration of INSET program in hours Extra hours spent on INSET activities Degree of participation Degree of satisfaction Training recommendable for other teachers INSET trainer’s instructional skills Practical recommendations presented Use of effective training components Implementation characteristics Conditions for implementation Principal’s steering functions Exchange of information

4

451

.86

5.15

SD

4 9

453 453

.84 .90

3.22 3.18 1.24

.5-l .84 .42

1 1 1

9

423

.93

4.03

.82

1

462

4

409

.84

3.19

.88

1

461

1.85

.60

466

2.94 2.06 2.05

.94 .70 .54

466 466 442

5 4 3

115 325 154

.I3

.90 .64

Note. k = number of scale items; N = number of respondents; n = number of respondents in reliability checks (in testing for reliability listwise deletion of missing values was used; this explains the differences between II and N); M = mean; SD = standard deviation; min = minimum item score; max = maximum item score.

or strategies to everyday practice cannot be guaranteed. According to the training model proposed by Joyce and Showers inservice activities with the component “coaching” plus three or four other components were rated as effective, inservice activities without coaching and less than three training components as ineffective. This rating constituted the variable “use of effective training components.” As stated before, implementation characteristics comprised three sets of variables: conditions for implementation, principals’ steering functions, and information exchange. Results of a

principal components analysis with varimax rotation performed on the teacher s’ ratings confirmed this grouping of implementation variables, In the individual-based INSET sample factor 1 (conditions for implementation) accounted for 36% of the rating variance, factor 2 (principal’s steering function) for 20%, factor 3 (school counsellor’s steering function) for 17% (not included in this study), and factor 4 (information exchange) for 9%. In the school-focused INSET sample also four factors were extracted, accounting for 76% of the variance. The conditions-for-implementation factor was divided in

Impact of Inservice Training

on Teacher Behaviour

311

Table 2 Contribution of rlre Independent Variables IO the Criteriott Variance: Proporrions of Variance Explained Incrementally blr Each Ser

Independent variables k

Dependent variables Impact on Impact on classroom school level level

Impact on knowledge use

1. School characteristics 2. Features of inservice programs 3. Implementation characteristics 4. Type of inservice activity 5. Interactions of type of INSET with school characteristics features of inservice programs implementation characteristics

4 11 3 1

.03 .19*** .03* .02**

.03 .25*** .08*** .oo

.02 .22*** .18*** .03***

4 11 3

.Ol .03 .03

.Ol .02 .oo

.Ol .02 .oo

Total explained (R’)

37

.32*** 325

.39*** 323

.48*** 325

N

Note. k = number of independent variables in a set. *p < .05, **p < .Ol, ***p < 001.

a planning and in a support factor, and the principal’s and school counsellor’s steering functions appeared in the first factor. The same analysis on principals’ ratings confirmed this solution; three factors were extracted because the subscale “conditions for implementation” was not used for principals. Finally, the variable type of inservice actioity was included in the analysis. A distinction was made between individual-based and schoolfocusedINSET to examine the differential effects of these two types of inservice training. Multiple regression analyses were performed - by the program “Regression” from SPSSX - to explore the relations of the five sets of independent variables with the teachers’selfreported effects of the inservice programs at the three impact levels. Variables were added blockwise ot the regressionequation according to the sequenceof the five setsof independent variables described above. This sequenceis in agreement with the model depicted in Figure 1. M issing values on the questions concerning implementation conditions and steering functions were replaced by their average scores on these variables. As a check the analyses were repeated twice with different ways of treating m issing values. The first reanalyseswere based on the assumption that m issingsindicate absence of the implementation conditions and of the

steering functions. So m issings were replaced by the lowest possible value. In the last run the analyses were performed with listwise deletion of m issing values. There were only m inor differencesin outcome betweenthese three ways of treating m issing values. In the analysesreported here the method of imputing mean values was used. There were also some m issing values on variables concerning school characteristics and features of inservice activities. Teachers with m issing values on these variables were excluded from all analyses. Results The contribution of each set of independent variables to the criterion variance at the three impact levels is presented in Table 2. The proportion of variance accounted by all the independent variables is partitioned incrementally by looking at the increase in the proportion of variance accounted which is attributable to a particular set of independentvariables.Note that the values reported depend upon the order of entry of the sets of variables. The results reported in Table 2 indicate that interaction effects are negligible and nonsignificant. This meansthat the featuresof the inservice programs and the characteristics of the implementation process on the self-reported impact

312

S. VEENMAN

et al.

Table 3 Results of Stepwise Regression Analyses: Standardized Regression Coeflcients (/I)

Dependent variables Independent variables

School characteristics Number of pupils Former experience with innovation projects Frequency of support by the school advisory service Features of inservice programs Being well informed in advance Duration of INSET program Extra hours spent on INSET activities Degree of participation Degree of satisfaction Practical recommendations presented Implementation characteristics Principal’s steering functions Exchange of information Conditions for implementation Type of inservice activity R2 (subset) RZ total)

Impact on classroom level t P

Impact on school level t B -.ll

-.16

-.26 -.I5

Impact on knowledge use t B

-2.3*

- 3.2**

-4.s*** - 2.9**

.I7

3.4***

.21

3.0**

.28

4.1***

.lS

2.9**

.14 .2s .26

2.8**

.13

2.7**

.ll .23

2.0* 4.8***

-.08 .I2

- 1.8 2.5’

.20

3.8***

.29

6.2***

.34 .36

.lS

3.6***

.31

6.7*+*

.39 .I1

8.8*** 2.s*

.19 .44 .4s

4.4***

Note. *p < .OS, **p < .Ol, ***p < Mll.

levels do not depend upon the type of inservice training (individual-based versus school-focused).Also, the type of training does not interact with school characteristics.Out of 54 interaction effectstested only one appearedto be significant. This m ight well be a chance hit; therefore interaction effects are left aside. Taken as a whole, school characteristics are not significantly related to the reported impact scores. However, the features of the inservice program and the implementation characteristics are significantly related to the impact variables. Inservice program characteristics contribute to all three levels of impact at just about the same amount. Implementation characteristics are most strongly related to knowledge use, the actual use of new knowledge and skills at the time of training or immediately thereafter. After controlling for all proceeding variables, type of training has a small but statistically significant

contribution at two of the three impact levels. For school-focused INSET greater impacts (at the levels of classroom and knowledge use) are reported than for individual-based INSET. After removing the interaction effects, 19 variables were left in the regression equations, only some of them being statistically significant. Stepwise multiple regression was used to determine a small subset of variables that would account for approximately the same amount of variance as the complete set of 19 variables. The resulting regression equations are presented in Table 3. This table contains standardized regression coefficients accompanied by t-values. The proportions of variance explained remained almost the sameas those for the complete set of 19 variables. (The scores on the R2 (subset) are comparable with those on the line R2 (total) in Table 3). For each impact level a somewhat different subset of significantly contributing variables has

Impact of InserviceTraining on TeacherBehaviour been obtained. The results will now be described for each of the four sets separately. School Characteristics

The number of pupils attending school influences the reported impact of inservice training at the school level: the school-level impact of the training (changes in productivity of school meetings and working conditions) is greater when the school is smaller. However, school size does not seem to be associated with classroom-level impact, nor with impact on knowledge. Former experience with innovation projects seems to have a negative effect at the classroom level. Frequency of support by a school advisory service is significantly related with impact, but only at the school level. More intensive support is associated with greater impact of the inservice training revealing itself in better teamwork conditions within the school. Features of the Inservice Programs

Being well-informed in advance about purposesand content of the training appears to have a positive influence on impact at the school level but a negative influence at the classroom level. The extent to which the participants are involved in the planning of the training program does not make any difference in the reported impact of the training. The duration of the training program leads to contradictory results. The length of the training is negatively related to classroom-level impact but positively to school-level impact. Extra hours spent on INSET activities (activities undertaken in addition to the planned training sessions)seem to have a significantly positive effect on classroom-level impact as well as on knowledge-use impact. The number of participants in training sessions seems unrelated to the three impact measures. But the degree of participation does show a positive effect at the school level. The degree of satisfaction with the inservice training contributes to the impact of the training at the classroom level. Whether a training is found recommendable for other teachers does not seem to make a difference with respect to the effects of an inservice training. The same appears to be true for the extent to which INSET

313

trainers used the effective training components as proposed by Joyce and Showers (1988). The extent to which the trainer presents the trainees with practical recommendations has a somewhat strong positive relation with impact at all three levels. The more practically oriented the trainer is the more the newly acquired knowledge and skills are applied and the more changes at the classroom and school level are achieved. Practical-mindedness of the trainer appears to be the training characteristic that has the strongest relations with the impact variables. In contrast, it is noteworthy that no significant relations with impact are found for the instructional skills of the trainers. Implementation

Characteristics

Favourable structural arrangements within the school that support the implementation of the training program have some effect on knowledge use. The principal’s steering functions of the implementation process appear to have a quite large effect on school-level impact, but no effect on the other two impact levels. Classroomlevel impact and knowledge-use impact scores are related to the exchange of information about the contents of the training within the school team. More information exchange leads to greater impact of inservice training on these two levels. Individual-Based

Versus School-Focused INSET

The type of inservice training appears to have a different impact depending upon the level at which impact is assessed.School-focused training is positively associated with classroom-level impact, and even more with knowledge use. However, no relation was found with schoollevel impact.

Discussion To examine the influence of inservice training on teacher behaviour the impact ratings of a sample of primary school teachers were regressed on their school characteristics, the features of their inservice programs, the implementation characteristics of their schools, and the types of inservice activities they attended.

314

S. VEENMAN

The results of the present study are consistent with some of the findings of Walberg and Genova (1992); knowledge utilization and implementation will be more extensive under the following conditions: (1) the school organization and climate are supportive and well organized, (2) the content of the inservice training activities is geared to the professional spheres of influence of the participants, (3) clear advanceexplanation of the goals of the inservice programs, (4) subject matter relevant to job, (5) practical skills presented, (6) extra time invested in inservice activities, and (7) active involvement of the participants in learning activities. This study also stressed the importance of the support and feedback from a consultant (school counsellor) and the principal’s active involvement, support and steering. These characteristics of inservice programs or staff-development programs can be influenced to a great degreeby the designersand trainers of inservice activities. As Walberg and Genova (1992)pointed out, these characteristics are more alterable than others, like age of the teaching staff, their personalities,and they physical structure of the school. To some extent, the subjective nature of the impact measures(based on the personal judgement of the teachers),the possible influence of background or psychological traits of the teachers,school bound factors, school environmental characteristics, and the aggregation of the ratings from two or three teachers who participated in school-focusedINSET to school ratings suggest some caution is warranted in generalizing the findings. Also, the low response rate in the questionnaire part of the study regarding individual-based INSET warrants some caution regarding the representatives of the findings. While the sample from the school year 1986-1987 cannot be considered ideal, it seems likely that the patterns of responsesreflected the wider picture of teachersparticipating in individual-based INSET. The study as a whole also included data from the school years 1984-1985 and 1985-1986 (not published here, see Van Tulder & Veenman, 1991). For 1984-1985 the response rate for the questionnaire part of the study was 55% (number of teacherscontacted 120),and for 1985-198644% (number of teachers contacted 165). No significant differenceswere found between these three samples from different school years. In

ct al.

1986-1987 questionnaires were also sent to a sample of individual teachers who participated in school-focused INSET to enlarge the pool of respondents for this type of inservice training. This attempt was not a successbecauseof the low responserate (18%). However, the findings from school-focused INSET are mainly based on the interview part of the study (responserate 65 %). Furthermore, principals’and school counsellers’ impact ratings (not reported here; see Van Tulder & Veenman, 1991;Van Tulder, 1992) in general corroborate the findings obtained from the teachers. The results of this study suggest that several aspects of schools, inservice programs, implementation processes,and type of inservice training are significantly associated with the impact of inservice activities. At the level of classroom impact 25% of the variance in teachers’ratings is explained by the four sets of independent variables. However, the explained variance for school-level impact (34%) and knowledge-useimpact (44%) is higher. This finding may suggest that there are lesser problems of transfer for the direct use of the contents of the inservice programs during and immediately after the training, in other words during the implementation process.During the training teachers are willing to use the new ideas in the classroom, to discuss their improvement efforts and to stimulate their colleagues to adopt new ideas or materials. After some time teachersfind it difficult to translate new ideas, materials and teaching skills with an enduring effect in the daily lives of their schools or classrooms. Future research should pay more attention to useful strategies for managing transfer of training before, during, and after training (e.g., proactive planning for transfer, involvement of teachersin needsanalysis procedures,participation in transfer action planning, relapseanticipation, support structures in the work environment, rewards and incentives in the job situation, supportive organizational cultures). Noteworthy is that the importance attributed to school-focusedINSET is not supported in the impact findings at the school level. One may expect that school-focused INSET would result in higher impacts at this level than individualbased INSET, for example, in more productive school meetingsand better working relationships among teachers. In this respect no differences

Impact of Inservice Training

were found between these two types of inservice activities. Closer inspection of the inservice programs revealed that although these programs were attended by subteams of teachers or entire school staffs (sometimes from two or more different schools) these programs were not geared to the specific needs of these schools, they were offered by teacher trainer institutions as readymade packages. However, differences between school-focusedINSET and individual-based INSET were found at the classroom level and at the level of knowledge utilization. At these levels school-focused INSET appeared to have influence on changes in didactic procedures in the classroom and on the actual use of new skills in the staff’s instructional repertoire. The influence of school characteristics on inservice training impact is lim ited. Larger schools show less impact than smaller schools. This finding suggeststhat large schools have to create support structures and networks for discussion and information sharing to implement the new knowledge and skills. Frequent support from the local school counsellor leads to improved productivity of school meetings and to better working relationships between teachers. Support in the work environment is important for maintaining the learned skills on the job. With respectto the inservice program features variables like being well informed in advance, involvement of participants in planning, and the use of effective training components show no strong associations with impact scores.Effective training components as defined by Joyce and Showers (1988) are especially important for the acquisition of new teaching skills and teaching strategies. The inservice programs included in this study were not primarily directed at effective teaching. Extra hours spent on INSET activities are associatedwith higher classroom impact and knowledge utilization, while inservice program duration is associated with school impact. The variable that shows a positive impact on all three levels is the degree of practicality of the recommendations given by the INSET trainers or teacher educators. This outcome underlines the practicality ethic of teachers as described by Doyle and Ponder (1978).Change proposals that ‘are perceived as practical are more likely to be incorporated into school and classroom procedures. Three criteria influence this vision of practicality: (1) instrumentality (does the change

on Teacher Behaviour

315

proposal describe a procedure in terms that depict classroom or school contingencies?);(2) congruence (does the proposed change fit into the way the teacher normally conducts school or classroom activities), and (3) cost (what is the ratio between amount of return and amount of investment?). Teacher educators were well prepared to meet teachers’need for practicality. In the framework for examining training transfer as described by Baldwin and Ford (1988)the transfer processis described in terms of training input factors, training outcomes, and conditions of transfer. Training input factors include training design (principles of learning, sequencing, training content), trainee characteristics (ability, personality, motivation), and work environment characteristics (support, opportunity to use). Surprisingly, trainer characteristics are not included in Baldwin and Ford’s framework of training input factors. The results of this study suggest a strong positive relationship between the practicality of the trainer (teacher educator) and the effects of inservice training on the application of the contents of the inservice programs at classroom and school level. After an analysis of the transfer literature, Broad and Newstrom (1992) and Simons (1990) underline the trainer’s important role in the transfer process(e.g.,providing realistic work-related tasks, monitoring trainee progress, acting as a metacognitive guide, and stimulating trainee selfregulation capabilities). Findings stressing the trainer’s role in the transfer processsuggestthat future models of the transfer processshould pay attention to trainer variables. Discussion and information exchangeseemto be an important aspect of the implementation process. Sharing of ideas with other teachers resulted in improvement at the classroom level and in knowledge utilization. Hearing about their colleagues’instructional problems and solutions makes teachersfeel lessisolated and more confident about their ability to make changesin their classrooms. Sparks (1983) regards discussing application as a crucially important aspect of staff development and teacher growth and adds it as a separate training activity to the list of effective training components suggested by Joyce and Showers (1980, 1988). The way principals performed their steering functions contribute to the impact of inservice activities at the school level. Qualitative data

S. VEENMAN

316

et

al.

from our casestudy revealednotable differences in the ways principals steered the implementation process. The most successful schools had principals who were giving considerable direction and providing directional pressure and support while lim iting teachers’discretionary power. They steered their teachers to the intended innovation goals (Van Tulder, Van der Vegt, & Veenman, 1993). For successfulimplementation of inservice activities the principal plays a crucial role in goal setting and planning for application, for monitoring the action plans for using new knowledge and skills, and for setting up follow-up meetings after the training at periodic intervals for further information sharing, problem solving, and support of the implementation effort. This multi-case study also made clear that inservice training can only be useful if the program is well-connected to the specific school situation. Schools must take an active part in making inservicetraining programs effective instruments of improvement through acquisition, positioning in their organization, and modification of its design and content. It is apparent from this study that a number of aspects of teachers,their schools, their inservice programs, and their workplace conditions for implementing the contents of inservice activities are significantly associatedwith their use of knowledge and skills. Variables like school counsellor’s support, hours spent on INSET activities, degree of participation, teacher educators’subject matter knowledge and the provision of realistic work-related tasks, discussion and information sharing, the way the principals perform their steering functions to monitor the implementation process,and support structures in the work environment are subject to alteration to increase inservice training effectiveness.

National Foundation for Educational Research in England and Wales, Bristol. Broad, M. L., & Newstrom, J. W. (1992).Transfer offruining:

References

Rosenholtz, S. J. (1991). Teachers’ workplace: The social organizafion ofschools. New York: TeachersCollege Press. Simons, P. R. J. (1990).Transfervermogen (Transfer capacity). Nijmegen: University of Nijmegen. Sparks, G. M. (1983). Synthesis of research on stagdevelopment for effective teaching. Educational Leadership, 40,

Action-packed strategies to ensure high pclvofffiom rraining invesfmenfs. Reading: Addison-Wesley.

Doyle, W., & Ponder, G. (1978). The practicality ethic in teacher decision making. Inrerchange, 8, l-12. Fenstermacher, G. D., & Berliner, D. C. (1985). Determining the value of staff development. Elementary School Journal, 85, 28 1-3 14. Fullan, M. (1982). The meaning of educational change. New York: Teacher College Press. Gist, M. E., Bavetta, A. G., & Stevens,C. K. (1990).Transfer training method: Its influence on skill generalization, skill repetition, and performance level. Personnel Psychology. 43, 501-523. Huberman, A. M., & Miles, M. B. (1984).Innovarion up close: How school improvement works. New York: Plenum. Joslin, P. A. (1980). Inservice teacher education: A metuanalysis of the research. Unpublished doctoral dissertation, University of Minnesota. Joyce, B., & Showers, B. (1980).Improving inservice training: The messages of research. Educational Leadership, 31, 379-385.

Joyce, B., & Showers, B. (1988).Studem achievement through &ff development. New York: Longman. Kirkpatrick, D. L. (1987). Evaluation. In R. L. Craig (Ed.), Training and developmenr handbook (3rd ed., pp. 301-319). New York: McGraw-Hill. Lawrence. G.. & Harrison. D. (1980). Policv imolications of the research on the professional development of education personnel: An analysis of fifty-nine studies. In C. E. Feistritzer (Ed.), The 1981 report on educational personnel developmenr (pp. 145-168). Washington, DC: Feistritzer Publications. Leithwood, K. A., & Montgomery, D. J. (1982). The role of the elementary school principal in program development. Review of EdkationalReseaich,

Si, 369-339.

Little, J. W. (1982). Norms of collegiality and experimentation: Workplace conditions of school success.American Educational Research Journal, 19, 325-340.

Robinson. D. G.. & Robinson. J. C. (1989). Trainina for impact. San Francisco: Jossey-Bass. Roelob, E. (1993). Teamgerichre nascholing en coaching (Staff development and coaching). Doctoral dissertation, University of Nijmegen, The Netherlands. Roelofs, E., Raemaekers,J., & Veenman, S. (1991).Improving instructional and classroom management skills: Effects of a staff development programme and coaching. Sc/rool ,

“..

Eflectiveness and School Improvement. 2, 192-212.

Bacon, F. (1892). The essays of counsels civil and moral. London: George Routledge and Sons. Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions [or future research. Personnel Psychology, 41, 63-105. Berman, P. (1981). Educational change: An implementation paradigm. In R. Lehning & M. Kane (Eds.), Improving schools: Using what we know (pp. 253-286). Beverly Hills: Sage. Bolam, R. (1987, December). What is effective INSET? Paper presented at the annual members conference of the

65-72.

Sparks, D., & Loucks-Horsley, S. (1990). Models of staff development. In W. R. Houston (Ed.), Handbook of research on teacher education (pp.234-250). New York: Macmillan. Van der Vegt, R., & Knip, H. (1988).The role of the principal in school improvement: Steering functions for implemen-

Impact of Inservice Training tation at the school level. Journal of Reseurch and Development in Education. 22, 60-68. Van Tulder, M. A. T. M. (1992). Nascholing en onderw&vernieuwing (In-service training and educational change). Doctoral dissertation, University of Nijmegen, The Netherlands. Van Tulder, M., & Veenman, S. (1991). Characteristics of effective in-service programmes and activities: Results of a Dutch survey. Educafionol Studies. 17, 25548. Van Tulder, M., Van der Vegt, R., & Veenman, S. (1993). In-service education in innovating schools:, A multi-case study. International Journal of Qualitative Studies in Educo/ion (in press). Van Tulder, M., Veenman, S., & Sieben, J. (1988). Features of effective in-service activities: Results of a Delphistudy. Educational Studies, 14, 209-223. Veenman, S., Lem, P., & Roelofs, E. (1989).Training teachers in mixed-age classrooms: Effects of a staff development programme. Educafional Studies, 15, 165-180.

on Teacher Behaviour

317

Wade, R. K. (1984). What makes a deference in inservice reacher education: A meta-analysis of the research. Unpublished doctoral dissertation, University of Massachusetts. Wade, R. K. (1985). What makes a difference in inservice teacher education: A meta-analysis of research.Educational Leadership, 42, 48-54.

Walberg, H. J. (1991). Productive teaching and instruction: Assessingthe knowledge base. In H. C. Waxman SCH. J. Walberg (Eds.), Effective reaching: Current research (pp. 33-62). Berkeley: McCutchan. Walberg, H. J., & Genova, W. J. (1982). Staff, school, and workshop influences on knowledge use in educational improvement efforts. Journal of Educational Research, 76, 69-80.

Submitted 16 April 1993 Accepted 10 October 1993