Evaluation of the North Carolina NET program: Dissemination of materials

RESEARCH ARTICLE Evaluation of the North Carolina NET Program: Dissemination of Materials KAREN

L.

GRAVES,l MARyANN

C.

FARTHING,2 JANET M. TURCHI, AND SALLY A. SMITH

Department of Nutrition, School of Public Health, 315 Pittsboro St., 325H, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599 ABSTRACT A survey was conducted to determine how many of the nutrition education materials distributed through the North Carolina Nutrition Education and Training (NET) Program were in use in a sample of school systems, how each school system received these materials, and what procedures were used to distribute these materials within the system. The study sample consisted of a stratified random sample of eightyone school systems. The survey instrument, a self-administered questionnaire, was mailed to the person in the school system who was responsible for curriculum development. The following observations were made: the majority of the persons completing the survey held child nutrition/school food service positions and not curriculum specialist positions; only four of the twelve materials asked about were present in three-fourths or more of the systems; the procedure employed to distribute materials to schools within a system was related to the type of nutrition education material being used; and the child nutrition/school food service director was the individual most often (JNE 21:11-15, 1989) involved in materials dissemination.

education materials. In most cases, workshops were held at the regional level, to which key personnel within the school systems were invited. The personnel most often included an administrator, a school food service or child nutrition director, and key teachers. These day-long workshops were designed to introduce the materials and to train the participants to organize workshops at the local level, in order to introduce the material to teachers. In North Carolina, evaluation of the NET Program was not an integral component of program development. However, a statewide evaluation effort was initiated in late 1984. This report describes a study designed to document the distribution of the nutrition education materials from the central office of the Division of Child Nutrition to the local school systems. A companion paper (6) details the study designed to determine teacher perceptions of the effectiveness of both training and curriculum materials provided. Outcomes of other evaluation components can be found elsewhere (7).

INTRODUCTION The impact of new instructional materials cannot be measured until those materials have been adopted by the intended user. Provision of materials alone does not effectively disseminate a program (1, 2). Creation of awareness, establishment of a commitment, training, help with implementation, problem solving and monitoring, and evaluation have all been identified as critical to the adoption of an innovative idea (3-5). The Division of Child Nutrition of the State Department of Public Instruction in North Carolina utilized a decentralized approach to the dissemination of nutrition

METHODS AND MATERIALS Instrument Development. A self-administered questionnaire was designed to elicit information on the procurement and distribution of nutrition materials made available to the school system by the NET Program between 1980-1984. Question formats included best answer and multiple response close-ended items, as well as openended questions. The questionnaire was reviewed by an expert panel for content validity and then pilot tested in two school systems to determine whether the questions could be understood, and whether they actually elicited the information needed. Revisions were based on pilot test data gathered. Open-ended questions were precoded by a member of the research team before the data were entered on the computer.

Division of Epidemiology, School of Public Health, 611 Beacon St., S.E., University of Minnesota, Minneapolis, Minnesota 55455. 2 Correspondence concerning this article should be directed to Dr. Farthing, Department of Nutrition, School of Public Health, 315 Pittsboro St., 325H, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599. 1

Selection of Study Sample. Coordination of educational resources within North Carolina is accomplished

0022-3182/89/2101-0011$02.00/0 © 1989 SOCIETY FOR NUTRITION EDUCATION 11

12

Graves et al. / NET MATERIALS DISSEMINATION

Table 1. Nutrition education materials, percent of sample with material, method by which material is acquired from the state office and method used to distribute to schools within the local school system.

Nutrition Education Material

A.

B.

Purchased Material Image: A Manual for Managers Introductory Principles of Nutrition Nutrition: Concepts and Controversies Developed Material Annotated Bibliography of Nutrition Education Resources Using Food Experiences to Reinforce Academic Objectives Nutrition Education Supplement to Textbooks Nutrition Discoveries: A Resource for Teachers of Seventh Grade Science Nutrition in Developing Countries: A Resource for Teachers of Seventh Grade Social Studies Nutrition Education Resource Package: A Resource for Teachers of Junior High Health Education The Agribusiness (Industry in North Carolina): Resources for Teachers of Eighth Grade Social Studies Nutrition Issues and the Political Process: A Resource for Teachers in Ninth Grade Social Studies Teens, Foods, Fitness and Sports

through eight regional administrative units. The school systems within the state were stratified by regions, and a random sample was drawn from each stratum so that the number of systems selected was proportionate to the number of school systems existing in the state. This sampling design allows generalizations not only at the regional level but also to the state of North Carolina. A stratified random sample of 60% (or 81) local school systems was selected. Implementation. The survey was mailed to the person in each school system who was responsible for the curriculum development, as identified in the Department of Education state directory. A cover letter was also sent to the school food service/child nutrition director to inform herlhim about the survey. A telephone follow-up procedure, which included two contacts made approximately two weeks apart, was employed to retrieve the late surveys. Seventy-five of the eighty-one school systems (93%) responded. Four of the eight regions had a response rate of 100%, with the lowest completion rate being 80%. Characteristics such as system size and location of the six school systems which did not submit a completed survey were compared with those school systems responding to the survey. No differences between the two categories were observed. Therefore, it was assumed that the non-response bias was negligible, and the sample was weighted according to the original strata weights, without adjustment for non-responders. Descriptive statistical analyses were conducted using

School Systems With Material

How Received

How Sent

With Training

Without Training

Local Workshop

Select Teachers

81 45 86

71 61 72

17 21 13

68 46 48

24 35 34

58 66 78 47

49 65 66 49

36 21 20 31

29 67 66 29

16 41 49 57

47

51

31

37

65

53

57

30

27

64

47

56

24

44

60

40

50

25

35

66

83

71

17

24

53

procedures outlined in the SAS (Statistical Analysis System) statistics package (8).

RESULTS AND DISCUSSION Description of respondents. The responder, the person who completed the returned survey, was in most cases different from the person to whom the survey was directed. The majority of individuals, 66%, held child nutrition/school food service director positions; this observation was consistent within all eight regions. Only 21 % of the respondents identified themselves as holding school administrative positions. On the average, respondents had spent seven years in their current position, suggesting that at least half had been in that particular position since the inception of the NET program in 1978. Nutrition education materials. The Division of Child Nutrition has sought to distribute twelve specific nutrition education documents to all school systems within the state. A listing of the materials, the availability of these materials within school systems, the principal method of receiving the materials from the central office, and the procedures used to distribute these materials to schools within the system are presented in Table 1. The materials can be categorized as either materials purchased for distribution or materials developed under contract to the NET Program. All the purchased materials are either teacher resource materials or material de-

J. ofNutr. Educ. Vol. 21, No.1 signed for the use of school food service managers; whereas, the developed materials were designed to meet specific needs, and included curricula, resource guides, and an annotated bibliography. Two documents were developed for grades K-6: Using Food Experiences to Reinforce Academic Objectives and Nutrition Education Supplement to Textbooks. The other curriculum materials were targeted for grades 7-9, with the exception of Teens, Foods, Fitness and Sports which is targeted to athletic personnel in high schools. Only four of the twelve materials available were present in 75% or more of the school systems. Acquisition and distribution of materials by the school system. The Division of Child Nutrition employed two principal techniques to distribute the materials, one involved training, the other involved mailing the material either upon request or as a mass mailing to selected school systems. This latter method did not provide any training in the use of the materials. The materials were distributed primarily through workshops. The majority of these training workshops were conducted at the state or regional level. The four materials which were identified as being available in at least 75% of the school systems were those materials which were reported to be most frequently acquired at a training workshop. In general, one-fifth of the sample reported receiving materials in a manner which did not provide any training in their use. The majority of the school systems reported use of several methods to disseminate nutrition education materials to individual schools. The procedures used appeared to be related to the type of nutrition education material to be distributed. Materials which can be considered grade specific were more likely to be discussed with selected teachers and provided to them individually at that time. The two K-6 curricula and the resource guide for the school food service manager were more likely to be made available through training workshops. The three resources for teachers were more likely to be found in resource centers within individual schools. The school food service/child nutrition director was reported by 84% of the sample as the person most often involved in the dissemination of nutrition education materials at the local level, an observation consistent across all eight regions. A locally sponsored training workshop was the method most frequently employed to inform teachers and school food service managers of any new nutrition education materials. The procedure least often used to inform school personnel was an organized curriculum meeting. The school systems acquired nutrition education materials from a variety of sources. As might be expected, 90% of the school systems indicated that they received materials from the Department of Public Instruction. Other important sources of materials included the Dairy

February 1989

13

Food and Nutrition Council, food companies and the Cooperative Extension Service. Six of the school systems responding noted that the North Carolina School Food Service Association was also an important resource. Improving the dissemination process. Approximately 60% of the respondents made recommendations concerning modifications to facilitate the dissemination of nutrition education materials from the Division of Child Nutrition to the school system and within the local school system itself. Only a small number of persons indicated that the present method of distribution of materials was acceptable. The suggestions for improvement included the following points: use of the health education coordinators to facilitate dissemination; more involvement of school administrators; more promotion of materials by the curriculum directors; a better system to alert teachers to new materials; creation of a system to monitor usage of the materials. This study indicates that many of the resources needed for the successful incorporation of nutrition education into the school curriculum are probably not available to the primary deliverer, namely the teacher. The NET Program, like many other educational programs, is multilevel in nature (9). The delivery of services occurs at the classroom level, but management of resources will occur to varying degrees at a number of other levels. In North Carolina, as elsewhere, nutrition education resources are channeled through the local school districts to teachers. The acquisition and management of these resources at the local school system level will affect the success of the NET Program. One important resource is the availability of program support materials. Many of the materials were not readily available at the level of the school systems. There is a dearth of information on the success of other state NET Programs in their dissemination of material to the school system level. Some indication of success may be garnered from those studies which assessed teacher satisfaction with availability of educational materials. Our results do not support those of Olson et al. (10) who observed that, in their sample of teachers, approximately one-half had nutrition education materials available to them. However, our results are consistent with other studies (11, 12) which indicated that teachers felt that they did not have access to sufficient nutrition information. It is well-documented that the availability of quality materials and ready access to these materials act as incentives to provide nutrition education to students (10). The establishment of an effective monitoring system would allow for the detection of those school systems lacking the necessary educational materials. In addition, the persons responsible for NET Program implementation often were not the individuals who are recognized as being responsible for overall curriculum implementation. Other researchers (11, 13) have noted

14

Graves et al. / NET MATERIALS DISSEMINATION

that this may be one of the obstructions to effective program implementation. School food service personnel are not generally recognized as nutrition education resources. Several of the suggestions made by the respondents stress the need for greater involvement of the curriculum director. There have been many attempts both in North Carolina (7) and elsewhere (10, 13) to involve in NET Program administration individuals who are directly involved in curriculum adoption, but commitment by such persons has been limited and the child nutrition director has by default become the focal point of nutrition education at the school system level. The extent of school system involvement in the nutrition education process is largely determined by the enthusiasm of the child nutrition director and his or her staff. In North Carolina, several school systems have hired a health education coordinator, and it is to be hoped that nutrition education efforts will be revitalized if and when the NET Program at the school comes under this person's jurisdiction. To date, the health education coordinators have shown a strong positive support for the incorporation of nutrition education into health education programs. This is recognized by the respondents, who suggest that this person assume a more central role. In this study, the material's content and level predicted the method of diffusion to the primary user, either through a workshop or by direct discussion with individual teachers. The School Health Evaluation conducted by Abt Associates Inc. (2) clearly demonstrated that success in the classroom was dependent upon several factors, one being the amount of in-service training provided. Teachers with full training more completely implemented health instruction than did those with partial training, who, in turn, implemented programs more fully than did those with no training. It seems evident that there is a need to re-evaluate the strategies currently employed to reach teachers, as provision of materials alone or in combination with a workshop may not suffice to encourage program adoption. One-time workshop attendance rarely leads to the successful implementation of a nutrition education program (14). A better strategy is to provide opportunities for follow-up training, so that problems which arise during implementation can be solved and more learning can occur (3). In addition, a planned series of training sessions would provide tangible evidence of a school district's commitment to nutrition education.

CONCLUSIONS In summary, dissemination is an essential part of any NET Program. Effective dissemination of materials and resources requires time, attention, planning and monitoring. To assure the adoption and appropriate use of

nutrition curriculum materials, the commitment of the school's administration should be secured. Additionally, the implementor must be made aware of those factors which lead to the successful adoption of materials, and the user of the materials should be assured of adequate assistance. 0

NOTES AND REFERENCES 1 Stearns, M. and C. Norwood. Evaluation of the field test of project information packages. Menlo Park, CA: Stanford Research Institute, 1977. 2 Connell, D.B., R.R Turner, and E.F. Mason. Summary of the findings of the School Health Education Evaluation: Health promotion effectiveness, implication and costs. Journal of School Health 55:316-323, 1985. 3 Loucks, S. F. Planning for dissemination. Vol. IX Implications for Action. Andover, MA: The Network, Inc., 1982. 4 Banta, T.W., J.L. Cunningham, W.W. Jozwiak, M.P. McCabe, and J.D. Skinner. Adapting a nutrition education evaluation to field and political realities. Journal of Nutrition Education 17:41-43, 1985. 5 U.S. General Accounting Office. Reports to the Secretary of Agriculture. What can be done to improve nutrition education efforts in schools? Gaithersburg, MD, 1982. 6 Farthing, M.A.C., K.L. Graves, J.M. Turchi, and S.A. Smith. Evaluation of the North Carolina NET Program: The Teacher's perspective. Journal of Nutrition Education 21:5-10, 1989. 7 Farthing, M.A.C., K.L. Graves, J.M. Turchi, and S.A. Smith. Nutrition Education and Training Program Evaluation: Project Report 1984-85. NET Program, Division of Child Nutrition, NC Department of Public Instruction, Raleigh, 1985. 8 SAS Institute Inc. SAS User's Guide: Statistics, Version 5 edition, Cary, NC: SAS Institute, 1985. 9 Hanson, RA. and RE. Schute. Rethinking evaluation design for nutrition education programs. Journal of Nutrition Education 13:8&89, 1981. 10 Olson, C.M., E.A. Frongello, and D.G. Schardt. Status of nutrition education in elementary schools: 1981 vs. 1975. Journal ofNutrition Education 18:4f.h54, 1986. 11 Soliah, L.L., G.K. Newell, A.G. Vaden, and A.D. Dayton. Establishing the need for nutrition education: II. Elementary teachers knowledge, attitudes and practices. Journal of the American Dietetic Association 83:447-453, 1983. 12 Gillespie, A. H. Evaluation of nutrition education and training minigrant programs. Journal of Nutrition Education 16:8--12, 1984. 13 St. Pierre, R.C. and V. Resmovic. An evaluation of the national Nutrition Education and Training Program. Journal of Nutrition Education 14:61---{)6, 1982. 14 Huberman, A.M. and D.P. Crandall. Policies, people and practices: Examining the chain of school improvement. Vol. IX: Implications for action. Andover, MA: The Network, Inc., 1982.

RESUME Un sondage a Me mene aupres d'un echantillon du reseau scolaire de la Caroline du Nord mn d' evaluer la quantite de materiel d' education en nutrition distribue par Ie biais du programme d' education en nutrition et formation (NET) de cet etat. Ce sondage evaluait aussi quelle quantite du materiel global recu etait en usage, qui avait achemine ce material educatif, et queUes procedures avaient ete utilisees pour Ie distribuer dans Ie reseau scolaire. L' enquete par sondage consistait en un echantillonage stratifie, pris au hasard de quatre-vingt un reseaux scolaires. L'instrument d' enquete, un questionnaire auto-gere, avait ete poste au responsable des programmes d' etudes du reseau scolaire. On observa ce qui suit: la majorite des personnes ayant com-

J. ofNutr. Educ. Vol. 21, No.1 plete Ie questionnaire etaiant a la fois responsables de l'education en nutrition de l' enfant et du service alimentaire, et non du programme d'etudes; seulement quatre des douze documents educatifs sur lesquels portaient les questionnaires se trouvaient dans trois-quart ou plus des reseaux scolaires; les methodes de distribution utilisees dans ces reseaux dependaient du type de material de l' education en nutrition utilise dans les ecoles, et la personne ala fois responsable de l'education en nutrition aupres des enfants, et du service alimentaire etait Ie plus souvent impJiquee dans la dissemination du maIJNE 21:11-15, 1989) teriel educatif. Source: Louise Desaulniers

RESUMEN

Se realiz6 una encuesta en una selecci6n de muestra de escuelas de la Carolina del Norte para determinar 1) la utilizaci6n de materiales pedag6gicos relativos a la nutrici6n, distribuidos por el programa educativo en nutrici6n y formaci6n profesional (NET) de este estado, 2) de que modo

February 1989

15

llegaron estos materiales a cada complejo escolar y 3) c6mo se distribuyeron tales materiales. La encuesta se llev6 a cabo en ochenta y uno sistemas escolares estratificados, escogidos al azar. EI instrumento del sondeo, un cuestionario auto-administrado, se despach6 por correo al responsable del program a de estudios en cada sistema. Se observ6 siguiente: la mayor parte de los oficiales que respondieron a la encuesta fueron los encargados de servicios alimentarios 0 de la educaci6n en nutrici6n de menores, y no del programa de estudios; s610 cuatro de los doce documentos pedag6gicos tratados en el cuestionario se encontraban en las tres cuartas partes 0 mas de estos sistemas educativos; el metodo empleado en la distribuci6n de materiales a escuelas dentro del sistema se relacionaba al tipo de materiales utilizados en el sistema para la instrucci6n en nutrici6n; y el individuo encargado tanto de nutrici6n de menores como del servicio alimentario de la escuela era el que con mas frecuencia diseminaba los materiales indicados. IJNE 21:11-15, 1989) Translated by H. T. Sturcken

LACK OF ADVERSE REACTIONS TO IRON·FORTIFIED FORMULA

Nelson et al. contrasted the effects of iron-fortified and non-iron-fortified formulas on behavior and stool characteristics in thirty-one infants (Pediatrics 81:360-364, 1988). The investigation involved three studies, each lasting 12-15 weeks, and included one and sixteen week old infants. All three studies employed a cross-over design (infants alternated between the high or low iron formulas for two week intervals). Two of the studies were double-blind. Parents made daily observations and reported on a four point scale (absent to severe) on their infants' fussiness, cramps, regurgitation, flatus, and colic, as well as stool characteristics of frequency, consistency (six descriptors), and color (five descriptors). The behavioral factors did not vary significantly according to whether the infants received high or low iron formula (probabilities ranged from 0.41 to 0.99). Cramps were virtually unreported, and flatus, regurgitation, and fussiness, while reported more frequently, were rarely severe. Frequency of stools varied with age but not with formula (probabilities 0.76 to 0.96). Reports of stool consistency were predominantely "soft" or "loose," but also did not differ significantly according to type of formula (probabilities 0.08 to 0.47). The only stool characteristic or behavior that showed a significant difference was stool color, in which the iron-fortified formula was associated with darker and/or greener stools. Results of this study suggest that considerations for the use of iron-fortified formula should be based on need and not on the fear of adverse reactions to the high iron. (5 references) Summary written by Garry Auld