- Email: [email protected]
5 A Day for Low-Income Families: Evaluation of an Advertising Campaign and Cooking Events
5 A Day for Low-Income Families: Evaluation o f an Advertising Campaign and Cooking Events M A R C I AWEAVER,',^ M E R E D I T HP O E H L I T Z ,A' N D S H I R L E YHUTCH IS ON^ 'Department of Health Services, University ofwashington, Seattle, Washington 98195; *Seattle-King County Department of Public Health, Seattle, Washington 98104
ABSTRACT This article reports the results of the evaluation of a 5 A Day demonstration project with an advertising campaign and cooking events for low-income families with young children.The goal of the evaluation was to provide meaningful feedback on how to improve the interventions, rather than to conduct a definitive trial on their effectiveness. Brief, selfadministered questionnaires were developed to measure two outcomes: (1) recognition of the 5 A Day message and (2) selfreported fruit and vegetable intake.To evaluate the advertising campaign, the results from the two intervention neighborhoods (n = 694) were compared with those from two control neighborhoods (n = 771). It was possible to evaluate the cooking events in relation to the advertising campaign because people who attended the cooking events were also exposed to the advertising campaign.The results from a sample of people who attended the cooking events (n = 188) were compared with those from the post-test sample from the intervention neighborhoods (n = 333). Recognition of the 5 A Day logos increased during the advertising campaign but the effect did not exceed the effect of the National Cancer Institute and state 5 A Day media campaigns. Self-reported fruit and vegetable intake did not change.The cooking events were significantly more effective than the advertising campaign in increasing recognition of the 5 A Day logos and understanding of the 5 A Day message.
INTRODUCTION O n e of theyear 2000 Health Goals is for all Americans to eat at least five servings of fruits or vegetables a day.' This goal is sometimes referred to as "5 A Day." There is reason to believe, however, that people with low income may have to ................................................... This project Wac funded in part with federal fund? from the Food and Nutrition Service, U.S. Department ofAgriculture, under Cooperative Agreement No. 58-3198-4-053. The contents of this publication do not necessarily reflect the view or policies of the U. S. Department ofAgr~culture,nor does mention of trade names, commercial products, or organizations Imply endorsement by the U.S. Government. Address for correspondence: Marcia Weaver. Ph.l)., 1122 E. Pike St.,#1132, Seattle,WA 98122;Tel: (206) 296-0241; Fax: (206) 205-5314; E-mall: [email protected]. 01999 SOCIETY FOR NUTRITION EIIUCATION
make equal or larger increases than people in other income groups in the number of servings of fruits and vegetables they eat to meet theyear 2000 Health Goal. Some researchers have shown that the number of servings of fruits and vegetables that people eat is similar across income groups,* while others have shown that it is lower among people with lower income or less ed~cation.'.~ There is also reason to believe that access to nutrition education for people with low income may be less than or equal to access for people in other income groups. Considering nutrition education in general, a 1995 literature review recommended that nutrition education programs devote more effort to reaching disadvantaged groups, especially those .~ who are less educated and have lower i n ~ o m e Considering the 5 A Day message in particular, the National Cancer Institute's (NCI) "5 A Day for Better Health" campaign was targeted at adults who had recently increased their consumption of fruits and vegetables but were still eating only 1.5 to 2.5 servings of fruits and vegetables per day.hWithin the stages of change f r a m e ~ o r k ,the ~ N C I media campaign focused on people who were preparing to eat more or were eating more fruits and vegetables, rather than those who were in the precontemplative or contemplative stages. Although the NCI media campaign did not exclude people with low income, the target audience included a disproportionate number of women who were employed outside the home." Further, the NCI media campaign did not face the challenge of calling the attention of people with low income to the 5 A Day message or showing them easy and inexpensive ways to eat more fruits and vegetables. To improve nutrition education for people with low income, the U.S. Department ofAgriculture (USDA) funded 10 Community Nutrition Education Cooperative (CNEC) Demonstration Projects to explore strategies to integrate nutrition education services with food assistance program^.^ Food assistance programs offer an opportunity to educate a large number of people with low income about nutrition because they serve several million people. In 1994, the average number of people served per month included 28.9 million people in the Food Stamps program, 6.5 million people
162
Weaver et a1./5 A DAY F O R LOW-INCOME FAMILIES
in the Special Supplemental Nutrition Program for Women, Infants and Children (WIC), 14 million students in the free and reduced-price School Lunch Program, and 5.2 million students in the free and reduced-price School Breakfast Program.9 Although USDA has supported nutrition education for more than 20 years through programs such as the ~ Expanded Food and Nutrition Education P r ~ g r a m 'and WIC," the C N E C initiative sought to encourage collaborative and community-based nutrition education interventions. T h e Seattle Nutrition Action Consortium (SNAC) received C N E C funding for the Taste and Health Project to implement a 5 A Day demonstration project. SNAC included representatives of each major food assistance program (Food Stamps,WIC, and school meal programs) as well as the Head Start/Early Childhood Assistance and Education Program, food pantries, nonprofit organizations, and businesses. The representatives decided to focus on a common nutrition message in each of their programs to maximize the impact of limited nutrition education resources.They selected the 5 A Day message in order to extend the N C I media campaign to low-income families with young children and because nutrition education materials were available from the NCI and state 5 A Day media campaigns. This article reports on the evaluation of two of the Taste and Health Project's 5 A Day interventions: an advertising campaign and cooking events. The evaluation questions for the advertising campaign were "Did the Taste and Health advertising campaign augment the effects of the NCI and state media campaigns on: (1) recognition of the 5 A Day message and (2) self-reported fruit and vegetable intake?" The evaluation question for the cooking events was "Did the 5 A Day cooking events significantly increase recognition of the 5 A Day message relative to the advertising campaign?"
METHODS Interventions.
Advertising. The goal of the advertising campaign was to extend the NCI and state media campaigns to low-income families with young children using advertisements targeted to them.The advertising campaign included (1) interior bus signs on bus routes in the intervention neighborhoods; (2) advertising and articles in community newspapers; (3) 5 A Day messages on school menus, grocery bags, milk cartons, and flyers; and (4) posters and promotional materials displayed and distributed by SNAC members. Some of the materials were obtained through NCI and state media campaigns, and others were developed by SNAC's Nutrition Messages Work Group and the staff of this project. Cooking events. The goal of the cooking events was to call attention to the 5 A Day message with colorful displays and the smell and taste of food samples. Outreach workers also
talked with people and showed them ways to serve fruits and vegetables that were easy to prepare and inexpensive. The events were conducted at food assistance program offices and other community settings and included a featured recipe, discussion of key points (such as "fruits and vegetables are the original fast food"), samples of the food, and promotional materials for the participants to take home (such as recipes, brochures, refrigerator magnets, coloring books, and stickers). The outreach workers were not necessarily nutrition professionals; they were people who received ongoing training in nutrition education and had an ability to work with community groups. There were two types of cooking events: demonstrations and classes. A demonstration served a large number of people at sites such as food pantries and the waiting rooms of food stamp offices. People could visit the displays at their discretion to talk with the outreach worker and sample food that was prepared on site. The demonstrations were developed to reach people who came to a site for another reason and could be attracted to the intervention, especially during a waiting period. In contrast, a class was taught to small audiences at sites such as W I C clinics, other health clinics, a homeless shelter for mothers and young children, and a methadone treatment center for pregnant women.The classes lasted 30 to 60 minutes. During this time, an outreach worker prepared a food sample from a featured recipe, offered tips on preparing fruits and vegetables and on encouraging children to eat more of them, and answered questions from the audience.
Target population. The target population for the interventions were low-income families with young children. SNAC was encouraged to focus on two neighborhoods for the demonstration project, so that it would be possible to compare the outcomes of the intervention neighborhoods to other neighborhoods. SNAC selected one inner-city and one residential neighborhood with a high incidence of poverty. Household income was less than 200% of the federal poverty level for 46% of the residents of the inner-city neighborhood and 35% of the residents of the residential neighborhood." Evaluation design. Several special features of the evaluation stemmed from the nature of the C N E C demonstration projects. SNAC was encouraged to try a variety of interventions and modi@ them based on experience. Consequently, the goal of the evaluation was to provide meaningful feedback on how to improve the interventions, rather than to conduct a definitive trial on their effectiveness. In addition, funding for the evaluation was relatively modest. Consequently, the researchers developed a strategy to sample a cross-section of people at food assistance program offices and other community settings, and self-administered questionnaires to measure two outcomes.
Advertising. Given that the goal of the advertising campaign was to extend the NCI and state media campaigns, and
May June 1999
Journal of Nutrition Education Volume 31 Number 3
that the NCI campaign was targeted at people who were preparing to eat or were already eating more fruits and vegetables, the evaluation addressed whether or not the intervention affected eating habits as well as recognition of the 5 A Day message. This part of the evaluation had a nonequivalent control group design13 in which the outcome variables of the two intervention neighborhoods were compared to two control neighborhoods before and after the intervention. One innercity and one residential control neighborhood were selected to match the demographic characteristics of W I C participants in the two intervention neighborhoods. The pre/post comparison controlled for differences between the intervention and control neighborhoods that predated the intervention and showed changes in the intervention and control groups over time.
Advertising versus cooking events. Given that the goal of the cooking events was to call attention to the 5 A Day message, the evaluation addressed whether the intervention affected recognition of the 5 A Day message, but not eating habits. It was possible to evaluate the cooking events in relation to the advertising campaign because people who attended the cooking events had the same opportunities to be exposed to the advertising campaign as the respondents to the advertising survey.The outcomes of cooking event participants were compared to the post-test sample of the intervention neighborhoods for the advertising survey. Sampling strategies and procedures.
Data were collected from the two intervention Advertisinx. and two control neighborhoods before the intervention began in November/December 1995 and 5 months after the intervention began in June 1996.A field data collection manager spent 1 day asking everyone at a site to complete a questionnaire, keeping track of refusals.There were four sites (two different W I C clinics, a food stamp office, and a food pantry) in each of the four neighborhoods, for a total of 16 sites.The sample excluded people under 18, staff, and people who did not speak English or Spanish.Two of the field data collection managers were fluent in Spanish and were assigned to sites where many of the respondents spoke Spanish, such as a food pantry adjacent to a church with a Hispanic congregation. For the pretest, there were 775 respondents of 1076 people at the 16 sites, for a response rate of 72%. For the posttest, there were 690 respondents of 1095 people at the 16 sites, for a response rate of 63%. The total sample size was 1465.
Cooking events. The sample included people who attended a cooking event between June 10 and July 31,1996.A field data collection manager attended the cooking event and asked every person who attended it to complete a questionnaire. Attendance at a demonstration was defined as approaching the outreach worker to sample food or talk with
163
him/her. Attendance at a class was defined as attending part of a class, including people who arrived late or left early.The sample excluded people under 18, staff, and people who did not speak English or Spanish. Some of the field data collection managers were fluent in Spanish and were assigned to cooking events where many of the people spoke Spanish. There were 188 respondents of 263 people who attended cooking events, for a response rate of 72%. Among the respondents, 117 attended demonstrations and 73 attended classes. Measurement instruments.
Development. The researchers developed self-administered questionnaires in English and Spanish. The first draft of the questions was based on a review of the literature. Then the questions were pilot tested with a total of 42 respondents at food assistance program offices. Feedback from the respondents was used to assess the questions for comprehension and the questions were revised accordingly. The final version of the questions was written at the fourth-grade reading level according to a readability analysis using the Rightwriter software.14 Recognition of the 5 A D a y message. Recognition of two different logos was measured: 5 A Day sun and 5 A Day hand. The 5 A Day hand, which was developed by NCI to promote 5 A Day week in the early 1990s, was adapted and used (with permission) on most of the Taste and Health Project's 5 A Day promotional materials. Recently, the 5 A Day hand was not used by the NCI and state media campaigns, so it was unique to the Taste and Health Project during the first 5 months of the advertising campaign. The questions were based on a questionnaire developed for the national media campaign.15Respondents were shown both logos and were asked "Which ads have you seen in the last few months? Please mark all that you have seen."Then the respondents were asked, "What does '5 A Day' mean to you?" and given a blank space in which to write an answer. The respondent was considered to have understood the meaning of 5 A Day if hidher answer included at least the word fruit or vegetable. Pretest data were used to test for ceiling effect.The questions did not have a ceiling effect because fewer than 25% of the pretest respondents had seen the ads or understood the meaning of 5 A Day. Self-reported fruit and vegetable intake. O n e of the challenges of the evaluation was to design self-administered questions to estimate the average number of servings of fruits and vegetables per day for a population. Two measures were included in the pretest with the goal of selecting only one for the post-test. Measure 1 was "How many servings of fruits and vegetables did you eat yesterday?" followed by information on serving sizes and a list of more than 25 fruits and vegetables. Measure 2 consisted of four modified questions from the Behavioral Risk Factor Surveillance System (BRFSS)
164
Weaver et a1./5 A DAY FOR LOW-INCOME FAMILIES
Fruit andvegetable M ~ d u l e . ' ~ The , " BRFSS was designed for telephone interviews that were too expensive for this evaluation, so the questions and response categories were modified for self-administered questionnaires (Shirley Beresford, personal communication, 1995). In general, Measure 1 appeared cognitively easier than Measure 2, but Measure 2 had the advantage of yielding results easy to compare with other BRFSS data. Pretest data were used to assess the reliability, convergent validity, and respondent burden of the measures.The assessment did not show that one measure was clearly superior to the other, so both measures were retained for the post-test. The assessment of reliability (Cronbach's alpha) was only relevant for Measure 2, which combines several questions about fruits and vegetables.The alpha for that measure was .77 and the corrected item-total correlation was greater than .4 for all four items, showing that the four questions could be combined into a single measure.The convergent validity tests gave mixed results.The correlation coefficient for the rank of cases by measure was .42 and significant (p < ,001). In contrast, Measure 1 showed that the difference in self-reported fruit and vegetable intake between the respondents in the intervention and control neighborhoods was significant (5.0 and 4.2, respectively; p value =. 08), but Measure 2 did not (2.6 and 2.5, respectively; p value = .54). Also, the number of servings per day was nearly twice as high according to Measure 1 than to Measure 2. Finally, the response rate was 97% for Measure 1, compared to 86% for Measure 2, showing that Measure 1 was less of a burden for respondents than Measure 2. Bivariate statistics. Chi-square tests were performed for two sets of comparisons of demographic characteristics: (1) a comparison of the post-test sample of intervention and control neighborhoods for the advertising survey and (2) a comparison of the cooking event participants and the post-test sample of the intervention neighborhood for the advertising survey.
to the residential intervention neighborhood, respectively. The covariates food stamp office andWIC controlled for the sampling strategy but did not have interactions with other factors because some of the cells were empty. All analyses were weighted to adjust for differences in the number of respondents at each site.
Advertising versus cooking events. The coefficients for the variables event and eventlresidential measured the effects of the cooking events and the effects that were specific to the residential intervention neighborhood, respectively.
RESULTS Advertising. There were significant differences in some demographic characteristics between the post-test sample of intervention and control neighborho0ds.A~shown inTable 1, the percentage of people who self-identified as black was significantly higher, the percentage of those who self-identified as white was significantly lower, and the percentage of those with one or more children was significantly lower for the intervention than the control neighborhoods.There was also a significantly lower percentage of people in the post-test sample than the pretest sample of control neighborhoods who completed the questionnaire in English (not shown). Consequently, the following variables for demographic characteristics were included in the multivariate analyses: black, one or more children, and English questionnaire. (A variable for white race/ethnicity was not included because the vari-
Table 1.
Comparison of demographic characteristics (percentages)
between the post-test sample of intervention and control neighborhoods for the advertising survey. Demographic Characteristics
Intervention
Control
p Value
Ethnicity
Multivariate analysis. Logistic regressions for measures of recognition of 5 A Day message (dichotomous variables) and ordinary least squares (OLS) regressions for self-reported fruit and vegetable intake (continuous variables) were performed. To validate the results, analyses of variance of the continuous variables were also performed, and those results on the effects of the interventions were identical to the results reported below. There were some differences in demographic characteristics between samples. Additional covariates were included in the multivariate analysis to adjust for these differences; these covariates are listed with the results on demographic characteristics for each intervention.
Advertising. The coefficients for the variables interventiodpost-test and interventiodpost-tesdresidential measured the effects of the intervention and effects that were specific
Asian Black Hispanic Native American White Multiethnic No response English questionnaire One or more children 12 or more years of school Employed full or part time Nutrition books at home Sample size Chi-square tests were performed for the comparisons of demographic variables. *p < .05; **p < .01.
May
Journal of Nutrition Education Volume 31 Number 3
able black adequately represented this difference in demographic characteristics between the two samples.) To the extent that comparable data were available, it appeared that the post-test sample of the intervention neighborhoods was representative of participants in food assistance programs in those neighborhoods. For example, the distribution of race/ethnic identity of W I C participants in the intervention neighborhoods was similar to the distribution reported in Table 1 for those neighborhoods. In August 1995,15% ofWIC participants were Asian, 37% were African Americadblack, 11% were Hispanic, 25% were white, and 12% gave other responses.Although the distributions are not identical, both samples were diverse, with a minority of people who self-identified as white. Differences between the distributions may reflect that the evaluation included participants in the food stamp program and food pantries in addition to the W I C program and excluded people who did not speak English or Spanish, as well as differences in the ways the questions about race/ethnicity were asked. Table 2 shows a comparison of outcomes for the post-test sample of respondents in the intervention and control neighborhoods. O f the intervention and control respondents, 15% and 8%, respectively, recalled seeing the 5 A Day hand, and 38% and 28%, respectively, recalled seeing one or both ads. The post-test average of self-reported fruit and vegetable intake was 5.3 and 4.6 for the intervention and control respondents, respectively, for Measure 1, and 2.9 and 2.4 for Measure 2. The multivariate analysis combined the pretest and posttest samples to control for differences between the intervention and control neighborhoods that predated the intervention. As shown in Table 3, recognition of the 5 A Day sun increased during the intervention, but the effects of the Taste and Health advertising campaign did not exceed the effects of the NCI and state media campaigns. The coefficients of the variable post reflected the effect of the NCI and
Table 2.
June 1999
165
state media campaigns and showed that there was a significant effect on recognition of the 5 A Day sun and one or both 5 A Day 1ogos.The coefficients of the variable interventiodpost reflected the effects of the Taste and Health advertising campaign and were not significant. The one exception was that respondents in the residential intervention neighborhood (see the coefficient for the variable interventiodpost/residential) were significantly more likely to recognize the 5 A Day hand, but they were not more likely to recognize one or both 5 A Day logos or understand the meaning of 5 A Day. Note that people in the intervention neighborhoods ate more servings of fruits and vegetables than people in the control neighborhoods both before and after the intervention. The coefficient of the variable intervention reflected differences between the intervention and control neighborhoods that predated the interventions, and it was positive and significant for Measure 1, the number of servings of fruits and vegetables consumed yesterday. The coeff~cientsof the variables food stamp offlce and W I C reflected differences in outcomes between respondents at these sites and food pantries (the omitted dummy variable). Respondents at food stamp offices and W I C clinics were significantly more likely than respondents at food pantries to understand the meaning of 5 A Day, and they ate more servings of fruits and vegetables according to Measure 2. The results for the demographic variables showed that respondents with one or more children were more likely to understand the meaning of 5 A Day as well as to recognize the 5 A Day sun and one or both 5 A Day 1ogos.The results for the other two demographic variables did not show consistent differences across racial/ethnic groups. Blacks had a significantly lower outcome than other respondents for one of the six outcome measures. Respondents who completed the questionnaire in English were more likely to recognize the 5 A Day sun but ate fewer servings of fruits and vegetables according to Measure l than those who completed the questionnaire in Spanish.
Recognition of the 5 A Day message, and self-reported fruit
and vegetable intake of the post-test sample of intervention and control neighborhoods for the advertising survey.
Intervention
Control
Recognition of 5 A Day message Have you seen: 5 A Day hand?
15 %
8%
5 A Day sunrise?
29 %
23 %
38 %
28 %
22 %
18 %
One or both ads? Understand the meaning of 5 A Day
Servings of fruits and vegetables per day Measure 1
5.3
4.6
Measure 2
2.9
2.4
Sample size
333
357
Advertising versus cooking events. There were significant differences in some demographic characteristics between the cooking event participants and the post-test sample of intervention neighborhoods for the advertising survey. As shown in Table 4, the percentage of people who self-identified as Hispanic was significantly higher, the percentage of those who completed the questionnaire in English was significantly lower, and the percentage of people with nutrition information at home was significantly lower for the cooking event participants than for the post-test sample of the intervention neighborhood for the advertising survey. Consequently, the following variables for demographic characteristics were included in the multivariate analyses: English questionnaire and nutrition books at home. (A variable for Hispanic race/ethnicity was not included because the variable English questionnaire adequately represented this difference in demographic characteristics between the two sanlples.)The
166
Weaver et a1./5 A DAY F O R LOW-INCOME FAMILIES
Table 3. Multivariate analysis of the effects of the advertising campaign on recognition of the 5 A Day message and self-reported fruit and vegetable intake. Servings of Fruits and Vegetables per Day
Recognition of 5 A Day Message
5 A Day Hand
Variables
5 A Day Sun
One or Both Understand Meaning 5 A Day Logos of 5 A Day
Measure 1
Measure 2
Intervention/post Intervention/post/residential lntervention Intervention/residential Post Posttresidential Residential Food stamp office WIC Black English questionnaire One or more children Constant Responses predicted correctly (%) Significance of model chi-square Adjusted R2 The multivariate analyses were logistic regressionsfor recognition of the 5 A Day message and OLS regressionsfor self-reported fruit and vegetable intake. Standard errors are reported in parenthesis. *p < .05; **p < .01.
Table 4.
Comparison of demographic characteristics (percentages)
between the cooking event participants and the post-test sample of intervention neighborhoods for the advertising survey. --
Demographic Characteristics Ethnicity Asian Black Hispanic
Post-test
Cooking
Intervention Neighborhoods
Event Participants
p Value
.59 .70 .oo**
Native American
.25
White
.96
Table 5.
.83
participants and the post-test sample of the intervention neighborhoods
.21
for the advertising survey.
Multiethnic No response English questionnaire
.OO**
One or more children
.14
12 or more years of school
.22
Employed full or part time Have nutrition information
.86
at home
Chi-square tests were performed for the comparisons of demographic variables.
s .05; **p $ .01.
Recognition of the 5 A Day message of the cooking event
Post-test
Cooking
Intervention
Event
Neighborhoods
Participants
5 A Day hand?
15 %
22 %
5 A Day sunrise?
29 %
40
One or both ads?
38 %
50 O h
22 %
50 %
Have you seen: .oo**
Sample size
'p
variables black and one or more children were also included in the multivariate analysis because they were significantly related to some of the outcomes of the advertising campaign. As shown in Table 5, 50% of the cooking event sample recalled seeing one or both of the 5 A Day logos and 50% understood the meaning of 5 A Day. In contrast, only 38% of the post-test sample of the intervention neighborhoods for the advertising survey recalled seeing one or both 5 A Day logos, and only 22% understood the meaning of 5 A Day. The multivariate analysis showed that the cooking events had a significantly larger effect than the advertising campaign
Understand the meaning of 5 A Day Sample size
333
188
Oh
May
Journal of Nutrition Education Volume 31 Number 3
on all four measures of recognition of the 5 A Day message. The coefficient of the variable event measured the effects of the cooking events, and, as shown in Table 6, people who attended the cooking events were significantly more likely than people in the post-test sample of the intervention neighborhoods for the advertising survey to recognize both 5 A Day logos and to understand the meaning of 5 A Day. The results for the demographic variables showed that respondents with nutrition books at home were more likely to recognize the 5 A Day hand and one or both 5 A Day logos. Recognition of the 5 A Day hand was greater among respondents in the residential neighborhood than those in the inner-city neighborhood. The results for the other demographic variables were similar to those for the advertising intervention presented above.
DISCUSSION
Advertising. The results of the evaluation suggest that neither of the two assumptions on which the advertising campaign was based were valid.The first assumption was that low-income families with young children would benefit from a special effort by theTaste and Health Project to extend the national and state media campaigns to them. Although recognition of the 5 A Day sun increased during the first 5 months of the Taste and Health advertising campaign, the results of the multivariate analysis suggest that the advertising campaign did not significantly augment the effects of the NCI and state media campaigns. The one exception was that respondents in the residential intervention neighborhood were significantly more likely to recognize the 5 A Day hand than other respondents. Remember that the 5 A Day hand was unique to the Taste and Health project during the first 5 months of the advertising campaign. The residents in the residential neighborhood may have had more exposure to the Taste and Health advertising campaign because it was concentrated on a -
Table 6.
~
June 1999
167
smaller neighborhood. For example, the food stamp office, food pantry, and one of the W I C clinics were all located within 1 mile of each other. Note that respondents in the residential neighborhood were also significantly more likely to recognize the 5 A Day hand than respondents in the innercity neighborhood in the comparison between the advertising campaign and the cooking events. The second assumption was that effective materials from the N C I and state media campaigns would be readily available. Neither the materials from the NCI and state media campaigns nor those adapted from the Taste and Health advertising campaign effectively conveyed the 5 A Day message to families with low income. Even in the post-test sample, where there was an increase in recognition of the 5 A Day sun, and in the residential intervention neighborhood, where there was an increase in recognition of the 5 A Day hand, there was no corresponding increase in understanding of the meaning of 5 A Day. Fewer than 25% of the post-test sample understood the meaning of 5 A Day, even though the 5 A Day sun and 5 A Day hand were printed on the questionnaire and previous knowledge was not necessary. As one SNAC representative said, "The 5 A Day sunrise looks like a cornucopia to me with fruits and vegetables coming out of it. It's colorful, but it doesn't jump out at you. It looks like a label for a produce manufacturer." Some may question whether the advertising campaign was really not effective or the evaluation was unable to measure the effects.The researchers are confident that the nonequivalent control group design controlled for the effects of the NCI and state media campaign and for differences between the intervention and control groups that predated the intervention. In addition, the measures of recognition of the 5 A Day message were sensitive; they detected significant differences between the pretest and post-test samples of the advertising survey and between the advertising campaign and cooking event samples. The practice measures may not have accurately estimated the number of servings of fruits and vegetables eaten per day;
Multivariate analysis of the effects of the cooking events relative to the advertising campaign on recognition of the 5 A Day message.
Variables
5 A Day Hand
5 A Day Sun
Event Eventlresidential Residential Black One or more children English questionnaire Nutrition books at home Constant Responses predicted correctly (%) Significance of model chi-square The multivariate analyses were logistic regressions. Standard errors are reported in parentheses. 'p < .05; p 5 .01.
One or Both
Understand Meaning
5 A Day
of 5 A Day
168
Weaver et a1./5 A DAY F O R LOW-INCOME FAMILIES
the average number of servings was nearly twice as high according to Measure 1 than to Measure 2 in both the pretest and post-test samples, and it was not possible to identify which, if either, of the measures was accurate. The practice measures did, however, provide valid information about the tendency to eat fruits and vegetab1es.A~stated in the Methods section, the correlation coefficient for the rank of cases by measure was .42 and significant (p < .001).A recent assessment of four self-administered measures that were similar to Measures 1 and 2 also concluded that they were more useful for ranking subjects than for estimating the number of servings of fruits and vegetables eaten per day.18 In addition, for this evaluation, the results were the same for both measures; there were no significant differences between the intervention and control neighborhoods. For this evaluation, the researchers would have had less confidence in the measures if the results according to one measure were inconsistent with the results according to the other. The sample size of the advertising survey was large, and the power of the statistical tests was high." In the multivariate analysis, the powers of the statistical tests were 1 and .7 for the OLS regressions with Measure 1 and Measure 2, respectively, as the dependent variable. Advertising versus cooking events. The cooking events succeeded in calling attention to the 5 A Day message with colorful displays, the smell and taste of food samples, and the chance to talk informally with an outreach worker. This intervention was an effective way to convey the 5 A Day message because virtually every person who attended a cooking event and recognized the 5 A Day logo understood its meaning.The demonstrations and classes were equally effective in increasing recognition of the 5 A Day message, and, on average, the number of people with young children per event who intended to eat more fruits and vegetables was the same."' The success of the cooking events was consistent with the lessons learned from Maryland's W I C 5 A Day Promotion Program Pilot Study;" the participants of the pilot study liked food demonstrations, group discussions, and hands-on learning activities.The success was also consistent with recent articles based on focus group research. Reicks et al. reported that food assistance participants would like fruit and vegetable recipes that were fast, easy, and tasty, as well as opportunities to meet in groups to learn cooking skills and try new recipes.22 Trieman et al. recommended that interventions include opportunities to meet in groups.23 Some may question whether the cooking events were really successful or the results reflected sample selection, such as unmeasured differences in motivation to learn about nutrition or eating habits between the cooking event and advertising sample. For this evaluation, motivation to learn about nutrition was measured by responses to the question "Do you have books or magazines about nutrition in your home?" In fact, the percentage of respondents who had nutrition information at home was significantly lower for the cooking event
sample than the post-test sample of the intervention neighborhoods for the advertising survey, and this variable was included in the multivariate analysis. Eating habits were measured by the number of servings of fruits and vegetables eaten yesterday (Measure 1).The difference in the average number of servings of fruits and vegetables was not significant (5.3 for the cooking events sample and 5.2 for the posttest sample of the intervention neighborhoods for the advertising survey; p value = .79),*O and this variable was not included in the multivariate analysis. Further, there were few opportunities for sample selection because the cooking events were designed for a "captive audience," such as a waiting room, or an established group, such as pregnant women in a methadone treatment program.
IMPLICATIONS F O R RESEARCH A N D PRACTICE Research. The evaluation provided meaningful feedback on the interventions to SNAC, despite its relatively modest funding.The survey of people at food assistance program sites provided a large sample of people in the intervention and control neighborhoods and a high response rate. The nonequivalent control group design and multivariate analysis were essential features of the evaluation of a communitybased intervention because it was important to control for differences between the neighborhoods that predated the intervention. The self-administered questionnaire had excellent measures of recognition of the 5 A Day message. The measures were sensitive enough to detect differences between the pretest and post-test samples of the advertising survey and between the advertising campaign and cooking events.The question "What does 5 A Day mean to you?" helped to distinguish recognizing the logos from understanding their meaning. Finally, none of the measures had a ceiling effect. The practice measures may not have accurately estimated the number of servings of fruits and vegetables eaten per day, but they provided consistent information about the tendency to eat fruits and vegetables. Both measures showed that the Taste and Health advertising campaign did not significantly increase self-reported fruit and vegetable intake. Practice. Nutrition education interventions rarely occur in isolation, and it is important for interventions to complement each other without duplicating effort. O n a local level, the collaboration among the members of SNAC was an important example of complementary interventions. SNAC's focus on a common nutrition message meant that the members' efforts reinforced each other. In addition, the collaboration facilitated access to food assistance program ofices and other community sites for the interventions and the evaluation. SNAC continues to work to promote common nutrition messages and coordinate nutrition education for lowincome families with young children.
May
Journal of Nutrition Educat~onVolume31 N u m b e r 3
TheTaste and Health advertising campaign appeared to be an example of duplication of effort because it appeared to be reaching the same families with low income who could be reached by the NCI and state media campaigns. To the extent that another national or state media campaign for a nutrition message focuses on people in a specific stage of change, it may not be necessary to make a special effort to extend that campaign to families with low income. Although it is possible that a longer or more intensive advertising campaign may have augmented the effects of the NCI and state media campaigns, there was reason to doubt that any of these interventions increased understanding of the 5 A Day message. Cooking events are an attractive, personal, and enjoyable way to provide nutrition education to families with low income at food assistance program offices and other community settings.The results of this evaluation suggest that the outreach workers were an effective source of information about nutrition.The cooking events, which are similar to a standard marketing tool of the food industry, have been continued by theTaste and Health project and merit replication for other nutrition messages and for families with low income in other locations.
June 1999
169
6. U.S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute, Office of Cancer Communications. Selecting and profiling the core target audience for the 5 A Day media campaign. Unpublished manuscript, March 1993. 7. Prochaska JO, DlClemente CC. Transtheoretical therapy: toward a more integrative model of change. Psychother Theory Res Pract 1982;10:276-88. 8. U.S. Department of Agriculture, Food and Nutrition Service. Community nutrition education cooperatwe agreements. RFA FNS-94014CDW. Alexandria,VA: 1994. 9. U.S. Bureau of the Census. Statistical Abstract of the United States: 1997. 117th ed. Washington, D C : U. S. Government Printing Office, 1997. 10. Chipman H , Kendall PA.Twenty years of EFNEP: changes and challenges. J Nutr Educ 1989;21:265-9. 11. Owen AL, Owen GM.Twenty years of W l C : a review of some of the effects of the program. J Am Diet Assoc 1997;97:777-82. 12. U.S. Bureau of the Census. Census of Population and Housing, 1990: Summary Tape File 3 (Washington State)[Machine Readable Data Flles]/prepared by the U.S. Bureau of the Census. Washington, DC, 1991. 13. Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Chicago: Rand McNally & Sons, 1966. 14. Q u e Software. Rightwriter,Version 6.0 for Windows. Prentice Hall, 1992.
ACKNOWLEDGMENTS
15. U. S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute, Office of Cancer Communica-
TheTaste and Health Project was a collaborative effort of the organizations that are members of the Seattle Nutrition Action Consortium (SNAC) and their representatives. The authors thank everyone in SNAC, with special thanks to Alice Kurle, Trish Twomey, Brenda Wolsey, and Mary Kay Dugan. The researchers would also like to acknowledge the essential efforts of the community volunteers, the nutrition education outreach workers, and the site representatives.
tions. 5 A Day for Better Health messages: reaching and educating the American public: a tracking study. Unpublished manuscript, October 1992. 16. U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control, National Center for Chronic Disease Prevention and Health Promotion. 1994 Behavioral Risk Factor
Questionnaire. 17. Serdula M, Coates R , Byers T, et al. Evaluation of a brief telephone
questionnaire to estimate fruit and vegetable consumption in diverse study populations. Epidemiology 1993;4:455-63.
REFERENCES
18. Field A, Colditz G, Fox MK, et al. Comparison of 4 qurstions for assessment of fruit and vegetable intake. Am J Public Health 1998;88:1216-8.
1. U.S. Department of Health and Human Services. Healthy People 2000. National health promotion and dlsease prevention objectives. DHHS publication PHS 91-50212. Washington, DC, 199 1.
19. Borenstein M, Cohen J. Statistical Power Analysis Program: a computer program. Hillsdale, NJ: Lawrence Erlbaum and Associates, 1988. 20. Seattle Nutrition Action Consortium.Taste and Health: making nutri-
2. Popkin BM, Siega-Riz AM. A comparison of dietary trends among
tion education fun. C N E C Demonstration Project final report sub-
racial and socioeconomic groups in the United States. N Engl J Med
mitted to the U.S. Department ofAgrlculture, Food and Nutrition Ser-
1996;335:716-20.
vice, October 1996.
3. Krebs-Smith SM, Cook A, Subar AF, Cleveland L, Friday J. US adults'
21. Havas S, Damron D,Trieman K, et al.The Maryland WIC 5 A Day Pro-
fruit and vegetable intakes, 1989 to 1991: a revised baseline for the
motion Pilot Study: rationale, results, and lessons learned. J Nutr Educ
Healthy People 2000 objective.Am J Public Health 1995;85:1623-9.
4. Patterson BH, Block G. Food choices and the cancer guidelines. Am J Public Health 1988;78:282-6. 5. Contento I, Balch GI, BronnerYL, et al.The effectiveness of nutrition education and implications for nutrition education policy, programs,
1997;29:343-50. 22. Reicks M, Randall JL, Haynes BJ. Factors affecting consumption of fruits and vegetables by low-income families. J Am Diet Assoc 1994;94:1309-11. 23. Trieman K, FreimuthV, Damron D, et al.Attitudes and behaviors related
and research: a review of research. Chapter 5: nutrition education for
to fruits and vegetables among low-income women in the WIC pro-
adults. J Nutr Educ 1995;25:312-28.
gram. J Nutr Educ 1996;28:149-56.
ABSTRACT This article reports the results of the evaluation of a 5 A Day demonstration project with an advertising campaign and cooking events for low-income families with young children.The goal of the evaluation was to provide meaningful feedback on how to improve the interventions, rather than to conduct a definitive trial on their effectiveness. Brief, selfadministered questionnaires were developed to measure two outcomes: (1) recognition of the 5 A Day message and (2) selfreported fruit and vegetable intake.To evaluate the advertising campaign, the results from the two intervention neighborhoods (n = 694) were compared with those from two control neighborhoods (n = 771). It was possible to evaluate the cooking events in relation to the advertising campaign because people who attended the cooking events were also exposed to the advertising campaign.The results from a sample of people who attended the cooking events (n = 188) were compared with those from the post-test sample from the intervention neighborhoods (n = 333). Recognition of the 5 A Day logos increased during the advertising campaign but the effect did not exceed the effect of the National Cancer Institute and state 5 A Day media campaigns. Self-reported fruit and vegetable intake did not change.The cooking events were significantly more effective than the advertising campaign in increasing recognition of the 5 A Day logos and understanding of the 5 A Day message.
INTRODUCTION O n e of theyear 2000 Health Goals is for all Americans to eat at least five servings of fruits or vegetables a day.' This goal is sometimes referred to as "5 A Day." There is reason to believe, however, that people with low income may have to ................................................... This project Wac funded in part with federal fund? from the Food and Nutrition Service, U.S. Department ofAgriculture, under Cooperative Agreement No. 58-3198-4-053. The contents of this publication do not necessarily reflect the view or policies of the U. S. Department ofAgr~culture,nor does mention of trade names, commercial products, or organizations Imply endorsement by the U.S. Government. Address for correspondence: Marcia Weaver. Ph.l)., 1122 E. Pike St.,#1132, Seattle,WA 98122;Tel: (206) 296-0241; Fax: (206) 205-5314; E-mall: [email protected]. 01999 SOCIETY FOR NUTRITION EIIUCATION
make equal or larger increases than people in other income groups in the number of servings of fruits and vegetables they eat to meet theyear 2000 Health Goal. Some researchers have shown that the number of servings of fruits and vegetables that people eat is similar across income groups,* while others have shown that it is lower among people with lower income or less ed~cation.'.~ There is also reason to believe that access to nutrition education for people with low income may be less than or equal to access for people in other income groups. Considering nutrition education in general, a 1995 literature review recommended that nutrition education programs devote more effort to reaching disadvantaged groups, especially those .~ who are less educated and have lower i n ~ o m e Considering the 5 A Day message in particular, the National Cancer Institute's (NCI) "5 A Day for Better Health" campaign was targeted at adults who had recently increased their consumption of fruits and vegetables but were still eating only 1.5 to 2.5 servings of fruits and vegetables per day.hWithin the stages of change f r a m e ~ o r k ,the ~ N C I media campaign focused on people who were preparing to eat more or were eating more fruits and vegetables, rather than those who were in the precontemplative or contemplative stages. Although the NCI media campaign did not exclude people with low income, the target audience included a disproportionate number of women who were employed outside the home." Further, the NCI media campaign did not face the challenge of calling the attention of people with low income to the 5 A Day message or showing them easy and inexpensive ways to eat more fruits and vegetables. To improve nutrition education for people with low income, the U.S. Department ofAgriculture (USDA) funded 10 Community Nutrition Education Cooperative (CNEC) Demonstration Projects to explore strategies to integrate nutrition education services with food assistance program^.^ Food assistance programs offer an opportunity to educate a large number of people with low income about nutrition because they serve several million people. In 1994, the average number of people served per month included 28.9 million people in the Food Stamps program, 6.5 million people
162
Weaver et a1./5 A DAY F O R LOW-INCOME FAMILIES
in the Special Supplemental Nutrition Program for Women, Infants and Children (WIC), 14 million students in the free and reduced-price School Lunch Program, and 5.2 million students in the free and reduced-price School Breakfast Program.9 Although USDA has supported nutrition education for more than 20 years through programs such as the ~ Expanded Food and Nutrition Education P r ~ g r a m 'and WIC," the C N E C initiative sought to encourage collaborative and community-based nutrition education interventions. T h e Seattle Nutrition Action Consortium (SNAC) received C N E C funding for the Taste and Health Project to implement a 5 A Day demonstration project. SNAC included representatives of each major food assistance program (Food Stamps,WIC, and school meal programs) as well as the Head Start/Early Childhood Assistance and Education Program, food pantries, nonprofit organizations, and businesses. The representatives decided to focus on a common nutrition message in each of their programs to maximize the impact of limited nutrition education resources.They selected the 5 A Day message in order to extend the N C I media campaign to low-income families with young children and because nutrition education materials were available from the NCI and state 5 A Day media campaigns. This article reports on the evaluation of two of the Taste and Health Project's 5 A Day interventions: an advertising campaign and cooking events. The evaluation questions for the advertising campaign were "Did the Taste and Health advertising campaign augment the effects of the NCI and state media campaigns on: (1) recognition of the 5 A Day message and (2) self-reported fruit and vegetable intake?" The evaluation question for the cooking events was "Did the 5 A Day cooking events significantly increase recognition of the 5 A Day message relative to the advertising campaign?"
METHODS Interventions.
Advertising. The goal of the advertising campaign was to extend the NCI and state media campaigns to low-income families with young children using advertisements targeted to them.The advertising campaign included (1) interior bus signs on bus routes in the intervention neighborhoods; (2) advertising and articles in community newspapers; (3) 5 A Day messages on school menus, grocery bags, milk cartons, and flyers; and (4) posters and promotional materials displayed and distributed by SNAC members. Some of the materials were obtained through NCI and state media campaigns, and others were developed by SNAC's Nutrition Messages Work Group and the staff of this project. Cooking events. The goal of the cooking events was to call attention to the 5 A Day message with colorful displays and the smell and taste of food samples. Outreach workers also
talked with people and showed them ways to serve fruits and vegetables that were easy to prepare and inexpensive. The events were conducted at food assistance program offices and other community settings and included a featured recipe, discussion of key points (such as "fruits and vegetables are the original fast food"), samples of the food, and promotional materials for the participants to take home (such as recipes, brochures, refrigerator magnets, coloring books, and stickers). The outreach workers were not necessarily nutrition professionals; they were people who received ongoing training in nutrition education and had an ability to work with community groups. There were two types of cooking events: demonstrations and classes. A demonstration served a large number of people at sites such as food pantries and the waiting rooms of food stamp offices. People could visit the displays at their discretion to talk with the outreach worker and sample food that was prepared on site. The demonstrations were developed to reach people who came to a site for another reason and could be attracted to the intervention, especially during a waiting period. In contrast, a class was taught to small audiences at sites such as W I C clinics, other health clinics, a homeless shelter for mothers and young children, and a methadone treatment center for pregnant women.The classes lasted 30 to 60 minutes. During this time, an outreach worker prepared a food sample from a featured recipe, offered tips on preparing fruits and vegetables and on encouraging children to eat more of them, and answered questions from the audience.
Target population. The target population for the interventions were low-income families with young children. SNAC was encouraged to focus on two neighborhoods for the demonstration project, so that it would be possible to compare the outcomes of the intervention neighborhoods to other neighborhoods. SNAC selected one inner-city and one residential neighborhood with a high incidence of poverty. Household income was less than 200% of the federal poverty level for 46% of the residents of the inner-city neighborhood and 35% of the residents of the residential neighborhood." Evaluation design. Several special features of the evaluation stemmed from the nature of the C N E C demonstration projects. SNAC was encouraged to try a variety of interventions and modi@ them based on experience. Consequently, the goal of the evaluation was to provide meaningful feedback on how to improve the interventions, rather than to conduct a definitive trial on their effectiveness. In addition, funding for the evaluation was relatively modest. Consequently, the researchers developed a strategy to sample a cross-section of people at food assistance program offices and other community settings, and self-administered questionnaires to measure two outcomes.
Advertising. Given that the goal of the advertising campaign was to extend the NCI and state media campaigns, and
May June 1999
Journal of Nutrition Education Volume 31 Number 3
that the NCI campaign was targeted at people who were preparing to eat or were already eating more fruits and vegetables, the evaluation addressed whether or not the intervention affected eating habits as well as recognition of the 5 A Day message. This part of the evaluation had a nonequivalent control group design13 in which the outcome variables of the two intervention neighborhoods were compared to two control neighborhoods before and after the intervention. One innercity and one residential control neighborhood were selected to match the demographic characteristics of W I C participants in the two intervention neighborhoods. The pre/post comparison controlled for differences between the intervention and control neighborhoods that predated the intervention and showed changes in the intervention and control groups over time.
Advertising versus cooking events. Given that the goal of the cooking events was to call attention to the 5 A Day message, the evaluation addressed whether the intervention affected recognition of the 5 A Day message, but not eating habits. It was possible to evaluate the cooking events in relation to the advertising campaign because people who attended the cooking events had the same opportunities to be exposed to the advertising campaign as the respondents to the advertising survey.The outcomes of cooking event participants were compared to the post-test sample of the intervention neighborhoods for the advertising survey. Sampling strategies and procedures.
Data were collected from the two intervention Advertisinx. and two control neighborhoods before the intervention began in November/December 1995 and 5 months after the intervention began in June 1996.A field data collection manager spent 1 day asking everyone at a site to complete a questionnaire, keeping track of refusals.There were four sites (two different W I C clinics, a food stamp office, and a food pantry) in each of the four neighborhoods, for a total of 16 sites.The sample excluded people under 18, staff, and people who did not speak English or Spanish.Two of the field data collection managers were fluent in Spanish and were assigned to sites where many of the respondents spoke Spanish, such as a food pantry adjacent to a church with a Hispanic congregation. For the pretest, there were 775 respondents of 1076 people at the 16 sites, for a response rate of 72%. For the posttest, there were 690 respondents of 1095 people at the 16 sites, for a response rate of 63%. The total sample size was 1465.
Cooking events. The sample included people who attended a cooking event between June 10 and July 31,1996.A field data collection manager attended the cooking event and asked every person who attended it to complete a questionnaire. Attendance at a demonstration was defined as approaching the outreach worker to sample food or talk with
163
him/her. Attendance at a class was defined as attending part of a class, including people who arrived late or left early.The sample excluded people under 18, staff, and people who did not speak English or Spanish. Some of the field data collection managers were fluent in Spanish and were assigned to cooking events where many of the people spoke Spanish. There were 188 respondents of 263 people who attended cooking events, for a response rate of 72%. Among the respondents, 117 attended demonstrations and 73 attended classes. Measurement instruments.
Development. The researchers developed self-administered questionnaires in English and Spanish. The first draft of the questions was based on a review of the literature. Then the questions were pilot tested with a total of 42 respondents at food assistance program offices. Feedback from the respondents was used to assess the questions for comprehension and the questions were revised accordingly. The final version of the questions was written at the fourth-grade reading level according to a readability analysis using the Rightwriter software.14 Recognition of the 5 A D a y message. Recognition of two different logos was measured: 5 A Day sun and 5 A Day hand. The 5 A Day hand, which was developed by NCI to promote 5 A Day week in the early 1990s, was adapted and used (with permission) on most of the Taste and Health Project's 5 A Day promotional materials. Recently, the 5 A Day hand was not used by the NCI and state media campaigns, so it was unique to the Taste and Health Project during the first 5 months of the advertising campaign. The questions were based on a questionnaire developed for the national media campaign.15Respondents were shown both logos and were asked "Which ads have you seen in the last few months? Please mark all that you have seen."Then the respondents were asked, "What does '5 A Day' mean to you?" and given a blank space in which to write an answer. The respondent was considered to have understood the meaning of 5 A Day if hidher answer included at least the word fruit or vegetable. Pretest data were used to test for ceiling effect.The questions did not have a ceiling effect because fewer than 25% of the pretest respondents had seen the ads or understood the meaning of 5 A Day. Self-reported fruit and vegetable intake. O n e of the challenges of the evaluation was to design self-administered questions to estimate the average number of servings of fruits and vegetables per day for a population. Two measures were included in the pretest with the goal of selecting only one for the post-test. Measure 1 was "How many servings of fruits and vegetables did you eat yesterday?" followed by information on serving sizes and a list of more than 25 fruits and vegetables. Measure 2 consisted of four modified questions from the Behavioral Risk Factor Surveillance System (BRFSS)
164
Weaver et a1./5 A DAY FOR LOW-INCOME FAMILIES
Fruit andvegetable M ~ d u l e . ' ~ The , " BRFSS was designed for telephone interviews that were too expensive for this evaluation, so the questions and response categories were modified for self-administered questionnaires (Shirley Beresford, personal communication, 1995). In general, Measure 1 appeared cognitively easier than Measure 2, but Measure 2 had the advantage of yielding results easy to compare with other BRFSS data. Pretest data were used to assess the reliability, convergent validity, and respondent burden of the measures.The assessment did not show that one measure was clearly superior to the other, so both measures were retained for the post-test. The assessment of reliability (Cronbach's alpha) was only relevant for Measure 2, which combines several questions about fruits and vegetables.The alpha for that measure was .77 and the corrected item-total correlation was greater than .4 for all four items, showing that the four questions could be combined into a single measure.The convergent validity tests gave mixed results.The correlation coefficient for the rank of cases by measure was .42 and significant (p < ,001). In contrast, Measure 1 showed that the difference in self-reported fruit and vegetable intake between the respondents in the intervention and control neighborhoods was significant (5.0 and 4.2, respectively; p value =. 08), but Measure 2 did not (2.6 and 2.5, respectively; p value = .54). Also, the number of servings per day was nearly twice as high according to Measure 1 than to Measure 2. Finally, the response rate was 97% for Measure 1, compared to 86% for Measure 2, showing that Measure 1 was less of a burden for respondents than Measure 2. Bivariate statistics. Chi-square tests were performed for two sets of comparisons of demographic characteristics: (1) a comparison of the post-test sample of intervention and control neighborhoods for the advertising survey and (2) a comparison of the cooking event participants and the post-test sample of the intervention neighborhood for the advertising survey.
to the residential intervention neighborhood, respectively. The covariates food stamp office andWIC controlled for the sampling strategy but did not have interactions with other factors because some of the cells were empty. All analyses were weighted to adjust for differences in the number of respondents at each site.
Advertising versus cooking events. The coefficients for the variables event and eventlresidential measured the effects of the cooking events and the effects that were specific to the residential intervention neighborhood, respectively.
RESULTS Advertising. There were significant differences in some demographic characteristics between the post-test sample of intervention and control neighborho0ds.A~shown inTable 1, the percentage of people who self-identified as black was significantly higher, the percentage of those who self-identified as white was significantly lower, and the percentage of those with one or more children was significantly lower for the intervention than the control neighborhoods.There was also a significantly lower percentage of people in the post-test sample than the pretest sample of control neighborhoods who completed the questionnaire in English (not shown). Consequently, the following variables for demographic characteristics were included in the multivariate analyses: black, one or more children, and English questionnaire. (A variable for white race/ethnicity was not included because the vari-
Table 1.
Comparison of demographic characteristics (percentages)
between the post-test sample of intervention and control neighborhoods for the advertising survey. Demographic Characteristics
Intervention
Control
p Value
Ethnicity
Multivariate analysis. Logistic regressions for measures of recognition of 5 A Day message (dichotomous variables) and ordinary least squares (OLS) regressions for self-reported fruit and vegetable intake (continuous variables) were performed. To validate the results, analyses of variance of the continuous variables were also performed, and those results on the effects of the interventions were identical to the results reported below. There were some differences in demographic characteristics between samples. Additional covariates were included in the multivariate analysis to adjust for these differences; these covariates are listed with the results on demographic characteristics for each intervention.
Advertising. The coefficients for the variables interventiodpost-test and interventiodpost-tesdresidential measured the effects of the intervention and effects that were specific
Asian Black Hispanic Native American White Multiethnic No response English questionnaire One or more children 12 or more years of school Employed full or part time Nutrition books at home Sample size Chi-square tests were performed for the comparisons of demographic variables. *p < .05; **p < .01.
May
Journal of Nutrition Education Volume 31 Number 3
able black adequately represented this difference in demographic characteristics between the two samples.) To the extent that comparable data were available, it appeared that the post-test sample of the intervention neighborhoods was representative of participants in food assistance programs in those neighborhoods. For example, the distribution of race/ethnic identity of W I C participants in the intervention neighborhoods was similar to the distribution reported in Table 1 for those neighborhoods. In August 1995,15% ofWIC participants were Asian, 37% were African Americadblack, 11% were Hispanic, 25% were white, and 12% gave other responses.Although the distributions are not identical, both samples were diverse, with a minority of people who self-identified as white. Differences between the distributions may reflect that the evaluation included participants in the food stamp program and food pantries in addition to the W I C program and excluded people who did not speak English or Spanish, as well as differences in the ways the questions about race/ethnicity were asked. Table 2 shows a comparison of outcomes for the post-test sample of respondents in the intervention and control neighborhoods. O f the intervention and control respondents, 15% and 8%, respectively, recalled seeing the 5 A Day hand, and 38% and 28%, respectively, recalled seeing one or both ads. The post-test average of self-reported fruit and vegetable intake was 5.3 and 4.6 for the intervention and control respondents, respectively, for Measure 1, and 2.9 and 2.4 for Measure 2. The multivariate analysis combined the pretest and posttest samples to control for differences between the intervention and control neighborhoods that predated the intervention. As shown in Table 3, recognition of the 5 A Day sun increased during the intervention, but the effects of the Taste and Health advertising campaign did not exceed the effects of the NCI and state media campaigns. The coefficients of the variable post reflected the effect of the NCI and
Table 2.
June 1999
165
state media campaigns and showed that there was a significant effect on recognition of the 5 A Day sun and one or both 5 A Day 1ogos.The coefficients of the variable interventiodpost reflected the effects of the Taste and Health advertising campaign and were not significant. The one exception was that respondents in the residential intervention neighborhood (see the coefficient for the variable interventiodpost/residential) were significantly more likely to recognize the 5 A Day hand, but they were not more likely to recognize one or both 5 A Day logos or understand the meaning of 5 A Day. Note that people in the intervention neighborhoods ate more servings of fruits and vegetables than people in the control neighborhoods both before and after the intervention. The coefficient of the variable intervention reflected differences between the intervention and control neighborhoods that predated the interventions, and it was positive and significant for Measure 1, the number of servings of fruits and vegetables consumed yesterday. The coeff~cientsof the variables food stamp offlce and W I C reflected differences in outcomes between respondents at these sites and food pantries (the omitted dummy variable). Respondents at food stamp offices and W I C clinics were significantly more likely than respondents at food pantries to understand the meaning of 5 A Day, and they ate more servings of fruits and vegetables according to Measure 2. The results for the demographic variables showed that respondents with one or more children were more likely to understand the meaning of 5 A Day as well as to recognize the 5 A Day sun and one or both 5 A Day 1ogos.The results for the other two demographic variables did not show consistent differences across racial/ethnic groups. Blacks had a significantly lower outcome than other respondents for one of the six outcome measures. Respondents who completed the questionnaire in English were more likely to recognize the 5 A Day sun but ate fewer servings of fruits and vegetables according to Measure l than those who completed the questionnaire in Spanish.
Recognition of the 5 A Day message, and self-reported fruit
and vegetable intake of the post-test sample of intervention and control neighborhoods for the advertising survey.
Intervention
Control
Recognition of 5 A Day message Have you seen: 5 A Day hand?
15 %
8%
5 A Day sunrise?
29 %
23 %
38 %
28 %
22 %
18 %
One or both ads? Understand the meaning of 5 A Day
Servings of fruits and vegetables per day Measure 1
5.3
4.6
Measure 2
2.9
2.4
Sample size
333
357
Advertising versus cooking events. There were significant differences in some demographic characteristics between the cooking event participants and the post-test sample of intervention neighborhoods for the advertising survey. As shown in Table 4, the percentage of people who self-identified as Hispanic was significantly higher, the percentage of those who completed the questionnaire in English was significantly lower, and the percentage of people with nutrition information at home was significantly lower for the cooking event participants than for the post-test sample of the intervention neighborhood for the advertising survey. Consequently, the following variables for demographic characteristics were included in the multivariate analyses: English questionnaire and nutrition books at home. (A variable for Hispanic race/ethnicity was not included because the variable English questionnaire adequately represented this difference in demographic characteristics between the two sanlples.)The
166
Weaver et a1./5 A DAY F O R LOW-INCOME FAMILIES
Table 3. Multivariate analysis of the effects of the advertising campaign on recognition of the 5 A Day message and self-reported fruit and vegetable intake. Servings of Fruits and Vegetables per Day
Recognition of 5 A Day Message
5 A Day Hand
Variables
5 A Day Sun
One or Both Understand Meaning 5 A Day Logos of 5 A Day
Measure 1
Measure 2
Intervention/post Intervention/post/residential lntervention Intervention/residential Post Posttresidential Residential Food stamp office WIC Black English questionnaire One or more children Constant Responses predicted correctly (%) Significance of model chi-square Adjusted R2 The multivariate analyses were logistic regressionsfor recognition of the 5 A Day message and OLS regressionsfor self-reported fruit and vegetable intake. Standard errors are reported in parenthesis. *p < .05; **p < .01.
Table 4.
Comparison of demographic characteristics (percentages)
between the cooking event participants and the post-test sample of intervention neighborhoods for the advertising survey. --
Demographic Characteristics Ethnicity Asian Black Hispanic
Post-test
Cooking
Intervention Neighborhoods
Event Participants
p Value
.59 .70 .oo**
Native American
.25
White
.96
Table 5.
.83
participants and the post-test sample of the intervention neighborhoods
.21
for the advertising survey.
Multiethnic No response English questionnaire
.OO**
One or more children
.14
12 or more years of school
.22
Employed full or part time Have nutrition information
.86
at home
Chi-square tests were performed for the comparisons of demographic variables.
s .05; **p $ .01.
Recognition of the 5 A Day message of the cooking event
Post-test
Cooking
Intervention
Event
Neighborhoods
Participants
5 A Day hand?
15 %
22 %
5 A Day sunrise?
29 %
40
One or both ads?
38 %
50 O h
22 %
50 %
Have you seen: .oo**
Sample size
'p
variables black and one or more children were also included in the multivariate analysis because they were significantly related to some of the outcomes of the advertising campaign. As shown in Table 5, 50% of the cooking event sample recalled seeing one or both of the 5 A Day logos and 50% understood the meaning of 5 A Day. In contrast, only 38% of the post-test sample of the intervention neighborhoods for the advertising survey recalled seeing one or both 5 A Day logos, and only 22% understood the meaning of 5 A Day. The multivariate analysis showed that the cooking events had a significantly larger effect than the advertising campaign
Understand the meaning of 5 A Day Sample size
333
188
Oh
May
Journal of Nutrition Education Volume 31 Number 3
on all four measures of recognition of the 5 A Day message. The coefficient of the variable event measured the effects of the cooking events, and, as shown in Table 6, people who attended the cooking events were significantly more likely than people in the post-test sample of the intervention neighborhoods for the advertising survey to recognize both 5 A Day logos and to understand the meaning of 5 A Day. The results for the demographic variables showed that respondents with nutrition books at home were more likely to recognize the 5 A Day hand and one or both 5 A Day logos. Recognition of the 5 A Day hand was greater among respondents in the residential neighborhood than those in the inner-city neighborhood. The results for the other demographic variables were similar to those for the advertising intervention presented above.
DISCUSSION
Advertising. The results of the evaluation suggest that neither of the two assumptions on which the advertising campaign was based were valid.The first assumption was that low-income families with young children would benefit from a special effort by theTaste and Health Project to extend the national and state media campaigns to them. Although recognition of the 5 A Day sun increased during the first 5 months of the Taste and Health advertising campaign, the results of the multivariate analysis suggest that the advertising campaign did not significantly augment the effects of the NCI and state media campaigns. The one exception was that respondents in the residential intervention neighborhood were significantly more likely to recognize the 5 A Day hand than other respondents. Remember that the 5 A Day hand was unique to the Taste and Health project during the first 5 months of the advertising campaign. The residents in the residential neighborhood may have had more exposure to the Taste and Health advertising campaign because it was concentrated on a -
Table 6.
~
June 1999
167
smaller neighborhood. For example, the food stamp office, food pantry, and one of the W I C clinics were all located within 1 mile of each other. Note that respondents in the residential neighborhood were also significantly more likely to recognize the 5 A Day hand than respondents in the innercity neighborhood in the comparison between the advertising campaign and the cooking events. The second assumption was that effective materials from the N C I and state media campaigns would be readily available. Neither the materials from the NCI and state media campaigns nor those adapted from the Taste and Health advertising campaign effectively conveyed the 5 A Day message to families with low income. Even in the post-test sample, where there was an increase in recognition of the 5 A Day sun, and in the residential intervention neighborhood, where there was an increase in recognition of the 5 A Day hand, there was no corresponding increase in understanding of the meaning of 5 A Day. Fewer than 25% of the post-test sample understood the meaning of 5 A Day, even though the 5 A Day sun and 5 A Day hand were printed on the questionnaire and previous knowledge was not necessary. As one SNAC representative said, "The 5 A Day sunrise looks like a cornucopia to me with fruits and vegetables coming out of it. It's colorful, but it doesn't jump out at you. It looks like a label for a produce manufacturer." Some may question whether the advertising campaign was really not effective or the evaluation was unable to measure the effects.The researchers are confident that the nonequivalent control group design controlled for the effects of the NCI and state media campaign and for differences between the intervention and control groups that predated the intervention. In addition, the measures of recognition of the 5 A Day message were sensitive; they detected significant differences between the pretest and post-test samples of the advertising survey and between the advertising campaign and cooking event samples. The practice measures may not have accurately estimated the number of servings of fruits and vegetables eaten per day;
Multivariate analysis of the effects of the cooking events relative to the advertising campaign on recognition of the 5 A Day message.
Variables
5 A Day Hand
5 A Day Sun
Event Eventlresidential Residential Black One or more children English questionnaire Nutrition books at home Constant Responses predicted correctly (%) Significance of model chi-square The multivariate analyses were logistic regressions. Standard errors are reported in parentheses. 'p < .05; p 5 .01.
One or Both
Understand Meaning
5 A Day
of 5 A Day
168
Weaver et a1./5 A DAY F O R LOW-INCOME FAMILIES
the average number of servings was nearly twice as high according to Measure 1 than to Measure 2 in both the pretest and post-test samples, and it was not possible to identify which, if either, of the measures was accurate. The practice measures did, however, provide valid information about the tendency to eat fruits and vegetab1es.A~stated in the Methods section, the correlation coefficient for the rank of cases by measure was .42 and significant (p < .001).A recent assessment of four self-administered measures that were similar to Measures 1 and 2 also concluded that they were more useful for ranking subjects than for estimating the number of servings of fruits and vegetables eaten per day.18 In addition, for this evaluation, the results were the same for both measures; there were no significant differences between the intervention and control neighborhoods. For this evaluation, the researchers would have had less confidence in the measures if the results according to one measure were inconsistent with the results according to the other. The sample size of the advertising survey was large, and the power of the statistical tests was high." In the multivariate analysis, the powers of the statistical tests were 1 and .7 for the OLS regressions with Measure 1 and Measure 2, respectively, as the dependent variable. Advertising versus cooking events. The cooking events succeeded in calling attention to the 5 A Day message with colorful displays, the smell and taste of food samples, and the chance to talk informally with an outreach worker. This intervention was an effective way to convey the 5 A Day message because virtually every person who attended a cooking event and recognized the 5 A Day logo understood its meaning.The demonstrations and classes were equally effective in increasing recognition of the 5 A Day message, and, on average, the number of people with young children per event who intended to eat more fruits and vegetables was the same."' The success of the cooking events was consistent with the lessons learned from Maryland's W I C 5 A Day Promotion Program Pilot Study;" the participants of the pilot study liked food demonstrations, group discussions, and hands-on learning activities.The success was also consistent with recent articles based on focus group research. Reicks et al. reported that food assistance participants would like fruit and vegetable recipes that were fast, easy, and tasty, as well as opportunities to meet in groups to learn cooking skills and try new recipes.22 Trieman et al. recommended that interventions include opportunities to meet in groups.23 Some may question whether the cooking events were really successful or the results reflected sample selection, such as unmeasured differences in motivation to learn about nutrition or eating habits between the cooking event and advertising sample. For this evaluation, motivation to learn about nutrition was measured by responses to the question "Do you have books or magazines about nutrition in your home?" In fact, the percentage of respondents who had nutrition information at home was significantly lower for the cooking event
sample than the post-test sample of the intervention neighborhoods for the advertising survey, and this variable was included in the multivariate analysis. Eating habits were measured by the number of servings of fruits and vegetables eaten yesterday (Measure 1).The difference in the average number of servings of fruits and vegetables was not significant (5.3 for the cooking events sample and 5.2 for the posttest sample of the intervention neighborhoods for the advertising survey; p value = .79),*O and this variable was not included in the multivariate analysis. Further, there were few opportunities for sample selection because the cooking events were designed for a "captive audience," such as a waiting room, or an established group, such as pregnant women in a methadone treatment program.
IMPLICATIONS F O R RESEARCH A N D PRACTICE Research. The evaluation provided meaningful feedback on the interventions to SNAC, despite its relatively modest funding.The survey of people at food assistance program sites provided a large sample of people in the intervention and control neighborhoods and a high response rate. The nonequivalent control group design and multivariate analysis were essential features of the evaluation of a communitybased intervention because it was important to control for differences between the neighborhoods that predated the intervention. The self-administered questionnaire had excellent measures of recognition of the 5 A Day message. The measures were sensitive enough to detect differences between the pretest and post-test samples of the advertising survey and between the advertising campaign and cooking events.The question "What does 5 A Day mean to you?" helped to distinguish recognizing the logos from understanding their meaning. Finally, none of the measures had a ceiling effect. The practice measures may not have accurately estimated the number of servings of fruits and vegetables eaten per day, but they provided consistent information about the tendency to eat fruits and vegetables. Both measures showed that the Taste and Health advertising campaign did not significantly increase self-reported fruit and vegetable intake. Practice. Nutrition education interventions rarely occur in isolation, and it is important for interventions to complement each other without duplicating effort. O n a local level, the collaboration among the members of SNAC was an important example of complementary interventions. SNAC's focus on a common nutrition message meant that the members' efforts reinforced each other. In addition, the collaboration facilitated access to food assistance program ofices and other community sites for the interventions and the evaluation. SNAC continues to work to promote common nutrition messages and coordinate nutrition education for lowincome families with young children.
May
Journal of Nutrition Educat~onVolume31 N u m b e r 3
TheTaste and Health advertising campaign appeared to be an example of duplication of effort because it appeared to be reaching the same families with low income who could be reached by the NCI and state media campaigns. To the extent that another national or state media campaign for a nutrition message focuses on people in a specific stage of change, it may not be necessary to make a special effort to extend that campaign to families with low income. Although it is possible that a longer or more intensive advertising campaign may have augmented the effects of the NCI and state media campaigns, there was reason to doubt that any of these interventions increased understanding of the 5 A Day message. Cooking events are an attractive, personal, and enjoyable way to provide nutrition education to families with low income at food assistance program offices and other community settings.The results of this evaluation suggest that the outreach workers were an effective source of information about nutrition.The cooking events, which are similar to a standard marketing tool of the food industry, have been continued by theTaste and Health project and merit replication for other nutrition messages and for families with low income in other locations.
June 1999
169
6. U.S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute, Office of Cancer Communications. Selecting and profiling the core target audience for the 5 A Day media campaign. Unpublished manuscript, March 1993. 7. Prochaska JO, DlClemente CC. Transtheoretical therapy: toward a more integrative model of change. Psychother Theory Res Pract 1982;10:276-88. 8. U.S. Department of Agriculture, Food and Nutrition Service. Community nutrition education cooperatwe agreements. RFA FNS-94014CDW. Alexandria,VA: 1994. 9. U.S. Bureau of the Census. Statistical Abstract of the United States: 1997. 117th ed. Washington, D C : U. S. Government Printing Office, 1997. 10. Chipman H , Kendall PA.Twenty years of EFNEP: changes and challenges. J Nutr Educ 1989;21:265-9. 11. Owen AL, Owen GM.Twenty years of W l C : a review of some of the effects of the program. J Am Diet Assoc 1997;97:777-82. 12. U.S. Bureau of the Census. Census of Population and Housing, 1990: Summary Tape File 3 (Washington State)[Machine Readable Data Flles]/prepared by the U.S. Bureau of the Census. Washington, DC, 1991. 13. Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Chicago: Rand McNally & Sons, 1966. 14. Q u e Software. Rightwriter,Version 6.0 for Windows. Prentice Hall, 1992.
ACKNOWLEDGMENTS
15. U. S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute, Office of Cancer Communica-
TheTaste and Health Project was a collaborative effort of the organizations that are members of the Seattle Nutrition Action Consortium (SNAC) and their representatives. The authors thank everyone in SNAC, with special thanks to Alice Kurle, Trish Twomey, Brenda Wolsey, and Mary Kay Dugan. The researchers would also like to acknowledge the essential efforts of the community volunteers, the nutrition education outreach workers, and the site representatives.
tions. 5 A Day for Better Health messages: reaching and educating the American public: a tracking study. Unpublished manuscript, October 1992. 16. U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control, National Center for Chronic Disease Prevention and Health Promotion. 1994 Behavioral Risk Factor
Questionnaire. 17. Serdula M, Coates R , Byers T, et al. Evaluation of a brief telephone
questionnaire to estimate fruit and vegetable consumption in diverse study populations. Epidemiology 1993;4:455-63.
REFERENCES
18. Field A, Colditz G, Fox MK, et al. Comparison of 4 qurstions for assessment of fruit and vegetable intake. Am J Public Health 1998;88:1216-8.
1. U.S. Department of Health and Human Services. Healthy People 2000. National health promotion and dlsease prevention objectives. DHHS publication PHS 91-50212. Washington, DC, 199 1.
19. Borenstein M, Cohen J. Statistical Power Analysis Program: a computer program. Hillsdale, NJ: Lawrence Erlbaum and Associates, 1988. 20. Seattle Nutrition Action Consortium.Taste and Health: making nutri-
2. Popkin BM, Siega-Riz AM. A comparison of dietary trends among
tion education fun. C N E C Demonstration Project final report sub-
racial and socioeconomic groups in the United States. N Engl J Med
mitted to the U.S. Department ofAgrlculture, Food and Nutrition Ser-
1996;335:716-20.
vice, October 1996.
3. Krebs-Smith SM, Cook A, Subar AF, Cleveland L, Friday J. US adults'
21. Havas S, Damron D,Trieman K, et al.The Maryland WIC 5 A Day Pro-
fruit and vegetable intakes, 1989 to 1991: a revised baseline for the
motion Pilot Study: rationale, results, and lessons learned. J Nutr Educ
Healthy People 2000 objective.Am J Public Health 1995;85:1623-9.
4. Patterson BH, Block G. Food choices and the cancer guidelines. Am J Public Health 1988;78:282-6. 5. Contento I, Balch GI, BronnerYL, et al.The effectiveness of nutrition education and implications for nutrition education policy, programs,
1997;29:343-50. 22. Reicks M, Randall JL, Haynes BJ. Factors affecting consumption of fruits and vegetables by low-income families. J Am Diet Assoc 1994;94:1309-11. 23. Trieman K, FreimuthV, Damron D, et al.Attitudes and behaviors related
and research: a review of research. Chapter 5: nutrition education for
to fruits and vegetables among low-income women in the WIC pro-
adults. J Nutr Educ 1995;25:312-28.
gram. J Nutr Educ 1996;28:149-56.