Development and Validation of a Beverage and Snack Questionnaire for Use in Evaluation of School Nutrition Policies

RESEARCH Research and Professional Briefs

Development and Validation of a Beverage and Snack Questionnaire for Use in Evaluation of School Nutrition Policies MARIAN L. NEUHOUSER, PhD, RD; SONYA LILLEY, MS, RD; ANNE LUND, MPH, RD; DONNA B. JOHNSON, PhD, RD

ABSTRACT School nutrition policies limiting access to sweetened beverages, candy, and salty snacks have the potential to improve the health of children. To effectively evaluate policy success, appropriate and validated dietary assessment instruments are needed. The objective of this study was to develop and validate a beverage and snack questionnaire suitable for use among young adolescents. A new 19-item Beverage and Snack Questionnaire (BSQ) was administered to middle school students on two occasions, 2 weeks apart, to measure test–retest reliability. The questionnaire inquired about frequency of consumption, both at school and away from school, of soft drinks, salty snacks, sweets, milk, and fruits and vegetables. Students also completed 4-day food records. To assess validity, food-record data were compared with BSQ data. Forty-six students of diverse backgrounds from metropolitan Seattle, WA, participated in this study. Participants answered the BSQ during class time and completed the food record at home. Pearson correlation coefficients assessed test–retest reliability and validity. Using frequency per week data, the test–retest reliability coefficients were r⫽0.85 for fruits and vegetables consumed at school and r⫽0.74 and r⫽0.72 for beverages and sweets/

M. L. Neuhouser is an associate member in the Cancer Prevention Program at the Fred Hutchison Cancer Research Center, Seattle, WA. S. Lilley is a registered dietitian, Cedars Sinai Medical Center, Los Angeles, CA; at the time of the study, she was a graduate student at the University of Washington, Seattle. A. Lund is a project coordinator at the Center for Public Health Nutrition, University of Washington, Seattle. D. B. Johnson is an associate professor in the Interdisciplinary Program in Nutritional Sciences and Department of Health Services, University of Washington, and codirector of the Center for Public Health Nutrition, University of Washington, Seattle. Address correspondence to: Marian L. Neuhouser, PhD, RD, Cancer Prevention Program, Fred Hutchinson Cancer Research Center, 1100 Fairview Ave North, M4B402, PO Box 19024, Seattle, WA 98109-1024. E-mail: [email protected] Manuscript accepted: March 20, 2009. Copyright © 2009 by the American Dietetic Association. 0002-8223/09/10909-0009$36.00/0 doi: 10.1016/j.jada.2009.06.365

© 2009 by the American Dietetic Association

snacks, respectively, consumed at school. Correlations ranged from r⫽0.73 to 0.77 for foods consumed outside of school. Compared with the criterion food record, validity coefficients were very good: r⫽0.69 to 0.71 for foods consumed at school and r⫽0.63 to 0.70 for foods consumed away from school. The validity coefficients for the 19 individual food items ranged from r⫽0.56 to 0.87. This easy-to-administer 19-item questionnaire captures data on sugar-sweetened beverages, salty snacks, sweets, milk, and fruit and vegetables as well as a more lengthy and expensive food record does. The BSQ can be used by nutrition researchers and practitioners to accurately evaluate student consumption of foods that are the focus of school nutrition policies. J Am Diet Assoc. 2009;109:1587-1592.

S

chool food environments influence the eating practices and nutritional status of American students (1-3). Of particular concern is easy access to unhealthful sugar-sweetened beverages, salty snacks, and sweets (1,4-7). The Institute of Medicine’s Committee on Nutrition Standards for Foods in Schools has developed rigorous guidelines regarding the composition of foods permissible in schools (4). State legislatures, school districts, and individual schools have also developed policies about food availability (8). However, little is known about the efficacy of widespread school policy change (4,9,10). One evaluation study indicated that policies limiting atschool access to soft drinks, chips, and candy improved student dietary behaviors, but much remains to be learned (11). Effective evaluation of school nutrition policies requires assessment of students’ dietary behaviors (12) and appropriate data collection instruments (9,10,13,14). However, existing tools have limitations; for example, food frequency questionnaires (FFQs) cannot capture eating location, and 24-hour recalls and food records are cost- and time-prohibitive (15-17). Further, recalls and records provide far more information than is needed for evaluation of a particular set of school-based dietary behaviors, such as soft drink consumption. Alternatively, short focused questionnaires have been successfully used to assess targeted dietary behaviors or intake of specific foods/nutrients (15,18-20). For example, instruments have been developed to assess fat and fiber behaviors (16,21), fruit and vegetable consumption (15,20), and calcium intake (18). Although some of these tools were developed for use in preadolescent or adolescent populations (18,21), others

Journal of the AMERICAN DIETETIC ASSOCIATION

1587

were developed for adults (15,16,19). Adult-based tools would not be appropriate for large-scale evaluations of school-based dietary behaviors because of questionnaire length (22-24), limited reliability (25,26), and lack of specificity toward foods addressed by school policies (27-29). This report describes the design, reliability, and validity of the Beverage and Snack Questionnaire (BSQ), which was developed as part of the Policy Legislation and Nutrition research project. METHODS Participants/Recruitment Participants were 7th-grade students in metropolitan Seattle, WA. To obtain a diverse sample, an upper-income suburban school and a lower-income inner-city school participated. Parents/guardians received informational study packets and signed informed consent. Students signed informed assent and received a $25 gift certificate upon study completion. Procedures were approved by the University of Washington Institutional Review Board. BSQ A multidisciplinary team used a three-stage process to develop the BSQ. The goals were to develop an instrument that could assess consumption of specific foods targeted by school nutrition policies (such as soft drinks, savory snacks, and sweets), and ensure that the assessment tool was valid and reliable across a range of racial/ ethnic groups and socioeconomic status. To meet these goals, the team first drafted 19 questions, based on previously published literature, to assess student consumption of foods that are commonly found in schools (1,3,6,7). The food list included items high in energy, but poor in nutrients; foods not permitted under the Institute of Medicine’s nutrition guidelines (4); as well as foods frequently missing from the diets of US adolescents, such as fruit, vegetables (30), and milk (31). The BSQ structure and format were based on a validated questionnaire of savory snack consumption (32). For stage 2 of development, the BSQ was cognitively pretested with 31 middle school students from various racial/ethnic and socioeconomic backgrounds. Students were asked to “think out loud” as they completed the questionnaire to assess the feasibility of using this format in middle school students and to determine whether students understood the words and food categories provided. Using qualitative feedback from the cognitive pretesting, modest changes were made to the questionnaire, including clarifying the food options, adding more brand names, and giving more examples. In stage 3, the BSQ was revised and pretested a second time with 50 students in two classrooms. The final BSQ included 19 items inquiring about the frequency of consumption during the previous week of nine beverages (100% juices; fruit drinks such as Capri Sun [Kraft Foods, Inc, Glenview, IL]/Snapple [Dr. Pepper Snapple Group, Plano, TX]; sport drinks such as Gatorade [Quaker Oats Co, PepsiCo, Chicago, IL]/Powerade [The Coca-Cola Company, Atlanta, GA]; flavored waters including vitamin-fortified water, diet, and regular soft drinks, energy drinks [eg, RockStar (RockStar Inc, Las

1588

September 2009 Volume 109 Number 9

Vegas, NV)/Red Bull (Red Bull GmbH, Fuschl am See, Austria)]; 1% or nonfat milk and whole/regular milk); eight salty snacks and sweets (low-fat/nonfat potato, tortilla/corn chips; regular potato, tortilla/corn chips; other snacks such as crackers; candy; doughnuts, Pop-Tarts (Kellogg Company, Battle Creek, MI), and pastries; cookies, brownies, cakes; low-fat frozen desserts, standard ice cream, and milkshakes); and two questions about fruit and vegetable consumption. Seven frequency response options were provided, ranging from “never or less than once/week” to “4⫹ times/day.” For each item, participants marked how often in the previous week they consumed the item listed both for “consumption at school” and “consumption away from school.” Mean frequency per week was calculated for each item at each location and these means were used for analysis. The BSQ included five questions about demographic characteristics (age, sex, race/ethnicity, and eligibility for the free and reduced-price school meal program as a proxy for income). BSQ Testing Procedures The BSQ was administered on two occasions (as test– retest), separated by 4 to 6 weeks. During the week prior to the second BSQ administration, participants completed a 4-day food record. 4-Day Food Records The food record booklet included detailed instructions about recording all intake (including beverages and snacks) and location of consumption (33). Students recorded 3 school days and 1 weekend day. For analysis, each item on the food record was first re-coded as a binary variable into one of 19 food categories that matched the BSQ line items and locations (eg, “at school” and “not at school”). For example, if a student consumed cereal with milk at home it was coded as a “1” for “milk at home” and if he or she consumed a cola beverage at school it was coded as a “1” for “soft drink at school.” Coding rules for fractional assignments were established for mixed foods, such as coffee beverages with milk (eg, lattes) and main dishes with portions of vegetables (eg, stir fry or casseroles). All variables were then multiplied by the number of times it appeared in the record to yield mean frequency per week, which was used in the analysis. Statistical Analysis Descriptive statistics assessed the overall data patterns; data were normally distributed. Student t tests compared the mean frequency of items reported on each instrument for each location of consumption (home and school). Pearson correlations assessed reliability and validity of the overall BSQ (34,35), the component sections (beverages, snacks and sweets; and fruits and vegetables), and all individual questions (1 through 19). The analyses were intended to demonstrate whether a food type (eg, soft drinks, salty snacks, sweets) and location of consumption on the food record (criterion measure) were also captured on the BSQ (first BSQ administration), thus indicating validity. Separate analyses evaluated the performance of the BSQ by demographic characteristics. SPSS was used for

all statistical analyses (SPSS, version 15.0, 2005, SPSS Inc, Chicago, IL). RESULTS AND DISCUSSION Of 46 study participants, 18 (39.1%) were male and 28 (60.9%) were female; mean age was 12.7 years. More than half (56.5%) of students were white; the remainder were Asian (15.2%), black/African-American (13.0%), Hispanic (6.5%), and other (American Indian, mixed race) (15.2%). The percentage of the study body who were minority was 93.9% at the urban school and 21.4% at the suburban school. Few students at the suburban school (1.9%) qualified for free or reduced-price school meals, whereas 74.9% of urban students qualified for reduced-priced meals. On average, the reported frequency of consumption was similar between the BSQ and the food record. For items consumed at school there were no significant differences (using a t test); mean frequency of consumption of all foods did not differ between the BSQ and the food record. For example, on average students reported consuming candy, orange juice, and milk once per week at school on both the food record and the BSQ. For foods consumed at school, although the mean frequency of consumption was nearly identical for milk (all kinds), doughnuts/breakfast pastries, ice cream, fruit, and vegetables, mean frequencies were slightly higher on the BSQ compared with the food record for sport drinks, flavored waters, energy drinks, and salty snacks (all P⬍0.05) (data not shown). The test–retest reliability of the BSQ was high (Table 1). The reliability coefficients ranged from r⫽0.72 for frequency of “snacks and sweets consumed at school” to r⫽0.85 for “fruits and vegetables consumed at school.” There were no apparent differences in reliability with regard to whether the foods were consumed at school (r⫽0.70) or away from school (r⫽0.67). There were also no differences in reliability between the two schools. Subgroup analysis demonstrated that reliability coefficients for suburban students were r⫽0.80 for foods eaten at school and r⫽0.70 for foods eaten away from school, whereas the reliability coefficients for urban students were r⫽0.77 for foods eaten at school and r⫽0.77 for foods eaten away from school. Table 2 gives data on BSQ validity. Pearson correlations are provided for location and food type for the BSQ compared with the mean frequency of intake from the food record. For estimates of beverages, snacks and sweets, and fruits and vegetables consumed at school, the correlations of the BSQ with the food record were r⫽0.71, 0.70, and 0.69, respectively. Estimates were similar for these foods consumed away from school and ranged from r⫽0.63 for fruits and vegetables to r⫽0.70 for beverages. The BSQ performed well in comparison to the criterion food record. For food consumed at school, correlations for frequency of consumption ranged from r⫽0.56 to 0.87 for beverages; r⫽0.61 to 0.79 for snacks and sweets, and r⫽0.66 and 0.72 for vegetables and fruits, respectively. Similarly, for foods consumed away from school, the BSQ performed well, with some variation in the correlations. For example, the correlation of the BSQ with the food record for energy drinks and milk consumed away from school was r⫽0.65 and 0.66, respectively, but the correlation for these beverages consumed at school was

r⫽0.56. The correlation of the BSQ with the food record for “away from school cookies, brownies, pies and cakes” was r⫽0.48, whereas it was r⫽0.65 for “at school.” Validity results for students at the suburban school had correlation coefficients of r⫽0.66 for foods eaten at school and r⫽0.59 for foods not eaten at school, whereas students at the urban school scored 0.71 for foods eaten at school and 0.77 for foods eaten away from school. The principal finding from this study among students from a range of backgrounds and socioeconomic status was that the 19-item BSQ captured data on frequency and location of consumption of specific foods (eg, soft drinks, salty snacks) in a manner that is comparable to collecting the same information (frequency and location) with a multiple-day food record (21,29). Because evaluation of school nutrition policies that limit availability of soft drinks and sweets has the potential to improve the body weight and nutritional status of American children, there is an urgent need to clearly understand the extent to which the policies are effective (1,10,11,36). The BSQ, shown here to be a simple, quick, and reliable method to obtain data on beverage, snack, and sweets consumption both at school and away from school, can be used by nutrition researchers and evaluators to measure student dietary behaviors and assess the impact of school nutrition polices. The reliability and validity coefficients for the BSQ were similar in magnitude to those reported elsewhere. In a study evaluating the measurement characteristics of a short screener to assess adult fruit and vegetable intake, correlations of the screener with a longer FFQ-type instrument overall ranged from 0.51 to 0.68 (20), but the validity of individual items was poorer, with correlations of the screener (test instrument) vs 24-hour recalls ranging from 0.25 for fruit juice consumption in women to more than 0.70 for fruit (20). For a 19-item food checklist developed for evaluation of community-level intervention studies, there was excellent agreement between items reported on the checklist and 24-hour dietary recalls (13). In a study that developed a focused recall specifically intended to assess intake of fruit, vegetables, and carotenoids, correlations of carotenoid intake from a full recall with the focused-recall ranged from 0.63 to 0.70, whereas those for fruit and vegetable consumption were 0.42 to 0.56 (15). Few suitable instruments have been explicitly developed and validated for use in adolescents. Yaroch and colleagues developed and tested a qualitative dietary fat index questionnaire in a sample of African-American adolescent females (21). As with the study presented here, validity was assessed by comparing results from the qualitative dietary fat index questionnaire to a reference instrument (three 24-hour recalls), and reliability was assessed by test–retest of the instrument. The validity and reliability coefficients reported in the Yaroch study were weaker than those for the BSQ, the former reported as r⫽0.54 for reliability and r⫽0.31 to ⫺0.23 for validity (21). In a separate study (37), Yaroch and colleagues reported results from the development of a picture-sort FFQ for use with low-income African-American adolescents. The test–retest correlations for the two administrations of the picture-sort FFQ ranged from 0.30 to 0.48 for macronutrients. A Web-based FFQ was developed and validated for adolescents. However, compared with 3-day

September 2009 ● Journal of the AMERICAN DIETETIC ASSOCIATION

1589

Table 1. Pearson correlations (test–retest reliability) of food items on a Beverage and Snack Questionnaire (BSQ) tested in middle school students (n⫽46) BSQ food categories

Location of consumption

Pearson correlations coefficients (test–retest reliability)

Beverages

School Not at school School Not at school School Not at school

0.74 0.77 0.72 0.75 0.85 0.73

Snacks and sweets Fruits and vegetables

BSQ food items Orange juice, apple juice, and other 100% juice Fruit drinks such as Snapplea, Capri Sunb, and Kool-Aidb Sport drinks such as Gatoradec and PowerAded Flavored waters such as Propele or vitamin-fortified waters Diet soda or pop Regular soda or pop Energy drinks such as RockStarf, Red Bullg, Monsterh, and Throttled 1% or nonfat milk Regular or 2% milk (whole and reduced fat) Low-fat or nonfat potato chips, tortilla chips, and corn chips Regular potato chips, tortilla chips, puffs, and corn chips Other salty snacks such as cheese nibs, Chex mixi, and Ritz Bitsj Candy, including chocolate, Jelly Belliesk, gummies, and Life Saversl Doughnuts, Pop-Tartsm, breakfast pastries Cookies, brownies, pies, and cakes Low-fat or nonfat frozen desserts such as low-fat ice cream and frozen yogurt Regular ice cream and milkshakes Vegetables, including green salad, peas, green beans, corn (not including fried potatoes or french fries) Fruit, such as banana, apple, or grapes (does not include juice)

Pearson correlations coefficients (test–retest reliability) for items consumed at school

Pearson correlations coefficients (test–retest reliability) for items not consumed at school

0.65 0.74 0.69 0.76 0.80 0.73

0.89 0.79 0.65 0.79 0.72 0.86

0.64 0.84 0.78 0.66 0.71 0.71

0.68 0.74 0.74 0.67 0.79 0.79

0.78 0.80 0.83

0.74 0.85 0.73

0.61 0.62

0.61 0.81

0.83 0.88

0.63 0.82

a

Dr. Pepper Snapple Group, Plano, TX. Kraft Foods, Inc, Glenview, IL. c Quaker Oats Co, PepsiCo, Chicago, IL. d Coca-Cola Co, Atlanta, GA. e Stokely-Van Kamp, Inc, Chicago, IL. f RockStar Inc, Las Vegas, NV. g Red Bull GmbH, Fuschl am See, Austria. h Monster Beverage Company, Corona, CA. i General Mills, Inc, Minneapolis, MN. j Nabisco World, Kraft Foods, Glenview, IL. k Jelly Belly, Inc, Fairfield, CA. l Wrigley Company, Mars, Incorporated, Chicago, IL. m Kellogg Company, Battle Creek, MI. b

food records, the measurement properties were modest (26). The mean reliability was r⫽0.62, but the validity coefficients ranged from r⫽0.20 to r⫽0.64 for individual foods. This compares to the BSQ validity correlations for individual foods, which ranged from 0.56 to 0.87 (26). Finally, it is worth noting that test questionnaires that elicit portion-size information may introduce an additional source of measurement error, which could possibly attenuate correlations to lower values than we observed

1590

September 2009 Volume 109 Number 9

with the BSQ. Correlations may have been higher in this report than could be expected in the absence of measurement error. This study has several strengths. First, the BSQ was developed and tested using a three-stage approach incorporating cognitive evaluation and two pilot tests. Second, students from schools of varying racial and socioeconomic backgrounds participated. Third, although dietary assessment is a complex task requiring cognitive develop-

Table 2. Pearson correlations (validity) of Beverage and Snack Questionnaire (BSQ) compared to a 4-day food record tested in middle school students (n⫽47) BSQ food category

Location of consumption

Validity coefficient

Beverage

School Not at school School Not at school School Not at school

0.71 0.70 0.70 0.66 0.69 0.63

Snacks and sweets Fruits and vegetables

BSQ food items Orange juice, apple juice, and other 100% juice Fruit drinks such as Snapplea, Capri Sunb, and Kool-Aidb Sport drinks such as Gatoradec and PowerAded Flavored waters such as Propele or vitamin-fortified waters Diet soda or pop Regular soda or pop Energy drinks such as RockStarf, Red Bullg, Monsterh, and Throttled 1% or nonfat milk Regular or 2% milk (whole and reduced fat) Low-fat or nonfat potato chips, tortilla chips, and corn chips Regular potato chips, tortilla chips, puffs, and corn chips Other salty snacks such as cheese nibs, Chex mixi, and Ritz Bitsj Candy, including chocolate, Jelly Belliesk, gummies, and Life Saversl Doughnuts, Pop-Tartsm, breakfast pastries Cookies, brownies, pies, and cakes Low-fat or nonfat frozen desserts such as low-fat ice cream and frozen yogurt Regular ice cream and milkshakes Vegetables, including green salad, peas, green beans, corn (not including fried potatoes or french fries) Fruit, such as banana, apple, or grapes (does not include juice)

Pearson correlations (validity coefficients) for items consumed at school

Pearson correlations (validity coefficients) for items not consumed at school

0.61 0.87 0.80 0.68 0.80 0.75

0.69 0.68 0.63 0.75 0.79 0.69

0.56 0.74 0.56 0.67 0.69 0.74

0.65 0.72 0.66 0.71 0.71 0.70

0.79 0.65 0.65

0.72 0.76 0.48

0.61 0.73

0.59 0.62

0.66 0.72

0.68 0.60

a

Dr. Pepper Snapple Group, Plano, TX. Kraft Foods, Inc, Glenview, IL. Quaker Oats Co, PepsiCo, Chicago, IL. d Coca-Cola Co, Atlanta, GA. e Stokely-Van Kamp, Inc, Chicago, IL. f RockStar Inc, Las Vegas, NV. g Red Bull GmbH, Fuschl am See, Austria. h Monster Beverage Company, Corona, CA. i General Mills, Inc, Minneapolis, MN. j Nabisco World, Kraft Foods, Glenview, IL. k Jelly Belly, Inc, Fairfield, CA. l Wrigley Company, Mars, Incorporated, Chicago, IL. m Kellogg Company, Battle Creek, MI. b c

ment, retrieval of events from memory, and literacy, participants performed these tasks well. There are also limitations to this study. The response rate was lower from the low-income vs the upper/middleclass school, so additional validation work is clearly needed. The BSQ is intended to be used for populationbased evaluation of school food policies. Because thousands of students will complete the BSQ, practicality and cost-efficiency necessitates a finite number of questions and questions that are closed-ended. Students may con-

sume beverages and snacks that do not appear on the BSQ. Finally, all self-reported dietary assessment instruments are subject to misreporting and measurement error (38,39). Food records may not be the perfect gold standard/criterion measure because they are subject to measurement error (38,39). In conclusion, the BSQ was developed as a cost-effective, rapid tool to assess the effectiveness of school nutrition policies, particularly those limiting access to sugar-sweetened beverages, salty snacks, and sweets. The BSQ can play

September 2009 ● Journal of the AMERICAN DIETETIC ASSOCIATION

1591

an important role in helping nutrition researchers and members of evaluation teams evaluate changes associated with school-based nutrition policies and build school nutrition environments that promote long-term health and optimize the nutritional status of America’s children.

19.

20.

STATEMENT OF POTENTIAL CONFLICT OF INTEREST: No potential conflict of interest was reported by the authors. FUNDING/SUPPORT: Funding was provided by the Robert Wood Johnson Foundation Healthy Eating Research Program and the Fred Hutchinson Cancer Research Center.

21.

22. 23.

References 1. Kubik MY, Lytle LA, Story M. Soft drinks, candy, and fast food: What parents and teachers think about the middle school food environment. J Am Diet Assoc. 2005;105:233-239. 2. Grimm GC, Harnack L, Story M. Factors associated with soft drink consumption in school-aged children. J Am Diet Assoc. 2004;104:12441249. 3. Rajeshwari R, Yang SJ, Nicklas TA, Berenson GS. Secular trends in children’s sweetened-beverage consumption (1973 to 1994): The Bogalusa Heart Study. J Am Diet Assoc. 2005;105:208-214. 4. Institute of Medicine, Food and Nutrition Board. Nutrition Standards for Foods in Schools: Leading the Way Towards a Healthier Youth. Washington, DC: National Academies Press; 2007. 5. Johnston LD, Delva J, O’Malley PM. Soft drink availability, contracts, and revenues in American secondary schools. Am J Prev Med. 2007; 33:S209-S225. 6. French SA, Lin BH, Guthrie JF. National trends in soft drink consumption among children and adolescents age 6 to 17 years: Prevalence, amounts, and sources, 1977/1978 to 1994/1998. J Am Diet Assoc. 2003;103:1326-1331. 7. Rampersaud GC, Bailey LB, Kauwell GP. National survey beverage consumption data for children and adolescents indicate the need to encourage a shift toward more nutritive beverages. J Am Diet Assoc. 2003;103:97-100. 8. Boehmer TK, Luke DA, Haire-Joshu DL, Bates HS, Brownson RC. Preventing childhood obesity through state policy predictors of bill enactment. Am J Prev Med. 2008;34:333-340. 9. Briggs M, Safaii S, Beall DL. Position of the American Dietetic Association, Society for Nutrition Education, and American School Food Service Association—Nutrition services: An essential component of comprehensive school health programs. J Am Diet Assoc. 2003:505514. 10. McDonnell E, Probart C, Weirich JE, Hartman T, Bailey-Davis L. School competitive food policies: Perceptions of Pennsylvania public high school foodservice directors and principals. J Am Diet Assoc. 2006;106:271-276. 11. Cullen KW, Watson K, Zakeri I. Improvements in middle school student dietary intake after implementation of the Texas Public School Nutrition Policy. Am J Public Health. 2008;98:111-117. 12. Framework for program evaluation in public health. MMWR Recomm Rep. 1999;48:1-40. 13. Kristal AR, Abrams BF, Thornquist MD, Disogra L, Croyle RT, Shattuck AL, Henry HJ. Development and validation of a food use checklist for evaluation of community nutrition interventions. Am J Public Health. 1990;80:1318-1322. 14. Kristal AR, Goldenhar L, Muldoon J, Morton RF. Evaluation of a supermarket intervention to increase consumption of fruits and vegetables. Am J Health Promotion. 1997;11:422-425. 15. Neuhouser ML, Patterson RE, Kristal AR, Eldridge AL, Vizenor NC. A brief dietary assessment instrument for assessing target foods, nutrients and eating patterns. Public Health Nutr. 2001;4:73-78. 16. Shannon J, Kristal AR, Curry SJ, Beresford SA. Application of a behavioral approach to measuring dietary change: The fat- and fiberrelated diet behavior questionnaire. Cancer Epidemiol Biomarkers Prev. 1997;6:355-361. 17. Zoumas-Morse C, Rock CL, Sobo EJ, Neuhouser ML. Children’s patterns of macronutrient intake and associations with restaurant and home eating. J Am Diet Assoc. 2001;101:923-925. 18. Jensen JK, Gustafson D, Boushey CJ, Auld G, Bock MA, Bruhn CM, Gabel K, Misner S, Novotny R, Peck L, Read M. Development of a food

1592

September 2009 Volume 109 Number 9

24. 25.

26. 27.

28.

29.

30. 31. 32. 33. 34. 35. 36.

37.

38.

39.

frequency questionnaire to estimate calcium intake of Asian, Hispanic, and white youth. J Am Diet Assoc. 2004;104:762-769. Thompson FE, Midthune D, Subar AF, Kipnis V, Kahle LL, Schatzkin A. Development and evaluation of a short instrument to estimate usual dietary intake of percentage energy from fat. J Am Diet Assoc. 2007;107:760-767. Thompson FE, Subar AF, Smith AF, Midthune D, Radimer KL, Kahle LL, Kipnis V. Fruit and vegetable assessment: Performance of 2 new short instruments and a food frequency questionnaire. J Am Diet Assoc. 2002;102:1764-1772. Yaroch AL, Resnicow K, Petty AD, Khan LK. Validity and reliability of a modified qualitative dietary fat index in low-income, overweight, African American adolescent girls. J Am Diet Assoc. 2000;100:15251529. Rockett HR, Breitenbach M, Frazier AL, Witschi J, Wolf AM, Field AE, Colditz GA. Validation of a youth/adolescent food frequency questionnaire. Prev Med. 1997;26:808-816. Rockett HRH, Wolf AM, Colditz GA. Development and reproducibility of a food frequency questionnaire to assess diets of older children and adolescents. J Am Diet Assoc. 1995;95:336-340. Cullen KW, Watson K, Zakeri I. Relative reliability and validity of the Block Kids’ Questionnaire among youth aged 10 to 17 years. J Am Diet Assoc. 2008;108:862-866. Field AE, Peterson KE, Gortmaker SL, Cheung L, Rockett H, Fox MK, Colditz GA. Reproducibility and validity of a food frequency questionnaire among fourth to seventh grade inner-city school children: Implications of age and day-to-day variation in dietary intake. Public Health Nutr. 1999;2:293-300. Matthys C, Pynaert I, De Keyzer W, De Henauw S. Validity and reproducibility of an adolescent web-based food frequency questionnaire. J Am Diet Assoc. 2007;107:605-610. Moore GF, Tapper K, Murphy S, Clark R, Lynch R, Moore L. Validation of a self-completion measure of breakfast foods, snacks and fruits and vegetables consumed by 9- to 11-year-old schoolchildren. Eur J Clin Nutr. 2007;61:420-430. Haraldsdóttir J, Thórsdóttir I, de Almeida MD, Maes L, Pérez Rodrigo C, Elmadfa I, Frost Andersen L. Validity and reproducibility of a precoded questionnaire to assess fruit and vegetable intake in European 11- to 12-year-old schoolchildren. Ann Nutr Metab. 2005;49:221227. Marshall TA, Eichenberger Gilmore JM, Broffitt B, Levy SM, Stumbo PJ. Relative validation of a beverage frequency questionnaire in children ages 6 months through 5 years using 3-day food and beverage diaries. J Am Diet Assoc. 2003;103:714-720. Larson NI, Neumark-Sztainer D, Hannan PJ, Story M. Trends in adolescent fruit and vegetable consumption, 1999-2004: Project EAT. Am J Prev Med. 2007;151:147-150. Kranz S, Lin P, Wagstaff D. Children’s dairy intake in the United States: Too little, too fat? J Pediatr. 2007;151:642-646. Neuhouser ML, Patterson RE, Kristal AR, Rock CL, NeumarkSztainer D, Thornquist MD, Cheskin LJ. Do consumers of savory snacks have poor quality diets? J Am Diet Assoc. 2000;100:576-579. Kolar AS, Patterson RE, White E, Neuhouser ML, Frank LL, Standley J, Potter JD, Kristal AR. A practical method for collecting 3-day food records in a large cohort. Epidemiology. 2005;16:579-583. Breslow NE, Day NE. Statistical methods in cancer research. Vol II, the analysis of cohort studies. Lyon, France: International Agency for Research on Cancer; 1987. Willett W. Nutritional Epidemiology. New York, NY: Oxford University Press, 1998. Stables GJ, Young EM, Howerton MW, Yaroch AL, Kuester S, Solera MK, Cobb K, Nebeling L. Small school-based effectiveness trials increase vegetable and fruit consumption among youth. J Am Diet Assoc. 2005;105:252-256. Yaroch AL, Resnicow K, Davis M, Davis A, Smith M, Khan LK. Development of a modified picture-sort food frequency questionnaire administered to low-income, overweight, African-American adolescent girls. J Am Diet Assoc. 2000;100:1050-1056. Subar A, Kipnis V, Troiano RP, Midthune D, Schoeller DA, Bingham S, Sharbaugh CO, Trabulsi J, Runswick S, Ballard-Barbash R, Sunshine J, Schatzkin A. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: The OPEN Study. Am J Epidemiol. 2003;158:1-13. Neuhouser ML, Tinker L, Shaw PA, Schoeller D, Bingham SA, Horn LV, Beresford SA, Caan B, Thomson C, Satterfield S, Kuller L, Heiss G, Smit E, Sarto G, Ockene J, Stefanick ML, Assaf A, Runswick S, Prentice RL. Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women’s Health Initiative. Am J Epidemiol. 2008;167:1247-1259.