- Email: [email protected]
Developing dietary assessment tools1
17. 18. 19. 20.
21.
RG, Potischman N, Schatzkin A, Hartman A, Swanson C, Kruse L, Hayes RB, Lewis DR, Harlan LC. Improving food frequency questionnaires: A qualitative approach using cognitive interviewing. J Am Diet Assoc. 1995;95:781-788. 1994-96, 1998 Continuing Survey of Food Intakes by Individuals [CD-ROM]. US Dept of Agriculture, Agricultural Research Service; 2000. Baranowski T, Domel SB. A cognitive model of children’s reporting of food intake. Am J Clin Nutr. 1994; 59(1 suppl):212S-217S. US Dept of Agriculture Nutrient Database for Standard Reference, Release No. 13, US Dept of Agriculture, Agricultural Research Service; 2001. Subar AF, Midthune D, Kulldorff M, Brown CC, Thompson FE, Kipnis V, Schatzkin A. Evaluation of alternative approaches to assign nutrient values to food groups in food frequency questionnaires. Am J Epidemiol. 2000;152:279-286. Willet W. Correction for the effects of measurement
22. 23.
24.
25. 26.
error. In: Nutritional Epidemiology. 2nd ed. New York, NY: Oxford University Press; 1998:302-320. Barr SI. Associations of social and demographic variables with calcium intakes of high school students. J Am Diet Assoc. 1994;94:260-266,269. Rockett HR, Wolf AM, Colditz GA. Development and reproducibility of a food frequency questionnaire to assess diets of older children and adolescents. J Am Diet Assoc. 1995;95:336-340. Rockett HR, Breitenbach M, Frazier AL, Witschi J, Wolf AM, Field AE, Colditz GA. Validation of a youth/ adolescent food frequency questionnaire. Prev Med. 1997;26:808-816. Cavadini C, Siega-Riz AM, Popkin BM. US adolescent food intake trends from 1965 to 1996. Arch Dis Child. 2000;83:18-24. Institute of Medicine, Food and Nutrition Board. Dietary Reference Intakes for Calcium, Phosphorus, Magnesium, Vitamin D, and Fluoride. Washington DC: National Academies Press; 1997.
APPLICATIONS
Developing Dietary Assessment Tools
W
hen investigators seek to assess dietary intakes at the population level, they need to consider many factors, such as how detailed, how population-specific, and how expensive a tool should be for their purposes. Although standardized methods exist for collecting food records and 24-hour dietary recalls, food frequency questionnaires (FFQs) tend to be more variable with respect to purpose (measuring total diet or selected foods/ nutrients), number of foods asked, whether or not (or how) portion size is queried, wording of questions, and food/nutrient database. The lure of the FFQ is its ease of administration, assessment of intake over an extended period of time, and low cost; the major drawbacks are less specificity and greater measurement error (1,2). An additional consideration with respect to frequencytype instruments, be they long or short, is whether to use “off the shelf ” questionnaires such as those developed at the National Cancer Institute (3), by Block (4), or by Willett (5), whether to adapt these instruments for specific uses if possible, or to develop a new tool. The paper by Jensen and colleagues (6) is an example of the latter— development of an FFQ targeted at specific age groups, specific races/ethnicities, and a single nutrient. As the authors indicate, doing this well requires a large time commitment and attention to detail if it is to succeed.
This article was written by Amy F. Subar, PhD, MPH, RD, Division of Cancer Control and Population Science, National Cancer Institute, Bethesda, MD. Amy is also on the Editorial Board of the Journal. doi: 10.1016/j.jada.2004.02.007
Fortunately, most researchers designing new tools are, at a minimum, able to borrow information from previous instruments, as was done for this calcium-specific FFQ. Here, the investigators used a youth-specific FFQ developed by Rockett and Colditz (7) as a model for how to format questions, but developed their own food list and nutrient database to meet their research needs. Creation of an FFQ The first rule of thumb in creating an FFQ for specific population subgroups is to seek out recent populationspecific data that can be used to determine the foods, portion sizes, and nutrient database to include. National dietary intake data for children indicate that only eight to 10 foods would be required to assess the majority of calcium intake (8). However, these data are not specific and do not contain enough respondents to assess such intake in Hispanic and/or Asian children. Therefore, it is clear why the investigators chose not to develop their instrument based on national dietary data. Ideally, investigators can create a data source by collecting recall or record data in the population of interest and use these data to determine the foods and portion sizes to appear on the FFQ and to create the nutrient database (Figure). Investigators can also choose to draw upon the judgment of nutrition professionals who are familiar with the population(s) of interest to create the food list. However this is done, extensive piloting or cognitive testing is useful to be sure that the target population both understands what is being queried and/or is capable of providing the answers sought by investigators. The article by Jensen and colleagues (6) shows some care
Journal of THE AMERICAN DIETETIC ASSOCIATION
769
Considerations in Selecting a Food Frequency Questionnaire (FFQ) Options ● Off the shelf ● Adapt a previously developed FFQ ● Develop (and validate) a new FFQ Pros ● Ease of administration ● Assess extended period of time ● Low cost Cons ● Less specificity ● Greater measurement error than other methods ● Validating the instrument can be time consuming and costly Selection ● Choose the most valid, accurate, and populationCriteria appropriate assessment method ● Consider if your needs are for individual- or group-level data* ● Choose a validated method whenever possible ● If additional information regarding a specific food or nutrient is needed, consider creating a specially developed questionnaire to address the food/nutrient of interest *Individual-level data are important for assessing intake in relation to specific biological or disease outcomes. Group-level data may be adequate for comparing mean nutrient intakes.
Figure. Considerations in selecting a food frequency questionnaire. in this regard as indicated in Figure 2— definitions of ethnic-specific foods provided to respondents to help them answer questions on the FFQ. Validating the FFQ Assessing the validity of any FFQ-type instrument is a difficult task. Without the existence of reference-type biomarkers, validating FFQs leave investigators with a sense of unease given that the reference instruments we compare them to, usually 24-hour recalls or food records, are flawed, measure intakes over different time periods, and have measurement error correlated with that on the FFQ (ie, underreporters on the FFQ also tend to underreport on the reference instrument) (1,2). However, there are currently no other reasonable methods for assessing validity of FFQs for most nutrients, so we use correlation coefficients adjusted for within-person variation based on statistical models that assume that the errors on FFQs and reference instruments are uncorrelated—an assumption we know is not true. Nevertheless, the correlations most investigators consider “good enough” tend to range from 0.4 to 0.7. The data from Jensen and colleagues indicate that their age-, race/ethnic-, and calcium-specific FFQ measured calcium intake well, with overall correlations around 0.7. However, their FFQ is highly specific
770
May 2004 Volume 104 Number 5
and may be of limited use given that it asks 80 foods to assess one nutrient, a number that is likely to be considered excessive for others interested in calcium intakes among youths. All instruments evolve over time. Further research on this FFQ should consider whether the instrument would still be valid if some foods that contributed less to overall calcium intakes were eliminated or whether the instrument performs well in other populations. Nutritionists often find themselves in the conundrum of wanting detailed and accurate information from an instrument without having the time or money necessary to either develop/ modify an instrument or its administration. It is a challenge to the profession to continue to approach instrument design with innovation, flexibility, and intent to share resources for others to build upon. Nutrition professionals in need of assessing intake of total diet or selected nutrients in a specific population should comprehensively research all resources to assess whether there are validated tools already available that can be used or adapted to meet their needs before embarking on the difficult task of designing and validating a new instrument. References 1. Subar AF, Kipnis V, Troiano RP, Midthune D, Schoeller DA, Bingham S, Sharbaugh CO, Trabulsi J, Runswick S, Ballard-Barbash R, Sunshine J, Schatzkin A. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: The OPEN Study. Am J Epidemiol. 2003;158:1-13. 2. Kipnis V, Subar AF, Midthune D, Freedman LS, Ballard-Barbash R, Troiano RP, Bingham S, Schoeller DA, Schatzkin A, Carroll RJ. The structure of dietary measurement error: Results of the OPEN biomarker study. Am J Epidemiol. 2003;158:14-21. 3. http://riskfactor.cancer.gov/tools/instruments/. Accessed January 16, 2004. 4. http://www.nutritionquest.com/. Accessed January 16, 2004. 5. http://www.hsph.harvard.edu/Academics/nutr/depart ment/Food%20Frequency%20Questionaire.html. Accessed January 16, 2004. 6. Jensen JK, Gustafson D, Boushey CJ, Auld G, Bock MA, Bruhn CM, Gabel K, Misner S, Novotny R, Peck L, Read M. Development of a food frequency questionnaire to estimate calcium intake in Asian, Hispanic and white youth. J Am Diet Assoc. 2004;104:762-769. 7. Rockett HR, Colditz GA. Assessing diets of children and adolescents. Am J Clin Nutr. 1997;65(4 Suppl): 1116S-1122S. 8. Subar AF, Krebs-Smith SM, Cook A, Kahle LL. Dietary sources of nutrients among US children: 198991. Pediatrics. 1998;102:913-923.
21.
RG, Potischman N, Schatzkin A, Hartman A, Swanson C, Kruse L, Hayes RB, Lewis DR, Harlan LC. Improving food frequency questionnaires: A qualitative approach using cognitive interviewing. J Am Diet Assoc. 1995;95:781-788. 1994-96, 1998 Continuing Survey of Food Intakes by Individuals [CD-ROM]. US Dept of Agriculture, Agricultural Research Service; 2000. Baranowski T, Domel SB. A cognitive model of children’s reporting of food intake. Am J Clin Nutr. 1994; 59(1 suppl):212S-217S. US Dept of Agriculture Nutrient Database for Standard Reference, Release No. 13, US Dept of Agriculture, Agricultural Research Service; 2001. Subar AF, Midthune D, Kulldorff M, Brown CC, Thompson FE, Kipnis V, Schatzkin A. Evaluation of alternative approaches to assign nutrient values to food groups in food frequency questionnaires. Am J Epidemiol. 2000;152:279-286. Willet W. Correction for the effects of measurement
22. 23.
24.
25. 26.
error. In: Nutritional Epidemiology. 2nd ed. New York, NY: Oxford University Press; 1998:302-320. Barr SI. Associations of social and demographic variables with calcium intakes of high school students. J Am Diet Assoc. 1994;94:260-266,269. Rockett HR, Wolf AM, Colditz GA. Development and reproducibility of a food frequency questionnaire to assess diets of older children and adolescents. J Am Diet Assoc. 1995;95:336-340. Rockett HR, Breitenbach M, Frazier AL, Witschi J, Wolf AM, Field AE, Colditz GA. Validation of a youth/ adolescent food frequency questionnaire. Prev Med. 1997;26:808-816. Cavadini C, Siega-Riz AM, Popkin BM. US adolescent food intake trends from 1965 to 1996. Arch Dis Child. 2000;83:18-24. Institute of Medicine, Food and Nutrition Board. Dietary Reference Intakes for Calcium, Phosphorus, Magnesium, Vitamin D, and Fluoride. Washington DC: National Academies Press; 1997.
APPLICATIONS
Developing Dietary Assessment Tools
W
hen investigators seek to assess dietary intakes at the population level, they need to consider many factors, such as how detailed, how population-specific, and how expensive a tool should be for their purposes. Although standardized methods exist for collecting food records and 24-hour dietary recalls, food frequency questionnaires (FFQs) tend to be more variable with respect to purpose (measuring total diet or selected foods/ nutrients), number of foods asked, whether or not (or how) portion size is queried, wording of questions, and food/nutrient database. The lure of the FFQ is its ease of administration, assessment of intake over an extended period of time, and low cost; the major drawbacks are less specificity and greater measurement error (1,2). An additional consideration with respect to frequencytype instruments, be they long or short, is whether to use “off the shelf ” questionnaires such as those developed at the National Cancer Institute (3), by Block (4), or by Willett (5), whether to adapt these instruments for specific uses if possible, or to develop a new tool. The paper by Jensen and colleagues (6) is an example of the latter— development of an FFQ targeted at specific age groups, specific races/ethnicities, and a single nutrient. As the authors indicate, doing this well requires a large time commitment and attention to detail if it is to succeed.
This article was written by Amy F. Subar, PhD, MPH, RD, Division of Cancer Control and Population Science, National Cancer Institute, Bethesda, MD. Amy is also on the Editorial Board of the Journal. doi: 10.1016/j.jada.2004.02.007
Fortunately, most researchers designing new tools are, at a minimum, able to borrow information from previous instruments, as was done for this calcium-specific FFQ. Here, the investigators used a youth-specific FFQ developed by Rockett and Colditz (7) as a model for how to format questions, but developed their own food list and nutrient database to meet their research needs. Creation of an FFQ The first rule of thumb in creating an FFQ for specific population subgroups is to seek out recent populationspecific data that can be used to determine the foods, portion sizes, and nutrient database to include. National dietary intake data for children indicate that only eight to 10 foods would be required to assess the majority of calcium intake (8). However, these data are not specific and do not contain enough respondents to assess such intake in Hispanic and/or Asian children. Therefore, it is clear why the investigators chose not to develop their instrument based on national dietary data. Ideally, investigators can create a data source by collecting recall or record data in the population of interest and use these data to determine the foods and portion sizes to appear on the FFQ and to create the nutrient database (Figure). Investigators can also choose to draw upon the judgment of nutrition professionals who are familiar with the population(s) of interest to create the food list. However this is done, extensive piloting or cognitive testing is useful to be sure that the target population both understands what is being queried and/or is capable of providing the answers sought by investigators. The article by Jensen and colleagues (6) shows some care
Journal of THE AMERICAN DIETETIC ASSOCIATION
769
Considerations in Selecting a Food Frequency Questionnaire (FFQ) Options ● Off the shelf ● Adapt a previously developed FFQ ● Develop (and validate) a new FFQ Pros ● Ease of administration ● Assess extended period of time ● Low cost Cons ● Less specificity ● Greater measurement error than other methods ● Validating the instrument can be time consuming and costly Selection ● Choose the most valid, accurate, and populationCriteria appropriate assessment method ● Consider if your needs are for individual- or group-level data* ● Choose a validated method whenever possible ● If additional information regarding a specific food or nutrient is needed, consider creating a specially developed questionnaire to address the food/nutrient of interest *Individual-level data are important for assessing intake in relation to specific biological or disease outcomes. Group-level data may be adequate for comparing mean nutrient intakes.
Figure. Considerations in selecting a food frequency questionnaire. in this regard as indicated in Figure 2— definitions of ethnic-specific foods provided to respondents to help them answer questions on the FFQ. Validating the FFQ Assessing the validity of any FFQ-type instrument is a difficult task. Without the existence of reference-type biomarkers, validating FFQs leave investigators with a sense of unease given that the reference instruments we compare them to, usually 24-hour recalls or food records, are flawed, measure intakes over different time periods, and have measurement error correlated with that on the FFQ (ie, underreporters on the FFQ also tend to underreport on the reference instrument) (1,2). However, there are currently no other reasonable methods for assessing validity of FFQs for most nutrients, so we use correlation coefficients adjusted for within-person variation based on statistical models that assume that the errors on FFQs and reference instruments are uncorrelated—an assumption we know is not true. Nevertheless, the correlations most investigators consider “good enough” tend to range from 0.4 to 0.7. The data from Jensen and colleagues indicate that their age-, race/ethnic-, and calcium-specific FFQ measured calcium intake well, with overall correlations around 0.7. However, their FFQ is highly specific
770
May 2004 Volume 104 Number 5
and may be of limited use given that it asks 80 foods to assess one nutrient, a number that is likely to be considered excessive for others interested in calcium intakes among youths. All instruments evolve over time. Further research on this FFQ should consider whether the instrument would still be valid if some foods that contributed less to overall calcium intakes were eliminated or whether the instrument performs well in other populations. Nutritionists often find themselves in the conundrum of wanting detailed and accurate information from an instrument without having the time or money necessary to either develop/ modify an instrument or its administration. It is a challenge to the profession to continue to approach instrument design with innovation, flexibility, and intent to share resources for others to build upon. Nutrition professionals in need of assessing intake of total diet or selected nutrients in a specific population should comprehensively research all resources to assess whether there are validated tools already available that can be used or adapted to meet their needs before embarking on the difficult task of designing and validating a new instrument. References 1. Subar AF, Kipnis V, Troiano RP, Midthune D, Schoeller DA, Bingham S, Sharbaugh CO, Trabulsi J, Runswick S, Ballard-Barbash R, Sunshine J, Schatzkin A. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: The OPEN Study. Am J Epidemiol. 2003;158:1-13. 2. Kipnis V, Subar AF, Midthune D, Freedman LS, Ballard-Barbash R, Troiano RP, Bingham S, Schoeller DA, Schatzkin A, Carroll RJ. The structure of dietary measurement error: Results of the OPEN biomarker study. Am J Epidemiol. 2003;158:14-21. 3. http://riskfactor.cancer.gov/tools/instruments/. Accessed January 16, 2004. 4. http://www.nutritionquest.com/. Accessed January 16, 2004. 5. http://www.hsph.harvard.edu/Academics/nutr/depart ment/Food%20Frequency%20Questionaire.html. Accessed January 16, 2004. 6. Jensen JK, Gustafson D, Boushey CJ, Auld G, Bock MA, Bruhn CM, Gabel K, Misner S, Novotny R, Peck L, Read M. Development of a food frequency questionnaire to estimate calcium intake in Asian, Hispanic and white youth. J Am Diet Assoc. 2004;104:762-769. 7. Rockett HR, Colditz GA. Assessing diets of children and adolescents. Am J Clin Nutr. 1997;65(4 Suppl): 1116S-1122S. 8. Subar AF, Krebs-Smith SM, Cook A, Kahle LL. Dietary sources of nutrients among US children: 198991. Pediatrics. 1998;102:913-923.