Reproducibility and relative validity of a self-administered semiquantitative food frequency questionnaire applied to younger women

] Clin Epidemiol Vol. 50, No. 3, pp. 303-311, Copyright 0 1997 Elsevier Science Inc.

0895-4356/97/$17.00 PII SOS95-4356(96)00379-4

1997

ELSEVIER

Reproducibility and Relative Validity of a SelfAdministered Semiquantitative Food Frequency Questionnaire Applied to Younger Women Sm-en Friis,’ AND

Susanne &-tiger

‘DANISH CANCER SOCIETY, DIVISION ‘INSTITUTE OF EPIDEMIOLOGY AND SOCIAL

Kjazr , ’ Connie

Stripp, ’ and Kim Overvud2

FOR CANCER EPIDEMIOLOGY, MEDICINE, AARHUS UNIVERSITY,

DK-2100 KBBENHAVN DK-8000 AARHUS

0, C,

DENMARK

ABSTRACT. We have evaluated the reproducibility and relative validity of a semiquantitative food frequency questionnaire (FFQ) used in a prospective study of risk factors for cervical neoplasia. The questionnaire is a modified version of one developed and evaluated in a middle-aged Danish population. In the present study, 122 women from the general population of Copenhagen, aged 20-29 years, completed the FFQ twice at a l-year interval, and provided three 4-day dietary records during the intervening year. The mean nutrient intakes calculated from the first and second questionnaire were similar and, for most nutrients, close to those obtained from the dietary records. The Pearson correlation coefficients between the mean nutrient intakes from the two questionnaires ranged from 0.53 (95% CI, 0.39-0.65) for vitamin E to 0.76 (95% CI, 0.67-0.83) for vitamin B12 (median, 0.67 [95% CI, 0.56-0.761). I n comparisons between the second FFQ and the dietary records, the correlations ranged from 0.24 (95% CI, 0.07-0.40) for vitamin D to 0.63 (95% CI, 0.51-0.73) for sucrose (median, 0.42 [95% CI, 0.26-0.561). Th e correlations between FFQ and dietary records were generally higher after adjustment for energy intake (median, 0.53 [95% CI, 0.39-0.651) and within-person variability (median, 0.64 [95% CI, 0.52-0.731). On average, 71% of the women were classified in the same (t 1) quintile in the second FFQ and the dietary records. An average of 3.8% of the women were grossly misclassified into the highest and lowest quintiles by the dietary records. The relative validity of the FFQ in this population was similar to that reported earlier. It is concluded that the FFQ is reproducible and provides a useful scale for categorizing individuals according to their intake of energy and nutrients. J CLIN EPIDEMIOL 50;3:303-311, 1997. 0 1997 Elsevier Science Inc. KEY WORDS. of results

Diet, dietary assessment methods, food frequency questionnaires,

INTRODUCTION The food frequency questionnaire (FFQ) has become an increasingly popular method for estimating dietary intake in epidemiological studies. Instead of measuring actual diet, as in short-term dietary recall and recording methods, FFQs are designed to measure long-term diet. The low costs of use of the FFQ, relative to other dietary assessment methods, make it applicable for large-scale studies involving thousands of individuals. Before an FFQ can be used in a specific study, however, its performance must be evaluated in a separate study to quantify its reproducibility and relative validity. Since an FFQ may perform differently in different cultural and demographic groups, a previously validated Address for correspondence: for Cancer Epldemlology, Kebenhavn 0, Denmark. Accepted for publication

Stiren Friis, Danish Strandboulevarden on

19 October

1996.

Cancer Society, 49, Box 839,

Division DK-2 100

dietary records, reproducibility

questionnaire must be evaluated for use in a population that is substantially different from that in which the FFQ was initially developed and validated. In the present study, we examined the reproducibility and relative validity of a modified version of a previously tested semiquantitative FFQ [l] among young women enrolled in a large-scale epidemiological study of risk factors for cervical neoplasia. Cumulative evidence implicates low intakes of vitamin C, beta-carotene, and possibly folate and vitamin E in the genesis of cervical neoplasia [2]. The questionnaire was designed to measure daily intake of these and other selected nutrients over the preceding year [3]. Multiple dietary records was chosen as the reference method, because this method is generally assumed to provide more accurate information on the full range of consumed foods than do other dietary methods, and because errors associated with dietary records are largely uncorrelated with errors associated with FFQs.

304

MATERIAL Subjects

S. Friis et al.

AND

METHODS

and Study Design

Over a two year period (1991-1993), a random sample of 16,345 women, aged 20-29 years and residing in Copenhagen, were invited to participate in a prospective populationbased study of risk factors for cervical neoplasia. Of those invited, 11,088 (68%) agreed to participate in the study, which included a personal interview on potential (non-dietary) risk factors, a gynecological examination, and collection of biological samples. At the interview, all participants were provided with instructions for filling in a semiquantitative FFQ, which they completed at home and returned by mail. A total of 9848 women (89% of participants; 60% of initial sample) returned the questionnaire. During the data collection phase, 300 women were invited to participate in a dietary validation study. For practical reasons, these women were selected by random sampling of two subpopulations of participants (sample A and B) who had visited the clinic during one of two 2-week periods two months apart and who had returned the dietary questionnaire. Similar proportions of women in the two samples agreed to participate: 59 of 120 (49%) women in sample A and 92 of 180 women (51%) in sample B. Approximately three months after the participants had completed the dietary questionnaire, an experienced dietician provided detailed instructions for recording and weighing all foods and beverages consumed during three 4day periods. The first period began the day after instruction, and the following two periods 4 and 8 months later. The days in the second and third periods were selected to allow all days of the week to be represented in the total 12-day period of dietary recording. The women were provided with a specially designed diary and a digital dietetic scale for weighing foods (Phillips model HR 2383), and they were instructed to measure precisely and describe in detail their consumption of all foods and beverages, including brand names, method of preparation, and recipe if possible. The instructions also included use of household measures as substitutes for weighed amounts for meals taken out of the home. After each period of recording, the participants returned the dietary records to the dietitian, who checked the records for completeness and errors and resolved any questions or inconsistencies with the participants. At the time of the last review after the third recording period, the participants were provided with the original semiquantitative FFQ, which they filled in at home and returned by mail. Twenty-two women were excluded because they did not fill in three complete sets of dietary records. Three women were excluded because they became pregnant during the study period. One woman was excluded because she reported having changed her diet significantly during the recording period. Also excluded from the analysis were three

women whose daily energy intake, as calculated from either of the two questionnaires, was outside the range of 3000 to 20,000 kJ [4,5]. One woman indicated unrealistic intakes in response to several questions in the first questionnaire. Therefore, the data from this woman’s first questionnaire were excluded. After exclusion, the analysis included 122 women, 48 women (40%) from sample A and 74 women (4 1%) from sample B. Demographic and other data for these women are presented in Table 1.

Food Frequency

Questionnaire

The self-administered questionnaire used in this study is a modified version of a semiquantitative FFQ that was developed for use in the Danish prospective study “Diet, Cancer and Health” [3]. A previous version of the questionnaire was validated in a pilot study among men and women aged 40-64 years [1,6]. The new questionnaire included additional food items considered to make an important contribution to the nutrient intake of younger women. In addition, more detailed questions were asked about the consumption of fruit and vegetables as these food items are main sources of vitamin C and carotenoids that have been associated fairly consistently with cervical neoplasia. Finally, open-ended questions were included for each food group in the questionnaire in order to identify foods not otherwise reported. The questionnaire was designed to rank individuals according to their average daily intake of selected nutrients considered to be of prime importance in human carcinogenesis [3]. The women were asked to indicate the average intake of each food item during the previous year on a scale of nine possible frequency responses. For foods that usually come in natural “units” (e.g., bread, milk, fruit), the serving sizes were based on assumed standard sizes for these “units.” For the remaining foods, we used standard portion sizes estimated from the data of the 1985 Danish National Dietary Survey [7,8], from the earlier validation study [I], and from the dietary records collected in the present study. Since the intake of foods specified as single items tends to be overestimated in FFQs [ 1,9], the questionnaire included a number of global questions designed to measure the frequency of intake of food groups. The sum of the frequencies of the specific items was compared with the global frequency; if the sum of the specific items exceeded the global frequency, overestimation was adjusted for on the basis of a weight, calculated as the global frequency divided by the sum of the specific frequencies. Such adjustment was especially necessary for meat and poultry, fish, vegetables, potatoes, and fruit. Daily intakes of energy and nutrients were calculated from the questionnaire on the basis of the official Danish food tables [lo]. Missing frequencies were assumed to be zero consumption. The use of vitamin supplements was not

Accuracy

TABLE

of a Semiquantitative

1.

Characteristics

Food

Frequency

of study participants

Age (ye=) Mean SD

Sample

No.

A

B

48 74

Total

122

Abbreviations:

SD = standard

305

Questionnaire

24.5 24.1 24.3

Married (%I

2.6 @ 2.7 deviation,

Schooling >lO years (%)

54 51 53 BMI

= body mass index,

98 97 98

Records

Three 4-day periods of dietary recording were chosen for validation in order to capture day-to-day and seasonal variation in dietary habits. The choice of three periods instead of one or two longer periods was also based on the assumption that the process of collecting dietary records tends to influence eating behavior [ 1 l] and that intakes on consecutive days are not independent [12-141. Furthermore, it has been reported that precision decreases after a few consecutive days of recording [ 151. The dietary records were coded by the same dietitian who collected and checked the records. For analysis, intake in grams was used, when recorded; when only household measures were available, standardized estimates in grams were applied. Single food items were coded according to the instructions of the computer program DANKOST [ 161, which calculates the intake of 46 nutrients on the basis of the official Danish food tables. The DANKOST system allows creation of specific recipes, and the women’s personal recipes were used if provided. Otherwise, standard recipes were constructed from a number of traditional Danish cookery books. Statistical

75 I!2 72

BMI

(kg/m’)

Mean

SD

21.8 21.9 21.9

2.8 j.J 3.0

Current smokers (%) 48 22 43

Current use of OCs (%) 44 42 43

OC = oral contraceptive.

uniformly noted on the questionnaires or dietary records and was therefore not considered in the calculations. Likewise, foods reported in the open-ended questions were not included in the computations, except for a mixture of butter and vegetable oils (KaergHrden), which was introduced in Denmark in 1990 and was used by many of the women in the study instead of regular butter. Dietary

Never pregnant (%)

Analyses

Group means and standard deviations of selected nutrient intakes were calculated from the three four-day dietary records (i.e., 12 days of recording) and from both FFQs. Because most distributions of nutrient intakes were skewed towards higher values, log, transformations were used to improve normality. A value of 0.001 was added to zero values to allow transformation of nutrients that were not consumed by all participants, Pearson correlation coefficients were used to compare the daily intake of nutrients from the two questionnaires and the dietary records. Spearman rank sum correlation coefficients were also computed, but since

the two analyses produced similar results, only Pearson correlations are presented. Energy-adjusted nutrient intakes were calculated by replacing the intake values from the dietary records and both questionnaires with their respective residuals from a linear regression model, with the log, of nutrient intake as the dependent and the log, of total energy intake as the independent variable [ 171. The crude and energy-adjusted correlation coefficients (Pearson) were corrected for variability due to within-person variation in nutrient intake as assessed in the dietary records [18-201. As the food intake on consecutive days tends to be correlated [ 12- 141, the average daily intake from each 4-day period was used in the analysis as the random unit of observation, The variance components due to within- and between-person variation were estimated from an ANOVA model [21]. The degree of misclassification across categories between the second questionnaire and the dietary records was examined by dividing nutrient intake into quintiles. Cut-off points were determined separately for the questionnaire and for the dietary records. Since the emphasis in this article is on quantification of measurement error rather than on hypothesis testing, confidence intervals are only presented for selected correlation coefficients. All analyses were performed using the SAS statistical package [22].

RESULTS The means and standard derivations of energy and nutrient intake calculated from the two questionnaires and the three 4-day dietary records are presented in Table 2. The mean values estimated from the first and second questionnaires were similar and, for most nutrients, close to those obtained from the dietary records. The intakes of energy and energyproviding nutrients were higher when estimated from the dietary records, the intake of energy being more than 20% higher. Relative overreporting in the questionnaires was observed for beta-carotene, with intakes twice as high as those in the dietary records. The mean percentages of energy in-

306

TABLE 2. Mean semiquantitative

S. Friis et al.

(and food

SD) daily frequency

intake of selected nutrients estimated from three 4.day dietary records questionnaire (FFQ) completed by 122 women aged 20-29 years

WQl

(n P?22)

mQ.2 (n = 122)

(II = 121)

Nutrient

Mean

SD

Energy (kl) Protein (g) Total fat (g) Cholesterol (mg) Carbohydrates, total (g) Sucrose (g) Dietary fiber (g) Alcohol (g) Retinol (pg) Beta-carotene (pup) Vitamin D (fig) Vitamin E (mg) Vitamin Bl (mg) Vitamin B2 (mg) Vitamin B6 (mg) Folate (pg) Vitamin B12 (pg) Vitamin C (mg) Calcium (mg) Magnesium (mg) Iron (mg) Zinc (mg) Percent calories from Fat Protein Carbohydrate

8907 68.7 81.4 265 252 47.2 19.0 12.3 412 3522 2.3 5.7 1.0 1.5 1.2 241 4.3 76.3 973 274 9.6 9.4

1772 14.4 22.0 98.1 57.1 26.7 6.8 11.1 280 2979 1.6 1.8 0.2 0.4 0.3 82.6 2.1 39.7 328 82.6 2.3 2.1

7367 63.0 62.3 294 220 48.9 19.0

34.6 13.2 48.2

5.7 1.7 7.0

Mean

(DR)

SD

Mean

47:.4 7294 2.2 5.7 1.0 1.7 1.2 309 4.7 82.9 922 287 8.1 9.4

2098 19.0 21.8 127 70.4 36.8 8.5 5.7 323 7779 1.1 2.6 0.3 0.7 0.4 108 2.6 51.6 473 139 2.8 3.0

7210 61.0 60.3 281 216 53.5 18.0 6.8 421 6939

31.9 14.7 50.7

5.7 2.4 6.9

and

from

a

(FFQ 14 (II = 171y SD

Mean

SD

7842 66.9 67.5 328 233 63.1 18.2 6.1 517 7925 2.5 6.3 1.0 1.8 1.2 313 5.4 92.1 1007 287 8.2 9.8

2779 24.7 28.0 145 89.7 50.3 7.8

::i 1.0 1.6 1.2 290 4.4 76.5 869 268 7.8 9.0

1938 17.4 20.4 133 69.5 42.0 7.3 6.6 241 7861 1.1 2.4 0.3 0.6 0.4 94.7 2.1 51.3 399 115 2.4 2.7

31.7 14.5 50.8

5.9 2.2 6.7

32.5 14.7 50.4

5.7 2.6 6.7

31::: 10683 1.3 3.2 0.4 0.8 0.5 115 2.8 60.1 569 133 2.7 3.7

“Comprises non-participants (n = 149) and participants with an incomplete set of dietary records (n = 22)

take from the energy-providing nutrients (fat, protein, and carbohydrates) were similar on the dietary records and the questionnaires. Non-participants had similar or somewhat higher mean nutrient intake values in a comparison of data from the first FFQ. Pearson coefficients for the correlation between the two FFQs ranged from 0.53 (95% CI, 0.39-0.65) for vitamin E to 0.76 (95% CI, 0.67-0.83) for vitamin B12 (median, 0.67 [95% CI, 0.56-0.761) for crude nutrient intake, including energy intake (Table 3). After adjustment for total energy intake, similar correlations were observed (range, 0.47 [95% CI, 0.31-0.591 to 0.74 [95% CI, 0.65-0.811; median 0.66 [95% CI, 0.55-0.751). Table 4 shows the correlations between nutrient intake from the two questionnaires and the average nutrient intake from the dietary records. The correlations for the first questionnaire were not appreciably different from those for the second questionnaire. Crude correlations between the first questionnaire and the dietary records ranged from 0.21 (95% CI, 0.03-0.37) for vitamin E to 0.63 (95% CI, OS0.73) for magnesium (median, 0.44 [95% CI, 0.28-0.571). A similar analysis for the second questionnaire provided correlations ranging from 0.24 (95% CI, 0.07-0.40) for vi-

tamin D to 0.63 (95% CI, 0.51-0.73) for sucrose, with a median correlation of 0.42 (95% CI, 0.26-0.56). Partial correlation coefficients adjusted for age and body mass index differed only slightly from the unadjusted coefficients (data not shown). Adjustment of nutrient intakes for total energy intake increased most correlations between the dietary records and the questionnaires (median correlations, 0.54 [95% CI, 0.40-0.661 and 0.53 [95% CI, 0.39-0.651, respectively). The correlations improved markedly for the energy-providing nutrients, particularly for fat and carbohydrates. The correlations for most other nutrients increased minimally or remained stable. Most of the adjusted values were between 0.4 and 0.7. Table 4 also shows the ratio of within-person to betweenperson variance in nutrient intake (o~/o[), as calculated from crude and energy-adjusted dietary record values. These ratios illustrate different day-to-day variability in nutrient intake. Nutrients consumed regularly (e.g., the energy-providing nutrients) had low ratios, whereas the ratios for nutrients with high within-person variability (e.g., retinol, beta-carotene, and vitamin D) were higher, indicating that 12 days of recording may not be sufficient to estimate usual

Accuracy of a Semiquantitative

307

Food Frequency Questionnaire

quency

Reproducibility of the semiquantitative food frequestionnaire: Pearson correlation coefficients

Nutrient”

Crude

TABLE

3.

Correlations

Energy Protein Total fat Cholesterol Carbohydrates, total Sucrose Dietary fiber Alcohol Retinol Beta-carotene Vitamin D Vitamin E Vitamin Bl Vitamin B2 Vitamin B6 Folate Vitamin B12 Vitamin C Calcium Magnesium Iron Zinc Percent calories from Fat Protein Carbohydrate ‘All

nutrient “Adjusted

values for total

0.61 0.67 0.62 0.64 0.60 0.73 0.60 0.63 0.68 0.68 0.71 0.53 0.62 0.71 0.67 0.66 0.76 0.73 0.72 0.64 0.54 0.68

Adjustedb 0.66 0.65 0.64 0.64 0.65 0.64 0.65 0.71 0.68 0.69 0.47 0.66 0.65 0.66 0.72 0.74 0.71 0.66 0.68 0.55 0.71

DISCUSSION

0.64 0.63 0.64

were loge transformed energy intake according

to improve FO Willett

normality. and Stampfer

be in the highest dietary record quintile were classified into the lowest quintile as estimated from the FFQ. Of the women categorized into the highest quintile on the basis of the dietary record, an average of 44% (24-64%) fell into the same quintile and 70% (56-92%) fell into the highest two quintiles when categorized by the second questionnaire. Similarly, 47% (20-64%) of the women in the lowest dietary record quintile fell into the lowest questionnaire quintile, and 70% (48-880/ o ) were categorized into the lowest two questionnaire quintiles. Similar results were found for unadjusted nutrient intakes (data not shown).

1171.

intake [ 111. When these ratios were used to correct correlations for deattenuation due to within-person variation [ 1820] (Table 4), the improvements were considerable. The corrected correlations between the first questionnaire and the dietary records ranged from 0.32 (95% CI, 0.15-0.47) for energy to 0.81 (95% CI, 0.74-0.86) for alcohol (median, 0.66 [95% CI, 0.55-0.751). Similar results were obtained in comparisons between the second questionnaire and the dietary records (range, 0.30 [95% CI, 0.13-0.451 to 0.88 [95% CI, 0.83-0.921; median, 0.64 [95% CI, 0.52-0.731). To evaluate further the ability of the FFQ to rank subjects by level of nutrient intake, we examined cross-classifications of adjusted nutrient intakes from the dietary records and the second questionnaire (Table 5). An average of 71% (58-84%) of the women were classified into the same or an adjacent quintile. Gross misclassification, i.e., classification of subjects into one extreme quintile on the basis of the dietary record but into the other extreme quintile by the FFQ, was seen for an average of 3.8% of the women. This gross misclassification was due primarily to misclassification of women from the highest quintile of intake measured by the dietary records to the lowest quintile of intake measured by the FFQ. For total fat, for example, 12% (three people) of the women estimated from the dietary records to

The reproducibility and relative validity of the FFQ in this study were comparable to those seen in studies of other FFQs [23-321 and to theresults of an earlier evaluation of the FFQ [l]. The studies are difficult to compare, because of differences in study populations, between-person variation in nutrient intake, and number of days of dietary recording. The problem of varying length of dietary recording among studies can be reduced, however, if comparisons are made by level of correlation corrected for within-person variation. In our study the .median values of the corrected correlations for the first andsecond administration of the questionnaire were similar to,those obtained in other studies in which adjustment were made for day-to-day variation in dietary record intakes [26,27,29,30]. Moreover, the correction for within-person variation illustrates that the low correlations observed for some nutrients (e.g., vitamin D and E) were due to high within-person variation (see Table 4), and that 12 days of recording were not sufficient to estimate the average long-term intake
308

TABLE

nutrient

S. Friis et al.

4. Comparison of nutrient scores from the first and second food frequency scores from dietary records Pearson

correlation

coefficients

WQl Nutrients” Energy Protein Total fat Cholesterol Carbohydrates, total Sucrose Dietary fiber Alcohol Retinol Beta-carotene Vitamin D Vitamin E Vitamin Bl Vitamin B2 Vitamin B6 Folate Vitamin B12 Vitamin C Calcium Magnesium Iron Zinc Percent calories from Fat Protein Carbohydrate

Crude 0.29 0.31 0.37 0.42 0.42 0.63 0.52 0.54 0.42 0.48 0.33 0.21 0.38 0.54 0.46 0.52 0.53 0.59 0.52 0.45 0.30 0.36 0.61 0.36 0.65

Adjusted’ 0.36 0.62 0.54 0.64 0.65 0.54 0.51 0.45 0.48 0.41 0.24 0.39 0.54 0.54 0.55 0.55 0.58 0.52 0.57 0.38 0.39

Crude

mQ2 Adjusted’

0.32 0.47 0.71 0.76 0.72 0.75 0.60 0.81 0.61 0.66 0.76 0.37 0.66 0.62 0.74 0.67 0.71 0.72 0.59 0.63 0.56 0.49

0.27 0.28 0.38 0.38 0.34 0.63 0.59 0.58 0.50 0.46 0.24 0.36 0.28 0.54 0.38 0.45 0.53 0.53 0.54 0.50 0.27 0.36

0.41 0.49 0.45 0.56 0.69 0.65 0.56 0.53 0.45 0.29 0.40 0.37 0.58 0.59 0.52 0.53 0.55 0.56 0.72 0.40 0.53

0.69 0.45 0.73

0.50 0.41 0.57

Correctedd

questionnaire

Correctedd 0.30 0.52 0.56 0.64 0.63 0.80 0.73 0.88 0.72 0.62 0.54 0.62 0.62 0.66 0.80 0.63 0.69 0.68 0.64 0.80 0.59 0.67

(FFQ 1 and 2) with

mean

Variance ratio b (cYoi3 Crude Adjusted’ 0.71 0.83 0.73 1.94 0.71 1.01 0.70 3.07 1.52 2.33 5.71 2.41 2.05 0.61 1.40 1.10 1.62 1.65 0.62 0.54 1.14 0.92

1.83 0.88 3.00 0.77 1.04 0.76 4.48 2.54 2.69 7.24 4.21 5.52 0.94 2.56 1.43 2.02 1.63 0.89 0.71 3.46 1.82

0.57 0.51 0.64

“All nutrient values were loge transformed to improve normality. hRatio of the within- to the between-person variance estimated from the foil owing ANOVA model: Y,) = p + A, + 4, where Y, is the average nutrient intake in the jth four-day recording period of the ith subject, A, represents between-person variation and c,,within-person variation. ‘Adjusted for total energy intake according to Willett and Stampfer [17]. “Corrected for within-person variability using the following formula. T<= ~“41 + [(d/d)/,,], where T
chosen from the study population, and there were similar attendance rates (40% and 42%) and other characteristics (Table 1). Consequently, we decided to report the results for the total group, considering the difference between the two samples as random error. Dietary records have been reported to underestimate energy intake, especially among obese individuals [5,31,33]. Using tables developed by Goldberg et al. [4] and predictions of basal metabolic rate from bodyweight on each participant [34], we identified eight women (bodyweight, range 47.484.7 kg) with implausibly low energy intakes (range, 43866430 kJ), which was due to obvious underreporting of habitual energy intake in the dietary records. Exclusion of these women from the analysis did not markedly change the results. At the group level, there was no indication of major underreporting in the dietary records, and the observed mean energy intake from the dietary records was similar to that for women in the 1985 Danish National Dietary Survey [7,8] and to that estimated on the basis of basal metabolic

rate and the average energy requirement for low physical activity [4]. Still, some underreporting of total energy intake was apparent among obese women, as was recently demonstrated in other Danish women [35]; among 14 women with body mass index higher than 25, four (29%) had implausible low energy intakes, whereas only four of 108 women with BMI lower than 25 were identified with implausible low energy intakes (4%). The mean intake of beta-carotene calculated from the questionnaires was twice as high as that estimated from the dietary records. Examination of the questionnaires suggested that the intake of carrots was in some cases overreported, owing to misinterpretation of the questions asked about this specific food item. The ranking of subjects was, however, acceptable when evaluated by correlation analysis and cross-classification. In future revisions of the questionnaire, efforts should be made to avoid this apparent overreporting. The intake of nutrients other than beta-carotene was

Accuracy

TABLE

of a Semiquantitative

Food

Frequency

Questionnaire

309

Cross-classification of nutrient intake into quintiles” after adjustment for total energy intake.6 (DR) and the second food frequency questionnaire (FFQ 2) (in percent)

5.

records

Lowest Lowest quintile onFFQ2 Energy Protein Total fat Cholesterol Carbohydrates, Sucrose Dietary fiber Alcohol Retinol Beta-carotene Vitamin D Vitamin E Vitamin Bl Vitamin B2 Vitamin B6 Folate Vitamin B12 Vitamin C Calcium Magnesium Iron Zinc

total

estimated

68 68 60 64 72 88 80 76 56 76 48 60 60 88 76

44 40 40 36 48 52 60 56 32 52 20 48 36 64 52 52 48 60 56 60 48 40

“Defined using the distribution ‘Except for total energy.

generally

quintile on DR Lowest 2 Highest quintiles quintile onFFQ2 onFFQ2

to

be

lower

4

ii: 60 68

from

intake

the

from

from

quintile on DR Highest 2 Lowest quintiles quintile onFFQ2 onFFQ2 60 68 72

i: 92 72 60 60 60 68 68 72 72 68 68 68 88 56 68

12 4 12 4 0 0 0 0 4 12 8 12 8 : 4 4 8 4 Y 4

Percentage within one quintile 60 68 66 75 67 82 75 75 65 72 58 65 65 77 75 76 70 75 70 84 66 73

12 days of diet records.

questionnaires

the dietary records. In most other studies of FFQs, subjects have been found to overreport their consumption of individual food items within food groups, thereby overestimating the true overall frequencies [9]. Few studies have adjusted for overestimation, using global questions to weight down the specific questions [1,28]. Such adjustment was necessary for several food groups. We chose to use the global questions as the standard, but it is possible that the sum of the specific frequencies is more valid for some foods. In both questionnaires, the mean energy intake calculated from unweighted nutrient values was similar to that determined from the dietary records. In interpreting these results, it should be borne in mind that the questionnaire was designed to rank individuals by level of intake and not to measure the “whole” diet. The first version of the questionnaire was estimated to cover approximately 80% of the average total intake of selected nutrients in the study population of middle-aged men and women [3]. This corresponds well to the results in this study, in which the mean energy intake calculated from either questionnaire, using weighted nutrient values, was some 20% lower than that estimated from the dietary records. Although the present study was conducted in a representative sample of women from the main study, we considered that the results could be biased if the participants (41% of than

24 32 48 52 40 48 56 56 52 40 32 36 40 52 48 64 44 48 44 52 32 36

: 8 8 0 1 0 0 2 3 0 3 1 2 1 1 1 0 1 4 3

it 80

of mean nutrient

Highest Highest quintile onFFQ2

Data from dietary

the invited women) were atypical in some way with regard to their eating pattern or their ability to complete a questionnaire or record their diet. In a comparison of participants and non-participants, they were found to be similar with respect to the intake of most nutrients, the mean percentages of energy-providing nutrients from total energy, and the number of blanks in the questionnaires. Furthermore, there were no significant differences between participants and non-participants with regard to a number of nondietary variables. We have described the relative validity of the FFQ in terms of correlation coefficients and percentages of individuals correctly classified in cross-classifications. The use of correlation coefficients to assess the relative validity of dietary assessment methods has been strongly criticized [36,37]. One argument is that correlation coefficients do not capture differential under- and overreporting. The extent of such bias is, however, difficult to address or quantify. Another possible problem in the use of correlation coefficients is the importance of between-person variation in nutrient intakes in the population. Although this dependence limits generalization of the results, it can also be viewed as an advantage, as the capacity of a questionnaire to assess dietdisease relationships depends on both its accuracy and the between-person variation, which is reflected in the correlation coefficient [l 11. At present, more appropriate analyses

310

in validation studies in nutritional epidemiology remain to be developed, although methods for achieving a more profound understanding of the variation and the sources of error underlying different dietary assessment methods have been proposed [38]. Recently, Garrow has advocated use of simple cross-classification of methods in dietary validation studies, arguing that this approach will make the different methods more readily comparable [39]. In our study, the relative validity of the FFQ for classifying subjects into quintiles was in agreement with that reported for other FFQs [1,23,24,27-291, and the gross misclassification was also comparable with that observed in most other studies. In conclusion, the reproducibility and relative validity observed in this study suggest a reasonable ability of the semiquantitative FFQ to rank individuals by levels of intake. Furthermore, the mean values for most nutrients calculated from the questionnaire were close to those obtained from the dietary records.

S. Friis et al.

14.

15.

16. 17.

18.

19.

20.

This study was funded by the National Cancer

Institute, USA, and the Danish Cancer Society. We are grateful to Mr. Anders Mplller, Danish Food Agency, for assistance in the computation of nutrients, and to Ms. Kutja Boll for technical assistance.

References 1. Tjanneland A, Overvad K, Haraldsdbttir J, Bang S, Ewertz M, Jensen OM. Validation of a semiquantitative food frequency questionnaire developed in Denmark. Int J Epidemiol 1991; 20: 906-912. 2. Potischman N, Brinton LA. Nutrition and cervical neoplasia. Cancer Causes Control 1996; 7: 113-126. 3. Overvad K, Tjanneland A, Haraldsdbttir J, Ewertz M, Jensen OM. Development of a semiquantitative food frequency questionnaire to assess food, energy and nutrient intake in Denmark. Int J Epidemiol 1991; 20: 900-905. 4. Goldberg GR, Black AE, Jebb SA, et al. Critical evaluation of energy intake data using fundamental principles of energy physiology: 1. Derivation of cut-off limits to identify underrecording. Eur J Clin Nutr 1991; 45: 569-581. 5. Haraldsdhttir J, Sandstriim B. Detection of underestimated energy intake in young adults. Int J Epidemiol 1994; 23: 577582. 6. Tjsnneland A, Overvad K, Thorling E, Ewertz M. Adipose tissue fatty acids as biomarkers of dietary exposure in Danish men and women. Am J Clin Nutr 1993; 57: 629-633. 7. Haraldsdbttir J, Holm L, Jensen JH, Mgller A. Dietary Habits in Denmark 1985; I. Main Results. Publ. No. 136. Copenhagen: Danish National Food Agency; 1986. 8. Haraldsdbttir J, Holm L, Jensen H, MGller A. Dietary Habits in Denmark; II. Who Eats What? Publ. No. 154. Copenhagen: Danish National Food Agency; 1987. 9. Haraldsdbttir J. Minimizing error in the field: Quality control in dietary surveys. Eur J Clin Nutr 1993; 47(Suppl. S2): S19-S24. 10. Meller A. Levnedsmiddeltabeller (Food Composition Tables). Copenhagen: Danish National Food Agency; 1989. 11. Willett WC. Nutritional Epidemiology. New York: Oxford University Press; 1990. 12. El Lozy M. Dietary variability and its impact on nutritional epidemiology. J Chron Dis 1983; 36: 237-249. 13. Morgan KJ, Johnson SR, Goungetas B. Variability of food in-

21.

22. 23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

takes. An analysis of a 12-day data series using persistence measures. Am J Epidemiol 1987; 126: 326-335. Tarasuk V, Beaton GH. The nature and individuality of within-subject variation in energy intake. Am J Clin Nutr 1991; 54: 464-470. Hankin JH, Wilkens LR, Kolonel LN, Yoshizawa CN. Validation of a quantitative diet history method in Hawaii. Am J Epidemiol 1991; 133: 616-628. Kostberegningsprogrammet DANKOST, Version 1.2.a. Copenhagen, Denmark: Danish National Food Agency; 1989. Willett W, Stampfer MJ. Total energy intake: Implications for epidemiologic analyses. Am J Epidemiol 1986; 124: 1727. Beaton GH, Milner J, Corey P, et al. Sources of variance in 24-hour dietary recall data: Implications for nutrition study design and interpretation. Am J Clin Nutr 1979; 32: 25462549. Liu K, Stamler J, Dyer A, McKever J, McKever I’. Statistical methods to assess and minimize the role of intra-individual variability in obscuring the relationship between dietary lipids and serum cholesterol. J Chron Dis 1978; 31: 399-418. Rosner B, Willett WC. Interval estimates for correlation coefficients corrected for within-person variation: Implications for study design and hypothesis testing. Am J Epidemiol 1988; 127: 377-386. Hunter DJ, Sampson L, Stampfer MJ, Colditz GA, Rosner B, Willett WC. Variability in portion sizes of commonly consumed foods among a population of women in the United States. Am J Epidemiol 1988; 127: 1240-1249. SAS/STAT User’s Guide, Version 6, Fourth Edition. Gary, NC, USA: SAS Institute Inc.; 1990. Willett WC, Sampson L, Stampfer MJ, et al. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol 1985; 122: 51-65. Pietinen P, Hartman AM, Haapa E, et al. Reproducibility and validity of dietary assessment instruments. II. A qualitative food frequency questionnaire. Am J Epidemiol 1988; 128: 667-676. Engle A, Lynn LL, Koury K, Boyar AI’. Reproducibility and comparability of a computerized, self-administered food frequency questionnaire. Nutr Cancer 1990; 13: 281-292. Block G, Thompson FE, Hartman AM, Larkin FA, Guire KE. Comparison of two dietary questionnaires validated against multiple dietary records collected during a l-year period. J Am Diet Assoc 1992; 92: 686-693. Rimm EB, Giovannucci EL, Stampfer MJ, Colditz GA, Litin LB, Willet WC. Reproducibility and validity of an expanded self-administered semiquantitative food frequency questionnaire among male health professionals. Am J Epidemiol 1992; 135: 1114-1126. Nes M, Frost Andersen L, Solvoll K, et al. Accuracy of a quantitative food frequency questionnaire applied in elderly Norwegian women. Eur J Clin Nutr 1992; 46: 809-821. Martin-Moreno JM, Boyle P, Gorgojo L, et al. Development and validation of a food frequency questionnaire in Spain. Int J Epidemiol 1993; 22: 512-519. Longnecker MI’, Lissner L, Holden JM, et al. The reproducibility and validity of a self-administered semiquantitative food frequency questionnaire in subjects from South Dakota and Wyoming. Epidemiology 1993; 4: 356-365. Lindroos AK, Lissner L, SjGstram L. Validity and reproducibility of a self-administered dietary questionnaire in obese and non-obese subjects. Eur J Clin Nutr 1993; 47: 461-481. Johansson G, Hallmans G, Eriksson S, et al. Evaluation of the accuracy of a dietary questionnaire aimed for the VHsterbotten study. &and J Nutr 1994; 38: 50-55.

Accuracy

33.

34.

35.

36.

of a Semiquantitative

Food

Frequency

311

Questionnaire

Black AE, Prentice AM, Goldberg GR, et al. Measurements of total energy expenditure provide insights into the validity of dietary measurements of energy intake. J Am Diet Assoc 1993; 93: 572-579. FAO/WHO/UNU. Report of a Joint Expert Consultation. Energy and Protein Requirements. WHO Tech Rep Series No. 724. Geneva: WHO; 1985. Heitmann BL, Lissner L. Dietary underreporting by obese individuals-Is it specific or non-specific? Br Med J 1995; 311: 986-989. Bellach 8. Remarks on the use of Pearson’s correlation coeffi-

37.

38.

39.

cient and other association measures in assessing validity and reliability of dietary assessment methods. Eur J Clin Nutr 1994; 47(Suppl. 2): S42-S45. Kohlmeier L. Gaps in dietary assessment methodology: Mealvs. list-based methods. Am J Clin Nutr 1994; 59s: 175S179s. Plummer M, Clayton D. Measurement error in dietary assessment: An investigation using covariance structure models. Part I and II. Stat Med 1993; 12: 925-948. Garrow JS. Validation of methods for estimating habitual diet: Proposed guidelines. Eur J Clin Nutr 1995; 49: 231-232.