- Email: [email protected]
Assessing Food Choice in School Children: Reliability and Construct Validity of a Method Stacking Food Photographs
Appetite, 1998, 30, 25–37
Assessing Food Choice in School Children: Reliability and Construct Validity of a Method Stacking Food Photographs
INGER JONSSON and LENA GUMMESON Department of Home Economics, University of Go¨teborg, Go¨teborg, Sweden
MARK CONNER Department of Psychology, University of Leeds, Leeds, U.K.
ELISABETH SVENSSON Department of Mathematics, Chalmers University of Technology, Go¨teborg, Sweden
A method utilizing pictures was used to assess food choice with 44 school boys in three different age groups (10, 13 and 16 years old). The aim of the study was to test reliability and construct validity (i.e. inter-instrument agreement) of food choice assessed when the boys were asked to compose breakfasts. The focus was on choices of milk, margarine, bread and breakfast cereals, as in these food groups it is possible to choose between low-fat/high-fat and low-fibre/high-fibre products. Subjects were asked to repeat their composing of breakfasts after eight weeks, and to complete a four-day food record on breakfasts after the second interview occasion. For comparisons between food choices a new non-parametric statistical method was used. Both reliability and construct validity were good for choices of milk and margarine. The unreliability could be explained as random in nature or in terms of healthier choices in the second interview. 1998 Academic Press Limited
I Mertz and Kelsay (1984) have pointed out that foods are not fully represented by their nutrient composition. It might, in certain contexts, be more appropriate to study choices of foods and types of products within food groups. For example, it is generally agreed that non-communicable diseases in many populations would decrease with a lower fat and a higher fibre intake. If a certain food group constitutes the predominant source of a nutrient in a population, as dairy products do for fat intake in several Western countries, it is interesting to examine whether, for example, choices Supported in part by a grant from the Swedish National Institute of Health. The authors would like to express their appreciation to Britt-Marie Sandstro¨m for helpful comments on an earlier version of this manuscript. Address correspondence to: Inger Jonsson, Department of Home Economics, University of Go¨teborg, Box 12 204, 402 42 Go¨teborg, Sweden. 0195–6663/98/010025+13 $25.00/0/ap970113
1998 Academic Press Limited
26
I. JONSSON ET AL.
of milk and cheese with a high fat content decrease as a result of health campaigns. In some epidemiology research, intake of foods and food groups rather than nutrients has been related to a lower risk of disease (Willet, 1990). Studies with groups representing whole populations and studies concerning certain subgroups have often different purposes, and need different approaches. For population screening, questionnaire techniques are appropriate. For studies with smaller groups, carried out with the purpose of getting a more detailed picture of food habits, and learning about reasons for food choices, interviews are often more suitable. Wardle (1993) argues that “sociological and anthropological approaches are well-established as methods of evaluating cultural beliefs and diet and health, but are based on unsystematic interviews and observations, and often use very small samples”, and points out that “there is a pressing need for development of assessment methods which reflect the point of view of the individual, while at the same time using reproducible measures and deriving quantitative data”. Reliability, validity and usability are the key characteristics for any measure of dietary habits. In developing measures of such habits for use with children the limited linguistic capacities of the respondents is likely to be an important factor influencing each of these characteristics. Measures that require sophisticated linguistic abilities are unlikely to produce reliable or valid results with younger children. One way to reduce the impact of linguistic skills is to make the task more visual in nature. If the child is required to select a food that it normally consumes from a selection of pictures the task is a simpler identification task rather than a task in verbally describing choices. Obviously the child still has to understand the verbal instructions and translate these instructions into a choice. However, the skills required are comprehension skills rather than skills of expression. In such a case, language is much less likely to be a barrier to adequate measurement. An approach with the above-mentioned characteristics was used to study school children’s food choice and their perceptions of the appropriateness of different combinations of food items in school lunches (Ahlstro¨m, Baird & Jonsson, 1990). From a specially designed apparatus, a stacking box, participants were asked to select cards with pictures of food items (one food item on each card) for meals according to given instructions, for example “Make up a breakfast you usually eat” and “Make up a healthy breakfast”. Each box consists of a topless solid aluminium box (25 cm×25 cm×6 cm deep) with 25 compartments ordered as a matrix five times five. Each compartment can hold up to ten pictures (all of a single food) stacked on top of each other. When one picture is removed a spring device in the bottom of the compartment allows the card below to pop up into view. This gives a uniform investigation situation, since the subject is not influenced by previous choices, as he/ she cannot see how many pictures of each food item they have already used. The approach makes written language a minor factor, which is important for exploring differences in food choice and preference for foods with school children at different grade levels. Through the construction of instructions to participants this use of photographs can be adapted to assess a variety of questions. The aim of the present study was to test reliability and construct validity (i.e. inter-instrument agreement) of a food choice measure based on utilization of food photographs with school children in three different age groups. Another aim was to apply a non-parametric statistical method in the evaluation of food choices. Photographs have, besides their common use as visual aids to decide quantities in dietary assessments, been used in studies concerning the satiating effects of foods (de Graff
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
27
et al., 1992) and children’s preferences for and knowledge of healthy/unhealthy foods (Calfas, Sallis & Nader, 1991) and meals (Gutezeit, Bloth & Hagenow, 1995). Williams et al. (1993) used visual aids (based on packaging) to distinguish milks of different fat content, and margarines of different levels of fat saturation. As far as we are aware no other study has used non-parametric methods to assess the disagreement between repeated food choices concerning the fibre and fat content recorded as ordered categorical data.
M Subjects The participants were 44 school boys from three grades (fourth, seventh, and tenth) who were 10, 13 and 16 years old. There were approximately equal numbers of boys from each grade (fourth and seventh grades, 15 boys each; tenth grade, 14 boys). The study was carried out in a suburb of Go¨teborg, Sweden, which has a socio-economically diverse population, and a rather low share of immigrants. Foodstuffs The breakfast meal was chosen to illustrate food choices, as a typical Swedish breakfast meal contains food items with several alternatives concerning fat and fibre content. Such foods are principally milk products and margarine, bread and breakfast cereals. Colour photographs of 50 different food items were available in two stacking boxes. Among these there were four types of milk with different fat contents, and five types of margarine, which varied in fat content and quality (butter is included here and will be through the paper). Five types of bread and four types of breakfast cereals, both product categories with a variation in fibre content, were also available. Foods within the four food categories were also chosen to represent the most frequently sold foods on the Swedish market. The rest of the 50 photographs covered foods typical of Swedish breakfast habits. These foods do not contribute at all or only to a limited extent to the fat and fibre intake compared to milk and margarine or bread and breakfast cereals. Moreover, most of them have no alternatives differing in fat and fibre content. For reasons of health it is important to diminish fat intake and increase fibre intake. This is most simply brought about in a population if people choose low-fat and high-fibre alternatives within those food categories where there are such alternatives. Colour photographs (5 cm×5 cm) were made of each item shown lying on a plate. Each photograph was mounted on a rigid card and covered by transparent plastic. To make it easier for subjects to find a certain type of food within a food group, pictures of milks or margarines or breads etc. were presented spatially close to one another in the stacking boxes. Procedure Subjects were asked to select food items from the stacking boxes according to different instructions: “Make up the breakfast you had this morning”; “Make up a breakfast you usually eat”; “Make up a breakfast you would like to eat” and “Make
28
I. JONSSON ET AL.
up a healthy breakfast”. The subjects chose freely among the pictures, and after each instruction they were asked to describe the reasons for the set of choices they had made. Their food choices were noted on a precoded sheet, and reasons were tape-recorded and evaluated according to Jonsson and Flink (1992; also Gummeson et al., 1996). Before reading out the next instruction, all the previously selected pictures were cleared away. The interviews were individual and lasted about 20 min. All the tests were performed on two occasions separated by two months, and with the same procedure used. Four-day Food Record A couple of weeks after the second interview the subjects completed a four-day food record of their breakfasts. The food record was framed as a diary, with instructions to report for all food items consumed at breakfast time, the food category (e.g. milk) and the type of the food (e.g. full-fat milk), though amounts were not requested. Three individuals did not complete their diaries, and one did not eat breakfast. Statistical Method The different types of milk and margarine represent decreasing levels of fat content, and correspondingly the types of bread and breakfast cereals represent increasing levels of fibre content. The fat content in milk was classified in four ordered categories: A, full-fat milk (3%); B, medium-fat milk (1·5%); C, low-fat milk (0·5%); and D, minimilk (0·1%). The choices of butter and margarine were classified according to fat content in five ordered categories: A, butter; B, margarine (80%) less unsaturated; C, margarine (80%) more unsaturated; D, margarine (60%); and E, margarine (40%). The fibre content in bread was recorded as: A, white bread, lower fibre content (2%); B, white bread, higher fibre content (3%); C, wheat and rye bread (5%); D, whole meal bread (7%); and E, coarse crisp bread (14%). Corresponding fibre content classifications among breakfast cereals were: A, corn flakes (2·5%); B, muesli with fruits and nuts (8% and sweet); C, plain muesli (9% and unsweetened); and D, oat porridge (porridge oats 10%). The choices of milk, margarine, bread and breakfast cereals represent the main fat and fibre sources of a breakfast and were recorded as ordered categorical data. Therefore non-parametric statistical methods were used for statistical treatment of data. The intra-object percentage agreement (PA) between the two occasions was calculated. Observed intra-object disagreement was described further by a non-parametric method for paired ordered categorical data developed by Svensson (1993; also Svensson & Holm, 1994). By this method it is possible to decompose and measure disagreement between choices in terms of systematic and random differences, which is a prerequisite for reliability assessments. Evidence of systematic difference limits the possibility of reaching a total agreement, which is equivalent to the highest level of reliability. Therefore, it is important to separate systematic differences from occasional, random differences in food choices, and this is possible with the method of Svensson. Possible systematic differences between food choices are of main interest in this study, as there was a time gap of two months between the two occasions of food choices that might attenuate the reliability.
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
29
T 1 The joint distribution of fat content in milk choices made at two occasions X and Y. The frequencies and the cumulative proportions of the levels are given Level of fat content in choice X Level of fat content in choice Y
A
B
C
D
D
Cumulative total percentage 0
100
C
2
5
7
100
B
9
4
13
77
2
11
35
A
9
total 9
11
11
Cumulative 29 percentage
65
100 10 0
0
31
The non-parametric statistical method and the empirical measures of the systematic and the random parts of disagreement are presented in the following application example from this study. Consider an example of two sets of milk choices registered in four ordered categories, A–D, representing a decreasing level of fat content in milk. There are 31 pairs of milk choices, and the two sets of ordered categories are labelled X and Y. The joint distribution of the paired observations is shown in Table 1. The agreement diagonal is oriented from lower left to upper right and is delineated by double lines in the table. There is agreement in 23 of the 31 choices, so that the proportion of agreement is 74%. Eight individuals changed the type of milk between the two occasions. Six of them changed from low-fat milk (C) to medium and full-fat milk (B or A). To what extent can this disagreement between the two occasions be regarded as systematic? There is a slight difference between the frequency distributions of the two occasions, see Table 1. This difference in the marginal distribution indicates systematic disagreement between the two sets of choices. The cumulative percentages for the marginal distributions are also shown in Table 1. A graphical presentation of the systematic differences between paired ordered categorical data is to plot the two sets of cumulative percentages against each other yielding a so-called Relative Operating Characteristic (ROC) curve (Svensson, 1993; Altham, 1973). In this example the ROC curve is constructed by the points (0,0), (29,35), (65,77) and (100,100), see Figure 1. By use of the ROC curve it is easy to identify and locate the most important reasons for systematic changes in the choices between the two occasions (Svensson, 1993). In case of total agreement, that is marginal homogeneity, the ROC curve is the diagonal of identical coordinates. In this example one set of choices (Y) is systematically shifted towards categories representing higher fat content in milk, and consequently the ROC curve will be located to the upper side of the diagonal of agreement. The greater the deviation, the stronger is the systematic disagreement between the two sets of choices. This type of systematic difference is defined by the
30
I. JONSSON ET AL.
100 + Low-fat milk 80
Occasion Y (2)
(65,77)
60 + Medium-fat milk 40 (29,35) Full-fat milk
20
0
20
40 60 Occasion X (1)
80
100
F 1. ROC curve for fat content in the milk choices made at two occasions X and Y (made from data with the instruction “Make up a breakfast you usually eat”).
difference between the probabilities P(X
]
RP=p0−p1 where p0=
v=1
m
(X) p(Y) v pv−1 and p1=
]p
(X) v
(X) (Y) P(Y) v−1 , and pv and pv
v=1
are the vth category proportions of the two marginal distributions X and Y respectively. The corresponding cumulative proportions are denoted P(X) and P(Y) v v where v denotes the categories, in this case the number of categories, m=4, for example A,B,C, and D. In this example 1 1 (13×9+7×20)=0·267 and p1= 2(11×11+11×24)=0·401 312 31
p0=
which implies RP=−0·13. The uncertainty of the RP value as an indication of systematic difference in position is given by the standard error of RP, which is calculated by the jack-knife technique (Svensson, 1993; see also Svensson & Holm, 1994). In this example RP is −0·13 which is 1·6 times the SE(RP) value (−0·080). The conclusion is that there is a tendency of systematic difference between the two occasions, toward milk with higher fat content, as the RP value is negative. Another type of systematic deviance from the agreement diagonal of the ROC curve might occur (Svensson, 1993). If one set of classification is concentrated to a limited part of the scale of categories compared to the other set, the ROC curve will be S-shaped, and there will be a systematic difference in concentration of the
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
31
classifications. The measure of systematic difference in concentration is defined by the difference between the probability of the distribution of Y being concentrated compared to X and vice versa; P(X1
R Construct Validity To test the construct validity (inter-instrument agreement) of the stacking box methodology, data from the instruction “Make up a breakfast you usually eat” on the second interview occasion was compared with data from the four-day food record of their own breakfast. The percentage agreements (PA values) were for milk,
32
I. JONSSON ET AL.
T 2 Assessment of construct validity and reliability for stacking box investigation of choices of four breakfast foods Milk Comparisons between: Assessment of construct validity “Make up a breakfast you usually eat” (second interview) and four-day food record Assessment of reliability “Make up the breakfast you had this morning” (first and second interview) “Make up a breakfast you usually eat” (first and second interview) “Make up a breakfast you would like to eat” (first and second interview) “Make up a healthy breakfast” (first and second interview)
Margarine
Bread
N
PA
N
PA
N
29
83c
28
71
35
29
76
26
77a
32
31
74b
29
79
16
76
27
16
69a,b
22
Breakfast cereals
PA
N
PA
13
77
53c
12
92
33
58c
14
86
78b
42
57c
16
69
36c
32
41a,b,c
38
37c
43a,b,c
Note. The Ns vary according to the number of pupils who reported choice of that food. PA, percentage agreement. a,b,c indicate the main reasons for disagreement: a systematic difference in concentration, b systematic difference in position, c random difference.
T 3 Number of subjects who reported consumption of one, two or three different types of milk, margarine, bread and breakfast cereals in their four-day food record. N=40 Number of different types of food
Milk1 Margarine1 Bread2 Breakfast cereals1 1 2
1
2
3
29 38 14 17
4 1 18 8
1 0 8 0
Four different types of the food category available. Five different types of the food category available.
margarine, bread and breakfast cereals 83, 71, 43, and 77% respectively (Table 2). The within-subject variations of those foods which were reported within the four different food categories in the four-day food record are shown in Table 3. The
33
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
100
Usual breakfast cumulative percent
+ Whole meal bread 80 + Wheat and rye bread
(91,83)
60 (26,54) 40 + Sweet white bread
20
(11,23) White bread
0
20 40 60 80 Food record cumulative percent
100
F 2. ROC curve for fibre content in the bread choices in breakfast composition done with the instruction “Make up a breakfast you usually eat” in the second interview, and according to the four-day food record.
highest variation was seen for bread. About two-thirds of the subjects consumed two or more different types of bread for breakfast. When more closely analysed with the method of Svensson (1993), the disagreements between those types of food chosen for “usual” breakfast, and those that were reported in the four-day food record, showed evidence of both systematic (RC and RP values) and random (RV value) character (Table 2). The disagreements for milks and bread were mainly due to intra-individual random variations [RV= 0·18, SE(RV)=0·11 and RV=0·31, SE(RV)=0·14]. For bread there was also a systematic difference [RC=0·22, SE(RC)=0·14 and RP=0·19, SE(RP)=0·13] between the second food choice and the food record, mainly explained by the fact that the individuals according to the four-day food record ate more wheat and rye bread, and less white bread than they chose from the stacking boxes, as is illustrated by the ROC curve in Fig. 2. There also were a few more choices of wholemeal bread from boxes. Reliability To test the reliability of the stacking box method the choices of milk, margarine, bread and breakfast cereals, elicited with each of four different instructions on the two interview occasions, were compared. For most of the instructions there was a high consistency for milk, margarine and breakfast cereals, i.e. good agreements between food choices in the two interviews (Table 2). Some of the disagreements showed evidence of systematic character (Table 2). The result for milk choices with the instruction “Make up a breakfast you usually eat” is presented in Table 1 and Fig. 1. Six of eight changed choices were towards
34
I. JONSSON ET AL.
100 + Skimmed milk 80
(94,75)
Occasion 2
+ Low-fat milk 60 (69,50) 40
+ Medium-fat milk
(25,25)
20
Full-fat milk
0
20
40 60 Occasion 1
80
100
F 3. ROC curve for fat content in the milk choices made at the two occasions with the instruction “Make up a healthy breakfast”.
milk with a higher fat content [RP=−0·13, SE(RP)=0·08]. With the instruction to make up one’s own breakfast eaten in the morning the interview was conducted, five of the six individuals who had different choices of margarine had chosen margarine with a lower fat content in the second interview (RC=0·14, SE(RC)= 0·08). In the “liked” breakfast the disagreement for margarine could mainly be explained by a shift towards margarine with a higher fat content [five of six changed choices; RP=−0·096, SE(RP)=0·038]. All five changes of milk choices with the instruction to make up a healthy breakfast were towards milk with a lower fat content [Fig. 3; RC=−0·28, SE(RC)= 0·13 and RP=0·19, SE(RP)=0·09]. Low percentage agreements for margarine, bread and breakfast cereals with the same instruction had evidence of random intraindividual differences [RV=0·22, SE(RV)=0·14; RV=0·30, SE(RV)=0·13 and RV= 0·30, SE(RV)=0·12 respectively]. For bread there also was a systematic difference between the two interview occasions [RC=−0·28, SE(RC)=0·17 and RP=0·31, SE(RP)=0·12]. Fourteen of the 19 boys who shifted their choices chose at the second interview occasion bread with a higher fibre content. For bread the agreement between the interviews was throughout lower than for the other food categories, and had obvious evidences of random character (RV=0·17–0·30), (Table 2).
D The method of Svensson (1993) was used to evaluate disagreements between the paired ordered categorical data, rather than the value of kappa, for several reasons. Kappa was originally developed for nominal data (Cohen, 1960). The kappa value
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
35
depends on the marginal distributions and in fact requires identical marginal distributions (Feinstein & Cicchetti, 1990), which was often not the case in this study. The value of kappa is also greatly influenced by the number of categories and increases when the number of categories decreases. This implies that kappa values obtained from categorical scales with different number of categories are not comparable (Maclure & Willet, 1987), and such scales occur in this study. The advantages with food photographs to describe food choices are numerous. It is our experience that pictures are likely to be more appropriate for children than complex food frequency questionnaires. The stacking box made it possible for the pupils to choose the foods they usually consumed, just by recognizing the packages. As such the method has face validity. This probably levels out effects of differences in memory and ability to express oneself verbally. Alternative types of food within a food category could easily be identified also by the youngest students. Pupils sometimes rejected all the types of food in the stacking boxes, and said they would rather eat something else, so they did not seem to feel compelled to choose among the stacking box alternatives. Of course in such circumstances many of the advantages with photographs of foods are obviated. The food they said they would rather eat was noted at the end of the precoded sheet or as a food item with a corresponding nutrition value, e.g. in regard to fat content and quality. With the pictures on the table, the pupils could survey their meals, and that seemed to make it easier for them to put together meals in constellations they considered suitable. Overall, students found the method simple and pleasant to use, and older as well as younger students appeared to appreciate it. The construct validity (inter-instrument agreement) was high for choices of milk and margarine, but lower for bread (Table 2). A small sample for breakfast cereals prevents a conclusion about the reason for the slight difference in percentage agreement. Subjects were asked to choose foods for one usual (their most usual) breakfast from the stacking boxes. To achieve better validity for bread, it would be necessary to ask subjects to compose several of their most usual breakfasts, as their four-day food records showed a high variation of the types of bread consumed (Table 3). Another explanation of low construct validity for bread may be confusion of brands with rather similar packages, resulting in wrong fibre classification. The within-subject variation in the food record could also be the explanation for a lower reliability for bread choices than for choices within the other three food categories. For milk, the reliability was high for all the instructions. However, there was a slight systematic difference in milk choices between the occasions with the instruction to make a usual breakfast (Fig. 1). The reason might be that in the first interview some students were not aware of which type of milk they most often had for their breakfasts, but learnt about it between the two interview occasions. Also for milk some students used more than one type (Table 3). In another study of Swedish schoolchildren, 17% used two different kinds of milk at breakfast according to a seven-day food record (Berg, G., personal communication). This variation in use of milk products could make it difficult for some boys to decide in the first interview which choice to report with the instruction “Make up a breakfast you usually eat”, and if they knew low-fat milk is healthier, they may have chosen to mention this kind of milk to us. Another possibility is that a debate in the media at the time of the investigation about recommendations of fat for children, or brochures from the dairy industry including words such as “young people need fat”, caused a true change over time in beliefs and habits among schoolchildren concerning milk.
36
I. JONSSON ET AL.
The result for milk with the instruction to make up a healthy breakfast showed that some of the students in the second interview had become more knowledgable about fat content of different types of milk, and that they knew it is desirable to cut down on fat intake, as they then chose more milk with a low fat content. For margarine the reliability was high for three of the four instructions (Table 2). Some students, however, seemed to be unaware in the first interview of which margarine they had for breakfast, to judge from changed choices in favour of more low-fat margarine in the second interview with the instruction “Make up the breakfast you had this morning”. Less knowledge of margarine brands in the first interview may also be the explanation of the slight systematically changed choices with the instruction “Make up a breakfast you would like to eat”. The evidence of random influences in the differences between the two occasions of measurement for margarine, bread and breakfast cereals with the instruction to make a healthy breakfast, may mean that not all of the students were as conscious about which the healthy choices are within these three food categories, as they were for milk. Williams et al. (1993) found a high level of uncertainty among young people about the healthiness of margarine. Eighty percent of Danish schoolchildren knew that butter has a high fat content, but fewer (55%) knew that margarine also contains a high percentage of fat (Osler & Tornberg Hansen, 1990). In summary, the used approach in assessing school boys’ food choices showed satisfactory reliability and construct validity (inter-instrument agreement) for milk and margarine. Disagreements could be explained as random in nature, or due to increased knowledge about the different types of foods involved. The approach used in the present study could be developed to be what Wardle (1993) calls for, a methodology producing reproducible quantitative measures, which also can reflect individual’s viewpoints on food choices (Gummeson et al., 1996). It is important to confirm hypotheses with several independent methods and measures. The stacking box methodology should be a valuable complement to existing methods in assessing school children’s food choices and determinants of such choices.
R Ahlstro¨m, R., Baird, J. C. & Jonsson, I. (1990). School children’s preferences for food combinations. Food Quality and Preference, 2, 155–165. Altham, P. M. (1973). A non-parametric measure of signal discriminability. British Journal of Mathematical and Statistical Psychology, 26, 1–12. Calfas, K. J., Sallis F. J. & Nader P. R. (1991). The development of scales to measure knowledge and preference for diet and physical activity behaviour in 4- to 8-year-old children. Journal of Developmental Behavioural Pediatrics, 12, 185–190. Cohen J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. de Graaf, C., Stafleu, A., Staal, P. & Wijne M. (1992). Beliefs about the satiating effect of bread with spread varying in macronutrient content. Appetite, 18, 121–128. Feinstein A. R. & Cicchetti, D. V. (1990). High agreement but low kappa: the problems of two paradoxes. Journal of Clinical Epidemiology, 43, 543–549. Gummeson L., Jonsson, I., Conner M. & Svensson, E. (1996). Assessing factors influencing food choice among 10–16-year-old school boys. A pilot study with a stacking box method. Journal of Human Nutrition and Dietetics, 9, 219–229. Gutezeit, G., Bloth, S. & Hagenow, J. (1995). Auswahl und Einscha¨tzung von Mahlzeiten durch Kinder verschiedener Altersstufen. Zeitschrift fu¨r Erna¨hrungswissenschaft, 34, 261–268.
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
37
Jonsson, I. & Flink, B. (1992). Barns och ungdomars motiveringar till val av ma˚ltidskomponenter. En pilotstudie. Scandinavian Journal of Nutrition, 36, 15–19. Summary in English. Maclure, M. & Willet, W. C. (1987). Misinterpretation and misuse of the kappa statistic. American Journal of Epidemiology, 126, 161–169. Mertz, W. & Kelsay, J. L. (1984). Rationale and design of the Beltsville one-year dietary intake study. American Journal of Clinical Nutrition, 40 (Suppl.), 1323–1326. Osler, M. & Tornberg Hansen, E. (1990). Frederiksbergske skoleelevers kostvaner og -viden. Ugeskrift for Læger, 152, 3527–3531. Summary in English. Svensson, E. (1993). Analysis of systematic and random differences between paired ordinal categorical data. Thesis, Department of Statistics, University of Go ¨ ¨ teborg, Go¨teborg. Svensson, E. & Holm S. (1994). Separation of systematic and random differences in ordinal rating scales. Statistics in Medicine, 13, 2437–2453. Wardle, J. (1993). Food choices and health evaluation. Psychology and Health, 8, 65–75. Willet, W. (1990). Nutritional epidemiology. P. 23. Oxford University Press, Oxford. Williams, H., Woodward, D., Ball, P., Cumming, F., Hornsby, H. & Boon, J. (1993). Food perceptions and food consumption among Tasmanian high school students. Australian Journal of Nutrition and Dietetics, 50, 154–161.
Received 7 July 1995, revision 7 May 1997
Assessing Food Choice in School Children: Reliability and Construct Validity of a Method Stacking Food Photographs
INGER JONSSON and LENA GUMMESON Department of Home Economics, University of Go¨teborg, Go¨teborg, Sweden
MARK CONNER Department of Psychology, University of Leeds, Leeds, U.K.
ELISABETH SVENSSON Department of Mathematics, Chalmers University of Technology, Go¨teborg, Sweden
A method utilizing pictures was used to assess food choice with 44 school boys in three different age groups (10, 13 and 16 years old). The aim of the study was to test reliability and construct validity (i.e. inter-instrument agreement) of food choice assessed when the boys were asked to compose breakfasts. The focus was on choices of milk, margarine, bread and breakfast cereals, as in these food groups it is possible to choose between low-fat/high-fat and low-fibre/high-fibre products. Subjects were asked to repeat their composing of breakfasts after eight weeks, and to complete a four-day food record on breakfasts after the second interview occasion. For comparisons between food choices a new non-parametric statistical method was used. Both reliability and construct validity were good for choices of milk and margarine. The unreliability could be explained as random in nature or in terms of healthier choices in the second interview. 1998 Academic Press Limited
I Mertz and Kelsay (1984) have pointed out that foods are not fully represented by their nutrient composition. It might, in certain contexts, be more appropriate to study choices of foods and types of products within food groups. For example, it is generally agreed that non-communicable diseases in many populations would decrease with a lower fat and a higher fibre intake. If a certain food group constitutes the predominant source of a nutrient in a population, as dairy products do for fat intake in several Western countries, it is interesting to examine whether, for example, choices Supported in part by a grant from the Swedish National Institute of Health. The authors would like to express their appreciation to Britt-Marie Sandstro¨m for helpful comments on an earlier version of this manuscript. Address correspondence to: Inger Jonsson, Department of Home Economics, University of Go¨teborg, Box 12 204, 402 42 Go¨teborg, Sweden. 0195–6663/98/010025+13 $25.00/0/ap970113
1998 Academic Press Limited
26
I. JONSSON ET AL.
of milk and cheese with a high fat content decrease as a result of health campaigns. In some epidemiology research, intake of foods and food groups rather than nutrients has been related to a lower risk of disease (Willet, 1990). Studies with groups representing whole populations and studies concerning certain subgroups have often different purposes, and need different approaches. For population screening, questionnaire techniques are appropriate. For studies with smaller groups, carried out with the purpose of getting a more detailed picture of food habits, and learning about reasons for food choices, interviews are often more suitable. Wardle (1993) argues that “sociological and anthropological approaches are well-established as methods of evaluating cultural beliefs and diet and health, but are based on unsystematic interviews and observations, and often use very small samples”, and points out that “there is a pressing need for development of assessment methods which reflect the point of view of the individual, while at the same time using reproducible measures and deriving quantitative data”. Reliability, validity and usability are the key characteristics for any measure of dietary habits. In developing measures of such habits for use with children the limited linguistic capacities of the respondents is likely to be an important factor influencing each of these characteristics. Measures that require sophisticated linguistic abilities are unlikely to produce reliable or valid results with younger children. One way to reduce the impact of linguistic skills is to make the task more visual in nature. If the child is required to select a food that it normally consumes from a selection of pictures the task is a simpler identification task rather than a task in verbally describing choices. Obviously the child still has to understand the verbal instructions and translate these instructions into a choice. However, the skills required are comprehension skills rather than skills of expression. In such a case, language is much less likely to be a barrier to adequate measurement. An approach with the above-mentioned characteristics was used to study school children’s food choice and their perceptions of the appropriateness of different combinations of food items in school lunches (Ahlstro¨m, Baird & Jonsson, 1990). From a specially designed apparatus, a stacking box, participants were asked to select cards with pictures of food items (one food item on each card) for meals according to given instructions, for example “Make up a breakfast you usually eat” and “Make up a healthy breakfast”. Each box consists of a topless solid aluminium box (25 cm×25 cm×6 cm deep) with 25 compartments ordered as a matrix five times five. Each compartment can hold up to ten pictures (all of a single food) stacked on top of each other. When one picture is removed a spring device in the bottom of the compartment allows the card below to pop up into view. This gives a uniform investigation situation, since the subject is not influenced by previous choices, as he/ she cannot see how many pictures of each food item they have already used. The approach makes written language a minor factor, which is important for exploring differences in food choice and preference for foods with school children at different grade levels. Through the construction of instructions to participants this use of photographs can be adapted to assess a variety of questions. The aim of the present study was to test reliability and construct validity (i.e. inter-instrument agreement) of a food choice measure based on utilization of food photographs with school children in three different age groups. Another aim was to apply a non-parametric statistical method in the evaluation of food choices. Photographs have, besides their common use as visual aids to decide quantities in dietary assessments, been used in studies concerning the satiating effects of foods (de Graff
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
27
et al., 1992) and children’s preferences for and knowledge of healthy/unhealthy foods (Calfas, Sallis & Nader, 1991) and meals (Gutezeit, Bloth & Hagenow, 1995). Williams et al. (1993) used visual aids (based on packaging) to distinguish milks of different fat content, and margarines of different levels of fat saturation. As far as we are aware no other study has used non-parametric methods to assess the disagreement between repeated food choices concerning the fibre and fat content recorded as ordered categorical data.
M Subjects The participants were 44 school boys from three grades (fourth, seventh, and tenth) who were 10, 13 and 16 years old. There were approximately equal numbers of boys from each grade (fourth and seventh grades, 15 boys each; tenth grade, 14 boys). The study was carried out in a suburb of Go¨teborg, Sweden, which has a socio-economically diverse population, and a rather low share of immigrants. Foodstuffs The breakfast meal was chosen to illustrate food choices, as a typical Swedish breakfast meal contains food items with several alternatives concerning fat and fibre content. Such foods are principally milk products and margarine, bread and breakfast cereals. Colour photographs of 50 different food items were available in two stacking boxes. Among these there were four types of milk with different fat contents, and five types of margarine, which varied in fat content and quality (butter is included here and will be through the paper). Five types of bread and four types of breakfast cereals, both product categories with a variation in fibre content, were also available. Foods within the four food categories were also chosen to represent the most frequently sold foods on the Swedish market. The rest of the 50 photographs covered foods typical of Swedish breakfast habits. These foods do not contribute at all or only to a limited extent to the fat and fibre intake compared to milk and margarine or bread and breakfast cereals. Moreover, most of them have no alternatives differing in fat and fibre content. For reasons of health it is important to diminish fat intake and increase fibre intake. This is most simply brought about in a population if people choose low-fat and high-fibre alternatives within those food categories where there are such alternatives. Colour photographs (5 cm×5 cm) were made of each item shown lying on a plate. Each photograph was mounted on a rigid card and covered by transparent plastic. To make it easier for subjects to find a certain type of food within a food group, pictures of milks or margarines or breads etc. were presented spatially close to one another in the stacking boxes. Procedure Subjects were asked to select food items from the stacking boxes according to different instructions: “Make up the breakfast you had this morning”; “Make up a breakfast you usually eat”; “Make up a breakfast you would like to eat” and “Make
28
I. JONSSON ET AL.
up a healthy breakfast”. The subjects chose freely among the pictures, and after each instruction they were asked to describe the reasons for the set of choices they had made. Their food choices were noted on a precoded sheet, and reasons were tape-recorded and evaluated according to Jonsson and Flink (1992; also Gummeson et al., 1996). Before reading out the next instruction, all the previously selected pictures were cleared away. The interviews were individual and lasted about 20 min. All the tests were performed on two occasions separated by two months, and with the same procedure used. Four-day Food Record A couple of weeks after the second interview the subjects completed a four-day food record of their breakfasts. The food record was framed as a diary, with instructions to report for all food items consumed at breakfast time, the food category (e.g. milk) and the type of the food (e.g. full-fat milk), though amounts were not requested. Three individuals did not complete their diaries, and one did not eat breakfast. Statistical Method The different types of milk and margarine represent decreasing levels of fat content, and correspondingly the types of bread and breakfast cereals represent increasing levels of fibre content. The fat content in milk was classified in four ordered categories: A, full-fat milk (3%); B, medium-fat milk (1·5%); C, low-fat milk (0·5%); and D, minimilk (0·1%). The choices of butter and margarine were classified according to fat content in five ordered categories: A, butter; B, margarine (80%) less unsaturated; C, margarine (80%) more unsaturated; D, margarine (60%); and E, margarine (40%). The fibre content in bread was recorded as: A, white bread, lower fibre content (2%); B, white bread, higher fibre content (3%); C, wheat and rye bread (5%); D, whole meal bread (7%); and E, coarse crisp bread (14%). Corresponding fibre content classifications among breakfast cereals were: A, corn flakes (2·5%); B, muesli with fruits and nuts (8% and sweet); C, plain muesli (9% and unsweetened); and D, oat porridge (porridge oats 10%). The choices of milk, margarine, bread and breakfast cereals represent the main fat and fibre sources of a breakfast and were recorded as ordered categorical data. Therefore non-parametric statistical methods were used for statistical treatment of data. The intra-object percentage agreement (PA) between the two occasions was calculated. Observed intra-object disagreement was described further by a non-parametric method for paired ordered categorical data developed by Svensson (1993; also Svensson & Holm, 1994). By this method it is possible to decompose and measure disagreement between choices in terms of systematic and random differences, which is a prerequisite for reliability assessments. Evidence of systematic difference limits the possibility of reaching a total agreement, which is equivalent to the highest level of reliability. Therefore, it is important to separate systematic differences from occasional, random differences in food choices, and this is possible with the method of Svensson. Possible systematic differences between food choices are of main interest in this study, as there was a time gap of two months between the two occasions of food choices that might attenuate the reliability.
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
29
T 1 The joint distribution of fat content in milk choices made at two occasions X and Y. The frequencies and the cumulative proportions of the levels are given Level of fat content in choice X Level of fat content in choice Y
A
B
C
D
D
Cumulative total percentage 0
100
C
2
5
7
100
B
9
4
13
77
2
11
35
A
9
total 9
11
11
Cumulative 29 percentage
65
100 10 0
0
31
The non-parametric statistical method and the empirical measures of the systematic and the random parts of disagreement are presented in the following application example from this study. Consider an example of two sets of milk choices registered in four ordered categories, A–D, representing a decreasing level of fat content in milk. There are 31 pairs of milk choices, and the two sets of ordered categories are labelled X and Y. The joint distribution of the paired observations is shown in Table 1. The agreement diagonal is oriented from lower left to upper right and is delineated by double lines in the table. There is agreement in 23 of the 31 choices, so that the proportion of agreement is 74%. Eight individuals changed the type of milk between the two occasions. Six of them changed from low-fat milk (C) to medium and full-fat milk (B or A). To what extent can this disagreement between the two occasions be regarded as systematic? There is a slight difference between the frequency distributions of the two occasions, see Table 1. This difference in the marginal distribution indicates systematic disagreement between the two sets of choices. The cumulative percentages for the marginal distributions are also shown in Table 1. A graphical presentation of the systematic differences between paired ordered categorical data is to plot the two sets of cumulative percentages against each other yielding a so-called Relative Operating Characteristic (ROC) curve (Svensson, 1993; Altham, 1973). In this example the ROC curve is constructed by the points (0,0), (29,35), (65,77) and (100,100), see Figure 1. By use of the ROC curve it is easy to identify and locate the most important reasons for systematic changes in the choices between the two occasions (Svensson, 1993). In case of total agreement, that is marginal homogeneity, the ROC curve is the diagonal of identical coordinates. In this example one set of choices (Y) is systematically shifted towards categories representing higher fat content in milk, and consequently the ROC curve will be located to the upper side of the diagonal of agreement. The greater the deviation, the stronger is the systematic disagreement between the two sets of choices. This type of systematic difference is defined by the
30
I. JONSSON ET AL.
100 + Low-fat milk 80
Occasion Y (2)
(65,77)
60 + Medium-fat milk 40 (29,35) Full-fat milk
20
0
20
40 60 Occasion X (1)
80
100
F 1. ROC curve for fat content in the milk choices made at two occasions X and Y (made from data with the instruction “Make up a breakfast you usually eat”).
difference between the probabilities P(X
]
RP=p0−p1 where p0=
v=1
m
(X) p(Y) v pv−1 and p1=
]p
(X) v
(X) (Y) P(Y) v−1 , and pv and pv
v=1
are the vth category proportions of the two marginal distributions X and Y respectively. The corresponding cumulative proportions are denoted P(X) and P(Y) v v where v denotes the categories, in this case the number of categories, m=4, for example A,B,C, and D. In this example 1 1 (13×9+7×20)=0·267 and p1= 2(11×11+11×24)=0·401 312 31
p0=
which implies RP=−0·13. The uncertainty of the RP value as an indication of systematic difference in position is given by the standard error of RP, which is calculated by the jack-knife technique (Svensson, 1993; see also Svensson & Holm, 1994). In this example RP is −0·13 which is 1·6 times the SE(RP) value (−0·080). The conclusion is that there is a tendency of systematic difference between the two occasions, toward milk with higher fat content, as the RP value is negative. Another type of systematic deviance from the agreement diagonal of the ROC curve might occur (Svensson, 1993). If one set of classification is concentrated to a limited part of the scale of categories compared to the other set, the ROC curve will be S-shaped, and there will be a systematic difference in concentration of the
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
31
classifications. The measure of systematic difference in concentration is defined by the difference between the probability of the distribution of Y being concentrated compared to X and vice versa; P(X1
R Construct Validity To test the construct validity (inter-instrument agreement) of the stacking box methodology, data from the instruction “Make up a breakfast you usually eat” on the second interview occasion was compared with data from the four-day food record of their own breakfast. The percentage agreements (PA values) were for milk,
32
I. JONSSON ET AL.
T 2 Assessment of construct validity and reliability for stacking box investigation of choices of four breakfast foods Milk Comparisons between: Assessment of construct validity “Make up a breakfast you usually eat” (second interview) and four-day food record Assessment of reliability “Make up the breakfast you had this morning” (first and second interview) “Make up a breakfast you usually eat” (first and second interview) “Make up a breakfast you would like to eat” (first and second interview) “Make up a healthy breakfast” (first and second interview)
Margarine
Bread
N
PA
N
PA
N
29
83c
28
71
35
29
76
26
77a
32
31
74b
29
79
16
76
27
16
69a,b
22
Breakfast cereals
PA
N
PA
13
77
53c
12
92
33
58c
14
86
78b
42
57c
16
69
36c
32
41a,b,c
38
37c
43a,b,c
Note. The Ns vary according to the number of pupils who reported choice of that food. PA, percentage agreement. a,b,c indicate the main reasons for disagreement: a systematic difference in concentration, b systematic difference in position, c random difference.
T 3 Number of subjects who reported consumption of one, two or three different types of milk, margarine, bread and breakfast cereals in their four-day food record. N=40 Number of different types of food
Milk1 Margarine1 Bread2 Breakfast cereals1 1 2
1
2
3
29 38 14 17
4 1 18 8
1 0 8 0
Four different types of the food category available. Five different types of the food category available.
margarine, bread and breakfast cereals 83, 71, 43, and 77% respectively (Table 2). The within-subject variations of those foods which were reported within the four different food categories in the four-day food record are shown in Table 3. The
33
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
100
Usual breakfast cumulative percent
+ Whole meal bread 80 + Wheat and rye bread
(91,83)
60 (26,54) 40 + Sweet white bread
20
(11,23) White bread
0
20 40 60 80 Food record cumulative percent
100
F 2. ROC curve for fibre content in the bread choices in breakfast composition done with the instruction “Make up a breakfast you usually eat” in the second interview, and according to the four-day food record.
highest variation was seen for bread. About two-thirds of the subjects consumed two or more different types of bread for breakfast. When more closely analysed with the method of Svensson (1993), the disagreements between those types of food chosen for “usual” breakfast, and those that were reported in the four-day food record, showed evidence of both systematic (RC and RP values) and random (RV value) character (Table 2). The disagreements for milks and bread were mainly due to intra-individual random variations [RV= 0·18, SE(RV)=0·11 and RV=0·31, SE(RV)=0·14]. For bread there was also a systematic difference [RC=0·22, SE(RC)=0·14 and RP=0·19, SE(RP)=0·13] between the second food choice and the food record, mainly explained by the fact that the individuals according to the four-day food record ate more wheat and rye bread, and less white bread than they chose from the stacking boxes, as is illustrated by the ROC curve in Fig. 2. There also were a few more choices of wholemeal bread from boxes. Reliability To test the reliability of the stacking box method the choices of milk, margarine, bread and breakfast cereals, elicited with each of four different instructions on the two interview occasions, were compared. For most of the instructions there was a high consistency for milk, margarine and breakfast cereals, i.e. good agreements between food choices in the two interviews (Table 2). Some of the disagreements showed evidence of systematic character (Table 2). The result for milk choices with the instruction “Make up a breakfast you usually eat” is presented in Table 1 and Fig. 1. Six of eight changed choices were towards
34
I. JONSSON ET AL.
100 + Skimmed milk 80
(94,75)
Occasion 2
+ Low-fat milk 60 (69,50) 40
+ Medium-fat milk
(25,25)
20
Full-fat milk
0
20
40 60 Occasion 1
80
100
F 3. ROC curve for fat content in the milk choices made at the two occasions with the instruction “Make up a healthy breakfast”.
milk with a higher fat content [RP=−0·13, SE(RP)=0·08]. With the instruction to make up one’s own breakfast eaten in the morning the interview was conducted, five of the six individuals who had different choices of margarine had chosen margarine with a lower fat content in the second interview (RC=0·14, SE(RC)= 0·08). In the “liked” breakfast the disagreement for margarine could mainly be explained by a shift towards margarine with a higher fat content [five of six changed choices; RP=−0·096, SE(RP)=0·038]. All five changes of milk choices with the instruction to make up a healthy breakfast were towards milk with a lower fat content [Fig. 3; RC=−0·28, SE(RC)= 0·13 and RP=0·19, SE(RP)=0·09]. Low percentage agreements for margarine, bread and breakfast cereals with the same instruction had evidence of random intraindividual differences [RV=0·22, SE(RV)=0·14; RV=0·30, SE(RV)=0·13 and RV= 0·30, SE(RV)=0·12 respectively]. For bread there also was a systematic difference between the two interview occasions [RC=−0·28, SE(RC)=0·17 and RP=0·31, SE(RP)=0·12]. Fourteen of the 19 boys who shifted their choices chose at the second interview occasion bread with a higher fibre content. For bread the agreement between the interviews was throughout lower than for the other food categories, and had obvious evidences of random character (RV=0·17–0·30), (Table 2).
D The method of Svensson (1993) was used to evaluate disagreements between the paired ordered categorical data, rather than the value of kappa, for several reasons. Kappa was originally developed for nominal data (Cohen, 1960). The kappa value
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
35
depends on the marginal distributions and in fact requires identical marginal distributions (Feinstein & Cicchetti, 1990), which was often not the case in this study. The value of kappa is also greatly influenced by the number of categories and increases when the number of categories decreases. This implies that kappa values obtained from categorical scales with different number of categories are not comparable (Maclure & Willet, 1987), and such scales occur in this study. The advantages with food photographs to describe food choices are numerous. It is our experience that pictures are likely to be more appropriate for children than complex food frequency questionnaires. The stacking box made it possible for the pupils to choose the foods they usually consumed, just by recognizing the packages. As such the method has face validity. This probably levels out effects of differences in memory and ability to express oneself verbally. Alternative types of food within a food category could easily be identified also by the youngest students. Pupils sometimes rejected all the types of food in the stacking boxes, and said they would rather eat something else, so they did not seem to feel compelled to choose among the stacking box alternatives. Of course in such circumstances many of the advantages with photographs of foods are obviated. The food they said they would rather eat was noted at the end of the precoded sheet or as a food item with a corresponding nutrition value, e.g. in regard to fat content and quality. With the pictures on the table, the pupils could survey their meals, and that seemed to make it easier for them to put together meals in constellations they considered suitable. Overall, students found the method simple and pleasant to use, and older as well as younger students appeared to appreciate it. The construct validity (inter-instrument agreement) was high for choices of milk and margarine, but lower for bread (Table 2). A small sample for breakfast cereals prevents a conclusion about the reason for the slight difference in percentage agreement. Subjects were asked to choose foods for one usual (their most usual) breakfast from the stacking boxes. To achieve better validity for bread, it would be necessary to ask subjects to compose several of their most usual breakfasts, as their four-day food records showed a high variation of the types of bread consumed (Table 3). Another explanation of low construct validity for bread may be confusion of brands with rather similar packages, resulting in wrong fibre classification. The within-subject variation in the food record could also be the explanation for a lower reliability for bread choices than for choices within the other three food categories. For milk, the reliability was high for all the instructions. However, there was a slight systematic difference in milk choices between the occasions with the instruction to make a usual breakfast (Fig. 1). The reason might be that in the first interview some students were not aware of which type of milk they most often had for their breakfasts, but learnt about it between the two interview occasions. Also for milk some students used more than one type (Table 3). In another study of Swedish schoolchildren, 17% used two different kinds of milk at breakfast according to a seven-day food record (Berg, G., personal communication). This variation in use of milk products could make it difficult for some boys to decide in the first interview which choice to report with the instruction “Make up a breakfast you usually eat”, and if they knew low-fat milk is healthier, they may have chosen to mention this kind of milk to us. Another possibility is that a debate in the media at the time of the investigation about recommendations of fat for children, or brochures from the dairy industry including words such as “young people need fat”, caused a true change over time in beliefs and habits among schoolchildren concerning milk.
36
I. JONSSON ET AL.
The result for milk with the instruction to make up a healthy breakfast showed that some of the students in the second interview had become more knowledgable about fat content of different types of milk, and that they knew it is desirable to cut down on fat intake, as they then chose more milk with a low fat content. For margarine the reliability was high for three of the four instructions (Table 2). Some students, however, seemed to be unaware in the first interview of which margarine they had for breakfast, to judge from changed choices in favour of more low-fat margarine in the second interview with the instruction “Make up the breakfast you had this morning”. Less knowledge of margarine brands in the first interview may also be the explanation of the slight systematically changed choices with the instruction “Make up a breakfast you would like to eat”. The evidence of random influences in the differences between the two occasions of measurement for margarine, bread and breakfast cereals with the instruction to make a healthy breakfast, may mean that not all of the students were as conscious about which the healthy choices are within these three food categories, as they were for milk. Williams et al. (1993) found a high level of uncertainty among young people about the healthiness of margarine. Eighty percent of Danish schoolchildren knew that butter has a high fat content, but fewer (55%) knew that margarine also contains a high percentage of fat (Osler & Tornberg Hansen, 1990). In summary, the used approach in assessing school boys’ food choices showed satisfactory reliability and construct validity (inter-instrument agreement) for milk and margarine. Disagreements could be explained as random in nature, or due to increased knowledge about the different types of foods involved. The approach used in the present study could be developed to be what Wardle (1993) calls for, a methodology producing reproducible quantitative measures, which also can reflect individual’s viewpoints on food choices (Gummeson et al., 1996). It is important to confirm hypotheses with several independent methods and measures. The stacking box methodology should be a valuable complement to existing methods in assessing school children’s food choices and determinants of such choices.
R Ahlstro¨m, R., Baird, J. C. & Jonsson, I. (1990). School children’s preferences for food combinations. Food Quality and Preference, 2, 155–165. Altham, P. M. (1973). A non-parametric measure of signal discriminability. British Journal of Mathematical and Statistical Psychology, 26, 1–12. Calfas, K. J., Sallis F. J. & Nader P. R. (1991). The development of scales to measure knowledge and preference for diet and physical activity behaviour in 4- to 8-year-old children. Journal of Developmental Behavioural Pediatrics, 12, 185–190. Cohen J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. de Graaf, C., Stafleu, A., Staal, P. & Wijne M. (1992). Beliefs about the satiating effect of bread with spread varying in macronutrient content. Appetite, 18, 121–128. Feinstein A. R. & Cicchetti, D. V. (1990). High agreement but low kappa: the problems of two paradoxes. Journal of Clinical Epidemiology, 43, 543–549. Gummeson L., Jonsson, I., Conner M. & Svensson, E. (1996). Assessing factors influencing food choice among 10–16-year-old school boys. A pilot study with a stacking box method. Journal of Human Nutrition and Dietetics, 9, 219–229. Gutezeit, G., Bloth, S. & Hagenow, J. (1995). Auswahl und Einscha¨tzung von Mahlzeiten durch Kinder verschiedener Altersstufen. Zeitschrift fu¨r Erna¨hrungswissenschaft, 34, 261–268.
ASSESSING FOOD CHOICE IN SCHOOL CHILDREN
37
Jonsson, I. & Flink, B. (1992). Barns och ungdomars motiveringar till val av ma˚ltidskomponenter. En pilotstudie. Scandinavian Journal of Nutrition, 36, 15–19. Summary in English. Maclure, M. & Willet, W. C. (1987). Misinterpretation and misuse of the kappa statistic. American Journal of Epidemiology, 126, 161–169. Mertz, W. & Kelsay, J. L. (1984). Rationale and design of the Beltsville one-year dietary intake study. American Journal of Clinical Nutrition, 40 (Suppl.), 1323–1326. Osler, M. & Tornberg Hansen, E. (1990). Frederiksbergske skoleelevers kostvaner og -viden. Ugeskrift for Læger, 152, 3527–3531. Summary in English. Svensson, E. (1993). Analysis of systematic and random differences between paired ordinal categorical data. Thesis, Department of Statistics, University of Go ¨ ¨ teborg, Go¨teborg. Svensson, E. & Holm S. (1994). Separation of systematic and random differences in ordinal rating scales. Statistics in Medicine, 13, 2437–2453. Wardle, J. (1993). Food choices and health evaluation. Psychology and Health, 8, 65–75. Willet, W. (1990). Nutritional epidemiology. P. 23. Oxford University Press, Oxford. Williams, H., Woodward, D., Ball, P., Cumming, F., Hornsby, H. & Boon, J. (1993). Food perceptions and food consumption among Tasmanian high school students. Australian Journal of Nutrition and Dietetics, 50, 154–161.
Received 7 July 1995, revision 7 May 1997