- Email: [email protected]
Accuracy of estimating food intake by observation
RESEARCH
Accupay of estimating food intake by observation JOEL GITTELSOHN, PhD, ANITA Vr SHANAR, MS, RAMP. POKHREL,FRCS, KEITH P WEST, Jr,DrPH,RD
AIBSTRACT Objective To determine the accuracy of direct observation in food-weight estimation as measured under controlled field conditions. Design and subjects Ten local Nepahis were trained in observational techniques and tested in food-weight estimation during a 3-month training period and for 4 months after training. Setting The study was carried out in the Sarlahi District, a rural, central lowland region of Nepal that borders India. Main outcome measures Thirty testing sessions (a total of 6,902 observations) were completed on more than 150 different foods. Estimates of observed food weight were compared with actual weights and were analyzed. Statistical analyses performed Pearson's correlation coefficients were calculated to examine associations between estimated and actual weights. Results Observer estimates of food weights were highly correlated with actual weights (r=.96) for the entire testing period. The linear regression equation (y=.96+1.3) suggests that the relationship between actual and observed food weights in grams) was also accurate. Most observers showed improvement with training. Substantial reductions in both mean and standard deviation of percentage error were achieved over time. Accuracy of estimates was influenced by characteristics of foods weighed; small quantities (less than 20 g), certain nonstaple foods, and foods of high volume but light weight had less accurate estimates. Conclusions Direct observation is an important method for assessing dietary intake that does not rely on a respondents' ability to recall Wis or her own or another's food consumption. It is feasible to train local observers to make visual estimates of food weight, but the accuracy of their estimates varies by food and portion size. JAm DietAssoc. 1994; 94:1273-1277
dietary assessment methodologies, including (a) retrospective methods, such as 24-hour recalls, food frequencies, and diet histories, based on narrative accounts of foods consumed in the past; and (b) prospective methods, such as food diaries, food weighings, and direct observation, where food intake is recorded at the time of consumption. This article examines the accuracy of direct observation, an underused but potentially important method of dietary assessment. In direct observation, trained personnel observe food consumption behavior and visually estimate food intake. Direct observation is a useful method for the study of dietary intakes of young children, because obtaining accurate assessments of their dietary intake is often problematic. Although children may be old enough to procure foods independently, they may be too young to recall their food consumption accurately (1,2). In such circumstances, dietary intake data of children are often gathered through the recall of a parent. Reliance on parental or caretaker recall of a child's food intake may provide reasonably accurate data for mealtime consumption, but may exclude foods eaten outside the view of adults (3,4). For instance, in her dietary study ofyoung Malay children, Wilson (5) found that a substantial portion of energy and vitan-dn C were consumed at times other than household meal hours. Other studies support this conclusion by reporting notable differences in quantity and variety of food consumed when observational methods were used and compared with dietary surveys of the same population (4,6). Prospective methods have other advantages over retrospective methods in that they are not subject to memory loss, inaccurate reporting, or error associated with question formlation (7-12). However, some prospective methods, such as self-administered food diaries or food weighing, are considered to be intrusive and may alter dietary behaviors. J. Gittelsohn is an assistant rofessor, A. Vr Shankaris a researchassociate, and P. West, Jr, is an associate professorin the Division of HumanNutrition, Department of InternationalHealth, School ofHygieneR and Public Health, The Johns Hopkins University, Baltimore, Md. R.P. Pokhrel is chairmann of the NationalSocietyfor the Prevention ofBlindness with the Nepal Ee Hospital, Kathmandu, Nepal. Address correspondenceto: Joel Gittelsohn, Phl) DivisionofHuman Nutrition, Departmentof Interntional Health, School fHygiene and Public Health, The Johns Hopkins Universiy 615 N Woffie St, Baltimore, MD 21205. JOURNAL OF THE AMERICAN DIETETIC ASSOCIATION 1273
....................................................................
..........................................................
RWSIICH
Table 1 Correlation coefficients for actual and estimated food weights for specific food groups Food Group
Sample size (n)
Correlation (r)
Fruits
265
.96
Vegetables
944
.93
Grains
2,773
.96
Liquids
1,436
.96
Mixed foods
595
.95
Manufactured foods
398
.98
Meat/fish
398
.97
6,809a
.96
All categorized foods
aTotal number of observer estimates analyzed was 6,902; 93 of these food estimates were not categorized by food group.
FIG 1. Accuracy offood-weight estimates.
1274/ NOVEMBER 1994 VOLUME 94 NUMBER 11
Direct observation is more advantageous than other prospective techniques because the method does not depend on the literacy level of the respondent, and it is less likely to alter natural patterns of mealtime behavior (13). In rural Nepal, children frequently share food, eat in groups from the same plate, and interact with the food server in a number of different ways that would be seriously disrupted by the need to weigh each food serving. In this setting, direct observations of food intake may provide the most accurate estimates of usual food intake for children, with the least disruption of natural behavior (14). Moreover, direct observation methods can capture details of feeding behavior that would otherwise not be obtained. In a study in Bangladesh, Guldan and colleagues (15) used observational techniques to examine child feeding practices; they found significant associations between factors such as frequency of feeding, freshness of foods, cleanliness of feeding areas, and feeding style. Unfortunately, recent reviews in nutritional epidemiology fail to mention observational techniques as a feasible alternative for dietary assessment (16,17). On the other hand, researchers who use observational techniques consider them the standard against which other dietary assessment techniques can be compared (1,18). The paucity of data on observational techniques, therefore, has resulted in only a few studies that examine the training of observers and standardization of procedures (10,19). We were able to locate only one study that attempted to characterize how food-weight estimation varies by visual attributes of food and food type (20). Such information is critical in assessing the validity of any dietary assessment technique and is of particular importance for studies conducted in less controllable, rural field conditions by lay observers. This article examines issues of accuracy and explores the feasibility of training local field workers in direct observation procedures. METHODS This study was conducted in the Sarlahi District, a rural, central, lowland region of Nepal that borders India. As part of a community-based study of dietary intake in children, 10 local Nepalis were trained for 3 months (September through November 1992) in observational techniques and estimation of food weight. These 3 months constituted the formal training period. Each week, validation sessions were conducted in which observer estimates of food quantities were compared with actual food weights for various foods. For 4 months after the training (December 1992 through March 1993), hereafter referred to as the posttraining period, we continued to conduct validation sessions to familiarize observers with new seasonally available foods and to review commonly eaten foods. During the testing period, which included both training and posttraining, 30 validation sessions were conducted and analyzed (14 during training and 16 during the posttraining period). All observers were men and were from the region. Each observer had a minimum of a high school diploma, and none had prior training in nutrition or research. Activities to improve observers' food-estimation skills included the following group and individual practice sessions: * Group practice estimation sessions were conducted regularly throughout the testing period to familiarize observers with estimating food-specific weights. Foods were presented in standard local plates or glasses, with their true weights disclosed to the group. Observers were permitted to examine and handle foods closely to develop an intuitive sense of the relationships of food weights, volume, and size. Later, observers were asked to estimate the weight of portions of the same foods of unknown weight based on this initial experience. Observers called out their estimates to the group; adjustment of subsequent answers based on previous responses was allowed. This process helped to standardize responses among observers.
* Individual practice estimation sessions were conducted every other day during the training period and weekly during the posttraining period. Observers independently estimated preweighed portions of different sizes for a small number of foods. Actual food weight was provided after each estimate to allow each observer to evaluate his food-specific estimation skills. * Validation sessions were conducted approximately once a week for the 7-month testing period. Observers were asked to estimate 10 to 30 different preweighed food quantities. Results were compared, and the observers were given feedback at the end of the testing period. No handling of food was permitted to simulate observation during meal conditions. Foods Used in Training The primary goal of training was to prepare field-workers to observe food consumption of young children in local households enrolled in a large case-control study. More than 150 different locally available foods and food mixtures were used for the training sessions. Practice and validation sessions concentrated on foods most commonly available and eaten by the indigenous population. Staple grains, vegetable and lentil preparations, and homemade unleavened breads were stressed. Less emphasis was placed on expensive foods, such as meat, or store-bought items (eg, loaf breads, candies), which are not commonly consumed. Observers estimated food weights of up to 1,300 g; however, the majority of food portions weighed reflected standard meal portions of less than 400 g. For any one testing session, food portions were randomly assigned but included a range of portion sizes possible for adults and children of all ages. Because shared-plate eating is common, portion sizes approximating "single bites" were included in most testing sessions. Food weights were not standardized between training and posttraining sessions to reduce observer bias resulting from "known" quantity estimates. However, the average food portions during training and posttraining testing did not differ significantly. All tested foods were in their prepared or ready-toeat form. As a result of limitations of availability and time, only a fraction of the total number of foods were used in any single training or validation session. Actual food weights were measured by means of two Sunbeam Digital Deluxe kitchen scales (Sunbeam/Sweeney & White, Waukesha, Wis). Statistical analyses were performed using the Statistical Analysis System for microcomputers (release 6.03, 1988, SAS Institute, Cary, NC) (21).
FIG 2. Observer-specific differences infood-quantity estimates with time.
FIG 3. Variation infood-quantity estimates by food group over time.
Calculations Results were expressed as a percentage error of estimated to actual food weight according to the following formula: percentage error = [estimated weight (g)/ actual weight (g) -1] x 100 For some analyses, the positive or negative sign associated with the percentage error was not used, and findings are expressed as the absolute value of the percentage error. Simple linear regression analysis was performed to examine the relationship between actual and estimated food weights. Pearson's correlation coefficients (r) were calculated to examine the linear goodness of fit between estimated and actual weights for all foods and within specific food groups. Effectiveness of training was determined by examining change in the absolute percentage error for all observers and over all foods over time. RESULTS For the testing period overall, observer estimates of food weights were highly correlated with actual weights (r=.96; No. of observaJOURNAL OF THE AMERICAN DIETETIC ASSOCIATION / 1275
RESEAH.................................................................... RESE^RCI ..................................................................
Table 2 Correlation between training week and absolute error for selected foodsa Specific food
Sample size (n)
Correlation (r)
Grains Rice Unleavened wheat bread Unleavened corn bread Unleavened millet bread
581 359 257 324
-. 31** -. 30*** - .31*** - 04
Mixed foods Rice and vegetables
131
- .15
Fruits Papaya Banana Vegetables Dark-green leafy vegetables Carrot
56 96
-. 35** .12
152 75
-. 26*** .02
ap= sample level of significance; **P<.01; ***P<.001.
FIG 4. Variation in accuracy by food characteristics.
FIG 5. Relationship between absolute error andfood weight over time.
1276 / NOVEMBER 1994 VOLUME 94 NUMBER 11
tions=6,902). Within defined food categories, this relationship held; r values ranged from .93 for vegetables to .98 for manufactured foods (Table 1). Linear regression analysis (Figure 1) suggested that the relationship between actual and observed food weights was also highly accurate, ie, the slope was close to 1.0 and a y intercept near 0 (y=.96x+1.3) (note that all food weights are expressed in grams). For the testing period, the accuracy of estimation did not vary considerably among observers, which was reflected by observer-specific mean percentage errors <6% across all food categories. The mean percentage errors of 80% of observers ranged from -4% to +4%. Individual standard deviations of up to 40%, however, reflected considerable food-specific variation. Separating the data into those collected during and after training revealed dramatic improvements in observer-specific estimates after training. Figure 2 demonstrates that with training and practice, nearly all observers reduced their variability in estimating food weights. Interobserver variability was also reduced; those observers initially showing the greatest deviations in food estimation improved the most. Improvements in observer estimates were seen within specific food groups (Figure 3). Estimates of fruits, vegetables, and manufactured foods (mostly sweets and candies) appeared to improve as a result of training. Some directionality in error for particular foods was found: staple grains, vegetables, and mixed foods tended to be underestimated, and fruits and liquids were more likely to be overestimated. Training did not appear to improve accuracy of estimates for all foods. Grains and mixed foods (including combinations of commonly eaten foods such as rice, vegetable, and dal curries) were accurately determined both during and after training. Table 2 shows changes in the absolute value of the percentage error over time for specific foods. Error in observer estimates of cooked rice, unleavened wheat and corn bread, dark-green leafy vegetables, and papaya decreased over time (reflected by a negative r), but estimated weights for some fruits and vegetables (ie, banana, carrot) and unleavened millet bread showed no reduction in error over time. One reason for variation in observer accuracy may be related to the ratio of volume to weight of a food item. To explore this possibility, we compared groups of foods of high volume and light weight (eg, puffed rice and corn, white bread, and glucose biscuits) with foods of medium volume and medium weight (eg, rice and vegetable preparations) and foods of low volume and heavy weights (eg, corn porridge and unleavened millet breads) (Figure 4). A substantially higher proportion (77%) of estimates of foods of low volume and heavy weight were within 20% of the actualfood weights, compared with only 57% of estimates offoods of high volume and light weight. Thus, the visual appearance of a food may affect an observer's ability to estimate its weight accurately. Foods of medium volume and medium weight (most commonly eaten by this population) were accurately estimated: more than 70% of all estimates were within 20% of actual weight. A second factor affecting accuracy of estimation may be the weight of food portion (Figure 5). Smaller portions of foods were less likely to be estimated accurately by observers than larger portions, regardless of food type. During training, accurate estimates (within 20% of actual weight) were seldom achieved for food portions with weights under 30 g. In the posttraining period, percentage error diminished notably for all food portion weights, and accurate estimates of food weights of 20+ g were attainable. DISCUSSION The use of direct observation for the assessment of dietary intake of children has been limited primarily to studies conducted in institutionalized settings, mostlyin developed nations (10,22,23). Studies done elsewhere tend to be anthropological and use small
samples observed by one or two trained investigators (5). Because direct observation procedures can be used to observe children throughout the day or during meal periods, they can provide information about the social and behavioral context of a meal, and more complete data on food consumed outside of structured mealtimes. Correlations between visually estimated food weight and actual food weight were found to be high and comparable to those reported in studies where food weights were visually estimated by trained dietitians (19,23). Our results concur with others that showthat continued, intensive training can improve the accuracy of observer estimates of food weight (22,24). Accuracy of estimation, however, clearly varies by type and quantity of food. One issue of concern is that observer estimates are relatively less accurate for small food portions. In young children, even very small portions of foodstuffs may be nutritionally important. For example, 30 g of a micronutrient-rich food such as mango contains approximately 120 RE vitamin A, or 30% of a small child's daily requirement. Fortunately, our results indicate that additional training can lead to accurate estimation of small quantities of fruits and vegetables. Our data also indicate that specific foods tend to be either overestimated or underestimated. If subsequent testing reveals consistent trends for certain foods, these data may be used to develop correction factors that may be applied in the analysis of our dietary data. Establishing the accuracy of visual estimation is a first step in the use of direct observation as a practical dietary assessment technique in the field. Our findings suggest that persons can be adequately trained to estimate quantities of foods by direct observation. Further research is needed to evaluate the performance of this technique in estimating dietary intake under usual household conditions. APPLICATIONS The findings reported here support the use of direct observation as a dietary assessment methodology applicable to various research settings. Visual estimates of food weights can be obtained accurately by trained observers, thereby making larger-scale dietary intake studies more feasible. Moreover, regular training can standardize observer proficiency of estimation in about 3 months for lay observers whose initial estimation skills are undeveloped. When tested 6 months after initial training (results not presented), observers were still, on average, able to make accurate estimates of food portions (within 20% of actual weight). Further applications of these findings relate to the preferred form and emphasis of observer training. To improve the accuracy of estimations, additional time should be spent on estimating smaller portion weights (<30 g). Moreover, repeated testing sessions that include seasonally available foods are essential. Periodic retesting, both during the training and study periods, is recommended to maintain high levels of accuracy for all observers. Finally, researchers who use observational techniques should be cognizant of biases that may be introduced because of the observer's ability or the particular food being estimated. These biases can be taken into account during data analysis. · The results presented here are part of a collaborativeeffort between The Johns Hopkins University, Baltimore,Md, and NepalNetra Jyoti Sangh (National Society for the Preventionof Blindness) and the Nepal Eye Hospital, Kathmandu, Nepal. The project isfunded through CooperativeAgreement DAN 0045-A between the Dana CenterforPreventive Ophthalmology, The Johns Hopkins University, and The Office of Nutrition, US Agency for InternationalDevelopment, with additionalsupportfrom
the Task Forcefor Sight and Life (Roche, Basel, Switzerland), and a NationalInstitutes of Health shared instrument grant S1 0-RR 04060. We acknowledge and appreciatethe considerable efforts of TaraP. Gnywali and Birendra Dahal on this project and thank the observerfield team in Sarlahifor their dedicated work. We thank GarretMehlfor his assistancein preparingthis manuscript. We also thank JoanneKatz, ScD, and Laura Caulfield, PhD,for theiruseful comments on drafts of this manuscript. References 1. BaranowskiT, Dworkin R, HenskeJC, Clearman DR, DunnJK, Nader PR, Hooks PC. The accuracy of children's self-reports of diet: Family Health Project. JAm Diet Assoc. 1986; 86:1381-1385. 2. Persson LA, Carlgren G. Measuring children's diets: evaluation of dietary assessment techniques in infancy and childhood. IntJEpidemiol. 1984; 13:506-517. 3. Baranowski T, Sprague D, Baranowski JH, Harrison JA. Accuracy of maternal dietary recall for preschool children. JAm DietAssoc. 1991; 91:669-674. 4. Davidson FR, Hayek LE, Altschul AM. Towards accurate assessment of children's food consumption. Ecol Food Nutr 1986; 18:309-317. 5. Wilson CS. Child following: a technic for learning food and nutrient intakes. J Trop Pediatr. 1974; 20:9-14. 6. Wahlqvist ML, Kouris A, Gracey M, Sullivan H. An anthropological approach to the study of food and health in an indigenous population. Food Nutr Bull. 1991; 13:145-149. 7. Madden JP, Goodman SJ, Guthrie HA. Validity of the 24-hour recall. JAm Diet Assoc. 1976; 68:143-147. 8. Block G. A review of validations of dietary assessment methods.Am JEpidemiol. 1982; 115:492-505. 9. Block G, Hartman AH. Issues in reproducibility and validity of dietary studies. Am J Clin Nutr. 1989; 50;1133-1138. 10. Simons-Morton BG, Baranowski T. Observation in assessment of children's dietary practices. JSch Health. 1991; 61:204-207.
11. Smith AF. Cognitive Processes in Long-term Dietary Recall. Washington, DC: Public Health Service; 1991. Vital and Health Statistics. Series 6, No. 4. Dept of Health and Human Services publication (PHS) 92-1079. 12. Haraldsdottir J. Minimizing error in the field: quality control in dietary surveys. Eur J Clin Nutr. 1993; 47(suppl 2):S19-S24. 13. Graves K, Shannon B. Using visual plate waste measurement to assess school lunch food behavior. JAm Diet Assoc. 1983; 82:163-165. 14. Gittelsohn J. Opening the box: intrahousehold food distribution in rural Nepal. Soc Sci Med. 1991; 33:1141-1154. 15. Guldan GS, Zeitlin MF, Beiser AS, Super CM, Gershoff SN, Datta S. Maternal education and child feeding practices in rural Bangladesh. Soc Sci Med. 1993; 36:925-935. 16. Kohlmeier L. Overview of validity, quality control and measurement error issues in nutritional epidemiology. Eur J Clin Nutr. 1993; 47(suppl 2):S1-S5. 17. Freudenheim JL. A review of study designs and methods of dietary assessment in nutritional epidemiology of chronic disease. J Nutr. 1993; 123:401-405. 18. Simons-Morton BG, Forthofer R, Huang IW, Baranowski T, Reed DB, Fleishman R. Reliability of direct observation of school children's consumption of bag lunches. JAm Diet Assoc. 1992; 92:219-221. 19. Dubois S. Accuracy of visual estimates of plate waste in the determination of food consumption.JAmDietAssoc. 1990; 90:382-387. 20. Thompson CH, Head MK, Rodman SM. Factors influencing accuracy in estimating plate waste. JAm DietAssoc. 1987; 87:1219-1220. 21. SAS ProceduresGuide. Release 6.03 ed. Cary, NC: SAS Institute; 1988. 22. Bolland JE, Yuhas JA, Bolland TW. Estimation of food portion sizes: effectiveness of training. JAm DietAssoc. 1988; 88:817-821. 23. Comstock EM, Symington LE. Distribution of serving sizes and plate waste in school lunches. JAm Diet Assoc. 1982; 82:413-422. 24. Bolland JE, Ward JY, Bolland TW. Improved accuracy of estimating food quantities up to four weeks after training. JAm Diet Assoc. 1990; 90:1402-1407. JOURNAL OF THE AMERICAN DIETETIC ASSOCIATION / 1277
Accupay of estimating food intake by observation JOEL GITTELSOHN, PhD, ANITA Vr SHANAR, MS, RAMP. POKHREL,FRCS, KEITH P WEST, Jr,DrPH,RD
AIBSTRACT Objective To determine the accuracy of direct observation in food-weight estimation as measured under controlled field conditions. Design and subjects Ten local Nepahis were trained in observational techniques and tested in food-weight estimation during a 3-month training period and for 4 months after training. Setting The study was carried out in the Sarlahi District, a rural, central lowland region of Nepal that borders India. Main outcome measures Thirty testing sessions (a total of 6,902 observations) were completed on more than 150 different foods. Estimates of observed food weight were compared with actual weights and were analyzed. Statistical analyses performed Pearson's correlation coefficients were calculated to examine associations between estimated and actual weights. Results Observer estimates of food weights were highly correlated with actual weights (r=.96) for the entire testing period. The linear regression equation (y=.96+1.3) suggests that the relationship between actual and observed food weights in grams) was also accurate. Most observers showed improvement with training. Substantial reductions in both mean and standard deviation of percentage error were achieved over time. Accuracy of estimates was influenced by characteristics of foods weighed; small quantities (less than 20 g), certain nonstaple foods, and foods of high volume but light weight had less accurate estimates. Conclusions Direct observation is an important method for assessing dietary intake that does not rely on a respondents' ability to recall Wis or her own or another's food consumption. It is feasible to train local observers to make visual estimates of food weight, but the accuracy of their estimates varies by food and portion size. JAm DietAssoc. 1994; 94:1273-1277
dietary assessment methodologies, including (a) retrospective methods, such as 24-hour recalls, food frequencies, and diet histories, based on narrative accounts of foods consumed in the past; and (b) prospective methods, such as food diaries, food weighings, and direct observation, where food intake is recorded at the time of consumption. This article examines the accuracy of direct observation, an underused but potentially important method of dietary assessment. In direct observation, trained personnel observe food consumption behavior and visually estimate food intake. Direct observation is a useful method for the study of dietary intakes of young children, because obtaining accurate assessments of their dietary intake is often problematic. Although children may be old enough to procure foods independently, they may be too young to recall their food consumption accurately (1,2). In such circumstances, dietary intake data of children are often gathered through the recall of a parent. Reliance on parental or caretaker recall of a child's food intake may provide reasonably accurate data for mealtime consumption, but may exclude foods eaten outside the view of adults (3,4). For instance, in her dietary study ofyoung Malay children, Wilson (5) found that a substantial portion of energy and vitan-dn C were consumed at times other than household meal hours. Other studies support this conclusion by reporting notable differences in quantity and variety of food consumed when observational methods were used and compared with dietary surveys of the same population (4,6). Prospective methods have other advantages over retrospective methods in that they are not subject to memory loss, inaccurate reporting, or error associated with question formlation (7-12). However, some prospective methods, such as self-administered food diaries or food weighing, are considered to be intrusive and may alter dietary behaviors. J. Gittelsohn is an assistant rofessor, A. Vr Shankaris a researchassociate, and P. West, Jr, is an associate professorin the Division of HumanNutrition, Department of InternationalHealth, School ofHygieneR and Public Health, The Johns Hopkins University, Baltimore, Md. R.P. Pokhrel is chairmann of the NationalSocietyfor the Prevention ofBlindness with the Nepal Ee Hospital, Kathmandu, Nepal. Address correspondenceto: Joel Gittelsohn, Phl) DivisionofHuman Nutrition, Departmentof Interntional Health, School fHygiene and Public Health, The Johns Hopkins Universiy 615 N Woffie St, Baltimore, MD 21205. JOURNAL OF THE AMERICAN DIETETIC ASSOCIATION 1273
....................................................................
..........................................................
RWSIICH
Table 1 Correlation coefficients for actual and estimated food weights for specific food groups Food Group
Sample size (n)
Correlation (r)
Fruits
265
.96
Vegetables
944
.93
Grains
2,773
.96
Liquids
1,436
.96
Mixed foods
595
.95
Manufactured foods
398
.98
Meat/fish
398
.97
6,809a
.96
All categorized foods
aTotal number of observer estimates analyzed was 6,902; 93 of these food estimates were not categorized by food group.
FIG 1. Accuracy offood-weight estimates.
1274/ NOVEMBER 1994 VOLUME 94 NUMBER 11
Direct observation is more advantageous than other prospective techniques because the method does not depend on the literacy level of the respondent, and it is less likely to alter natural patterns of mealtime behavior (13). In rural Nepal, children frequently share food, eat in groups from the same plate, and interact with the food server in a number of different ways that would be seriously disrupted by the need to weigh each food serving. In this setting, direct observations of food intake may provide the most accurate estimates of usual food intake for children, with the least disruption of natural behavior (14). Moreover, direct observation methods can capture details of feeding behavior that would otherwise not be obtained. In a study in Bangladesh, Guldan and colleagues (15) used observational techniques to examine child feeding practices; they found significant associations between factors such as frequency of feeding, freshness of foods, cleanliness of feeding areas, and feeding style. Unfortunately, recent reviews in nutritional epidemiology fail to mention observational techniques as a feasible alternative for dietary assessment (16,17). On the other hand, researchers who use observational techniques consider them the standard against which other dietary assessment techniques can be compared (1,18). The paucity of data on observational techniques, therefore, has resulted in only a few studies that examine the training of observers and standardization of procedures (10,19). We were able to locate only one study that attempted to characterize how food-weight estimation varies by visual attributes of food and food type (20). Such information is critical in assessing the validity of any dietary assessment technique and is of particular importance for studies conducted in less controllable, rural field conditions by lay observers. This article examines issues of accuracy and explores the feasibility of training local field workers in direct observation procedures. METHODS This study was conducted in the Sarlahi District, a rural, central, lowland region of Nepal that borders India. As part of a community-based study of dietary intake in children, 10 local Nepalis were trained for 3 months (September through November 1992) in observational techniques and estimation of food weight. These 3 months constituted the formal training period. Each week, validation sessions were conducted in which observer estimates of food quantities were compared with actual food weights for various foods. For 4 months after the training (December 1992 through March 1993), hereafter referred to as the posttraining period, we continued to conduct validation sessions to familiarize observers with new seasonally available foods and to review commonly eaten foods. During the testing period, which included both training and posttraining, 30 validation sessions were conducted and analyzed (14 during training and 16 during the posttraining period). All observers were men and were from the region. Each observer had a minimum of a high school diploma, and none had prior training in nutrition or research. Activities to improve observers' food-estimation skills included the following group and individual practice sessions: * Group practice estimation sessions were conducted regularly throughout the testing period to familiarize observers with estimating food-specific weights. Foods were presented in standard local plates or glasses, with their true weights disclosed to the group. Observers were permitted to examine and handle foods closely to develop an intuitive sense of the relationships of food weights, volume, and size. Later, observers were asked to estimate the weight of portions of the same foods of unknown weight based on this initial experience. Observers called out their estimates to the group; adjustment of subsequent answers based on previous responses was allowed. This process helped to standardize responses among observers.
* Individual practice estimation sessions were conducted every other day during the training period and weekly during the posttraining period. Observers independently estimated preweighed portions of different sizes for a small number of foods. Actual food weight was provided after each estimate to allow each observer to evaluate his food-specific estimation skills. * Validation sessions were conducted approximately once a week for the 7-month testing period. Observers were asked to estimate 10 to 30 different preweighed food quantities. Results were compared, and the observers were given feedback at the end of the testing period. No handling of food was permitted to simulate observation during meal conditions. Foods Used in Training The primary goal of training was to prepare field-workers to observe food consumption of young children in local households enrolled in a large case-control study. More than 150 different locally available foods and food mixtures were used for the training sessions. Practice and validation sessions concentrated on foods most commonly available and eaten by the indigenous population. Staple grains, vegetable and lentil preparations, and homemade unleavened breads were stressed. Less emphasis was placed on expensive foods, such as meat, or store-bought items (eg, loaf breads, candies), which are not commonly consumed. Observers estimated food weights of up to 1,300 g; however, the majority of food portions weighed reflected standard meal portions of less than 400 g. For any one testing session, food portions were randomly assigned but included a range of portion sizes possible for adults and children of all ages. Because shared-plate eating is common, portion sizes approximating "single bites" were included in most testing sessions. Food weights were not standardized between training and posttraining sessions to reduce observer bias resulting from "known" quantity estimates. However, the average food portions during training and posttraining testing did not differ significantly. All tested foods were in their prepared or ready-toeat form. As a result of limitations of availability and time, only a fraction of the total number of foods were used in any single training or validation session. Actual food weights were measured by means of two Sunbeam Digital Deluxe kitchen scales (Sunbeam/Sweeney & White, Waukesha, Wis). Statistical analyses were performed using the Statistical Analysis System for microcomputers (release 6.03, 1988, SAS Institute, Cary, NC) (21).
FIG 2. Observer-specific differences infood-quantity estimates with time.
FIG 3. Variation infood-quantity estimates by food group over time.
Calculations Results were expressed as a percentage error of estimated to actual food weight according to the following formula: percentage error = [estimated weight (g)/ actual weight (g) -1] x 100 For some analyses, the positive or negative sign associated with the percentage error was not used, and findings are expressed as the absolute value of the percentage error. Simple linear regression analysis was performed to examine the relationship between actual and estimated food weights. Pearson's correlation coefficients (r) were calculated to examine the linear goodness of fit between estimated and actual weights for all foods and within specific food groups. Effectiveness of training was determined by examining change in the absolute percentage error for all observers and over all foods over time. RESULTS For the testing period overall, observer estimates of food weights were highly correlated with actual weights (r=.96; No. of observaJOURNAL OF THE AMERICAN DIETETIC ASSOCIATION / 1275
RESEAH.................................................................... RESE^RCI ..................................................................
Table 2 Correlation between training week and absolute error for selected foodsa Specific food
Sample size (n)
Correlation (r)
Grains Rice Unleavened wheat bread Unleavened corn bread Unleavened millet bread
581 359 257 324
-. 31** -. 30*** - .31*** - 04
Mixed foods Rice and vegetables
131
- .15
Fruits Papaya Banana Vegetables Dark-green leafy vegetables Carrot
56 96
-. 35** .12
152 75
-. 26*** .02
ap= sample level of significance; **P<.01; ***P<.001.
FIG 4. Variation in accuracy by food characteristics.
FIG 5. Relationship between absolute error andfood weight over time.
1276 / NOVEMBER 1994 VOLUME 94 NUMBER 11
tions=6,902). Within defined food categories, this relationship held; r values ranged from .93 for vegetables to .98 for manufactured foods (Table 1). Linear regression analysis (Figure 1) suggested that the relationship between actual and observed food weights was also highly accurate, ie, the slope was close to 1.0 and a y intercept near 0 (y=.96x+1.3) (note that all food weights are expressed in grams). For the testing period, the accuracy of estimation did not vary considerably among observers, which was reflected by observer-specific mean percentage errors <6% across all food categories. The mean percentage errors of 80% of observers ranged from -4% to +4%. Individual standard deviations of up to 40%, however, reflected considerable food-specific variation. Separating the data into those collected during and after training revealed dramatic improvements in observer-specific estimates after training. Figure 2 demonstrates that with training and practice, nearly all observers reduced their variability in estimating food weights. Interobserver variability was also reduced; those observers initially showing the greatest deviations in food estimation improved the most. Improvements in observer estimates were seen within specific food groups (Figure 3). Estimates of fruits, vegetables, and manufactured foods (mostly sweets and candies) appeared to improve as a result of training. Some directionality in error for particular foods was found: staple grains, vegetables, and mixed foods tended to be underestimated, and fruits and liquids were more likely to be overestimated. Training did not appear to improve accuracy of estimates for all foods. Grains and mixed foods (including combinations of commonly eaten foods such as rice, vegetable, and dal curries) were accurately determined both during and after training. Table 2 shows changes in the absolute value of the percentage error over time for specific foods. Error in observer estimates of cooked rice, unleavened wheat and corn bread, dark-green leafy vegetables, and papaya decreased over time (reflected by a negative r), but estimated weights for some fruits and vegetables (ie, banana, carrot) and unleavened millet bread showed no reduction in error over time. One reason for variation in observer accuracy may be related to the ratio of volume to weight of a food item. To explore this possibility, we compared groups of foods of high volume and light weight (eg, puffed rice and corn, white bread, and glucose biscuits) with foods of medium volume and medium weight (eg, rice and vegetable preparations) and foods of low volume and heavy weights (eg, corn porridge and unleavened millet breads) (Figure 4). A substantially higher proportion (77%) of estimates of foods of low volume and heavy weight were within 20% of the actualfood weights, compared with only 57% of estimates offoods of high volume and light weight. Thus, the visual appearance of a food may affect an observer's ability to estimate its weight accurately. Foods of medium volume and medium weight (most commonly eaten by this population) were accurately estimated: more than 70% of all estimates were within 20% of actual weight. A second factor affecting accuracy of estimation may be the weight of food portion (Figure 5). Smaller portions of foods were less likely to be estimated accurately by observers than larger portions, regardless of food type. During training, accurate estimates (within 20% of actual weight) were seldom achieved for food portions with weights under 30 g. In the posttraining period, percentage error diminished notably for all food portion weights, and accurate estimates of food weights of 20+ g were attainable. DISCUSSION The use of direct observation for the assessment of dietary intake of children has been limited primarily to studies conducted in institutionalized settings, mostlyin developed nations (10,22,23). Studies done elsewhere tend to be anthropological and use small
samples observed by one or two trained investigators (5). Because direct observation procedures can be used to observe children throughout the day or during meal periods, they can provide information about the social and behavioral context of a meal, and more complete data on food consumed outside of structured mealtimes. Correlations between visually estimated food weight and actual food weight were found to be high and comparable to those reported in studies where food weights were visually estimated by trained dietitians (19,23). Our results concur with others that showthat continued, intensive training can improve the accuracy of observer estimates of food weight (22,24). Accuracy of estimation, however, clearly varies by type and quantity of food. One issue of concern is that observer estimates are relatively less accurate for small food portions. In young children, even very small portions of foodstuffs may be nutritionally important. For example, 30 g of a micronutrient-rich food such as mango contains approximately 120 RE vitamin A, or 30% of a small child's daily requirement. Fortunately, our results indicate that additional training can lead to accurate estimation of small quantities of fruits and vegetables. Our data also indicate that specific foods tend to be either overestimated or underestimated. If subsequent testing reveals consistent trends for certain foods, these data may be used to develop correction factors that may be applied in the analysis of our dietary data. Establishing the accuracy of visual estimation is a first step in the use of direct observation as a practical dietary assessment technique in the field. Our findings suggest that persons can be adequately trained to estimate quantities of foods by direct observation. Further research is needed to evaluate the performance of this technique in estimating dietary intake under usual household conditions. APPLICATIONS The findings reported here support the use of direct observation as a dietary assessment methodology applicable to various research settings. Visual estimates of food weights can be obtained accurately by trained observers, thereby making larger-scale dietary intake studies more feasible. Moreover, regular training can standardize observer proficiency of estimation in about 3 months for lay observers whose initial estimation skills are undeveloped. When tested 6 months after initial training (results not presented), observers were still, on average, able to make accurate estimates of food portions (within 20% of actual weight). Further applications of these findings relate to the preferred form and emphasis of observer training. To improve the accuracy of estimations, additional time should be spent on estimating smaller portion weights (<30 g). Moreover, repeated testing sessions that include seasonally available foods are essential. Periodic retesting, both during the training and study periods, is recommended to maintain high levels of accuracy for all observers. Finally, researchers who use observational techniques should be cognizant of biases that may be introduced because of the observer's ability or the particular food being estimated. These biases can be taken into account during data analysis. · The results presented here are part of a collaborativeeffort between The Johns Hopkins University, Baltimore,Md, and NepalNetra Jyoti Sangh (National Society for the Preventionof Blindness) and the Nepal Eye Hospital, Kathmandu, Nepal. The project isfunded through CooperativeAgreement DAN 0045-A between the Dana CenterforPreventive Ophthalmology, The Johns Hopkins University, and The Office of Nutrition, US Agency for InternationalDevelopment, with additionalsupportfrom
the Task Forcefor Sight and Life (Roche, Basel, Switzerland), and a NationalInstitutes of Health shared instrument grant S1 0-RR 04060. We acknowledge and appreciatethe considerable efforts of TaraP. Gnywali and Birendra Dahal on this project and thank the observerfield team in Sarlahifor their dedicated work. We thank GarretMehlfor his assistancein preparingthis manuscript. We also thank JoanneKatz, ScD, and Laura Caulfield, PhD,for theiruseful comments on drafts of this manuscript. References 1. BaranowskiT, Dworkin R, HenskeJC, Clearman DR, DunnJK, Nader PR, Hooks PC. The accuracy of children's self-reports of diet: Family Health Project. JAm Diet Assoc. 1986; 86:1381-1385. 2. Persson LA, Carlgren G. Measuring children's diets: evaluation of dietary assessment techniques in infancy and childhood. IntJEpidemiol. 1984; 13:506-517. 3. Baranowski T, Sprague D, Baranowski JH, Harrison JA. Accuracy of maternal dietary recall for preschool children. JAm DietAssoc. 1991; 91:669-674. 4. Davidson FR, Hayek LE, Altschul AM. Towards accurate assessment of children's food consumption. Ecol Food Nutr 1986; 18:309-317. 5. Wilson CS. Child following: a technic for learning food and nutrient intakes. J Trop Pediatr. 1974; 20:9-14. 6. Wahlqvist ML, Kouris A, Gracey M, Sullivan H. An anthropological approach to the study of food and health in an indigenous population. Food Nutr Bull. 1991; 13:145-149. 7. Madden JP, Goodman SJ, Guthrie HA. Validity of the 24-hour recall. JAm Diet Assoc. 1976; 68:143-147. 8. Block G. A review of validations of dietary assessment methods.Am JEpidemiol. 1982; 115:492-505. 9. Block G, Hartman AH. Issues in reproducibility and validity of dietary studies. Am J Clin Nutr. 1989; 50;1133-1138. 10. Simons-Morton BG, Baranowski T. Observation in assessment of children's dietary practices. JSch Health. 1991; 61:204-207.
11. Smith AF. Cognitive Processes in Long-term Dietary Recall. Washington, DC: Public Health Service; 1991. Vital and Health Statistics. Series 6, No. 4. Dept of Health and Human Services publication (PHS) 92-1079. 12. Haraldsdottir J. Minimizing error in the field: quality control in dietary surveys. Eur J Clin Nutr. 1993; 47(suppl 2):S19-S24. 13. Graves K, Shannon B. Using visual plate waste measurement to assess school lunch food behavior. JAm Diet Assoc. 1983; 82:163-165. 14. Gittelsohn J. Opening the box: intrahousehold food distribution in rural Nepal. Soc Sci Med. 1991; 33:1141-1154. 15. Guldan GS, Zeitlin MF, Beiser AS, Super CM, Gershoff SN, Datta S. Maternal education and child feeding practices in rural Bangladesh. Soc Sci Med. 1993; 36:925-935. 16. Kohlmeier L. Overview of validity, quality control and measurement error issues in nutritional epidemiology. Eur J Clin Nutr. 1993; 47(suppl 2):S1-S5. 17. Freudenheim JL. A review of study designs and methods of dietary assessment in nutritional epidemiology of chronic disease. J Nutr. 1993; 123:401-405. 18. Simons-Morton BG, Forthofer R, Huang IW, Baranowski T, Reed DB, Fleishman R. Reliability of direct observation of school children's consumption of bag lunches. JAm Diet Assoc. 1992; 92:219-221. 19. Dubois S. Accuracy of visual estimates of plate waste in the determination of food consumption.JAmDietAssoc. 1990; 90:382-387. 20. Thompson CH, Head MK, Rodman SM. Factors influencing accuracy in estimating plate waste. JAm DietAssoc. 1987; 87:1219-1220. 21. SAS ProceduresGuide. Release 6.03 ed. Cary, NC: SAS Institute; 1988. 22. Bolland JE, Yuhas JA, Bolland TW. Estimation of food portion sizes: effectiveness of training. JAm DietAssoc. 1988; 88:817-821. 23. Comstock EM, Symington LE. Distribution of serving sizes and plate waste in school lunches. JAm Diet Assoc. 1982; 82:413-422. 24. Bolland JE, Ward JY, Bolland TW. Improved accuracy of estimating food quantities up to four weeks after training. JAm Diet Assoc. 1990; 90:1402-1407. JOURNAL OF THE AMERICAN DIETETIC ASSOCIATION / 1277