Validity and Reliability of a Food Skills Questionnaire

ARTICLE IN PRESS Research Article Validity and Reliability of a Food Skills Questionnaire Lauren G. Kennedy, MScFN, RD1,*; Emily J. Kichler, MScFN, RD1,*; Jamie A. Seabrook, PhD1,2,3,4,5,6; June I. Matthews, PhD, RD, PHEc1; Paula D.N. Dworatzek, PhD, RD, PHEc1,7 ABSTRACT Objective: Develop, validate, and assess reliability of a food skills questionnaire. Design: Phase 1: Questionnaire development categorized questions into domains (Food Selection and Planning, Food Preparation, and Food Safety and Storage). Phase 2: Content validity included expert panel quantitative and qualitative feedback. Phase 3: Face validity involved pilot testing. Phase 4: Reliability assessed test-retest and inter-item reliability. Setting: Phase 1: The authors developed a draft questionnaire in London, Ontario, Canada. Participants: Phase 2: Dietitians, home economists, academics, and chefs completed content validity (n = 17; 57% response rate). Phase 3: A convenience sample of students completed face validity (n = 20; 17% response rate). Phase 4: Randomly selected students completed test-retest reliability (time 1: n = 189, time 2: n = 165; 9% response rate). Main Outcome Measures: Lawshe content validity ratio, Lawshe content validity index, intraclass correlation coefficients (ICCs), and Cronbach a. Analysis: Test-retest reliability was evaluated using ICC, and inter-item reliability by Cronbach a coefficient. Results: In phase 2, Lawshe content validity index was 0.80 (90% expert panel consensus). In phase 3, 85% of respondents identified the main construct. In phase 4, Cronbach a coefficients were .67−.88 for domains and .90 for the questionnaire overall, and ICC scores ranged from 0.67−0.92 for questions, 0.86 −0.93 for domains, and 0.92 for the questionnaire overall. Conclusions and Implications: This questionnaire demonstrated strong content validity, face validity, test-retest reliability, and good inter-item reliability. It is appropriate for evaluating food skills in a population with basic to intermediate skills (eg, young adults). Key Words: cooking, food safety, food skills, surveys and questionnaires, validation studies (J Nutr Educ Behav. 2019;000:1−8.) Accepted February 10, 2019.

INTRODUCTION The concept of food skills is complex;1 however, it can be represented by 3 domains: Food Selection and

Planning, Food Preparation, and Food Safety and Storage. Each domain includes knowledge, attitudes, behaviors, and conceptualization of outcomes.1−3 Food skills are a

1

School of Food and Nutritional Sciences, Brescia University College, Western University, London, Ontario, Canada 2 Department of Paediatrics, Western University, London, Ontario, Canada 3 Department of Epidemiology and Biostatistics, Western University, London, Ontario, Canada 4 Children’s Health Research Institute, London, Ontario, Canada 5 Lawson Health Research Institute, London, Ontario, Canada 6 Human Environments Analysis Laboratory, Western University, London, Ontario, Canada 7 Schulich Interfaculty Program in Public Health, Western University, London, Ontario, Canada *These authors contributed equally to this study. Conflict of Interest Disclosure: The authors have not stated any conflicts of interest. Address for correspondence: Paula Dworatzek, PhD, RD, PHEc, School of Food and Nutritional Sciences, Brescia University College, Western University, 1285 Western Rd, London, ON N6G 1H2, Canada; E-mail: [email protected] Ó 2019 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jneb.2019.02.003

Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019

component of food literacy, which is an understanding of the full cycle of food practices (farm to table to waste), including impacts on health, and economic, social, and environmental sustainability.4,5 Low food skills or cooking self-efficacy are barriers to healthy eating.6,7 Conversely, higher food preparation skills are associated with better diet quality7−9 including increased intake of vegetables and fruits,1,8,9 lower intake of calories, fat, and sugar,2 and lower intake of fast food,1,2,7−9 which are all associated with reduced risk for diet-related chronic disease.10 Various programs and interventions have been developed to improve food skills;11−14 however, methodological limitations including nonexperimental study designs, lack of follow-up data, and use of non-validated assessment tools14,15 highlight the need for more stringent study designs and development of validated

1

ARTICLE IN PRESS 2

Kennedy et al

measurement tools.15 Measuring or quantifying food skills is methodologically challenging because there is no reference standard for comparison.16 Self-reported familiarity with cooking techniques was identified as a motivator for food preparation.17 Confidence in food preparation skills may influence purchasing behaviors and diet quality,18 and attitudes and beliefs affect the quantity, frequency, and time spent preparing food.17,18 Currently, validated surveys and scales measure only some aspects of food skills, such as food safety19 or self-reported technical cooking skills,20 and may not assess food skills as an overall construct. Others identified cooking skills as distinct from food skills.21 However, cooking skills would generally be identified as a component of food skills. Conversely, other surveys focus on nutrition literacy in the name of food literacy.22 Although nutrition literacy is important, it is only 1 aspect of food literacy. The objective of this study was to assess the validity and reliability of a single questionnaire measuring basiclevel food skills (eg, following a simple recipe; peeling, chopping, and slicing; checking best-before dates) to intermediate-level food skills (eg, adjusting a recipe to make it healthier; planning a meal using foods already in the home; cooking foods to the correct internal temperature). In addition, some skills were practical (eg, knife skills) whereas others were more conceptual (eg, meal planning). Advanced food skills (eg, creating a new recipe, canning and preserving food, making bread) were not included.

METHODS This study developed and validated a food skills questionnaire over 4 phases: (1) questionnaire development, (2) content validity, (3) face validity, and (4) reliability. Through full review, the Western University Research Ethics Board approved the study. All participants provided informed consent.

Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019 and previous surveys,25−27 occurred between 2012 and 2015. To minimize measurement error, the researchers designed questions to be concise and clearly understood, using basic sentence structure and providing definitions.28 Questions were categorized into Food Selection and Planning, Food Preparation, and Food Safety and Storage domains. Most questions used an 11-point scale (0−100); however, attitudinal questions were on a 6-point scale of strongly disagree to strongly agree and food safety questions were primarily on a 5-point scale of never to always. Personal and environmental factors must be acknowledged when researching food skills, because these affect self-efficacy, attitudes, and abilities. Questions about demographics, barriers to food skills, food skills history, and contextual factors (eg, living situation) were developed to form a pool of supplemental questions, but were not included in validity and reliability assessment.

Phase 2: Content Validity Content validity is the degree to which a tool captures components of the construct that it is intended to measure.29 Lawshe content validity ratio30 was used because it is a rigorous methodological approach to assess the validity of individual items and the overall questionnaire. It has been used in diverse areas of research and is considered 1 of the most accepted tools for content validity assessments. A panel of 5−10 experts is considered sufficient,31 but the researchers invited a larger sample (n = 30) of registered dietitians, public health nutritionists, academics, home economists, family studies/ home economics teachers, and chefs, who were recognized for their

expertise, to participate. Most were Canadian; others resided in the US, Australia, and Ireland. Anonymous responses were collected electronically in July and September, 2015. Quantitative responses. Following Lawshe’s32 content evaluation panel methodology, participants indicated through an online survey whether each question was essential, useful but not essential, or not necessary. Content validity for each question was calculated using Lawshe content validity ratio (CVR) equation (Figure),32 which yielded values between −1 and +1. Positive values indicated that over half of experts reported the question to be essential. The minimum CVR required for an item to be valid depends on the number of experts providing feedback.32 For example, when there are 15 experts, a minimum CVR of 0.49 is required, meaning that there is 80% agreement that the question is essential. Lawshe provided minimum CVR values for expert panels of 5−15, 20, 25, 30, 35, or 40. In the current study, the closest conservative minimum CVR value was used, depending on the final sample size of expert respondents per question. Questions were retained when they had valid CVRs, or when CVRs were not valid but were supported by experts’ qualitative feedback. Overall content validity was assessed by the content validity index (CVI), which was calculated by averaging CVRs of retained questions (Figure).33 Qualitative responses. Two authors independently assessed these data. Experts’ comments informed the adequacy of chosen domains, as well as the clarity and breadth of food skills components in the questionnaire. For example, in the initial assessment, a question regarding the frequency of

Phase 1: Questionnaire Development Development and refinement of questionnaire items, informed by expert opinion, relevant literature,12,23,24

Figure. Lawshe content validity equations. 32

ARTICLE IN PRESS Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019 home-prepared meals was not included; however, it was recommended by some expert reviewers and was added for the reassessment. Content validity reassessment. The expert panel reassessed questions that had been added, modified, or assigned to a different domain. Content validity for each item was recalculated. Retained questions from both phases comprised the revised questionnaire.

Phase 3: Face Validity Face validity is the degree to which a measure captures the intended construct in the view of intended participants.29 A convenience sample of 20 undergraduate students who were previously shown to have basic to intermediate food skills26,34 was recruited in October, 2015 at Western University. Students enrolled in food and nutrition programs were excluded. Participants completed a paper version of the questionnaire and participated in a 10-minute interview to provide additional feedback and identify what they thought the questionnaire measured. The percentage of respondents identifying at least 1 domain of food skills was recorded, as were individual and mean completion times.

Phase 4: Reliability The revised questionnaire was formatted online using Fluidsurveys software (Fluidware, Inc, Ottawa, Ontario, Canada, 2016). Given a sampling target of 200 undergraduate students, 2,110 were randomly selected and invited to participate. From January to March, 2016, a hyperlink to the questionnaire was e-mailed to participating students. Those who completed the first questionnaire (time 1, test) were sent another link to the survey 2 weeks later and asked to complete it again (time 2, retest). Questionnaire responses were analyzed for interitem reliability and test-retest reliability. Data were analyzed using SPSS Statistics software (version 24, IBM Corporation, Armonk, NY, 2016).

Inter-item reliability. Cronbach a coefficient determined how well items within each domain correlated with each other (internal consistency).29,33 When participants’ scores were consistent for questions within a domain, questions in that domain were deemed an appropriate representation of that construct. Inter-item reliabilities were interpreted as <0.50 = unacceptable, 0.50−0.59 = poor, 0.60−0.69 = questionable, 0.70−0.79 = acceptable, and >0.80 = good.33 Cronbach a coefficients were calculated at time 1 for each domain and the questionnaire. Test-retest reliability. A 2-week timeframe between tests was considered short enough to minimize changes in food skills and long enough to avoid allowing participants to remember previous responses.35 At times 1 and 2, mean food skills scores were calculated for each domain and the questionnaire. Most response categories had an 11-point scale (range, 0−100); others were reweighted to be out of 100 to ensure that all questions were equally represented. Participant scores were included in the mean if they responded to at least half of a domain’s questions. The intraclass correlation coefficient (ICC) was calculated for individual questions to assess the similarity of responses at times 1 and 2.36 Thus, data only from participants who responded at both times 1 and 2 were included. The ICC was calculated for each domain and the questionnaire. The level of agreement indicated by ICC was interpreted as 0.00−0.10 = virtually no agreement, 0.11−0.40 = slight agreement, 0.41−0.60 = fair agreement, 0.61−0.80 = moderate agreement, and 0.81−1.0 = substantial agreement.37

RESULTS Phase 1: Questionnaire Development After multiple drafts, the questionnaire contained 51 items, with 16, 27, and 8 questions in Food Selection and Planning, Food Preparation, and Food Safety and Storage, respectively. Given the wide range of skills involved in food preparation, there were more questions in that domain than in the other 2.

Kennedy et al

3

Phase 2: Content Validity Quantitative and qualitative feedback. TaggedPOf the 30 experts invited, 17 participated, 16 of whom provided qualitative feedback on clarity, content, and length. Lawshe CVRs (Table 1) were calculated for each question. The initial CVI for the overall questionnaire was 0.49, achieving the required CVI for validity. This translated to 75% expert consensus that 24 items were essential. Of the 51 original questions, 24 were retained, 16 were discarded, and 11 were modified or moved to a different domain. Fifteen new questions were added, which resulted in 26 questions requiring reassessment. Content validity reassessment. Of the 17 experts, 11 reassessed the 26 questions; 15 were retained and 11 were eliminated. The refined questionnaire contained 39 items, with 9, 20, and 10 items in Food Selection and Planning, Food Preparation, and Food Safety and Storage, respectively. Four retained items in the Food Preparation domain had low Lawshe CVRs; however, these items were kept. Using a triangulated approach, these decisions were based on qualitative feedback (Table 2), Cronbach a of each domain (Table 1), support from the literature, and discussion among the authors. The final Lawshe CVI of the questionnaire was 0.80, translating to 90% consensus that retained questions were essential (Table 1). For a final expert sample size of 11, a CVI of 0.59 is acceptable.32 Phase 3: Face Validity. A total of 85% of participants (17 of 20; 85% female) identified that the survey measured food skills or ≥1 component in the 3 domains. Completion time for the food skills questionnaire, together with the supplemental questions, was 16 § 4 minutes. Moreover, 95% of participants reported that the questionnaire length was appropriate. The researchers made minor modifications to flow and clarity.

Phase 4: Reliability Of the students invited, 189 participated at time 1 (9% response rate). At time 2, 165 of participants (mean age, 22 § 6 years; 73% female)

4

Table 1. Content Validity, Test-Retest Reliability,and Inter-item Reliability for Food Skills Questionnaire

Phase 2: Content Validity

Intraclass Correlation Coefficient (95% Confidence Interval)

CVRa

Time 1 (Mean § SD)

Time 2 (Mean § SD)

How often do you check best-before dates? Rate your confidence in: Budgeting for groceries Meal planning Selecting vegetables and fruits Reading food labels Planning a meal using only foods already in your home Adjusting a recipe to make it healthier What percentage of the time do you: Use a grocery list? Buy a variety of vegetables? Totals for food selection and planningc How many times per week do you make the following home-prepared meals? Breakfast Lunch Dinner What percentage of the meals you prepare are balanced? Describe your ability to prepare mealse To what extent do you agree or disagree: Food preparation takes too much timef Food preparation is too much workj Making meals at home helps me to eat healthier Food preparation is enjoyable Rate your confidence in: Using knives in the kitchen Peeling, chopping, and slicing vegetables and fruits Using vegetables in food preparation Using beans and lentils in food preparation Preparing food from basic ingredients Following a simple recipe Boiling, steaming, or stewing

9/11

0.64

83.9 § 23.9

81.6 § 24.1

0.84 (0.78−0.88)

11/11 15/16 11/11 11/11 16/16

1.0 0.88 1.0 1.0 1.0

69.7 § 22.2 65.6 § 24.6 80.3 § 17.5 77.9 § 21.0 79.7 § 18.0

70.7 § 21.4 68.1 § 21.6 81.9 § 15.7 77.5 § 19.0 80.9 § 15.7

0.72 (0.61−0.80) 0.73 (0.62−0.81) 0.74 (0.63−0.82) 0.84 (0.77−0.89) 0.75 (0.64−0.82)

13/16

0.63

71.4 § 25.3

72.5 § 23.0

0.83 (0.76−0.88)

12/16 13/16

0.50 0.63

61.1 § 32.8 61.1 § 24.5 73.8 § 14.2

61.7 § 31.1 73.8 § 22.4 74.8 § 13.0

0.92 (0.88−0.94) 0.85 (0.79−0.89) 0.90 (0.86−0.93)

11/11 11/11 11/11 8/11

1.0 1.0 1.0 0.45d

56.8 § 38.4 52.6 § 33.0 51.9 § 33.4 64.9 § 23.7

60.5 § 35.8 54.4 § 30.4 55.6 § 31.7 66.6 § 22.6

0.90 (0.86−0.92) 0.83 (0.77−0.88) 0.88 (0.84−0.91) 0.82 (0.76−0.87)

15/16

0.88

87.0 § 23.7

85.7 § 24.2

0.85 (0.80−0.89)

14/16 14/16 15/16

0.75 0.75 0.88

44.5 § 29.7 53.4 § 29.9 90.5 § 14.2

43.0 § 28.6 50.7 § 29.3 90.1 § 15.2

0.84 (0.78−0.88) 0.83 (0.77−0.88) 0.67 (0.58−0.77)

12/16

0.50

69.1 § 25.7

69.5 § 24.3

0.88 (0.84−0.91)

16/16 16/16

1.0 1.0

82.3 § 17.3 82.4 § 18.3

82.1 § 15.5 82.2 § 15.5

0.82 (0.75−0.88) 0.77 (0.67−0.84)

10/11 8/11 14/16 15/16 11/11

0.82 0.45h 0.75 0.88 1.0

85.1 § 16.9 65.6 § 27.8 81.2 § 17.8 88.6 § 16.7 85.1 § 18.2

84.4 § 14.6 64.0 § 25.8 80.9 § 16.0 88.9 § 15.3 85.4 § 14.0

0.69 (0.56−0.78) 0.86 (0.80−0.90) 0.84 (0.78−0.89) 0.80 (0.72−0.86) 0.75 (0.65−0.83)

Cronbach ab .77

(continued)

ARTICLE IN PRESS

Ne/N

Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019

Question

Phase 4: Inter-Item Reliability

Kennedy et al

Phase 4: Test-Retest Reliability

Phase 4: Test-Retest Reliability Phase 2: Content Validity Ne/N

CVRa

Stir-frying or pan-frying Baking, grilling, or roasting Choosing a spice or herb Preparing new foods or recipes Totals for food preparationg How often do you: Wash countertops before preparing food? Wash your hands before preparing food? Use microwave/fridge/cold water to thaw frozen meat? Separate raw meat, poultry, and seafood? Cook foods to the correct internal temperature? Wash fruits and vegetables? Put leftovers in the refrigerator within 2 h? Follow instructions for storage on packaged foods? Check that food is heated throughout when reheating? Rate your confidence in: Staying safe in the kitchen (eg, avoiding burns and cuts) Totals for food safety and storageh Total food skills scorei

11/11 11/11 10/16 11/16

1.0 1.0 0.25h 0.38h

Time 1 (Mean § SD)

Time 2 (Mean § SD)

84.2 § 18.4 80.7 § 19.4 67.9 § 25.2 67.6 § 24.7 72.1 § 13.6

84.8 § 15.9 78.8 § 19.2 69.2 § 21.9 73.0 § 19.6 72.5 § 12.3

0.79 (0.71−0.85) 0.83 (0.76−0.88) 0.78 (0.69−0.84) 0.75 (0.63−0.82) 0.93 (0.90−0.95)

16/16 16/16 10/11

1.0 1.0 0.82

65.8 § 30.8 84.8 § 22.2 77.0 § 27.4

68.5 § 28.9 84.2 § 21.2 76.0 § 26.1

0.79 (0.71−0.85) 0.81 (0.74−0.86) 0.84 (0.77−0.88)

16/16 15/16

1.0 0.88

89.2 § 18.5 75.7 § 24.1

90.3 § 18.4 78.1 § 24.9

0.75 (0.65−0.82) 0.74 (0.64−0.81)

12/16 16/16 12/16

0.50 1.0 0.50

84.7 § 221.0 89.8 § 16.6 82.0 § 24.8

83.3 § 21.9 86.6 § 17.5 80.6 § 21.6

0.86 (0.81−0.90) 0.69 (0.57−0.77) 0.75 (0.66−0.82)

13/15

0.73

74.3 § 27.1

75.3 § 26.2

0.75 (0.65−0.81)

10/11

0.82

83.1 § 17.8

83.5 § 16.1

0.79 (0.70−0.85)

0.80

80.7 § 12.1 75.7 § 10.3

80.4 § 12.7 76.1 § 9.8

0.86 (0.81−0.90) 0.92 (0.88−0.94)

Phase 4: Inter-Item Reliability Cronbach ab

.88

.67 .90

Kennedy et al

CVR indicates content validity ratio; N, number of experts who provided feedback; Ne, number of experts who reported the question as essential. Notes: The wording of some questions was modified for ease of reporting. For N, samples sizes decreased to 11 for questions that were reassessed. a The CVR depends on the number of experts providing feedback on a question. Minimum CVRs for question retention were: 11 experts = 0.59; >11 experts = 0.49; bCronbach a coefficient was calculated at time 1; cParticipant scores were included in the domain mean if responses were given to ≥5 of 9 domain questions; dRetained in questionnaire owing to expert qualitative feedback, support from literature, and Cronbach a values; eResponse categories included: (1) I have no food preparation ability (eg, heat pre-prepared foods), (2) I have some food preparation ability (eg, make sandwiches, salads, or scrambled eggs), (3) I can use a combination of pre-prepared ingredients and basic ingredients to prepare homemade meals (eg, use pre-prepared rotisserie chicken in a home-made casserole), and (4) I can prepare meals from basic ingredients (eg, make a chicken and vegetable stir-fry with rice). Responses were reweighted out of 100 for consistency with other questions; fQuestions were reverse coded; gParticipant scores were included in the domain mean if responses were given to ≥10 of 20 domain questions; hParticipant scores were included in the domain mean if responses were given to ≥5 of 10 domain questions; iFinal sample size for time 1 = 187 and time 2 = 126.

ARTICLE IN PRESS

Question

Intraclass Correlation Coefficient (95% Confidence Interval)

Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019

Table 1. (Continued)

5

ARTICLE IN PRESS 6

Kennedy et al

Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019

Table 2. Qualitative Feedback From Expert Panel Justifying Retained Questions With Low Content Validity Ratios Questions Retained/Modified

Expert Feedback

What percentage of the meals you prepare are balanced meals? A balanced meal would include vegetables and fruits, grain products, and meat/ alternatives (eg, beans or nuts) or dairy/alternatives (eg, soy milk).

“Maybe a better way to ask about balanced meals . . . ‘How often do you cook a meal that includes vegetables, meat or protein, and a grain product?’” “There should be questions about menu planning (eg, ‘What percentage of your meals includes all the food groups?’).”

Please rate your confidence in:

“Specific questions about fruit and meat, eggs, legumes might be useful.”

Using beans and lentils in food preparation Choosing a spice or herb that goes well with food you are cooking and eating

Preparing new foods and recipes

completed the questionnaire again. Participants were allowed to skip questions, which resulted in varying response rates per question. The lowest number of participants answering the same questions at times 1 and 2 was 126. Inter-item reliability. The Cronbach a coefficient was calculated at time 1. Coefficients were .77, .88, and .67 for Food Selection and Planning, Food Preparation, and Food Safety and Storage, respectively. Cronbach a for the overall questionnaire was .90 (n = 148, because responses to all 39 questions were required for the analyses). Test-retest reliability. Respondents who completed at least half of the questions within a domain were included in the analyses (n = 126). For each question, ICC coefficients ranged from 0.67 to 0.92 and mean domain scores were 0.90, 0.93, and 0.86 for Selection and Planning, Preparation, and Food Safety and Storage, respectively (Table 1). Overall, there was substantial agreement between total food skills at times 1 and 2 (ICC = 0.92).

DISCUSSION This questionnaire has strong content and face validity, good interitem reliability, and substantial testretest reliability. It is a valid and

“Perhaps a question could be ‘Knowing or researching a spice or herb that goes well with food I am cooking/ eating.’” “I suggest a question on seasoning food (use of herbs and spices).” “Include a question on how they decide what they will prepare for their meals (eg, available ingredients, new recipes, etc).”

reliable tool that can be used to evaluate basic to intermediate food skills. The Lawshe method30 is considered 1 of the most accepted tools to establish content validity. Although a panel of 5−10 experts is considered sufficient,31 this study exceeded this expectation. Inter-item reliability showed that individual questions in this questionnaire capture the construct of food skills overall because the multiple questions highly correlate with each other.29 Face validity adds further confidence that target populations (ie, those with basic to intermediate food skills) will find the questionnaire to be acceptable and understandable. The high ICC scores, which demonstrated good test-retest reliability, showed that this tool will produce repeated food skill domain measurements that will be consistent for a given individual.29 Lawshe CVRs of 1.0 indicated unanimous consensus among experts that 16 questions were essential and 19 achieved the minimum or higher CVR to be retained. The CVRs were low for 4 retained questions, which suggested that they were not essential; however, these questions were deemed appropriate because qualitative expert feedback indicated that these were components of food skills and significant to health. Furthermore, justification for including these questions is supported by literature, because beans and lentils are nutrient-dense plant sources of

protein,38 and, for many, a major behavioral barrier to incorporating these pulses in one’s diet is the lack of preparation skills and lack of familiarity with these nutritious food items.39 Similarly, retaining preparing balanced meals was important because this spoke not only to the nutritional quality of their diets, but also to their ability to conceptualize and plan a meal that included foods from multiple food groups.1 In addition, individuals who are confident in their ability to prepare new foods and recipes are more likely to prepare a wider variety of foods and to experiment to increase their repertoire of dishes;40 therefore, these skills could translate into better diet quality.7 Finally, meals incorporating spices and herbs may be lower in fat and sodium while remaining flavorful;41 therefore, these food skills are important to health and may provide motivation to prepare meals at home. Retention of these questions led to a good Cronbach a of .88, which indicated that responses to these questions correlated well with responses to other questions within the Food Preparation domain, contributing to good inter-item reliability. In the final version of the questionnaire, the Food Preparation Domain contained twice the number of questions as Food Planning and Selection or Food Safety and Storage, because it included basic to intermediate food preparation skills, as well as attitudes

ARTICLE IN PRESS Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019 toward and frequency of preparing food. The Food Safety and Storage Domain had a lower inter-item reliability compared with other domains. This could be because of the variety of skills that constitute food safety, some of which may be more advanced but are essential (eg, cooking food to the correct internal temperature). Despite the lower inter-item reliability for the Food Safety and Storage Domain, the overall questionnaire had an excellent interitem reliability of 0.90. Food agency has been identified as an important factor in empowering an individual to cook.42 Many factors such as physical, social, and economic environments26 can be facilitators or barriers to developing and using one’s food skills. For example, a lack of adequate kitchen facilities or equipment may make someone less likely to prepare meals, but not because he or she lacks food skills. Therefore, some of these factors were included in the supplemental questions, providing context or subgroups for additional analyses. When the questionnaire is used in different populations (eg, homeless individuals, seniors, or people with diabetes), it will necessitate asking supplemental population-specific questions. The strength of this study lies in its rigorous examination of validity and reliability. Input from a range of experts, in addition to a high Lawshe CVI (0.80), builds confidence that this questionnaire accurately and reliably assesses basic to intermediate food skills. Assessment of criterion validity was not feasible because there are no reference standards for measuring food skills.29 The degree of association between self-reported and actual food skills is unknown, but attitudes toward and confidence in cooking and household cooking frequency were shown to predict food preparation, diet quality, and chronic disease risk.18,28 Thus, self-reporting is a meaningful lens through which to measure food skills. In all study phases, there were more female participants than males. This is likely because females tend to be more interested in food and nutrition, which aligns with gender biases in cooking.43 Face validity and

reliability were evaluated in postsecondary students, who are a culturally and economically diverse population of young adults, many of whom have limited years of experience in home food preparation, and who represent a segment of the population with basic to intermediate food skills. Nevertheless, the response rate of 9% may limit generalizability.

IMPLICATIONS FOR RESEARCH AND PRACTICE This food skills questionnaire demonstrates strong content and face validity, strong test-retest reliability, and good inter-item reliability. It is an appropriate instrument for evaluating basic to intermediate food skills in research or practice-based settings. Nutrition education interventionscould focus more on food skills, and although they may not address the full range of food skills, this questionnaire would be able to capture changes in specific food skills, domains, or overall food skills from planning to food safety. Ultimately, higher food skills in the population could result in increased intakes of vegetables and fruits1,8,9 and lower intakes of calories, fat, sugar,2 and fast food,1,2,7−9 potentially reducing the risk for chronic diseases.10

ACKNOWLEDGMENTS Funding in support of this research was provided by an internal grant from Brescia University College.

REFERENCES 1. Hartmann C, Dohle S, Siegrist M. Importance of cooking skills for balanced food choices. Appetite. 2013;65:125-131. 2. Wolfson JA, Bleich SN. Is cooking at home associated with better diet quality or weight-loss intention? Public Health Nutr. 2015;18:1397-1406. 3. Ontario Ministry of Health Promotion. Healthy Eating, Physical Activity, and Healthy Weights: Guidance Document. Ontario, Canada: Queens Printer for Ontario; 2010. 4. Sumner J. Food literacy and adult education: learning to read the world by eating. Can J Study Adult Educ. 2013; 25:79-92.

Kennedy et al

7

5. Cullen T, Hatch J, Martin W, Higgins JW, Sheppard R. Food literacy: definition and framework for action. Can J Diet Pract Res. 2015;76:140-145. 6. Greaney ML, Less FD, White AA, et al. College students’ barriers and enablers for healthful weight management: a qualitative study. J Nutr Educ Behav. 2009;41:281-286. 7. Murray DW, Mahadevan M, Gatto K, et al. Culinary efficacy: an exploratory study of skills, confidence, and healthy cooking competencies among university students. Perspect Public Health. 2015;136:143-151. 8. Thorpe MG, Kestin M, Riddell LJ, Keast RS, McNaughton SA. Diet quality in young adults and its association with food-related behaviours. Public Health Nutr. 2014;17:1767-1775. 9. Utter J, Larson N, Laska M, Winkler M, Neumark-Sztainer D. Self-perceived cooking skills in emerging adulthood predict better dietary behaviours and intake 10 years later: a longitudinal study. J Nutr Educ Behav. 2018;50:494-500. 10. World Health Organization. Diet, Nutrition and the Prevention of Chronic Diseases: Report of a Joint WHO/FAO Expert Consultation (2002: Geneva, Switzerland). Geneva, Switzerland: World Health Organization; 2003. 11. Clifford D, Anderson J, Auld G, Champ J. Good grubbin’: impact of a TV cooking show for college students living off campus. J Nutr Educ Behav. 2009; 41:194-200. 12. Wrieden WL, Anderson AS, Longbottom PJ, et al. The impact of a communitybased food skills intervention on cooking confidence, food preparation methods and dietary choices—an exploratory trial. Public Health Nutr. 2007;10:203-211. 13. Levy J, Auld G. Cooking classes outperform cooking demonstrations for college sophomores. J Nutr Educ Behav. 2004;36:197-203. 14. Reicks M, Trofholz AC, Stang JS, Laska MN. Impact of cooking and home food preparation interventions among adults: outcomes and implications for future programs. J Nutr Educ Behav. 2014;46: 259-276. 15. Kelly NR, Mazzeo SE, Bean MK. Systematic review of dietary interventions with college students: directions for future research and practice. J Nutr Educ Behav. 2013;45:304-313. 16. Short F. Domestic cooking skills—what are they? J Home Econ Inst Aust. 2003; 10:13-22.

ARTICLE IN PRESS 8

Kennedy et al

17. Jones SA, Walter J, Soliah L, Phifer JT. Perceived motivators to home food preparation: focus group findings. J Acad Nutr Diet. 2014;114:1552-1556. 18. Winkler E, Turrell G. Confidence to cook vegetables and the buying habits of Australian households. J Am Diet Assoc. 2009;109:1759-1768. 19. Medeiros LC, Hillers VN, Chen G, Bergmann V, Kendall P, Schroeder M. Design and development of food safety knowledge and attitude scales for consumer food safety education. J Am Diet Assoc. 2004;104:1671-1677. 20. Condrasky MD, Williams JE, Catalano PM, Griffin SF. Development of psychosocial scales for evaluating the impact of a culinary nutrition education program on cooking and healthful eating. J Nutr Educ Behav. 2011;43:511516. 21. Lavelle F, McGowan L, Hollywood L, et al. The development and validation of measures to assess cooking skills and food skills. Int J Behav Nutr Phys Act. 2017;14:118-130. 22. Krause CG, Beer-Borst S, Sommerhalder K, Hayoz S, Abel T. A short food literacy questionnaire (SFLQ) for adults: findings from a Swiss validation study. Appetite. 2018;120:275-280. 23. Anderson AS, Bell A, Adamson A, Moynihan P. A questionnaire assessment of nutrition knowledge—validity and reliability issues. Public Health Nutr. 2002;5:497-503. 24. Barton KL, Wrieden WL, Anderson AS. Validity and reliability of a short questionnaire for assessing the impact of cooking skills interventions: questionnaire assessing cooking interventions. J Hum Nutr Diet. 2011;24:588-595.

Journal of Nutrition Education and Behavior  Volume 000, Number 000, 2019 25. Matthews JI, Doerr L, Dworatzek PDN. University students intend to eat better but lack coping self-efficacy and knowledge of dietary recommendations. J Nutr Educ Behav. 2016;48:12-19. 26. Wilson CK, Matthews JI, Seabrook JA, Dworatzek PDN. Self-reported food skills of university students. Appetite. 2017;108:270-276. 27. Kohn D, Hammingh J, Bellman J, Armstrong A. How to Do a Food Demonstration. Toronto, Canada: Community Food Advisor (CFA) Food Skills Questionnaire. Community Food Advisor and the Nutrition Resource Centre; 2010. 28. Engler-Stringer R. Food, cooking skills, and health: a literature review. Can J Diet Pract Res. 2010;71:141-145. 29. Gleason PM, Harris J, Sheean PM, Boushey CJ, Bruemmer B. Publishing nutrition research: validity, reliability, and diagnostic test assessment in nutrition-related research. J Am Diet Assoc. 2010;110:409-419. 30. Ayre C, Scally AJ. Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Meas Eval Couns Dev. 2014;47:79-86. 31. Gilbert GE, Prion S. Making sense of methods and measurement: Lawshe’s content validity index. Clin Simul Nurs. 2016;12:530-531. 32. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28: 563-575. 33. George D, Mallery P. SPSS for Windows:Step by Step Guide: A Simple Guide and Reference. 4th ed. Boston, MA: Allyn & Bacon; 2003. 34. Courtney SM, Majowicz SE, Dubin JA. Food safety knowledge of undergraduate students at a Canadian university: results

35.

36.

37.

38.

39.

40.

41.

42.

43.

of an online survey. BMC Public Health. 2016;16:1147-1162. Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to Their Development and Use. New York, NY: Oxford University Press; 2008. Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005;19:231-240. Shrout PE. Measurement reliability and agreement in psychiatry. Stat Methods Med Res. 1998;7:301-317. Mudryj AN, Yu N, Aukema HM. Nutritional and health benefits of pulses. Appl Physiol Nutr Metab. 2014; 39:1197-1204. Karlsen MC, Ellmore GS, McKeown N. Seeds—health benefits, barriers to incorporation, and strategies for practitioners in supporting consumption among consumers. Nutr Today. 2016;51:50-59. Stead M, Caraher M, Wrieden W, Longbottom P, Valentine K, Anderson A. Confident, fearful and hopeless cooks: findings from the development of a food skills initiative. Br Food J. 2004;106:274-287. Ghawi SK, Rowland I, Methven L. Enhancing consumer liking of low salt tomato soup over repeated exposure by herb and spice seasonings. Appetite. 2014;81:20-29. Trubek AB, Carabello M, Morgan C, Lahne J. Empowered to cook: the crucial role of “food agency” in making meals. Appetite. 2017;116:297-305. Daniels S, Glorieux I, Minnen J, van Tienoven TP. More than preparing a meal? Concerning the meanings of home cooking. Appetite. 2012;58:10501056.