- Email: [email protected]
Measuring children's self-reported sport participation, risk perception and injury history: Development and validation of a survey instrument
Available online at www.sciencedirect.com
Journal of Science and Medicine in Sport 14 (2011) 22–26
Original paper
Measuring children’s self-reported sport participation, risk perception and injury history: Development and validation of a survey instrument Emma J. Siesmaa, Jennifer D. Blitvich, Peta E. White, Caroline F. Finch ∗ School of Human Movement and Sport Sciences, University of Ballarat, Australia Received 26 November 2009; received in revised form 16 March 2010; accepted 5 April 2010
Abstract Despite the health benefits associated with children’s sport participation, the occurrence of injury in this context is common. The extent to which sport injuries impact children’s ongoing involvement in sport is largely unknown. Surveys have been shown to be useful for collecting children’s injury and sport participation data; however, there are currently no published instruments which investigate the impact of injury on children’s sport participation. This study describes the processes undertaken to assess the validity of two survey instruments for collecting self-reported information about child cricket and netball related participation, injury history and injury risk perceptions, as well as the reliability of the cricket-specific version. Face and content validity were assessed through expert feedback from primary and secondary level teachers and from representatives of peak sporting bodies for cricket and netball. Test–retest reliability was measured using a sample of 59 child cricketers who completed the survey on two occasions, 3–4 weeks apart. Based on expert feedback relating to face and content validity, modification and/or deletion of some survey items was undertaken. Survey items with low test–retest reliability (κ ≤ 0.40) were modified or deleted, items with moderate reliability (κ = 0.41–0.60) were modified slightly and items with higher reliability (κ ≥ 0.61) were retained, with some undergoing minor modifications. This is the first survey of its kind which has been successfully administered to cricketers aged 10–16 years to collect information about injury risk perceptions and intentions for continued sport participation. Implications for its generalisation to other child sport participants are discussed. © 2010 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved. Keywords: Sports injury; Risk perception; Children; Reliability; Validity; Survey
1. Introduction Child sport injury is a major public health concern in Australia and internationally.1–4 For many children, organised sport makes an important contribution to their overall physical activity levels,5,6 thereby enhancing their health and wellbeing. To ensure children’s long-term participation in sport, however, it is necessary to establish and address potential barriers towards continued participation. Several factors, such as wanting to do other things, lack of enjoyment and wanting to play other sports have been identified as influencers of children’s participation in sport, and have been linked to child sport dropout.7,8 Sports injury is another possible barrier to sport participation,9 but the extent to which it impacts children is not well understood. ∗
Corresponding author. E-mail address: [email protected] (C.F. Finch).
Whilst survey methods have been used to collect children’s self-reported data on injury and attitudes towards protective equipment (e.g. Refs. 10–12), there are no published survey instruments which collect data specifically about the impact of sport injuries and injury risk perceptions on children’s ongoing involvement in sport. Accordingly, the role of injury and injury risk perception on children’s sport choices remains unclear. This paper reports the development and validation of a survey instrument to collect information about children’s (aged 9–17 years) organised sport behaviours (including intention to continue or to withdraw), sport injury history and sport injury risk perceptions in two popular sports: cricket and netball. Two difficulties encountered when using survey measures with children are their capacity to recall such information without the data being susceptible to recall biases, and the possibility of children experiencing difficulties associated with low literacy skills.13 It is widely accepted that only chil-
1440-2440/$ – see front matter © 2010 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.jsams.2010.04.006
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
dren aged 10+ years can reliably and accurately self-report their physical activity behaviour.13,14 For these reasons, previous physical activity surveys among children have had mixed success.15–18 Furthermore, when studies based on surveys are published, they do not always report rigorous testing of validity and reliability, which is a vital aspect in the development of a survey instrument.13 The purpose of this study was to establish a valid and reliable survey instrument to investigate the role of recent sports injury history and sports injury risk perceptions on children’s sport participation and choices in two common sports in Australia. Cricket and netball are ranked among the 10 most popular sports by participation for children in Australia, with cricket being male dominant and netball being dominated by females.19 This paper outlines the development of two sports specific versions (cricket and netball) of the instrument and provides suggestions for its subsequent adaptation to a generic survey for use in other sports.
2. Method Following a detailed literature search and collation of previous questionnaires examining children’s sport/physical activity habits and injury risk perceptions, a survey was constructed to investigate key factors that potentially influence a child’s ongoing participation in sport. These factors were related to injury history/perceived injury risks, enjoyment, peer involvement, family influences, socioeconomic influences, skill/ability levels, school/work commitments, sport specific factors and the influence of role models. Where possible, survey items were direct replicates of questions successfully administered in Australian surveys used in previous child studies.2,18,20–26 Other questions were modified slightly to improve their relevance to this study’s goals, while some new questions were also developed. Several experts from the Human Movement field, some with previous direct experience in developing and validating surveys for use among children, assessed the draft survey and their input prompted some modifications to the initial survey. The initial survey was made specific to cricket or netball by including wording that linked the items to these sports, respectively. The two sports specific surveys were later adapted to become one generic version to allow administration of the survey in the wider child sport setting. To assess the survey’s face and content validity, a draft was sent to several experts, in addition to those in the Human Movement field, for feedback on its age specificity, readability and item content, and suggestions for any modifications, additions and deletions. Two secondary and two primary level school teachers commented specifically on the age appropriateness of the language, readability and formatting. Five sport experts representing Australian state and national bodies for cricket and netball provided feedback regarding the survey’s sport specific content and relevance to those sports. Quali-
23
tative feedback from all experts was collated and informed changes to the survey. A test–retest reliability study was conducted with a small subset of junior cricket participants who had been recruited in a larger, concurrent, cohort study of junior cricket.27 Fifty-nine cricket participants completed the survey on two occasions, within a 3–4-week interval. All but two participants were male and all were 10–16 years of age; all participated in organised after-school cricket in one of three selected age levels (12 years and under [U12], U14 and U16). The time period for the test–retest reliability assessment was based on convenience for the participants and sufficient time between the two survey administrations. Consistency between responses to the two survey administrations was compared using a Cohen’s Kappa (κ) statistic or Weighted Kappa (κw2 ) statistic for ordinal items, as appropriate. Kappa values were graded according to Landis and Koch.28 Findings from the validity and reliability assessments were summarised and a more generic version of the survey was developed for administration among children in the wider sports setting. The final version of the survey can be found in Supplement File 1. This research was approved by the University’s Human Research and Ethics Committee and informed consent was obtained from all participants.
3. Results The teacher experts who reviewed the survey for face and content validity raised some concerns about whether children aged less than 10 years would be capable of completing the survey considering its complexity and length. Survey readability was generally identified as being appropriate for the target age group, but more so for children aged 12+ years. In response, the survey size was reduced by deleting items that were considered to be non-essential when administered in a general sport setting. Some formatting changes were also made to decrease item complexity. A Flesch Kincaid Grade Level Test was conducted on the final survey, which indicated that the readability was appropriate for children aged 12+ years. The survey’s content in terms of its relevance to children’s cricket and netball was identified as appropriate by the peak sports bodies with very few suggestions for content alterations. Suggestions for some wording refinements and improvements were provided, and adjustment to the survey formatting and the incorporation of graphics to improve its layout were recommended and subsequently made. The injury history aspects of the survey were identified as requiring modification to improve their clarity. This was addressed by the inclusion of a clearer injury definition to facilitate children’s recall of relevant injury events. This injury definition was modelled on questions previously used to classify sport injury within other sport injury surveys.2,20,22,29 The particular definition adopted was
24
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
related to the loss of time from sport participation following the sport injury incident. Table 1 summarises the survey item modifications that were undertaken in response to the test–retest reliability assessments and the expert feedback. Typically, items with very low Kappa/Weighted Kappa values (κ ≤ 0.30 [poorslight]) were deleted, while other low scoring items (κ ≤ 0.40 [poor-fair]) underwent more modification than those with moderate reliability (κ = 0.41–0.60 [moderate]). Items with higher values (κ ≥ 0.61 [substantial-almost perfect]) required only minor modification or none at all. Supplement File 2 provides the specific reliability scores for all survey items in the cricket-specific survey. Questions with high reliability (κ ≥ 0.61) related to demographics (age, gender, family structure); sports played over the last year; and sports in which children expected to withdraw. Items relating to reasons for discontinued participation were also generally associated with high reliability scores (κ ≥ 0.61), or could not be computed due to a lack of variability among children’s responses. Questions with a large number of possible response options, especially items regarding reasons for participation, typically demonstrated moderate reliability (κ = 0.41–0.60). Questions with low reliability (κ ≤ 0.40) included some of the injury history and risk perception items. A final version of the survey was generated by converting the two sport specific (cricket and netball) surveys into one generic form, taking into account all the validity and reliability assessments.
4. Discussion This study has verified the face and content validity of two sport specific versions of a survey instrument (for cricket or netball), and determined the test–retest reliability of the cricket-specific version. Both versions of the survey were similar and designed to collect data pertaining to self-reported sport participation, recent injury history and injury risk perception, among children aged 9–16 years. Self-reported information from child sport participants is useful, but like all survey data, can be subject to biases.13,14 It is therefore important when using this type of instrument to ensure that it collects accurate and reliable data. Validity was determined through review of expert feedback from sporting representatives and teachers who identified some areas where improvements to wording and format were required. Overall, the survey’s readability was considered appropriate for the target age group, although for children under the age of 12 years, and/or children who experience difficulty with reading and writing, assistance from an adult may be required when completing the survey. The test–retest reliability of the cricket-specific survey prompted other modifications. Some items with a large number of response options, such as those regarding reasons why children participated in sport, typically demonstrated only
moderate reliability. To address this, the number of total response options was reduced by removing those considered non-essential for survey administration in a general sport setting. Conversely, the multi-response items which related to reasons why children discontinued sport participation, had high reliability. Only two children were required to respond to the sports discontinuation items as these children were the only ones who reported that they planned to cease sport participation. Because so few children planned to discontinue their sport participation, it was not possible to compute a reliability score for some of these items, as there was no variability in responses for the two children on each occasion. Reliability assessments of injury history and injury risk perception items had low κ scores. This issue was also emphasised by the expert feedback providers during the face and content validity phase, when the possibility of memory influencing injury recall for some children was first highlighted. As a result, a clearer definition of injury, modelled on previous studies,2,20,22,29 was included in the final version to facilitate children’s recall of injury events, viz.: ‘Sport injury is an incident during a sport game or training session, where you hurt yourself and had to miss a game or training session(s) afterwards [e.g. sprain, graze, fracture, bruise, etc.]’. As the majority of children participating in the test–retest reliability assessment were male, no gender comparisons could be made. Therefore, it is unknown whether gender influenced the reliability of the survey; however, the validity assessment found that the content of both the cricket (97% male respondents) and netball (100% female) specific surveys, was appropriate with only minor modifications suggested. In the initial cricket survey, many items were targeted to this sport for specific testing with a sample of junior cricket participants. Where necessary, items retained after assessment were modified slightly to become generic, extending the survey’s relevance to participants of other formal sports. As only minor modifications to wording were made to convert the cricket-specific survey to the generic form, it was assumed that all items found to be valid for the cricket-specific survey would remain valid once converted. For example, ‘How many matches did you play in the past week for cricket?’ was modified to, ‘How many times did you play sport last week?’. Despite the final survey containing generic items to enable its administration in different sports settings, it is unreasonable to automatically assume that this will ensure the successful administration in every childhood sport. However, as the modifications were guided by both the feedback from the expert panel and the reliability assessment, it is likely that the changes undertaken would facilitate children’s completion of the survey. The survey was only validated in cricket and netball and the reliability tested only in cricket, hence the survey’s ability to collect valid and reliable data in the wider context of other sports cannot be assumed. Surveys for sport settings that differ noticeably from these will require further modification and it is recommended that validation be confirmed in this case.
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
25
Table 1 Summary of survey item modifications based on expert feedback and results of the test–retest reliability assessment. Type of item
Kappa valuesa κ = ≤0.40 (low reliability) Moderate to substantial item modification
Demographics e.g., How old are you now? Cricket/netball participation e.g., How many matches/training sessions played in the past week? Reasons for participating e.g., Why do you play cricket?
Other sport participation e.g., Apart from cricket, have you played any other sports over the past year? Reasons for discontinuing sport e.g., Why will you stop playing these sports next year? Injury history e.g., Did you get sore, hurt or injured playing cricket last season?
Injury risk perception e.g., How much chance of getting injured do you think you have when playing cricket? a
κ = 0.41–0.60 (moderate reliability) κ = 0.61–1.00 (high reliability) Minor to moderate item modification Minor item modification or items unchanged All items remained unchanged.
Format changed to improve readability and graphic layout.
Modification to wording to enhance clarity and to become general to sport. Number of response options reduced from 20 choices to 13 choices. Modification to become general to sport. Minor modification to wording to enable survey to become general to sport. Number of response options reduced from 19 choices to 13 choices.
Wording modification to improve clarity and to link items to the newly included injury definition. Some of the items that scored κ ≤ 0.30 were considered non-essential in a general sport setting and were deleted. Wording made general to sport.
Inclusion of clear injury definition and some modification to wording to improve clarity and to become general to sport.
Wording modified to improve clarity and to become general to sport.
Either Kappa or Weighted Kappa, as appropriate.
The nature of children’s sports varies markedly, particularly across nations. Unlike cricket and netball, competition and/or training sessions for some sports may not be scheduled on a weekly basis. Further, although cricket and netball are popular sports among Australian children, they are not, for example, typical sports that are played throughout North America and in many parts of Europe. Sports of interest in these countries may be quite different to cricket and/or netball and may not correspond completely to the items included in the final generic survey. It should be noted that in the cricket and netball-specific surveys, question items could be directed purposefully to children’s participation in these sports. However, after the survey was converted from sport specific to a generic format, questions could no longer focus on one sport alone. To ensure that this did not cause confusion for children who played more than one sport, the generic survey instructions explain that in this circumstance, children should base their responses on the sport that they play most often. This method has been adopted for other sport injury surveys.2,10 In summary, there have been no published studies investigating the influence of sport injury and injury risk perception on children’s continued sport participation. It is likely that the lack of a reliable and valid survey tool has contributed to
this. This paper verifies the validity of netball and cricketspecific instruments and determines the reliability of the cricket-specific survey tool to enable these factors to be explored among children involved in these sports. This study has developed and validated the first survey instrument of its kind that can be used to collect data of this nature in these two sports and also offers a generic format which could be applied to other organised sport settings in future studies.
5. Practical implications • Valid and reliable information about the role of injury and injury risk perception on children’s ongoing participation in, and choices of, sport can be obtained through a selfreport survey. • The survey can be successfully administered to children aged 9–16 years participating in cricket; however, children under the age of 12 years may require greater assistance from an adult than children above this age, to complete the survey. • Application of the survey to other sports settings is recommended, although further evaluation of the instrument in new settings, outside netball and cricket, may be required.
26
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
Acknowledgements This research was supported by research grants from the Department of Planning and Community Development - Sport and Recreation Victoria Division and the Injury Prevention Community Grants Program of the Australian Government Department of Health and Ageing. Emma Siesmaa is supported by an Australian Postgraduate Award. Caroline Finch is supported by a National Health and Medical Research Council Principal Research Fellowship. The authors would also like to thank Associate Professor Amanda Telford for her input in the survey development process.
Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/ j.jsams.2010.04.006.
References 1. Conn JM, Annest JL, Gilchrist J. Sports and recreation related injury episodes in the US population, 1997–99. Inj Prev 2003;9:117–23. 2. Finch C, Cassell E. The public health impact of injury during sport and active recreation. J Sci Med Sport 2006;9:490–7. 3. Nicholl JP, Coleman P, Williams BT. The epidemiology of sports and exercise related injury in the United Kingdom. Br J Sports Med 1995;29(4):232–8. 4. Caine D, Maffulli N, Caine C. Epidemiology of injury in child and adolescent sports: injury rates, risk factors, and prevention. Clin J Sport Med 2008;27:19–50. 5. Hoffman JR, Kang J, Faigenbaum AD, et al. Recreational sports participation is associated with enhanced physical fitness in children. Res Sports Med 2005;13:149–61. 6. Olds T, Dollman J, Maher C. Adolescent sport in Australia: who, when, where and what? ACHPER Healthy Lifestyles J 2009;56(1):11–6. 7. Gould D, Feltz D, Horn T, et al. Reasons for attrition in competitive youth swimming. J Sport Behav 1982;5(3):155–65. 8. Butcher J, Lindner KJ, Johns DP. Withdrawal from competitive youth sport: a retrospective ten-year study. J Sport Behav 2002;25(2):145–63. 9. Finch C, Owen N, Price R. Current injury or disability as a barrier to being more physically active. Med Sci Sports Exerc 2001;33(5):778–82. 10. Emery CA, Meeuwisse WH, McAllister JR. Survey of sport participation and sport injury in Calgary and area high schools. Clin J Sport Med 2006;16:20–6. 11. Williams JM, Wright P, Currie CE, et al. Sports related injuries in Scottish adolescents aged 11–15. Br J Sports Med 1998;32:291–6.
12. Finch CF, McIntosh AS, McCrory P. What do under 15 year old schoolboy rugby union players think about protective headgear? Br J Sports Med 2001;35:89–94. 13. Dollman J, Okely AD, Hardy L, et al. A hitchhiker’s guide to assessing young people’s physical activity: deciding what method to use. J Sci Med Sport 2009;12:518–25. 14. Trost SG. State of the art reviews: measurement of physical activity in children and adolescents. Am J Lifestyle Med 2007;1(4):299–314. 15. Booth ML, Okely AD, Bauman A. The reliability and validity of the physical activity questions in the WHO health behaviour in schoolchildren (HBSC) survey: a population study. Br J Sports Med 2001;35:263–7. 16. Chen X, Sekine M, Hamanishi S, et al. Validation of a selfreported physical activity questionnaire for schoolchildren. J Epidemiol 2003;13(5):278–87. 17. Sallis JF, Buono MJ, Roby RR, et al. Seven-day recall and other physical activity self-reports in children and adolescents. Med Sci Sports Exerc 1993;25(1):99–108. 18. Telford A, Salmon J, Jolley D, et al. Reliability and validity of questionnaire-based measures for younger and older children’s physical activity: the Children’s Leisure Activities Study Surveys (CLASS). Pediatr Exerc Sci 2004;16:64–78. 19. Australian Bureau of Statistics. Children’s participation in cultural and leisure activities, Australia; April 2006. 20. Northern Sydney Area Health Service. New South Wales Youth Sports Injury Report; July 1997. 21. Centre for Epidemiology and Research, NSW Department of Health. The health behaviours of secondary school students in New South Wales 2002. N S W Public Health Bull 2004;15(S-2). 22. Finch C, Donohue S, Garnham A. Safety attitudes and beliefs of junior Australian football players. Inj Prev 2002;8:151–4. 23. Salmon J, Ball K, Crawford D, et al. Reducing sedentary behaviour and increasing physical activity among 10-year-old children: overview and process evaluation of the ‘Switch-Play’ intervention. Health Promot Int 2005;20(1):7–17. 24. Booth M, Macaskill P, McLellan L, et al. NSW schools fitness and physical activity survey, 1997. Sydney: NSW Dept of School Education; 1997. 25. Finch C, Donohue S, Garnham A, et al. The playing habits and other commitments of elite junior Australian football players. J Sci Med Sport 2002;5(3):266–73. 26. Telford R, Bass S, Budge M, et al. The lifestyle of our kids (LOOK) project: outline of methods. J Sci Med Sport 2009;12(1):156– 63. 27. Finch CF, White P, Dennis R, et al. Fielders and batters are injured too: a prospective cohort study of injuries in junior club cricket. J Sci Med Sport; Corrected Proof, available online 22 December 2009. doi:10.1016/j.jsams.2009.10.489. 28. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. 29. Finch C, McIntosh A, McCrory P, et al. A pilot study of the attitudes of Australian rules footballers towards protective headgear. J Sci Med Sport 2003;6(4):505–11.
Journal of Science and Medicine in Sport 14 (2011) 22–26
Original paper
Measuring children’s self-reported sport participation, risk perception and injury history: Development and validation of a survey instrument Emma J. Siesmaa, Jennifer D. Blitvich, Peta E. White, Caroline F. Finch ∗ School of Human Movement and Sport Sciences, University of Ballarat, Australia Received 26 November 2009; received in revised form 16 March 2010; accepted 5 April 2010
Abstract Despite the health benefits associated with children’s sport participation, the occurrence of injury in this context is common. The extent to which sport injuries impact children’s ongoing involvement in sport is largely unknown. Surveys have been shown to be useful for collecting children’s injury and sport participation data; however, there are currently no published instruments which investigate the impact of injury on children’s sport participation. This study describes the processes undertaken to assess the validity of two survey instruments for collecting self-reported information about child cricket and netball related participation, injury history and injury risk perceptions, as well as the reliability of the cricket-specific version. Face and content validity were assessed through expert feedback from primary and secondary level teachers and from representatives of peak sporting bodies for cricket and netball. Test–retest reliability was measured using a sample of 59 child cricketers who completed the survey on two occasions, 3–4 weeks apart. Based on expert feedback relating to face and content validity, modification and/or deletion of some survey items was undertaken. Survey items with low test–retest reliability (κ ≤ 0.40) were modified or deleted, items with moderate reliability (κ = 0.41–0.60) were modified slightly and items with higher reliability (κ ≥ 0.61) were retained, with some undergoing minor modifications. This is the first survey of its kind which has been successfully administered to cricketers aged 10–16 years to collect information about injury risk perceptions and intentions for continued sport participation. Implications for its generalisation to other child sport participants are discussed. © 2010 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved. Keywords: Sports injury; Risk perception; Children; Reliability; Validity; Survey
1. Introduction Child sport injury is a major public health concern in Australia and internationally.1–4 For many children, organised sport makes an important contribution to their overall physical activity levels,5,6 thereby enhancing their health and wellbeing. To ensure children’s long-term participation in sport, however, it is necessary to establish and address potential barriers towards continued participation. Several factors, such as wanting to do other things, lack of enjoyment and wanting to play other sports have been identified as influencers of children’s participation in sport, and have been linked to child sport dropout.7,8 Sports injury is another possible barrier to sport participation,9 but the extent to which it impacts children is not well understood. ∗
Corresponding author. E-mail address: [email protected] (C.F. Finch).
Whilst survey methods have been used to collect children’s self-reported data on injury and attitudes towards protective equipment (e.g. Refs. 10–12), there are no published survey instruments which collect data specifically about the impact of sport injuries and injury risk perceptions on children’s ongoing involvement in sport. Accordingly, the role of injury and injury risk perception on children’s sport choices remains unclear. This paper reports the development and validation of a survey instrument to collect information about children’s (aged 9–17 years) organised sport behaviours (including intention to continue or to withdraw), sport injury history and sport injury risk perceptions in two popular sports: cricket and netball. Two difficulties encountered when using survey measures with children are their capacity to recall such information without the data being susceptible to recall biases, and the possibility of children experiencing difficulties associated with low literacy skills.13 It is widely accepted that only chil-
1440-2440/$ – see front matter © 2010 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.jsams.2010.04.006
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
dren aged 10+ years can reliably and accurately self-report their physical activity behaviour.13,14 For these reasons, previous physical activity surveys among children have had mixed success.15–18 Furthermore, when studies based on surveys are published, they do not always report rigorous testing of validity and reliability, which is a vital aspect in the development of a survey instrument.13 The purpose of this study was to establish a valid and reliable survey instrument to investigate the role of recent sports injury history and sports injury risk perceptions on children’s sport participation and choices in two common sports in Australia. Cricket and netball are ranked among the 10 most popular sports by participation for children in Australia, with cricket being male dominant and netball being dominated by females.19 This paper outlines the development of two sports specific versions (cricket and netball) of the instrument and provides suggestions for its subsequent adaptation to a generic survey for use in other sports.
2. Method Following a detailed literature search and collation of previous questionnaires examining children’s sport/physical activity habits and injury risk perceptions, a survey was constructed to investigate key factors that potentially influence a child’s ongoing participation in sport. These factors were related to injury history/perceived injury risks, enjoyment, peer involvement, family influences, socioeconomic influences, skill/ability levels, school/work commitments, sport specific factors and the influence of role models. Where possible, survey items were direct replicates of questions successfully administered in Australian surveys used in previous child studies.2,18,20–26 Other questions were modified slightly to improve their relevance to this study’s goals, while some new questions were also developed. Several experts from the Human Movement field, some with previous direct experience in developing and validating surveys for use among children, assessed the draft survey and their input prompted some modifications to the initial survey. The initial survey was made specific to cricket or netball by including wording that linked the items to these sports, respectively. The two sports specific surveys were later adapted to become one generic version to allow administration of the survey in the wider child sport setting. To assess the survey’s face and content validity, a draft was sent to several experts, in addition to those in the Human Movement field, for feedback on its age specificity, readability and item content, and suggestions for any modifications, additions and deletions. Two secondary and two primary level school teachers commented specifically on the age appropriateness of the language, readability and formatting. Five sport experts representing Australian state and national bodies for cricket and netball provided feedback regarding the survey’s sport specific content and relevance to those sports. Quali-
23
tative feedback from all experts was collated and informed changes to the survey. A test–retest reliability study was conducted with a small subset of junior cricket participants who had been recruited in a larger, concurrent, cohort study of junior cricket.27 Fifty-nine cricket participants completed the survey on two occasions, within a 3–4-week interval. All but two participants were male and all were 10–16 years of age; all participated in organised after-school cricket in one of three selected age levels (12 years and under [U12], U14 and U16). The time period for the test–retest reliability assessment was based on convenience for the participants and sufficient time between the two survey administrations. Consistency between responses to the two survey administrations was compared using a Cohen’s Kappa (κ) statistic or Weighted Kappa (κw2 ) statistic for ordinal items, as appropriate. Kappa values were graded according to Landis and Koch.28 Findings from the validity and reliability assessments were summarised and a more generic version of the survey was developed for administration among children in the wider sports setting. The final version of the survey can be found in Supplement File 1. This research was approved by the University’s Human Research and Ethics Committee and informed consent was obtained from all participants.
3. Results The teacher experts who reviewed the survey for face and content validity raised some concerns about whether children aged less than 10 years would be capable of completing the survey considering its complexity and length. Survey readability was generally identified as being appropriate for the target age group, but more so for children aged 12+ years. In response, the survey size was reduced by deleting items that were considered to be non-essential when administered in a general sport setting. Some formatting changes were also made to decrease item complexity. A Flesch Kincaid Grade Level Test was conducted on the final survey, which indicated that the readability was appropriate for children aged 12+ years. The survey’s content in terms of its relevance to children’s cricket and netball was identified as appropriate by the peak sports bodies with very few suggestions for content alterations. Suggestions for some wording refinements and improvements were provided, and adjustment to the survey formatting and the incorporation of graphics to improve its layout were recommended and subsequently made. The injury history aspects of the survey were identified as requiring modification to improve their clarity. This was addressed by the inclusion of a clearer injury definition to facilitate children’s recall of relevant injury events. This injury definition was modelled on questions previously used to classify sport injury within other sport injury surveys.2,20,22,29 The particular definition adopted was
24
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
related to the loss of time from sport participation following the sport injury incident. Table 1 summarises the survey item modifications that were undertaken in response to the test–retest reliability assessments and the expert feedback. Typically, items with very low Kappa/Weighted Kappa values (κ ≤ 0.30 [poorslight]) were deleted, while other low scoring items (κ ≤ 0.40 [poor-fair]) underwent more modification than those with moderate reliability (κ = 0.41–0.60 [moderate]). Items with higher values (κ ≥ 0.61 [substantial-almost perfect]) required only minor modification or none at all. Supplement File 2 provides the specific reliability scores for all survey items in the cricket-specific survey. Questions with high reliability (κ ≥ 0.61) related to demographics (age, gender, family structure); sports played over the last year; and sports in which children expected to withdraw. Items relating to reasons for discontinued participation were also generally associated with high reliability scores (κ ≥ 0.61), or could not be computed due to a lack of variability among children’s responses. Questions with a large number of possible response options, especially items regarding reasons for participation, typically demonstrated moderate reliability (κ = 0.41–0.60). Questions with low reliability (κ ≤ 0.40) included some of the injury history and risk perception items. A final version of the survey was generated by converting the two sport specific (cricket and netball) surveys into one generic form, taking into account all the validity and reliability assessments.
4. Discussion This study has verified the face and content validity of two sport specific versions of a survey instrument (for cricket or netball), and determined the test–retest reliability of the cricket-specific version. Both versions of the survey were similar and designed to collect data pertaining to self-reported sport participation, recent injury history and injury risk perception, among children aged 9–16 years. Self-reported information from child sport participants is useful, but like all survey data, can be subject to biases.13,14 It is therefore important when using this type of instrument to ensure that it collects accurate and reliable data. Validity was determined through review of expert feedback from sporting representatives and teachers who identified some areas where improvements to wording and format were required. Overall, the survey’s readability was considered appropriate for the target age group, although for children under the age of 12 years, and/or children who experience difficulty with reading and writing, assistance from an adult may be required when completing the survey. The test–retest reliability of the cricket-specific survey prompted other modifications. Some items with a large number of response options, such as those regarding reasons why children participated in sport, typically demonstrated only
moderate reliability. To address this, the number of total response options was reduced by removing those considered non-essential for survey administration in a general sport setting. Conversely, the multi-response items which related to reasons why children discontinued sport participation, had high reliability. Only two children were required to respond to the sports discontinuation items as these children were the only ones who reported that they planned to cease sport participation. Because so few children planned to discontinue their sport participation, it was not possible to compute a reliability score for some of these items, as there was no variability in responses for the two children on each occasion. Reliability assessments of injury history and injury risk perception items had low κ scores. This issue was also emphasised by the expert feedback providers during the face and content validity phase, when the possibility of memory influencing injury recall for some children was first highlighted. As a result, a clearer definition of injury, modelled on previous studies,2,20,22,29 was included in the final version to facilitate children’s recall of injury events, viz.: ‘Sport injury is an incident during a sport game or training session, where you hurt yourself and had to miss a game or training session(s) afterwards [e.g. sprain, graze, fracture, bruise, etc.]’. As the majority of children participating in the test–retest reliability assessment were male, no gender comparisons could be made. Therefore, it is unknown whether gender influenced the reliability of the survey; however, the validity assessment found that the content of both the cricket (97% male respondents) and netball (100% female) specific surveys, was appropriate with only minor modifications suggested. In the initial cricket survey, many items were targeted to this sport for specific testing with a sample of junior cricket participants. Where necessary, items retained after assessment were modified slightly to become generic, extending the survey’s relevance to participants of other formal sports. As only minor modifications to wording were made to convert the cricket-specific survey to the generic form, it was assumed that all items found to be valid for the cricket-specific survey would remain valid once converted. For example, ‘How many matches did you play in the past week for cricket?’ was modified to, ‘How many times did you play sport last week?’. Despite the final survey containing generic items to enable its administration in different sports settings, it is unreasonable to automatically assume that this will ensure the successful administration in every childhood sport. However, as the modifications were guided by both the feedback from the expert panel and the reliability assessment, it is likely that the changes undertaken would facilitate children’s completion of the survey. The survey was only validated in cricket and netball and the reliability tested only in cricket, hence the survey’s ability to collect valid and reliable data in the wider context of other sports cannot be assumed. Surveys for sport settings that differ noticeably from these will require further modification and it is recommended that validation be confirmed in this case.
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
25
Table 1 Summary of survey item modifications based on expert feedback and results of the test–retest reliability assessment. Type of item
Kappa valuesa κ = ≤0.40 (low reliability) Moderate to substantial item modification
Demographics e.g., How old are you now? Cricket/netball participation e.g., How many matches/training sessions played in the past week? Reasons for participating e.g., Why do you play cricket?
Other sport participation e.g., Apart from cricket, have you played any other sports over the past year? Reasons for discontinuing sport e.g., Why will you stop playing these sports next year? Injury history e.g., Did you get sore, hurt or injured playing cricket last season?
Injury risk perception e.g., How much chance of getting injured do you think you have when playing cricket? a
κ = 0.41–0.60 (moderate reliability) κ = 0.61–1.00 (high reliability) Minor to moderate item modification Minor item modification or items unchanged All items remained unchanged.
Format changed to improve readability and graphic layout.
Modification to wording to enhance clarity and to become general to sport. Number of response options reduced from 20 choices to 13 choices. Modification to become general to sport. Minor modification to wording to enable survey to become general to sport. Number of response options reduced from 19 choices to 13 choices.
Wording modification to improve clarity and to link items to the newly included injury definition. Some of the items that scored κ ≤ 0.30 were considered non-essential in a general sport setting and were deleted. Wording made general to sport.
Inclusion of clear injury definition and some modification to wording to improve clarity and to become general to sport.
Wording modified to improve clarity and to become general to sport.
Either Kappa or Weighted Kappa, as appropriate.
The nature of children’s sports varies markedly, particularly across nations. Unlike cricket and netball, competition and/or training sessions for some sports may not be scheduled on a weekly basis. Further, although cricket and netball are popular sports among Australian children, they are not, for example, typical sports that are played throughout North America and in many parts of Europe. Sports of interest in these countries may be quite different to cricket and/or netball and may not correspond completely to the items included in the final generic survey. It should be noted that in the cricket and netball-specific surveys, question items could be directed purposefully to children’s participation in these sports. However, after the survey was converted from sport specific to a generic format, questions could no longer focus on one sport alone. To ensure that this did not cause confusion for children who played more than one sport, the generic survey instructions explain that in this circumstance, children should base their responses on the sport that they play most often. This method has been adopted for other sport injury surveys.2,10 In summary, there have been no published studies investigating the influence of sport injury and injury risk perception on children’s continued sport participation. It is likely that the lack of a reliable and valid survey tool has contributed to
this. This paper verifies the validity of netball and cricketspecific instruments and determines the reliability of the cricket-specific survey tool to enable these factors to be explored among children involved in these sports. This study has developed and validated the first survey instrument of its kind that can be used to collect data of this nature in these two sports and also offers a generic format which could be applied to other organised sport settings in future studies.
5. Practical implications • Valid and reliable information about the role of injury and injury risk perception on children’s ongoing participation in, and choices of, sport can be obtained through a selfreport survey. • The survey can be successfully administered to children aged 9–16 years participating in cricket; however, children under the age of 12 years may require greater assistance from an adult than children above this age, to complete the survey. • Application of the survey to other sports settings is recommended, although further evaluation of the instrument in new settings, outside netball and cricket, may be required.
26
E.J. Siesmaa et al. / Journal of Science and Medicine in Sport 14 (2011) 22–26
Acknowledgements This research was supported by research grants from the Department of Planning and Community Development - Sport and Recreation Victoria Division and the Injury Prevention Community Grants Program of the Australian Government Department of Health and Ageing. Emma Siesmaa is supported by an Australian Postgraduate Award. Caroline Finch is supported by a National Health and Medical Research Council Principal Research Fellowship. The authors would also like to thank Associate Professor Amanda Telford for her input in the survey development process.
Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/ j.jsams.2010.04.006.
References 1. Conn JM, Annest JL, Gilchrist J. Sports and recreation related injury episodes in the US population, 1997–99. Inj Prev 2003;9:117–23. 2. Finch C, Cassell E. The public health impact of injury during sport and active recreation. J Sci Med Sport 2006;9:490–7. 3. Nicholl JP, Coleman P, Williams BT. The epidemiology of sports and exercise related injury in the United Kingdom. Br J Sports Med 1995;29(4):232–8. 4. Caine D, Maffulli N, Caine C. Epidemiology of injury in child and adolescent sports: injury rates, risk factors, and prevention. Clin J Sport Med 2008;27:19–50. 5. Hoffman JR, Kang J, Faigenbaum AD, et al. Recreational sports participation is associated with enhanced physical fitness in children. Res Sports Med 2005;13:149–61. 6. Olds T, Dollman J, Maher C. Adolescent sport in Australia: who, when, where and what? ACHPER Healthy Lifestyles J 2009;56(1):11–6. 7. Gould D, Feltz D, Horn T, et al. Reasons for attrition in competitive youth swimming. J Sport Behav 1982;5(3):155–65. 8. Butcher J, Lindner KJ, Johns DP. Withdrawal from competitive youth sport: a retrospective ten-year study. J Sport Behav 2002;25(2):145–63. 9. Finch C, Owen N, Price R. Current injury or disability as a barrier to being more physically active. Med Sci Sports Exerc 2001;33(5):778–82. 10. Emery CA, Meeuwisse WH, McAllister JR. Survey of sport participation and sport injury in Calgary and area high schools. Clin J Sport Med 2006;16:20–6. 11. Williams JM, Wright P, Currie CE, et al. Sports related injuries in Scottish adolescents aged 11–15. Br J Sports Med 1998;32:291–6.
12. Finch CF, McIntosh AS, McCrory P. What do under 15 year old schoolboy rugby union players think about protective headgear? Br J Sports Med 2001;35:89–94. 13. Dollman J, Okely AD, Hardy L, et al. A hitchhiker’s guide to assessing young people’s physical activity: deciding what method to use. J Sci Med Sport 2009;12:518–25. 14. Trost SG. State of the art reviews: measurement of physical activity in children and adolescents. Am J Lifestyle Med 2007;1(4):299–314. 15. Booth ML, Okely AD, Bauman A. The reliability and validity of the physical activity questions in the WHO health behaviour in schoolchildren (HBSC) survey: a population study. Br J Sports Med 2001;35:263–7. 16. Chen X, Sekine M, Hamanishi S, et al. Validation of a selfreported physical activity questionnaire for schoolchildren. J Epidemiol 2003;13(5):278–87. 17. Sallis JF, Buono MJ, Roby RR, et al. Seven-day recall and other physical activity self-reports in children and adolescents. Med Sci Sports Exerc 1993;25(1):99–108. 18. Telford A, Salmon J, Jolley D, et al. Reliability and validity of questionnaire-based measures for younger and older children’s physical activity: the Children’s Leisure Activities Study Surveys (CLASS). Pediatr Exerc Sci 2004;16:64–78. 19. Australian Bureau of Statistics. Children’s participation in cultural and leisure activities, Australia; April 2006. 20. Northern Sydney Area Health Service. New South Wales Youth Sports Injury Report; July 1997. 21. Centre for Epidemiology and Research, NSW Department of Health. The health behaviours of secondary school students in New South Wales 2002. N S W Public Health Bull 2004;15(S-2). 22. Finch C, Donohue S, Garnham A. Safety attitudes and beliefs of junior Australian football players. Inj Prev 2002;8:151–4. 23. Salmon J, Ball K, Crawford D, et al. Reducing sedentary behaviour and increasing physical activity among 10-year-old children: overview and process evaluation of the ‘Switch-Play’ intervention. Health Promot Int 2005;20(1):7–17. 24. Booth M, Macaskill P, McLellan L, et al. NSW schools fitness and physical activity survey, 1997. Sydney: NSW Dept of School Education; 1997. 25. Finch C, Donohue S, Garnham A, et al. The playing habits and other commitments of elite junior Australian football players. J Sci Med Sport 2002;5(3):266–73. 26. Telford R, Bass S, Budge M, et al. The lifestyle of our kids (LOOK) project: outline of methods. J Sci Med Sport 2009;12(1):156– 63. 27. Finch CF, White P, Dennis R, et al. Fielders and batters are injured too: a prospective cohort study of injuries in junior club cricket. J Sci Med Sport; Corrected Proof, available online 22 December 2009. doi:10.1016/j.jsams.2009.10.489. 28. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. 29. Finch C, McIntosh A, McCrory P, et al. A pilot study of the attitudes of Australian rules footballers towards protective headgear. J Sci Med Sport 2003;6(4):505–11.