- Email: [email protected]
Assessing Youth Risk Behavior in a Clinical Trial Setting: Lessons From the Infant Health and Development Program
Journal of Adolescent Health 46 (2010) 429–436
Original article
Assessing Youth Risk Behavior in a Clinical Trial Setting: Lessons From the Infant Health and Development Program Elizabeth R. Woods, M.D., M.P.H.a,*, Stephen L. Buka, Sc.D.b, Camilia R. Martin, M.D., M.S.c, Mikhail Salganik, Ph.D.d, Mary Beth Howard, B.S.e, Jennifer A. Gueguen, Ed.M., S.M.f, Jeanne Brooks-Gunn, Ph.D.g, and Marie C. McCormick, M.D., Sc.D.c,f a
Divisions of Adolescent/Young Adult Medicine, Children’s Hospital Boston, Harvard Medical School, Boston, Massachusetts b Brown University Program in Public Health, Providence, Rhode Island (Harvard School of Public Health) c Division of Newborn Medicine, Department of Pediatrics, Harvard Medical School, Boston, Massachusetts d Cytel Inc., Cambridge, Massachusetts, (Harvard School of Public Health) e Cornell University, Ithaca, New York f Harvard School of Public Health, Boston, Massachusetts g Teachers College and the College of Physicians and Surgeons, Columbia University, New York Manuscript received June 21, 2009; manuscript accepted October 27, 2009
Abstract
Purpose: The purpose of this article was to describe the use of the Youth Risk Behavior Surveillance System (YRBSS) with known 17–18-year-old patients in follow-up of a multisite randomized clinical trial, and to develop a new scoring algorithm indicating the degree of risk-taking behavior for betweengroup analyses. Methods: Seventy-five questions from the YRBSS were incorporated into the study questionnaire, with the development of safety plans to guide the disposition of participants. The YRBSS questions were grouped into two categories (with three subdomains each) named problem behaviors (conduct problems, sexual behavior, and suicide/hopelessness) and substance use (cigarettes, alcohol, and marijuana use), with scores for each subdomain indicating high, moderate, and low risk. Results: Of the 677 participants, the safety plan was activated 215 times for 199 (29.4%) of participants. Risk behaviors included binge drinking (149), alcohol/substance use and driving (41), depression (22), hopelessness (37), and suicidal ideation (13; all in the past). No emergency room evaluations were required. The subdomain scaling was analyzed by demographic characteristics, and findings were consistent with the literature; for example, higher rates of conduct problems in males, more suicidal ideation in females, greater sexual risk in African Americans, more substance use in males and whites, and more alcohol use in youth with mothers with higher levels of education. Conclusions: YRBSS can be administered in a research setting with appropriate safety precautions. These results should provide a useful guide to the application of the YRBSS to other adolescent populations in the future. Ó 2010 Society for Adolescent Medicine. All rights reserved.
Keywords:
Youth Risk Behavior Surveillance System; Adolescents; Safety plan; Suicidal ideation; Depression; Substance use; Risk behaviors
Participating Sites in IHDP: University of Washington, Seattle, WA; University of Texas Southwestern Medical Center, Dallas, TX; University of Pennsylvania School of Medicine, Children’s Hospital of Philadelphia, Philadelphia, PA; University of Miami School of Medicine, Miami, FL; Department of Neonatology, Beth Israel Deaconess Medical Center, Boston, MA; Children’s Hospital Boston, Boston, MA; University of Arkansas for
Medical Sciences, Little Rock, AR. For a complete list of participants, please see reference 18. *Address correspondence to: Elizabeth R. Woods, M.D., M.P.H., Division of Adolescent/Young Adult Medicine, LO-306, Children’s Hospital Boston, 300 Longwood Ave., Boston, MA 02115. E-mail address: [email protected]
1054-139X/10/$ – see front matter Ó 2010 Society for Adolescent Medicine. All rights reserved. doi:10.1016/j.jadohealth.2009.10.010
430
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
The Youth Risk Behavior Surveillance System (YRBSS) was developed by the Centers for Disease Control and Prevention in 1990 for the purpose of monitoring health risk behaviors that contribute to morbidity, mortality, and social problems among adolescents [1]. The survey is administered biennially and the data are used to determine the prevalence of health-risk behaviors among middle- and high-school students, assess trends in these behaviors, and allow for the analyses of co-occurrence of health-risk behaviors [2]. Due to the strong connection between adolescent health behaviors and specific health outcomes [3], the information reflects both current and future health status. This, combined with reliable test characteristics [4–6], makes the YRBSS a useful tool for monitoring trends in adolescent health risk behaviors [7–10]. The availability of nationally representative data has been essential for identifying health care needs, and has been critical for planning of services for adolescents [11]. The questions from these surveys can be useful in other research settings. Conventionally, used as a surveillance tool, the YRBSS is administered in the classroom as an anonymous self-administered questionnaire [2]. Attempt to use the YRBSS in situations where the identity of the respondent is known to raise additional considerations in the research team. Many items of the questionnaire address sensitive issues that require confidentiality and may incur legal liability if negative consequences result from the risk behaviors [2]. A certificate of confidentiality may provide additional protection of research participants’ responses, though this protection may not be absolute [12]. Protection of participants, in turn, helps to promote participation and achieve accurate reporting by assuring privacy [13]. Another implication of nonanonymous administration of national surveys is the necessity for a safety plan to be used in response to potentially injurious behaviors revealed by participant responses. Many items of the YRBSS questionnaire (e.g., suicide ideation, drinking and driving, and substance abuse) can reveal troublesome or dangerous behaviors that may need intervention if the participant is known to the investigators [2]. Therefore, a plan to manage these situations should be established and explained to participants at the time of consent, but there is limited literature discussing the use of safety plans. The utility of the YRBSS could be enhanced by creating algorithms to produce analyzable scores. When used by the Centers for Disease Control and Prevention, only individual risk behaviors are analyzed [1]. However, many of the behaviors measured may be of low prevalence and may not be independent of each other and can be grouped. In addition, Jessor described that risk and problem behaviors cluster to form a risk-behavior syndrome [10]. As examples, suicide attempts have been associated with physical fights and gun carrying, substance use, alcohol/substance use before sexual activity, and lack of seat belt use [14], and sexual risk behaviors have been associated with problem drug behaviors [15]. Development of summary subscales can improve subgroup
analyses and assessment of risk associations. Only one attempt to develop a combined question scoring algorithm for the YRBSS has been published for three subscales (substance use, sexual behaviors, and violence/victimization) [16]. Consideration of levels of overall risk behavior via a ‘‘risk behavior score’’ is likely to produce a better understanding of the degree of adolescents’ involvement in the behavior. In this article, the YRBSS is implemented in the follow-up at age 17–18 years in a multisite randomized controlled trial of early educational intervention, the Infant Health and Development Program (IHDP) [17–21]. In previous studies of preschool educational interventions, participants have shown reductions in delinquent and criminal behavior in early adulthood without any subsequent intervention [21, 22]. An examination of risk behaviors is important in this clinical trial setting where the respondent is known to the study team, and also requires the derivation of composite outcome measures that could be analyzed to investigate group differences. Methods Study population IHDP was a multisite, randomized controlled trial of an educational intervention until 3 years of age for low birth weight (<2500 g) and preterm (<38 weeks of gestation) infants born in 1985 at eight heterogeneous sites; the results of this trial have been described elsewhere [17–20]. To assess the effect of the intervention in adolescence, the cohort was reconstructed when the participants were aged 17–18 years and were examined using both questionnaire information and direct observations [23]. In this most recent data collection, 677 (68.7%) of the 985 of the original participants of IHDP had interview data on youth risk behaviors. Of these, 46.8% were male, 52.1% African American and 9.0% Hispanic, and 36.5% had maternal education of less than high-school graduation at the time of the youth’s birth. Data collection Risk behaviors were assessed using youth self-report on questions taken from the YRBSS. Seventy-five items from this survey were administered as a face-to-face interview to characterize the youth’s involvement in behaviors including suicidal ideation/attempts, smoking, alcohol and marijuana use, and sexual activity. YRBSS has been shown to have excellent test–retest reliability, especially for sex, drug, alcohol, and tobacco items [24]. More general behavioral problems were ascertained by maternal report and youth self-reported questionnaires, using the Behavior Problem Index. We also administered 24 items from the parent–child version of the Conflict Tactics Scale to assess the youth’s report of verbal and psychological violence of their primary caregiver. These are frequently administered scales with robust psychometric properties [23,25]. Questions about
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
injection drug use were not included because of the potential need for intervention that could not be addressed adequately in the research setting. Human subjects issues These instruments presented human subjects concerns in that responses to some of the questions might indicate behaviors or situations that place the youth at risk for poor health outcomes. The questions that addressed elicit illegal behavior could also confer criminal liability. Finally, questions about violence in the home might trigger the need for referral to protective services or other violence prevention programs. To protect the participants in the study, several steps were taken. The first was to obtain a certificate of confidentiality from the National Institutes of Health (NIH) to protect the confidentiality of information obtained in research and to reduce the risk of subpoena of research data in subsequent court cases that may be self-incriminating [13]. Second, during the pilot phase, investigators at the Children’s Hospital Boston and the Harvard School of Public Health developed a template protocol based on early experience with adolescent/young adult research. This protocol involved flagging specific questions in the instrument for the youth that could suggest danger to their health and welfare, such as suicidal ideation, depression reported by youth or parent, a sense of hopelessness, and high-risk substance use. A positive response to these flagged questions mandated activation of a specific safety plan at the time of the interview. The use of this approach minimized variability in response that might have occurred with interviewers of different levels of clinical expertise, and most of our interviewers were not health professionals. The safety plan contained guidance for further assessment depending on scores or responses to specific questions, and referral if needed using scripts with specific wording for discussion and disposition. A concerning score for depressive symptoms reported by parent or youth, or positive response by the youth to suicide-related questions, drinking and driving, or binge drinking led to activation of the safety plan. For depressive symptoms by parent or youth report, the safety plan suggested an assessment by the research staff, referral for mental health services, follow-up with their mental health providers, or connection to emergency evaluation. For youth who reported suicidal ideation, the timing was assessed, and referred to medical or mental health services, or emergency services for current suicidal ideation. For youth who admitted to drinking/substance use and driving, providing a ‘‘Contract for Life’’ (e.g., Mothers Against Drunk Driving http://www.madd.ca/english/youth/contract_ for_life.pdf) was recommended if not already in place within the family. For those who reported binge drinking/substance use, referral numbers for local substance programs were provided. Reports obtained through the Parent-Child Conflict Scale were not felt to indicate abuse, but if the youth or parent disclosed ongoing abuse as part of the interview process, then
431
a report was made to the Department of Social Services as required by state laws for participants aged less than 18 years. For those aged over 18 years, the safety plan recommended informing the primary care or mental health provider and/ or the referring the youth for mental health services. The activation of the safety plan was done in consultation with the local site director, all of whom were experienced clinicians. Evaluations were performed at locations where emergency rooms were available, and off-site evaluations identified emergency facilities in the area prior to the interview. As part of the preparation for data collection, each participating site was required to obtain approval from their local Institutional Review Board (IRB), with any modifications needed for that site to identify appropriate referral sources for the issues above. A standardized form for reporting any activation of the safety plan was designed and submitted to the main study office at the Harvard School of Public Health. An analysis of the safety plan activation forms was performed. A structured coding sheet was used and all forms were scored and entered by one author (J.A.G.), with 10% evaluated by a second author (E.R.W.), and the few differences or unclear results were discussed and reconciled by both authors. Specific data elements summarized included the participation site, the reason for initiation of the safety plan, and disposition resulting from these observations. To protect the confidentiality of the respondents, these reports did not include identifying information. Scaling of the YRBSS The YRBSS questions were grouped into two broad categories: (1) ‘‘problem behavior,’’ including conduct problems, sexual behavior and suicidality/hopelessness, and (2) ‘‘substance use,’’ including cigarettes, alcohol and marijuana. The number of items in the six subdomains ranged from two questions on marijuana use (number of times used during past 30 days and lifetime) to eight questions on sexual behavior (ever had intercourse, first age, number of partners during past 3 months and lifetime, use of alcohol or drugs before intercourse, use of condom, number of times pregnant/impregnated, age first time pregnant/impregnated). There were seven questions concerning conduct problems, four questions on suicidality/hopelessness, three items on cigarette use, and three items reflecting alcohol use. For each of the subdomains studied, a marked ‘‘Lshaped’’ distribution for the composite responses was observed. Due to the non-normal distribution, we applied cutpoints to the approximately top 10%, the next 20%, and the large majority of approximately 70% of the sample with no or low problem behaviors. The Appendix lists the individual behaviors and coding approach for all six subdomains. For instance, 63% of the sample reported having never tried cigarettes and were assigned a value of 0 (low risk); 22% reported occasional cigarette use and were assigned a value of 1 (medium risk); and 15% reported regular smoking (daily for 30 days or more) and were assigned a value of 2 (high
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
432
Table 1 Rates of safety plan initiation by subject demographic characteristics
Table 2 Review of safety plan reports yielded 215 safety plan initiations during the studya
Subject Plans N
Number
%
Reason
n b
Total Gender Male Female Youth race White Black Hispanic Other Youth age (yr) 17 18 Maternal education Less than High School High School More than High School
677
199
29.4
317 360
104 95
32.8 26.4
228 353 61 35
100 69 19 11
43.9 19.6 31.2 31.4
102 575
17 182
16.7 31.6
247 202 228
60 62 77
24.3 30.7 33.8
risk). Lower thresholds were established for less prevalent behaviors: 6% of the youth reported suicidal thoughts or attempts during the past 12 months were rated as high risk, an additional 11% reported feeling sad or hopeless for two weeks or longer were rated as moderate risk, and the remaining 83% with no reports related to suicide or hopelessness were rated as low risk. Data analyses The rates of safety plan initiation according to subject demographic characteristics was described, and number of times youth respondents reported risk behaviors warranting the initiation of a safety plan and disposition was summarized. Next, we present descriptive statistics on the distribution of six subdomains derived from the YRBSS, according to subject and family demographic characteristics. Chisquare statistics were calculated for bivariate analyses to determine whether level of risk behavior differed according to subject gender, race/ethnicity, and maternal education level. Results Of the 677 participants, 199 respondents generated 215 activations of a safety plan, and the demographic distribution is in Table 1. More white patients overall had activated safety plans, but otherwise the demographic distribution is similar to that of the participants in the original study. The incidence of safety plan activations ranged from 2 to 53 times among the eight sites. One site was required by their local IRB to develop a safety plan only for suicidal ideation, and therefore had the least number of safety plan initiations. The reason for the safety plan initiations and the dispositions are described in Table 2. More than one initiation may have been required per participant depending on the risks
Alcohol/substance problem Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and providers in place No clear action Contract for Life given Depression by caregiver report Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and provider in place No clear action Depression by youth report Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and provider in place Suicidal ideation/hopelessnessc Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and provider in place Evaluation and provider discussion No clear action (hopelessness only) Family conflictd Information and referral Discussion with family No clear action
180 99 37 5 5 4 47 17 5 1 2 9 1 22 8 6 6 2 53 19 13 4 13 2 2 14 8 1 5
a The reasons and eventual dispositions are summarized. (Note: more than one reason may be given for the report and more than one disposition may be indicated for each reason). b 149 youth reported binge drinking/substance use and/or 41 reported drinking and driving (10 with both). c 37 youth responded positively to the hopelessness question only, 7 responded to hopelessness in the past 2 weeks, 13 responded positively to the suicidal ideation question, and 0 to suicidal ideation in the past 2 weeks. d 0 youth disclosed physical or sexual abuse.
assessed in the child’s and/or caretaker’s responses, and more than one disposition may have been noted. Teams varied in the amount of information described on the safety plan activation form, with 7.2% providing minimal or no data, so that not all dispositions could be inferred. The most common reasons for safety plan initiations were 149 due to binge drinking/substance use, and 41 reporting drinking/substance use and driving. Report of depression by the parent in 17 youth was slightly lower than the 22 youth self-reporting depression. Also, 13 participants reported positive responses to the suicidal ideation question, but none in the past 2 weeks. The administration of the family conflict scale suggested a high degree of family conflict in 14, but there was no discloser of physical or sexual abuse. After assessment by the research team, no emergency room evaluations were required. Most youth with depression, hopelessness, or suicidal ideation had providers in place and their family members were aware of their symptoms.
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436 Table 3 Distribution of YRBSS subscores for six subdomains Category/Subdomain Problem behavior Conduct problems Sexual behavior Suicide/hopeless Substance use Cigarettes Alcohol Marijuana
N
Low (%)
Medium (%)
High (%)
605 598 605
61 75 84
25 19 11
14 6 5
606 606 603
64 65 70
22 26 20
14 9 10
Table 3 presents the distribution of the six subdomains of youth risk behavior, according to the three-level scoring approach. For some subdomains, the number of questions available and the low rate of endorsement resulted in different distributions, for example, 84% of the youth reported no suicidal thoughts or behaviors, and only 5% reported the highest level of risk (suicidal ideation or attempt in the past 12 months). Table 4 indicates the rates of the YRBSS subscores by subject demographic characteristics for problem behaviors. Males were more likely to report high levels of conduct problems than females. African American participants and children of mothers with less education had higher scores for sexual risk behaviors. Females reported higher levels of suicidal ideation, attempts, and hopelessness than males. Males and whites reported significantly higher levels of tobacco and marijuana use. Males, whites, and youth with mothers who had higher levels of education reported higher scores for alcohol use. Discussion Administering the YRBSS and other sensitive questions as a personal interview was substantially different from the standard anonymous administration in the school setting. Most IRBs preferred a detailed safety plan to address potential risks reported by known participants at the time of interview. There was variability in the requests from IRBs for safety plans concerning depression and substance use. The multisite protocols have identified variability in IRB requirements across institutions [26]. Also, during the development of the study protocol, the safety plan needed some modification when participants reached the age of 18 to allow young adult participants to make more of their own decisions about sources of care, disclosure to family and providers, and preferred provider type for needed referrals. Although no participants reported current suicidal ideation or abuse, all IRBs required intervention for those participants if they were identified. A certificate of confidentiality provided added protection for research participants to report risk behaviors and limit the ability of the research material to be subpoenaed in court cases, but this may not provide absolute protection under all circumstances [12]. These steps attempt to ensure the safety and privacy of research participants participating in an individual interview setting.
433
The retrospective evaluation of the implementation of the safety plan provides insight into the initiation of the safety plan during the study. Most of the episodes of initiation of the safety plan were related to unsafe alcohol/substance use. Few participants expressed suicidal ideation and most endorsed primarily feelings of hopelessness; and none were assessed to be at current risk that required immediate emergency room evaluation. The documentation of the management of the participant when the safety plan was initiated varied some and could not always be ascertained by retrospective review. However, there was a similar approach to safety plan activation implemented and documented across sites depending on the seriousness of the reported risk. In general, the severity of the mental health issues was remarkably low, none required emergency room evaluation, most families were already aware of the issues, and many youth had treatment plans in place. Future studies could obtain feedback from youth and families about the management and information obtained from the safety plan activations. A novel aspect of this article was the grouping of participants’ responses to YRBSS questions into six subdomains of risk behavior, and the classification of these six subdomains into low-, medium-, and high-risk scores. Only one earlier study was identified that attempted to develop a scoring system for some of the questions of the YRBSS [16]. The current investigation expands this approach and generates summary scores for three classes of substance use and three types of problem behavior. Because the approach is based on the empirical results for the specific population the cutpoints for the categories will vary some from population to population. The 10th and 30th percentile cut-points were selected to be comparable with other investigations that have attempted to identify the extreme values (e.g., top decile, top quartile) for semi-continuous scales (deviant responder analyses). Members of our team have previously developed an illness severity measure for neonatal intensive care unit patients, which similarly generate summary scores of 26 items, with final classification into three categories reflecting low, moderate, and high severity [27]. Unfortunately, due to the different response levels of the initial YRBSS questions, statistical methods such as Cronbach’s alpha could not be used to show the internal consistency of questions within each subscale. However, this provided additional motivation to develop the categorical composite approach that we adopted. Other approaches may be considered by individual investigators who might wish to analyze the total count of certain risk domains (e.g., number of conduct problems in the past 12 months or total lifetime uses of marijuana). This approach of identifying the extremes for subdomains of risk behavior has face validity, and we have provided initial information concerning construct validity for the classification system applied in these analyses [28]. With the exception of sexual behavior, there were significant gender differences for all of the risk behavior scales, as predicted
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
434
Table 4 Rates of YRBSS response in six subdomain scores by subject demographic characteristics Problem behavior Conduct problems Male Female White Black Hispanic Maternal education Less than high school High school More than high school Sexual behavior Male Female White Black Hispanic Maternal education Less than high school High school More than high school Suicide/hopelessness Male Female White Black Hispanic Maternal education Less than high school High school More than high school
N
Low Medium High p (%) (%) (%)
White Black Hispanic Maternal education Less than high school High school More than high school Alcohol use Male Female White Black Hispanic Maternal education Less than high school High school More than high school Marijuana use Male Female
Rates of YRBSS response in six subdomain scores by subject demographic characteristics Substance use
283 49 322 71
29 21
21 7
<.001
236 62 316 61 53 53
26 25 24
12 14 23
.36
222 61 170 61 213 61
23 26 27
16 13 13
.81
281 72 317 78
21 18
7 4
.25
233 86 313 66 52 77
12 25 19
2 8 4
<.001 (B vs. W)
222 68 166 72 210 85
26 20 12
7 8 2
<.001 (HS)
284 88 321 80
9 12
3 8
.01
236 84 316 83 53 89
11 11 8
5 6 4
.82
222 81 171 83 212 88
13 12 8
7 5 4
.33
White Black Hispanic Maternal education Less than high school High school More than high school
236 314 53
64 75 72
24 17 23
12 8 6
.05 (B vs. W)
221 170 212
75 66 68
16 23 23
9 11 9
0.27
Parentheses indicate subgroup comparisons.
Substance use Cigarette use Male Female
Table 4 (Continued )
284 322
56 70
28 17
15 13
<.001
238 315 53
56 70 66
24 20 28
21 10 6
<.001 (B vs. W) (H vs. W)
222 170 214
64 62 64
21 24 22
15 14 14
0.94
284 322
59 70
29 22
12 7
.01
238 315 53
50 76 66
37 17 28
13 7 6
<.001 (B vs. W)
222 170 214
71 67 58
21 24 32
8 9 10
.04 (HS)
282 321
62 77
24 17
13 6
<.001 (Continued )
by earlier research [14–16]. In particular, females reported higher levels of suicide/hopelessness, and males reported higher levels of conduct problems and all forms of substance use. White youth reported statistically higher levels of problem use for all three areas of substance use. In addition, high levels of sexual risk behavior were significantly elevated for African American youth and among offspring of mothers with less educational attainment. Higher levels of maternal education attainment were directly associated with higher levels of problem substance use. Validation of this approach requires more empiric studies. Changing the administration mode from survey to interview may have resulted in under-reporting [4]. Adolescents’ responses seem more sensitive to mode of data collection than adults’ responses [29]. This observation is of particular relevance to the YRBSS, as many items address sensitive issues because they are potentially illegal or socially undesirable. In studies comparing rates of risk behaviors, greater privacy afforded in self-administered questionnaires leads to higher self-reported rates of these behaviors in adolescents [30–32]. Turner et al found greater effects of administration mode on reports of cocaine use than alcohol use [29], but the effects of mode of administration on depression and antisocial behavior reports have not been delineated. Asking sensitive questions may lead the participant to view the interviewer as having a counseling role, and may seek support and guidance from the interviewer. This misconception is common among participants in clinical trials, who believe that both therapy and research are governed by the goal of advancing the individual participant’s best interests [33]. Though only described in clinical research [34–36], a similar misconception may exist for participants in interviewer-administered behavioral research. Development of safety protocols provides explicit guidance to research staff on their roles, responsibilities and limitations in this regard, as well as a safety net for individual participants. This study evaluates the approach to developing a safety plan for this population of known participants receiving individual interviews, and the outcomes of the initiation of the safety plan. IRB requirements varied from site to site; however, when a detailed safety plan was required a standard approach was provided. The study further demonstrates that groups of YRBSS questions can be combined and scaled
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
depending on the range of responses of the specific population. This provides a ranking of risk within problem behaviors and substance use categories across multiple questions, and can provide a way to compare the distribution of risks between populations. Both the issues of implementation sensitive questionnaires and the approach of developing analyzable summary scores have not received much attention in the literature related to adolescent research. Thus, this article provides important approaches for the implementation and analysis of sensitive behavioral questions in research settings where the participant is known to the investigators, and provides summary scores that can lead to group and population comparisons. Acknowledgments We are grateful for the research and administrative assistance of Julie Goldman, Jennifer Yu, and Alaina Kessler for this study, as well as the collaboration of all the IHDP program directors and coordinators. This research is supported in part by a grant from Robert Wood Johnson Foundation (ID No. 039543), and Leadership Education in Adolescent Health Training grant numbers T71MC00009 and T76MC0001 from the Maternal and Child Health Bureau, Health Resources and Services Administration. References [1] U.S. Department of Health and Human Services, Centers for Disease Control, Atlanta, GA. An epidemiological surveillance system to monitor the prevalence of youth behaviors that most affect health. Chronic Disease and health Promotion: Reprints from the MMWR 1990 Youth Surveillance System 1990. [2] Kolbe LJ, Kann L, Collins JL. Overview of the Youth Risk Behavior Surveillance System. Public Health Rep 1993;108(supp 1):2–10. [3] Brener ND, Kann L, Kinchen SA, et al. Methodology of the youth risk behavior surveillance system. MMWR Recomm Rep 2004;53: 1–13. [4] Brener ND, Billy JO, Grady WR. Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. J Adolesc Health 2003;33:436–57. [5] Johnston LD, O’Malley PM. Issues of validity and population coverage in student surveys of drug use. In: Rouse BA, Kozel NJ, Richards LG, eds. Self-Report Methods of Estimating Drug Use: Meeting Current Challenges to Validity. Washington, DC: U.S. Government Printing Office, 1985:31–54. National Institute on Drug Abuse Research Monograph 57, DHHS Publication No. (ADM 85-1402). [6] Winters KC, Stinchfield RD, Henly GA, et al. Validity of adolescent self-report of alcohol and other drug involvement. Int J Addict 1991; 25:1379–95. [7] Blanken AJ. Measuring use of alcohol and other drugs among adolescents. Public Health Rep 1993;108:25–30. [8] Lowry R, Powell KE, Kann L, et al. Weapon-carrying, physical fighting, and fight-related injury among U.S. adolescents. Am J Prev Med 1998;14:122–9. [9] Morris L, Warren CW, Aral SO. Measuring adolescent sexual behaviors and related health outcomes. Public Health Rep 1993;108: 31–6. [10] Jessor R. Risk behavior in adolescence: A psychological framework for understanding and action. J Adolesc Health 1991;12:597–605.
435
[11] Klein JD, Barrett MS, Blythe MJ, et al. American Academy of Pediatrics Committee on Adolescence. Contraception and adolescents. Pediatrics 2007;120:1135–48. [12] Boskow LM, Dame L, Costello EJ. Certificates of confidentiality and compelled disclosure of data. Science 2008;322:1054–5. [13] Wolf LE, Zandecki J, Lo B. The certificate of confidentiality application: A view from the NIH Institutes. IRB 2004;26:14–8. [14] Woods ER, Lin YG, Middleman A, et al. The associations of suicide attempts in adolescents. Pediatrics 1997;99:791–6. [15] Shrier LA, Emans SJ, Woods ER, et al. The association of sexual risk behaviors and problem drug behaviors in high school students. J Adolesc Health 1997;20:377–83. [16] Garofalo R, Wolf C, Wissow LS, et al. Sexual orientation and risk of suicide attempts among a representative sample of youth. Arch Pediatr Adolesc Med 1999;153:487–93. [17] The Infant Health and Development Program. Enhancing the outcomes of low-birth-weight, premature infants: A multisite, randomized trial. JAMA 1990;263:3035–42. [18] Brooks-Gunn J, McCarton CM, Casey PH, et al. Early intervention in lowbirth-weight premature infants: Results through age 5 years from the Infant Health and Development Program. JAMA 1994;272:1257–62. [19] McCarton CM, Brooks-Gunn J, Wallace IF, et al. Results at age 8 years of early intervention for low-birth-weight premature infants. The Infant Health and Development Program. JAMA 1997;277:126–32. [20] McCormick MC, McCarton C, Brooks-Gunn J, et al. The Infant Health and Development Program: Interim summary. J Dev Behav Pediatr 1998;19:359–70. [21] Campbell FA, Ramey CT, Pungello E, et al. Early childhood education: Young adult outcomes from the Abecedarian Project. App Dev Sci 2002;6:42–57. [22] Schweinhart LJ, Barnes HV, Weikart DP, et al. Significant benefits: The High/Scope Perry preschool study through age 27. Monographs of the High/Scope Educational Research Foundation, No. 10. Ypsilanti, MI: High/Scope Educational Research Foundation, 1993. [23] McCormick MC, Brooks-Gunn J, Buka SL, et al. Early intervention in low birth weight premature infants: Results at 18 years for the Infant Health and Development Program. Pediatrics 2006;117:771–80. [24] Rosenblum JE. Truth or consequences: The intertemporal consistency of adolescent self-report on the Youth Risk Behavior Survey. Am J Epidemiol 2009;169:1388–97. [25] Straus MA. Measuring intrafamily conflict and violence: The Conflict Tactics (CT) scales. J Marriage Fam 1979;41:75–88. [26] Helfand BT, Moniu AK, Roehrborn CG, et al. Variation in institutional review board responses to a standard protocol for a multicenter randomized, controlled surgical trial. J Urol 2009;181:2674–9. [27] Richardson DK, Gray JE, McCormick MC, et al. Score for Neonatal Acute Physiology: A physiologic severity index for neonatal intensive care. Pediatrics 1993;91:617–23. [28] Nunnally J. Psychometric Theory. New York: McGraw-Hill, 1978. [29] Turner CF, Lessler JT, Devore JW. Effects of mode of administration and wording on reporting of drug use. In: Turner CF, Lessler JF, Gfroerer JC, eds. Survey Measurement of Drug Use: Methodological Studies. Washington, DC: U.S. Government Printing Office, 1992:177–220. [30] Schober SE, Fe Caces M, Pergamit MR, et al. Effect of mode of administration on reporting of drug use in the National Longitudinal Survey. In: Turner CF, Lessler JT, Gfroerer JC, eds. Survey Measurement of Drug Use. Washington, DC: U.S. Government Printing Office, 1992:267–76. [31] Brittingham A, Tourangeau R, Kay W. Reports of smoking in a national survey: Data from screening and detailed interviews, and from self- and interviewer-administered questions. Ann Epidemiol 1998;8:393–401. [32] Davoli M, Perucci CA, Sangalli M, et al. Reliability of sexual behavior data among high school students in Rome. Epidemiology 1992;3:531–5.
436
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
[33] Appelbaum PS, Roth LH, Lidz CW, et al. False hopes and best data: Consent to research and the therapeutic misconception. Hastings Cent Rep 1987;17:20–4. [34] Lidz CW, Appelbaum PS. The therapeutic misconception: Problems and solutions. Med Care 2002;40:55–63.
[35] Lidz CW, Appelbaum PS, Grisso T, et al. Therapeutic misconception and the appreciation of risks in clinical trials. Soc Sci Med 2004;58: 1689–97. [36] Appelbaum PS, Lidz CW, Grisso T. Therapeutic misconception in clinical research: Frequency and risk factors. IRB 2004;26:1–8.
Appendix. Description of 27 items from the YRBSS used to generate six composite indices of youth problem behaviors and substance use Conduct problems (past 12 months) Participated in physical fights Taken something from store Damaged school property Skipped school Stayed out at night Stopped and questioned by police Arrested by the police Count of total number of events in past 12 months: 2þ ¼ high risk; 1 ¼ moderate risk; 0 ¼ low risk. Sexual behavior Ever had sex Age at first sexual intercourse Number of partners (lifetime) Number of partners (past 3 months) Use alcohol or drugs last sex Didn’t use condom last sex Ever pregnant/gotten someone pregnant Age first time Count of following risk behaviors: 4þ lifetime partners; sex before age 15; sexually active past 3 months without condom use; sexually active past 3 months with multiple partners; ever pregnant; used alcohol or drugs before last intercourse. 4þ risks ¼ high risk; 2-3 risks ¼ moderate risk; 0-1 risks ¼ low risk. Suicide/hopelessness (past 12 months) Felt sad or hopeless and stopped usual activities for 2 weeks Seriously considered attempting suicide Made plan for suicide attempt Attempted suicide Ideation or attempt ¼ high risk; sad/hopeless only ¼ moderate risk; no positive responses ¼ low risk. Cigarette use Number of days smoked (past month) Cigarettes per day (past month) Ever smoked daily Ever smoked daily ¼ high risk; other positive response ¼ moderate risk; never smoked ¼ low risk. Alcohol use Number of days drank alcohol (lifetime) Number of days drank alcohol (past 30 days) Binge drinking past 30 days (5þ drinks) 100þ drinks lifetime or 10þ drinks past month or 3þ times binging past month ¼ high risk; <10 drinks lifetime ¼ low risk; all others ¼ moderate risk. Marijuana use Number of days used marijuana (lifetime) Number of days used marijuana (past 30 days) 100þ use lifetime ¼ high risk; <3 uses lifetime ¼ low risk; all others ¼ moderate risk
Original article
Assessing Youth Risk Behavior in a Clinical Trial Setting: Lessons From the Infant Health and Development Program Elizabeth R. Woods, M.D., M.P.H.a,*, Stephen L. Buka, Sc.D.b, Camilia R. Martin, M.D., M.S.c, Mikhail Salganik, Ph.D.d, Mary Beth Howard, B.S.e, Jennifer A. Gueguen, Ed.M., S.M.f, Jeanne Brooks-Gunn, Ph.D.g, and Marie C. McCormick, M.D., Sc.D.c,f a
Divisions of Adolescent/Young Adult Medicine, Children’s Hospital Boston, Harvard Medical School, Boston, Massachusetts b Brown University Program in Public Health, Providence, Rhode Island (Harvard School of Public Health) c Division of Newborn Medicine, Department of Pediatrics, Harvard Medical School, Boston, Massachusetts d Cytel Inc., Cambridge, Massachusetts, (Harvard School of Public Health) e Cornell University, Ithaca, New York f Harvard School of Public Health, Boston, Massachusetts g Teachers College and the College of Physicians and Surgeons, Columbia University, New York Manuscript received June 21, 2009; manuscript accepted October 27, 2009
Abstract
Purpose: The purpose of this article was to describe the use of the Youth Risk Behavior Surveillance System (YRBSS) with known 17–18-year-old patients in follow-up of a multisite randomized clinical trial, and to develop a new scoring algorithm indicating the degree of risk-taking behavior for betweengroup analyses. Methods: Seventy-five questions from the YRBSS were incorporated into the study questionnaire, with the development of safety plans to guide the disposition of participants. The YRBSS questions were grouped into two categories (with three subdomains each) named problem behaviors (conduct problems, sexual behavior, and suicide/hopelessness) and substance use (cigarettes, alcohol, and marijuana use), with scores for each subdomain indicating high, moderate, and low risk. Results: Of the 677 participants, the safety plan was activated 215 times for 199 (29.4%) of participants. Risk behaviors included binge drinking (149), alcohol/substance use and driving (41), depression (22), hopelessness (37), and suicidal ideation (13; all in the past). No emergency room evaluations were required. The subdomain scaling was analyzed by demographic characteristics, and findings were consistent with the literature; for example, higher rates of conduct problems in males, more suicidal ideation in females, greater sexual risk in African Americans, more substance use in males and whites, and more alcohol use in youth with mothers with higher levels of education. Conclusions: YRBSS can be administered in a research setting with appropriate safety precautions. These results should provide a useful guide to the application of the YRBSS to other adolescent populations in the future. Ó 2010 Society for Adolescent Medicine. All rights reserved.
Keywords:
Youth Risk Behavior Surveillance System; Adolescents; Safety plan; Suicidal ideation; Depression; Substance use; Risk behaviors
Participating Sites in IHDP: University of Washington, Seattle, WA; University of Texas Southwestern Medical Center, Dallas, TX; University of Pennsylvania School of Medicine, Children’s Hospital of Philadelphia, Philadelphia, PA; University of Miami School of Medicine, Miami, FL; Department of Neonatology, Beth Israel Deaconess Medical Center, Boston, MA; Children’s Hospital Boston, Boston, MA; University of Arkansas for
Medical Sciences, Little Rock, AR. For a complete list of participants, please see reference 18. *Address correspondence to: Elizabeth R. Woods, M.D., M.P.H., Division of Adolescent/Young Adult Medicine, LO-306, Children’s Hospital Boston, 300 Longwood Ave., Boston, MA 02115. E-mail address: [email protected]
1054-139X/10/$ – see front matter Ó 2010 Society for Adolescent Medicine. All rights reserved. doi:10.1016/j.jadohealth.2009.10.010
430
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
The Youth Risk Behavior Surveillance System (YRBSS) was developed by the Centers for Disease Control and Prevention in 1990 for the purpose of monitoring health risk behaviors that contribute to morbidity, mortality, and social problems among adolescents [1]. The survey is administered biennially and the data are used to determine the prevalence of health-risk behaviors among middle- and high-school students, assess trends in these behaviors, and allow for the analyses of co-occurrence of health-risk behaviors [2]. Due to the strong connection between adolescent health behaviors and specific health outcomes [3], the information reflects both current and future health status. This, combined with reliable test characteristics [4–6], makes the YRBSS a useful tool for monitoring trends in adolescent health risk behaviors [7–10]. The availability of nationally representative data has been essential for identifying health care needs, and has been critical for planning of services for adolescents [11]. The questions from these surveys can be useful in other research settings. Conventionally, used as a surveillance tool, the YRBSS is administered in the classroom as an anonymous self-administered questionnaire [2]. Attempt to use the YRBSS in situations where the identity of the respondent is known to raise additional considerations in the research team. Many items of the questionnaire address sensitive issues that require confidentiality and may incur legal liability if negative consequences result from the risk behaviors [2]. A certificate of confidentiality may provide additional protection of research participants’ responses, though this protection may not be absolute [12]. Protection of participants, in turn, helps to promote participation and achieve accurate reporting by assuring privacy [13]. Another implication of nonanonymous administration of national surveys is the necessity for a safety plan to be used in response to potentially injurious behaviors revealed by participant responses. Many items of the YRBSS questionnaire (e.g., suicide ideation, drinking and driving, and substance abuse) can reveal troublesome or dangerous behaviors that may need intervention if the participant is known to the investigators [2]. Therefore, a plan to manage these situations should be established and explained to participants at the time of consent, but there is limited literature discussing the use of safety plans. The utility of the YRBSS could be enhanced by creating algorithms to produce analyzable scores. When used by the Centers for Disease Control and Prevention, only individual risk behaviors are analyzed [1]. However, many of the behaviors measured may be of low prevalence and may not be independent of each other and can be grouped. In addition, Jessor described that risk and problem behaviors cluster to form a risk-behavior syndrome [10]. As examples, suicide attempts have been associated with physical fights and gun carrying, substance use, alcohol/substance use before sexual activity, and lack of seat belt use [14], and sexual risk behaviors have been associated with problem drug behaviors [15]. Development of summary subscales can improve subgroup
analyses and assessment of risk associations. Only one attempt to develop a combined question scoring algorithm for the YRBSS has been published for three subscales (substance use, sexual behaviors, and violence/victimization) [16]. Consideration of levels of overall risk behavior via a ‘‘risk behavior score’’ is likely to produce a better understanding of the degree of adolescents’ involvement in the behavior. In this article, the YRBSS is implemented in the follow-up at age 17–18 years in a multisite randomized controlled trial of early educational intervention, the Infant Health and Development Program (IHDP) [17–21]. In previous studies of preschool educational interventions, participants have shown reductions in delinquent and criminal behavior in early adulthood without any subsequent intervention [21, 22]. An examination of risk behaviors is important in this clinical trial setting where the respondent is known to the study team, and also requires the derivation of composite outcome measures that could be analyzed to investigate group differences. Methods Study population IHDP was a multisite, randomized controlled trial of an educational intervention until 3 years of age for low birth weight (<2500 g) and preterm (<38 weeks of gestation) infants born in 1985 at eight heterogeneous sites; the results of this trial have been described elsewhere [17–20]. To assess the effect of the intervention in adolescence, the cohort was reconstructed when the participants were aged 17–18 years and were examined using both questionnaire information and direct observations [23]. In this most recent data collection, 677 (68.7%) of the 985 of the original participants of IHDP had interview data on youth risk behaviors. Of these, 46.8% were male, 52.1% African American and 9.0% Hispanic, and 36.5% had maternal education of less than high-school graduation at the time of the youth’s birth. Data collection Risk behaviors were assessed using youth self-report on questions taken from the YRBSS. Seventy-five items from this survey were administered as a face-to-face interview to characterize the youth’s involvement in behaviors including suicidal ideation/attempts, smoking, alcohol and marijuana use, and sexual activity. YRBSS has been shown to have excellent test–retest reliability, especially for sex, drug, alcohol, and tobacco items [24]. More general behavioral problems were ascertained by maternal report and youth self-reported questionnaires, using the Behavior Problem Index. We also administered 24 items from the parent–child version of the Conflict Tactics Scale to assess the youth’s report of verbal and psychological violence of their primary caregiver. These are frequently administered scales with robust psychometric properties [23,25]. Questions about
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
injection drug use were not included because of the potential need for intervention that could not be addressed adequately in the research setting. Human subjects issues These instruments presented human subjects concerns in that responses to some of the questions might indicate behaviors or situations that place the youth at risk for poor health outcomes. The questions that addressed elicit illegal behavior could also confer criminal liability. Finally, questions about violence in the home might trigger the need for referral to protective services or other violence prevention programs. To protect the participants in the study, several steps were taken. The first was to obtain a certificate of confidentiality from the National Institutes of Health (NIH) to protect the confidentiality of information obtained in research and to reduce the risk of subpoena of research data in subsequent court cases that may be self-incriminating [13]. Second, during the pilot phase, investigators at the Children’s Hospital Boston and the Harvard School of Public Health developed a template protocol based on early experience with adolescent/young adult research. This protocol involved flagging specific questions in the instrument for the youth that could suggest danger to their health and welfare, such as suicidal ideation, depression reported by youth or parent, a sense of hopelessness, and high-risk substance use. A positive response to these flagged questions mandated activation of a specific safety plan at the time of the interview. The use of this approach minimized variability in response that might have occurred with interviewers of different levels of clinical expertise, and most of our interviewers were not health professionals. The safety plan contained guidance for further assessment depending on scores or responses to specific questions, and referral if needed using scripts with specific wording for discussion and disposition. A concerning score for depressive symptoms reported by parent or youth, or positive response by the youth to suicide-related questions, drinking and driving, or binge drinking led to activation of the safety plan. For depressive symptoms by parent or youth report, the safety plan suggested an assessment by the research staff, referral for mental health services, follow-up with their mental health providers, or connection to emergency evaluation. For youth who reported suicidal ideation, the timing was assessed, and referred to medical or mental health services, or emergency services for current suicidal ideation. For youth who admitted to drinking/substance use and driving, providing a ‘‘Contract for Life’’ (e.g., Mothers Against Drunk Driving http://www.madd.ca/english/youth/contract_ for_life.pdf) was recommended if not already in place within the family. For those who reported binge drinking/substance use, referral numbers for local substance programs were provided. Reports obtained through the Parent-Child Conflict Scale were not felt to indicate abuse, but if the youth or parent disclosed ongoing abuse as part of the interview process, then
431
a report was made to the Department of Social Services as required by state laws for participants aged less than 18 years. For those aged over 18 years, the safety plan recommended informing the primary care or mental health provider and/ or the referring the youth for mental health services. The activation of the safety plan was done in consultation with the local site director, all of whom were experienced clinicians. Evaluations were performed at locations where emergency rooms were available, and off-site evaluations identified emergency facilities in the area prior to the interview. As part of the preparation for data collection, each participating site was required to obtain approval from their local Institutional Review Board (IRB), with any modifications needed for that site to identify appropriate referral sources for the issues above. A standardized form for reporting any activation of the safety plan was designed and submitted to the main study office at the Harvard School of Public Health. An analysis of the safety plan activation forms was performed. A structured coding sheet was used and all forms were scored and entered by one author (J.A.G.), with 10% evaluated by a second author (E.R.W.), and the few differences or unclear results were discussed and reconciled by both authors. Specific data elements summarized included the participation site, the reason for initiation of the safety plan, and disposition resulting from these observations. To protect the confidentiality of the respondents, these reports did not include identifying information. Scaling of the YRBSS The YRBSS questions were grouped into two broad categories: (1) ‘‘problem behavior,’’ including conduct problems, sexual behavior and suicidality/hopelessness, and (2) ‘‘substance use,’’ including cigarettes, alcohol and marijuana. The number of items in the six subdomains ranged from two questions on marijuana use (number of times used during past 30 days and lifetime) to eight questions on sexual behavior (ever had intercourse, first age, number of partners during past 3 months and lifetime, use of alcohol or drugs before intercourse, use of condom, number of times pregnant/impregnated, age first time pregnant/impregnated). There were seven questions concerning conduct problems, four questions on suicidality/hopelessness, three items on cigarette use, and three items reflecting alcohol use. For each of the subdomains studied, a marked ‘‘Lshaped’’ distribution for the composite responses was observed. Due to the non-normal distribution, we applied cutpoints to the approximately top 10%, the next 20%, and the large majority of approximately 70% of the sample with no or low problem behaviors. The Appendix lists the individual behaviors and coding approach for all six subdomains. For instance, 63% of the sample reported having never tried cigarettes and were assigned a value of 0 (low risk); 22% reported occasional cigarette use and were assigned a value of 1 (medium risk); and 15% reported regular smoking (daily for 30 days or more) and were assigned a value of 2 (high
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
432
Table 1 Rates of safety plan initiation by subject demographic characteristics
Table 2 Review of safety plan reports yielded 215 safety plan initiations during the studya
Subject Plans N
Number
%
Reason
n b
Total Gender Male Female Youth race White Black Hispanic Other Youth age (yr) 17 18 Maternal education Less than High School High School More than High School
677
199
29.4
317 360
104 95
32.8 26.4
228 353 61 35
100 69 19 11
43.9 19.6 31.2 31.4
102 575
17 182
16.7 31.6
247 202 228
60 62 77
24.3 30.7 33.8
risk). Lower thresholds were established for less prevalent behaviors: 6% of the youth reported suicidal thoughts or attempts during the past 12 months were rated as high risk, an additional 11% reported feeling sad or hopeless for two weeks or longer were rated as moderate risk, and the remaining 83% with no reports related to suicide or hopelessness were rated as low risk. Data analyses The rates of safety plan initiation according to subject demographic characteristics was described, and number of times youth respondents reported risk behaviors warranting the initiation of a safety plan and disposition was summarized. Next, we present descriptive statistics on the distribution of six subdomains derived from the YRBSS, according to subject and family demographic characteristics. Chisquare statistics were calculated for bivariate analyses to determine whether level of risk behavior differed according to subject gender, race/ethnicity, and maternal education level. Results Of the 677 participants, 199 respondents generated 215 activations of a safety plan, and the demographic distribution is in Table 1. More white patients overall had activated safety plans, but otherwise the demographic distribution is similar to that of the participants in the original study. The incidence of safety plan activations ranged from 2 to 53 times among the eight sites. One site was required by their local IRB to develop a safety plan only for suicidal ideation, and therefore had the least number of safety plan initiations. The reason for the safety plan initiations and the dispositions are described in Table 2. More than one initiation may have been required per participant depending on the risks
Alcohol/substance problem Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and providers in place No clear action Contract for Life given Depression by caregiver report Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and provider in place No clear action Depression by youth report Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and provider in place Suicidal ideation/hopelessnessc Information and referral Evaluation and assessment Evaluation and family discussion Evaluation and provider in place Evaluation and provider discussion No clear action (hopelessness only) Family conflictd Information and referral Discussion with family No clear action
180 99 37 5 5 4 47 17 5 1 2 9 1 22 8 6 6 2 53 19 13 4 13 2 2 14 8 1 5
a The reasons and eventual dispositions are summarized. (Note: more than one reason may be given for the report and more than one disposition may be indicated for each reason). b 149 youth reported binge drinking/substance use and/or 41 reported drinking and driving (10 with both). c 37 youth responded positively to the hopelessness question only, 7 responded to hopelessness in the past 2 weeks, 13 responded positively to the suicidal ideation question, and 0 to suicidal ideation in the past 2 weeks. d 0 youth disclosed physical or sexual abuse.
assessed in the child’s and/or caretaker’s responses, and more than one disposition may have been noted. Teams varied in the amount of information described on the safety plan activation form, with 7.2% providing minimal or no data, so that not all dispositions could be inferred. The most common reasons for safety plan initiations were 149 due to binge drinking/substance use, and 41 reporting drinking/substance use and driving. Report of depression by the parent in 17 youth was slightly lower than the 22 youth self-reporting depression. Also, 13 participants reported positive responses to the suicidal ideation question, but none in the past 2 weeks. The administration of the family conflict scale suggested a high degree of family conflict in 14, but there was no discloser of physical or sexual abuse. After assessment by the research team, no emergency room evaluations were required. Most youth with depression, hopelessness, or suicidal ideation had providers in place and their family members were aware of their symptoms.
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436 Table 3 Distribution of YRBSS subscores for six subdomains Category/Subdomain Problem behavior Conduct problems Sexual behavior Suicide/hopeless Substance use Cigarettes Alcohol Marijuana
N
Low (%)
Medium (%)
High (%)
605 598 605
61 75 84
25 19 11
14 6 5
606 606 603
64 65 70
22 26 20
14 9 10
Table 3 presents the distribution of the six subdomains of youth risk behavior, according to the three-level scoring approach. For some subdomains, the number of questions available and the low rate of endorsement resulted in different distributions, for example, 84% of the youth reported no suicidal thoughts or behaviors, and only 5% reported the highest level of risk (suicidal ideation or attempt in the past 12 months). Table 4 indicates the rates of the YRBSS subscores by subject demographic characteristics for problem behaviors. Males were more likely to report high levels of conduct problems than females. African American participants and children of mothers with less education had higher scores for sexual risk behaviors. Females reported higher levels of suicidal ideation, attempts, and hopelessness than males. Males and whites reported significantly higher levels of tobacco and marijuana use. Males, whites, and youth with mothers who had higher levels of education reported higher scores for alcohol use. Discussion Administering the YRBSS and other sensitive questions as a personal interview was substantially different from the standard anonymous administration in the school setting. Most IRBs preferred a detailed safety plan to address potential risks reported by known participants at the time of interview. There was variability in the requests from IRBs for safety plans concerning depression and substance use. The multisite protocols have identified variability in IRB requirements across institutions [26]. Also, during the development of the study protocol, the safety plan needed some modification when participants reached the age of 18 to allow young adult participants to make more of their own decisions about sources of care, disclosure to family and providers, and preferred provider type for needed referrals. Although no participants reported current suicidal ideation or abuse, all IRBs required intervention for those participants if they were identified. A certificate of confidentiality provided added protection for research participants to report risk behaviors and limit the ability of the research material to be subpoenaed in court cases, but this may not provide absolute protection under all circumstances [12]. These steps attempt to ensure the safety and privacy of research participants participating in an individual interview setting.
433
The retrospective evaluation of the implementation of the safety plan provides insight into the initiation of the safety plan during the study. Most of the episodes of initiation of the safety plan were related to unsafe alcohol/substance use. Few participants expressed suicidal ideation and most endorsed primarily feelings of hopelessness; and none were assessed to be at current risk that required immediate emergency room evaluation. The documentation of the management of the participant when the safety plan was initiated varied some and could not always be ascertained by retrospective review. However, there was a similar approach to safety plan activation implemented and documented across sites depending on the seriousness of the reported risk. In general, the severity of the mental health issues was remarkably low, none required emergency room evaluation, most families were already aware of the issues, and many youth had treatment plans in place. Future studies could obtain feedback from youth and families about the management and information obtained from the safety plan activations. A novel aspect of this article was the grouping of participants’ responses to YRBSS questions into six subdomains of risk behavior, and the classification of these six subdomains into low-, medium-, and high-risk scores. Only one earlier study was identified that attempted to develop a scoring system for some of the questions of the YRBSS [16]. The current investigation expands this approach and generates summary scores for three classes of substance use and three types of problem behavior. Because the approach is based on the empirical results for the specific population the cutpoints for the categories will vary some from population to population. The 10th and 30th percentile cut-points were selected to be comparable with other investigations that have attempted to identify the extreme values (e.g., top decile, top quartile) for semi-continuous scales (deviant responder analyses). Members of our team have previously developed an illness severity measure for neonatal intensive care unit patients, which similarly generate summary scores of 26 items, with final classification into three categories reflecting low, moderate, and high severity [27]. Unfortunately, due to the different response levels of the initial YRBSS questions, statistical methods such as Cronbach’s alpha could not be used to show the internal consistency of questions within each subscale. However, this provided additional motivation to develop the categorical composite approach that we adopted. Other approaches may be considered by individual investigators who might wish to analyze the total count of certain risk domains (e.g., number of conduct problems in the past 12 months or total lifetime uses of marijuana). This approach of identifying the extremes for subdomains of risk behavior has face validity, and we have provided initial information concerning construct validity for the classification system applied in these analyses [28]. With the exception of sexual behavior, there were significant gender differences for all of the risk behavior scales, as predicted
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
434
Table 4 Rates of YRBSS response in six subdomain scores by subject demographic characteristics Problem behavior Conduct problems Male Female White Black Hispanic Maternal education Less than high school High school More than high school Sexual behavior Male Female White Black Hispanic Maternal education Less than high school High school More than high school Suicide/hopelessness Male Female White Black Hispanic Maternal education Less than high school High school More than high school
N
Low Medium High p (%) (%) (%)
White Black Hispanic Maternal education Less than high school High school More than high school Alcohol use Male Female White Black Hispanic Maternal education Less than high school High school More than high school Marijuana use Male Female
Rates of YRBSS response in six subdomain scores by subject demographic characteristics Substance use
283 49 322 71
29 21
21 7
<.001
236 62 316 61 53 53
26 25 24
12 14 23
.36
222 61 170 61 213 61
23 26 27
16 13 13
.81
281 72 317 78
21 18
7 4
.25
233 86 313 66 52 77
12 25 19
2 8 4
<.001 (B vs. W)
222 68 166 72 210 85
26 20 12
7 8 2
<.001 (
284 88 321 80
9 12
3 8
.01
236 84 316 83 53 89
11 11 8
5 6 4
.82
222 81 171 83 212 88
13 12 8
7 5 4
.33
White Black Hispanic Maternal education Less than high school High school More than high school
236 314 53
64 75 72
24 17 23
12 8 6
.05 (B vs. W)
221 170 212
75 66 68
16 23 23
9 11 9
0.27
Parentheses indicate subgroup comparisons.
Substance use Cigarette use Male Female
Table 4 (Continued )
284 322
56 70
28 17
15 13
<.001
238 315 53
56 70 66
24 20 28
21 10 6
<.001 (B vs. W) (H vs. W)
222 170 214
64 62 64
21 24 22
15 14 14
0.94
284 322
59 70
29 22
12 7
.01
238 315 53
50 76 66
37 17 28
13 7 6
<.001 (B vs. W)
222 170 214
71 67 58
21 24 32
8 9 10
.04 (
282 321
62 77
24 17
13 6
<.001 (Continued )
by earlier research [14–16]. In particular, females reported higher levels of suicide/hopelessness, and males reported higher levels of conduct problems and all forms of substance use. White youth reported statistically higher levels of problem use for all three areas of substance use. In addition, high levels of sexual risk behavior were significantly elevated for African American youth and among offspring of mothers with less educational attainment. Higher levels of maternal education attainment were directly associated with higher levels of problem substance use. Validation of this approach requires more empiric studies. Changing the administration mode from survey to interview may have resulted in under-reporting [4]. Adolescents’ responses seem more sensitive to mode of data collection than adults’ responses [29]. This observation is of particular relevance to the YRBSS, as many items address sensitive issues because they are potentially illegal or socially undesirable. In studies comparing rates of risk behaviors, greater privacy afforded in self-administered questionnaires leads to higher self-reported rates of these behaviors in adolescents [30–32]. Turner et al found greater effects of administration mode on reports of cocaine use than alcohol use [29], but the effects of mode of administration on depression and antisocial behavior reports have not been delineated. Asking sensitive questions may lead the participant to view the interviewer as having a counseling role, and may seek support and guidance from the interviewer. This misconception is common among participants in clinical trials, who believe that both therapy and research are governed by the goal of advancing the individual participant’s best interests [33]. Though only described in clinical research [34–36], a similar misconception may exist for participants in interviewer-administered behavioral research. Development of safety protocols provides explicit guidance to research staff on their roles, responsibilities and limitations in this regard, as well as a safety net for individual participants. This study evaluates the approach to developing a safety plan for this population of known participants receiving individual interviews, and the outcomes of the initiation of the safety plan. IRB requirements varied from site to site; however, when a detailed safety plan was required a standard approach was provided. The study further demonstrates that groups of YRBSS questions can be combined and scaled
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
depending on the range of responses of the specific population. This provides a ranking of risk within problem behaviors and substance use categories across multiple questions, and can provide a way to compare the distribution of risks between populations. Both the issues of implementation sensitive questionnaires and the approach of developing analyzable summary scores have not received much attention in the literature related to adolescent research. Thus, this article provides important approaches for the implementation and analysis of sensitive behavioral questions in research settings where the participant is known to the investigators, and provides summary scores that can lead to group and population comparisons. Acknowledgments We are grateful for the research and administrative assistance of Julie Goldman, Jennifer Yu, and Alaina Kessler for this study, as well as the collaboration of all the IHDP program directors and coordinators. This research is supported in part by a grant from Robert Wood Johnson Foundation (ID No. 039543), and Leadership Education in Adolescent Health Training grant numbers T71MC00009 and T76MC0001 from the Maternal and Child Health Bureau, Health Resources and Services Administration. References [1] U.S. Department of Health and Human Services, Centers for Disease Control, Atlanta, GA. An epidemiological surveillance system to monitor the prevalence of youth behaviors that most affect health. Chronic Disease and health Promotion: Reprints from the MMWR 1990 Youth Surveillance System 1990. [2] Kolbe LJ, Kann L, Collins JL. Overview of the Youth Risk Behavior Surveillance System. Public Health Rep 1993;108(supp 1):2–10. [3] Brener ND, Kann L, Kinchen SA, et al. Methodology of the youth risk behavior surveillance system. MMWR Recomm Rep 2004;53: 1–13. [4] Brener ND, Billy JO, Grady WR. Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. J Adolesc Health 2003;33:436–57. [5] Johnston LD, O’Malley PM. Issues of validity and population coverage in student surveys of drug use. In: Rouse BA, Kozel NJ, Richards LG, eds. Self-Report Methods of Estimating Drug Use: Meeting Current Challenges to Validity. Washington, DC: U.S. Government Printing Office, 1985:31–54. National Institute on Drug Abuse Research Monograph 57, DHHS Publication No. (ADM 85-1402). [6] Winters KC, Stinchfield RD, Henly GA, et al. Validity of adolescent self-report of alcohol and other drug involvement. Int J Addict 1991; 25:1379–95. [7] Blanken AJ. Measuring use of alcohol and other drugs among adolescents. Public Health Rep 1993;108:25–30. [8] Lowry R, Powell KE, Kann L, et al. Weapon-carrying, physical fighting, and fight-related injury among U.S. adolescents. Am J Prev Med 1998;14:122–9. [9] Morris L, Warren CW, Aral SO. Measuring adolescent sexual behaviors and related health outcomes. Public Health Rep 1993;108: 31–6. [10] Jessor R. Risk behavior in adolescence: A psychological framework for understanding and action. J Adolesc Health 1991;12:597–605.
435
[11] Klein JD, Barrett MS, Blythe MJ, et al. American Academy of Pediatrics Committee on Adolescence. Contraception and adolescents. Pediatrics 2007;120:1135–48. [12] Boskow LM, Dame L, Costello EJ. Certificates of confidentiality and compelled disclosure of data. Science 2008;322:1054–5. [13] Wolf LE, Zandecki J, Lo B. The certificate of confidentiality application: A view from the NIH Institutes. IRB 2004;26:14–8. [14] Woods ER, Lin YG, Middleman A, et al. The associations of suicide attempts in adolescents. Pediatrics 1997;99:791–6. [15] Shrier LA, Emans SJ, Woods ER, et al. The association of sexual risk behaviors and problem drug behaviors in high school students. J Adolesc Health 1997;20:377–83. [16] Garofalo R, Wolf C, Wissow LS, et al. Sexual orientation and risk of suicide attempts among a representative sample of youth. Arch Pediatr Adolesc Med 1999;153:487–93. [17] The Infant Health and Development Program. Enhancing the outcomes of low-birth-weight, premature infants: A multisite, randomized trial. JAMA 1990;263:3035–42. [18] Brooks-Gunn J, McCarton CM, Casey PH, et al. Early intervention in lowbirth-weight premature infants: Results through age 5 years from the Infant Health and Development Program. JAMA 1994;272:1257–62. [19] McCarton CM, Brooks-Gunn J, Wallace IF, et al. Results at age 8 years of early intervention for low-birth-weight premature infants. The Infant Health and Development Program. JAMA 1997;277:126–32. [20] McCormick MC, McCarton C, Brooks-Gunn J, et al. The Infant Health and Development Program: Interim summary. J Dev Behav Pediatr 1998;19:359–70. [21] Campbell FA, Ramey CT, Pungello E, et al. Early childhood education: Young adult outcomes from the Abecedarian Project. App Dev Sci 2002;6:42–57. [22] Schweinhart LJ, Barnes HV, Weikart DP, et al. Significant benefits: The High/Scope Perry preschool study through age 27. Monographs of the High/Scope Educational Research Foundation, No. 10. Ypsilanti, MI: High/Scope Educational Research Foundation, 1993. [23] McCormick MC, Brooks-Gunn J, Buka SL, et al. Early intervention in low birth weight premature infants: Results at 18 years for the Infant Health and Development Program. Pediatrics 2006;117:771–80. [24] Rosenblum JE. Truth or consequences: The intertemporal consistency of adolescent self-report on the Youth Risk Behavior Survey. Am J Epidemiol 2009;169:1388–97. [25] Straus MA. Measuring intrafamily conflict and violence: The Conflict Tactics (CT) scales. J Marriage Fam 1979;41:75–88. [26] Helfand BT, Moniu AK, Roehrborn CG, et al. Variation in institutional review board responses to a standard protocol for a multicenter randomized, controlled surgical trial. J Urol 2009;181:2674–9. [27] Richardson DK, Gray JE, McCormick MC, et al. Score for Neonatal Acute Physiology: A physiologic severity index for neonatal intensive care. Pediatrics 1993;91:617–23. [28] Nunnally J. Psychometric Theory. New York: McGraw-Hill, 1978. [29] Turner CF, Lessler JT, Devore JW. Effects of mode of administration and wording on reporting of drug use. In: Turner CF, Lessler JF, Gfroerer JC, eds. Survey Measurement of Drug Use: Methodological Studies. Washington, DC: U.S. Government Printing Office, 1992:177–220. [30] Schober SE, Fe Caces M, Pergamit MR, et al. Effect of mode of administration on reporting of drug use in the National Longitudinal Survey. In: Turner CF, Lessler JT, Gfroerer JC, eds. Survey Measurement of Drug Use. Washington, DC: U.S. Government Printing Office, 1992:267–76. [31] Brittingham A, Tourangeau R, Kay W. Reports of smoking in a national survey: Data from screening and detailed interviews, and from self- and interviewer-administered questions. Ann Epidemiol 1998;8:393–401. [32] Davoli M, Perucci CA, Sangalli M, et al. Reliability of sexual behavior data among high school students in Rome. Epidemiology 1992;3:531–5.
436
E.R. Woods et al. / Journal of Adolescent Health 46 (2010) 429–436
[33] Appelbaum PS, Roth LH, Lidz CW, et al. False hopes and best data: Consent to research and the therapeutic misconception. Hastings Cent Rep 1987;17:20–4. [34] Lidz CW, Appelbaum PS. The therapeutic misconception: Problems and solutions. Med Care 2002;40:55–63.
[35] Lidz CW, Appelbaum PS, Grisso T, et al. Therapeutic misconception and the appreciation of risks in clinical trials. Soc Sci Med 2004;58: 1689–97. [36] Appelbaum PS, Lidz CW, Grisso T. Therapeutic misconception in clinical research: Frequency and risk factors. IRB 2004;26:1–8.
Appendix. Description of 27 items from the YRBSS used to generate six composite indices of youth problem behaviors and substance use Conduct problems (past 12 months) Participated in physical fights Taken something from store Damaged school property Skipped school Stayed out at night Stopped and questioned by police Arrested by the police Count of total number of events in past 12 months: 2þ ¼ high risk; 1 ¼ moderate risk; 0 ¼ low risk. Sexual behavior Ever had sex Age at first sexual intercourse Number of partners (lifetime) Number of partners (past 3 months) Use alcohol or drugs last sex Didn’t use condom last sex Ever pregnant/gotten someone pregnant Age first time Count of following risk behaviors: 4þ lifetime partners; sex before age 15; sexually active past 3 months without condom use; sexually active past 3 months with multiple partners; ever pregnant; used alcohol or drugs before last intercourse. 4þ risks ¼ high risk; 2-3 risks ¼ moderate risk; 0-1 risks ¼ low risk. Suicide/hopelessness (past 12 months) Felt sad or hopeless and stopped usual activities for 2 weeks Seriously considered attempting suicide Made plan for suicide attempt Attempted suicide Ideation or attempt ¼ high risk; sad/hopeless only ¼ moderate risk; no positive responses ¼ low risk. Cigarette use Number of days smoked (past month) Cigarettes per day (past month) Ever smoked daily Ever smoked daily ¼ high risk; other positive response ¼ moderate risk; never smoked ¼ low risk. Alcohol use Number of days drank alcohol (lifetime) Number of days drank alcohol (past 30 days) Binge drinking past 30 days (5þ drinks) 100þ drinks lifetime or 10þ drinks past month or 3þ times binging past month ¼ high risk; <10 drinks lifetime ¼ low risk; all others ¼ moderate risk. Marijuana use Number of days used marijuana (lifetime) Number of days used marijuana (past 30 days) 100þ use lifetime ¼ high risk; <3 uses lifetime ¼ low risk; all others ¼ moderate risk