- Email: [email protected]
Comparing Adolescent Response Bias Between Internet and Telephone Surveys
S36
Abstracts / 44 (2009) S14–S47
intervention. A total of 1,049 (97%) had sufficient data for analyses. Respondents completed the 7-item connectedness measure and other questions about the visit immediately post-visit. Participants were asked to indicate on a 5-point scale (1 ¼ ‘‘Not at all’’ to 5 ¼ ‘‘Very much’’) how much they felt the provider cared about them; listened to them; respected them; was someone they could talk to about a problem; would keep what was said confidential; was honest with them; and was judging them (reverse-coded). We examined reliability with Cronbach’s alpha and validity by comparing the connectedness scale score with hypothesized correlates: number of prior visits, satisfaction with the visit, rating of the way the provider gave advice or information, and change in understanding of substance use health risks. Results: Participants had a mean age of 16 + 2 years; 62% were females; 37% were non-White; and 44% had parents with at least a college degree. Connectedness item scores tended to be high, with the proportion of respondents giving ‘‘5’’ ratings ranging from 36% for the ‘‘cares’’ item to 71% for the reverse-coded ‘‘judging.’’ The scale had high internal consistency (alpha ¼ 0.83) although the ‘‘judging’’ item had low correlations with other scale items (r range, 0.12-0.19). Patients with three or more prior visits with the same provider had significantly higher total scores than those with fewer visits (MannWhitney U; p < 0.001), and total scores were correlated with satisfaction with visit (Spearman’s rho ¼ 0.57), rating of how well the provider gave advice (0.30), and reported likelihood of following the provider’s advice (0.53). Youth with higher scores were more likely to report that their understanding of the risks of substance use changed quite a bit or a great deal as a result of the medical visit (OR ¼ 1.07, 95% CI 1.04-1.10). Conclusions: This initial evaluation suggests that this connectedness measure is reliable and valid, although the one negative item, ‘‘judging,’’ needs improvement. Sources of Support: NIH/NIDA R01 DA018848; LEAH #T71MC00009, MCHB, HRSA
54. COMPARING ADOLESCENT RESPONSE BIAS BETWEEN INTERNET AND TELEPHONE SURVEYS Jonathan D. Klein, MD, MPH1, Caryn Graff Havens, MPH, MBA1, and Randall K. Thomas2. 1Dept. of Pediatrics, University of Rochester, Rochester, NY; 2Harris Interactive, Rochester, NY Purpose: To examine variation in response bias between internet vs. telephone survey methods. Methods: We surveyed 15-24-year-olds using: (1) an age and gender quota sample from the Harris Poll Online (n ¼ 859) weighted to represent the US population by age, gender, ethnicity and propensity to be online, and (2) a nationally representative random digit dial cluster sample (n ¼ 609) weighted to represent the US population by region, gender, age, and ethnicity. Scaled response options were included to assess satisficing (choosing the ‘easiest’ answer) and other response bias. Topics include demographics; emergency contraception; sexual behaviors and attitudes; condom use; access; and quality of care and confidentiality. Results: In agree/disagree questions regarding condom use and efficacy, respondents were asked if they strongly agreed, somewhat agreed, agreed/disagreed, somewhat disagreed, or strongly disagreed with statements. The online respondents were between 2 to 3.5 times more likely to select "neither agree/disagree" option than phone respondents for questions on: having unprotected sex (14.7 vs. 5.8%; p < 0.001), whether buying condoms was embarrassing (19.1 vs. 6.7%; p < 0.001), if condoms broke a lot (39.4 vs. 12.0%; p < 0.001), if sex without a condom was worth the risk (12.4 vs. 6.3%; p < 0.001), and though waiting to have sex is a nice idea,
nobody really does (21.6 vs. 7.1%; p < 0.001). Higher endorsement of the neither category was seen regardless of whether the adolescents were sexually experienced or not. Conclusions: Online respondents were more likely to choose not to express an opinion. This may be due to satisficing, choosing the ’easiest’ answer for the online group, or to social desirability bias against ’not’ having an opinion about a reproductive health issue for the phone group, or to both. Greater understanding of mode effects on self report are needed to accurately reflect adolescents’ attitudes towards reproductive health and health related behaviors. Sources of Support: The Centers for Disease Control and Prevention.
55. TEST-RETEST RELIABILITY OF SELF-REPORTED SEXUAL HISTORY AMONG HETEROSEXUAL YOUTH Sylvie Lombardo, PhD, and Jessica Bukowski. Dept. of Psychology Oakland Univ., Rochester, MI Purpose: Sexual health researchers typically rely on self-reports to assess the prevalence of sexual behaviors and the impact of interventions aimed at reducing sexual risks among youth. Biases due to incorrect recall and desirable responding can introduce errors in predicting individual risk or assessing changes. In addition, available test-retest studies often rely on inadequate assessment periods and the scope of sexual behaviors examined is often limited. Moreover, test-retest studies are rarely partner-specific and rather focus on sexual behaviors occurring within a specific time period. Thus, the goal of the present study is to assess the reliability of self-reported sexual history using partner and time referent methods. Methods: Eighty-two heterosexual male and female college students (age 19-21; 69% females) attending a 4-year college in the Midwest participated in the study. All participants were enrolled in psychology classes and received class credit for their participation. A paper-pencil questionnaire consisting of approximately 300 items was developed for the present study and administered in a classroom setting using a two-week interval (two participants did not return for the retest and were not included in the present analyses). Sexual and contraceptive behaviors occurring in the specific interpersonal context of the last three sexual partners were assessed. In addition, a series of items examined the sexual timetables of sexual behaviors (e.g., heavy petting, oral, vaginal, and anal intercourse) as well as lifetime, 3-month, and 6-month prevalence. In order to maximize the reliability of self-reports and improve accuracy time frames were anchored in specific dates (e.g., ‘‘In the last three months, which is since [date], have you ever.’’) and terms were clearly operationalized using familiar language (e.g., vaginal intercourse means ‘‘penis in the vagina’’). All variables reported in the present analyses were continuous and thus test-retest reliability was computed using intraclass correlations (ICC). Results: Overall, respondents reliably reported the sexual timetables of various sexual behaviors (ICC 0.72-0.96) as well as lifetime (ICC ¼ 0.79-0.98) 3-month (ICC ¼ 0.89-0.96), and 6-month (ICC ¼ 0.84-0.94) prevalence of sexual behaviors. However, the reliability of condom and birth control use in the past 6 months was lower (ICC ¼ 0.59-0.69) than in the past 3 months (ICC ¼ 0.72-0.75). When the assessment of sexual behaviors was partner specific, test-retest reliability was substantially lower than when assessed using specific time frames. Conclusions: Overall substantial test-retest reliability was obtained on various sexual and contraceptive behaviors using specific time
Abstracts / 44 (2009) S14–S47
intervention. A total of 1,049 (97%) had sufficient data for analyses. Respondents completed the 7-item connectedness measure and other questions about the visit immediately post-visit. Participants were asked to indicate on a 5-point scale (1 ¼ ‘‘Not at all’’ to 5 ¼ ‘‘Very much’’) how much they felt the provider cared about them; listened to them; respected them; was someone they could talk to about a problem; would keep what was said confidential; was honest with them; and was judging them (reverse-coded). We examined reliability with Cronbach’s alpha and validity by comparing the connectedness scale score with hypothesized correlates: number of prior visits, satisfaction with the visit, rating of the way the provider gave advice or information, and change in understanding of substance use health risks. Results: Participants had a mean age of 16 + 2 years; 62% were females; 37% were non-White; and 44% had parents with at least a college degree. Connectedness item scores tended to be high, with the proportion of respondents giving ‘‘5’’ ratings ranging from 36% for the ‘‘cares’’ item to 71% for the reverse-coded ‘‘judging.’’ The scale had high internal consistency (alpha ¼ 0.83) although the ‘‘judging’’ item had low correlations with other scale items (r range, 0.12-0.19). Patients with three or more prior visits with the same provider had significantly higher total scores than those with fewer visits (MannWhitney U; p < 0.001), and total scores were correlated with satisfaction with visit (Spearman’s rho ¼ 0.57), rating of how well the provider gave advice (0.30), and reported likelihood of following the provider’s advice (0.53). Youth with higher scores were more likely to report that their understanding of the risks of substance use changed quite a bit or a great deal as a result of the medical visit (OR ¼ 1.07, 95% CI 1.04-1.10). Conclusions: This initial evaluation suggests that this connectedness measure is reliable and valid, although the one negative item, ‘‘judging,’’ needs improvement. Sources of Support: NIH/NIDA R01 DA018848; LEAH #T71MC00009, MCHB, HRSA
54. COMPARING ADOLESCENT RESPONSE BIAS BETWEEN INTERNET AND TELEPHONE SURVEYS Jonathan D. Klein, MD, MPH1, Caryn Graff Havens, MPH, MBA1, and Randall K. Thomas2. 1Dept. of Pediatrics, University of Rochester, Rochester, NY; 2Harris Interactive, Rochester, NY Purpose: To examine variation in response bias between internet vs. telephone survey methods. Methods: We surveyed 15-24-year-olds using: (1) an age and gender quota sample from the Harris Poll Online (n ¼ 859) weighted to represent the US population by age, gender, ethnicity and propensity to be online, and (2) a nationally representative random digit dial cluster sample (n ¼ 609) weighted to represent the US population by region, gender, age, and ethnicity. Scaled response options were included to assess satisficing (choosing the ‘easiest’ answer) and other response bias. Topics include demographics; emergency contraception; sexual behaviors and attitudes; condom use; access; and quality of care and confidentiality. Results: In agree/disagree questions regarding condom use and efficacy, respondents were asked if they strongly agreed, somewhat agreed, agreed/disagreed, somewhat disagreed, or strongly disagreed with statements. The online respondents were between 2 to 3.5 times more likely to select "neither agree/disagree" option than phone respondents for questions on: having unprotected sex (14.7 vs. 5.8%; p < 0.001), whether buying condoms was embarrassing (19.1 vs. 6.7%; p < 0.001), if condoms broke a lot (39.4 vs. 12.0%; p < 0.001), if sex without a condom was worth the risk (12.4 vs. 6.3%; p < 0.001), and though waiting to have sex is a nice idea,
nobody really does (21.6 vs. 7.1%; p < 0.001). Higher endorsement of the neither category was seen regardless of whether the adolescents were sexually experienced or not. Conclusions: Online respondents were more likely to choose not to express an opinion. This may be due to satisficing, choosing the ’easiest’ answer for the online group, or to social desirability bias against ’not’ having an opinion about a reproductive health issue for the phone group, or to both. Greater understanding of mode effects on self report are needed to accurately reflect adolescents’ attitudes towards reproductive health and health related behaviors. Sources of Support: The Centers for Disease Control and Prevention.
55. TEST-RETEST RELIABILITY OF SELF-REPORTED SEXUAL HISTORY AMONG HETEROSEXUAL YOUTH Sylvie Lombardo, PhD, and Jessica Bukowski. Dept. of Psychology Oakland Univ., Rochester, MI Purpose: Sexual health researchers typically rely on self-reports to assess the prevalence of sexual behaviors and the impact of interventions aimed at reducing sexual risks among youth. Biases due to incorrect recall and desirable responding can introduce errors in predicting individual risk or assessing changes. In addition, available test-retest studies often rely on inadequate assessment periods and the scope of sexual behaviors examined is often limited. Moreover, test-retest studies are rarely partner-specific and rather focus on sexual behaviors occurring within a specific time period. Thus, the goal of the present study is to assess the reliability of self-reported sexual history using partner and time referent methods. Methods: Eighty-two heterosexual male and female college students (age 19-21; 69% females) attending a 4-year college in the Midwest participated in the study. All participants were enrolled in psychology classes and received class credit for their participation. A paper-pencil questionnaire consisting of approximately 300 items was developed for the present study and administered in a classroom setting using a two-week interval (two participants did not return for the retest and were not included in the present analyses). Sexual and contraceptive behaviors occurring in the specific interpersonal context of the last three sexual partners were assessed. In addition, a series of items examined the sexual timetables of sexual behaviors (e.g., heavy petting, oral, vaginal, and anal intercourse) as well as lifetime, 3-month, and 6-month prevalence. In order to maximize the reliability of self-reports and improve accuracy time frames were anchored in specific dates (e.g., ‘‘In the last three months, which is since [date], have you ever.’’) and terms were clearly operationalized using familiar language (e.g., vaginal intercourse means ‘‘penis in the vagina’’). All variables reported in the present analyses were continuous and thus test-retest reliability was computed using intraclass correlations (ICC). Results: Overall, respondents reliably reported the sexual timetables of various sexual behaviors (ICC 0.72-0.96) as well as lifetime (ICC ¼ 0.79-0.98) 3-month (ICC ¼ 0.89-0.96), and 6-month (ICC ¼ 0.84-0.94) prevalence of sexual behaviors. However, the reliability of condom and birth control use in the past 6 months was lower (ICC ¼ 0.59-0.69) than in the past 3 months (ICC ¼ 0.72-0.75). When the assessment of sexual behaviors was partner specific, test-retest reliability was substantially lower than when assessed using specific time frames. Conclusions: Overall substantial test-retest reliability was obtained on various sexual and contraceptive behaviors using specific time