- Email: [email protected]
Interviewer effects on self-reported substance use among homeless persons
Addictive
Pergamon
Behawm. Vol. 19, No. I, pp. X3-Y3. lYY4 Copyright :i: lYY4 Elvevier Science Ltd PI-inted in the USA. All right\ reserved
0306-46031Y4 Sh.00 + .oo
INTERVIEWER
EFFECTS ON SELF-REPORTED SUBSTANCE AMONG HOMELESS PERSONS
TIMOTHY
P. JOHNSON
Survey
Research
and JENNIFER
Laboratory,
University
USE
A. PARSONS of Illinois
Abstract - This study examines respondent editing of survey responses to questions regarding substance use. Previous research has identified several types of interviewer effects that may be associated with respondent editing. including direct and social distance effects. Little is known. however, regarding how these potential effects may influence self-reported substance use behavior. Data analyzed for this study were collected as part of a survey of substance use among homeless persons conducted in Cook County, Illinois. A total of 481 respondents were sampled at random in emergency and transitional shelters. soup kitchens, drop-in centers. and single room occupancy (SRO) hotels. Overall, a direct. gender-based effect was observed, with homeless respondents of both genders more willing to reveal substance use behaviors to male interviewers. Some race- and age-related direct effects were also identified, with White and older interviewers obtaining more reports of substance use. These findings yield important evidence regarding respondent editing of self-reported substance use and have implications for the common practice of demographically matching interviewers and respondents.
Editing of responses for social desirability is now recognized as one of the four main stages of the survey response process (Strack & Martin, 1987; Tourangeau & Rasinski, 1988; Willis, Royston, & Bercini, 1991). It is during this stage, researchers believe, that formulated responses are compared with relevant social norms and may be revised or edited in order to be brought into closer agreement with any norms perceived to have been violated by the unedited responses. As a consequence, socially undesirable activities, such as excessive or illicit substance use, may be underestimated in survey interviews (of course, the same process may inflate estimates of socially desirable actions, such as voting). Indeed, several elements of the survey process have been associated with relative under-reporting of substance use, including interview mode (Aquilino & LoSciuto, 1990; Gfroerer & Hughes, 1991), and respondent characteristics (Aquiline, 1992: Johnson, Hougland, & Moore, 1991). Very little information is currently available regarding interviewer effects in substance use surveys, despite the volume of research individually addressing each of these topics. Mensch and Kandel (1988) evaluated interviewer effects on self-reported substance use in a national survey of young adults, concluding that there were “no strong interviewer-respondent interactions for sex or race with regard to reporting of drug use” (p. 117). Schaeffer (1980) found that African-American respondents reported significantly more drinking of alcohol when speaking to White interviewers, but considered this finding inconclusive given small sample sizes. In this study, An earlier version of this paper was presented at the Annual Meeting of the Midwest Association for Public Opinion Research, Chicago, November 1992. Data analyzed in this study was collected as part of contract number 90219 with the State of Illinois Department of Alcoholism and Substance Abuse. The authors would also like to thank Mr. Hyung-sup (Stephen) Kim for his assistance with computer analysis. Requests for reprints should be sent to Timothy P. Johnson. Survey Research Laboratory, University of Illinois, P.O. Box 6905, Chicago, IL 60680. 83
84
several potential examined among abuse.
T. P. JOHNSON
types of interviewer homeless persons,
and J. A. PARSONS
effects on self-reported a population at increased
substance use are risk of substance
Some of the earliest literature regarding homelessness emphasized widespread alcohol use in this population (Bahr & Caplow, 1974; Bogue, 1963: Rice, 1918; Straus, 1946). Research completed in the past several years continues to find high levels of alcohol use and alcoholism in this group (Breakey et al., 1989; Kogcl & Burnam. 1987; Rossi, 1989; Sosin, Colsin, & Grossman, 1988). It is currently estimated that just under half of the homeless population probably have alcohol-related problems (Fischer, 1989; McCarty, Argeriou, Huebner, & Lubran, 1991; Mulkern & Spence, 1984). Several studies have also documented patterns of drug use among the homeless (Breakey et al., 1989; Susser, Struening. & Conover, 1989; Gelberg & Linn, 1989). Estimates suggest that, although drug use may not be as widespread as alcohol abuse among homeless persons. a sizeable minority are nonetheless involved in drug use (Fischer & Breakey, 1991). It should be noted that although much of what is known about substance use among homeless persons comes from survey interviews, little information is available regarding the quality of data collected from this special population.
Previous research has, in general, found several types of interviewer effects on responses to survey questions. These have fallen into three categories: direct effects, social distance effects, and null effects. Direct effects suggest that survey respondents infer interviewer attitudes from observable characteristics, such as age, race, and gender, and adjust their responses where necessary to be more acceptable to the interviewer’s perceived values (Finkel, Guterbock, & Borg, 1991; Groves & Fultz, 1985; Hatchett & Schuman, 19715; Johnson & Delamater, 1976). In practical terms, direct effects are expected to present themselves in the form of a main effect of interviewer characteristics on respondent answers, such that all respondents tend to answer in a similar manner to interviewer characteristics. In contrast, the social distance perspective holds that it is not interviewer chardcteristics alone, but rather the interplay of respondent and interviewer characteristics, that most adequately accounts for interviewer effects. This approach suggests that response editing is associated with increased differences in social group identification between respondent and interviewer (Freeman & Butler, 1976; Landis, Sullivan, & Sheley, 197.5). Contrary to the direct effects model, social distance effects are expected to take the form of a statistical interaction between interviewer and rcspondent characteristics. Thus, respondents would be expected to answer questions differently, depending on whether or not they shared social identities with the interviewer. Most discussions of the social distance hypothesis presume that there is a linear association between the amount of interviewer-respondent social distance and the likelihood of respondent editing of socially undesirable answers. Finally. null interviewer effects are also common. Conventional wisdom suggests that interviewer effects are only possible when the survey question topic is relevant to the personal characteristics in question (Groves. 1989). If this is correct, race-ofinterviewer effects should only be found in response to racially-sensitive questions and interviewer-gender effects should only be observed with respect to gender-
Interviewer
effects
on self-reported
substance
use
85
relevant survey questions (Weiss, 1975). Within this framework, questions unrelated to observed interviewer and respondent characteristics would not be expected to produce interviewer effects. Much of the available research investigating these potential effects has examined them in relation to demographic characteristics of interviewers, including gender, race, and age (Groves, 1989; Sudman & Bradburn, 1974). Of the studies that have examined the effects of interviewer gender, a majority have produced evidence consistent with the direct effects model (Axinn, 1991; Benny, Riesman, & Star, 1956; Brorsson, 1984; Grimes & Hansen, 1984; Groves & Fultz, 1985: Hyman. 1954; Kane & MaCaulay, 1993; Landis, Sullivan and Sheley, 1973; Loewenstein & Varma, 1970; Nealon, 1983). Support for social distance (Delamater, 1974; Schofield, 196s) and null effects (Colombotos, Elinson, & Loewenstein, 1968; Johnson & Moore, 1993; Mensch & Kandel, 1988; Reiss, 1967) of interviewer gender, however, have also been reported. In contrast, research on race-of-interviewer effects has tended to favor the social distance perspective (Campbell, 1981; Cotter, Cohen, & Coulter, 1982; Reese, Danielson, Shoemaker, Chang, & Hsu, 1986; Schaeffer, 1980). This research has consistently demonstrated a pattern in which respondents provide more deferential responses to interviewers of other races, usually only when answering questions regarding race-related topics. Several other studies have also found data consistent with the social distance hypothesis, although they only examined respondents of one race or ethnic group (Anderson, Silver, & Abramson, 1988; Finkel, Guterbock, & Borg, 1991; Hatchett & Schuman, 1975; Hyman, 1954; Schuman & Converse, 1971; Summers & Hammonds, 1966; Weeks & Moore, 1981). While these studies appear consistent with the social distance hypothesis, the absence of a comparison race or ethnic group makes it difficult to assess whether direct effects might actually be operating. A few race-of-interviewer studies reporting findings of direct effects (Loewenstein & Varma, 1970) and null effects (David, 1962; Mensch & Kandel, 1988) are also available, although the topics examined in these studies were not directly related to race issues. Compared to research involving interviewer race and gender effects, very little work has been done to assess the effects of interviewer age on response patterns. The two available studies were completed over 30 years ago. Ehrlich and Riesman (1961) found a social distance effect among young female respondents, who were more open to interviewers with ages closer to those of the respondents. Because only younger females were interviewed, though, it was not possible to rule out the possibility of direct, rather than social distance, effects to interviewer-respondent age differences. An earlier study by Benny, Riesman, and Star (1956) concluded that communication was least inhibited between younger respondents and interviewers of the same gender, and that the most inhibited communication was between people of the same age, but of different genders. METHODS
Data source
The data analyzed in this study were collected as part of a survey of alcohol and other drug use among homeless persons in Cook County, Illinois. Homeless persons were screened and interviewed in emergency and transitional shelters, soup kitchens, drop-in centers, and single room occupancy (SRO) hotels. These facilities
xfl
T. P. JOHNSON
and J. A. PAKSONS
were selected with probability proportional to size, based upon operator estimates of the numbers of persons served at each facility per day. Interviewing took place in a total of 36 facilities. Field work was completed during October and early November, 1990. A total of 48 I face-to-face interviews were completed, and the study’s completion rate (completed interviews/eligible sample) was 78.2%. All respondents were paid $10 for their participation. Interviews were conducted by a staff of 14 interviewers. Of these, I3 were experienced field staff, several of which had previously interviewed homeless persons in conjunction with other studies. The single interviewer with no previous experience was himself a former homeless person who was trained specifically for employment on this study. Interviewers were not always randomly assigned to respondents. Within transitional shelters, soup kitchens, and drop-in centers, interviewing assignments were generally made at random. Within overnight shelters. however, efforts were made to match interviewers with respondents of the same gender whenever possible, given that shelter accommodations were usually segregated by gender, and interviewing necessarily took place during evening hours in these facilities. Interviewers were assigned at random to complete all interviewing within individual SRO hotels included in the sample.
Responses to 15 substance use questions were included in this analysis, including lifetime, last year, and last-30-day use of: (a) alcohol, (b) marijuana, (c) cocaine, (d) other street drugs (including hallucinogens, inhalants, heroin, and “street speed”), and (e) psychotherapeutic drugs (including nonprescription use of analgesics, stimulants, sedatives, and tranquilizers). All substance use questions were worded to be consistent with the 1990 National Household Survey of Drug Abuse (NIDA, 1990). Unweighted data are presented in this analysis. Information was available to assess the effects of three interviewer characteristics: gender, race, and age. A majority of the interviewers were male (71.4%), AfricanAmerican (64.3%), and over the age of 35 (64.3%). Males completed 62.6% of all interviews: African-Americans completed 7 I .9%; and interviewers over the age of 35 completed 68.4% of all interviews. Responses to each substance use question were initially examined separately for: (a) males and females by interviewer gender, (b) African-Americans and Whites by interviewer race (for this analysis, Asian and American Indian respondents were not included), and (c) for young and old respondents by interviewer age. Recognizing that the sample sizes used in this study may have inadequate power to detect some important differences, the nonparametric sign test (Dixon & Massey, 1969) was also used to assess potential trends. Given that substance use is generally viewed as a socially deviant behavior in Western culture, greater self-reports of substance use are considered to be an indicator of less response editing (Bradburn, Sudman, Blair, & Stoching, 1978). Analysis of covariance (ANCOVA) was next used to determine if there were direct and/or interaction effects of interviewer characteristics on substance use reports independent of the effects of respondent characteristics. Previous research has found a number of persona1 characteristics, most notably gender and age, to be related to self-reported drug use (Clayton, Voss, Robbins, & Skinner, 1986; Kandel, 1980).
Interviewer
effects
on self-reported
substance
use
87
Substance use responses were combined into several scales for the multivariate analyses. Five scales were developed to measure overall reported use of each of the five types of substances being examined. For instance, overall reported use of marijuana was measured on a 4-point scale: 0 1 2 3
= = = =
reported reported reported reported
never having used marijuana, lifetime marijuana use only, marijuana use in last year, marijuana use in last 30 days.
Three additional scales were developed to measure total reported substance use during each of the three time frames asked about in the study: lifetime, last year, and last-30-day use. These scales ranged in value from 0 to 5 substances reported to have been used during each of these reporting periods. A final scale measured total reported substance use. This variable had a range of 0 to 15, reflecting the total reported use of all substances across all time periods. A summary measure of respondent-interviewer similarity was also developed as a measure of total social distance. This measure ranged from 0 to 3, with 0 indicating total dissimilarity of demographic characteristics within the dyad (i.e., young African-American male interviewing an older White female), and 3 representing respondent-interviewer congruence in terms of gender, race and age. RESULTS
Bivariate eflects Examination of bivariate interviewer and gender effects revealed that male respondents reported higher rates of substance use to male interviewers than female interviewers in 14 of the 15 comparisons made. A sign test found this distribution to be significant (two-tailed p-value < .OOl). Female respondents also tended to report substance use more to male than female interviewers (12 of I5 comparisons: twotailed sign test p-value = .03). An interesting contrast was observed for females. though. While all drug use reports were greater when interviewed by males (12 of 12), reported alcohol use among women was greater when interviewed by females. The effects of interviewer race on self-reported substance use were also examined. In general, there appeared to be no systematic differences in the answers given by White respondents to White and African-American interviewers. Overall, AfricanAmericans interviewed by Whites reported more substance use for 10 of the 15 comparisons made. A two-tailed sign test of this trend was not significant. Associations between interviewer/respondent age and self-reports revealed that, in 11 of 15 comparisons, higher proportions of the younger respondent group (i.e., those age 18-35) indicated a history of substance use when interviewed by older (i.e., over age 35) interviewers. A two-tailed sign test of this trend was not significant, however. As with younger respondents, comparison of the 15 self-reports of older persons showed that greater proportions reported substance use to older interviewers (in 12 of 15 comparisons). This trend was found to be significant using the sign test (two-tailed p-value = .036). Detailed statistical tables containing the bivariate findings reported in this section are available from the authors.
X8
‘I‘. P. JOHNSON
and J. A. PARSONS
A series of ANCOVAs were next estimated to assess the direct, social distance, and null hypotheses with regards to each demographic characteristic. As previously described, responses to individual substance use questions were aggregated to develop counts of the number of self-reports provided by each respondent. In Table I. responses were aggregated by substance type to produce measures of alcohol, marijuana, cocaine, other street drug, and psychotherapeutic use. Respondent characteristics were strongly associated with reported use of all five substances. AfricanAmericans. males, and younger respondents were each found to be more likely to use most of the substances examined. Consistent with the results presented earlier. several intervicwcr characteristics were also found to be associated with respondent reports. Most notably, interviewer gender was independently related to the likelihood that respondents would reveal marijuana, cocaine, and psychotherapeutic drug USC. For each of these three drugs, substance use was more likely to be reported to males. One effect of interviewer race was also observed, as homeless persons were more likely to report alcohol use to White interviewers. Interviewer age was also related to one substance use report, as older interviewers (over age 35) were more likely to obtain reports of marijuana use than were younger interviewers. Among the respondent-by-interviewer interactions examined. only one reached statistical significance. When reporting alcohol use, male respondents were more likely to admit use to male interviewers. and female respondents were more likely to report use to female interviewers. Substance use reports were also aggregated by recall period for multivariate analysis. These results are presented in the first three columns of Table 2. Respondent gender, ethnic status, and age were found to be significantly associated with likelihood of reporting substance use ever, during the last year, and during the last 30 days. Among interviewer characteristics, gender was associated with likelihood of reporting substance use during each of these time frames. In each case, males were more likely to elicit reports of substance use than were females. Interviewer age was
Alcohol
Marijuana
Cocaine 25.Y)‘-! Y.24’ I I .Y3‘
Respondent Respondent Respondent
race gender age
O.XY 50.74” 0.00
Interviewer Interviewer Interviewer
race gender age
h.l3.> 0.7 I 0.67
0.49 9.63 0.57
0.00 4.79.. 0.5 I
0.03 0.04 0.0 I
Race interaction Gender interaction Age interaction
ii? /v ,‘Partial I--value5 *p C’ .05. *./I J .Ol. i*c[’ .ooi
0. I5 451 arc repel-ted.
35
0. I4 I
Other
8 / i
\treet 12.37’-‘-’ 12.7X’ I.71
dnlg\
Psq~hothenlpruti~~ 6.73. 3.47 -1. I6
0. I7 3.?Y I.3
3.36 1.12 0. IS
I.22 I .12 O.OY
2.44 0.33 0.56
0.07 44x
0.05 145
Interviewer
Table 2. ANCOVA
analysis
effects
on self-reported
substance
89
use
of interviewer and respondent characteristics reports and by reporting period“
on total substance
use
Reports Lifetime
Last year
Last 30 days
Total
Respondent Respondent Respondent
race gender age
21.53*** 38.20**” 16.82”**
18.93*“* 25.65*** 6.88*”
14.52*** 42.61*** 2.92
23.63**” 39.5X*** ,3,-Y,*** i
Interviewer Interviewer Interviewer
race gender age
1.20 10.71*** 4.71*
3.60* 7.95** I .53
0.65 4.39* 4.54*
2.02 8.73** 2.75
I .47 0.03 0.34
0.00 0.12 0.71
0.04 0.60 0.15
0.74 0.11 0.10
0.18 451
0.14 451
0.15 417
0. I9 417
Race interaction Gender interaction Age interaction R’ N “Partial
F-values
are reported
*p < .05. **p i
.Ol.
***p < ,001.
also related to likelihood of reporting lifetime and last-30-day substance use, and interviewer race was related to reports of use during the last year. In these instances, White interviewers and those over the age of 35 were more likely to obtain reports of substance use compared to interviewers who were African-American and/or younger. No interactions between interviewer and respondent characteristics were found in these analyses. The final column of Table 2 examined the measure of substance use that was aggregated over all substances and time frames. When regressed against respondent and interviewer characteristics, this measure provided an overall assessment of interviewer effects on self-reported substance use. The results of this analysis again demonstrated that all three respondent characteristics (gender, race, age) were associated with substance use reports. Among interviewer characteristics, the only effect that was significant across all substance use reports was interviewer gender. Consistent with earlier findings, respondents were more likely to admit substance use to male interviewers. Overall, no direct effects of interviewer race and age were found, and no respondent-by-interviewer interactions were observed. An additional set of analyses (not shown) examined the possibility of an additive effect of social distance on self reports. Similar ANCOVA equations were estimated for each of the dependent variables previously shown in Tables 1 and 2. Respondent characteristics (age, race, gender) were again included as independent variables, along with a measure of degree of demographic similarity between interviewer and respondent. When controlling for respondent characteristics, this measure of total social distance was not found to be significantly associated with self-reported substance use in any of the equations examined. Studies of interviewer effects are often criticized for failure to control for the fact that respondents are clustered by interviewer (Dijkstra, 1983). To address this potential problem, a final set of models (also not shown) were estimated that included error terms for the effects of individual interviewers nested within each of the three inter-
90
T. P. JOHNSON
and J. A. PARSONS
viewer characteristics. In several of these models, this additional parameter was significant, suggesting that substance use reports sometimes varied across interviewers, net of their demographic characteristics. The general findings presented in Tables 1 and 2, however, were unchanged.
II 1 S C U S S I 0 N
This study yields important evidence regarding respondent editing of self-reported substance use. Previously, there has been little direct information regarding this potential source of response error. Overall, a gender-based direct effect was observed, with homeless respondents of both genders more willing to reveal substance use to male interviewers. Several instances of race- and age-related direct effects were also identified, with older interviewers and White interviewers obtaining more reports of substance use. A single instance of a social distance effect was also identified, as both males and females were more likely to report alcohol use to samegender interviewers. A “likely-user” related line of reasoning is proposed to account for the general direct effect of interviewer gender. It is possible that substance users are more likely to reveal their habits to male interviewers, given that male interviewers may be generally regarded as being more likely to be tither current or previous substance users themselves. As a consequence, male interviewers may be expected to have more accepting or at least more tolerant opinions regarding alcohol and other drug use, making it less threatening for respondents to reveal such behavior to them. Given general societal sanctions prohibiting many forms of substance use, it would appear all the more likely that respondents would be more willing to admit substance use to persons viewed as less offended by these behaviors. This explanation. though, would not account for the tendency of older interviewers to obtain greater substance use reports. The possibility that substance use is being over-reported to some interviewers must also be considered. This is an intriguing possibility, given that more affirmative substance use reports were obtained by interviewers with characteristics representative of the authority structure in this country (i.e., those who were older, White, and male). Respondents may have been more likely to tell these interviewers what they perceived the interviewers wanted to hear, perhaps in hopes of eliciting some form of assistance from them. Alternatively, they may merely have been unwilling to provide these symbols of the power structure with accurate information. Moreover, the 4- and S-point drug-use scales used in much of our multivariate analyses may not have been sensitive enough indicators to detect some differences. The social distance effects commonly reported in race-of-interviewer studies were not found in these data. Although substance use is generally regarded as a highly sensitive behavior (Sudman and Bradburn. 1982, p. 84), it does not appear to be racesensitive. Our findings are thus consistent with the general belief that race-of-interviewer effects should only be expected in response to questions that are relevant to this respondent characteristic. There are several important limitations of these data that should be recognized. First, interviewers and respondents were not always randomly assigned to one another. Random assignment should be considered a prerequisite for rigorous evaluation of any response effects in social surveys. In addition, it should be acknowledged that this study is based upon a relatively small sample of interviewers. Also, there is
Interviewer
effects
on self-reported
hubstance
use
91
no guarantee that these findings can be generalized to other populations of survey respondents in substance use surveys. Available evidence, though, does suggest that self-reports from homeless persons are as accurate as those obtained from other disadvantaged populations (Annis, 1979; Bahr & Houts. 1971; Calsyn, Allen, Morse, Smith, & Tempelhoff, 1993). Nonetheless, replication of this research is important. and can be undertaken using virtually any substance-use data set for which both respondent and interviewer characteristics can be identified. It is particularly important to replicate these findings in general household surveys, given the volume of substance use data currently being collected nationally. Finally, these findings have implications for the common practice of matching respondents with interviewers of similar characteristics. Although this practice is widespread, and was to some degree undertaken in this study (see Methods section), there remains considerable disagreement regarding its relative merits. Most adherents recommend matching, in order to minimize one or more of the types of interviewer effects that have been documented in this and past research (Couper. 1991; Groves, 1989; Schaeffer, 1980). Others suggest more caution, recommending random assignment and an emphasis on interviewer training, rather than demographics (Collins, 1980; Freeman & Butler, 1976). Although evidence of an interviewer gender effect was found to be compelling, it is questionable whether the relatively small amount of variation in reported substance use accounted for by this factor would warrant the exclusion of experienced female interviewers from participation in faceto-face substance use surveys of the homeless or any other population. Overemphasizing respondent-interviewer matching would, in the long run, probably do more harm than good.
REFERENCES Anderson, B. A.. Silver, B. D.. & Abramson. P. R. (1988). The effects of the race of the interviewer on race-related attitudes of Black respondents in SRCiCPS national election studies. Plrhlic, Opinion Ql~urterl~. 52, 289-324. Annis. H. M. (1979). Self-report reliability of skid-row alcoholics. British Jo~lrnu/ c$‘Ps~c~hicctr~, 134. 459465. Aquilino. W. S. (1992). Telephone versus face-to-face interviewing for household drug use surveys. Internutionul Jmrrncd o.f the Addic,tions, 27, I I-91. Aquilino. W. S., & LoSciuto. L. A. (1990). Interview mode effects in drug use surveys. Public, Opinion Qucrrterly. 54. 362-395. Axinn, W. G. (1991). The influence of interviewer sex on responses to sensitive questions in Nepal. Socicrl Science Research, 20, 303-318. Bahr. H. M.. & Caplow, T. (1974). O/d men: Drunk und .sober. New York: New York University Press. Bahr. H. M., & Houts, K. C. (1971). Can you trust a homeless man’? A comparison of official records and interview responses by Bowery men. Public Opinim Quarter/y, 35. 374-382. Benny. M.. Riesman. D., & Star, S. A. (19.56). Age and sex in the interview. Anic~ric~r~nJorrrntrl of Socio/ogy, 62. 143-152. Bogue, D. B. (1963). Skid rm in Americ,m cirics. Chicago: Community and Family Center, University of Chicago. Bradburn, N. M.. Sudman, S., Blair, E., & Stoching. C. (1978). Question threat and response bias. Puhlk Opinion Quurterly. 42. 22 l-234. Breakey, W. R., Fischer. P. J., Kramer. G., Nestadt, G.. Romanoski, A. J., Ross. A., Royall. R. M.. & Stine, 0. C. (1989). Health and mental health problems of homeless men and women in Baltimore. Journal of thr American Medical Assoc,icrtion. 262. 1352-1357. Brorsson. B. (1984). Effects of interviewer characteristics and interviewer variability on interview responses. In C. F. Cannell and R. M. Groves (Ed?.). N~~tr/r/r.s~rrucyre.~wrc/~ ~~~c~t/~oc/s: Procc~edirr~.~of the fourrh Cmfirc~ncr on Hccrlth Suruey Rr.tc~rlrc~hMerhod~ (DHHS Publication No. PHS 84.3346). Washington. DC: U.S. Government Printing Office. Calsyn, R. J., Allen, G., Morse. G. A., Smith. R.. & Tempelhoff, B. (1993) Can you trust self-report data provided by homeless mentally ill individuals‘? Eutrlrrcrtion Rruietr,, 17, 353-366.
Y2
T. I’. JOHNSON
and J. A. PAKSONS
Campbell. B. A. (19X I). Kate-o-tntervtewer effects among southern adolescents. Public, O/~iniorr Qtrtrrtel/~. 45, 23 I-244. Clayton, K. K.. Voss. H. L.. R&bins. C. & Skinner. W. F. t IYXh). Gender differences in drug use: An epidemiological perspective. In B. A. Ray and M. C. Braude (Ed\.). W~~nre,! trnd drrr,qs: A neu‘ er(~ fogrcsc~trrclr (DHHS Publication No. ADM X6-1447. Research Monograph Series No. 65. pp. X0-YY). Kockville. MD: National Institute on Drug Abuse. Collins. M. (19X0). Interviewer variability: A review of the problem. ./o/fr/ltr/ c!f’ r/tr Mtrrr(e/ K~,.\c,trrc~/f soc~ic~t.v.22. 77-95. Colombotos. J.. Elinson. J.. Xr Loewen\tein. K. (196X). Effect of interviewers’ sex on interviewer responses. P/rh/ic Hctrlth Rcpor.t.\. 83. 6X5-6YO. Cotter. P. K., Cohen. J.. & Coulter, I’. B. (1982). Race-of-interviewer effects in telephone Interviews. Pohlic. Opiuiotl Qcctcrlarly. 46. 27X-286. Couper, M. P. ( 1991 ). Modeling survey participation at the interviewer Icvel. Proccc~dir~,q.~r!f’t/rc Sec.tion
on Swvcy
Resccrrc~h Method.\,
A~~rrrictrt~ .S/utivticcr/ A.\.soc~irrtion. 9% 107.
David, M. ( 1962). The validity of income reported by a sample of families who received welfare assistance during 19%. Jorm7trl c(f //I(> Atnc,ricrr/~ S~aris~ic~rrl A.~.~oc~ic~lion.57. 680-685. Delamater. J. D. (1974). Methodological issues in the study of premarital sexuality. .S~~io/o~ic~c~/ ,M~~thot/.~ & Rr.\c,rrrc~/r. 3, 30-61. Dijkstra. W. (19X3). How interviewer variance can bias the results of research on interviewer effects. @di/~ trrrd grrrrntity. 17, 17Y-187. Dixon. W. J.. R Massey, F. J. (1969). Intrrx/r~~tion to .srcr/i.c/ic.c~/trncr/yvi.\ (3rd ed. t. New York: McG,-awHill. Ehrlich. J. S.. 8: Kiesman. I>. (1961). Age and authority in the interview. P/lh/rc, Opiniorr Q/~ortcr/\.. 25. 39-56. Finkel, S. E., Guterbock. T. M., 6i Borg, M. J. (1991). Race-of-interviewer effects in a preelection poll: Virginia 19x9. Plrhlic, Opinion Qtrtrrtcr!\.. 55. 3 13-330. Fischer. P. J. (1989). Estimating the prevalence of alcohol. drug, and mental health problems in the contemporary homeless population: A review of the literature. Co/r/c,ntportrrv Drrlg f’rohlr~ns. 16. 333-390. Fischer, P. J., & Breakey. W. K. (1991). The epidemiology of alcohol. drug, and mental disorders among homeless persons. Amc~rictrn P,s~~~l~~/o,qi.~f.46. I I I-I 12X. Freeman, J., & Butler. E. W. (1976). Some sources of interviewer variance tn surveys. P/rh/ic, Opiniorr QMll./CI.!\.. 40. 79-92. Gelberg. L.. & Linn. L. S. (1989). Assessing the physical health of homeless adults. Jorr~,rtr/ of’ fhe Antcric.trn Mec/icci/ A.\.\o&ttio/r, 262. 1973-1979. Gfroerer. J. C., & Hughes. A. L. (IYYI). The feasibility of collecting drug abuse data by telephone. Public Hroltk Rcycwr.s. 106, 3X4-393. Grimes. M. D.. & Hansen. G. L. (1984). Response bias in sex-role attitude measurement. SPY Ro/c.s. 10. 67-72. Groves. R. M. (I 989). Sltr-vev e1’1’or.\ trf~d .s,rruc~~cosf.\. New York: John Wiley & Sons. Groves, R. M.. & Fultz. N. H. (1985).Gender effects among telephone interviews in a survey of economic attitudes. Soc~io/o~ic~r~/ Merizods &I Rcsc~tr~h, 14, 3 l-52. Hatchett. S., & Schuman, H. (1975). White respondents and race-of-interviewer effects. Public, Opitrior~ QIIlII.IPI.!\., 39. 523-52X. Hyman. H. ( 1954). Irrrrrvicwir~g in .soc,itr/ rc.~cwc~/z. Chicago: University of Chicago Press. Johnson. T. P.. Hougland. J. G.. & Moore. R. W. (I991 ). Sex differences in reporting sensitive behavior: A comparison of interview methods. SC,.\-Roles. 24. 669-680. Johnson. T. P.. & Moore. K. W. (1993). Gender interactions between interviewer and survey respondents: ls\ues of pornography and community standards. Se.\- Rolrs. 28. I-19. Johnson. W. T.. Delamater. J. D. (1976). Response effects in sex surveys. Pclh/ic~ Opirrirm C)~/urt~r/\.. 40. 16%181. Kandel. D. B. (19X0).Drug and drinking behavior among youth. Atrtrrrrrl Rwim ~~t’.S~~io/op~. 6. X5-285. Kane. E. W.. MaCaulay. L. J. (1993) Interviewer gender and gender attitudes. Prthlic, Opinir>fr Q/f(!r/cr/\.. 57. l-28. Kogel, P.. & Burnam. A. ( 1987). Traditional and nontraditional homeless alcoholics. A/~~oho/ Hcrrlrlr trtrd Landis.
J. K.. Sullivan.
D.. & Shelev. J. (1973). Feminist attitudes as related to sex of the interviewer. 16.’ 305-3 14. Loewenstein. K.. & Varma. .4. (1970). Effect of interaction of interviewers and respondents in health surveys. P~rhlk Opiniot~ L)u~irtw!\‘. 34. 472. McCarty, D.. Argeriou. M.. Huebner. R. B.. & Lubran. B. (1991). Alcoholism, drug abuse. and the homeless. A!?7eric~n Ps~cl~o/ogi.t/. 46. I 139-4X. Mensch. B. S.. & Kandel. D. B. (19RX). Youths‘ underreporting of substance use. Public Opirlim Qrrc~r1d\~. 52. 100-124.
Pac,ific Soc.io/o~ic~u/ Rcuicw,
Interviewer
effects
on self-reported
substance
use
93
Mulkern,
V., & Spence, R. (1984). Alcohol ubuselulcholism among homeless persons: A review, of the Rockville, MD: National Institute of Alcohol Abuse and Alcoholism. National Institute on Drug Abuse [NIDA]. (1990). Nurionul household .srrrue~ on drng trhr~e: Main jinding.s 1990 (DHHS Publication No. ADM 91-1788). Rockville. MD: Author. Nealon, J. (1983). The effects of male vs. female telephone interviewers. Proceedings of the Section on Survey Research Methods. Americun Statisticul Associution, 139-141. Reese, S. D., Danielson, W. A., Shoemaker, P. J.. Chang, T.-K., & Hsu. H.-L. (1986). Ethnicity-ofinterviewer effects among Mexican-Americans and Angles. Public Opinion Quurterly, 50, 563-572. Reiss, I. L. (1967). The sociu/ context ofpremuritul sesucrlpermissiuerI4s. New York: Holt, Rinehart and Winston. Rice, S. A. (1918). The homeless. Annuls. 77. 140-153. Rossi. P. H. (1989). Dobvn and out in Americuc The origins of home/es.sne.ss, Chicago: University of Chicago Press. Schaeffer, N. C. (1980). Evaluating race-of-interviewer effects in a national survey. Soc~iologicul Merhods literuture.
& Reseurch,
Schofield, Schuman. 1968. Sosin. M.
8, 400-419.
M. (1965). The sex/Au/ behuuiorrr ofyoung people. London: Longmans. H.. & Converse. J. (1971). The effects of Black and White interviewers Public,
Opinion
Quur/er/y,
Green. on Black responses
in
35, 44-68.
S. (1988). Homelessness in Chicugo: Pouerty und pathology, Chicago: Chicago Community Trust. Strack. F.. & Martin. L. L. (1987). Thinking, judging. and communicating: A process account of context effects in attitude surveys. In H.-J. Hippler. N. Schwarz, S. Sudman (Eds.). Sociu/ irzformution processing und survey methodology (pp. 123-148). New York: Springer-Verlag. Straus. R. (1946). Alcohol and the homeless man. Qucrrter/y Journul of Sfudies on A/coho/. 7. 360-404. Sudman, S., & Bradburn. N. M. (1974). Response qff‘ects in Anrueys. Chicago: Aldine. Sudman. S., & Bradburn, N. M. (1982). A.tking yuesrions: A prucfkul guide to yrtestionnuire design. San Francisco: Jossey-Bass. Summers. G., & Hammonds, A. (1966). Effect of racial characteristics of investigator on self-enumerated responses to a Negro prejudice scale. Sociul Force.5, 48, 515-5 18. Susser, S., Struening, E. L., & Conover. S. (1989). Psychiatric problems in homeless men: Lifetime psychosis, substance use and current distress in new arrivals at New York City shelters. Arc,hiues of sociu/
R., Colsin,
P., & Grossman.
institutions
und social
Generul
Psychiutry,
change.
46, 845-850.
Tourdngeau, R., & Rasinski, K. A. (1988). Cognitive processes measurement. Psycho/o&u/ Bulletin, 103, 299-314. Weeks, M. F., & Moore, R. P. (1981). Ethnicity-of-interviewer Opinion
Weiss.
Quurtedy,
C. H. (1975).
underlying effects
context
on ethnic
effects respondents.
in attitude Pub/it
45, 245-249.
in evaluation research. In E. L. Struening & M. Guttentag (Eds.). (Vol. I. pp. 355-395). Beverly Hills, CA: Sage. Willis. G. B., Royston, P., & Bercini, D. (1991). The use of verbal report methods in the development and testing of survey questionnaires. Applied Cognifiue Ps,vchn/o~y. 5. 251-267. Hundhook
Interviewing
of euuluution
reseurch
Pergamon
Behawm. Vol. 19, No. I, pp. X3-Y3. lYY4 Copyright :i: lYY4 Elvevier Science Ltd PI-inted in the USA. All right\ reserved
0306-46031Y4 Sh.00 + .oo
INTERVIEWER
EFFECTS ON SELF-REPORTED SUBSTANCE AMONG HOMELESS PERSONS
TIMOTHY
P. JOHNSON
Survey
Research
and JENNIFER
Laboratory,
University
USE
A. PARSONS of Illinois
Abstract - This study examines respondent editing of survey responses to questions regarding substance use. Previous research has identified several types of interviewer effects that may be associated with respondent editing. including direct and social distance effects. Little is known. however, regarding how these potential effects may influence self-reported substance use behavior. Data analyzed for this study were collected as part of a survey of substance use among homeless persons conducted in Cook County, Illinois. A total of 481 respondents were sampled at random in emergency and transitional shelters. soup kitchens, drop-in centers. and single room occupancy (SRO) hotels. Overall, a direct. gender-based effect was observed, with homeless respondents of both genders more willing to reveal substance use behaviors to male interviewers. Some race- and age-related direct effects were also identified, with White and older interviewers obtaining more reports of substance use. These findings yield important evidence regarding respondent editing of self-reported substance use and have implications for the common practice of demographically matching interviewers and respondents.
Editing of responses for social desirability is now recognized as one of the four main stages of the survey response process (Strack & Martin, 1987; Tourangeau & Rasinski, 1988; Willis, Royston, & Bercini, 1991). It is during this stage, researchers believe, that formulated responses are compared with relevant social norms and may be revised or edited in order to be brought into closer agreement with any norms perceived to have been violated by the unedited responses. As a consequence, socially undesirable activities, such as excessive or illicit substance use, may be underestimated in survey interviews (of course, the same process may inflate estimates of socially desirable actions, such as voting). Indeed, several elements of the survey process have been associated with relative under-reporting of substance use, including interview mode (Aquilino & LoSciuto, 1990; Gfroerer & Hughes, 1991), and respondent characteristics (Aquiline, 1992: Johnson, Hougland, & Moore, 1991). Very little information is currently available regarding interviewer effects in substance use surveys, despite the volume of research individually addressing each of these topics. Mensch and Kandel (1988) evaluated interviewer effects on self-reported substance use in a national survey of young adults, concluding that there were “no strong interviewer-respondent interactions for sex or race with regard to reporting of drug use” (p. 117). Schaeffer (1980) found that African-American respondents reported significantly more drinking of alcohol when speaking to White interviewers, but considered this finding inconclusive given small sample sizes. In this study, An earlier version of this paper was presented at the Annual Meeting of the Midwest Association for Public Opinion Research, Chicago, November 1992. Data analyzed in this study was collected as part of contract number 90219 with the State of Illinois Department of Alcoholism and Substance Abuse. The authors would also like to thank Mr. Hyung-sup (Stephen) Kim for his assistance with computer analysis. Requests for reprints should be sent to Timothy P. Johnson. Survey Research Laboratory, University of Illinois, P.O. Box 6905, Chicago, IL 60680. 83
84
several potential examined among abuse.
T. P. JOHNSON
types of interviewer homeless persons,
and J. A. PARSONS
effects on self-reported a population at increased
substance use are risk of substance
Some of the earliest literature regarding homelessness emphasized widespread alcohol use in this population (Bahr & Caplow, 1974; Bogue, 1963: Rice, 1918; Straus, 1946). Research completed in the past several years continues to find high levels of alcohol use and alcoholism in this group (Breakey et al., 1989; Kogcl & Burnam. 1987; Rossi, 1989; Sosin, Colsin, & Grossman, 1988). It is currently estimated that just under half of the homeless population probably have alcohol-related problems (Fischer, 1989; McCarty, Argeriou, Huebner, & Lubran, 1991; Mulkern & Spence, 1984). Several studies have also documented patterns of drug use among the homeless (Breakey et al., 1989; Susser, Struening. & Conover, 1989; Gelberg & Linn, 1989). Estimates suggest that, although drug use may not be as widespread as alcohol abuse among homeless persons. a sizeable minority are nonetheless involved in drug use (Fischer & Breakey, 1991). It should be noted that although much of what is known about substance use among homeless persons comes from survey interviews, little information is available regarding the quality of data collected from this special population.
Previous research has, in general, found several types of interviewer effects on responses to survey questions. These have fallen into three categories: direct effects, social distance effects, and null effects. Direct effects suggest that survey respondents infer interviewer attitudes from observable characteristics, such as age, race, and gender, and adjust their responses where necessary to be more acceptable to the interviewer’s perceived values (Finkel, Guterbock, & Borg, 1991; Groves & Fultz, 1985; Hatchett & Schuman, 19715; Johnson & Delamater, 1976). In practical terms, direct effects are expected to present themselves in the form of a main effect of interviewer characteristics on respondent answers, such that all respondents tend to answer in a similar manner to interviewer characteristics. In contrast, the social distance perspective holds that it is not interviewer chardcteristics alone, but rather the interplay of respondent and interviewer characteristics, that most adequately accounts for interviewer effects. This approach suggests that response editing is associated with increased differences in social group identification between respondent and interviewer (Freeman & Butler, 1976; Landis, Sullivan, & Sheley, 197.5). Contrary to the direct effects model, social distance effects are expected to take the form of a statistical interaction between interviewer and rcspondent characteristics. Thus, respondents would be expected to answer questions differently, depending on whether or not they shared social identities with the interviewer. Most discussions of the social distance hypothesis presume that there is a linear association between the amount of interviewer-respondent social distance and the likelihood of respondent editing of socially undesirable answers. Finally. null interviewer effects are also common. Conventional wisdom suggests that interviewer effects are only possible when the survey question topic is relevant to the personal characteristics in question (Groves. 1989). If this is correct, race-ofinterviewer effects should only be found in response to racially-sensitive questions and interviewer-gender effects should only be observed with respect to gender-
Interviewer
effects
on self-reported
substance
use
85
relevant survey questions (Weiss, 1975). Within this framework, questions unrelated to observed interviewer and respondent characteristics would not be expected to produce interviewer effects. Much of the available research investigating these potential effects has examined them in relation to demographic characteristics of interviewers, including gender, race, and age (Groves, 1989; Sudman & Bradburn, 1974). Of the studies that have examined the effects of interviewer gender, a majority have produced evidence consistent with the direct effects model (Axinn, 1991; Benny, Riesman, & Star, 1956; Brorsson, 1984; Grimes & Hansen, 1984; Groves & Fultz, 1985: Hyman. 1954; Kane & MaCaulay, 1993; Landis, Sullivan and Sheley, 1973; Loewenstein & Varma, 1970; Nealon, 1983). Support for social distance (Delamater, 1974; Schofield, 196s) and null effects (Colombotos, Elinson, & Loewenstein, 1968; Johnson & Moore, 1993; Mensch & Kandel, 1988; Reiss, 1967) of interviewer gender, however, have also been reported. In contrast, research on race-of-interviewer effects has tended to favor the social distance perspective (Campbell, 1981; Cotter, Cohen, & Coulter, 1982; Reese, Danielson, Shoemaker, Chang, & Hsu, 1986; Schaeffer, 1980). This research has consistently demonstrated a pattern in which respondents provide more deferential responses to interviewers of other races, usually only when answering questions regarding race-related topics. Several other studies have also found data consistent with the social distance hypothesis, although they only examined respondents of one race or ethnic group (Anderson, Silver, & Abramson, 1988; Finkel, Guterbock, & Borg, 1991; Hatchett & Schuman, 1975; Hyman, 1954; Schuman & Converse, 1971; Summers & Hammonds, 1966; Weeks & Moore, 1981). While these studies appear consistent with the social distance hypothesis, the absence of a comparison race or ethnic group makes it difficult to assess whether direct effects might actually be operating. A few race-of-interviewer studies reporting findings of direct effects (Loewenstein & Varma, 1970) and null effects (David, 1962; Mensch & Kandel, 1988) are also available, although the topics examined in these studies were not directly related to race issues. Compared to research involving interviewer race and gender effects, very little work has been done to assess the effects of interviewer age on response patterns. The two available studies were completed over 30 years ago. Ehrlich and Riesman (1961) found a social distance effect among young female respondents, who were more open to interviewers with ages closer to those of the respondents. Because only younger females were interviewed, though, it was not possible to rule out the possibility of direct, rather than social distance, effects to interviewer-respondent age differences. An earlier study by Benny, Riesman, and Star (1956) concluded that communication was least inhibited between younger respondents and interviewers of the same gender, and that the most inhibited communication was between people of the same age, but of different genders. METHODS
Data source
The data analyzed in this study were collected as part of a survey of alcohol and other drug use among homeless persons in Cook County, Illinois. Homeless persons were screened and interviewed in emergency and transitional shelters, soup kitchens, drop-in centers, and single room occupancy (SRO) hotels. These facilities
xfl
T. P. JOHNSON
and J. A. PAKSONS
were selected with probability proportional to size, based upon operator estimates of the numbers of persons served at each facility per day. Interviewing took place in a total of 36 facilities. Field work was completed during October and early November, 1990. A total of 48 I face-to-face interviews were completed, and the study’s completion rate (completed interviews/eligible sample) was 78.2%. All respondents were paid $10 for their participation. Interviews were conducted by a staff of 14 interviewers. Of these, I3 were experienced field staff, several of which had previously interviewed homeless persons in conjunction with other studies. The single interviewer with no previous experience was himself a former homeless person who was trained specifically for employment on this study. Interviewers were not always randomly assigned to respondents. Within transitional shelters, soup kitchens, and drop-in centers, interviewing assignments were generally made at random. Within overnight shelters. however, efforts were made to match interviewers with respondents of the same gender whenever possible, given that shelter accommodations were usually segregated by gender, and interviewing necessarily took place during evening hours in these facilities. Interviewers were assigned at random to complete all interviewing within individual SRO hotels included in the sample.
Responses to 15 substance use questions were included in this analysis, including lifetime, last year, and last-30-day use of: (a) alcohol, (b) marijuana, (c) cocaine, (d) other street drugs (including hallucinogens, inhalants, heroin, and “street speed”), and (e) psychotherapeutic drugs (including nonprescription use of analgesics, stimulants, sedatives, and tranquilizers). All substance use questions were worded to be consistent with the 1990 National Household Survey of Drug Abuse (NIDA, 1990). Unweighted data are presented in this analysis. Information was available to assess the effects of three interviewer characteristics: gender, race, and age. A majority of the interviewers were male (71.4%), AfricanAmerican (64.3%), and over the age of 35 (64.3%). Males completed 62.6% of all interviews: African-Americans completed 7 I .9%; and interviewers over the age of 35 completed 68.4% of all interviews. Responses to each substance use question were initially examined separately for: (a) males and females by interviewer gender, (b) African-Americans and Whites by interviewer race (for this analysis, Asian and American Indian respondents were not included), and (c) for young and old respondents by interviewer age. Recognizing that the sample sizes used in this study may have inadequate power to detect some important differences, the nonparametric sign test (Dixon & Massey, 1969) was also used to assess potential trends. Given that substance use is generally viewed as a socially deviant behavior in Western culture, greater self-reports of substance use are considered to be an indicator of less response editing (Bradburn, Sudman, Blair, & Stoching, 1978). Analysis of covariance (ANCOVA) was next used to determine if there were direct and/or interaction effects of interviewer characteristics on substance use reports independent of the effects of respondent characteristics. Previous research has found a number of persona1 characteristics, most notably gender and age, to be related to self-reported drug use (Clayton, Voss, Robbins, & Skinner, 1986; Kandel, 1980).
Interviewer
effects
on self-reported
substance
use
87
Substance use responses were combined into several scales for the multivariate analyses. Five scales were developed to measure overall reported use of each of the five types of substances being examined. For instance, overall reported use of marijuana was measured on a 4-point scale: 0 1 2 3
= = = =
reported reported reported reported
never having used marijuana, lifetime marijuana use only, marijuana use in last year, marijuana use in last 30 days.
Three additional scales were developed to measure total reported substance use during each of the three time frames asked about in the study: lifetime, last year, and last-30-day use. These scales ranged in value from 0 to 5 substances reported to have been used during each of these reporting periods. A final scale measured total reported substance use. This variable had a range of 0 to 15, reflecting the total reported use of all substances across all time periods. A summary measure of respondent-interviewer similarity was also developed as a measure of total social distance. This measure ranged from 0 to 3, with 0 indicating total dissimilarity of demographic characteristics within the dyad (i.e., young African-American male interviewing an older White female), and 3 representing respondent-interviewer congruence in terms of gender, race and age. RESULTS
Bivariate eflects Examination of bivariate interviewer and gender effects revealed that male respondents reported higher rates of substance use to male interviewers than female interviewers in 14 of the 15 comparisons made. A sign test found this distribution to be significant (two-tailed p-value < .OOl). Female respondents also tended to report substance use more to male than female interviewers (12 of I5 comparisons: twotailed sign test p-value = .03). An interesting contrast was observed for females. though. While all drug use reports were greater when interviewed by males (12 of 12), reported alcohol use among women was greater when interviewed by females. The effects of interviewer race on self-reported substance use were also examined. In general, there appeared to be no systematic differences in the answers given by White respondents to White and African-American interviewers. Overall, AfricanAmericans interviewed by Whites reported more substance use for 10 of the 15 comparisons made. A two-tailed sign test of this trend was not significant. Associations between interviewer/respondent age and self-reports revealed that, in 11 of 15 comparisons, higher proportions of the younger respondent group (i.e., those age 18-35) indicated a history of substance use when interviewed by older (i.e., over age 35) interviewers. A two-tailed sign test of this trend was not significant, however. As with younger respondents, comparison of the 15 self-reports of older persons showed that greater proportions reported substance use to older interviewers (in 12 of 15 comparisons). This trend was found to be significant using the sign test (two-tailed p-value = .036). Detailed statistical tables containing the bivariate findings reported in this section are available from the authors.
X8
‘I‘. P. JOHNSON
and J. A. PARSONS
A series of ANCOVAs were next estimated to assess the direct, social distance, and null hypotheses with regards to each demographic characteristic. As previously described, responses to individual substance use questions were aggregated to develop counts of the number of self-reports provided by each respondent. In Table I. responses were aggregated by substance type to produce measures of alcohol, marijuana, cocaine, other street drug, and psychotherapeutic use. Respondent characteristics were strongly associated with reported use of all five substances. AfricanAmericans. males, and younger respondents were each found to be more likely to use most of the substances examined. Consistent with the results presented earlier. several intervicwcr characteristics were also found to be associated with respondent reports. Most notably, interviewer gender was independently related to the likelihood that respondents would reveal marijuana, cocaine, and psychotherapeutic drug USC. For each of these three drugs, substance use was more likely to be reported to males. One effect of interviewer race was also observed, as homeless persons were more likely to report alcohol use to White interviewers. Interviewer age was also related to one substance use report, as older interviewers (over age 35) were more likely to obtain reports of marijuana use than were younger interviewers. Among the respondent-by-interviewer interactions examined. only one reached statistical significance. When reporting alcohol use, male respondents were more likely to admit use to male interviewers. and female respondents were more likely to report use to female interviewers. Substance use reports were also aggregated by recall period for multivariate analysis. These results are presented in the first three columns of Table 2. Respondent gender, ethnic status, and age were found to be significantly associated with likelihood of reporting substance use ever, during the last year, and during the last 30 days. Among interviewer characteristics, gender was associated with likelihood of reporting substance use during each of these time frames. In each case, males were more likely to elicit reports of substance use than were females. Interviewer age was
Alcohol
Marijuana
Cocaine 25.Y)‘-! Y.24’ I I .Y3‘
Respondent Respondent Respondent
race gender age
O.XY 50.74” 0.00
Interviewer Interviewer Interviewer
race gender age
h.l3.> 0.7 I 0.67
0.49 9.63 0.57
0.00 4.79.. 0.5 I
0.03 0.04 0.0 I
Race interaction Gender interaction Age interaction
ii? /v ,‘Partial I--value5 *p C’ .05. *./I J .Ol. i*c[’ .ooi
0. I5 451 arc repel-ted.
35
0. I4 I
Other
8 / i
\treet 12.37’-‘-’ 12.7X’ I.71
dnlg\
Psq~hothenlpruti~~ 6.73. 3.47 -1. I6
0. I7 3.?Y I.3
3.36 1.12 0. IS
I.22 I .12 O.OY
2.44 0.33 0.56
0.07 44x
0.05 145
Interviewer
Table 2. ANCOVA
analysis
effects
on self-reported
substance
89
use
of interviewer and respondent characteristics reports and by reporting period“
on total substance
use
Reports Lifetime
Last year
Last 30 days
Total
Respondent Respondent Respondent
race gender age
21.53*** 38.20**” 16.82”**
18.93*“* 25.65*** 6.88*”
14.52*** 42.61*** 2.92
23.63**” 39.5X*** ,3,-Y,*** i
Interviewer Interviewer Interviewer
race gender age
1.20 10.71*** 4.71*
3.60* 7.95** I .53
0.65 4.39* 4.54*
2.02 8.73** 2.75
I .47 0.03 0.34
0.00 0.12 0.71
0.04 0.60 0.15
0.74 0.11 0.10
0.18 451
0.14 451
0.15 417
0. I9 417
Race interaction Gender interaction Age interaction R’ N “Partial
F-values
are reported
*p < .05. **p i
.Ol.
***p < ,001.
also related to likelihood of reporting lifetime and last-30-day substance use, and interviewer race was related to reports of use during the last year. In these instances, White interviewers and those over the age of 35 were more likely to obtain reports of substance use compared to interviewers who were African-American and/or younger. No interactions between interviewer and respondent characteristics were found in these analyses. The final column of Table 2 examined the measure of substance use that was aggregated over all substances and time frames. When regressed against respondent and interviewer characteristics, this measure provided an overall assessment of interviewer effects on self-reported substance use. The results of this analysis again demonstrated that all three respondent characteristics (gender, race, age) were associated with substance use reports. Among interviewer characteristics, the only effect that was significant across all substance use reports was interviewer gender. Consistent with earlier findings, respondents were more likely to admit substance use to male interviewers. Overall, no direct effects of interviewer race and age were found, and no respondent-by-interviewer interactions were observed. An additional set of analyses (not shown) examined the possibility of an additive effect of social distance on self reports. Similar ANCOVA equations were estimated for each of the dependent variables previously shown in Tables 1 and 2. Respondent characteristics (age, race, gender) were again included as independent variables, along with a measure of degree of demographic similarity between interviewer and respondent. When controlling for respondent characteristics, this measure of total social distance was not found to be significantly associated with self-reported substance use in any of the equations examined. Studies of interviewer effects are often criticized for failure to control for the fact that respondents are clustered by interviewer (Dijkstra, 1983). To address this potential problem, a final set of models (also not shown) were estimated that included error terms for the effects of individual interviewers nested within each of the three inter-
90
T. P. JOHNSON
and J. A. PARSONS
viewer characteristics. In several of these models, this additional parameter was significant, suggesting that substance use reports sometimes varied across interviewers, net of their demographic characteristics. The general findings presented in Tables 1 and 2, however, were unchanged.
II 1 S C U S S I 0 N
This study yields important evidence regarding respondent editing of self-reported substance use. Previously, there has been little direct information regarding this potential source of response error. Overall, a gender-based direct effect was observed, with homeless respondents of both genders more willing to reveal substance use to male interviewers. Several instances of race- and age-related direct effects were also identified, with older interviewers and White interviewers obtaining more reports of substance use. A single instance of a social distance effect was also identified, as both males and females were more likely to report alcohol use to samegender interviewers. A “likely-user” related line of reasoning is proposed to account for the general direct effect of interviewer gender. It is possible that substance users are more likely to reveal their habits to male interviewers, given that male interviewers may be generally regarded as being more likely to be tither current or previous substance users themselves. As a consequence, male interviewers may be expected to have more accepting or at least more tolerant opinions regarding alcohol and other drug use, making it less threatening for respondents to reveal such behavior to them. Given general societal sanctions prohibiting many forms of substance use, it would appear all the more likely that respondents would be more willing to admit substance use to persons viewed as less offended by these behaviors. This explanation. though, would not account for the tendency of older interviewers to obtain greater substance use reports. The possibility that substance use is being over-reported to some interviewers must also be considered. This is an intriguing possibility, given that more affirmative substance use reports were obtained by interviewers with characteristics representative of the authority structure in this country (i.e., those who were older, White, and male). Respondents may have been more likely to tell these interviewers what they perceived the interviewers wanted to hear, perhaps in hopes of eliciting some form of assistance from them. Alternatively, they may merely have been unwilling to provide these symbols of the power structure with accurate information. Moreover, the 4- and S-point drug-use scales used in much of our multivariate analyses may not have been sensitive enough indicators to detect some differences. The social distance effects commonly reported in race-of-interviewer studies were not found in these data. Although substance use is generally regarded as a highly sensitive behavior (Sudman and Bradburn. 1982, p. 84), it does not appear to be racesensitive. Our findings are thus consistent with the general belief that race-of-interviewer effects should only be expected in response to questions that are relevant to this respondent characteristic. There are several important limitations of these data that should be recognized. First, interviewers and respondents were not always randomly assigned to one another. Random assignment should be considered a prerequisite for rigorous evaluation of any response effects in social surveys. In addition, it should be acknowledged that this study is based upon a relatively small sample of interviewers. Also, there is
Interviewer
effects
on self-reported
hubstance
use
91
no guarantee that these findings can be generalized to other populations of survey respondents in substance use surveys. Available evidence, though, does suggest that self-reports from homeless persons are as accurate as those obtained from other disadvantaged populations (Annis, 1979; Bahr & Houts. 1971; Calsyn, Allen, Morse, Smith, & Tempelhoff, 1993). Nonetheless, replication of this research is important. and can be undertaken using virtually any substance-use data set for which both respondent and interviewer characteristics can be identified. It is particularly important to replicate these findings in general household surveys, given the volume of substance use data currently being collected nationally. Finally, these findings have implications for the common practice of matching respondents with interviewers of similar characteristics. Although this practice is widespread, and was to some degree undertaken in this study (see Methods section), there remains considerable disagreement regarding its relative merits. Most adherents recommend matching, in order to minimize one or more of the types of interviewer effects that have been documented in this and past research (Couper. 1991; Groves, 1989; Schaeffer, 1980). Others suggest more caution, recommending random assignment and an emphasis on interviewer training, rather than demographics (Collins, 1980; Freeman & Butler, 1976). Although evidence of an interviewer gender effect was found to be compelling, it is questionable whether the relatively small amount of variation in reported substance use accounted for by this factor would warrant the exclusion of experienced female interviewers from participation in faceto-face substance use surveys of the homeless or any other population. Overemphasizing respondent-interviewer matching would, in the long run, probably do more harm than good.
REFERENCES Anderson, B. A.. Silver, B. D.. & Abramson. P. R. (1988). The effects of the race of the interviewer on race-related attitudes of Black respondents in SRCiCPS national election studies. Plrhlic, Opinion Ql~urterl~. 52, 289-324. Annis. H. M. (1979). Self-report reliability of skid-row alcoholics. British Jo~lrnu/ c$‘Ps~c~hicctr~, 134. 459465. Aquilino. W. S. (1992). Telephone versus face-to-face interviewing for household drug use surveys. Internutionul Jmrrncd o.f the Addic,tions, 27, I I-91. Aquilino. W. S., & LoSciuto. L. A. (1990). Interview mode effects in drug use surveys. Public, Opinion Qucrrterly. 54. 362-395. Axinn, W. G. (1991). The influence of interviewer sex on responses to sensitive questions in Nepal. Socicrl Science Research, 20, 303-318. Bahr. H. M.. & Caplow, T. (1974). O/d men: Drunk und .sober. New York: New York University Press. Bahr. H. M., & Houts, K. C. (1971). Can you trust a homeless man’? A comparison of official records and interview responses by Bowery men. Public Opinim Quarter/y, 35. 374-382. Benny. M.. Riesman. D., & Star, S. A. (19.56). Age and sex in the interview. Anic~ric~r~nJorrrntrl of Socio/ogy, 62. 143-152. Bogue, D. B. (1963). Skid rm in Americ,m cirics. Chicago: Community and Family Center, University of Chicago. Bradburn, N. M.. Sudman, S., Blair, E., & Stoching. C. (1978). Question threat and response bias. Puhlk Opinion Quurterly. 42. 22 l-234. Breakey, W. R., Fischer. P. J., Kramer. G., Nestadt, G.. Romanoski, A. J., Ross. A., Royall. R. M.. & Stine, 0. C. (1989). Health and mental health problems of homeless men and women in Baltimore. Journal of thr American Medical Assoc,icrtion. 262. 1352-1357. Brorsson. B. (1984). Effects of interviewer characteristics and interviewer variability on interview responses. In C. F. Cannell and R. M. Groves (Ed?.). N~~tr/r/r.s~rrucyre.~wrc/~ ~~~c~t/~oc/s: Procc~edirr~.~of the fourrh Cmfirc~ncr on Hccrlth Suruey Rr.tc~rlrc~hMerhod~ (DHHS Publication No. PHS 84.3346). Washington. DC: U.S. Government Printing Office. Calsyn, R. J., Allen, G., Morse. G. A., Smith. R.. & Tempelhoff, B. (1993) Can you trust self-report data provided by homeless mentally ill individuals‘? Eutrlrrcrtion Rruietr,, 17, 353-366.
Y2
T. I’. JOHNSON
and J. A. PAKSONS
Campbell. B. A. (19X I). Kate-o-tntervtewer effects among southern adolescents. Public, O/~iniorr Qtrtrrtel/~. 45, 23 I-244. Clayton, K. K.. Voss. H. L.. R&bins. C. & Skinner. W. F. t IYXh). Gender differences in drug use: An epidemiological perspective. In B. A. Ray and M. C. Braude (Ed\.). W~~nre,! trnd drrr,qs: A neu‘ er(~ fogrcsc~trrclr (DHHS Publication No. ADM X6-1447. Research Monograph Series No. 65. pp. X0-YY). Kockville. MD: National Institute on Drug Abuse. Collins. M. (19X0). Interviewer variability: A review of the problem. ./o/fr/ltr/ c!f’ r/tr Mtrrr(e/ K~,.\c,trrc~/f soc~ic~t.v.22. 77-95. Colombotos. J.. Elinson. J.. Xr Loewen\tein. K. (196X). Effect of interviewers’ sex on interviewer responses. P/rh/ic Hctrlth Rcpor.t.\. 83. 6X5-6YO. Cotter. P. K., Cohen. J.. & Coulter, I’. B. (1982). Race-of-interviewer effects in telephone Interviews. Pohlic. Opiuiotl Qcctcrlarly. 46. 27X-286. Couper, M. P. ( 1991 ). Modeling survey participation at the interviewer Icvel. Proccc~dir~,q.~r!f’t/rc Sec.tion
on Swvcy
Resccrrc~h Method.\,
A~~rrrictrt~ .S/utivticcr/ A.\.soc~irrtion. 9% 107.
David, M. ( 1962). The validity of income reported by a sample of families who received welfare assistance during 19%. Jorm7trl c(f //I(> Atnc,ricrr/~ S~aris~ic~rrl A.~.~oc~ic~lion.57. 680-685. Delamater. J. D. (1974). Methodological issues in the study of premarital sexuality. .S~~io/o~ic~c~/ ,M~~thot/.~ & Rr.\c,rrrc~/r. 3, 30-61. Dijkstra. W. (19X3). How interviewer variance can bias the results of research on interviewer effects. @di/~ trrrd grrrrntity. 17, 17Y-187. Dixon. W. J.. R Massey, F. J. (1969). Intrrx/r~~tion to .srcr/i.c/ic.c~/trncr/yvi.\ (3rd ed. t. New York: McG,-awHill. Ehrlich. J. S.. 8: Kiesman. I>. (1961). Age and authority in the interview. P/lh/rc, Opiniorr Q/~ortcr/\.. 25. 39-56. Finkel, S. E., Guterbock. T. M., 6i Borg, M. J. (1991). Race-of-interviewer effects in a preelection poll: Virginia 19x9. Plrhlic, Opinion Qtrtrrtcr!\.. 55. 3 13-330. Fischer. P. J. (1989). Estimating the prevalence of alcohol. drug, and mental health problems in the contemporary homeless population: A review of the literature. Co/r/c,ntportrrv Drrlg f’rohlr~ns. 16. 333-390. Fischer, P. J., & Breakey. W. K. (1991). The epidemiology of alcohol. drug, and mental disorders among homeless persons. Amc~rictrn P,s~~~l~~/o,qi.~f.46. I I I-I 12X. Freeman, J., & Butler. E. W. (1976). Some sources of interviewer variance tn surveys. P/rh/ic, Opiniorr QMll./CI.!\.. 40. 79-92. Gelberg. L.. & Linn. L. S. (1989). Assessing the physical health of homeless adults. Jorr~,rtr/ of’ fhe Antcric.trn Mec/icci/ A.\.\o&ttio/r, 262. 1973-1979. Gfroerer. J. C., & Hughes. A. L. (IYYI). The feasibility of collecting drug abuse data by telephone. Public Hroltk Rcycwr.s. 106, 3X4-393. Grimes. M. D.. & Hansen. G. L. (1984). Response bias in sex-role attitude measurement. SPY Ro/c.s. 10. 67-72. Groves. R. M. (I 989). Sltr-vev e1’1’or.\ trf~d .s,rruc~~cosf.\. New York: John Wiley & Sons. Groves, R. M.. & Fultz. N. H. (1985).Gender effects among telephone interviews in a survey of economic attitudes. Soc~io/o~ic~r~/ Merizods &I Rcsc~tr~h, 14, 3 l-52. Hatchett. S., & Schuman, H. (1975). White respondents and race-of-interviewer effects. Public, Opitrior~ QIIlII.IPI.!\., 39. 523-52X. Hyman. H. ( 1954). Irrrrrvicwir~g in .soc,itr/ rc.~cwc~/z. Chicago: University of Chicago Press. Johnson. T. P.. Hougland. J. G.. & Moore. R. W. (I991 ). Sex differences in reporting sensitive behavior: A comparison of interview methods. SC,.\-Roles. 24. 669-680. Johnson. T. P.. & Moore. K. W. (1993). Gender interactions between interviewer and survey respondents: ls\ues of pornography and community standards. Se.\- Rolrs. 28. I-19. Johnson. W. T.. Delamater. J. D. (1976). Response effects in sex surveys. Pclh/ic~ Opirrirm C)~/urt~r/\.. 40. 16%181. Kandel. D. B. (19X0).Drug and drinking behavior among youth. Atrtrrrrrl Rwim ~~t’.S~~io/op~. 6. X5-285. Kane. E. W.. MaCaulay. L. J. (1993) Interviewer gender and gender attitudes. Prthlic, Opinir>fr Q/f(!r/cr/\.. 57. l-28. Kogel, P.. & Burnam. A. ( 1987). Traditional and nontraditional homeless alcoholics. A/~~oho/ Hcrrlrlr trtrd Landis.
J. K.. Sullivan.
D.. & Shelev. J. (1973). Feminist attitudes as related to sex of the interviewer. 16.’ 305-3 14. Loewenstein. K.. & Varma. .4. (1970). Effect of interaction of interviewers and respondents in health surveys. P~rhlk Opiniot~ L)u~irtw!\‘. 34. 472. McCarty, D.. Argeriou. M.. Huebner. R. B.. & Lubran. B. (1991). Alcoholism, drug abuse. and the homeless. A!?7eric~n Ps~cl~o/ogi.t/. 46. I 139-4X. Mensch. B. S.. & Kandel. D. B. (19RX). Youths‘ underreporting of substance use. Public Opirlim Qrrc~r1d\~. 52. 100-124.
Pac,ific Soc.io/o~ic~u/ Rcuicw,
Interviewer
effects
on self-reported
substance
use
93
Mulkern,
V., & Spence, R. (1984). Alcohol ubuselulcholism among homeless persons: A review, of the Rockville, MD: National Institute of Alcohol Abuse and Alcoholism. National Institute on Drug Abuse [NIDA]. (1990). Nurionul household .srrrue~ on drng trhr~e: Main jinding.s 1990 (DHHS Publication No. ADM 91-1788). Rockville. MD: Author. Nealon, J. (1983). The effects of male vs. female telephone interviewers. Proceedings of the Section on Survey Research Methods. Americun Statisticul Associution, 139-141. Reese, S. D., Danielson, W. A., Shoemaker, P. J.. Chang, T.-K., & Hsu. H.-L. (1986). Ethnicity-ofinterviewer effects among Mexican-Americans and Angles. Public Opinion Quurterly, 50, 563-572. Reiss, I. L. (1967). The sociu/ context ofpremuritul sesucrlpermissiuerI4s. New York: Holt, Rinehart and Winston. Rice, S. A. (1918). The homeless. Annuls. 77. 140-153. Rossi. P. H. (1989). Dobvn and out in Americuc The origins of home/es.sne.ss, Chicago: University of Chicago Press. Schaeffer, N. C. (1980). Evaluating race-of-interviewer effects in a national survey. Soc~iologicul Merhods literuture.
& Reseurch,
Schofield, Schuman. 1968. Sosin. M.
8, 400-419.
M. (1965). The sex/Au/ behuuiorrr ofyoung people. London: Longmans. H.. & Converse. J. (1971). The effects of Black and White interviewers Public,
Opinion
Quur/er/y,
Green. on Black responses
in
35, 44-68.
S. (1988). Homelessness in Chicugo: Pouerty und pathology, Chicago: Chicago Community Trust. Strack. F.. & Martin. L. L. (1987). Thinking, judging. and communicating: A process account of context effects in attitude surveys. In H.-J. Hippler. N. Schwarz, S. Sudman (Eds.). Sociu/ irzformution processing und survey methodology (pp. 123-148). New York: Springer-Verlag. Straus. R. (1946). Alcohol and the homeless man. Qucrrter/y Journul of Sfudies on A/coho/. 7. 360-404. Sudman, S., & Bradburn. N. M. (1974). Response qff‘ects in Anrueys. Chicago: Aldine. Sudman. S., & Bradburn, N. M. (1982). A.tking yuesrions: A prucfkul guide to yrtestionnuire design. San Francisco: Jossey-Bass. Summers. G., & Hammonds, A. (1966). Effect of racial characteristics of investigator on self-enumerated responses to a Negro prejudice scale. Sociul Force.5, 48, 515-5 18. Susser, S., Struening, E. L., & Conover. S. (1989). Psychiatric problems in homeless men: Lifetime psychosis, substance use and current distress in new arrivals at New York City shelters. Arc,hiues of sociu/
R., Colsin,
P., & Grossman.
institutions
und social
Generul
Psychiutry,
change.
46, 845-850.
Tourdngeau, R., & Rasinski, K. A. (1988). Cognitive processes measurement. Psycho/o&u/ Bulletin, 103, 299-314. Weeks, M. F., & Moore, R. P. (1981). Ethnicity-of-interviewer Opinion
Weiss.
Quurtedy,
C. H. (1975).
underlying effects
context
on ethnic
effects respondents.
in attitude Pub/it
45, 245-249.
in evaluation research. In E. L. Struening & M. Guttentag (Eds.). (Vol. I. pp. 355-395). Beverly Hills, CA: Sage. Willis. G. B., Royston, P., & Bercini, D. (1991). The use of verbal report methods in the development and testing of survey questionnaires. Applied Cognifiue Ps,vchn/o~y. 5. 251-267. Hundhook
Interviewing
of euuluution
reseurch