- Email: [email protected]
A successful conversion or double refusal: A study of the process of refusal conversion in telephone survey research
Available online at www.sciencedirect.com
The Social Science Journal 44 (2007) 434–446
A successful conversion or double refusal: A study of the process of refusal conversion in telephone survey research Kana Fuse a , Dong Xie b,∗ a
b
Department of Sociology, The Ohio State University, Columbus, OH 43210, USA Department of Psychology and Counseling, University of Central Arkansas, Conway, AR 72035, USA
Abstract This study investigated factors that affect the outcome of refusal conversion efforts in telephone surveys. We used data collected from three types of forms filled out by telephone interviewers: a Refusal Report Form was filled out when the respondent/household member refused to do the survey at the initial contact, and a Refusal Conversion Report Form or a Double Refusal Report Form was filled out during the refusal conversion phase. These forms included information such as the degree of interaction between interviewer and respondent/household member, reasons for refusal, information about the survey revealed during the introduction, and demographics. Results from logistic regressions suggested that the degree of interviewer–respondent/household member interaction during the second call attempt significantly influenced the outcome of refusal conversions. Results also suggested that revealing information about the legitimacy of the survey did not affect the outcome, and that there might even be a “boomerang effect” if interviewers rush to reveal such information. While respondent/household member’s age did not affect the outcome, women appeared to be more difficult to be converted than men, especially when the degree of interviewer–respondent/household member interaction is low. Implication of these results and direction for future research directions are also discussed. © 2007 Elsevier Inc. All rights reserved.
1. Introduction Nonresponse error due to sampled household/respondent’s noncooperation to participate in a telephone survey is one type of survey error of the total survey error perspective (Groves & Couper, 1998). Various methods to boost response rates have been implemented in study ∗
Corresponding author. Tel.: +1 501 450 5422. E-mail address: [email protected] (D. Xie).
0362-3319/$ – see front matter © 2007 Elsevier Inc. All rights reserved. doi:10.1016/j.soscij.2007.07.011
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
435
designs to reduce nonresponse errors. For instance, nonresponse error can be reduced by using incentives (Groves et al., 2004). Nonresponse error can also be reduced by conducting rigorous refusal aversion training for the interviewers (Mayer & O’Brien, 2001). Another way to increase response rates is to implement refusal conversions, in which households/respondents that initially refused to participate in a survey are recontacted at a later date. Refusal conversion rates are reported to be as high as 40% (Groves et al., 2004). Since response rates in telephone surveys have seemingly become lower, refusal conversions have become an important part of survey operations to boost response rates. Studies have investigated the effectiveness of the use of incentives to increase response rates (e.g., Singer, Groves, & Corning, 1999; Singer, Van Hoewyk, & Maher, 2000). Many studies have also examined household profile, interviewer characteristics, and interviewing styles and their effect on household cooperation to participate in telephone surveys (e.g., Hawkins, 1975; Herzog, Rodgers, & Kulka, 1983; Groves, 1979; Miller & Cannell, 1982). However, not much is known about the process of refusal conversion efforts. Therefore, the purpose of this study is to investigate various aspects of the refusal conversion process and their effects on the outcome of refusal conversion efforts. We expect our study to provide a better understanding of the refusal conversion process in telephone surveys and consequently help improve the effectiveness of refusal conversion efforts. Since there has been little research on the topic, our study is exploratory in nature. We hope it will stimulate more research on refuse conversion processes. Telephone interviews are different from face-to-face interviews in many ways. Miller and Cannell (1982) compared and contrasted differences in communication between telephone and face-to-face interviews. There are certain limitations inherent in communication over the phone not only because it is less intimate than in-person dialogues, but also because visual cues such as facial expressions and gestures are unavailable. There is often heightened uncertainty about the affective meaning behind the words and whether people understand what they are trying to convey. Communication over the phone is less effective than face-to-face communication unless efforts are made to clarify, emphasize, and explain what one is trying to convey to the other. Moreover, telephone survey requests may often be misidentified by householders as sales calls and decisions to decline such a request are often made in less than 30 s (Groves et al., 2004). Therefore, interviewers need to carefully manage the communication in order to better inform respondents about the nature of the survey and to motivate them to participate (Miller & Cannell, 1982). For this reason, many survey operations require telephone interviewers to read verbatim a standardized introduction to deliver a telephone survey request and use standardized fallback statements to respond to householders’ concerns (Groves et al., 2004). An example of such a standardized introduction may read “Hello, my name is Mary Smith. I am calling from the University of Michigan in Ann Arbor. Here at the University we are currently working on a nationwide research project. First, I would like to make sure I dialed the right number. Is this 301-555-2222?” (Groves et al., 2004, p. 175). Mishler (1986) has argued that the interaction between the interviewer and the interviewee (respondent) is one of the key elements in collecting valid survey data. He has argued against the traditional standardized style of interviewing and instead proposed a contrasting approach. Unlike standardized interviewing that requires interviewers to follow a standardized interviewing procedure, this approach emphasizes a more narrative and conversational approach to
436
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
interviewing in which both the interviewer and the interviewee assume a more active role in formulating and responding to questions. Although Mishler (1986) has not specified whether this approach should be applied to conducting introductions of the survey, we can expect that engaging the householder/respondent in an on-going conversation during the introduction stage will help gain their cooperation to do the survey. Previous research suggests that increased interaction between the interviewer and the householder/respondent is associated with gaining their cooperation to participate. Groves (1979) compared respondents’ reaction to different modes of interviewing in order to understand why national telephone interviews have lower response rates than personal interviews. He argued that the response rates are lower in telephone surveys because there are limitations in telephone interviews, that is, information exchange is restricted and not as effective as in the context of face-to-face interviews. To compensate for this limitation, telephone interviewers need to make more effort to engage the householder/respondent in an on-going conversation. In order to gain cooperation in telephone surveys, Groves (1979) recommended “to delay the request for an interview, to devote the initial dialog to detailed descriptions of the survey, the survey organization, the role of the respondent, and the proposed use of the data” (p. 195). In another study, Groves, Cialdin, and Couper (1992) examined factors that influence respondents’ decision on whether to participate in an interview. They conducted focus groups with interviewers to find out effective strategies that help gain cooperation. Two components were found useful at the introduction stage: tailoring and maintaining interaction. They found that inexperienced interviewers often create “soft-refusals” in their first interactions by pressing the respondents to make a decision too quickly. Based on their findings, they suggest that instead of trying to get immediate acceptance, interviewers should focus on trying not to terminate the discussion prematurely. We relate previous research on interviewer–householder/respondent interaction to refusal conversion efforts in telephone surveys. In this study, we are particularly interested in examining whether interviewer’s efforts to interact with the respondent/household member at the initial call attempt and/or interaction during the refusal conversion phase lead to a successful conversion. While increased interaction between telephone interviewer and respondent/household member may help gain cooperation, it should be noted that there is a high correlation between the degree of interaction and the amount and type of information (about the survey) revealed to the respondent/household member. It is unclear whether interviewers just need to engage the respondent/household member in a plain “chit-chat” conversation, or if interviewers have to reveal certain information to enhance the legitimacy of the interview. Therefore, we investigate whether being plain chatty during the refusal conversion introduction helps to convert a case or if conveying certain pieces of information helps lead to a successful conversion. Researchers have tried to identify household demographics that may be related to their decision to participate in interviews. Hawkins (1975) attempted to construct a demographic profile of nonrespondents and found that middle-aged and older householders were more likely to refuse. Similarly, DeMaio (1980) found that older people (50 years and older) were more likely to refuse during the first contact, but they were somewhat more likely to be converted during follow-up attempts. The author did not find any difference by race or gender, however.
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
437
Herzog et al. (1983) compared representation of older adults in telephone surveys with face-toface surveys and found that older adults, particularly those who are less educated tended to be under-represented in telephone surveys. These results suggested difficulty not only in gaining cooperation from older respondents at the initial survey request, but also in converting them at the refusal conversion phase. In this study, we explore the relationship between demographics of the respondent/household member and their decision to participate in telephone surveys. In addition, we explore how respondent’s gender and age interact with other variables such as the degree of interviewer–respondent/household member interaction. This exploratory study examines whether the outcome of refusal conversion efforts (i.e., successful conversion or double refusal) are influenced by (1) age and gender of the respondent/household member, (2) degree of interaction between the interviewer and the respondent/household member, and (3) whether the interviewer conveyed information (e.g., how the respondent was sampled, the nature and purpose of the survey) to the respondent/household member.
2. Method Data are based on information collected from three types of forms filled out by interviewers at a Survey Research Center affiliated with a large Midwestern university from January 2002 to August 2002 across three distinct telephone surveys. Random digit dialing (RDD) was used to first sample households. Then, the “last birthday method” (Oldendick, Bishop, Sorenson, & Tuchfarber, 1988) was employed to randomly select a respondent within each household. Advance letters to alert households/respondents about the survey were not sent out because there are no means to identify addresses of respondents who are randomly chosen through RDD. Interviewers filled out a Refusal Report Form when the respondent/household member refused to do the survey. During the refusal conversion phase, interviewers were asked to complete a Refusal Conversion Report Form if they successfully converted a refusal or a Double Refusal Report Form if the interview was refused again. The main contents of these forms were refuser’s (respondent or household member) demographic information (actual or interviewer’s perception), degree of interaction between the interviewer and the refuser, reasons for first/second refusal, strength of refusal, whether the interviewer conveyed information about the nature/legitimacy of the survey, and whether interviewer had revealed the previous refusal history to the respondent/household member. Altogether there were 157 successful conversions and 550 double refusals that were with valid documentations. Our study is based on 157 randomly selected double refusals cases and 157 successful conversions cases, yielding a total of 314 cases. Although a great amount of information was collected from these original forms, we focused on examining variables that were present in both outcome situations. These variables include data from the Refusal Report Form, which include (a) degree of interaction during first attempt, (b) reasons for initial refusal, (c) information about the survey revealed to the respondent/household member, and (d) demographics (respondent/household member’s age and gender reported by telephone interviewer). Specifically, the variables examined in this study are as follows.
438
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
2.1. Dependent variable 2.1.1. Outcome of second call attempt The dependent variable is the outcome of the second interview attempt following a refusal. Successfully converted cases are coded as “1” and double refusals are coded as “0.” 2.2. Independent variables 2.2.1. Degree of interaction Interviewers reported their perceived degree of interaction with the respondent/household member during the introduction on a scale of 1–7 where 1 being “no interaction at all” and 7 being “much interaction.” The degree of interaction was reported after the first call attempt (on the Refusal Report Form) as well as after the second attempt (on the Refusal Conversion Form or the Double Refusal Report Form). 2.2.2. Reason for initial refusal Interviewers indicated whether the refusal was due to any of the following reasons: (a) immediate hang-up, (b) time-related concerns, reasons, and excuses, (c) objections to surveys, topics, and sponsors, and (d) concerns about confidentiality and legitimacy of survey. Each of the four variables was coded as binary variables where 1 means that the refusal was due to the given reason and 0 means that it was not. 2.2.3. Information revealed Interviewers were given a set of standardized statements to address specific concerns often brought up by respondents. Interviewers reported whether they revealed any of the following information to the respondent/household member: (a) how he/she was sampled,1 (b) the nature, purpose of survey beyond the standard introduction, (c) confidentiality, (d) how the data would be used, (e) verification with supervisor/sponsor. This information was collected after the first call attempt (on the Refusal Report Form) as well as after the second attempt (on the Refusal Conversion Form or the Double Refusal Report Form). These variables were also coded as binary variables where 1 means that the given information was revealed and 0 means that it was not revealed. 2.2.4. Previous refusal history revealed to the householder Interviewers indicated whether they used the phrase, “Recently we contacted someone in your household about participating in a very important study . . . unfortunately, it was not a good time for them” during the introduction. Interviewers were encouraged to use this phrase when attempting refusal conversions. Again, this is a binary variable where 1 means that the interviewer used this phase and 0 means that they did not mention the phrase. This information was collected from the Refusal Conversion Form as well as the Double Refusal Report Form. 2.2.5. Demographics On the Refusal Report Form and Double Refusal Report Form, interviewers reported their perceived gender (male, female, or uncertain) and age (child, adult <30, 30–59, 60+, uncertain)
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
439
of the person whom they spoke to. The gender variable was recoded (1 = female; 0 = male; missing = uncertain) to be able to make comparisons between the genders. Age was also recoded (1 = 60+; 0 = under 60; missing = uncertain) in order to compare elderly respondents with other age groups. Perceived demographics of the respondent were not asked on the Refusal Conversion Form; instead, demographics of the converted respondent were pulled from the actual completed survey. As with “perceived” demographics, “actual” gender was coded as 1 = female; 0 = male, and “actual” age was recoded as 1 = 60+; 0 = under 60.
3. Results T tests, χ2 tests, and logistic regression analysis were performed using the outcome of refusal conversion (successful refusal conversion or double refusal) as the dependent variable and other aforementioned variables as predictors. We investigate factors that impact the refusal conversion process. Table 1 presents descriptive statistics of all variables included in the study. To get a general idea of how the independent variables are related to the outcome variable, we performed t tests and χ2 tests. We ran t tests to look at whether there is a mean difference in the degree of interviewer and respondent/household member interaction at first and second call attempts between converted cases and double refused cases (see Table 2). Results suggested that the mean level of interaction during the initial contact was slightly higher for converted cases than for double refused cases, although the individual contacted at the first attempt may not have necessarily been the individual contacted during the second call attempt. The difference in the mean degree of interaction between the two outcomes was greater at the refusal conversion phase. While the mean level of interaction was 1.84 for double refused cases, that of the converted cases was 3.70, indicating that during the refusal conversion phase, significantly higher levels of interaction between the interviewer and respondent/household member occurred for successfully converted cases as opposed to double refused cases. We also created a variable that measures the extent to which information about the survey was revealed to the respondent/household member. This variable combines five items on whether certain information was revealed (i.e., how she was sampled, the nature/purpose of the survey, confidentiality, how the data would be used, verification with supervisor/sponsor). This variable ranges from 0 to 5 where 0 means none of them were revealed and 5 means all five of them were provided. We examined mean differences in these scores. The amount of information revealed to the respondent/household member at the initial contact stage was not significantly different between the two refusal conversion effort outcomes. However, during the second call attempt, significantly more information was revealed when cases resulted in a successful conversion than when they resulted in a double refusal. Table 3 summarizes χ2 test results that examined the relationship between the dependent variable and independent variables that are binary. These results suggested that if the initial refusal was due to immediate hang-up or time-related concerns by the respondent/household member during the first call attempt, it was more likely to result in a double refusal at the
440
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
Table 1 Descriptive statistics of variables in the study Valid n
Mean
S.D.
Dependent variable Outcome of second call attempt
314
0.50
0.50
Independent variables Information from first call attempt (Refusal Report Form) Degree of interaction
288
1.79
1.20
Reason for initial refusal Immediate hang-up Time-related concers, reasons, excuses Objects to surveys, topics, sponsors Concerned about confidentiality, legitimacy of survey
309 308 308 308
0.60 0.12 0.14 0.01
0.49 0.33 0.35 0.10
Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
314 314 314 314 314
0.27 0.22 0.09 0.09 0.04
0.45 0.42 0.29 0.29 0.18
Demographics Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
292 308
0.60 0.05
0.49 0.22
310
2.78
1.72
Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
314 314 314 314 314
0.28 0.43 0.34 0.33 0.05
0.45 0.50 0.47 0.47 0.22
Previous refusal history revealed to the respondent/household member
305
0.58
0.49
Demographics (perceived or actual) Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
307 227
0.59 0.32
0.49 0.47
Information from second call attempt (RC Form or DR Report Form) Degree of interaction
Table 2 Mean differences in independent variables between refusal conversion effort outcomes
Degree of interaction during first call attempt Information revealed during first call attempt
*
Degree of interation during second call attempt** Information revealed during second call attempt** ∗ ∗∗
Mean difference significant at p < .05. Mean difference significant at p < .001.
Outcome = RC
Outcome = DR
Mean
S.D.
Mean
S.D.
1.93 0.81
1.31 1.30
1.64 0.61
1.06 1.21
3.70 2.27
1.67 1.73
1.84 0.59
1.17 1.33
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
441
Table 3 Summary of χ2 test results Independent variables
Related to DV (p < .05)?
Information from first call attempt (Refusal Report Form) Reason for initial refusal Immediate hang-up Time-related concers, reasons, excuses Objects to surveys, topics, sponsors Concerned about confidentiality, legitimacy of survey
Yes Yes No No
Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
No No No No No
Demographics Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
No No
Information from second call attempt (RC Form or DR Report Form) Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
Yes Yes Yes Yes No
Previous refusal history revealed to the respondent/household member
Yes
Demographics (perceived or actual) Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
No No
second call attempt. Other variables obtained from the Refusal Report Form did not appear to affect the outcome of the refusal conversion. However, we need to be reminded that we do not know whether the person who refused or was converted at the second attempt was the same individual we contacted during the initial attempt. Therefore, it is misleading to assume that the nature of a refusal during the first call attempt directly affects the outcome of the second call attempt. Results indicate that the amount of information revealed to the respondent/household member during the refusal conversion phase affected the outcome. Clearly, trying to convey information that justifies the legitimacy of the survey and revealing the previous call history were effective in refusal conversions. The only variable that did not appear to affect the outcome based on χ2 results was whether the respondent sought to verify with the supervisor/sponsor. However, this may be due to the fact that very few people in the sample sought verification. Since there is not much previous research that guides us to propose a model in predicting the refusal conversion outcomes, the models in the following logistic regression analyses are exploratory (see Table 4). We simply included independent variables that were significant at the bivariate level (t test and χ2 analyses) and demographics as covariates in our models. The predictors in Model 1 include the two reasons for initial refusal (i.e., immediate hang-up
442
Table 4 Logistic regression estimates for determinants of refusal conversion effort outcomes Independent variables
Model 1 B
Model 2 Odds ratio
Note. Standard errors are in parentheses. ∗ p ≤ .05 two-tailed test. ∗∗ p ≤ .01 two-tailed test. ∗∗∗ p ≤ .001 two-tailed test.
B
Odds ratio
0.155 (0.428) 0.764 (0.643)
1.167 2.148
B
Model 4 Odds ratio
B
Odds ratio
0.742 (0.630)
2.101
0.764 (0.639)
2.146
0.141 (0.174)
0.868
0.130 (0.174)
0.878
−1.376 (0.721) 0.770 (0.930)
0.253 2.159
−1.607* (0.806) 1.419 (1.009)
0.200 4.134
−1.518 (0.834) 1.558 (1.061)
0.219 4.748
−0.798 (0.844) 0.816 (0.630) 0.502* (0.201)
0.450 2.261 1.652
−0.725 (0.926) 0.764 (0.682) 0.891*** (0.219)
0.484 2.146 2.437
−1.013 (0.991) 0.526 (0.688) 0.506* (0.218)
1.045 1.692 1.659
0.563 (0.422)
1.757
0.672 (0.427)
1.959
0.595 (0.435)
1.812
−3.133*** (0.864)
0.044
−0.939* (0.418)
0.391
−2.916*** (0.881)
0.054
−0.143 (0.418) 0.940** (0.310)
0.867 2.560
0.029 (0.427)
1.030
−0.060 (0.444) 0.851** (0.322)
0.942 2.342
−0.730 172.6 104.1 (11) 212
−1.450 164.7 85.8 (10) 196
−0.473 157.6 92.9 (11) 196
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
Information from first call attempt (Refusal Report Form) Reason for initial refusal Immediate hang-up 0.062 (0.410) 1.064 Time-related concers, 0.709 (0.635) 2.031 reasons, excuses Degree of interaction during first call attempt Information from second call attempt (RC Form or DR Report Form) Information revealed 0.235 How s/he was sampled −1.450* (0.696) 0.631 (0.887) 1.880 The nature, purpose of survey beyond the standard introduction Confidentiality −0.491 (0.801) 0.612 How the data would be used 1.081 (0.624) 2.949 Degree of interaction during 0.925*** (0.215) 2.521 second call attempt Previous refusal history 0.612 (0.409) 1.843 revealed to the householder Demographics (perceived or actual) Gender (1 = female; −0.897* (0.405) 0.408 0 = male) Age (1 = 60+; 0 = under 60) −0.039 (0.401) 0.961 Gender × Degree of interaction during second call attempt (interaction term) Constant −1.795 −2 log likelihood 182.0 94.7 (10) χ2 (d.f.) n 212
Model 3
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
443
and time related concerns). Other predictors included in Model 1 are from the refusal conversion phase, including specific information revealed to respondent/household member, degree of interaction in the conversion phase, whether the previous refusal history was revealed to respondent/household member, and interviewers’ perceived respondent/household member’s age and gender. Model 2 is the same as Model 1 except for the added interaction term between gender and degree of interviewer–respondent/household member interaction. Models 3 and 4 are the same as Models 1 and 2, respectively, except that immediate hang-up as a reason for initial refusal was substituted by degree of interviewer–respondent/household member interaction during the initial call attempt. These two variables were not included in the same model because they are highly correlated with each other. The results indicate that across the four models, reasons for refusal (immediate hang-up and time-related concerns) and degree of interviewer–respondent/household member interaction during the initial call attempt were no longer predictive of conversion outcomes when variables from the second call attempt and demographic variables were controlled. All factors that appear to affect the conversion outcomes were from the refusal conversion phase. Specifically, the degree of interviewer–respondent/household member interaction during the conversion phase appeared to be a significant predictor, that is, when the interaction between the interviewer and the respondent/household member was greater, the likelihood in resulting in a successful conversion was greater. Respondent/household member’s gender also appeared be related to the outcome. Female respondents/household members were less likely to be successfully converted. A significant interaction effect between gender and the degree of interviewer–householder interaction was found in both Models 2 and 4. This interaction effect suggested that female respondents/householders were less likely to be converted at the refusal conversion stage when the interaction between her and the interviewer was low. Overall, the results indicated that the type of information revealed to respondent/householder did not affect the outcome very much. The only variable that had an impact was the information, “how he/she was sampled” was revealed. However, the direction of this effect was not what we had expected since providing this information tended to result in a double refusal. Contrary to our expectation, perceived respondent/householder’s age did not affect the outcome.
4. Discussion Results indicate that the degree of interviewer–respondent/household member interaction during the second call attempt mattered more than any other variable in converting a refused case. This result provided further empirical support for the importance of interviewer–respondent/household member interaction in gaining cooperation to participate in telephone interviews that has been emphasized by previous researchers (Groves, 1979; Groves et al., 1992; Groves & McGonagale, 2001; Miller & Cannell, 1982). Our results also provide support for the narrative interviewing style proposed by Mishler (1986) as opposed to traditional standardized interviewing. Our results suggest an interviewing approach with an emphasis on maintaining interaction with the respondent/household member is effective in successfully converting a refused case.
444
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
It is believed that conveying certain information about the survey (i.e., how he/she was sampled, how the data will be used, who the sponsor of the survey is, and confidentiality) influences the respondent/household member’s perceived legitimacy of the survey thus helps gain their cooperation to do the survey. As we mentioned earlier, the amount of information revealed during the introduction and interaction between the interviewer and respondent/household member is highly correlated. Therefore, we attempted to explore whether the interviewer simply needs to engage the householder in a “chit-chat” conversation, or has to reveal certain information during their interaction in order to convince respondents to participate in an interview. Our results suggested that engaging the respondent/household member in a more conversation-like interaction matters more than revealing certain information during the introduction. Interestingly, the only type of information revealed to the respondent/household member that had an impact on the outcome was how she/he was sampled, though the relationship was in an unexpected direction. Our findings suggested that when the degree of interviewer–respondent/household member interaction was held constant, revealing such information was associated with less success in converting than when it was not revealed. Therefore, revealing how respondents were sampled seemed to have a “boomerang effect.” This effect is based on theory of psychological reactance that postulates that people have a motive to protect or restore one’s sense of freedom and will react against attempts to control their behavioral freedom (Brehm, 1966). In the context of telephone survey interviews, this boomerang effect is a negative mental set that the respondent/household member has toward any call from a stranger. Perhaps the recent increase in the number of telemarketing calls reinforces this mindset. With such a negative attitude towards phone calls from strangers, the respondent/household member might be likely to refuse if the interviewer rushed to reveal information about the survey. At the survey research center where our data were collected, information on how respondents were sampled is on the top of the list of information that interviewers are trained to bring up when respondents/household members appear skeptical about the legitimacy of the survey. Therefore, it was likely that refusal conversion attempts often fail quickly after the interviewer reveals the first piece of information that justifies the survey. Thus, a general strategy to successfully convert a refused case might be to engage respondents/household members in an interaction as much as possible and not rush to reveal information concerning the legitimacy of the survey. However, such strategy might be more effective in converting a female respondent/household member than a male. Our findings suggested an interaction effect between respondent/household member’s gender and degree of interaction where women are more likely to refuse during the refusal conversion stage if their interaction with the interviewer is not heightened. Previous research found that respondent’s age might affect their decision to participate in a survey. Older householders were found to be more likely to refuse in the initial request of survey (DeMaio, 1980; Hawkins, 1975; Herzog et al., 1983). The present study did not find that age is related to the outcome of refusal conversions, although this might be due to interviewer’s inaccurate perception of the respondent/household member’s age. One major limitation of the present study is that we were unable to determine whether the person contacted at the refusal conversion phase was the same individual we contacted at the
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
445
initial attempt. Due to this limitation, it is difficult to examine how factors from the first call attempt affect the outcome of refusal conversion efforts. In the future, we may also need to focus on variables such as respondent’s interest in the topic of the survey and their perceived legitimacy of the survey, as these variables also serve to shape people’s decision on whether to participate in a survey (Groves, Singer, & Corning, 2000).
Note 1. For instance, the standardized language interviewers were instructed to use to describe how households were selected reads, “Your number was randomly chosen using a technique called Random-Digit Dialing. The way this works is that a computer randomly chooses an area code and a three-digit prefix that works within that area code. Then, the computer randomly adds four more numbers to make up a telephone number, like the one I used that reached your home. Many times we dial numbers that are not working numbers; others reach businesses, and some reach households such as yours. We do this because it’s important that we interview a random sample of [place] residents regardless of whether they have a listed or unlisted telephone number.”
Acknowledgements This manuscript was based on a paper presented at the 60th Annual Conference of American Association of Public Opinion Research (AAPOR), Miami Beach, Florida, May 12–15, 2005. Authors contributed equally to the research.
References Brehm, J. (1966). A theory of psychological reactance. New York: Academic Press. DeMaio, T. J. (1980). Refusals: Who, where, and why. Public Opinion Quarterly, 44, 223–233. Groves, R. M. (1979). Actors and questions in telephone and personal interview surveys. Public Opinion Quarterly, 43, 190–205. Groves, R. M., Cialdin, R. B., & Couper, M. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475–495. Groves, R., & Couper, M. (1998). Nonresponse in household interview surveys. New York: Wiley & Sons Inc.. Groves, R., Fowler, F., Couper, M., Lepkowski, J., Singer, E., & Tourangeau, R. (2004). Survey methodology. New York: Wiley. Groves, R., & McGonagale, K. (2001). A theory-guided interviewer training protocol regarding survey participation. Journal of Official Statistics, 17, 249–265. Groves, R., Singer, E., & Corning, A. (2000). Leverage-salience theory of survey participation: Description and an illustration. Public Opinion Quarterly, 64, 299–308. Hawkins, D. F. (1975). Estimation of nonresponse bias. Sociological Methods and Research, 3, 462–485. Herzog, A. R., Rodgers, W. L., & Kulka, R. A. (1983). Interviewing older adults: A comparison of telephone and face-to-face modalities. Public Opinion Quarterly, 47, 405–418. Mayer, T. S., & O’Brien, E. (2001). Interviewer refusal aversion training to increase survey participation. In Proceedings of the annual meeting of the American Statistical Association.
446
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
Miller, P., & Cannell, C. (1982). A study of experimental techniques for telephone interviewing. Public Opinion Quarterly, 46, 250–269. Mishler, E. G. (1986). Research Interviewing. Cambridge: Harvard University Press. Oldendick, R., Bishop, G., Sorenson, S., & Tuchfarber, A. (1988). A comparison of the Kish and last birthday methods of respondent selection in telephone surveys. Journal of Official Statistics, 4, 307–318. Singer, E., Groves, R., & Corning, A. (1999). Differential incentives: Beliefs about practices, perceptions of equity, and effects on survey participation. Public Opinion Quarterly, 63, 251–260. Singer, E., Van Hoewyk, J., & Maher, M. P. (2000). Experiments with incentives in telephone surveys. Public Opinion Quarterly, 64, 171–188.
The Social Science Journal 44 (2007) 434–446
A successful conversion or double refusal: A study of the process of refusal conversion in telephone survey research Kana Fuse a , Dong Xie b,∗ a
b
Department of Sociology, The Ohio State University, Columbus, OH 43210, USA Department of Psychology and Counseling, University of Central Arkansas, Conway, AR 72035, USA
Abstract This study investigated factors that affect the outcome of refusal conversion efforts in telephone surveys. We used data collected from three types of forms filled out by telephone interviewers: a Refusal Report Form was filled out when the respondent/household member refused to do the survey at the initial contact, and a Refusal Conversion Report Form or a Double Refusal Report Form was filled out during the refusal conversion phase. These forms included information such as the degree of interaction between interviewer and respondent/household member, reasons for refusal, information about the survey revealed during the introduction, and demographics. Results from logistic regressions suggested that the degree of interviewer–respondent/household member interaction during the second call attempt significantly influenced the outcome of refusal conversions. Results also suggested that revealing information about the legitimacy of the survey did not affect the outcome, and that there might even be a “boomerang effect” if interviewers rush to reveal such information. While respondent/household member’s age did not affect the outcome, women appeared to be more difficult to be converted than men, especially when the degree of interviewer–respondent/household member interaction is low. Implication of these results and direction for future research directions are also discussed. © 2007 Elsevier Inc. All rights reserved.
1. Introduction Nonresponse error due to sampled household/respondent’s noncooperation to participate in a telephone survey is one type of survey error of the total survey error perspective (Groves & Couper, 1998). Various methods to boost response rates have been implemented in study ∗
Corresponding author. Tel.: +1 501 450 5422. E-mail address: [email protected] (D. Xie).
0362-3319/$ – see front matter © 2007 Elsevier Inc. All rights reserved. doi:10.1016/j.soscij.2007.07.011
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
435
designs to reduce nonresponse errors. For instance, nonresponse error can be reduced by using incentives (Groves et al., 2004). Nonresponse error can also be reduced by conducting rigorous refusal aversion training for the interviewers (Mayer & O’Brien, 2001). Another way to increase response rates is to implement refusal conversions, in which households/respondents that initially refused to participate in a survey are recontacted at a later date. Refusal conversion rates are reported to be as high as 40% (Groves et al., 2004). Since response rates in telephone surveys have seemingly become lower, refusal conversions have become an important part of survey operations to boost response rates. Studies have investigated the effectiveness of the use of incentives to increase response rates (e.g., Singer, Groves, & Corning, 1999; Singer, Van Hoewyk, & Maher, 2000). Many studies have also examined household profile, interviewer characteristics, and interviewing styles and their effect on household cooperation to participate in telephone surveys (e.g., Hawkins, 1975; Herzog, Rodgers, & Kulka, 1983; Groves, 1979; Miller & Cannell, 1982). However, not much is known about the process of refusal conversion efforts. Therefore, the purpose of this study is to investigate various aspects of the refusal conversion process and their effects on the outcome of refusal conversion efforts. We expect our study to provide a better understanding of the refusal conversion process in telephone surveys and consequently help improve the effectiveness of refusal conversion efforts. Since there has been little research on the topic, our study is exploratory in nature. We hope it will stimulate more research on refuse conversion processes. Telephone interviews are different from face-to-face interviews in many ways. Miller and Cannell (1982) compared and contrasted differences in communication between telephone and face-to-face interviews. There are certain limitations inherent in communication over the phone not only because it is less intimate than in-person dialogues, but also because visual cues such as facial expressions and gestures are unavailable. There is often heightened uncertainty about the affective meaning behind the words and whether people understand what they are trying to convey. Communication over the phone is less effective than face-to-face communication unless efforts are made to clarify, emphasize, and explain what one is trying to convey to the other. Moreover, telephone survey requests may often be misidentified by householders as sales calls and decisions to decline such a request are often made in less than 30 s (Groves et al., 2004). Therefore, interviewers need to carefully manage the communication in order to better inform respondents about the nature of the survey and to motivate them to participate (Miller & Cannell, 1982). For this reason, many survey operations require telephone interviewers to read verbatim a standardized introduction to deliver a telephone survey request and use standardized fallback statements to respond to householders’ concerns (Groves et al., 2004). An example of such a standardized introduction may read “Hello, my name is Mary Smith. I am calling from the University of Michigan in Ann Arbor. Here at the University we are currently working on a nationwide research project. First, I would like to make sure I dialed the right number. Is this 301-555-2222?” (Groves et al., 2004, p. 175). Mishler (1986) has argued that the interaction between the interviewer and the interviewee (respondent) is one of the key elements in collecting valid survey data. He has argued against the traditional standardized style of interviewing and instead proposed a contrasting approach. Unlike standardized interviewing that requires interviewers to follow a standardized interviewing procedure, this approach emphasizes a more narrative and conversational approach to
436
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
interviewing in which both the interviewer and the interviewee assume a more active role in formulating and responding to questions. Although Mishler (1986) has not specified whether this approach should be applied to conducting introductions of the survey, we can expect that engaging the householder/respondent in an on-going conversation during the introduction stage will help gain their cooperation to do the survey. Previous research suggests that increased interaction between the interviewer and the householder/respondent is associated with gaining their cooperation to participate. Groves (1979) compared respondents’ reaction to different modes of interviewing in order to understand why national telephone interviews have lower response rates than personal interviews. He argued that the response rates are lower in telephone surveys because there are limitations in telephone interviews, that is, information exchange is restricted and not as effective as in the context of face-to-face interviews. To compensate for this limitation, telephone interviewers need to make more effort to engage the householder/respondent in an on-going conversation. In order to gain cooperation in telephone surveys, Groves (1979) recommended “to delay the request for an interview, to devote the initial dialog to detailed descriptions of the survey, the survey organization, the role of the respondent, and the proposed use of the data” (p. 195). In another study, Groves, Cialdin, and Couper (1992) examined factors that influence respondents’ decision on whether to participate in an interview. They conducted focus groups with interviewers to find out effective strategies that help gain cooperation. Two components were found useful at the introduction stage: tailoring and maintaining interaction. They found that inexperienced interviewers often create “soft-refusals” in their first interactions by pressing the respondents to make a decision too quickly. Based on their findings, they suggest that instead of trying to get immediate acceptance, interviewers should focus on trying not to terminate the discussion prematurely. We relate previous research on interviewer–householder/respondent interaction to refusal conversion efforts in telephone surveys. In this study, we are particularly interested in examining whether interviewer’s efforts to interact with the respondent/household member at the initial call attempt and/or interaction during the refusal conversion phase lead to a successful conversion. While increased interaction between telephone interviewer and respondent/household member may help gain cooperation, it should be noted that there is a high correlation between the degree of interaction and the amount and type of information (about the survey) revealed to the respondent/household member. It is unclear whether interviewers just need to engage the respondent/household member in a plain “chit-chat” conversation, or if interviewers have to reveal certain information to enhance the legitimacy of the interview. Therefore, we investigate whether being plain chatty during the refusal conversion introduction helps to convert a case or if conveying certain pieces of information helps lead to a successful conversion. Researchers have tried to identify household demographics that may be related to their decision to participate in interviews. Hawkins (1975) attempted to construct a demographic profile of nonrespondents and found that middle-aged and older householders were more likely to refuse. Similarly, DeMaio (1980) found that older people (50 years and older) were more likely to refuse during the first contact, but they were somewhat more likely to be converted during follow-up attempts. The author did not find any difference by race or gender, however.
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
437
Herzog et al. (1983) compared representation of older adults in telephone surveys with face-toface surveys and found that older adults, particularly those who are less educated tended to be under-represented in telephone surveys. These results suggested difficulty not only in gaining cooperation from older respondents at the initial survey request, but also in converting them at the refusal conversion phase. In this study, we explore the relationship between demographics of the respondent/household member and their decision to participate in telephone surveys. In addition, we explore how respondent’s gender and age interact with other variables such as the degree of interviewer–respondent/household member interaction. This exploratory study examines whether the outcome of refusal conversion efforts (i.e., successful conversion or double refusal) are influenced by (1) age and gender of the respondent/household member, (2) degree of interaction between the interviewer and the respondent/household member, and (3) whether the interviewer conveyed information (e.g., how the respondent was sampled, the nature and purpose of the survey) to the respondent/household member.
2. Method Data are based on information collected from three types of forms filled out by interviewers at a Survey Research Center affiliated with a large Midwestern university from January 2002 to August 2002 across three distinct telephone surveys. Random digit dialing (RDD) was used to first sample households. Then, the “last birthday method” (Oldendick, Bishop, Sorenson, & Tuchfarber, 1988) was employed to randomly select a respondent within each household. Advance letters to alert households/respondents about the survey were not sent out because there are no means to identify addresses of respondents who are randomly chosen through RDD. Interviewers filled out a Refusal Report Form when the respondent/household member refused to do the survey. During the refusal conversion phase, interviewers were asked to complete a Refusal Conversion Report Form if they successfully converted a refusal or a Double Refusal Report Form if the interview was refused again. The main contents of these forms were refuser’s (respondent or household member) demographic information (actual or interviewer’s perception), degree of interaction between the interviewer and the refuser, reasons for first/second refusal, strength of refusal, whether the interviewer conveyed information about the nature/legitimacy of the survey, and whether interviewer had revealed the previous refusal history to the respondent/household member. Altogether there were 157 successful conversions and 550 double refusals that were with valid documentations. Our study is based on 157 randomly selected double refusals cases and 157 successful conversions cases, yielding a total of 314 cases. Although a great amount of information was collected from these original forms, we focused on examining variables that were present in both outcome situations. These variables include data from the Refusal Report Form, which include (a) degree of interaction during first attempt, (b) reasons for initial refusal, (c) information about the survey revealed to the respondent/household member, and (d) demographics (respondent/household member’s age and gender reported by telephone interviewer). Specifically, the variables examined in this study are as follows.
438
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
2.1. Dependent variable 2.1.1. Outcome of second call attempt The dependent variable is the outcome of the second interview attempt following a refusal. Successfully converted cases are coded as “1” and double refusals are coded as “0.” 2.2. Independent variables 2.2.1. Degree of interaction Interviewers reported their perceived degree of interaction with the respondent/household member during the introduction on a scale of 1–7 where 1 being “no interaction at all” and 7 being “much interaction.” The degree of interaction was reported after the first call attempt (on the Refusal Report Form) as well as after the second attempt (on the Refusal Conversion Form or the Double Refusal Report Form). 2.2.2. Reason for initial refusal Interviewers indicated whether the refusal was due to any of the following reasons: (a) immediate hang-up, (b) time-related concerns, reasons, and excuses, (c) objections to surveys, topics, and sponsors, and (d) concerns about confidentiality and legitimacy of survey. Each of the four variables was coded as binary variables where 1 means that the refusal was due to the given reason and 0 means that it was not. 2.2.3. Information revealed Interviewers were given a set of standardized statements to address specific concerns often brought up by respondents. Interviewers reported whether they revealed any of the following information to the respondent/household member: (a) how he/she was sampled,1 (b) the nature, purpose of survey beyond the standard introduction, (c) confidentiality, (d) how the data would be used, (e) verification with supervisor/sponsor. This information was collected after the first call attempt (on the Refusal Report Form) as well as after the second attempt (on the Refusal Conversion Form or the Double Refusal Report Form). These variables were also coded as binary variables where 1 means that the given information was revealed and 0 means that it was not revealed. 2.2.4. Previous refusal history revealed to the householder Interviewers indicated whether they used the phrase, “Recently we contacted someone in your household about participating in a very important study . . . unfortunately, it was not a good time for them” during the introduction. Interviewers were encouraged to use this phrase when attempting refusal conversions. Again, this is a binary variable where 1 means that the interviewer used this phase and 0 means that they did not mention the phrase. This information was collected from the Refusal Conversion Form as well as the Double Refusal Report Form. 2.2.5. Demographics On the Refusal Report Form and Double Refusal Report Form, interviewers reported their perceived gender (male, female, or uncertain) and age (child, adult <30, 30–59, 60+, uncertain)
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
439
of the person whom they spoke to. The gender variable was recoded (1 = female; 0 = male; missing = uncertain) to be able to make comparisons between the genders. Age was also recoded (1 = 60+; 0 = under 60; missing = uncertain) in order to compare elderly respondents with other age groups. Perceived demographics of the respondent were not asked on the Refusal Conversion Form; instead, demographics of the converted respondent were pulled from the actual completed survey. As with “perceived” demographics, “actual” gender was coded as 1 = female; 0 = male, and “actual” age was recoded as 1 = 60+; 0 = under 60.
3. Results T tests, χ2 tests, and logistic regression analysis were performed using the outcome of refusal conversion (successful refusal conversion or double refusal) as the dependent variable and other aforementioned variables as predictors. We investigate factors that impact the refusal conversion process. Table 1 presents descriptive statistics of all variables included in the study. To get a general idea of how the independent variables are related to the outcome variable, we performed t tests and χ2 tests. We ran t tests to look at whether there is a mean difference in the degree of interviewer and respondent/household member interaction at first and second call attempts between converted cases and double refused cases (see Table 2). Results suggested that the mean level of interaction during the initial contact was slightly higher for converted cases than for double refused cases, although the individual contacted at the first attempt may not have necessarily been the individual contacted during the second call attempt. The difference in the mean degree of interaction between the two outcomes was greater at the refusal conversion phase. While the mean level of interaction was 1.84 for double refused cases, that of the converted cases was 3.70, indicating that during the refusal conversion phase, significantly higher levels of interaction between the interviewer and respondent/household member occurred for successfully converted cases as opposed to double refused cases. We also created a variable that measures the extent to which information about the survey was revealed to the respondent/household member. This variable combines five items on whether certain information was revealed (i.e., how she was sampled, the nature/purpose of the survey, confidentiality, how the data would be used, verification with supervisor/sponsor). This variable ranges from 0 to 5 where 0 means none of them were revealed and 5 means all five of them were provided. We examined mean differences in these scores. The amount of information revealed to the respondent/household member at the initial contact stage was not significantly different between the two refusal conversion effort outcomes. However, during the second call attempt, significantly more information was revealed when cases resulted in a successful conversion than when they resulted in a double refusal. Table 3 summarizes χ2 test results that examined the relationship between the dependent variable and independent variables that are binary. These results suggested that if the initial refusal was due to immediate hang-up or time-related concerns by the respondent/household member during the first call attempt, it was more likely to result in a double refusal at the
440
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
Table 1 Descriptive statistics of variables in the study Valid n
Mean
S.D.
Dependent variable Outcome of second call attempt
314
0.50
0.50
Independent variables Information from first call attempt (Refusal Report Form) Degree of interaction
288
1.79
1.20
Reason for initial refusal Immediate hang-up Time-related concers, reasons, excuses Objects to surveys, topics, sponsors Concerned about confidentiality, legitimacy of survey
309 308 308 308
0.60 0.12 0.14 0.01
0.49 0.33 0.35 0.10
Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
314 314 314 314 314
0.27 0.22 0.09 0.09 0.04
0.45 0.42 0.29 0.29 0.18
Demographics Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
292 308
0.60 0.05
0.49 0.22
310
2.78
1.72
Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
314 314 314 314 314
0.28 0.43 0.34 0.33 0.05
0.45 0.50 0.47 0.47 0.22
Previous refusal history revealed to the respondent/household member
305
0.58
0.49
Demographics (perceived or actual) Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
307 227
0.59 0.32
0.49 0.47
Information from second call attempt (RC Form or DR Report Form) Degree of interaction
Table 2 Mean differences in independent variables between refusal conversion effort outcomes
Degree of interaction during first call attempt Information revealed during first call attempt
*
Degree of interation during second call attempt** Information revealed during second call attempt** ∗ ∗∗
Mean difference significant at p < .05. Mean difference significant at p < .001.
Outcome = RC
Outcome = DR
Mean
S.D.
Mean
S.D.
1.93 0.81
1.31 1.30
1.64 0.61
1.06 1.21
3.70 2.27
1.67 1.73
1.84 0.59
1.17 1.33
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
441
Table 3 Summary of χ2 test results Independent variables
Related to DV (p < .05)?
Information from first call attempt (Refusal Report Form) Reason for initial refusal Immediate hang-up Time-related concers, reasons, excuses Objects to surveys, topics, sponsors Concerned about confidentiality, legitimacy of survey
Yes Yes No No
Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
No No No No No
Demographics Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
No No
Information from second call attempt (RC Form or DR Report Form) Information revealed How s/he was sampled The nature, purpose of survey beyond the standard introduction Confidentiality How the data would be used Verification with supervisor/sponsor
Yes Yes Yes Yes No
Previous refusal history revealed to the respondent/household member
Yes
Demographics (perceived or actual) Gender (1 = female; 0 = male) Age (1 = 60+; 0 = under 60)
No No
second call attempt. Other variables obtained from the Refusal Report Form did not appear to affect the outcome of the refusal conversion. However, we need to be reminded that we do not know whether the person who refused or was converted at the second attempt was the same individual we contacted during the initial attempt. Therefore, it is misleading to assume that the nature of a refusal during the first call attempt directly affects the outcome of the second call attempt. Results indicate that the amount of information revealed to the respondent/household member during the refusal conversion phase affected the outcome. Clearly, trying to convey information that justifies the legitimacy of the survey and revealing the previous call history were effective in refusal conversions. The only variable that did not appear to affect the outcome based on χ2 results was whether the respondent sought to verify with the supervisor/sponsor. However, this may be due to the fact that very few people in the sample sought verification. Since there is not much previous research that guides us to propose a model in predicting the refusal conversion outcomes, the models in the following logistic regression analyses are exploratory (see Table 4). We simply included independent variables that were significant at the bivariate level (t test and χ2 analyses) and demographics as covariates in our models. The predictors in Model 1 include the two reasons for initial refusal (i.e., immediate hang-up
442
Table 4 Logistic regression estimates for determinants of refusal conversion effort outcomes Independent variables
Model 1 B
Model 2 Odds ratio
Note. Standard errors are in parentheses. ∗ p ≤ .05 two-tailed test. ∗∗ p ≤ .01 two-tailed test. ∗∗∗ p ≤ .001 two-tailed test.
B
Odds ratio
0.155 (0.428) 0.764 (0.643)
1.167 2.148
B
Model 4 Odds ratio
B
Odds ratio
0.742 (0.630)
2.101
0.764 (0.639)
2.146
0.141 (0.174)
0.868
0.130 (0.174)
0.878
−1.376 (0.721) 0.770 (0.930)
0.253 2.159
−1.607* (0.806) 1.419 (1.009)
0.200 4.134
−1.518 (0.834) 1.558 (1.061)
0.219 4.748
−0.798 (0.844) 0.816 (0.630) 0.502* (0.201)
0.450 2.261 1.652
−0.725 (0.926) 0.764 (0.682) 0.891*** (0.219)
0.484 2.146 2.437
−1.013 (0.991) 0.526 (0.688) 0.506* (0.218)
1.045 1.692 1.659
0.563 (0.422)
1.757
0.672 (0.427)
1.959
0.595 (0.435)
1.812
−3.133*** (0.864)
0.044
−0.939* (0.418)
0.391
−2.916*** (0.881)
0.054
−0.143 (0.418) 0.940** (0.310)
0.867 2.560
0.029 (0.427)
1.030
−0.060 (0.444) 0.851** (0.322)
0.942 2.342
−0.730 172.6 104.1 (11) 212
−1.450 164.7 85.8 (10) 196
−0.473 157.6 92.9 (11) 196
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
Information from first call attempt (Refusal Report Form) Reason for initial refusal Immediate hang-up 0.062 (0.410) 1.064 Time-related concers, 0.709 (0.635) 2.031 reasons, excuses Degree of interaction during first call attempt Information from second call attempt (RC Form or DR Report Form) Information revealed 0.235 How s/he was sampled −1.450* (0.696) 0.631 (0.887) 1.880 The nature, purpose of survey beyond the standard introduction Confidentiality −0.491 (0.801) 0.612 How the data would be used 1.081 (0.624) 2.949 Degree of interaction during 0.925*** (0.215) 2.521 second call attempt Previous refusal history 0.612 (0.409) 1.843 revealed to the householder Demographics (perceived or actual) Gender (1 = female; −0.897* (0.405) 0.408 0 = male) Age (1 = 60+; 0 = under 60) −0.039 (0.401) 0.961 Gender × Degree of interaction during second call attempt (interaction term) Constant −1.795 −2 log likelihood 182.0 94.7 (10) χ2 (d.f.) n 212
Model 3
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
443
and time related concerns). Other predictors included in Model 1 are from the refusal conversion phase, including specific information revealed to respondent/household member, degree of interaction in the conversion phase, whether the previous refusal history was revealed to respondent/household member, and interviewers’ perceived respondent/household member’s age and gender. Model 2 is the same as Model 1 except for the added interaction term between gender and degree of interviewer–respondent/household member interaction. Models 3 and 4 are the same as Models 1 and 2, respectively, except that immediate hang-up as a reason for initial refusal was substituted by degree of interviewer–respondent/household member interaction during the initial call attempt. These two variables were not included in the same model because they are highly correlated with each other. The results indicate that across the four models, reasons for refusal (immediate hang-up and time-related concerns) and degree of interviewer–respondent/household member interaction during the initial call attempt were no longer predictive of conversion outcomes when variables from the second call attempt and demographic variables were controlled. All factors that appear to affect the conversion outcomes were from the refusal conversion phase. Specifically, the degree of interviewer–respondent/household member interaction during the conversion phase appeared to be a significant predictor, that is, when the interaction between the interviewer and the respondent/household member was greater, the likelihood in resulting in a successful conversion was greater. Respondent/household member’s gender also appeared be related to the outcome. Female respondents/household members were less likely to be successfully converted. A significant interaction effect between gender and the degree of interviewer–householder interaction was found in both Models 2 and 4. This interaction effect suggested that female respondents/householders were less likely to be converted at the refusal conversion stage when the interaction between her and the interviewer was low. Overall, the results indicated that the type of information revealed to respondent/householder did not affect the outcome very much. The only variable that had an impact was the information, “how he/she was sampled” was revealed. However, the direction of this effect was not what we had expected since providing this information tended to result in a double refusal. Contrary to our expectation, perceived respondent/householder’s age did not affect the outcome.
4. Discussion Results indicate that the degree of interviewer–respondent/household member interaction during the second call attempt mattered more than any other variable in converting a refused case. This result provided further empirical support for the importance of interviewer–respondent/household member interaction in gaining cooperation to participate in telephone interviews that has been emphasized by previous researchers (Groves, 1979; Groves et al., 1992; Groves & McGonagale, 2001; Miller & Cannell, 1982). Our results also provide support for the narrative interviewing style proposed by Mishler (1986) as opposed to traditional standardized interviewing. Our results suggest an interviewing approach with an emphasis on maintaining interaction with the respondent/household member is effective in successfully converting a refused case.
444
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
It is believed that conveying certain information about the survey (i.e., how he/she was sampled, how the data will be used, who the sponsor of the survey is, and confidentiality) influences the respondent/household member’s perceived legitimacy of the survey thus helps gain their cooperation to do the survey. As we mentioned earlier, the amount of information revealed during the introduction and interaction between the interviewer and respondent/household member is highly correlated. Therefore, we attempted to explore whether the interviewer simply needs to engage the householder in a “chit-chat” conversation, or has to reveal certain information during their interaction in order to convince respondents to participate in an interview. Our results suggested that engaging the respondent/household member in a more conversation-like interaction matters more than revealing certain information during the introduction. Interestingly, the only type of information revealed to the respondent/household member that had an impact on the outcome was how she/he was sampled, though the relationship was in an unexpected direction. Our findings suggested that when the degree of interviewer–respondent/household member interaction was held constant, revealing such information was associated with less success in converting than when it was not revealed. Therefore, revealing how respondents were sampled seemed to have a “boomerang effect.” This effect is based on theory of psychological reactance that postulates that people have a motive to protect or restore one’s sense of freedom and will react against attempts to control their behavioral freedom (Brehm, 1966). In the context of telephone survey interviews, this boomerang effect is a negative mental set that the respondent/household member has toward any call from a stranger. Perhaps the recent increase in the number of telemarketing calls reinforces this mindset. With such a negative attitude towards phone calls from strangers, the respondent/household member might be likely to refuse if the interviewer rushed to reveal information about the survey. At the survey research center where our data were collected, information on how respondents were sampled is on the top of the list of information that interviewers are trained to bring up when respondents/household members appear skeptical about the legitimacy of the survey. Therefore, it was likely that refusal conversion attempts often fail quickly after the interviewer reveals the first piece of information that justifies the survey. Thus, a general strategy to successfully convert a refused case might be to engage respondents/household members in an interaction as much as possible and not rush to reveal information concerning the legitimacy of the survey. However, such strategy might be more effective in converting a female respondent/household member than a male. Our findings suggested an interaction effect between respondent/household member’s gender and degree of interaction where women are more likely to refuse during the refusal conversion stage if their interaction with the interviewer is not heightened. Previous research found that respondent’s age might affect their decision to participate in a survey. Older householders were found to be more likely to refuse in the initial request of survey (DeMaio, 1980; Hawkins, 1975; Herzog et al., 1983). The present study did not find that age is related to the outcome of refusal conversions, although this might be due to interviewer’s inaccurate perception of the respondent/household member’s age. One major limitation of the present study is that we were unable to determine whether the person contacted at the refusal conversion phase was the same individual we contacted at the
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
445
initial attempt. Due to this limitation, it is difficult to examine how factors from the first call attempt affect the outcome of refusal conversion efforts. In the future, we may also need to focus on variables such as respondent’s interest in the topic of the survey and their perceived legitimacy of the survey, as these variables also serve to shape people’s decision on whether to participate in a survey (Groves, Singer, & Corning, 2000).
Note 1. For instance, the standardized language interviewers were instructed to use to describe how households were selected reads, “Your number was randomly chosen using a technique called Random-Digit Dialing. The way this works is that a computer randomly chooses an area code and a three-digit prefix that works within that area code. Then, the computer randomly adds four more numbers to make up a telephone number, like the one I used that reached your home. Many times we dial numbers that are not working numbers; others reach businesses, and some reach households such as yours. We do this because it’s important that we interview a random sample of [place] residents regardless of whether they have a listed or unlisted telephone number.”
Acknowledgements This manuscript was based on a paper presented at the 60th Annual Conference of American Association of Public Opinion Research (AAPOR), Miami Beach, Florida, May 12–15, 2005. Authors contributed equally to the research.
References Brehm, J. (1966). A theory of psychological reactance. New York: Academic Press. DeMaio, T. J. (1980). Refusals: Who, where, and why. Public Opinion Quarterly, 44, 223–233. Groves, R. M. (1979). Actors and questions in telephone and personal interview surveys. Public Opinion Quarterly, 43, 190–205. Groves, R. M., Cialdin, R. B., & Couper, M. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475–495. Groves, R., & Couper, M. (1998). Nonresponse in household interview surveys. New York: Wiley & Sons Inc.. Groves, R., Fowler, F., Couper, M., Lepkowski, J., Singer, E., & Tourangeau, R. (2004). Survey methodology. New York: Wiley. Groves, R., & McGonagale, K. (2001). A theory-guided interviewer training protocol regarding survey participation. Journal of Official Statistics, 17, 249–265. Groves, R., Singer, E., & Corning, A. (2000). Leverage-salience theory of survey participation: Description and an illustration. Public Opinion Quarterly, 64, 299–308. Hawkins, D. F. (1975). Estimation of nonresponse bias. Sociological Methods and Research, 3, 462–485. Herzog, A. R., Rodgers, W. L., & Kulka, R. A. (1983). Interviewing older adults: A comparison of telephone and face-to-face modalities. Public Opinion Quarterly, 47, 405–418. Mayer, T. S., & O’Brien, E. (2001). Interviewer refusal aversion training to increase survey participation. In Proceedings of the annual meeting of the American Statistical Association.
446
K. Fuse, D. Xie / The Social Science Journal 44 (2007) 434–446
Miller, P., & Cannell, C. (1982). A study of experimental techniques for telephone interviewing. Public Opinion Quarterly, 46, 250–269. Mishler, E. G. (1986). Research Interviewing. Cambridge: Harvard University Press. Oldendick, R., Bishop, G., Sorenson, S., & Tuchfarber, A. (1988). A comparison of the Kish and last birthday methods of respondent selection in telephone surveys. Journal of Official Statistics, 4, 307–318. Singer, E., Groves, R., & Corning, A. (1999). Differential incentives: Beliefs about practices, perceptions of equity, and effects on survey participation. Public Opinion Quarterly, 63, 251–260. Singer, E., Van Hoewyk, J., & Maher, M. P. (2000). Experiments with incentives in telephone surveys. Public Opinion Quarterly, 64, 171–188.