- Email: [email protected]
Participation Rates in a Case-Control Study:
Participation Rates in a Case-Control Study: The Impact of Age, Race, and Race of Interviewer PATRICIA G. MOORMAN, PhD, BETH NEWMAN, PhD, ROBERT C. MILLIKAN, DVM, PhD, CHIU-KIT J. TSE, MSPH, AND DALE P. SANDLER, PhD
PURPOSE: Despite concerns about declining participation rates in epidemiologic studies in recent years, relatively few papers have discussed obstacles to recruiting study participants or strategies for optimizing response rates. This report describes factors associated with nonparticipation in a populationbased, case-control study of breast cancer and discusses ways to overcome barriers to participation. METHODS: Contact and cooperation rates were calculated for participants in the Carolina Breast Cancer Study (CBCS), stratified by case status, age, race, and race of interviewer. Demographic and breast cancer risk factor characteristics of partial and full responders also were compared. RESULTS: Contact rates and cooperation rates varied by case/control status and demographic characteristics. Contact rates were lower among controls, younger women, and black women. Cooperation rates were lower among controls, older women, and black cases. Cooperation rates were higher among both black and nonblack women when participants and interviewers were concordant on race. CONCLUSIONS: Obstacles to recruitment seem to differ among race and age subgroups, suggesting that recruitment strategies may need to be tailored to potential participants based upon demographic characteristics. Strategies have been implemented to improve response rates in this and other epidemiologic studies; however, additional research and innovation in this area are needed. Ann Epidemiol 1999;9:188–195. 1999 Elsevier Science Inc. All rights reserved. KEY WORDS:
Response Rates, Epidemiologic Methods, Breast Cancer, Blacks.
INTRODUCTION High response rates are important in epidemiologic research for both validity and generalizability to the target population. However, the perception among epidemiologists is that response rates are lower currently than 10 to 15 years ago (1). In addition, there is concern about participation rates among certain subsets of the population, among these, minorities and elderly persons (2, 3). Minorities are underrepresented in research studies for a variety of reasons, including study area demographics (4, 5), mistrust of medical research because of past abuses, differences in sociocultural beliefs, and socioeconomic characteristics (2, 3). In the case of elderly persons, reluctance to participate in research may be caused by safety concerns and fears that they might become targets of financial swindles. In addition, poor
health status may be a real or perceived barrier to elderly citizens’ participation in research studies (6, 7). Despite widespread concern about the impact of decreased participation in epidemiologic research, relatively few papers have discussed strategies for optimizing response rates. This report describes response rates in a populationbased, case-control study of breast cancer, with a particular emphasis on differences by age and race of the respondent, as well as the impact of the interviewer’s race. Comparisons also are presented between women who completed a full, in-person interview and those who did only an abbreviated telephone survey. The discussion focuses on obstacles to achieving contact with and interviewing potential study participants and methods to improve response rates.
METHODS From the Department of Epidemiology and Public Health (P.G.M.), Yale University School of Medicine, New Haven, CT 06520; Department of Epidemiology (B.N., R.C.M., C-K.J.T.), University of North CarolinaChapel Hill, Chapel Hill, NC 27599; and Epidemiology Branch (D.P.S.), National Institute of Environmental Health Sciences, Research Triangle Park, NC 27709. Address reprint requests to: Dr. Patricia G. Moorman, Department of Epidemiology and Public Health, Yale University School of Medicine, 60 College Street, P.O. Box 208034, New Haven, CT 06520-8034. Received October 27, 1998; accepted October 27, 1998. 1999 Elsevier Science Inc. All rights reserved. 655 Avenue of the Americas, New York, NY 10010
The Carolina Breast Cancer Study (CBCS) is an on-going, population-based, case-control study that began in 1993 (8). Phase I of the study, during which 889 cases and 841 controls were interviewed, was completed in 1996. Cases were identified through a rapid case ascertainment system developed in conjunction with the North Carolina Central Cancer Registry (CCR) (9). Pathology reports for all breast cancers diagnosed in the hospitals in the study area were forwarded 1047-2797/99/$–see front matter PII S1047-2797(98)00057-X
AEP Vol. 9, No. 3 April 1999: 188–195
Selected Abbreviations and Acronyms CBCS 5 Carolina Breast Cancer Study CCR 5 Central Cancer Registry DMV 5 Division of Motor Vehicles HCFA 5 Health Care Financing Administration
to the CCR and then to the CBCS, most within 1 month of diagnosis. Eligible cases were between the ages of 20 and 74 years, resided in a 24-county area of central and eastern North Carolina, and had no prior history of breast cancer. Black women and women under age 50 were oversampled, using a modification of randomized recruitment (10), with the goal of obtaining roughly equal numbers of black and nonblack women and women diagnosed before and after age 50. Permission to contact breast cancer cases was requested from the physician of record on the pathology report. Controls between the ages of 20 and 64 years were randomly selected from North Carolina Division of Motor Vehicle (DMV) lists of women residing in the study area who had either a driver’s license or an identification card. Controls between the ages of 65 and 74 years were selected from U.S. Health Care Financing Administration (HCFA) lists. Sampling fractions based on race (dichotomized as black or nonblack based on race information from the DMV or HCFA) and 5-year age groups were used to select controls who were approximately frequency-matched to cases using randomized recruitment. Cases for whom physician consent was obtained and controls were mailed a letter describing the study and inviting them to participate. The letters were followed by telephone calls from a registered nurse, who scheduled an interview, usually in the home, for women who agreed to participate. During the interview visit, the nurse administered a 1-hour questionnaire, took body measurements (height, weight, and waist and hip circumferences), and obtained a 30-mL blood sample. Participants were paid $25 for completing the interview. Women who declined the home visit were asked to complete an abbreviated (approximately 5-minute) telephone interview. The study protocol was approved by the institutional review boards of the University of North Carolina School of Medicine and the participating hospitals. Neither the DMV nor the HCFA list provided telephone numbers for potential controls; therefore, multiple strategies were employed to identify telephone numbers, including computerized directories on CD-ROM, conventional telephone directories, directory assistance, and reverse directories. Repeated mailings, including one sent by certified mail, each included a postage-paid card for the woman to return if we did not have her telephone number or if the number we identified was incorrect. A toll-free telephone number was also included for women to use when contacting the
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
189
study office. Other strategies to contact or resolve the status of unlocatable women included checking the National Change of Address Service (11), and checking state death records. In addition, lists of unlocatable women were checked periodically against current DMV records in an attempt to identify new addresses or moves out of state. As a last resort, the nurse-interviewers made visits to the residences of those who could not be contacted by telephone. Measures suggested by Slattery et al. (1) were used to describe various aspects of response. “Contact rate” is defined as the percentage of women who were identified as potential study participants with whom contact was achieved. “Cooperation rate” is the percentage of women interviewed of those who were contacted and eligible. “Over-all response rate” is the percentage of women interviewed of those who were selected and eligible for the study. Over-all response rate includes individuals who were not contacted in the denominator, although these women are excluded from the denominator for cooperation rate. Women of all races were included in the study; however, white women and black women comprised the majority of the population in the North Carolina study area. Asian Americans, Native Americans, and women of other races represented only 1.6% of the study population. For the racespecific analyses presented in this paper, the “black” category includes all women described as black or African American, and the “nonblack” category includes all other women. When the CBCS was initiated, interviewers and participants were not matched on race. However, after interviewers reported more refusals from women not of their race, particularly older women, matching on race among participants age 50 and older was initiated. The impact of interviewer’s race was assessed by comparing cooperation rates when the interviewer and potential participant were concordant or discordant on race. The differences in the proportion of women who consented to be interviewed and 95% confidence intervals were calculated. Demographic characteristics and breast cancer risk factors were compared for women who consented to the complete, in-person interview and those who agreed only to a partial, telephone interview. Differences in mean values or proportions between women completing the full questionnaire versus those doing the partial survey, stratified by case status, were calculated along with the 95% confidence intervals for the differences.
RESULTS Table 1 is a detailed breakdown of the disposition of potential study participants. Of the 1285 identified breast cancer patients, 889 were interviewed. Of 1797 potential control participants, 841 were interviewed. Those not interviewed included ineligible women, unlocatable women, physician
190
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
AEP Vol. 9, No. 3 April 1999: 188–195
TABLE 1. Responses of women selected as potential participants for the CBCS by case-control status, race, and age Cases Whites Total Sampled (N) Ineligible (%)a Deceased (%) Uncontactable (%) Physician refusal (%) Participant refusal (%) Interviewed Full (%) Partial (%) Contact rateb Cooperation ratec Overall response rated
1285 6.1 0.9 3.3 5.7 14.9 67.2 1.9 96.7% 77.1% 74.4%
20–39 yrs 119 3.4 0 2.5 5.9 5.9 81.5 0.8 97.5% 87.5% 85.2%
40–49 yrs
Blacks
50–64 yrs
318 4.1 0.3 3.1 6.6 11.0
164 7.9 0.6 3.0 9.8 11.6
73.6 1.3 96.9% 81.0% 78.2%
65.2 1.8 97.0% 75.9% 73.3%
. 64 yrs
20–39 yrs
138 7.2 0 0 8.0 18.8
86 2.3 2.3 4.7 2.3 14.0
64.5 1.4 100.0% 71.1% 71.1%
70.9 3.5 95.3% 82.1% 78.0%
40–49 yrs 168 8.3 0 4.8 4.2 12.5
50–64 yrs 162 4.3 2.5 5.6 2.5 23.5
67.9 2.4 95.2% 80.8% 76.6%
. 64 yrs 130 11.5 3.1 2.3 3.8 25.4
59.3 2.5 94.4% 70.4% 66.2%
50.8 3.1 97.7% 64.8% 63.1%
50–64 yrs
. 64 yrs
Controls Whites Total Sampled (N) Ineligible (%)a Deceased (%) Uncontactable (%) Participant refusal (%) Interviewed Full (%) Partial (%) Contact rateb Cooperation ratec Overall response rated
1797 9.8 1.6 19.4 22.5 44.0 2.8 80.6% 67.6% 52.8%
20–39 yrs 119 17.6 0 20.2 15.1 46.2 0.8 79.8% 75.7% 57.1%
40–49 yrs 350 9.7 0.3 14.3 20.9 51.7 3.1 85.7% 72.5% 61.0%
Blacks
50–64 yrs 265 11.3 1.5 10.9 25.7 46.4 4.2 89.1% 66.3% 58.0%
. 64 yrs 191 9.9 2.1 5.2 27.2 51.8 3.7 94.8% 67.1% 63.1%
20–39 yrs 140 10.0 0 39.3 17.9 32.1 0.7 60.7% 64.8% 36.5%
40–49 yrs 283 7.1 1.1 27.6 16.6 44.5 3.2 72.4% 74.2% 51.9%
253 3.6 2.4 23.7 26.5 40.7 3.2 76.3% 62.4% 46.6%
196 14.8 5.1 21.4 27.6 29.6 1.5 78.6% 53.0% 38.9%
a
Eligibility criteria include age (20–74 years), female gender, residence in 24-county study area, able to complete an interview in English, and no prior history of breast cancer. b Contact rate 5 # of women contacted/# of women identified as potential cases or controls. c Cooperation rate 5 # of completed interviews/# of women contacted and eligible. d Over-all response rate 5 # of completed interviews/# of women selected for study—ineligible and deceased women.
refusals (for cases), and participant refusals. Within these categories, there was variation by case-control status, age, and race. Among cases, 6% were found to be ineligible, and among controls, 10% were ineligible. Ineligibility was determined by age at diagnosis (for cases) or selection (for controls) older than 74 years, permanent residence outside study area, prior history of breast cancer, or inability to complete an interview in English (i.e., did not speak English or was cognitively impaired by Alzheimer’s disease, mental retardation, etc.). Ineligibility among cases was most often because of a prior history of breast cancer; whereas, current residence outside the study area was the most common reason for ineligibility among controls. Physician consent was not obtained for approximately 6% of eligible cases, ranging from 2.3% of the youngest black women to 9.8% of nonblack women aged 50–64 years. Approximately half the refusals were from four physicians (out of more than 150 physicians who treat breast cancer
in the study area) who declined consent for all of their patients. The others were for reasons specific to a given patient (e.g., patient was considered too ill or emotionally distraught). The higher percentage of physician refusals for nonblack women, particularly at older ages, largely reflected the demographics of the practices of those doctors who chose not to allow us to invite their patients to participate in the study. The proportion of women who were unlocatable or uncontactable differed among case-control, race, and age strata. Only a small proportion of cases (3%) were in this category, consistent with the availability of current contact information from our rapid case ascertainment system. However, nearly one-fifth of all potential controls could not be located or contacted. More black women than nonblack women, and within each race, more younger than older women could not be contacted. Participant refusal rates were lower among cases. Women age 50 and older were more likely to refuse to participate,
AEP Vol. 9, No. 3 April 1999: 188–195
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
191
TABLE 2. Cooperation rates for cases and controls by age, race of participant, and race of interviewer Interviewer race Concordant
Nonblack cases, age , 50 Nonblack cases, age > 50 Black cases, age , 50 Black cases, age > 50 Nonblack controls, age , 50 Nonblack controls, age > 50 Black controls, age , 50 Black controls, age > 50 a
Discordant
N
Cooperation rate (%)a
N
Cooperation rate (%)a
Difference in cooperation rate (95% CI)
199 203 142 169 154 302 154 218
92.5 83.7 89.4 77.5 75.3 69.5 77.9 63.3
168 34 66 62 181 51 97 69
90.5 91.2 83.3 62.9 72.9 58.8 62.9 49.3
2.0 (23.7–7.7) 27.5 (218.3–3.2) 6.1 (24.2–16.4) 14.6 (1.0–28.2) 2.4 (27.0–11.8) 10.7 (23.8–25.2) 15.0 (3.4–26.6) 14.0 (0.6–27.4)
Denominators exclude cases for whom physician consent was not obtained, women who were never located, or women who called the study office declining to participate after they received a letter of invitation but before being telephoned by the nurse.
whether they were cases or controls. Race differences were observed among cases, with black women less likely to participate. Among controls, however, within each age category, refusal rates were similar for black and nonblack women. The lower over-all response rates for blacks reflected the lower contact rates for these women. Table 2 presents cooperation rates by participant race and interviewer race among older and younger women. Because these analyses are intended to assess the influence of interviewer race on the likelihood of a woman agreeing to be interviewed, the reported cooperation rates are for women who were actually contacted by a nurse. The denominators exclude cases for whom physician consent was not obtained and women who called the study office declining to participate after they received a letter of invitation but before being telephoned by a nurse. Consequently, the cooperation rates in this table are higher than the cooperation rates for these age/race categories calculated according to the criteria used in Table 1, which were 83, 74, 81, and 68% for younger and older nonblack cases and younger and older and black cases, respectively. The corresponding rates for these age and race strata among controls were 73, 67, 72, and 59%. In seven of the eight age, race, and case/control categories, cooperation rates were higher when the interviewer and participant were concordant on race. However, the magnitude of this effect varied by age and race of the participant. Over-all cooperation rates were highest among younger nonblack women for whom race of the interviewer had little impact. Among older nonblack controls, the cooperation rate was about 10% higher when they were invited to participate by white interviewers, although this difference was not statistically significant. Among older nonblack cases, cooperation rates were actually higher among those assigned to a black interviewer; however, this is based on small numbers. Cooperation rates were higher among black women when they were invited to participate by a black interviewer; this was true for cases and controls as well as
for younger and older women. Having the interviewer and participant concordant on race improved cooperation rates by 6% (young black cases) to 15% (young black controls). As shown in Table 3, comparisons were made between women who completed the full, in-person questionnaire with those who did only a partial, telephone interview, stratified by case-control status. Among sociodemographic characteristics, women who completed the partial interview were slightly older, had less education, and were less likely to have been married or to have held a job for at least 6 months. Comparisons of potential breast cancer risk factors were less consistent in characterizing who was likely to complete the full interview. Those who were partially interviewed were less likely to report having used oral contraceptives or estrogen replacement therapy, but were more likely to report a family history of breast or ovarian cancer. They were similar to women who completed the full questionnaire for a number of breast cancer risk factors, including age at menarche, age at menopause, pregnancy history, and body mass index. For most characteristics, differences between those fully participating and those agreeing to the partial interview were in the same direction for cases and controls. However, race, menopausal status, and history of benign biopsy were exceptions to this observation. It should be noted that the number of partial interviews was relatively small, and the confidence intervals around the differences in means or proportions for all risk factors, except use of estrogens, were consistent with no difference between women completing the full versus the partial interview.
DISCUSSION In this study, over-all response rates varied by case-control status, age, and race of participant. Examination of the components of the over-all response rate—the contact rate and the cooperation rate—revealed different obstacles to achieving
192
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
AEP Vol. 9, No. 3 April 1999: 188–195
TABLE 3. Characteristics of women completing full, in-home interview compared to those completing a partial, telephone interview, by case-control status Cases Characteristic Race (% black) Education (% , 12 years) Marital status (% never married) Ever had a job (%) Ever use of oral contraceptives (%) Ever use of estrogens (%) Premenopausal (%) History of benign biopsy (%) Breast cancer in 1st degree relative (%) Ovarian cancer in 1st degree relative (%) Age (mean) Age at menarche (mean) Age at menopause (mean) Number of pregnancies (mean) Age at 1st full-term pregnancy (mean) Body mass index (mean)
Controls
Full N 5 864
Partial N 5 25
Difference (95% CI)
Full N 5 790
Partial N 5 51
Difference (95% CI)
39.0 18.4 9.5 98.4 64.1 22.2 42.4 16.8 15.0 2.5 50.3 12.6 43.8 2.9 22.5 28.1
60.0 36.0 20.0 88.0 54.2 8.0 40.0 20.0 28.0 4.0 51.6 12.8 43.9 2.2 22.3 28.5
221.0 (240.0–1.5) 217.6 (236.6–1.3) 210.5 (226.3–5.3) 10.4 (22.4–23.1) 9.9 (210.3–30.1) 14.2 (3.2–25.2) 2.4 (217.1–21.8) 23.2 (219.0–12.7) 213.0 (230.7–4.8) 21.5 (29.2–6.2) 21.3 (26.1–3.4) 20.2 (20.9–0.4) 20.1 (24.4–4.2) 0.7 (20.2–1.6) 0.2 (22.1–2.5) 20.4 (23.2–2.5)
42.0 20.9 7.5 98.2 59.6 29.6 36.6 17.6 11.7 1.2 51.8 12.7 43.4 3.2 22.3 28.9
41.2 29.4 16.0 94.1 47.1 5.9 45.1 11.8 15.7 3.9 54.7 12.9 43.9 2.8 21.3 28.3
0.8 (213.0–14.8) 28.5 (221.4–4.3) 28.5 (218.9–1.8) 4.1 (22.4–10.6) 12.5 (21.6–26.6) 23.7 (16.5–30.9) 28.5 (222.5–5.6) 5.8 (23.3–15.1) 24.0 (214.2–6.3) 22.7 (28.1–2.6) 22.9 (26.2–0.4) 20.2 (20.7–0.2) 20.5 (23.5–2.5) 0.4 (20.3–1.1) 1.0 (20.6–2.5) 0.6 (21.4–2.6)
participation in specific subgroups. Lower contact rates were observed for controls, younger women, and black women. Cooperation rates were lower among black cases than nonblack cases, lower among controls of both races than cases, and lower among older women of both races than younger women. Achieving contact with the individuals selected as potential study participants, especially controls, is a major challenge of population-based research. Despite the multiple strategies employed to identify current addresses and telephone numbers, 19% of women identified as potential controls from the DMV and HCFA lists were never contacted. Contact efforts were less successful for younger women and for black women, presumably because of greater mobility and a smaller likelihood of having a telephone number listed in their name. A smaller proportion of women identified through the HCFA list could not be contacted. The more accurate address information from HCFA reflects both the lower mobility of older women and the underlying purposes of this roster (i.e., health insurance and social security). The difficulty in contacting women is attributable, in part, to the limitations of the sources used for selecting controls. Although DMV and HCFA lists are commonly used in epidemiologic research, have fairly complete coverage of the population, and provide address, race, and birthdate information, the contact information is not optimal. Telephone numbers generally are unavailable from either list, and address information is frequently not current. Outdated address information from the DMV list was a substantial limitation to achieving contact with potential participants and is likely to become an even greater problem now that some states are extending the length of time between license renewals. Other difficulties in using DMV lists as
sources of controls include increasingly restricted availability of these data as a result of a federal law limiting public access to personal information from drivers’ license records and a privacy option available in at least one state that allows drivers to indicate that their personal information cannot be released for business or survey purposes (12). Refusals represented the other major category of nonresponse. A number of factors have been identified that influence the likelihood of participation in research studies (13–15). Positive factors include case status, physician endorsement of the study, perceived legitimacy of the study, perceived importance of the study to health, and monetary incentives. Often cited negative factors include concerns about the time required to participate, privacy issues, and opposition to surveys in general. We attempted to account for these concerns when designing the contact materials and protocols. For example, the letter of invitation emphasized that physician consent had been obtained (for cases), and an enclosed question-and-answer sheet addressed concerns often raised by participants, including how they were selected, how their privacy would be protected, and how participating in the study would benefit them. In addition to factors applicable to the general population, reasons for non-participation specific to subgroups of the population have also been identified. Higher refusal rates among blacks have been attributed to the legacy of the Tuskegee syphilis study (2, 16) and to differences in attitudes and beliefs about disease (2, 3). Interviewer characteristics may come into play as well, as has been demonstrated in studies that show communication is least inhibited between respondents and interviewers of the same race and gender (17). Studies examining interviewer effects generally
AEP Vol. 9, No. 3 April 1999: 188–195
have been done in the context of in-person interviews and have focused on qualitative differences in responses based upon the interviewers’ and respondents’ race and/or gender (17, 18). It has been assumed that interviewer effects are of less importance in telephone surveys, because it is more difficult to differentiate characteristics of the interviewer (19). However, our study found that having interviewers and potential participants concordant on race had a substantial effect on the likelihood of participation, although the interviewers’ first contact with the women by telephone. It seems that some women’s decisions to participate were based on the perceived race of the interviewer, a judgment presumably made on the basis of accent or language patterns. It is important to note that the decision to match interviewer and older participants on race was made approximately 1 year into the study. It is possible that the improved cooperation rates when matching on race could have been attributable in part to the interviewers being more experienced later in the study. However, the magnitude of the effect in certain subgroups (e.g., the nearly 15% difference in cooperation rates among black controls and older black cases) as compared to the much smaller effects observed in others argues against this. Although matching on race seemed to have a substantial effect on cooperation rates, two observations can be made. First, the interviewers’ perception that race was a more important issue for older women was not entirely accurate, because the race of the interviewer made the largest difference in cooperation rates among young black controls. Second, even when matching on race, cooperation rates among black women tended to be lower than for nonblack women. The use of black interviewers increased the likelihood of participation, possibly by overcoming mistrust of medical researchers or conveying the message that the research was relevant to women of their race; however, the lower cooperation rates suggest that other barriers to participation remained. Older women were the other major subgroup for whom refusal rates tended to be high. Among these women, the biggest obstacle to achieving high participation rates seemed not to be in contacting them, but in overcoming their fears about taking part in a research study, especially one involving in-home interviews. Interviewers were sometimes able to alleviate these fears by offering to meet at another location (e.g., local hospital or health department), talking with a family member to confirm the legitimacy of the study, letting the participant know that it was acceptable to have another person present for the interview, or reassuring them, when appropriate, that existing medical conditions were not barriers to participation. However, in many cases, it was impossible to overcome the apprehension that older women had about speaking with a stranger, whether in their own homes or at another location. The failure to interview individuals selected as potential participants in a population-based study, either because of
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
193
inability to contact or refusals, raises concern about possible bias caused by nonresponse. Neither high rates of nonresponse nor different response rates between cases and controls in and of themselves introduce bias (20). Rather, it depends upon whether the proportions of exposed individuals among the case and control groups differ between those who participated and those who did not. It is usually impossible to compare risk factor characteristics of respondents and nonrespondents; however, in this study, we were able to compare characteristics of women who were fully cooperative to those who agreed only to a brief telephone interview. We found that women who were only partially interviewed were more likely to be older, less educated, and black (if cases), which is consistent with other studies suggesting that these factors are predictors of lower participation rates (7, 21, 22). There was not a consistent pattern between breast cancer risk factors and likelihood of participation. Among women who completed only the partial interview, some risk factors were reported more frequently than among women who were fully cooperative; whereas, others were reported less frequently. Thus, these data do not suggest that high-risk women are more likely to participate in a research study. This is consistent with other research that suggests that participants differ from nonparticipants more in terms of sociodemographic factors than in disease-specific risk factors (23). The analyses comparing full and partial respondents have several limitations. The number of partial interviews is relatively small and includes only women who could be contacted. No information beyond race, age, and area of residence was available for the unlocatable women, who comprised a sizable proportion of the potential controls. Furthermore, if these results are used to make inferences about how nonrespondents differ from participants, we must assume that the partial respondents are similar to women who completely refused to participate. Some researchers have argued that willingness to participate in research studies is a continuum, and the characteristics of “reluctant” respondents are likely to be similar to nonrespondents (24). In contrast, other researchers have argued that there are situations in which this assumption is not valid (25). The uncertainty regarding risk factor characteristics of individuals who were never interviewed underscores the importance of having as few nonrespondents as possible. Our experience in recruiting study participants points out that no single strategy will ensure high response rates, especially when the study population is diverse in terms of age, ethnicity, and socioeconomic characteristics. Multiple sources were necessary to identify telephone numbers, because listings were sometimes found in one source but not another. Each additional strategy beyond the primary method of identifying telephone numbers through computerized directories resulted in incremental increases in the number of women contacted and interviewed. For example,
194
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
approximately 10% of women who were mailed a certified letter after earlier attempts at contact were unsuccessful called the study office and agreed to participate. Likewise, leaving messages on answering machines after multiple telephone calls resulted in no personal contact also led some women to contact the study office. The much more laborintensive approach of making unannounced home visits also resulted in additional interviews, sometimes at the time of that visit and sometimes in response to contact materials that were left behind by the interviewer. Because different approaches were pursued simultaneously, it was not possible to come up with firm estimates of the increase in participation rates associated with each of the strategies employed. This report is intended, in part, to stimulate discussion of issues surrounding response rates and to encourage sharing of recruitment strategies between epidemiologists. Informal discussions with other researchers indicate that a number of creative strategies are currently being employed to bolster response rates. For example, gift certificates or entry into a lottery for a large cash prize have been used as alternatives to the more common cash incentive in some studies. Other researchers have employed individuals affected by the disease under study (e.g., breast cancer survivors) as recruiters. In addition, the Internet offers a number of services that may have utility in contacting individuals for research studies. These include “people finder” web sites available at no charge and tracking services that provide information on individuals for a fee (26). In an era when it seems to be ever more challenging to recruit study participants, it is important for researchers to share their successes as well as the pitfalls encountered with various methods of contacting individuals. Although no single strategy will ensure high response rates, especially when the study population is diverse in terms of age, ethnicity, and socioeconomic characteristics, it remains imperative that researchers strive to include as large a proportion of the targeted population as possible. Multiple strategies for achieving contact with and encouraging cooperation by potential participants are necessary, including possibly tailoring the approach used based on demographic characteristics. However, whatever methods are used to contact potential study participants, the scientific need to achieve high response rates must be weighed against the ethical responsibility to respect individual privacy.
AEP Vol. 9, No. 3 April 1999: 188–195
This work was supported by a Specialized Program of Research Excellence (SPORE) in Breast Cancer grant (P50-CA58223) and a FIRST award (R29-CA67285), both funded by the National Cancer Institute.
REFERENCES 1. Slattery ML, Edwards SL, Caan BJ, Kerber RA, Potter JD. Response rates among control subjects in case-control studies. Ann Epidemiol. 1995;5:245–249. 2. Swanson GM, Ward AJ. Recruiting minorities into clinical trials: Toward a participant-friendly system. J Natl Cancer Inst. 1995; 87:1747–1759. 3. Harris Y, Gorelick PB, Samuels P, Bempong I. Why African Americans may not be participating in clinical trials. J Natl Med Assn. 1996; 88:630–634. 4. Bostick RM, Potter JD, Kushi LH, Sellers TA, Steinmetz KA, McKenzie DR, Gapstur SM, Folsom AR. Sugar, meat, and fat intake, and nondietary risk factors for colon cancer incidence in Iowa women (United States). Cancer Causes Control. 1994;5:38–52. 5. Longnecker MP, Newcomb PA, Mittendorf R, Greenberg ER, Clapp RW, Bogdan GF, Baron J, MacMahon B, Willett WC. Risk of breast cancer in relation to lifetime alcohol consumption. J Natl Cancer Inst. 1995;87:923–929. 6. Tennstedt SL, Dettling U, McKinlay JB. Refusal rates in a longitudinal study of older people: Implications for field methods. J Gerontol. 1992;47:S313–318. 7. Koval JJ, Ecclestone NA, Paterson DH, Brown B, Cunningham DA, Rechnitzer PA. Response rates in a survey of physical capacity among older persons. J Gerontol. 1992;47:S140–148. 8. Newman B, Moorman PG, Millikan R, Qaqish BF, Geradts J, Aldrich TE, Liu ET. The Carolina Breast Cancer Study: Integrating populationbased epidemiology and molecular biology. Breast Cancer Res Treat. 1995;35:51–60. 9. Aldrich TE, Vann D, Moorman PG, Newman B. Rapid reporting of cancer incidence in a population-based study of breast cancer: One constructive use of a central cancer registry. Breast Cancer Res Treat. 1995;35:61–64. 10. Weinberg CR, Sandler DP. Randomized recruitment in case-control studies. Am J Epidemiol. 1991;134:421–432. 11. National Change of Address Technical Information, Lorton Data, Inc., Minneapolis, MN, 1994. 12. Anonymous. New driver privacy protection law may hamper epidemiologic research. Epidemiol Monitor. 1997;18:1–3. 13. DeMaio TJ. Refusals: Who, where, and why. Pub Op Quart. 1980; 44:223–233. 14. Berk ML, Wilensky GR, Cohen SB. Methodological issues in health surveys. An evaluation of procedures used in the National Medical Care Expenditure Survey. Eval Rev. 1984;8:307–326. 15. Savitz DA, Hamman RF, Grace C, Stroo K. Respondents’ attitudes regarding participation in an epidemiologic study. Am J Epidemiol. 1986;123:362–366. 16. Brandt AM. Racism and research: The case of the Tuskegee syphilis study. Hastings Center Report. December 1978;21–29.
Note Added in Proof
17. Johnson TP, Parsons JA. Interviewer effects on self-reported substance use among homeless persons. Addict Behav. 1994;19:83–93.
Three additional cases were found to be ineligible after submission of this manuscript: 2 black women aged 57 and 59 and 1 white woman aged 44. None of the results or conclusions were changed substantially by the reclassification of these cases.
18. Schuman H, Converse J. The effects of black and white interviewers on black responses in 1968. Pub Op Quart. 1971;35:44–66. 19. Stokes L, Yeh M-Y. Searching for causes of interviewer effects in telephone surveys. In: Groves RM, Biememr PP, Lyberg LE, Massey JT, Nicholls WL II, Waksberg J, eds. Telephone Survey Methodology. New York: Wiley; 1988:357–383.
AEP Vol. 9, No. 3 April 1999: 188–195
20. Schlesselman JJ, Stolley PD. Sources of bias. In: Schlesselman JJ, ed. Case-Control Studies: Design, Conduct, Analysis. New York: Oxford University Press; 1982:131–132. 21. Grotzinger KM, Stuart BC, Ahern F. Assessment and control of nonresponse bias in a survey of medicine use by the elderly. Med Care. 1994;32:989–1003. 22. Schwartz SM, Siscovick DS, Weinmann S, Raghunathan TE, Psaty BM, Koepsell TD, Albright J, Beverly RK, Longstreth WT. Jr. Brief telephone interviews to assess characteristics of nonrespondents in population-based case-control studies. Am J Epidemiol. 1995;141:S20. 23. Freudenheim JL, Marshall JR, Vena JE, Laughlin R, Brasure JR, Swan-
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
195
son MK, Nemoto T, Graham S. Premenopausal breast cancer risk and intake of vegetables, fruits, and related nutrients. J Natl Cancer Inst. 1996;88:340–348. 24. Hebert R, Bravo G, Korner-Bitensky N, Voyer L. Refusal and information bias associated with postal questionnaires and face-to-face interviews in very elderly subjects. J Clin Epidemiol. 1996;49:373–381. 25. Brenner H. Alternative approaches for estimating prevalence in epidemiologic surveys with two waves of respondents. Am J Epidemiol. 1995;142:1235–1245. 26. Bernstein N. On line, high-tech sleuths find private facts. New York Times, Sept. 15, 1997.
PURPOSE: Despite concerns about declining participation rates in epidemiologic studies in recent years, relatively few papers have discussed obstacles to recruiting study participants or strategies for optimizing response rates. This report describes factors associated with nonparticipation in a populationbased, case-control study of breast cancer and discusses ways to overcome barriers to participation. METHODS: Contact and cooperation rates were calculated for participants in the Carolina Breast Cancer Study (CBCS), stratified by case status, age, race, and race of interviewer. Demographic and breast cancer risk factor characteristics of partial and full responders also were compared. RESULTS: Contact rates and cooperation rates varied by case/control status and demographic characteristics. Contact rates were lower among controls, younger women, and black women. Cooperation rates were lower among controls, older women, and black cases. Cooperation rates were higher among both black and nonblack women when participants and interviewers were concordant on race. CONCLUSIONS: Obstacles to recruitment seem to differ among race and age subgroups, suggesting that recruitment strategies may need to be tailored to potential participants based upon demographic characteristics. Strategies have been implemented to improve response rates in this and other epidemiologic studies; however, additional research and innovation in this area are needed. Ann Epidemiol 1999;9:188–195. 1999 Elsevier Science Inc. All rights reserved. KEY WORDS:
Response Rates, Epidemiologic Methods, Breast Cancer, Blacks.
INTRODUCTION High response rates are important in epidemiologic research for both validity and generalizability to the target population. However, the perception among epidemiologists is that response rates are lower currently than 10 to 15 years ago (1). In addition, there is concern about participation rates among certain subsets of the population, among these, minorities and elderly persons (2, 3). Minorities are underrepresented in research studies for a variety of reasons, including study area demographics (4, 5), mistrust of medical research because of past abuses, differences in sociocultural beliefs, and socioeconomic characteristics (2, 3). In the case of elderly persons, reluctance to participate in research may be caused by safety concerns and fears that they might become targets of financial swindles. In addition, poor
health status may be a real or perceived barrier to elderly citizens’ participation in research studies (6, 7). Despite widespread concern about the impact of decreased participation in epidemiologic research, relatively few papers have discussed strategies for optimizing response rates. This report describes response rates in a populationbased, case-control study of breast cancer, with a particular emphasis on differences by age and race of the respondent, as well as the impact of the interviewer’s race. Comparisons also are presented between women who completed a full, in-person interview and those who did only an abbreviated telephone survey. The discussion focuses on obstacles to achieving contact with and interviewing potential study participants and methods to improve response rates.
METHODS From the Department of Epidemiology and Public Health (P.G.M.), Yale University School of Medicine, New Haven, CT 06520; Department of Epidemiology (B.N., R.C.M., C-K.J.T.), University of North CarolinaChapel Hill, Chapel Hill, NC 27599; and Epidemiology Branch (D.P.S.), National Institute of Environmental Health Sciences, Research Triangle Park, NC 27709. Address reprint requests to: Dr. Patricia G. Moorman, Department of Epidemiology and Public Health, Yale University School of Medicine, 60 College Street, P.O. Box 208034, New Haven, CT 06520-8034. Received October 27, 1998; accepted October 27, 1998. 1999 Elsevier Science Inc. All rights reserved. 655 Avenue of the Americas, New York, NY 10010
The Carolina Breast Cancer Study (CBCS) is an on-going, population-based, case-control study that began in 1993 (8). Phase I of the study, during which 889 cases and 841 controls were interviewed, was completed in 1996. Cases were identified through a rapid case ascertainment system developed in conjunction with the North Carolina Central Cancer Registry (CCR) (9). Pathology reports for all breast cancers diagnosed in the hospitals in the study area were forwarded 1047-2797/99/$–see front matter PII S1047-2797(98)00057-X
AEP Vol. 9, No. 3 April 1999: 188–195
Selected Abbreviations and Acronyms CBCS 5 Carolina Breast Cancer Study CCR 5 Central Cancer Registry DMV 5 Division of Motor Vehicles HCFA 5 Health Care Financing Administration
to the CCR and then to the CBCS, most within 1 month of diagnosis. Eligible cases were between the ages of 20 and 74 years, resided in a 24-county area of central and eastern North Carolina, and had no prior history of breast cancer. Black women and women under age 50 were oversampled, using a modification of randomized recruitment (10), with the goal of obtaining roughly equal numbers of black and nonblack women and women diagnosed before and after age 50. Permission to contact breast cancer cases was requested from the physician of record on the pathology report. Controls between the ages of 20 and 64 years were randomly selected from North Carolina Division of Motor Vehicle (DMV) lists of women residing in the study area who had either a driver’s license or an identification card. Controls between the ages of 65 and 74 years were selected from U.S. Health Care Financing Administration (HCFA) lists. Sampling fractions based on race (dichotomized as black or nonblack based on race information from the DMV or HCFA) and 5-year age groups were used to select controls who were approximately frequency-matched to cases using randomized recruitment. Cases for whom physician consent was obtained and controls were mailed a letter describing the study and inviting them to participate. The letters were followed by telephone calls from a registered nurse, who scheduled an interview, usually in the home, for women who agreed to participate. During the interview visit, the nurse administered a 1-hour questionnaire, took body measurements (height, weight, and waist and hip circumferences), and obtained a 30-mL blood sample. Participants were paid $25 for completing the interview. Women who declined the home visit were asked to complete an abbreviated (approximately 5-minute) telephone interview. The study protocol was approved by the institutional review boards of the University of North Carolina School of Medicine and the participating hospitals. Neither the DMV nor the HCFA list provided telephone numbers for potential controls; therefore, multiple strategies were employed to identify telephone numbers, including computerized directories on CD-ROM, conventional telephone directories, directory assistance, and reverse directories. Repeated mailings, including one sent by certified mail, each included a postage-paid card for the woman to return if we did not have her telephone number or if the number we identified was incorrect. A toll-free telephone number was also included for women to use when contacting the
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
189
study office. Other strategies to contact or resolve the status of unlocatable women included checking the National Change of Address Service (11), and checking state death records. In addition, lists of unlocatable women were checked periodically against current DMV records in an attempt to identify new addresses or moves out of state. As a last resort, the nurse-interviewers made visits to the residences of those who could not be contacted by telephone. Measures suggested by Slattery et al. (1) were used to describe various aspects of response. “Contact rate” is defined as the percentage of women who were identified as potential study participants with whom contact was achieved. “Cooperation rate” is the percentage of women interviewed of those who were contacted and eligible. “Over-all response rate” is the percentage of women interviewed of those who were selected and eligible for the study. Over-all response rate includes individuals who were not contacted in the denominator, although these women are excluded from the denominator for cooperation rate. Women of all races were included in the study; however, white women and black women comprised the majority of the population in the North Carolina study area. Asian Americans, Native Americans, and women of other races represented only 1.6% of the study population. For the racespecific analyses presented in this paper, the “black” category includes all women described as black or African American, and the “nonblack” category includes all other women. When the CBCS was initiated, interviewers and participants were not matched on race. However, after interviewers reported more refusals from women not of their race, particularly older women, matching on race among participants age 50 and older was initiated. The impact of interviewer’s race was assessed by comparing cooperation rates when the interviewer and potential participant were concordant or discordant on race. The differences in the proportion of women who consented to be interviewed and 95% confidence intervals were calculated. Demographic characteristics and breast cancer risk factors were compared for women who consented to the complete, in-person interview and those who agreed only to a partial, telephone interview. Differences in mean values or proportions between women completing the full questionnaire versus those doing the partial survey, stratified by case status, were calculated along with the 95% confidence intervals for the differences.
RESULTS Table 1 is a detailed breakdown of the disposition of potential study participants. Of the 1285 identified breast cancer patients, 889 were interviewed. Of 1797 potential control participants, 841 were interviewed. Those not interviewed included ineligible women, unlocatable women, physician
190
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
AEP Vol. 9, No. 3 April 1999: 188–195
TABLE 1. Responses of women selected as potential participants for the CBCS by case-control status, race, and age Cases Whites Total Sampled (N) Ineligible (%)a Deceased (%) Uncontactable (%) Physician refusal (%) Participant refusal (%) Interviewed Full (%) Partial (%) Contact rateb Cooperation ratec Overall response rated
1285 6.1 0.9 3.3 5.7 14.9 67.2 1.9 96.7% 77.1% 74.4%
20–39 yrs 119 3.4 0 2.5 5.9 5.9 81.5 0.8 97.5% 87.5% 85.2%
40–49 yrs
Blacks
50–64 yrs
318 4.1 0.3 3.1 6.6 11.0
164 7.9 0.6 3.0 9.8 11.6
73.6 1.3 96.9% 81.0% 78.2%
65.2 1.8 97.0% 75.9% 73.3%
. 64 yrs
20–39 yrs
138 7.2 0 0 8.0 18.8
86 2.3 2.3 4.7 2.3 14.0
64.5 1.4 100.0% 71.1% 71.1%
70.9 3.5 95.3% 82.1% 78.0%
40–49 yrs 168 8.3 0 4.8 4.2 12.5
50–64 yrs 162 4.3 2.5 5.6 2.5 23.5
67.9 2.4 95.2% 80.8% 76.6%
. 64 yrs 130 11.5 3.1 2.3 3.8 25.4
59.3 2.5 94.4% 70.4% 66.2%
50.8 3.1 97.7% 64.8% 63.1%
50–64 yrs
. 64 yrs
Controls Whites Total Sampled (N) Ineligible (%)a Deceased (%) Uncontactable (%) Participant refusal (%) Interviewed Full (%) Partial (%) Contact rateb Cooperation ratec Overall response rated
1797 9.8 1.6 19.4 22.5 44.0 2.8 80.6% 67.6% 52.8%
20–39 yrs 119 17.6 0 20.2 15.1 46.2 0.8 79.8% 75.7% 57.1%
40–49 yrs 350 9.7 0.3 14.3 20.9 51.7 3.1 85.7% 72.5% 61.0%
Blacks
50–64 yrs 265 11.3 1.5 10.9 25.7 46.4 4.2 89.1% 66.3% 58.0%
. 64 yrs 191 9.9 2.1 5.2 27.2 51.8 3.7 94.8% 67.1% 63.1%
20–39 yrs 140 10.0 0 39.3 17.9 32.1 0.7 60.7% 64.8% 36.5%
40–49 yrs 283 7.1 1.1 27.6 16.6 44.5 3.2 72.4% 74.2% 51.9%
253 3.6 2.4 23.7 26.5 40.7 3.2 76.3% 62.4% 46.6%
196 14.8 5.1 21.4 27.6 29.6 1.5 78.6% 53.0% 38.9%
a
Eligibility criteria include age (20–74 years), female gender, residence in 24-county study area, able to complete an interview in English, and no prior history of breast cancer. b Contact rate 5 # of women contacted/# of women identified as potential cases or controls. c Cooperation rate 5 # of completed interviews/# of women contacted and eligible. d Over-all response rate 5 # of completed interviews/# of women selected for study—ineligible and deceased women.
refusals (for cases), and participant refusals. Within these categories, there was variation by case-control status, age, and race. Among cases, 6% were found to be ineligible, and among controls, 10% were ineligible. Ineligibility was determined by age at diagnosis (for cases) or selection (for controls) older than 74 years, permanent residence outside study area, prior history of breast cancer, or inability to complete an interview in English (i.e., did not speak English or was cognitively impaired by Alzheimer’s disease, mental retardation, etc.). Ineligibility among cases was most often because of a prior history of breast cancer; whereas, current residence outside the study area was the most common reason for ineligibility among controls. Physician consent was not obtained for approximately 6% of eligible cases, ranging from 2.3% of the youngest black women to 9.8% of nonblack women aged 50–64 years. Approximately half the refusals were from four physicians (out of more than 150 physicians who treat breast cancer
in the study area) who declined consent for all of their patients. The others were for reasons specific to a given patient (e.g., patient was considered too ill or emotionally distraught). The higher percentage of physician refusals for nonblack women, particularly at older ages, largely reflected the demographics of the practices of those doctors who chose not to allow us to invite their patients to participate in the study. The proportion of women who were unlocatable or uncontactable differed among case-control, race, and age strata. Only a small proportion of cases (3%) were in this category, consistent with the availability of current contact information from our rapid case ascertainment system. However, nearly one-fifth of all potential controls could not be located or contacted. More black women than nonblack women, and within each race, more younger than older women could not be contacted. Participant refusal rates were lower among cases. Women age 50 and older were more likely to refuse to participate,
AEP Vol. 9, No. 3 April 1999: 188–195
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
191
TABLE 2. Cooperation rates for cases and controls by age, race of participant, and race of interviewer Interviewer race Concordant
Nonblack cases, age , 50 Nonblack cases, age > 50 Black cases, age , 50 Black cases, age > 50 Nonblack controls, age , 50 Nonblack controls, age > 50 Black controls, age , 50 Black controls, age > 50 a
Discordant
N
Cooperation rate (%)a
N
Cooperation rate (%)a
Difference in cooperation rate (95% CI)
199 203 142 169 154 302 154 218
92.5 83.7 89.4 77.5 75.3 69.5 77.9 63.3
168 34 66 62 181 51 97 69
90.5 91.2 83.3 62.9 72.9 58.8 62.9 49.3
2.0 (23.7–7.7) 27.5 (218.3–3.2) 6.1 (24.2–16.4) 14.6 (1.0–28.2) 2.4 (27.0–11.8) 10.7 (23.8–25.2) 15.0 (3.4–26.6) 14.0 (0.6–27.4)
Denominators exclude cases for whom physician consent was not obtained, women who were never located, or women who called the study office declining to participate after they received a letter of invitation but before being telephoned by the nurse.
whether they were cases or controls. Race differences were observed among cases, with black women less likely to participate. Among controls, however, within each age category, refusal rates were similar for black and nonblack women. The lower over-all response rates for blacks reflected the lower contact rates for these women. Table 2 presents cooperation rates by participant race and interviewer race among older and younger women. Because these analyses are intended to assess the influence of interviewer race on the likelihood of a woman agreeing to be interviewed, the reported cooperation rates are for women who were actually contacted by a nurse. The denominators exclude cases for whom physician consent was not obtained and women who called the study office declining to participate after they received a letter of invitation but before being telephoned by a nurse. Consequently, the cooperation rates in this table are higher than the cooperation rates for these age/race categories calculated according to the criteria used in Table 1, which were 83, 74, 81, and 68% for younger and older nonblack cases and younger and older and black cases, respectively. The corresponding rates for these age and race strata among controls were 73, 67, 72, and 59%. In seven of the eight age, race, and case/control categories, cooperation rates were higher when the interviewer and participant were concordant on race. However, the magnitude of this effect varied by age and race of the participant. Over-all cooperation rates were highest among younger nonblack women for whom race of the interviewer had little impact. Among older nonblack controls, the cooperation rate was about 10% higher when they were invited to participate by white interviewers, although this difference was not statistically significant. Among older nonblack cases, cooperation rates were actually higher among those assigned to a black interviewer; however, this is based on small numbers. Cooperation rates were higher among black women when they were invited to participate by a black interviewer; this was true for cases and controls as well as
for younger and older women. Having the interviewer and participant concordant on race improved cooperation rates by 6% (young black cases) to 15% (young black controls). As shown in Table 3, comparisons were made between women who completed the full, in-person questionnaire with those who did only a partial, telephone interview, stratified by case-control status. Among sociodemographic characteristics, women who completed the partial interview were slightly older, had less education, and were less likely to have been married or to have held a job for at least 6 months. Comparisons of potential breast cancer risk factors were less consistent in characterizing who was likely to complete the full interview. Those who were partially interviewed were less likely to report having used oral contraceptives or estrogen replacement therapy, but were more likely to report a family history of breast or ovarian cancer. They were similar to women who completed the full questionnaire for a number of breast cancer risk factors, including age at menarche, age at menopause, pregnancy history, and body mass index. For most characteristics, differences between those fully participating and those agreeing to the partial interview were in the same direction for cases and controls. However, race, menopausal status, and history of benign biopsy were exceptions to this observation. It should be noted that the number of partial interviews was relatively small, and the confidence intervals around the differences in means or proportions for all risk factors, except use of estrogens, were consistent with no difference between women completing the full versus the partial interview.
DISCUSSION In this study, over-all response rates varied by case-control status, age, and race of participant. Examination of the components of the over-all response rate—the contact rate and the cooperation rate—revealed different obstacles to achieving
192
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
AEP Vol. 9, No. 3 April 1999: 188–195
TABLE 3. Characteristics of women completing full, in-home interview compared to those completing a partial, telephone interview, by case-control status Cases Characteristic Race (% black) Education (% , 12 years) Marital status (% never married) Ever had a job (%) Ever use of oral contraceptives (%) Ever use of estrogens (%) Premenopausal (%) History of benign biopsy (%) Breast cancer in 1st degree relative (%) Ovarian cancer in 1st degree relative (%) Age (mean) Age at menarche (mean) Age at menopause (mean) Number of pregnancies (mean) Age at 1st full-term pregnancy (mean) Body mass index (mean)
Controls
Full N 5 864
Partial N 5 25
Difference (95% CI)
Full N 5 790
Partial N 5 51
Difference (95% CI)
39.0 18.4 9.5 98.4 64.1 22.2 42.4 16.8 15.0 2.5 50.3 12.6 43.8 2.9 22.5 28.1
60.0 36.0 20.0 88.0 54.2 8.0 40.0 20.0 28.0 4.0 51.6 12.8 43.9 2.2 22.3 28.5
221.0 (240.0–1.5) 217.6 (236.6–1.3) 210.5 (226.3–5.3) 10.4 (22.4–23.1) 9.9 (210.3–30.1) 14.2 (3.2–25.2) 2.4 (217.1–21.8) 23.2 (219.0–12.7) 213.0 (230.7–4.8) 21.5 (29.2–6.2) 21.3 (26.1–3.4) 20.2 (20.9–0.4) 20.1 (24.4–4.2) 0.7 (20.2–1.6) 0.2 (22.1–2.5) 20.4 (23.2–2.5)
42.0 20.9 7.5 98.2 59.6 29.6 36.6 17.6 11.7 1.2 51.8 12.7 43.4 3.2 22.3 28.9
41.2 29.4 16.0 94.1 47.1 5.9 45.1 11.8 15.7 3.9 54.7 12.9 43.9 2.8 21.3 28.3
0.8 (213.0–14.8) 28.5 (221.4–4.3) 28.5 (218.9–1.8) 4.1 (22.4–10.6) 12.5 (21.6–26.6) 23.7 (16.5–30.9) 28.5 (222.5–5.6) 5.8 (23.3–15.1) 24.0 (214.2–6.3) 22.7 (28.1–2.6) 22.9 (26.2–0.4) 20.2 (20.7–0.2) 20.5 (23.5–2.5) 0.4 (20.3–1.1) 1.0 (20.6–2.5) 0.6 (21.4–2.6)
participation in specific subgroups. Lower contact rates were observed for controls, younger women, and black women. Cooperation rates were lower among black cases than nonblack cases, lower among controls of both races than cases, and lower among older women of both races than younger women. Achieving contact with the individuals selected as potential study participants, especially controls, is a major challenge of population-based research. Despite the multiple strategies employed to identify current addresses and telephone numbers, 19% of women identified as potential controls from the DMV and HCFA lists were never contacted. Contact efforts were less successful for younger women and for black women, presumably because of greater mobility and a smaller likelihood of having a telephone number listed in their name. A smaller proportion of women identified through the HCFA list could not be contacted. The more accurate address information from HCFA reflects both the lower mobility of older women and the underlying purposes of this roster (i.e., health insurance and social security). The difficulty in contacting women is attributable, in part, to the limitations of the sources used for selecting controls. Although DMV and HCFA lists are commonly used in epidemiologic research, have fairly complete coverage of the population, and provide address, race, and birthdate information, the contact information is not optimal. Telephone numbers generally are unavailable from either list, and address information is frequently not current. Outdated address information from the DMV list was a substantial limitation to achieving contact with potential participants and is likely to become an even greater problem now that some states are extending the length of time between license renewals. Other difficulties in using DMV lists as
sources of controls include increasingly restricted availability of these data as a result of a federal law limiting public access to personal information from drivers’ license records and a privacy option available in at least one state that allows drivers to indicate that their personal information cannot be released for business or survey purposes (12). Refusals represented the other major category of nonresponse. A number of factors have been identified that influence the likelihood of participation in research studies (13–15). Positive factors include case status, physician endorsement of the study, perceived legitimacy of the study, perceived importance of the study to health, and monetary incentives. Often cited negative factors include concerns about the time required to participate, privacy issues, and opposition to surveys in general. We attempted to account for these concerns when designing the contact materials and protocols. For example, the letter of invitation emphasized that physician consent had been obtained (for cases), and an enclosed question-and-answer sheet addressed concerns often raised by participants, including how they were selected, how their privacy would be protected, and how participating in the study would benefit them. In addition to factors applicable to the general population, reasons for non-participation specific to subgroups of the population have also been identified. Higher refusal rates among blacks have been attributed to the legacy of the Tuskegee syphilis study (2, 16) and to differences in attitudes and beliefs about disease (2, 3). Interviewer characteristics may come into play as well, as has been demonstrated in studies that show communication is least inhibited between respondents and interviewers of the same race and gender (17). Studies examining interviewer effects generally
AEP Vol. 9, No. 3 April 1999: 188–195
have been done in the context of in-person interviews and have focused on qualitative differences in responses based upon the interviewers’ and respondents’ race and/or gender (17, 18). It has been assumed that interviewer effects are of less importance in telephone surveys, because it is more difficult to differentiate characteristics of the interviewer (19). However, our study found that having interviewers and potential participants concordant on race had a substantial effect on the likelihood of participation, although the interviewers’ first contact with the women by telephone. It seems that some women’s decisions to participate were based on the perceived race of the interviewer, a judgment presumably made on the basis of accent or language patterns. It is important to note that the decision to match interviewer and older participants on race was made approximately 1 year into the study. It is possible that the improved cooperation rates when matching on race could have been attributable in part to the interviewers being more experienced later in the study. However, the magnitude of the effect in certain subgroups (e.g., the nearly 15% difference in cooperation rates among black controls and older black cases) as compared to the much smaller effects observed in others argues against this. Although matching on race seemed to have a substantial effect on cooperation rates, two observations can be made. First, the interviewers’ perception that race was a more important issue for older women was not entirely accurate, because the race of the interviewer made the largest difference in cooperation rates among young black controls. Second, even when matching on race, cooperation rates among black women tended to be lower than for nonblack women. The use of black interviewers increased the likelihood of participation, possibly by overcoming mistrust of medical researchers or conveying the message that the research was relevant to women of their race; however, the lower cooperation rates suggest that other barriers to participation remained. Older women were the other major subgroup for whom refusal rates tended to be high. Among these women, the biggest obstacle to achieving high participation rates seemed not to be in contacting them, but in overcoming their fears about taking part in a research study, especially one involving in-home interviews. Interviewers were sometimes able to alleviate these fears by offering to meet at another location (e.g., local hospital or health department), talking with a family member to confirm the legitimacy of the study, letting the participant know that it was acceptable to have another person present for the interview, or reassuring them, when appropriate, that existing medical conditions were not barriers to participation. However, in many cases, it was impossible to overcome the apprehension that older women had about speaking with a stranger, whether in their own homes or at another location. The failure to interview individuals selected as potential participants in a population-based study, either because of
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
193
inability to contact or refusals, raises concern about possible bias caused by nonresponse. Neither high rates of nonresponse nor different response rates between cases and controls in and of themselves introduce bias (20). Rather, it depends upon whether the proportions of exposed individuals among the case and control groups differ between those who participated and those who did not. It is usually impossible to compare risk factor characteristics of respondents and nonrespondents; however, in this study, we were able to compare characteristics of women who were fully cooperative to those who agreed only to a brief telephone interview. We found that women who were only partially interviewed were more likely to be older, less educated, and black (if cases), which is consistent with other studies suggesting that these factors are predictors of lower participation rates (7, 21, 22). There was not a consistent pattern between breast cancer risk factors and likelihood of participation. Among women who completed only the partial interview, some risk factors were reported more frequently than among women who were fully cooperative; whereas, others were reported less frequently. Thus, these data do not suggest that high-risk women are more likely to participate in a research study. This is consistent with other research that suggests that participants differ from nonparticipants more in terms of sociodemographic factors than in disease-specific risk factors (23). The analyses comparing full and partial respondents have several limitations. The number of partial interviews is relatively small and includes only women who could be contacted. No information beyond race, age, and area of residence was available for the unlocatable women, who comprised a sizable proportion of the potential controls. Furthermore, if these results are used to make inferences about how nonrespondents differ from participants, we must assume that the partial respondents are similar to women who completely refused to participate. Some researchers have argued that willingness to participate in research studies is a continuum, and the characteristics of “reluctant” respondents are likely to be similar to nonrespondents (24). In contrast, other researchers have argued that there are situations in which this assumption is not valid (25). The uncertainty regarding risk factor characteristics of individuals who were never interviewed underscores the importance of having as few nonrespondents as possible. Our experience in recruiting study participants points out that no single strategy will ensure high response rates, especially when the study population is diverse in terms of age, ethnicity, and socioeconomic characteristics. Multiple sources were necessary to identify telephone numbers, because listings were sometimes found in one source but not another. Each additional strategy beyond the primary method of identifying telephone numbers through computerized directories resulted in incremental increases in the number of women contacted and interviewed. For example,
194
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
approximately 10% of women who were mailed a certified letter after earlier attempts at contact were unsuccessful called the study office and agreed to participate. Likewise, leaving messages on answering machines after multiple telephone calls resulted in no personal contact also led some women to contact the study office. The much more laborintensive approach of making unannounced home visits also resulted in additional interviews, sometimes at the time of that visit and sometimes in response to contact materials that were left behind by the interviewer. Because different approaches were pursued simultaneously, it was not possible to come up with firm estimates of the increase in participation rates associated with each of the strategies employed. This report is intended, in part, to stimulate discussion of issues surrounding response rates and to encourage sharing of recruitment strategies between epidemiologists. Informal discussions with other researchers indicate that a number of creative strategies are currently being employed to bolster response rates. For example, gift certificates or entry into a lottery for a large cash prize have been used as alternatives to the more common cash incentive in some studies. Other researchers have employed individuals affected by the disease under study (e.g., breast cancer survivors) as recruiters. In addition, the Internet offers a number of services that may have utility in contacting individuals for research studies. These include “people finder” web sites available at no charge and tracking services that provide information on individuals for a fee (26). In an era when it seems to be ever more challenging to recruit study participants, it is important for researchers to share their successes as well as the pitfalls encountered with various methods of contacting individuals. Although no single strategy will ensure high response rates, especially when the study population is diverse in terms of age, ethnicity, and socioeconomic characteristics, it remains imperative that researchers strive to include as large a proportion of the targeted population as possible. Multiple strategies for achieving contact with and encouraging cooperation by potential participants are necessary, including possibly tailoring the approach used based on demographic characteristics. However, whatever methods are used to contact potential study participants, the scientific need to achieve high response rates must be weighed against the ethical responsibility to respect individual privacy.
AEP Vol. 9, No. 3 April 1999: 188–195
This work was supported by a Specialized Program of Research Excellence (SPORE) in Breast Cancer grant (P50-CA58223) and a FIRST award (R29-CA67285), both funded by the National Cancer Institute.
REFERENCES 1. Slattery ML, Edwards SL, Caan BJ, Kerber RA, Potter JD. Response rates among control subjects in case-control studies. Ann Epidemiol. 1995;5:245–249. 2. Swanson GM, Ward AJ. Recruiting minorities into clinical trials: Toward a participant-friendly system. J Natl Cancer Inst. 1995; 87:1747–1759. 3. Harris Y, Gorelick PB, Samuels P, Bempong I. Why African Americans may not be participating in clinical trials. J Natl Med Assn. 1996; 88:630–634. 4. Bostick RM, Potter JD, Kushi LH, Sellers TA, Steinmetz KA, McKenzie DR, Gapstur SM, Folsom AR. Sugar, meat, and fat intake, and nondietary risk factors for colon cancer incidence in Iowa women (United States). Cancer Causes Control. 1994;5:38–52. 5. Longnecker MP, Newcomb PA, Mittendorf R, Greenberg ER, Clapp RW, Bogdan GF, Baron J, MacMahon B, Willett WC. Risk of breast cancer in relation to lifetime alcohol consumption. J Natl Cancer Inst. 1995;87:923–929. 6. Tennstedt SL, Dettling U, McKinlay JB. Refusal rates in a longitudinal study of older people: Implications for field methods. J Gerontol. 1992;47:S313–318. 7. Koval JJ, Ecclestone NA, Paterson DH, Brown B, Cunningham DA, Rechnitzer PA. Response rates in a survey of physical capacity among older persons. J Gerontol. 1992;47:S140–148. 8. Newman B, Moorman PG, Millikan R, Qaqish BF, Geradts J, Aldrich TE, Liu ET. The Carolina Breast Cancer Study: Integrating populationbased epidemiology and molecular biology. Breast Cancer Res Treat. 1995;35:51–60. 9. Aldrich TE, Vann D, Moorman PG, Newman B. Rapid reporting of cancer incidence in a population-based study of breast cancer: One constructive use of a central cancer registry. Breast Cancer Res Treat. 1995;35:61–64. 10. Weinberg CR, Sandler DP. Randomized recruitment in case-control studies. Am J Epidemiol. 1991;134:421–432. 11. National Change of Address Technical Information, Lorton Data, Inc., Minneapolis, MN, 1994. 12. Anonymous. New driver privacy protection law may hamper epidemiologic research. Epidemiol Monitor. 1997;18:1–3. 13. DeMaio TJ. Refusals: Who, where, and why. Pub Op Quart. 1980; 44:223–233. 14. Berk ML, Wilensky GR, Cohen SB. Methodological issues in health surveys. An evaluation of procedures used in the National Medical Care Expenditure Survey. Eval Rev. 1984;8:307–326. 15. Savitz DA, Hamman RF, Grace C, Stroo K. Respondents’ attitudes regarding participation in an epidemiologic study. Am J Epidemiol. 1986;123:362–366. 16. Brandt AM. Racism and research: The case of the Tuskegee syphilis study. Hastings Center Report. December 1978;21–29.
Note Added in Proof
17. Johnson TP, Parsons JA. Interviewer effects on self-reported substance use among homeless persons. Addict Behav. 1994;19:83–93.
Three additional cases were found to be ineligible after submission of this manuscript: 2 black women aged 57 and 59 and 1 white woman aged 44. None of the results or conclusions were changed substantially by the reclassification of these cases.
18. Schuman H, Converse J. The effects of black and white interviewers on black responses in 1968. Pub Op Quart. 1971;35:44–66. 19. Stokes L, Yeh M-Y. Searching for causes of interviewer effects in telephone surveys. In: Groves RM, Biememr PP, Lyberg LE, Massey JT, Nicholls WL II, Waksberg J, eds. Telephone Survey Methodology. New York: Wiley; 1988:357–383.
AEP Vol. 9, No. 3 April 1999: 188–195
20. Schlesselman JJ, Stolley PD. Sources of bias. In: Schlesselman JJ, ed. Case-Control Studies: Design, Conduct, Analysis. New York: Oxford University Press; 1982:131–132. 21. Grotzinger KM, Stuart BC, Ahern F. Assessment and control of nonresponse bias in a survey of medicine use by the elderly. Med Care. 1994;32:989–1003. 22. Schwartz SM, Siscovick DS, Weinmann S, Raghunathan TE, Psaty BM, Koepsell TD, Albright J, Beverly RK, Longstreth WT. Jr. Brief telephone interviews to assess characteristics of nonrespondents in population-based case-control studies. Am J Epidemiol. 1995;141:S20. 23. Freudenheim JL, Marshall JR, Vena JE, Laughlin R, Brasure JR, Swan-
Moorman et al. RESPONSE RATES IN CASE-CONTROL STUDIES
195
son MK, Nemoto T, Graham S. Premenopausal breast cancer risk and intake of vegetables, fruits, and related nutrients. J Natl Cancer Inst. 1996;88:340–348. 24. Hebert R, Bravo G, Korner-Bitensky N, Voyer L. Refusal and information bias associated with postal questionnaires and face-to-face interviews in very elderly subjects. J Clin Epidemiol. 1996;49:373–381. 25. Brenner H. Alternative approaches for estimating prevalence in epidemiologic surveys with two waves of respondents. Am J Epidemiol. 1995;142:1235–1245. 26. Bernstein N. On line, high-tech sleuths find private facts. New York Times, Sept. 15, 1997.