- Email: [email protected]
A mail survey of United States hematologists and oncologists:
Journal of Clinical Epidemiology 54 (2001) 430–432
A mail survey of United States hematologists and oncologists: A comparison of business reply versus stamped return envelopes Michael B. Streiff a, Lauren Dundesb,*, Jerry L. Spivaka a Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD Department of Sociology, Western Maryland College, 2 College Hill, Westminster, MD 21157 Received 1 December 1999; received in revised form 12 May 2000; accepted 12 May 2000
b
Abstract Mailed surveys are a popular means of obtaining data on large populations. In July 1999 a mail survey was conducted among 3000 randomly selected members of the American Society of Hematology to assess their approach to diagnosis and treatment of polycythemia vera. Because the researchers and the study population are members of the same professional organization with a vested interest in the results, we anticipated that the advantages of return stamped postage seen in previous studies would be less significant. The response rate for stamped return envelopes was 38% versus 32% for business reply envelopes. This statistically significant difference (P ⫽ .0005) of six percentage points is comparable to previous research. Excluding labor, the total cost per returned survey was $2.62 for business reply envelopes versus $1.82 for stamped return envelopes. We conclude that stamped return envelopes are a more effective and cost-efficient means of procuring data from physician specialists. © 2001 Elsevier Science Inc. All rights reserved. Keywords: Physician surveys; Mail surveys; Response rates; Costs; Business reply postage; Stamped return postage
1. Introduction Mail surveys are a popular means of obtaining research data on a large population of subjects. As for any type of research, response rate is critical for an accurate assessment of the survey population’s opinions. Previous researchers have identified a number of different strategies to maximize response rates [1]. These include personalized correspondence, multiple mailings, mailed and telephone reminders, as well as monetary and other forms of inducement for survey completion. Several previous studies have applied these strategies to mail surveys targeted at physicians [2–7]. However, it is unclear whether such strategies are necessary when the investigators are surveying colleagues in a subspecialty professional society about a topic of unique interest to the survey population. The purpose of this study was to determine the impact of return postage on the response rate and the economics of a mail survey of a large group of medical subspecialists. 2. Methods In July 1999, two hematologists at a large academic health center mailed an anonymous one-page, 12-item sur-
vey and cover letter to their hematologist and oncologist colleagues who were members of the American Society of Hematology (ASH).1 The survey, which took approximately 5 minutes to complete, asked clinicians about their approach to the diagnosis and treatment of polycythemia vera, an acquired bone marrow disorder characterized by elevated blood counts and a predisposition to thrombosis, bleeding, and rarely, leukemia. Although a large number of clinical studies have been conducted to establish diagnostic and therapeutic strategies for this disorder, no large-scale survey has been conducted to assess the impact of these trials on clinical practice. A random sample of 3000 of the total United States ASH membership of 7100 were sent surveys. To determine the impact of different forms of return postage on response rate and the costs of the survey, half of the respondents were sent either a business reply return envelope or a stamped return envelope. All stamped return envelopes bore the same 33-cent stamp, which featured an American flag in the foreground, with looming beige skyscrapers and a blue-green sky in the background. All respondents received the same cover letter and one-page questionnaire. Only a single mailing was performed without telephone reminders or follow-ups.
* Corresponding author. Tel: 410-857-2534; fax: 410-386-4601. E-mail address: [email protected] 0895-4356/01/$ – see front matter © 2001 Elsevier Science Inc. All rights reserved. PII: S0895-4356(00)00 2 7 7 - 8
1
See www.hematology.org for more information about ASH.
M.B. Streiff et al. / Journal of Clinical Epidemiology 54 (2001) 430–432
3. Results Of the 1500 stamped return envelopes mailed out, 570 were returned by October 1, 1999 for a response rate of 38%. In contrast, 479 of 1489 business reply envelopes were returned for a response rate of 32%. Eleven surveys containing business reply envelopes were returned to sender due to a change of address. Thus, the return rate for stamped envelopes was six percentage points greater than for business reply envelopes. This is equivalent to a 19% greater yield for stamped return envelopes. Using a test of significance between proportions, this difference is statistically significant at the P ⫽ .0005 level. Of equal importance to the survey researcher is the cost of each returned survey. Although attractive on the surface because only returned mail pieces are charged, business reply envelopes have a number of hidden costs as well as time-consuming requirements. First one must fill out paperwork and pay for a business reply permit. There are different kinds of permits; one is cheaper, $100, but means that each envelope returned is billed at 63 cents. The other permit is designed for those who expect a higher volume of responses wherein an additional fee of $300 is required but each returned envelope costs only 41 cents. Furthermore, one must pay to have the template made for the business reply envelope. Samples of the printed business reply envelope must be approved by a postal employee to ensure that the postal machines can correctly scan them for postage costs. Another logistic issue involves financial arrangements with the local post-office so that the costs of incoming envelopes can be deducted from a designated account.2 In our study, we compared the average cost for business reply and stamped return envelopes by dividing the overall costs of each by the number of responses received in each category. When taking into account postage costs and other factors related to creating business reply envelopes, the cost per returned survey was $2.62 for business reply envelopes versus $1.82 for stamped return envelopes, making stamped return envelopes 80 cents less expensive per survey returned (see Table 1). 4. Discussion Survey researchers are confronted with the need to maximize response rate while minimizing costs. A number of strategies have been found to increase response rate including personalizing correspondence with handwritten address
2 In this study, the investigators had to make multiple trips to the post office to leave a check to pay for the return envelopes. After calling to find out what to bring and when, ostensibly to avoid wasted trips, study employees were told the following, on different occasions, after waiting in long lines: 1) It would not be possible to drop off the check without bringing a copy of the postal permit; 2) Return at some other time because the person who handles these matters is not present to accept the check. The point is not to malign the postal bureaucracy, but rather to illustrate potential problems that are obviated by the use of stamps.
431
Table 1 Costs of using business reply versus stamped return envelopes (total costs divided by number of responses) Business reply: (response rate: 32%) Stamps to mail survey to respondents Cost of 1500 envelopes holding survey, cover letter, and return envelope Business reply permit Return postage (for 479 respondents) Cost of 1500 envelopes (prior to being printed with business reply design) Costs for making 1500 business reply envelopes ($317) Printing the business reply envelopes Design charge for business reply envelopes Printing charge for business reply envelopes Post office processing fee (recordation) Tax (for total business reply envelope job) Total Cost per returned business reply survey: $2.62
$495 $22 $100 $302 $22 $225 $45 $25 $7 $15 $1,256
Stamped envelopes: (response rate: 38%) Stamps to mail survey to respondents Return postage Cost of 1500 envelopes holding survey, cover letter, and return envelope Cost of 1500 envelopes to be returned to investigators 1500 return address labels Total
$495 $495 $22 $22 $6 $1,040
Cost per returned stamped reply envelope: $1.82 Note: The above costs do not include labor costs involved for stuffing envelopes or placing address labels on envelopes, expenses incurred for both types of envelopes.
labels [8], limiting questionnaire length [9], using monetary incentives, having university sponsorship [10], using special commemorative stamps [11], and including telephone or mailed follow-up [7]. Other strategies include enclosing a pencil in the mailing which doubled the response rate in one study of physicians who had not responded to an initial mailing with stamped return envelopes [3]. One particularly effective technique—if resources permit—is personalizing the mailing: in a 70-item survey of physician attitudes about issues pertinent to patient care, nonrespondents were more likely to respond to a second mailing when a personalized thank you was added to the cover letter (41% increase) and among nonrespondents sent a third mailing, there was a 53% increase in responses when the doctors’ names were typed individually instead of using computerized address labels [2]. Another tactic employed includes having the cover letter sent by respondent’s own general practitioner versus a research unit which yielded a ten percentage point higher response rate [6]. While a number of previous studies have included physician respondents, the present study is unique in a number of ways. This survey represents the largest performed to date examining differential response rates by type of postage among medical subspecialists. Although several previous studies have examined the impact of return envelope type
432
M.B. Streiff et al. / Journal of Clinical Epidemiology 54 (2001) 430–432
upon response rates and study costs, it was unclear whether these results would apply when the study organizers and respondents are colleagues in the same subspecialty professional organization. Since the survey concerned a disorder of unique interest to the survey population and the results were to be presented at the annual meeting of the American Society of Hematology, we anticipated that the effect of return postage would be minimal. Nevertheless, our results indicate that advantages of stamped return postage hold regardless of the differences between our study and previous research. In fact, the difference in response rate between stamped and business reply envelopes in our study is comparable to previous surveys of both physicians [7] and nonphysicians [12,13]. For example, in their survey about pregnancy among resident physicians, Shiono and Klebanoff [4] found that stamped returned envelopes had an advantage of only about 3% points over franked envelopes (for which postage is slightly less than stamps) yet among men, the difference was about six percentage points. Furthermore, our response rates are equivalent to those obtained after first mailing in other studies [7,13]. Likewise, the cost-effectiveness of stamped return envelopes demonstrated in previous studies also was documented in the present study. We surmise that respondents are more likely to feel guilty about throwing away a stamp that has been paid for than discarding a business reply envelope. It is noteworthy that this phenomenon appears to be at work even for a commodity worth only 33 cents. A limitation of the present study is the 35% response rate. We anticipated a higher response rate because of the topic and study population. Unfortunately, use of additional strategies to improve response rates (personalizing cover letters and address labels, use of monetary incentives, follow-up telephone calls or additional mailings) were not feasible given the large study population. 5. Conclusion A mail survey of physician members of a professional subspecialty organization was conducted. In congruence with previous research, stamped envelopes were found more cost effective than business reply envelopes. These results might not be applicable to studies with very large sample sizes and an anticipated low rate of return for which the costs incurred in creating and implementing business reply
envelopes can be averaged over a large number of mail pieces. While the costs involved with business reply return envelopes were apparent before even mailing out the survey, the goal was to determine whether stamps yielded a higher response rate, even among a professional society of physicians whom one might expect to either return or discard the survey, regardless of the type of return postage. This study, however, has shown that they, too, are more likely to return stamped envelopes, information that may be of use to those concerned with both response rate and costefficiency in survey research. Acknowledgment Funding for this project was provided by the Myeloproliferative Disorders Fund.
References [1] Dillman DA, Sangster RL. Mail surveys: A comprehensive bibliography 1974–1989. Chicago, IL: Council of Planning Librarians, CPL Bibliography 272. 1991. [2] Maheux B, Legault C, Lambert J. Increasing response rates in physicians’ mail surveys: an experimental study. Am J Public Health 1989; 79:638–9. [3] Sallis JF, Fortmann SP, Solomon DS, Farquhar JW. Increasing returns of physician surveys. Am J Public Health 1984;74:1043. [4] Shino PH, Klebanoff MA. The effect of two mailing strategies on the response to a survey of physicians. Am J Epidemiol 1991;134:539–42. [5] Shosteck H, Fairweather WR. Physician response rates to mail and person interview surveys. Public Opinion Q 1979;43:206–17. [6] Smith WCS, Crombie IK, Campion PD, Knox JDE. Comparison of response rates to a postal questionnaire from a general practice and a research unit. Br Med J 1985;291:1483–5. [7] Urban N, Anderson GL, Tseng A. Effects on response rates and costs of stamps vs business reply in a mail survey of physicians. J Clin Epidemiol 1993;46:455–9. [8] Nederhof AJ. Effects of repetition and consistency of personalization treatments on response rates in mail surveys. Soc Sci Res 1983;12:1–9. [9] Lund E, Gram IT. Response rate according to title and length of questionnaire. Scand J Soc Med 1998;26:154–60. [10] Ransdell LB. Maximinzing [sic] response rate in questionnaire research. Am J Health Behavior 1996;20:50–6. [11] Hensley WE. Increasing response rate by choice of postage stamps. Public Opinion Quarterly 1974;38:280–3. [12] Armstrong JS, Lusk EJ. Return postage in mail surveys. Public Opinion Quarterly 1987;51:233–48. [13] Harris JR, Guffey HJ. Questionnaire returns: stamps versus business reply envelopes revisited. J Marketing Res 1978;15:290–93.
A mail survey of United States hematologists and oncologists: A comparison of business reply versus stamped return envelopes Michael B. Streiff a, Lauren Dundesb,*, Jerry L. Spivaka a Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD Department of Sociology, Western Maryland College, 2 College Hill, Westminster, MD 21157 Received 1 December 1999; received in revised form 12 May 2000; accepted 12 May 2000
b
Abstract Mailed surveys are a popular means of obtaining data on large populations. In July 1999 a mail survey was conducted among 3000 randomly selected members of the American Society of Hematology to assess their approach to diagnosis and treatment of polycythemia vera. Because the researchers and the study population are members of the same professional organization with a vested interest in the results, we anticipated that the advantages of return stamped postage seen in previous studies would be less significant. The response rate for stamped return envelopes was 38% versus 32% for business reply envelopes. This statistically significant difference (P ⫽ .0005) of six percentage points is comparable to previous research. Excluding labor, the total cost per returned survey was $2.62 for business reply envelopes versus $1.82 for stamped return envelopes. We conclude that stamped return envelopes are a more effective and cost-efficient means of procuring data from physician specialists. © 2001 Elsevier Science Inc. All rights reserved. Keywords: Physician surveys; Mail surveys; Response rates; Costs; Business reply postage; Stamped return postage
1. Introduction Mail surveys are a popular means of obtaining research data on a large population of subjects. As for any type of research, response rate is critical for an accurate assessment of the survey population’s opinions. Previous researchers have identified a number of different strategies to maximize response rates [1]. These include personalized correspondence, multiple mailings, mailed and telephone reminders, as well as monetary and other forms of inducement for survey completion. Several previous studies have applied these strategies to mail surveys targeted at physicians [2–7]. However, it is unclear whether such strategies are necessary when the investigators are surveying colleagues in a subspecialty professional society about a topic of unique interest to the survey population. The purpose of this study was to determine the impact of return postage on the response rate and the economics of a mail survey of a large group of medical subspecialists. 2. Methods In July 1999, two hematologists at a large academic health center mailed an anonymous one-page, 12-item sur-
vey and cover letter to their hematologist and oncologist colleagues who were members of the American Society of Hematology (ASH).1 The survey, which took approximately 5 minutes to complete, asked clinicians about their approach to the diagnosis and treatment of polycythemia vera, an acquired bone marrow disorder characterized by elevated blood counts and a predisposition to thrombosis, bleeding, and rarely, leukemia. Although a large number of clinical studies have been conducted to establish diagnostic and therapeutic strategies for this disorder, no large-scale survey has been conducted to assess the impact of these trials on clinical practice. A random sample of 3000 of the total United States ASH membership of 7100 were sent surveys. To determine the impact of different forms of return postage on response rate and the costs of the survey, half of the respondents were sent either a business reply return envelope or a stamped return envelope. All stamped return envelopes bore the same 33-cent stamp, which featured an American flag in the foreground, with looming beige skyscrapers and a blue-green sky in the background. All respondents received the same cover letter and one-page questionnaire. Only a single mailing was performed without telephone reminders or follow-ups.
* Corresponding author. Tel: 410-857-2534; fax: 410-386-4601. E-mail address: [email protected] 0895-4356/01/$ – see front matter © 2001 Elsevier Science Inc. All rights reserved. PII: S0895-4356(00)00 2 7 7 - 8
1
See www.hematology.org for more information about ASH.
M.B. Streiff et al. / Journal of Clinical Epidemiology 54 (2001) 430–432
3. Results Of the 1500 stamped return envelopes mailed out, 570 were returned by October 1, 1999 for a response rate of 38%. In contrast, 479 of 1489 business reply envelopes were returned for a response rate of 32%. Eleven surveys containing business reply envelopes were returned to sender due to a change of address. Thus, the return rate for stamped envelopes was six percentage points greater than for business reply envelopes. This is equivalent to a 19% greater yield for stamped return envelopes. Using a test of significance between proportions, this difference is statistically significant at the P ⫽ .0005 level. Of equal importance to the survey researcher is the cost of each returned survey. Although attractive on the surface because only returned mail pieces are charged, business reply envelopes have a number of hidden costs as well as time-consuming requirements. First one must fill out paperwork and pay for a business reply permit. There are different kinds of permits; one is cheaper, $100, but means that each envelope returned is billed at 63 cents. The other permit is designed for those who expect a higher volume of responses wherein an additional fee of $300 is required but each returned envelope costs only 41 cents. Furthermore, one must pay to have the template made for the business reply envelope. Samples of the printed business reply envelope must be approved by a postal employee to ensure that the postal machines can correctly scan them for postage costs. Another logistic issue involves financial arrangements with the local post-office so that the costs of incoming envelopes can be deducted from a designated account.2 In our study, we compared the average cost for business reply and stamped return envelopes by dividing the overall costs of each by the number of responses received in each category. When taking into account postage costs and other factors related to creating business reply envelopes, the cost per returned survey was $2.62 for business reply envelopes versus $1.82 for stamped return envelopes, making stamped return envelopes 80 cents less expensive per survey returned (see Table 1). 4. Discussion Survey researchers are confronted with the need to maximize response rate while minimizing costs. A number of strategies have been found to increase response rate including personalizing correspondence with handwritten address
2 In this study, the investigators had to make multiple trips to the post office to leave a check to pay for the return envelopes. After calling to find out what to bring and when, ostensibly to avoid wasted trips, study employees were told the following, on different occasions, after waiting in long lines: 1) It would not be possible to drop off the check without bringing a copy of the postal permit; 2) Return at some other time because the person who handles these matters is not present to accept the check. The point is not to malign the postal bureaucracy, but rather to illustrate potential problems that are obviated by the use of stamps.
431
Table 1 Costs of using business reply versus stamped return envelopes (total costs divided by number of responses) Business reply: (response rate: 32%) Stamps to mail survey to respondents Cost of 1500 envelopes holding survey, cover letter, and return envelope Business reply permit Return postage (for 479 respondents) Cost of 1500 envelopes (prior to being printed with business reply design) Costs for making 1500 business reply envelopes ($317) Printing the business reply envelopes Design charge for business reply envelopes Printing charge for business reply envelopes Post office processing fee (recordation) Tax (for total business reply envelope job) Total Cost per returned business reply survey: $2.62
$495 $22 $100 $302 $22 $225 $45 $25 $7 $15 $1,256
Stamped envelopes: (response rate: 38%) Stamps to mail survey to respondents Return postage Cost of 1500 envelopes holding survey, cover letter, and return envelope Cost of 1500 envelopes to be returned to investigators 1500 return address labels Total
$495 $495 $22 $22 $6 $1,040
Cost per returned stamped reply envelope: $1.82 Note: The above costs do not include labor costs involved for stuffing envelopes or placing address labels on envelopes, expenses incurred for both types of envelopes.
labels [8], limiting questionnaire length [9], using monetary incentives, having university sponsorship [10], using special commemorative stamps [11], and including telephone or mailed follow-up [7]. Other strategies include enclosing a pencil in the mailing which doubled the response rate in one study of physicians who had not responded to an initial mailing with stamped return envelopes [3]. One particularly effective technique—if resources permit—is personalizing the mailing: in a 70-item survey of physician attitudes about issues pertinent to patient care, nonrespondents were more likely to respond to a second mailing when a personalized thank you was added to the cover letter (41% increase) and among nonrespondents sent a third mailing, there was a 53% increase in responses when the doctors’ names were typed individually instead of using computerized address labels [2]. Another tactic employed includes having the cover letter sent by respondent’s own general practitioner versus a research unit which yielded a ten percentage point higher response rate [6]. While a number of previous studies have included physician respondents, the present study is unique in a number of ways. This survey represents the largest performed to date examining differential response rates by type of postage among medical subspecialists. Although several previous studies have examined the impact of return envelope type
432
M.B. Streiff et al. / Journal of Clinical Epidemiology 54 (2001) 430–432
upon response rates and study costs, it was unclear whether these results would apply when the study organizers and respondents are colleagues in the same subspecialty professional organization. Since the survey concerned a disorder of unique interest to the survey population and the results were to be presented at the annual meeting of the American Society of Hematology, we anticipated that the effect of return postage would be minimal. Nevertheless, our results indicate that advantages of stamped return postage hold regardless of the differences between our study and previous research. In fact, the difference in response rate between stamped and business reply envelopes in our study is comparable to previous surveys of both physicians [7] and nonphysicians [12,13]. For example, in their survey about pregnancy among resident physicians, Shiono and Klebanoff [4] found that stamped returned envelopes had an advantage of only about 3% points over franked envelopes (for which postage is slightly less than stamps) yet among men, the difference was about six percentage points. Furthermore, our response rates are equivalent to those obtained after first mailing in other studies [7,13]. Likewise, the cost-effectiveness of stamped return envelopes demonstrated in previous studies also was documented in the present study. We surmise that respondents are more likely to feel guilty about throwing away a stamp that has been paid for than discarding a business reply envelope. It is noteworthy that this phenomenon appears to be at work even for a commodity worth only 33 cents. A limitation of the present study is the 35% response rate. We anticipated a higher response rate because of the topic and study population. Unfortunately, use of additional strategies to improve response rates (personalizing cover letters and address labels, use of monetary incentives, follow-up telephone calls or additional mailings) were not feasible given the large study population. 5. Conclusion A mail survey of physician members of a professional subspecialty organization was conducted. In congruence with previous research, stamped envelopes were found more cost effective than business reply envelopes. These results might not be applicable to studies with very large sample sizes and an anticipated low rate of return for which the costs incurred in creating and implementing business reply
envelopes can be averaged over a large number of mail pieces. While the costs involved with business reply return envelopes were apparent before even mailing out the survey, the goal was to determine whether stamps yielded a higher response rate, even among a professional society of physicians whom one might expect to either return or discard the survey, regardless of the type of return postage. This study, however, has shown that they, too, are more likely to return stamped envelopes, information that may be of use to those concerned with both response rate and costefficiency in survey research. Acknowledgment Funding for this project was provided by the Myeloproliferative Disorders Fund.
References [1] Dillman DA, Sangster RL. Mail surveys: A comprehensive bibliography 1974–1989. Chicago, IL: Council of Planning Librarians, CPL Bibliography 272. 1991. [2] Maheux B, Legault C, Lambert J. Increasing response rates in physicians’ mail surveys: an experimental study. Am J Public Health 1989; 79:638–9. [3] Sallis JF, Fortmann SP, Solomon DS, Farquhar JW. Increasing returns of physician surveys. Am J Public Health 1984;74:1043. [4] Shino PH, Klebanoff MA. The effect of two mailing strategies on the response to a survey of physicians. Am J Epidemiol 1991;134:539–42. [5] Shosteck H, Fairweather WR. Physician response rates to mail and person interview surveys. Public Opinion Q 1979;43:206–17. [6] Smith WCS, Crombie IK, Campion PD, Knox JDE. Comparison of response rates to a postal questionnaire from a general practice and a research unit. Br Med J 1985;291:1483–5. [7] Urban N, Anderson GL, Tseng A. Effects on response rates and costs of stamps vs business reply in a mail survey of physicians. J Clin Epidemiol 1993;46:455–9. [8] Nederhof AJ. Effects of repetition and consistency of personalization treatments on response rates in mail surveys. Soc Sci Res 1983;12:1–9. [9] Lund E, Gram IT. Response rate according to title and length of questionnaire. Scand J Soc Med 1998;26:154–60. [10] Ransdell LB. Maximinzing [sic] response rate in questionnaire research. Am J Health Behavior 1996;20:50–6. [11] Hensley WE. Increasing response rate by choice of postage stamps. Public Opinion Quarterly 1974;38:280–3. [12] Armstrong JS, Lusk EJ. Return postage in mail surveys. Public Opinion Quarterly 1987;51:233–48. [13] Harris JR, Guffey HJ. Questionnaire returns: stamps versus business reply envelopes revisited. J Marketing Res 1978;15:290–93.