- Email: [email protected]
Methodological considerations in corporate mail surveys: A research note
Methodological Considerations in Corporate Mail Surveys: A Research Note Roger A.
Michael
Southern Methodist
University
Academic interest in analyzing corporate operations has produced considerable research of a mail survey variety. However, recent evidence suggests that executive participation in university sponsored mail surveys is declining dramatically [41-a particular11 acute problem for academicians, due to the relatively limited corporate population from which sample surveys can be drawn. Accordingly, research designed to seek improvements in corporate survey response rates is necessary to preserve the mail questionnaire as a visible research instrument. In spite of the voluminous body of empirical research on improving mail returns [ 2,3], stimulating response rates in surveys directed toward corporations has been largely ignored. Reportedly, corporations fail to return questionnaires because of (1) the completion cost-time ofexecutives to complete questionnaire; (2) the nature of information requested (sensitivity of competitive data); and (3) questionnaire design and length [4] indicating that researchers should give increased attention to questionnaire construction as well as the more subtle inducements which may be also required. The purpose of this investigation was to determine whether three inducement techniques frequently used in mail surveys of the general population would influence response sates in corporate mail surveys. The threle techniques were: (1) format of the cover letkr; (2) appeal used in the cover letter; and (3) availability of postage on return envelopes.
Method An experiment w$asconducted to test the effect of three variables on questionnaire return rates in a university sponsored mail survey of Fortune 500 corporate presidents. The four page questionnaire foiiugust, 1976 Volume 4, Number 3
275
Journal of Business Research
Table 1: Return Rates bv Treatment Treatment Fwzmt/Appeal/Postage I e--
Return Percent
Individual/altruistic/stamp
27/55 = 49%
Individual/altruistic/nostamp
26/55 = 47
Individual/egoistic/rtamp
20/55 = 36
Individual/egoistic/nostamp
20155 = 36
Form/altruistic/stamp
15155 = 27
Form/aliruistic/nostamp
23/55 = 42
Pam/egoistic/stamp
13155 = 24
Form/egoistic/nostamp
13/55 = 24
Overall return rate
1571440 = 36
cused on product recall practices among Fortune 500 organizations. The techniques were manipulated to create a 2 x 2 x 2 factorial design, or eight treatments. The inducement techniques tested were: (1) cover letter format (individual vs. form); (2) the appeal used in cover letters (altruistic vs. egoistic); and (3) availability of return postage on LIslelf-addressed return envelope (stamp vs. no stamp) incltded with all questionnaires. Two applications of each factor were operationalized as follows.1 Letter format was divided into two separate formats, one consisting of a cover letter individually typed with the president’s inside address, individualized salutation, and a handwritten signature of the researcher; the second format was a mimeographed “form” cover letter included with the questionnaire. Appeal was defined as *whether the letter emphasized the importance of a reply to assist the researcher in hi8 investigation (altruistic), or emphasized the president’s reply due to his knowledge of corporate activities and the importance of his firm as a member of F~tiune’s 500 (egoistic). Return postage was defined simply as whether or not a stamp was placed on the return envelope. Questionnaires were preceded for identification hy treatment. Fifty-five fums were assigned randomly to each treatment for a total mailing of 440 questionnaires. Sixty firms were not included in the study bEtcause they were non-manufacturing firms (e.g., service
Methodological
Considerations
279
Table;2: Analysis of Variance Effects of Three Variables on Return Dates Source
ss
df
MS
F
P
Letter format
1.912
1
1.912
8.50
c.005
B: Appeal
1.421
1
1.421
6.32
c.025
-112
1
.112
.50
n.6.
A?3
.002
1
.002
.Ol
n.s.
AC
.184
1
,184
.82
n.s.
BC
.:11
1
.lll
.49
n.s.
ABC
.184
1
,184
.82
n.s.
97.054
432
-225
A:
C:
Postage
Error Total
100.98
Note: F .g75%432)
439
= 5.02; Fsg9&,432) _'
= 7.88
and retailing firms). Response rates for each treatment were examined using the analysis of variance. This procedure has been shown to be appropriate for dichotomous dependent variables provided the number of observations in each treatment are equal and exceed fifty [I]. Results
and Dbcussion
Table I shows the response rates by treatment, The overall response rate was 36 percent, and is based on returned questionnaires only. Instances where corporations replied, but were unwilling to participate in the survey, were coded as non-returned questionnaires.2 Table 2 presents the analysis of variance describing the effects of the response inducement techniques. Two ‘statistically significant main effects for letter format (pc.005) and appeal (pc.025) were observed. Inspection of response rates reveals that individualized letters produced higher returns than form letters (42 percent vs. 29 percent), An altruistic appeal geI\erated a larger return race than the egoistic appeal (4I percent vs. 30 percent). Availability of return postage had no statistically significant impact on response rates. Furthermore, no higher order interaction effects between variables were found.
280
journal of Business Research
The observed results provide valuable insights into the dynamics of response behavior when surveying corporate executives. For example, postage on return envelopes had no promotional impact on return rates even though this inducement has consistently produced higher response in mail surveys of the general population [2, 31. Inspection of return envelopes in this study indicated that most return envelopes were metered, including envelopes with postage. Furthermore, the results suggest that asking for an executive’s assistance outright rather than appealing to the executive’s ego is more productive. This result is somewhat contradictory to the research of Petry and Quackenbush 141which suggested that executives often hesitate to respond because they believe university personnel desire to publish studies for personal benefit with little value to the firm. Finally, individualized cover letters achieved higher returns than form letters indicating that executives prefer to be queried personally rather than en masse. This finding is consistent with previous research [4] which has shown that questionnaires with form cover letters are not likely to be returned by some executives. However, the substantial response differential observed in this study indicates that letter format may be more important than executives consciously admit. The results of this study have direct implications for prospective users of corporate mail surveys. First, researchers should ind.ividualize cover letters appended to questionnaires rather than enclosing form letters. Secretarial expense and time associated with this procedure can be minimized by using an MT/ST typing system. Second, if researchers cannot individualize cover letters due to expense or time constraints, an altruistic appeal should be included in the body of the cover letter. Virtually no incremental cost is associated with using this appeal; yet, the differential in response rate over an egoistic appeal is substantial. Finally, this research indicates that it is unnecessary to prov:de postage on return envelopes for corporate executives. This finding is particularly relevant from the perspective of the direct and increasing cost of postage. This research has investigated three response inducement techniques commonly used in mail survey research. Other means for increasing corporate response should be tested in the hture sucli as (f) questionnaire length; (2) deadline dates; (3) size, reproduction, and color; (4) preliminary notice; and (5) follow-ups to the initial questionnaire. Additional research of this nature should help insure
MethodokogicaC Considerations
the future viability of the mail questionnaire research instrument.
281
as a useful academic
Footnotes l Copies of the cover letters used in the research may be obtained by WI iting the authors. * Twenty-four corporations fell into this category of which eighteen returned the questionnaire itself. Treatment codes indicated that the questionnaires were evenly distributed among the eight treatments.
References 1. Glass. G. et. al. “Consequences of F;rilure to Meet Assumptions Underlying the Fixed ELcts Analysis of Variance and Cobidance.” Weoieu: of Educational Research 42 (Summer 1972): 237-88. 2. Kanuk, Leslie and Berenson, Conrad “Mail Survey Response Bates: A Literature Review.” Journal of Mat-keting Research 12 (November 1975): 440-53. 3. Linsky, A. “Stimulating Response to Mailed Questionnaires: A Beview.” Public Opinion QuarterZy 39 (Summer 1975): 82-301, 4. Petry, G. and Quackenbush, S. “The Conversation of the Questionnaire as a Research Besrsurce.” Business ~ofizons 17 (Au@st 1974): 43-47.
Michael
Southern Methodist
University
Academic interest in analyzing corporate operations has produced considerable research of a mail survey variety. However, recent evidence suggests that executive participation in university sponsored mail surveys is declining dramatically [41-a particular11 acute problem for academicians, due to the relatively limited corporate population from which sample surveys can be drawn. Accordingly, research designed to seek improvements in corporate survey response rates is necessary to preserve the mail questionnaire as a visible research instrument. In spite of the voluminous body of empirical research on improving mail returns [ 2,3], stimulating response rates in surveys directed toward corporations has been largely ignored. Reportedly, corporations fail to return questionnaires because of (1) the completion cost-time ofexecutives to complete questionnaire; (2) the nature of information requested (sensitivity of competitive data); and (3) questionnaire design and length [4] indicating that researchers should give increased attention to questionnaire construction as well as the more subtle inducements which may be also required. The purpose of this investigation was to determine whether three inducement techniques frequently used in mail surveys of the general population would influence response sates in corporate mail surveys. The threle techniques were: (1) format of the cover letkr; (2) appeal used in the cover letter; and (3) availability of postage on return envelopes.
Method An experiment w$asconducted to test the effect of three variables on questionnaire return rates in a university sponsored mail survey of Fortune 500 corporate presidents. The four page questionnaire foiiugust, 1976 Volume 4, Number 3
275
Journal of Business Research
Table 1: Return Rates bv Treatment Treatment Fwzmt/Appeal/Postage I e--
Return Percent
Individual/altruistic/stamp
27/55 = 49%
Individual/altruistic/nostamp
26/55 = 47
Individual/egoistic/rtamp
20/55 = 36
Individual/egoistic/nostamp
20155 = 36
Form/altruistic/stamp
15155 = 27
Form/aliruistic/nostamp
23/55 = 42
Pam/egoistic/stamp
13155 = 24
Form/egoistic/nostamp
13/55 = 24
Overall return rate
1571440 = 36
cused on product recall practices among Fortune 500 organizations. The techniques were manipulated to create a 2 x 2 x 2 factorial design, or eight treatments. The inducement techniques tested were: (1) cover letter format (individual vs. form); (2) the appeal used in cover letters (altruistic vs. egoistic); and (3) availability of return postage on LIslelf-addressed return envelope (stamp vs. no stamp) incltded with all questionnaires. Two applications of each factor were operationalized as follows.1 Letter format was divided into two separate formats, one consisting of a cover letter individually typed with the president’s inside address, individualized salutation, and a handwritten signature of the researcher; the second format was a mimeographed “form” cover letter included with the questionnaire. Appeal was defined as *whether the letter emphasized the importance of a reply to assist the researcher in hi8 investigation (altruistic), or emphasized the president’s reply due to his knowledge of corporate activities and the importance of his firm as a member of F~tiune’s 500 (egoistic). Return postage was defined simply as whether or not a stamp was placed on the return envelope. Questionnaires were preceded for identification hy treatment. Fifty-five fums were assigned randomly to each treatment for a total mailing of 440 questionnaires. Sixty firms were not included in the study bEtcause they were non-manufacturing firms (e.g., service
Methodological
Considerations
279
Table;2: Analysis of Variance Effects of Three Variables on Return Dates Source
ss
df
MS
F
P
Letter format
1.912
1
1.912
8.50
c.005
B: Appeal
1.421
1
1.421
6.32
c.025
-112
1
.112
.50
n.6.
A?3
.002
1
.002
.Ol
n.s.
AC
.184
1
,184
.82
n.s.
BC
.:11
1
.lll
.49
n.s.
ABC
.184
1
,184
.82
n.s.
97.054
432
-225
A:
C:
Postage
Error Total
100.98
Note: F .g75%432)
439
= 5.02; Fsg9&,432) _'
= 7.88
and retailing firms). Response rates for each treatment were examined using the analysis of variance. This procedure has been shown to be appropriate for dichotomous dependent variables provided the number of observations in each treatment are equal and exceed fifty [I]. Results
and Dbcussion
Table I shows the response rates by treatment, The overall response rate was 36 percent, and is based on returned questionnaires only. Instances where corporations replied, but were unwilling to participate in the survey, were coded as non-returned questionnaires.2 Table 2 presents the analysis of variance describing the effects of the response inducement techniques. Two ‘statistically significant main effects for letter format (pc.005) and appeal (pc.025) were observed. Inspection of response rates reveals that individualized letters produced higher returns than form letters (42 percent vs. 29 percent), An altruistic appeal geI\erated a larger return race than the egoistic appeal (4I percent vs. 30 percent). Availability of return postage had no statistically significant impact on response rates. Furthermore, no higher order interaction effects between variables were found.
280
journal of Business Research
The observed results provide valuable insights into the dynamics of response behavior when surveying corporate executives. For example, postage on return envelopes had no promotional impact on return rates even though this inducement has consistently produced higher response in mail surveys of the general population [2, 31. Inspection of return envelopes in this study indicated that most return envelopes were metered, including envelopes with postage. Furthermore, the results suggest that asking for an executive’s assistance outright rather than appealing to the executive’s ego is more productive. This result is somewhat contradictory to the research of Petry and Quackenbush 141which suggested that executives often hesitate to respond because they believe university personnel desire to publish studies for personal benefit with little value to the firm. Finally, individualized cover letters achieved higher returns than form letters indicating that executives prefer to be queried personally rather than en masse. This finding is consistent with previous research [4] which has shown that questionnaires with form cover letters are not likely to be returned by some executives. However, the substantial response differential observed in this study indicates that letter format may be more important than executives consciously admit. The results of this study have direct implications for prospective users of corporate mail surveys. First, researchers should ind.ividualize cover letters appended to questionnaires rather than enclosing form letters. Secretarial expense and time associated with this procedure can be minimized by using an MT/ST typing system. Second, if researchers cannot individualize cover letters due to expense or time constraints, an altruistic appeal should be included in the body of the cover letter. Virtually no incremental cost is associated with using this appeal; yet, the differential in response rate over an egoistic appeal is substantial. Finally, this research indicates that it is unnecessary to prov:de postage on return envelopes for corporate executives. This finding is particularly relevant from the perspective of the direct and increasing cost of postage. This research has investigated three response inducement techniques commonly used in mail survey research. Other means for increasing corporate response should be tested in the hture sucli as (f) questionnaire length; (2) deadline dates; (3) size, reproduction, and color; (4) preliminary notice; and (5) follow-ups to the initial questionnaire. Additional research of this nature should help insure
MethodokogicaC Considerations
the future viability of the mail questionnaire research instrument.
281
as a useful academic
Footnotes l Copies of the cover letters used in the research may be obtained by WI iting the authors. * Twenty-four corporations fell into this category of which eighteen returned the questionnaire itself. Treatment codes indicated that the questionnaires were evenly distributed among the eight treatments.
References 1. Glass. G. et. al. “Consequences of F;rilure to Meet Assumptions Underlying the Fixed ELcts Analysis of Variance and Cobidance.” Weoieu: of Educational Research 42 (Summer 1972): 237-88. 2. Kanuk, Leslie and Berenson, Conrad “Mail Survey Response Bates: A Literature Review.” Journal of Mat-keting Research 12 (November 1975): 440-53. 3. Linsky, A. “Stimulating Response to Mailed Questionnaires: A Beview.” Public Opinion QuarterZy 39 (Summer 1975): 82-301, 4. Petry, G. and Quackenbush, S. “The Conversation of the Questionnaire as a Research Besrsurce.” Business ~ofizons 17 (Au@st 1974): 43-47.