- Email: [email protected]
The effect of two variables on industrial mail survey returns
The Effect of Two Variables on Industrial Mail Survey Returns David Jobber StuartSanderson An experiment was designed to test the eflectiveness on response rates of (1) an offer of a full survey report with respondents entering their names and addresses on the questionnaire and (2) the use of typed and handwritten postscripts for an industrial population. Results demonstrate that the use of postscripts was ineffective, while the offer actually reduced response. The ineffectiveness of the handwritten postscript strengthens the viewpoint that methods that may be successful for household populations may not always be effective ,for industrial populations.
The problem of identifying methods of increasing response rates of mail surveys to industrial populations is particularly important since response is, in general, lower for these populations than for household populations [ 11. One group of inducement methods that has been successful in this respect is the use of rewards. Experimental studies have shown that both monetary and nonmonetary inducements have stimulated increased returns from industrial populations. Watson [2], Erdos [ 31, Address correspondence to: David Jobber, Lecturer in Marketing. University of Bradford, Management Centre, Emm Lane. Bradford, West Yorkshire BD9-4JL England. fndusrrial Mar!wiq Manugemmt 14, I 19- 12 I ( 1985) 0 Elszvier Science Publishing Co.. Inc., 1985 52 Vanderbilt Ave., New York, New York 10017
and Hansen [4] achieved significantly higher response using a 25e coin. The inclusion of a 10~ coin was, also, supported by experimental evidence in studies conducted by Kimball [S], Watson 121, Erdos [3], Pressley and Tullar [6], and Tullar, Pressley and Gentry [7]. The efficacy of using a small nonmonetary incentive was determined by Hansen [4], who obtained a significant increase in response with the inclusion of a ballpoint pen worth about 25~ compared with the control group. Another nonmonetary reward that has been used to stimulate response in industrial mail survey surveys is the offer of a free copy of the survey report with respondents entering their name and address on the questionnaire so that the report could be sent to them. In one academic institution, this form of response inducement has been used in five separate surveys in recent years [8- 121. However, a literature search revealed that the effects of this form of offer to businessmen had not been tested experimentally. Consequently, an experiment was designed in order to reveal the effectiveness of this approach. The results were of particular interest since it was by no means certain that an offer made in this way would improve response rates. The addition of the re119 OOIY-8.501/85/$3.30
spondent’s name and address on the questionnaire would remove the cloak of anonymity from his answers. Indeed, Futrell and Hise [ 131 found that granting anonymity was effective in increasing response among an industrial population when sensitive information was sought. The second objective of the experiment was to provide further evidence regarding the hypothesis that inducement methods which are effective for household populations may not be effective for industrial populations [ 14, 151. One inducement method that has increased response in household populations is the use of a handwritten postscript. Frazier and Bird [ 161 found that a handwritten
METHOD A university-sponsored mail survey was conducted among 440 senior marketing executives of large manufacturing companies, exploring the design and extent of implementation of marketing information systems. Each company was randomly allocated into one of three experimental groups and one control group. The first experimental group was mailed a covering letter which contained the offer of a free copy of the results of the survey as the final paragraph of the body of the letter. The second experimental group received the same covering letter except that the offer, using identical wording,
Methods thatappear to increaseresponse from household populationsmay not be successful in industrial mail surveys. postscript had a significant effect on response rates among a sample of Idaho residents. Hoppe [ 17 ] also found a significant increase in response when a handwritten postscript was used among a sample of car drivers. However, Pressley [ 141, in two surveys of executives, found that a handwritten postscript was ineffective in stimulating response, as did Childers, Pride and Ferrell [ 181 among a sample of businessmen. The present study was designed, therefore, to test the effectiveness of a handwritten postscript compared with no postscript, and with a typed postscript.
DAVID JOBBER teaches at the Bradford Management Centre, Unrversrty of Bradford. Prior to entering academic work he gained marketing experience with T. I. Raleigh He is the coauthor of a book on selling and sales management and his research interests lie in the provision of information for marketing decision making. STUART SANDERSON untrl recently was a Pnncrpal Lecturer in Marketing at Huddersfield Polytechnrc and Course Tutor for the B. A. Textile Marketing Course. He IS now Lecturer in Business Policy, Bradford Management Centre, University of Bradford. He gained industrial experience with ICI
120
was made as a typed postscript; and the third experimental group received the offer in the form of a handwritten postscript. The control group received the same covering letter except that no offer of the survey report was made. in the experimental groups, who The respondents, wished to receive a copy of the report were asked to fill in their name and address at the end of the questionnaire so that the report could be posted to them. No such provision was made for respondents in the control group. All covering letters stressed that replies would be treated in the strictest confidence. The sample size exceeded the minimum as determined by the Feldt-Mahmoud [ 191 method of sample size specification in experimental design. RESULTS Table 1 shows the response rates associated with each treatment. The treatment associated with the highest response was the no offer condition, the difference between the 27.4% associated with the control group and the 15.3% of subjects who replied having been offered a free report was statistically significant at p > 0.01. (x2 = 7.8 1). The form the offer took made no significant difference
TABLE 1 Results of Offer of Free Survey Report and Use of Postscripts Treatment Offer in body of covering letter Offer as typed postscript Offer as handwritten postscript No offer
Percent
w
response 17.8 14.5 13.5 27.4
107 110 104 I06
OOriginal ns of I10 were reduced marginally be return letters indicating that the questionnaire was inapplicable as the office was that of a holding company and, therefore, would not be in a position to require a MKIS.
to response. The difference between the 17.8% of respondents who received the offer in the body of the covering letter, the 14.5% of respondents who received it as a typed postscript, and the 13.5% of respondents who received it as a handwritten postscript was not statistically significant at p < 0.05 (x2 = 0.83).
2. Watson, J. J., Improving Advertising
3
Erdos, P. L., Professional
4
Hansen, R. A., A Self-Perception Interpretation of the Effect of Monetary and Non-monetary Incentives on Mail Survey Respondent Behavior, Journul of Marketing Research 17, 77-83 (February 1980).
5
The practice of offering free survey results and asking respondents to identify themselves on the questionnaire in order to send them the report is not supported in this study. The results suggest that previous research which has used this method may have suffered from reduced response as a consequence. The likely explanation is that the loss of anonymity suppressed response for some recipients of the questionnaire. The results of this experiment confirmed those from earlier studies of industrial populations in that the use of a handwritten postscript did not increase response rates. This finding strengthens the viewpoint that methods that appear to increase response from household populations may not always be successful in industrial mail surveys.
Thompson, Association
M., Industrial Postal Surveys, April 1984.
Seminur.
New York, 1970.
Kimball, A. E., Increasing the Rate of Return in Mail Surveys, Journul of 25, 63-64 (December 1961).
6
Pressley, M. M., and Tullar, W. L., A Factor Interactive Investigation of Mail Survey Response Rates from a Commercial Population, Journcrl of Marketing Reseurch 14 10% I1 I (February 1977).
7
Tullar, W. L., Pressley, M. M., and Gentry, D. L., Toward a Theoretical Framework for Mail Survey Response, Proceedings of the Third Annual Conference of the Academy of Marketing Science (Miami, Floridu), 1979, pp. 243-249.
8 Higgins, J. C., and Finn, R., The Organization and Practice of Corporate Planning in the UK, Long Range Planning 10, 88-92 (August 1977). 9
Jobber, D., and Rainbow, C., A Study of the Development and Implementation of Marketing Information Systems in British Industry, Journul of the Murket Research Society 19(3), IO4- I I I (I 977).
10 Higgins, J. C., and Romano, D. J., Socio-Political Forecasting and Management of Information Systems, OrneRo 83, 303-309 (May 1980). Jobber, D., Hooley, G., and Sanderson, S. M., Marketing in a Hostile Environment: The British Textile Industry, Industricrl Marketing Munugement, forthcoming.
12. Higgins, J. C., and Opdebeeck, E. J., The Microcomputer as a Tool in Financial Planning and Control: Some Survey Results, Accounting und Business Research (Autumn 1984). 13 Futrell, C.. and Hize, R. T., The Effects of Anonymity and Same Day Deadline on the Response Rate to Mail Surveys, Europetin Research I7 I 173 (October 1982). 14. Pressley, M. M., Care Needed When Selecting Response Inducements Mail Surveys to Commercial Populations, Journul qf the Academy Murkefinx Science 6, 336-343 (Fall 1979).
in of
15 Jobber, D., and Sanderson, S. M., The Effects of a Prior Letter and Coloured Questionnaire Paper on Mail Survey Response Rates, Journrrl of the Market
Research Society 25(4),
339-349
(1983).
16 Frazier, G.. and Bird, K.. Increasing the Response of a Mail Questionnaire, Journal qf Marketing 23, l86- I87 (October 1958). 17 Hoppe, D. A., Certain Factors Found to Improve Mail Survey Returns, Proceedings of the Iowa Academy of Science 59, 374-376 (Summer 1952). 18. Childers, T. I., Pride, W. M.. and Ferrell. 0. C., A Reassessment of the Effects of Appeals on Response to Mail Surveys, Journal of Marketing Research 17, 365-370 (August 1980).
REFERENCES I.
Mail Surveys. McGraw-Hill,
Marketing
II
IMPLICATIONS
the Response Rate in Mail Research, Journul of (June 1965).
Research 5, 48-50
Industriul
Marker
Research
19. Feldt, L. S., and Mahmoud, M. W., Power Function Charts for Spccification of Sample Size, Psychometriku 23, 201-210 (1958).
121
The problem of identifying methods of increasing response rates of mail surveys to industrial populations is particularly important since response is, in general, lower for these populations than for household populations [ 11. One group of inducement methods that has been successful in this respect is the use of rewards. Experimental studies have shown that both monetary and nonmonetary inducements have stimulated increased returns from industrial populations. Watson [2], Erdos [ 31, Address correspondence to: David Jobber, Lecturer in Marketing. University of Bradford, Management Centre, Emm Lane. Bradford, West Yorkshire BD9-4JL England. fndusrrial Mar!wiq Manugemmt 14, I 19- 12 I ( 1985) 0 Elszvier Science Publishing Co.. Inc., 1985 52 Vanderbilt Ave., New York, New York 10017
and Hansen [4] achieved significantly higher response using a 25e coin. The inclusion of a 10~ coin was, also, supported by experimental evidence in studies conducted by Kimball [S], Watson 121, Erdos [3], Pressley and Tullar [6], and Tullar, Pressley and Gentry [7]. The efficacy of using a small nonmonetary incentive was determined by Hansen [4], who obtained a significant increase in response with the inclusion of a ballpoint pen worth about 25~ compared with the control group. Another nonmonetary reward that has been used to stimulate response in industrial mail survey surveys is the offer of a free copy of the survey report with respondents entering their name and address on the questionnaire so that the report could be sent to them. In one academic institution, this form of response inducement has been used in five separate surveys in recent years [8- 121. However, a literature search revealed that the effects of this form of offer to businessmen had not been tested experimentally. Consequently, an experiment was designed in order to reveal the effectiveness of this approach. The results were of particular interest since it was by no means certain that an offer made in this way would improve response rates. The addition of the re119 OOIY-8.501/85/$3.30
spondent’s name and address on the questionnaire would remove the cloak of anonymity from his answers. Indeed, Futrell and Hise [ 131 found that granting anonymity was effective in increasing response among an industrial population when sensitive information was sought. The second objective of the experiment was to provide further evidence regarding the hypothesis that inducement methods which are effective for household populations may not be effective for industrial populations [ 14, 151. One inducement method that has increased response in household populations is the use of a handwritten postscript. Frazier and Bird [ 161 found that a handwritten
METHOD A university-sponsored mail survey was conducted among 440 senior marketing executives of large manufacturing companies, exploring the design and extent of implementation of marketing information systems. Each company was randomly allocated into one of three experimental groups and one control group. The first experimental group was mailed a covering letter which contained the offer of a free copy of the results of the survey as the final paragraph of the body of the letter. The second experimental group received the same covering letter except that the offer, using identical wording,
Methods thatappear to increaseresponse from household populationsmay not be successful in industrial mail surveys. postscript had a significant effect on response rates among a sample of Idaho residents. Hoppe [ 17 ] also found a significant increase in response when a handwritten postscript was used among a sample of car drivers. However, Pressley [ 141, in two surveys of executives, found that a handwritten postscript was ineffective in stimulating response, as did Childers, Pride and Ferrell [ 181 among a sample of businessmen. The present study was designed, therefore, to test the effectiveness of a handwritten postscript compared with no postscript, and with a typed postscript.
DAVID JOBBER teaches at the Bradford Management Centre, Unrversrty of Bradford. Prior to entering academic work he gained marketing experience with T. I. Raleigh He is the coauthor of a book on selling and sales management and his research interests lie in the provision of information for marketing decision making. STUART SANDERSON untrl recently was a Pnncrpal Lecturer in Marketing at Huddersfield Polytechnrc and Course Tutor for the B. A. Textile Marketing Course. He IS now Lecturer in Business Policy, Bradford Management Centre, University of Bradford. He gained industrial experience with ICI
120
was made as a typed postscript; and the third experimental group received the offer in the form of a handwritten postscript. The control group received the same covering letter except that no offer of the survey report was made. in the experimental groups, who The respondents, wished to receive a copy of the report were asked to fill in their name and address at the end of the questionnaire so that the report could be posted to them. No such provision was made for respondents in the control group. All covering letters stressed that replies would be treated in the strictest confidence. The sample size exceeded the minimum as determined by the Feldt-Mahmoud [ 191 method of sample size specification in experimental design. RESULTS Table 1 shows the response rates associated with each treatment. The treatment associated with the highest response was the no offer condition, the difference between the 27.4% associated with the control group and the 15.3% of subjects who replied having been offered a free report was statistically significant at p > 0.01. (x2 = 7.8 1). The form the offer took made no significant difference
TABLE 1 Results of Offer of Free Survey Report and Use of Postscripts Treatment Offer in body of covering letter Offer as typed postscript Offer as handwritten postscript No offer
Percent
w
response 17.8 14.5 13.5 27.4
107 110 104 I06
OOriginal ns of I10 were reduced marginally be return letters indicating that the questionnaire was inapplicable as the office was that of a holding company and, therefore, would not be in a position to require a MKIS.
to response. The difference between the 17.8% of respondents who received the offer in the body of the covering letter, the 14.5% of respondents who received it as a typed postscript, and the 13.5% of respondents who received it as a handwritten postscript was not statistically significant at p < 0.05 (x2 = 0.83).
2. Watson, J. J., Improving Advertising
3
Erdos, P. L., Professional
4
Hansen, R. A., A Self-Perception Interpretation of the Effect of Monetary and Non-monetary Incentives on Mail Survey Respondent Behavior, Journul of Marketing Research 17, 77-83 (February 1980).
5
The practice of offering free survey results and asking respondents to identify themselves on the questionnaire in order to send them the report is not supported in this study. The results suggest that previous research which has used this method may have suffered from reduced response as a consequence. The likely explanation is that the loss of anonymity suppressed response for some recipients of the questionnaire. The results of this experiment confirmed those from earlier studies of industrial populations in that the use of a handwritten postscript did not increase response rates. This finding strengthens the viewpoint that methods that appear to increase response from household populations may not always be successful in industrial mail surveys.
Thompson, Association
M., Industrial Postal Surveys, April 1984.
Seminur.
New York, 1970.
Kimball, A. E., Increasing the Rate of Return in Mail Surveys, Journul of 25, 63-64 (December 1961).
6
Pressley, M. M., and Tullar, W. L., A Factor Interactive Investigation of Mail Survey Response Rates from a Commercial Population, Journcrl of Marketing Reseurch 14 10% I1 I (February 1977).
7
Tullar, W. L., Pressley, M. M., and Gentry, D. L., Toward a Theoretical Framework for Mail Survey Response, Proceedings of the Third Annual Conference of the Academy of Marketing Science (Miami, Floridu), 1979, pp. 243-249.
8 Higgins, J. C., and Finn, R., The Organization and Practice of Corporate Planning in the UK, Long Range Planning 10, 88-92 (August 1977). 9
Jobber, D., and Rainbow, C., A Study of the Development and Implementation of Marketing Information Systems in British Industry, Journul of the Murket Research Society 19(3), IO4- I I I (I 977).
10 Higgins, J. C., and Romano, D. J., Socio-Political Forecasting and Management of Information Systems, OrneRo 83, 303-309 (May 1980). Jobber, D., Hooley, G., and Sanderson, S. M., Marketing in a Hostile Environment: The British Textile Industry, Industricrl Marketing Munugement, forthcoming.
12. Higgins, J. C., and Opdebeeck, E. J., The Microcomputer as a Tool in Financial Planning and Control: Some Survey Results, Accounting und Business Research (Autumn 1984). 13 Futrell, C.. and Hize, R. T., The Effects of Anonymity and Same Day Deadline on the Response Rate to Mail Surveys, Europetin Research I7 I 173 (October 1982). 14. Pressley, M. M., Care Needed When Selecting Response Inducements Mail Surveys to Commercial Populations, Journul qf the Academy Murkefinx Science 6, 336-343 (Fall 1979).
in of
15 Jobber, D., and Sanderson, S. M., The Effects of a Prior Letter and Coloured Questionnaire Paper on Mail Survey Response Rates, Journrrl of the Market
Research Society 25(4),
339-349
(1983).
16 Frazier, G.. and Bird, K.. Increasing the Response of a Mail Questionnaire, Journal qf Marketing 23, l86- I87 (October 1958). 17 Hoppe, D. A., Certain Factors Found to Improve Mail Survey Returns, Proceedings of the Iowa Academy of Science 59, 374-376 (Summer 1952). 18. Childers, T. I., Pride, W. M.. and Ferrell. 0. C., A Reassessment of the Effects of Appeals on Response to Mail Surveys, Journal of Marketing Research 17, 365-370 (August 1980).
REFERENCES I.
Mail Surveys. McGraw-Hill,
Marketing
II
IMPLICATIONS
the Response Rate in Mail Research, Journul of (June 1965).
Research 5, 48-50
Industriul
Marker
Research
19. Feldt, L. S., and Mahmoud, M. W., Power Function Charts for Spccification of Sample Size, Psychometriku 23, 201-210 (1958).
121