An examination of the effects of questionnaire factors on response to an industrial mail survey

129

An examination of the effects of questionnaire factors on response to an industrial mail survey David JOBBER *

The importance of the mail questionnaire in gathering data from industrial populations and the inappropriateness of much of the research on the effects of questionnaire factors on response rates provided the impetus for this study. The results suggest that industrial mail surveys which do not employ a follow-up mailing should use double-sided printing. For those which do use a follow-up, no differences in response rates between 5- and g-page questionnaires, and single- and doublesided printing was found. The theoretical and practical implications of the findings are discussed.

1. Introduction The mail survey has proven to be a valuable method of collecting data from industrial populations. For example, recent marketing studies of advertising budgeting practices (Blasko and Patti, 1984), product elimination (Avlonitis, 1985), buyers’ perceptions of the customer orientation of industrial salespeople (Michaels and Day, 1985), salesforce socialization (Dubinsky, Howell, Ingram and Bellenger, 1986), macro-segmentation (Woodside and Wilson, 1986), power sources of an industrial machinery manufacturer (Gaski, 1986) and role stress among industrial buyers (Michaels, Day and Joachimsthaler, 1987) were all based upon data collected by mail questionnaire. However, care * David Jobber is a Lecturer in Marketing at the University of Bradford, Emm Lane, Bradford, West Yorkshire, United Kingdom BD9 4JL. Intern. J. of Research in Marketing 6 (1989) 129-140 North-Holland 0167-8116/89/$3.50

must be exercised in its use, for example the intricacies and subtle influences on an industrial buying process are unlikely to be captured because of its rigid structure. A mail questionnaire is most suited to surveys where the scheme of questions is not over-elaborate, and in which the questions- require straightforward and brief answers. When such conditions occur it can be extremely effective for no other survey method can compete in terms of cost for reaching widely-dispersed populations. Its main drawback however is a relatively low response rate compared to personal and telephone interviews (Yu and Cooper, 1983) and response rates from industrial populations are particularly low (Jobber and Saunders, 1986). Powerful motivators are likely to be the content of the questionnaire and its perceived usefulness. Furthermore, experimental evidence suggests that higher response can be expected by the use of monetary incentives (e.g., Furse and Stewart, 1982; Hansen, 1980; Heads and Thrift 1966; Kimball, 1961; Pressley and Tullar, 1977); stamped addressed return envelopes (e.g., Gullahorn and Gullahorn, 1965; Hammond, 1959; Hewett, 1974; Kimball, 1961; Watson, 1965), although Perry (1974) reported no improvement over franked mail; telephone notifications (e.g., Hornik, 1982; Jobber, Allen and Oakland, 1985; Roscoe, Lang and Sheth, 1975; Wiseman, 1972); and a follow-up to the initial mailing (e.g., Comer and Kelly, 1982; Ferrell and Krugman, 1983; Tullar, Pressley and Gentry, 1979; Watson, 1965). A further method - a prenotification letter -

0 1989, Elsevier Science Publishers B.V. (North-Holland)

.

130

D. Jobber / Effects of questionnaire factors on response lo industrial mail suruey

has consistently raised response rates in household surveys 1 (e.g., Ford, 1967; Heaton, 1965; Smith and Hewett, 1972; Walker and Burdick, 1977) but in the only test using an industrial population it was associated with a lower response rate (Jobber and Sanderson, 1983). Personalisation has been successful in raising response rates in industrial mail surveys (e.g., Kerin and Harvey, 1981; Thompson, 1984) but in a number of other industrial studies it has had no effect on response rate (e.g., Kimball, 1961; Simon, 1967; Watson, 1965). A common factor among these techniques is that they all increase survey costs. The objective of this paper is to explore a set of factors associated with the questionnaire itself which can be manipulated by the researcher at little or no extra cost and yet may influence response to the survey. Specifically two issues are examined: questionnaire length and printing the questionnaire on one side versus two sides of each sheet of paper. Both of these variables could affect response rates and yet have not been the subject of much rigorous experimental investigation. Furthermore, no rationale has been proffered to explain their effect using well-developed psychological theories of behaviour in previous studies. It is our intention to provide such a theoretical base and to examine experimentally the impact of these variables on response to an industrial mail survey.

2. Literature review

2. I. Questionnaire length Experimental research into the effects of questionnaire length on mail survey response rates has had a chequered history. Most studies upon close examination appear to be ’ Furthermore, an anonymous referee points out’that the use of a warning letter is well-accepted practice in the U.K. research industry.

methodologically flawed. The most persistent problem is the confounding effect of content on experimental results. For example, simply adding items to the ‘short’ version of the questionnaire in order to produce a ‘long’ one may introduce content bias if the additional questions heighten or lower the interest of the questionnaire to subjects. The results of the Cartwright (1986), Cartwright and Ward (1986), Clausen and Ford (1947), Hansen and Robinson (1980), Mason, Dressel and Bain (1961) Marks (1981), Powers and Alderman (1982) and Roscoe, Lang and Sheth (1975) studies may have been contaminated by this effect. To avoid this difficulty it is necessary to use two short questionnaires with different questions, and to compare them with a long questionnaire consisting of the two short ones put together. Scott (1961) employed this approach and found no difference between the mean response to the two short questionnaires and that for the long version. A limitation on his results however is that even the ‘long’ version was only two pages, and so it could be argued that negative returns to length had not had an opportunity to set in. Another experiment which attempted to overcome content bias (Berdie, 1973) found no significant differences between one-page, two-page and three-page treatments. However, this experiment suffered through loss of statistical power in the tests for differences due to inadequate sample size (n = 35 per treatment). A third experiment which used this approach was conducted by Sletto (1940) who compared responses to ten and twenty-five page questionnaires with a combined thirty-five page questionnaire. No difference in response rate between the shorter versions and the thirty-five page questionnaire was found. However a limitation of his research was that the results can only be generalized over the page lengths investigated. Sletto recognized that “it may be that the effect of length is more pronounced within the limits of one and ten pages”.

D. Jobber / Effecis of questionnaire factors on response to industrial mail suroey

Two surveys which did find significant differences between responses to short and long questionnaires (Brown, 1965; Stanton, 1939) also suffered from measurement error. In both cases the short one was in the form of a postcard while the long questionnaire was typed on paper. Clearly, the effect of questionnaire length on response was confounded by differing formats. A further quasi-experiment reported by Bartram and Flack (1973) indicated that in two travel surveys a questionnaire which required details of upto four journeys obtained a 77% response while one which asked for similar information for upto twenty journeys achieved only 40% response. However, since the surveys were conducted by different research agencies it is not certain that other variables which might affect response (eg question wording) were held constant. The study by Champion and Sear (1969) and a second experiment by Scott (1961) also addressed the relationship between questionnaire length and response rates but in both cases an identical number of questions was spread over varying page lengths. In both cases higher response rates were associated with the longer questionnaire. A further study (Blythe and Essex, 1981) measured the effects of reducing a sixteen page questionnaire to ten and six page booklets. Again higher response was associated with the longer questionnaire although the difference was not statistically significant. Although these studies were valuable in highlighting the importance of adequately spacing questions they throw no light on the issue of questionnaire variation by number of questions. Although the literature reviews by Linsky (1975) and Kanuk and Berensen (1975) concluded that the evidence does not support the view that questionnaire length affects response rates the analysis above shows that most of the experiments upon which this conclusion were based possessed methodological limitations which throw doubt upon the inter-

131

nal and external validity of the results. Perhaps a purer assessment is that the experimental evidence is inconclusive. A second approach to the assessment of questionnaire length on response is by means of regression analysis where response rate is the dependent variable and length is one of a number of candidate predictor variables. This approach has yielded fairly consistent results with studies by Goyder (1982), Heberlein and Baumgartner (1978) and Jobber and Saunders (1986) indicating marginal reductions in response as questionnaires become longer. 2.2. Single vs. double-sided printing of questionnaires Another questionnaire issue which has received little attention is the relative merits of single-sided and double-sided printing. Only three studies were identified which tested this issue - with conflicting results. A study by Hyett and Farr (1977) of telephone subscribers found that for a sixteen page questionnaire a higher response rate was found for the single-sided questionnaire. However, with a similar population Blythe and Essex (1981) found no significant difference in response to single and double-sided questionnaires. Moreover, a study of businessmen by Childers and Ferrell(l979) found a directional effect in favour of the doublesided questionnaire although the difference (6%) was not statistically significant.

3. The theoretical base and hypotheses The theoretical base of this study relies upon the temporal-cue notion within cuesearch theory which states that subjects are searching for cues to estimate time and that these cues do in fact influence behaviour ’ 2 This is not to deny that other factors such as personal interest in the survey and relevance to an individual’s job may also be important.

132

D. Jobber / Efjects of questionnaire factors on response to industrial mail survey

(Cottle, 1976; Hornik, 1981; Thomas and Weaver, 1975). Thus a salient cue to questionnaire completion time might be perceived questionnaire length. And as time perceptions of questionnaire completion time lengthen so might the negative evaluations of the outcome of completing the questionnaire. This is particularly so since mail surveys to businessmen are usually sent to their work address and completed during company time. Businessmen may be particularly sensitive to questionnaire length because of job-related demands on their time. Thus the longer a questionnaire is perceived to intrude into the subject’s workday the less likely (s)he is to complete it. It should also be recognised that the person who answers the questionnaire is not the only influence on response rate in the industrial situation. In a survey of marketing managers, Ferrell, Childers and Reukert (1984) found that two-thirds of the respondent sample had their mail screened at work. Clearly time perceptions of completion may influence this screening process with shorter questionnaires being more likely to pass through this filter. 3 Both the number of pages and printing format may influence perceived questionnaire length, and thus perceived completion time. Consequently, the following hypotheses are proposed: HA: A short questionnaire (number of pages) will be associated with a higher response (both rate and quality) than a long questionnaire. HB: A questionnaire which has double-sided printing will have a higher response than a questionnaire which has single-sided printing. 3 An exception to this may be when a very short questionnaire (i.e., one or two pages) may be perceived as trivial and therefore not passed on. Since the shortest version of the questionnaire in our experiment was five pages a small number of secretaries were asked whether they would consider a five-page questionnaire too trivial to pass onto their superordinate. All stated that they did not consider a fivepage questionnaire too trivial to pass on.

Furthermore, the impact of questionnaire page length on response may depend upon whether the form is single or double-sided. Thus: H,: There will be interaction effects between page length and printing format. The testing of these hypotheses will provide evidence to aid research designers when deciding questionnaire page length or printing format. If page length does not influence response, researchers can gather much more information about a phenomenon in each survey. Further, if printing format does affect response then this knowledge can be used by researchers to increase returns. The importance of these issues and the inappropriateness of much of the evidence regarding them provided the stimulus for this study.

4. Survey procedure and design Two variables were tested, each at two levels; these were the short/long questionnaire conditions and the one-sided/ two-sided conditions. A 22 factorial design was used in order to test the effects of these treatments and also their interaction (see Fig. 1). One short questionnaire contained questions relating to the types of selling approach and orientations used by the sample firms; the other questioned respondents about the criteria used to evaluate their salespeople. Both questionnaires contained an identical final page which asked a number of classificatory questions such as size of firm, industry category and number of salespersons employed. Version one contained thirty-five and version two forty-seven questions. Both questionnaires were five pages in length. The long version of the questionnaire was a combination of the substantive questions in the two short questionnaires plus the final page of classificatory questions. The’long questionnaire was therefore nine pages. By combining two short questionnaires to form the long

D. Jobber / Effects of questionnaire factors on response to industrial mail survey

version content and layout are, in effect, controlled. The five and nine page comparison was chosen for two reasons: (i) the range was fairly representative of page length in industrial mail surveys. A study by Jobber and Saunders (1986) revealed that out of twenty six studies for which data were available eleven cases (42%) were within the 5-9 page range and fifteen cases (58%) were within the 4-10 page range; (ii) the difference between the short and the long versions was assumed to be sufficient to allow negative returns to size to set in. Pilot testing with a small sample of sales managers indicated that both short questionnaires took approximately fifteen minutes to complete. The sample consisted of a stratified random sample of 600 industrial goods companies with industry type being used as the basis for stratification. The 1986 edition of Kompass - Register of U.K. Companies was used as a sampling frame. Three industries were chosen: chemicals, electrical engineering, and machinery and equipment. The two hundred firms in each stratum were randomly allocated to the four treatment combin a t i o n s - short questionnaire/ one-sided printing, short questionnaire/ two-sided printing, long questionnaire/ one-sided printing, and long questionnaire/ two-sided printing. 4 For the short questionnaires random allocation between the two versions was similarly performed. The sample size exceeded the m i n i m u m a s d e t e r m i n e d b y t h e Feldt-

4 The number of sheets associated with each of these treatments was 5, 3, 9, and 5 respectively. An interesting question concerns the perception of the two intermediate states short/one-sided questionnaire and long/double-sided questionnaire. Since both involve the use of an identical number of sheets (5) it is possible that perception of completion time could be similar. A test with ten sales managers who were asked to rank the four versions of the questionnaire in terms of perceived completion time demonstrated that in all cases the short/one-sided questionnaire was perceived as being quicker than the long double-sided questionnaire to complete. Thanks are given to an anonymous reviewer for this suggestion.

Short Single-sided Double-sided

133

Long

150150 150 150

Fig. 1. Experimental design. (a) There were two versions of the short questionnaire which when combined formed the long questionnaire. (b) This is a factorial design which allows the calculation of interaction. i.e., the situations where the response to changes in the levels of one treatment variable (e.g. page length) is dependent upon the level of some other treatment variable (e.g. printing format) in the experiment. If the interaction term is significant, ordinarily the calculation of main effects is superfluous, since the experimenter will customarily be interested in the best combination of variables (see Green and Tull. 1978. pp. 356-361).

Mahmoud (1958) method for sample size specification in experimental design. The envelopes were addressed to the Sales Manager and the questionnaire was accompanied by a covering letter which was identical for all treatments. It briefly explained the purpose of the survey, stressed that replies would be treated in the strictest confidence (Futrell, 1981) and thanked respondents for their cooperation. The covering letters were not personalised because the mixed evidence (cited earlier) suggested that the benefits of obtaining the names of the 600 sales managers were unlikely to exceed the costs. Large (33 X 23; cm) white envelopes were used for both out-going and return postage and a stamp was attached to the return envelope. A follow-up letter and questionnaire was sent to non-respondents approximately four weeks after the initial mailing when the initial response had virtually ceased. Although an interval of two weeks is favoured by many researchers we waited until the first-wave responses had ended because we wished to analyse results for both waves. To have sent the reminder earlier may have contaminated data collection for these distinct analyses. Once more an addressed, stamped return envelope was used to facilitate response and the covering letter was identical for all treatments. Two dependent variables were studied. A limitation of previous questionnaire research

D. Jobber / Effects of questionnaire factors on response to industrial mail survey

134

Table 1 Response rates by treatment combinations. Treatment combination

Overall 4%

Actual no.

Short/one-sided Short/double-sided Long/one-sided Long/double-sided

50.6 56.0 56.6 50.0

76 84 85 75

First wave response

response

Second wave response

(NJ

%

Actual no.

(N)

%

Actual no.

(150) (150) (150) (150)

37.3 45.3 32.0 42.6

56 68 48 64

(150) (150) (150) (150)

21.3 19.5 36.3 12.8

20 16 3-I 11

is that it has focused on only one criterion variable, namely response rate. But questionnaire manipulations can have effects on response quality also. It can be hypothesised that questionnaires which are perceived as being longer may suffer from a lower propensity for respondents to answer all of the questions (items). Thus the effects of treatments were measured in terms of both response rate and item omissions. 5 These variables are defined as follows:

(9

Response rate is defined as the percentage of total questionnaires mailed (and not returned by the postal service as undeliverable) that were returned by respondents (Wiseman and Billington, 1984). Item omission was defined as the per(ii) centage of fixed-response items unanswered per returned questionnaire (Houston and Ford, 1976). The results relating to response rate were analysed using the analysis of variance technique which allows for the testing of both main effects and the interaction effect between treatments. Despite the binomial nature of this dependent variable several researchers including Cochran (1950) and Hsu

’ It is acknowledged that this is not a complete measure of response quality since it does not capture the r$iability or validity of response. Furthermore, another response quality measure - responses to open-ended questions (Hansen, 1980) - was not applicable since all questions were closed.

(f”) (94)

(82) (102) (86)

and Feldt (1969) support its use given certain conditions. Provided that cell sizes exceed 50 and are equal between treatments, and that the probability of the occurrence of the event lies between 0.25 and 0.75, the analysis of variance technique may be applied to such data. All of these conditions applied in this experiment. Analysis of variance has been used in a number of mail survey studies in the way proposed in this experiment (e.g., Childers, Pride and Ferrell, 1980; Hornik, 1982; Houston and Nevin, 1977; Jones and Linda, 1978; Little and Pressley, 1980; Peterson, 1975; Pressley and Tullar, 1977). Analysis of item omissions was carried out using the normal approximation to the appropriate z transforms so that the standard test for significance of differences in proportions could be applied.

5. Results

5.1. Response rate An overall response rate of 53.5% was achieved which was quite high for an industrial mail survey (Hart, 1987). The response rates associated with the treatments applied in this experiment are given in Table 1. The influence of each of these treatments on response rate is now ascertained by analy-

D. Jobber / Effects of questionnaire factors on response to industrial mail

suroey

135

Table 2 Overall response - ANOVA

results.

Source of variation

Sum of squares

d.f.

Mean

F

Significance of F

Main effects Short vs. long Single vs. double

0.005 0.000 0.004

2 1 1

0.002 0.000 0.004

0.010 0.002 0.017

0.991 0.965 0.896

2-way interaction Length x sides

0.400

1

0.400

1.600

0.206

Table 3 First wave response rates - ANOVA

square

results.

Source of variation

Sum of squares

d.f.

Mean

F

Significance of F

Main effects Short vs. long Single vs. double

1.424 0.223 1.199

2 1 1

0.712 0.223 1.199

2.992 0.938 5.040

0.051 0.333 0.025 a

2-way interaction Length X sides

0.034

1

0.034

0.142

0.707

square

a Significant at p < 0.05.

sis of overall response and response to first and second wave mailings. 5.1.1. Overall response rate

In order to test HA, HB and H,, that overall response is not associated with (i) questionnaire length, (ii) whether the questionnaire was single or double-sided, and (iii) interaction between these conditions, an analysis of variance on rate of response to these treatments was performed. The results are displayed in Table 2. Table 1 indicates that the overall effects on response rate of questionnaire page length 6 6 There was no significant difference between response rates to the two short versions of the questionnaire.

Table 4 Second wave response rates - ANOVA

and one-sided vs. double-sided printing were insignificant. The analysis of variance results in Table 2 confirmed that this was the case. ’ The response to a long (or short) questionnaire might depend on whether it is single or double-sided. Similarly the response to a single-sided (or double-sided) questionnaire might depend on the length of the questionnaire. Indeed inspection of Table 1 suggests that such interaction effects may be present with short/ double-sided and long/one-sided ’ It is interesting to note that the treatments with the least (3) and most (9) number of sheets achieved the higher absolute response rates (56.0 and 56.3%) compared to those with the medium (5) number of sheets (50.6 and 50.0%). However these differences were not statistically significant.

results. F

Significance of F

0.815 0.309 1.317

4.664 1.766 7.533

0.010 0.185 0.006 ’

0.830

4.746

0.030 a

Source. of variation

Sum of squares

d.f.

Mean

Main effects Short vs. long Single vs. double

1.631 0.309 1.317

2 1 1

2-way interaction Length X sides

0.830

1

a Significant at p < 0.05.

square

136

D. Jobber / Effects of questionnaire factors on response to industrial mail survey

treatments recording response rates at 56 and 56.6% while short/one-sided and long/double-sided conditions producing only 50.6 and 50.0% response respectively. However, the analysis of variance results in Table 2 demonstrate that these differences were not statistically significant. 5.1.2. Responses to first and second wave questionnaires (i) First wave responses. Further analyses were performed to assess whether the nonsignificant results at the overall response rate level masked differences which could have occurred at each of the two mailing stages. Once more analysis of variance was used to identify significant effects and the results are given in Tables 3 and 4. The difference in response rates between the short and the long questionnaires was insignificant for the first mailing. In percentage terms response rates were 41.3% (short) and 37.3% (long). However, the difference in response rates between the single-sided (34.7%) and the double-sided (44.0%) questionnaires was statistically significant ( p < 0.05). 8 Interaction effects were not significant. Thus returns to the first mailing indicated that double-sided questionnaires could be expected to result in higher response rates for both long (9-page) and short (5-page) versions of the questionnaire. (ii) Second wave responses. Analysis of the responses to the follow-up questionnaire which was sent to non-respondents to the first wave showed that there was no significant difference between the response rates to the short (20.4%) and the long (25.5%) questionnaires (see Table 4). However a significant difference did occur between the single-sided (29.1%) and double-sided (16.1%) questionnaires. And so, whereas at the first mailing

Table 5 Item omission rates. Treatment

combination

Short/one-sided Short/double-sided Long/one-sided Long/double-sided

Mean per questionnaire

Percentage of items a

0.75 0.87 8.27 0.45

0.68 0.79 3.92 0.21

a There were more items than questions since some questions required more than one response.

stage the double-sided questionnaire had resulted in a significantly higher response rate, at the follow-up stage this result was reversed with the single-sided questionnaire being more effective. Moreover, when the interaction effect between questionnaire length and single vs. double-sided printing was examined a significant difference was found. An inspection of Table 1 reveals that this is attributed to the much higher response rate (36.3%) associated with the long/one-sided condition. This is an interesting finding given that this treatment combination recorded the lowest absolute returns to the first mailing. 5.2. Item omission For all respondents the average number of items omitted per questionnaire was 2.6% or 1.4% of the items. Table 5 shows the distribution of item omissions over the four treatment combinations. Although the long/single-sided treatment was associated with the highest absolute omission rate a series of z-tests of differences of proportions showed no significant differences between all pairs of treatment combinations. Clearly, neither questionnaire length nor printing format affected response quality as measured by item omission.

6. Discussion and conclusions * Note that although the short, double-sided format achieved a higher absolute response (45.3%) than the long, do;ble-sided format (42.6%) the difference was not statistically significant).

I

With the popularity of the industrial mail questionnaire among academic marketing re-

D. Jobber / Effects of questionnaire factors on response to industrial mail suruey

searchers being so high it is important that as much as possible is known about the design issues that can raise or depress response. This paper has examined two variables which could be expected to have significant effects on response. Questionnaire page length and single vs. double-sided printing could influence perceptions of the time it takes to answer a questionnaire. Since businessmen could be expected to be sensitive to time perceptions response may be affected by these factors and their interaction. Moreover assessment of their effects is important since changing page length and printing has only marginal effects on costs. The hypothesis that a questionnaire with double-sided printing would have a higher response than one with single-sided printing was not supported by the overall response to the survey but was supported at the first wave stage. This suggests that mail surveys which do not include a follow-up should use double-sided printing. In total, this produced a response increase of almost ten percentage points; for the short questionnaire the increase was eight percentage points (45.3% for the double-sided versus 37.3% for the singlesided questionnaire) and for the long version the increase was even higher at over ten percentage points (42.6 versus 32.0%). Thus the impact of the double-sided printing is to raise response rates for both short (5-page) and long (9-page) questionnaires targeted at businessmen. An interpretation of this result is that double-sided printing makes the questionnaire appear shorter and thus less time-consuming than single-sided printing. The hypothesis that a short questionnaire (number of pages) would be associated with a higher response than a long questionnaire was not supported by the results of this experiment. Although directional impact of the page length was also in favour of the short version (4 percentage points) at the first wave stage, the difference was not statistically significant. However, another salient difference to analyse

137

was that between the two extreme versions of the questionnaires, that is between the short/ double-sided questionnaire and the long/ single-sided versions. It is reasonable to assume that the former would appear the quickest to complete while the latter may be perceived as requiring the longest time out of the four treatment combinations. In the event the short/ double-sided form obtained the highest response rate (45.3%) at the first mailing stage while the long/single-sided version recorded the lowest (32.0%). The difference (13.3 percentage points) was statistically significant ( p < 0.05). One interpretation of this result is that time perception effects on response were active for the extreme versions of the questionnaire. The hypothesis that there would be interaction effects between page length and printing format was supported by the returns to the second-stage mailing. An analysis of secondstage returns showed a significant difference in response in favour of the single-sided (versus double-sided) printing. However, interaction analysis proved significant and inspection of Table 1 revealed the cause: the positive impact of single-sided printing was due almost entirely to the long/single-sided treatment. This treatment recorded response of 36.3% at the follow-up stage which was fifteen percentage points higher than its nearest alternative (short/single-sided). The impact of the follow-up was greatest on those treatments which had fared worst at the initial mailing stage. This then is the reason for the non-significant results to the survey overall. It appears that the act of sending a followup letter and questionnaire negates the time-perception effects which were detectable at the initial mailing stage. An explanation of this is possible. It is plausible that some receipients of the long version of the questionnaire at the first mailing stage would have completed it except that the opportunity cost was too high because the questionnaire looked

138

D. Jobber / Effects of questionnaire factors on response to industrial mail survey

lengthy (unfavourable time perception) and pressure of other work was great. However, by the time the follow-up questionnaire arrived (four weeks later) the opportunity cost may have fallen because of reduced job demands. Thus for some subjects time perception sensitivity to the long questionnaire may have fallen between mailings and it was these extra respondents to the long questionnaire (9-page/single-sided) who brought about the balancing effect of the follow-up. The previous discussion of the situation facing businessmen when they receive mail questionnaires suggested that time perception would be an important influence on response. The results of this study partially support this contention. Furthermore the study provides evidence that these effects can be attenuated by means of a follow-up. From a practical perspective the follow-up mailing had significant results: response rates increased by an average of 9.1 percentage points for the double-sided questionnaires and 19.0 percentage points for the single-sided versions. This had the effect of enhancing the representativeness of the sample and increasing the power of the statistical tests performed on the substantive results of the survey (Co&ran, 1977). However, if only one mailing is employed (e.g., because of resource constraints) our results support the use of double-sided printing. This result is congruent with the Childers and Ferrell (1979) study of businessmen in a directional sense although their result was not statistically significant. Another practical implication from this research is that researchers can use the industrial mail survey to gather a wide range of information rather than restricting themselves to a short questionnaire. 9 These findings extend those of Scott (1961) who, using a similar methodology, found no difference in response 9 A limitation to this implication is likely to bl where the additional questions are of less interest to the respondent and/or involve more sensitive information.

rates to a one versus two-page questionnaire and Sletto (1940) who found no difference in response to ten, twenty-five and a combined thirty-five page questionnaire. An interesting question for further research is the effect of one page versus, for instance, five pages; it could be hypothesised that the longer questionnaire might produce higher returns if the one-page questionnaire were perceived as trivial. A final implication of this research is that future experimental studies should be analysed at both the initial and follow-up mailing stages as well as at the aggregate level. This is rarely done; yet in this study a greater understanding of response behaviour was gained by these additional analyses. This study is only the third to investigate questionnaire design factors which could affect response to an industrial mail survey. This is an important research domain since mail questionnaire instruments are often used to elicit information from businessmen. Much more knowledge is required so that academics can be confident that their research design will produce acceptable response in an age when researchers are placing increasing demands on businessmen’s time. It is hoped that this study will stimulate more theorybased research which provides a better understanding of the influences on behaviour and a heightened awareness of how to use the industrial mail questionnaire effectively.

References Avlonitis, G.J., 1984. The techno-economic ecology of the product elimination process. International Journal of Research in Marketing 2, no. 3, 175-184. Berdie, D.R., 1973. Questionnaire length and response rate. Journal of Applied Psychology 58, no. 2, 278-280. Bartram, P. and M. Flack, 1973. What can be done: A review of the development of self completion survey techniques. Market Research Society Annual Conference, Brighton, 245-268. Blasko, V.J. and C.H. Patti, 1984. The advertising budgetary

D. Jobber / Effects of questionnaire factors on response to industrial mail practices of industrial marketers. Journal of Marketing 48, no. 4, 104-110. Blythe, 1. and P. Essex, 1981. Variations on a postal theme. Market Research Society Annual Conference, Brighton, 35-51. Brown, K.L., 1965. Use of a postcard query in mail surveys. Public Opinion Quarterly 29 (Winter), 635-637. Cartwright, A., 1986. Some experiments with factors that might affect the response of mothers to a postal questionnaire. Statistics in Medicine 5, 607-617. Cartwright, A. and A.W.M. Ward, 1968. Variations in general practitioner’s response to postal questionnaires. British Journal of Preventative and Social Medicine 22, 199-205. Champion, D.J. and A.M. Sear, 1969. Questionnaire response rate: A methodological analysis. Social Faces 47, 335-339. Childers, T.L. and O.C. Ferrell, 1979. Response rates and perceived questionnaire length in mail surveys. Journal of Marketing Research 16, 429-431. Childers, T.L., W.M. Pride and O.C. Ferrell, 1980. A reassessment of the effects of appeals on response to mail surveys. Journal of Marketing Research 17, 365-370. Clausen, J.A. and R.N. Ford, 1947. Controlling bias in mail questionnaires. Journal of the American Statistical Association 42, 497-511. Cochran, W.G., 1950. The comparison of percentages in matched samples. Biometrika 37, 256-266. Cochran, W.G., 1977. Sampling techniques, 3rd ed. New York: Wiley. Comer, J.M. and J.S. Kelly, 1982. Follow-up techniques: The effect of method and source appeal. Proceedings of the American Marketing Association Educators’ Conference, 430-434. Cottle, T.J., 1976. Perceiving time: a psychological investigation with men and women. New York: Wiley. Dubinsky, A.J., R.D. Howell, T.N. Ingram and D.N. Bellenger, 1986. Salesforce socialization. Journal of Marketing 50, 192-207. Feldt, L.S. and M.W. Mahmoud, 1985. Power function charts for specification of sample size. Psychometrika 23,201-210. Ferrell, O.C., T.L. Childers and R.W. Reukert, 1984. Effects of situational factors on mail survey response. Proceedings of the American Marketing Association Educators’ Conference, 364-367. Ferrell, O.C. and D. Krugman, 1983. Response patterns and the importance of the follow-up duplicate questionnaire in a mail survey of advertising managers. European Research 11 (October), 157-163. Ford, N.M., 1967. The advance letter in mail surveys. Journal of Marketing Research 4, no. 2, 202-204. Furse, D.H. and D.W. Stewart, 1982. Monetary incentives versus promised contribution to charity: New evidence on mail survey response. Journal of Marketing Research 19, 375-380. Futrell, C., 1981. The effects of using both confidentiality and anonymity in survey research of the sales force. Proceedings of the Americam Marketing Association Educators’ Conference 47, 391-393. Gaski, J.F., 1986. Interrelations among a channel entity’s power sources: Impact of the exercise of reward and coercion on

suroey

139

expert, referent and legitimate power sources. Journal of Marketing Research, 23, 62-77. Goyder, J., 1982. Further evidence on factors affecting response rates to mailed questionnaires, American Sociological Review 47, 550-553. Green, P.E. and D.S. Tull, 1978. Research for marketing decisions. Englewood Cliffs, NJ: Prentice-Hall. Gullahorn, J. and J. Gullahom, 1965. An investigation of the effects of three factors on response to mail questionnaires. Public Opinion Quarterly 27, 294-296. Hammond, E.C., 1959. Isolation in relation to type and amount of smoking. American Statistical Association Journal, 54, 39-51. Hansen, R.A., 1980. A self-perception theory of the effect of monetary and non-monetary incentives on mail survey respondent behaviour. Journal of Marketing Research, 17-83. Hansen, R.A. and L.M. Robinson, 1980. Testing the effectiveness of alternative foot-in-the door manipulations. Journal of Marketing Research 17, 359-364. Hart, S., 1987. The use of the survey industrial marketing research. Journal of Marketing Management 3, no. 1.25-38. Heads, J. and H.J. Thrift, 1966. Notes on a study in postal response rates. Journal of the Market Research Society 8, no. 4, 257-262. Heaton, E.E., 7965. Increasing mail questionnaire returns with a preliminary letter. Journal of Advertising Research 5, 35-39. Heberlein, T. and R. Baumgartner, 1978. Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. American Sociological Review 43,,447-462. Hewett, W.C., 1974. How different combinations of postage on out-going and return envelopes affect questionnaire return. Journal of the Market Research Society 16, 49-50. Homik, J., 1981. Time cue and time perception effect on response to mail surveys. Journal of Marketing Research 18, 243-248. Homik, J., 1982. Impact of pre-call request form and gender interaction on response to a mail survey. Journal of Marketing Research 19, 144-151. Houston, M.J. and N.W. Ford, 1976. Broadening the scope of methodological research in mail surveys. Journal of Marketing Research 13, 397-403. Houston, M.J. and J.R. Nevin, 1977. The effect of source and appeal on mail survey response patterns. Journal of Marketing Research 14, 374-378. Hsu, T.C. and L.S. Feldt, 1969. The effect of limitations on the number of criterion scores on the significance level of the F-test. American Educations Research Journal 6, 515-527. Hyett, G.P. and D.L. Farr, 1977. Postal questionnaires: Double-sided printing compared with single-sided printing. European Research 5, 136-37. Jobber, D., N. Allen and J. Oakland, 1985. The impact of telephone notification strategies on response to an industrial mail survey. International Journal of Research in Marketing 2, 291-296. Jobber, D. and J. Saunders, 1986. The specification and estimation of a robust mail survey response model. Proceed-

140

D. Jobber / Effects of questionnaire factors on response to industrial mail survey

ings of the XVth Annual Conference of the European Marketing Academy, 865-880. Jobber, D. and SM. Sanderson, 1983. The effects of a prior letter and coloured questionnaires on mail survey response rates. Journal of the Market Research Society 25, 339-349. Jones, W.H. and G. Linda, 1978. Multiple criteria effects in a mail survey experiment. Journal of Marketing Research 15, 280-284. Kanuk, L. and C. Berenson, 1975. Mail surveys and response rates: A literature review. Journal of Marketing Research 12, 440-453. Kerin, R.A. and M.G. Harvey, 1976. Methodological considerations in corporate mail surveys: A research note. Journal of Business Research 4, no. 3, 277-281. Kimball, A.E., 1961. Increasing the rate of return in mail surveys. Journal of Marketing 25, 63-64. Linsky, A.S., 1975. Stimulating responses to mailed questionnaires. Public Opinion Quarterly 39, 82-101. Little, T.E. and M.M. Pressley, 1980. A multi-factor experiment on the generalizability of direct mail advertising response techniques to mail survey design. Journal of the Academy of Marketing Science 4, 390-404. Marks, R.B., 1981. A factorial experiment in stimulating response to mail surveys. Proceedings of the American Marketing Association Educators’ Conference, 398-400. Mason, W.S., R.J. Dressel and R.K. Bain, 1961. An experimental study of factors affecting response to a mail survey of beginning teachers. Public Opinion Quarterly 25, 296-299. Michaels, R.E. and R.L. Day, 1985. Measuring customer orientation of salespeople: A replication with industrial buyers. Journal of Marketing Research 22, 443-446. Michael& R.E., R.L. Day and E.A. Joachimsthaler, 1987. Role stress in industrial buyers: An integrative model. Journal of Marketing, 51, 28-45. Perry, N., 1974. Postage combinations in postal questionnaire surveys - another view. Journal of the Market Research Society 16, 245-246. Peterson, R.A., 1975. An experiment investigation of mail survey responses. Journal of Business Research 3, 199-209. Pressley, M.M. and W. Tullar, 1977. A factor interactive investigation of mail survey response rates from a commercial population. Journal of Marketing Research 41,108-112. Powers, D.E. and D.L. Alderman, 1982. Feedback as an incen-

tive for responding to a mail questionnaire. Research in Higher Education 17, 207-211. Roscoe, A.M., D. Lang and J.N. Sheth, 1975. Follow-up methods, questionnaire length, and market-differences in mail surveys. Journal of Marketing 39, 20-27. Scott, C., 1961. Research on mail surveys. Journal of the Royal Statistical Association 124. 143-192. Simon, R., 1967. Responses to personal and form letters in mail surveys. Journal of Advertising Research 7, 28-30. Sletto, R.F., 1940. Pretesting of questionnaires. American Sociological Review 5, 193-200. Smith, E.N. and W. Hewett, 1972. The value of a preliminary letter in postal survey response. Journal of the Market Research Society 14, 145-151. Stanton, F.. 1939. Notes on the validity of mail questionnaire returns. Journal of Applied Psychology 23, 95-104. Thomas, E.A. and W.B. Weaver, 1975. Cognitive processing and time perception. Perception and Psychophysics 17, 363-367. Thompson, M., 1984. Industrial postal surveys. Industrial Market Research Association Seminar, April. Tullar, W.L., M.M. Pressley and D.L. Gentry, 1979. Toward a theoretical framework for mail survey response. Proceedings of the 3rd Annual Conference of the Academy of Marketing Science, 243-249. Walker, B.J. and R.K. Burdick, 1977. Advance correspondence and error in mail surveys. Journal of Marketing Research 14, 379-382. Watson, J.J., 1965. Improving the response rate in mail research. Journal of Advertising Research 5, 48-50. Wiseman, F., 1972. Factor interaction effects in mail survey response rates. Journal of Marketing Research 10, 330-333. Wiseman, F. and M. Billington, 1984. Comment on a standard definition of response rates. Journal of Marketing Research 21, 336-338. Woodside, A.G. and E.J. Wilson, 1986. Combing macro and micro industrial market segmentation. In: A.G. Woodside (ed.), Advances in business marketing Vol. 1. Greenwich: JAI Press, 241-256. Yu, J. and M. Cooper, 1983. A quantitative review of research design effects on response rates to questionnaires. Journal of Marketing Research 20, 36-44.