- Email: [email protected]
Survey Non-response in the Netherlands
Survey Non-response in the Netherlands: Effects on Prevalence Estimates and Associations A. JEANNE M. VAN LOON, PHD, MARJA TIJHUIS, PHD, H. SUSAN J. PICAVET, PHD, PAUL G. SURTEES, PHD, AND JOHAN ORMEL, PHD
PURPOSE: Differences in respondent characteristics may lead to bias in prevalence estimates and bias in associations. Both forms of non-response bias are investigated in a study on psychosocial factors and cancer risk, which is a sub-study of a large-scale monitoring survey in the Netherlands. METHODS: Respondents of a cross-sectional monitoring project (MORGEN; N 22,769) were also asked to participate in a prospective study on psychosocial factors and cancer risk (HLEQ; N 12,097). To investigate diverse aspects of non-response in the HLEQ on prevalence estimates and associations are studied, based on information gathered in the MORGEN-project. RESULTS: A response percentage of 45% was obtained in the MORGEN-project. Response rates were found to be lower among men and younger people. The HLEQ showed a response percentage of 56%, and respondents reported higher socioeconomic status, better subjective health and healthier lifestyle behaviors than non-respondents. However, associations between smoking status and either socioeconomic status or subjective health based on respondents only were not statistically different from those based on the entire MORGEN-population. CONCLUSION: Non-response leads to bias in prevalence estimates of current smoking, current alcohol intake, and low physical activity or poor subjective health. However, non-response did not cause bias in the examined associations. Ann Epidemiol 2003;13:105–110. © 2003 Elsevier Science Inc. KEY WORDS:
Bias, Health, Lifestyle, Non-response, Socio-demographic.
INTRODUCTION Non-response bias is a major concern for studies based entirely upon data collected through mailed questionnaires. In general, response rates vary by socioeconomic status (1– 4), by gender (5–7), by age (4, 8, 9) and by marital status (3, 10). In addition, respondents reported more often better health status and more positive health-related behaviors than non-respondents did (1, 2, 5–7, 11–13). These differences in respondent characteristics may lead to bias in prevalence estimates and bias in associations (2, 14). However, in most studies little information is available from non-respondents. Consequently, it is usually impossible to quantify response bias.
In an ongoing large-scale prospective study examining psychosocial factors, lifestyle and cancer risk, using the Health and Life Experiences Questionnaire (HLEQ), the sampling frame consisted of participants in the monitoring project on chronic disease risk factors (MORGEN-project). Therefore, it was possible to compare respondents and nonrespondents on the HLEQ, using data collected in the MORGEN-project, to quantify possible response bias in prevalence estimation and establish whether non-response contributed to bias in associations.
METHODS Study Population
From the National Institute of Public Health and the Environment, Bilthoven, The Netherlands (A.J.M.L., M.T., H.S.J.P.); MRC Biostatistics Unit, Institute of Public Health, University of Cambridge, Great Britain (P.G.S.); and Department of Psychiatry, University of Groningen, The Netherlands (J.O.). Address correspondences to: A.J.M. Van Loon, National Institute of Public Health and the Environment, P.O. Box 1, 3720 BA Bilthoven, The Netherlands. Tel.: 31 30 2743210; Fax: 31 30 2744407. E-mail: [email protected] Received 12 February 2001; revised 28 November 2001; accepted 13 December 2001. © 2003 Elsevier Science Inc. All rights reserved. 360 Park Avenue South, New York, NY 10010
Between 1993 and 1997, in the MORGEN-project, data on health status and the prevalence of risk factors were collected in a random sample of the general population, aged 20–59 years, drawn from three towns in the Netherlands. In Amsterdam and Maastricht each year a new sample was examined and in Doetinchem, participants of an earlier study (1987 to 1991) were re-examined. A total of 50,766 persons received an invitation to participate in the MORGEN-project. Those who returned a reply card had received two questionnaires by mail (a general question1047-2797/03/$–see front matter PII S1047-2797(02)00257-0
106
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
AEP Vol. 13, No. 2 February 2003: 105–110
naire on socio-demographic factors, lifestyle and health indicators and a food frequency questionnaire). They were then asked to visit the local Public Health Service for a medical examination. All participants in the MORGENproject who gave informed consent for further research (94%) received a HLEQ, measuring psychosocial factors like life events and personality characteristics (15).
tivity and diet), and health aspects were used. Dietary aspects were measured with a food frequency questionnaire (16). Health aspects under study were self-rated subjective health, body mass index, physical functioning and mental health (measured with the RAND-36) (17), neuroticism (18), and positive and negative social experiences (19). When quantifying possible response bias in prevalence estimation, particular attention was given to smoking habits and alcohol intake. Relationships between subjective health or socioeconomic status and smoking were studied to check whether non-response would lead to bias in associations.
Independent Variables
Statistical Methods
When comparing respondents with non-respondents in the MORGEN-project, attention was given to gender, age and place of residence. When comparing respondents to the HLEQ with non-respondents, information available from the MORGEN-questionnaire concerning socio-demographic characteristics (sex, age, marital status, socioeconomic status, place of residence), lifestyle risk factors for chronic disease (smoking, alcohol intake, low physical ac-
Available information on the chosen independent variables was described for both respondents and non-respondents. Statistical significance was assessed by chi-square tests for categorical variables and by ANOVA (F-test) for continuous variables (p 0.05). To quantify possible response bias in prevalence estimation, weighted prevalence figures were calculated. To check whether non-response would lead to bias in associations, multivariate analyses
Selected Abbreviations and Acronyms HLEQ Health and Life Experiences Questionnaire
FIGURE 1. Flow diagram (non-) response in MORGEN and HLEQ.
AEP Vol. 13, No. 2 February 2002: 105–110
were conducted, using MORGEN baseline data. Response status on the HLEQ (yes or no) was included as independent variable in the analysis and effect modification by response status was statistically tested by means of maximum likelihood (p 0.10). All statistical analyses were completed using SAS statistical software (version 6.12) (SAS institute Inc., Cary, North Carolina, USA).
RESULTS In total, 50,766 people were approached for participation in the MORGEN-project. Finally, 22,769 persons (45%) completed the total survey including questionnaires and a medical examination (Fig. 1). Overall, response percentages were higher among women (49%) than men (41%) and were higher among older persons (30% among people aged 20–29 and 54% among people aged 50–65). Response percentages were highest in Doetinchem (68%), intermediate in Maastricht (45%) and lowest in Amsterdam (34%). A decline in response percentages was seen throughout the years, from 48% in 1993 until 40% in 1997. For people who did not participate in the MORGENproject, but who returned the reply card, or who participated in a non-response survey, the main reasons for refusal were “no time” (35%), “already have a medical check-up on a regular basis” (25%), “I am healthy, there is no reason to participate” (16%) and “no interest” (15%). For about 30% of the sample, we had no information except the sample frame characteristics (sex, age, and place of residence). A comparison of the characteristics of respondents with those who gave only limited information (for the period 1995 to 1997) showed that there were only minor differences in level of education or the proportion of current smokers (Table 1). The proportion of current alcohol drinkers and the proportion practicing sports were found higher among respondents as compared with those who gave only limited information. A total of 21,426 persons were invited to complete the HLEQ (Fig. 1). Of these, 10,050 persons returned the questionnaire after the first mailing and 2,047 after the second mailing yielding a response percentage of 56%. Response percentages were only slightly lower in Amsterdam (54%), compared with Doetinchem (58%) and Maastricht (57%). A comparison of HLEQ respondent characteristics with those of non-respondents was completed, based on data from the MORGEN-project. Response percentages were comparable between men (55%) and women (57%) with no age differences between respondents and non-respondents (Table 2). A higher percentage of male respondents were married and a lower percentage of all respondents, were either low SES, or had no paid job. Current smoking was less prevalent among respondents, current alcohol intake was more prevalent among respondents and low physi-
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
107
TABLE 1. Characteristics of respondents and non-respondents with incomplete information in The MORGENa-project: 1995 to 1997 Complete response N13,490 N Sex: male Mean age City: Amsterdam Doetinchem Maastricht Level of education % low % high % current smoker (cig) % current drinker % playing sports a b
6072 41.9 4490 4292 4708 6196 3437 4846 11,413 6348
Incomplete responseb N 7221 %
N
% 48.0
33.3 31.8 34.9
3463 38.9 3338 1004 2879
46.3 25.7 36.0 85.1 47.2
3140 1584 2305 5204 2952
48.8 24.6 37.1 75.5 43.2
45.0
46.2 13.9 39.9
Monitoring Project on Chronic Disease Risk Factors. Only general questionnaire or reply card or non-response survey by telephone.
cal activity was found less often among respondents. The intake of energy percentage of fat between respondents and non-respondents was rather small, whereas the intake of vitamin C was slightly higher among respondents. Finally, among respondents a smaller proportion reported poor subjective health and fewer respondents had a body mass index above 30 than non-respondents. Only small differences in measures of physical and mental functioning were found between respondents and non-respondents. To assess whether the non-response rate to the HLEQ (of about 44%) led to bias in the prevalence of lifestyle risk factors, weighted figures for lifestyle behaviors and health aspects were determined, assuming only indirect selective non-response (Table 2). Figures were weighted for socioeconomic status (low, medium, high), marital status (married vs. not married), occupational status (paid job vs. no paid job) and place of residence (Amsterdam, Doetinchem, Maastricht). Distributions of these characteristics showed the greatest dissimilarities between respondents and nonrespondents. No substantial differences were found between non-weighted and weighted prevalence estimations. For example, the prevalence of current smoking among respondents changed from 33% to 32% after weighting and from 40% to 39% among non-respondents. The greatest effect of weighting was found for vitamin C intake among nonrespondents. The mean intake in milligram per day changed from 105 mg to 109 mg after weighting. Finally, a check was made as to whether non-response led to bias in the association between subjective health and smoking or the association between SES and smoking. The analyses revealed no essential differences in those associations based upon the total MORGEN-population and those based only upon respondents to the HLEQ (Table 3).
108
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
AEP Vol. 13, No. 2 February 2003: 105–110
TABLE 2. Comparison of non-respondents and respondents to the HLEQ a, based on MORGENb-baseline characteristics (1993 to 1997)c N Sociodemographic Sex: male (%) Mean age (years) Not married (%) Low SES (%) No paid job (%) Place of residence (%) Amsterdam Doetinchem Maastricht Lifestyle behaviors: unweighted Current smoking (%) # packyears (mean) Current alcohol intake # glasses p.d. (mean) Low physical activity (%) Energy % fat Vitamin C intake (mg/day) Lifestyle behaviors: weighted Current smoking (%) # packyears (mean) Current alcohol intake # glasses p.d. (mean) Low physical activity (%) Energy % fat Vitamin C intake (mg/day) Health aspects: unweighted Bad subjective health (%) 1993 to 1994 1995 to 1997 Body mass index (%) 18.5 30 Physical functioningd (mean) Mental well-beingd (mean) Neuroticisme (mean) Positive social exp. (mean) Negative social exp. (mean) Health aspects: weighted Bad subjective health (%) 1993 to 1994 1995 to 1997 Body mass index (%) 18.5 30 Physical functioningd (mean) Mental well–beingd (mean) Neuroticisme (mean) Positive social exp. (mean) Negative social exp. (mean) a
21,426 21,426 21,338 21,321 20,563 21,426
21,375 7649 21,339 13,148 21,426 21,364 21,364
8812 12,537 21,442
12,485 12,471 8536 20,292 20,333
Non-respondents (9329)
Respondents (12,097)
Difference [95% CI]
46.3 42.4 36.6 56.2 40.8
44.5 42.2 34.3 42.7 33.0
1.8 [0.5–3.1] 0.2 [0.02–0.4] 2.3 [1.3–3.3] 13.5 [12.2–14.8] 7.8 [6.5–9.1]
32.9 31.8 35.3
29.6 34.0 36.4
3.3 [1.1–5.5] 2.2 [0.0–4.4] 1.1 [3.3–1.1]
40.1 18.3 82.4 2.0 49.1 35.1 104.9
33.1 16.8 88.6 1.9 43.6 34.9 109.0
7.0 [5.7–8.3] 1.5 [0.1–2.0] 6.2 [7.2–5.2] 0.1 [0.06–0.14] 5.5 [4.2–6.8] 0.2 [0.1–0.3] 4.1 [5.4–2.8]
39.3 17.9 83.5 1.9 48.3 34.9 108.5
33.7 17.2 88.1 1.9 43.8 34.9 107.4
5.6 [4.3–6.9] 0.7 [0.2–1.3] 4.6 [5.6– 3.6] 0 [0.1–0.1] 4.5 [3.2-5.8] 0 [0.1–0.1] 1.1 [0.1–2.3]
9.8 18.9
5.2 12.0
1.8 12.1 86.6 71.8 35.1 22.9 13.4
1.4 8.8 89.2 74.5 36.3 23.4 13.2
8.5 17.1
5.4 12.3
1.7 11.4 87.5 72.3 35.3 23.0 13.3
1.3 9.4 88.8 74.4 36.2 23.3 13.2
4.6 [3.5–5.7] 6.9 [5.6–8.2] 0.4 [2.3–3.1] 3.3 [0.8–5.8] 2.6 [3.1– 2.1] 2.7 [3.2– 2.2] 1.2 [1.4– 1.0] 0.5 [0.6– 0.4] 0.2 [0.1–0.3]
3.1 [2.0–4.2] 4.8 [3.5–6.1] 0.4 [0.2–3.0] 2.0 [0.5–4.5] 1.3 [1.8– 0.8] 2.1 [–2.7–1.5] 0.9 [1.1–0.7] 0.3 [0.4–0.2] 0.1 [0.04–0.2]
Health and Life Experiences Questionnaire. Monitoring project on chronic disease risk factors. c Statistical significance was assessed by chi-square tests for categorical variables and by analysis of variance (F-test) for continuous variables. d Data available in 1995, 1996 and 1997. e Data available in 1993 and 1994. b
AEP Vol. 13, No. 2 February 2002: 105–110
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
Moreover, a logistic regression analysis suggested that HLEQ response status did not statistically significant (2 log likelihood, P-value 0.10) interact with associations between subjective health and smoking or between SES and smoking.
DISCUSSION The response rate of 45% in the MORGEN-project was only slightly lower than published values for comparable studies in the Netherlands (50–70%) (20). This may be explained by a general increase in non-response over the past 10 years and the length of the questionnaires (21). The mean response rate in the MORGEN-project was elevated by the relatively high response rate in Doetinchem (68%), which consisted largely of a selected sub-population (participants in the previous monitoring project). Response rates to studies undertaken in the Netherlands tend to be low compared with those obtained from similar studies completed in other European countries (21, 22). This might be caused by the heavy response burden in the Netherlands (21). About 30% of the people to whom the invitation was sent did not respond. It seems reasonable to assume that some of these people did not receive the invitation because they had moved to another address. Between 1993 and 1997 the annual mean removal rate varied be-
TABLE 3. Comparison of associations within total study population (sample frame) and among respondents to the HLEQ a,b Odds ratios current smokers
Total study population
Responders
Socioeconomic status 1993 to 1997 Low Medium High
N 21,445
N 12,039
1.66 (1.54–1.78) 1.29 (1.19–1.40) 1*
1.55 (1.41–1.71) 1.29 (1.16–1.43) 1*
N 8,799
N 4,805
1* 1.29 (1.12–1.48) 1.70 (1.46–1.97) 1.82 (1.47–2.25) 1.96 (1.34–2.88) N 12,499
1* 1.37 (1.14–1.64) 1.79 (1.46–2.20) 2.03 (1.48–2.81) 1.42 (0.75–2.69) N 7,229
1* 1.11 (0.95–1.29) 1.49 (1.30–1.70) 2.09 (1.78–2.45) 3.38 (2.40–4.77)
1* 1.15 (0.95–1.40) 1.45 (1.22–1.72) 2.18 (1.76–2.69) 3.95 (2.44–6.39)
Subjective Health 1993 to 1994 Excellent Good Moderate Fair Bad Subjective Health 1995 to 1997 Excellent Very good Good Fair Bad
* Reference category. Health And Life Experiences Questionnaire. b With adjustment for sex. a
109
tween 98 and 163 per 1,000 in Amsterdam, Doetinchem and Maastricht (23). The major reasons for non-response (lack of time and interest and had already had a medical check up) were also reported in other studies (6, 13). Response rates in MORGEN were found lower among men and younger people, comparable with findings from other studies (4, 5, 7–9). In comparison with the general population of the Netherlands within the same age range (24), the percentage of male current smokers in the MORGEN-project (36%) was lower than the general population (42%) while among women this was reversed (MORGEN, 36%; general population 32%) (24). The percentages of current drinkers of alcohol were found somewhat higher in MORGEN (91% for men and 81% for women) as compared with the general population (88% for men and 73% for women) (24). Since response rates, smoking habits and alcohol consumption were all associated with socioeconomic status, these differences may be due to lower response rates in the lower socioeconomic status categories. The response percentage to the HLEQ was 56%. The main reasons given for non-response were “questionnaire was too personal” or “too lengthy”. There were no substantial differences in response rates among people with different sex or age. Main differences were found in socioeconomic status and occupational status, which was comparable with findings obtained in other studies (1–4). Respondents to the HLEQ tended to report better lifestyle behaviors and subjective health, also reported in other studies (1, 2, 5, 7, 11–13). Weighted prevalence estimates were calculated to evaluate and correct for indirect selective non-response and our results suggested no bias in prevalence estimation. However, an important assumption inherent in this method was that the prevalence of lifestyle risk factors among respondents is comparable with that among non-respondents. Based on a small non-response survey in the MORGENproject (25), this seems—at least for smoking—not a valid assumption. Moreover, we found a higher proportion current drinkers and a higher proportion practicing sports among respondents on the HLEQ as compared with those who gave only limited information. Therefore, bias in these prevalence estimates should be considered likely. When studying associations between smoking status and either socioeconomic status or subjective health for respondents and non-respondents separately, it was found that these associations did not vary according to response status. This is comparable with results from a study on estimates of health care utilization, where estimations based on respondents only were slightly different from those based on the entire target population (10). Therefore, when using a twophase sampling design, non-response in the second phase might introduce bias in prevalence estimates (especially in estimations of unhealthy lifestyles), although non-response bias in associations is less likely.
110
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
In conclusion, prevalence estimates of current smoking, low physical activity or bad subjective health based upon respondents to the HLEQ are likely to be underestimates of the rates found in the general population of the Netherlands. However, no differences were found among respondents and non-respondents in the association between smoking and socioeconomic status or subjective health. Therefore, we conclude that non-response did not cause bias in the examined associations. The MORGEN project was financially supported by the Ministry of Public Health, Welfare and Sports of The Netherlands and the National Institute of Public Health and the Environment. The authors thank the epidemiologists and field workers of the Municipal Health Services in Amsterdam, Doetinchem and Maastricht for their important contribution to the data collection. The project steering committee consisted of Dr. H.B. Bueno de Mesquita, Dr. H.A. Smit, Dr. W.M.M. Verschuren and J.C. Seidell (project leader). Data management was provided by A. Blokstra, P. Steinberger and A. van Kessel. The study on psychosocial factors and cancer risk is supported by a grant from the Dutch Cancer Society. Use of the questionnaire on lifestyle factors was in part made possible by the financial support received from the Europe against Cancer Program of the Commission of the European Communities in relation to the European Prospective Investigation into Cancer and Nutrition. The psychosocial component of the EPIC in Norfolk is funded in the UK by the Medical Research Council.
REFERENCES 1. Boeing H, Korfmann A, Bergmann MM. Recruitment procedures of EPIC-Germany. Ann Nutr Metab. 1999;43:205–215. 2. Boström G, Hallqvist J, Haglund BJA, Romelsjö A, Svanström L, Diderichsen F. Socioeconomic differences in smoking in an urban Swedish population. Scan J Soc Med. 1993;21:77–81. 3. Bergstrand R, Vedin A, Wiehelmsson C, Wilhelmsen L. Bias due to non-participation and heterogenous sub-groups in population surveys. J Chron Dis. 1983;36:725–728. 4. Rodes A, Sans S, Balana LL, Paluzie G, Aguilera R, Balaguer-Vintro I. Recruitment methods and differences in early, late and non-respondents in the first MONICA-Catalonia population survey. Rev Epidém et Santé Publ. 1990;38:447–453. 5. Jacobsen BK, Thelle DS. The Tromsø Heart Study: Responders and non-responders to a health questionnaire. Do they differ? Scand J Soc Med. 1988;16:101–104. 6. Janzon L, Hanson BS, Isacsson S-O, Lindell S-E, Steen B. Factors influencing participation in health surveys. Results from prospective population study ‘men born in 1914’ in Malmö. Sweden. J Epidemiol Comm Health. 1986;40:174–177.
AEP Vol. 13, No. 2 February 2003: 105–110
7. Macera CA, Jackson KL, Davis DR, Kronenfeld JJ, Blair SN. Patterns of non-response to a mail survey. J Clin Epidemiol. 1990;43:1427–1430. 8. Etter J-F, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol. 1997;50:1123–1128. 9. Eastwood BJ, Gregor RD, Maclean DR, Wolf HK. Effects of recruitment strategy on response rates and risk factor profile in two cardiovascular surveys. Int J Epidemiol. 1996;25:763–769. 10. Reijneveld SA, Stronks K. The impact of response bias on estimates of health care utilization in a metropolitan area: the use of administrative data. Int J Epidemiol. 1999;28:1134–1140. 11. Hill A, Roberts J, Ewings P, Gunnell D. Non-response bias in a lifestyle survey. J Public Health Med. 1997;19:203–207. 12. O’Neill TW, Marsden D, Silman AJ. and the European Vertebral Osteoporosis Study Group. Differences in the characteristics of responders and non-responders in a prevalence survey of vertebral osteoporosis. Osteoporosis Int. 1995;5:327–334. 13. Criqui MH. Response bias and risk ratios in epidemiologic studies. Am J Epidemiol. 1979;109:394–399. 14. Criqui MH, Barrett-Connor E, Austin M. Differences between respondents and non-respondents in a population-based cardiovascular disease study. Am J Epidemiol. 1978;108:367–372. 15. Surtees PG, Wainwright NWJ, Brayne C. Psychosocial aetiology of chronic disease: a pragmatic approach to the assessment of lifetime affective morbidity in an EPIC component study. J Epidemiol Comm Health. 2000;54:114–122. 16. Ocké M, Bueno de Mesquita B, Goddijn E, Ocké M, Bueno de Mesquita B, Goddyn E, Jansen A, Pols M, van Staveren W, Kromhout D. The Dutch EPIC food frequency questionnaire. I. Description of the questionnaire and relative validity and reproducibility for food groups. Int J Epidemiol. 1997;26:s37–s48. 17. Van der Zee KI, Sanderman H, Heyink J. A comparison of two multidimensional measures of health status: the Nottingham health profile and the RAND 36-item health survey 1.0. Qual Life Res. 1996;5:165–174. 18. Ormel J. Neuroticism and well-being inventories: measuring traits or states? Psychol Med. 1983;13:165–176. 19. Van Oostrom MA, Tijhuis MAR, De Haes JCJM, Tempelaar R, Kromhout D. A measurement of social support in epidemiological research: the social experiences checklist tested in a general population in the Netherlands. J Epidemiol Comm Health. 1995;49:518–524. 20. Kalfs N, Kool E. Experiences with non-response. Rapport NIMMO Nederlands Instituut voor Maatschappij en Markt-onderzoek. 1994. 21. Hupkens CLH, Van den Berg J, Van der Zee J. National health interview surveys in Europe: an overview. Health Policy. 1999;47:145–168. 22. De Heer WF, Israëls AZ, eds. Response trends in Europe. American Statistical Association; Proceedings of the section on Survey Research Methods 1992; 92–101. 23. Central Bureau of Statistics. Statistisch Bestand Nederlandse Gemeenten 1999. Voorburg/Heerlen; 1999. 24. Central Bureau of Statistics. Netherlands Health Interview Survey. 1981–1995. The Hague, The Netherlands. 1996. 25. Steenbrink-Van Woerden J. Non-response pilotstudy MORGENproject 1993. The Netherlands; RIVM; 1994.
PURPOSE: Differences in respondent characteristics may lead to bias in prevalence estimates and bias in associations. Both forms of non-response bias are investigated in a study on psychosocial factors and cancer risk, which is a sub-study of a large-scale monitoring survey in the Netherlands. METHODS: Respondents of a cross-sectional monitoring project (MORGEN; N 22,769) were also asked to participate in a prospective study on psychosocial factors and cancer risk (HLEQ; N 12,097). To investigate diverse aspects of non-response in the HLEQ on prevalence estimates and associations are studied, based on information gathered in the MORGEN-project. RESULTS: A response percentage of 45% was obtained in the MORGEN-project. Response rates were found to be lower among men and younger people. The HLEQ showed a response percentage of 56%, and respondents reported higher socioeconomic status, better subjective health and healthier lifestyle behaviors than non-respondents. However, associations between smoking status and either socioeconomic status or subjective health based on respondents only were not statistically different from those based on the entire MORGEN-population. CONCLUSION: Non-response leads to bias in prevalence estimates of current smoking, current alcohol intake, and low physical activity or poor subjective health. However, non-response did not cause bias in the examined associations. Ann Epidemiol 2003;13:105–110. © 2003 Elsevier Science Inc. KEY WORDS:
Bias, Health, Lifestyle, Non-response, Socio-demographic.
INTRODUCTION Non-response bias is a major concern for studies based entirely upon data collected through mailed questionnaires. In general, response rates vary by socioeconomic status (1– 4), by gender (5–7), by age (4, 8, 9) and by marital status (3, 10). In addition, respondents reported more often better health status and more positive health-related behaviors than non-respondents did (1, 2, 5–7, 11–13). These differences in respondent characteristics may lead to bias in prevalence estimates and bias in associations (2, 14). However, in most studies little information is available from non-respondents. Consequently, it is usually impossible to quantify response bias.
In an ongoing large-scale prospective study examining psychosocial factors, lifestyle and cancer risk, using the Health and Life Experiences Questionnaire (HLEQ), the sampling frame consisted of participants in the monitoring project on chronic disease risk factors (MORGEN-project). Therefore, it was possible to compare respondents and nonrespondents on the HLEQ, using data collected in the MORGEN-project, to quantify possible response bias in prevalence estimation and establish whether non-response contributed to bias in associations.
METHODS Study Population
From the National Institute of Public Health and the Environment, Bilthoven, The Netherlands (A.J.M.L., M.T., H.S.J.P.); MRC Biostatistics Unit, Institute of Public Health, University of Cambridge, Great Britain (P.G.S.); and Department of Psychiatry, University of Groningen, The Netherlands (J.O.). Address correspondences to: A.J.M. Van Loon, National Institute of Public Health and the Environment, P.O. Box 1, 3720 BA Bilthoven, The Netherlands. Tel.: 31 30 2743210; Fax: 31 30 2744407. E-mail: [email protected] Received 12 February 2001; revised 28 November 2001; accepted 13 December 2001. © 2003 Elsevier Science Inc. All rights reserved. 360 Park Avenue South, New York, NY 10010
Between 1993 and 1997, in the MORGEN-project, data on health status and the prevalence of risk factors were collected in a random sample of the general population, aged 20–59 years, drawn from three towns in the Netherlands. In Amsterdam and Maastricht each year a new sample was examined and in Doetinchem, participants of an earlier study (1987 to 1991) were re-examined. A total of 50,766 persons received an invitation to participate in the MORGEN-project. Those who returned a reply card had received two questionnaires by mail (a general question1047-2797/03/$–see front matter PII S1047-2797(02)00257-0
106
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
AEP Vol. 13, No. 2 February 2003: 105–110
naire on socio-demographic factors, lifestyle and health indicators and a food frequency questionnaire). They were then asked to visit the local Public Health Service for a medical examination. All participants in the MORGENproject who gave informed consent for further research (94%) received a HLEQ, measuring psychosocial factors like life events and personality characteristics (15).
tivity and diet), and health aspects were used. Dietary aspects were measured with a food frequency questionnaire (16). Health aspects under study were self-rated subjective health, body mass index, physical functioning and mental health (measured with the RAND-36) (17), neuroticism (18), and positive and negative social experiences (19). When quantifying possible response bias in prevalence estimation, particular attention was given to smoking habits and alcohol intake. Relationships between subjective health or socioeconomic status and smoking were studied to check whether non-response would lead to bias in associations.
Independent Variables
Statistical Methods
When comparing respondents with non-respondents in the MORGEN-project, attention was given to gender, age and place of residence. When comparing respondents to the HLEQ with non-respondents, information available from the MORGEN-questionnaire concerning socio-demographic characteristics (sex, age, marital status, socioeconomic status, place of residence), lifestyle risk factors for chronic disease (smoking, alcohol intake, low physical ac-
Available information on the chosen independent variables was described for both respondents and non-respondents. Statistical significance was assessed by chi-square tests for categorical variables and by ANOVA (F-test) for continuous variables (p 0.05). To quantify possible response bias in prevalence estimation, weighted prevalence figures were calculated. To check whether non-response would lead to bias in associations, multivariate analyses
Selected Abbreviations and Acronyms HLEQ Health and Life Experiences Questionnaire
FIGURE 1. Flow diagram (non-) response in MORGEN and HLEQ.
AEP Vol. 13, No. 2 February 2002: 105–110
were conducted, using MORGEN baseline data. Response status on the HLEQ (yes or no) was included as independent variable in the analysis and effect modification by response status was statistically tested by means of maximum likelihood (p 0.10). All statistical analyses were completed using SAS statistical software (version 6.12) (SAS institute Inc., Cary, North Carolina, USA).
RESULTS In total, 50,766 people were approached for participation in the MORGEN-project. Finally, 22,769 persons (45%) completed the total survey including questionnaires and a medical examination (Fig. 1). Overall, response percentages were higher among women (49%) than men (41%) and were higher among older persons (30% among people aged 20–29 and 54% among people aged 50–65). Response percentages were highest in Doetinchem (68%), intermediate in Maastricht (45%) and lowest in Amsterdam (34%). A decline in response percentages was seen throughout the years, from 48% in 1993 until 40% in 1997. For people who did not participate in the MORGENproject, but who returned the reply card, or who participated in a non-response survey, the main reasons for refusal were “no time” (35%), “already have a medical check-up on a regular basis” (25%), “I am healthy, there is no reason to participate” (16%) and “no interest” (15%). For about 30% of the sample, we had no information except the sample frame characteristics (sex, age, and place of residence). A comparison of the characteristics of respondents with those who gave only limited information (for the period 1995 to 1997) showed that there were only minor differences in level of education or the proportion of current smokers (Table 1). The proportion of current alcohol drinkers and the proportion practicing sports were found higher among respondents as compared with those who gave only limited information. A total of 21,426 persons were invited to complete the HLEQ (Fig. 1). Of these, 10,050 persons returned the questionnaire after the first mailing and 2,047 after the second mailing yielding a response percentage of 56%. Response percentages were only slightly lower in Amsterdam (54%), compared with Doetinchem (58%) and Maastricht (57%). A comparison of HLEQ respondent characteristics with those of non-respondents was completed, based on data from the MORGEN-project. Response percentages were comparable between men (55%) and women (57%) with no age differences between respondents and non-respondents (Table 2). A higher percentage of male respondents were married and a lower percentage of all respondents, were either low SES, or had no paid job. Current smoking was less prevalent among respondents, current alcohol intake was more prevalent among respondents and low physi-
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
107
TABLE 1. Characteristics of respondents and non-respondents with incomplete information in The MORGENa-project: 1995 to 1997 Complete response N13,490 N Sex: male Mean age City: Amsterdam Doetinchem Maastricht Level of education % low % high % current smoker (cig) % current drinker % playing sports a b
6072 41.9 4490 4292 4708 6196 3437 4846 11,413 6348
Incomplete responseb N 7221 %
N
% 48.0
33.3 31.8 34.9
3463 38.9 3338 1004 2879
46.3 25.7 36.0 85.1 47.2
3140 1584 2305 5204 2952
48.8 24.6 37.1 75.5 43.2
45.0
46.2 13.9 39.9
Monitoring Project on Chronic Disease Risk Factors. Only general questionnaire or reply card or non-response survey by telephone.
cal activity was found less often among respondents. The intake of energy percentage of fat between respondents and non-respondents was rather small, whereas the intake of vitamin C was slightly higher among respondents. Finally, among respondents a smaller proportion reported poor subjective health and fewer respondents had a body mass index above 30 than non-respondents. Only small differences in measures of physical and mental functioning were found between respondents and non-respondents. To assess whether the non-response rate to the HLEQ (of about 44%) led to bias in the prevalence of lifestyle risk factors, weighted figures for lifestyle behaviors and health aspects were determined, assuming only indirect selective non-response (Table 2). Figures were weighted for socioeconomic status (low, medium, high), marital status (married vs. not married), occupational status (paid job vs. no paid job) and place of residence (Amsterdam, Doetinchem, Maastricht). Distributions of these characteristics showed the greatest dissimilarities between respondents and nonrespondents. No substantial differences were found between non-weighted and weighted prevalence estimations. For example, the prevalence of current smoking among respondents changed from 33% to 32% after weighting and from 40% to 39% among non-respondents. The greatest effect of weighting was found for vitamin C intake among nonrespondents. The mean intake in milligram per day changed from 105 mg to 109 mg after weighting. Finally, a check was made as to whether non-response led to bias in the association between subjective health and smoking or the association between SES and smoking. The analyses revealed no essential differences in those associations based upon the total MORGEN-population and those based only upon respondents to the HLEQ (Table 3).
108
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
AEP Vol. 13, No. 2 February 2003: 105–110
TABLE 2. Comparison of non-respondents and respondents to the HLEQ a, based on MORGENb-baseline characteristics (1993 to 1997)c N Sociodemographic Sex: male (%) Mean age (years) Not married (%) Low SES (%) No paid job (%) Place of residence (%) Amsterdam Doetinchem Maastricht Lifestyle behaviors: unweighted Current smoking (%) # packyears (mean) Current alcohol intake # glasses p.d. (mean) Low physical activity (%) Energy % fat Vitamin C intake (mg/day) Lifestyle behaviors: weighted Current smoking (%) # packyears (mean) Current alcohol intake # glasses p.d. (mean) Low physical activity (%) Energy % fat Vitamin C intake (mg/day) Health aspects: unweighted Bad subjective health (%) 1993 to 1994 1995 to 1997 Body mass index (%) 18.5 30 Physical functioningd (mean) Mental well-beingd (mean) Neuroticisme (mean) Positive social exp. (mean) Negative social exp. (mean) Health aspects: weighted Bad subjective health (%) 1993 to 1994 1995 to 1997 Body mass index (%) 18.5 30 Physical functioningd (mean) Mental well–beingd (mean) Neuroticisme (mean) Positive social exp. (mean) Negative social exp. (mean) a
21,426 21,426 21,338 21,321 20,563 21,426
21,375 7649 21,339 13,148 21,426 21,364 21,364
8812 12,537 21,442
12,485 12,471 8536 20,292 20,333
Non-respondents (9329)
Respondents (12,097)
Difference [95% CI]
46.3 42.4 36.6 56.2 40.8
44.5 42.2 34.3 42.7 33.0
1.8 [0.5–3.1] 0.2 [0.02–0.4] 2.3 [1.3–3.3] 13.5 [12.2–14.8] 7.8 [6.5–9.1]
32.9 31.8 35.3
29.6 34.0 36.4
3.3 [1.1–5.5] 2.2 [0.0–4.4] 1.1 [3.3–1.1]
40.1 18.3 82.4 2.0 49.1 35.1 104.9
33.1 16.8 88.6 1.9 43.6 34.9 109.0
7.0 [5.7–8.3] 1.5 [0.1–2.0] 6.2 [7.2–5.2] 0.1 [0.06–0.14] 5.5 [4.2–6.8] 0.2 [0.1–0.3] 4.1 [5.4–2.8]
39.3 17.9 83.5 1.9 48.3 34.9 108.5
33.7 17.2 88.1 1.9 43.8 34.9 107.4
5.6 [4.3–6.9] 0.7 [0.2–1.3] 4.6 [5.6– 3.6] 0 [0.1–0.1] 4.5 [3.2-5.8] 0 [0.1–0.1] 1.1 [0.1–2.3]
9.8 18.9
5.2 12.0
1.8 12.1 86.6 71.8 35.1 22.9 13.4
1.4 8.8 89.2 74.5 36.3 23.4 13.2
8.5 17.1
5.4 12.3
1.7 11.4 87.5 72.3 35.3 23.0 13.3
1.3 9.4 88.8 74.4 36.2 23.3 13.2
4.6 [3.5–5.7] 6.9 [5.6–8.2] 0.4 [2.3–3.1] 3.3 [0.8–5.8] 2.6 [3.1– 2.1] 2.7 [3.2– 2.2] 1.2 [1.4– 1.0] 0.5 [0.6– 0.4] 0.2 [0.1–0.3]
3.1 [2.0–4.2] 4.8 [3.5–6.1] 0.4 [0.2–3.0] 2.0 [0.5–4.5] 1.3 [1.8– 0.8] 2.1 [–2.7–1.5] 0.9 [1.1–0.7] 0.3 [0.4–0.2] 0.1 [0.04–0.2]
Health and Life Experiences Questionnaire. Monitoring project on chronic disease risk factors. c Statistical significance was assessed by chi-square tests for categorical variables and by analysis of variance (F-test) for continuous variables. d Data available in 1995, 1996 and 1997. e Data available in 1993 and 1994. b
AEP Vol. 13, No. 2 February 2002: 105–110
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
Moreover, a logistic regression analysis suggested that HLEQ response status did not statistically significant (2 log likelihood, P-value 0.10) interact with associations between subjective health and smoking or between SES and smoking.
DISCUSSION The response rate of 45% in the MORGEN-project was only slightly lower than published values for comparable studies in the Netherlands (50–70%) (20). This may be explained by a general increase in non-response over the past 10 years and the length of the questionnaires (21). The mean response rate in the MORGEN-project was elevated by the relatively high response rate in Doetinchem (68%), which consisted largely of a selected sub-population (participants in the previous monitoring project). Response rates to studies undertaken in the Netherlands tend to be low compared with those obtained from similar studies completed in other European countries (21, 22). This might be caused by the heavy response burden in the Netherlands (21). About 30% of the people to whom the invitation was sent did not respond. It seems reasonable to assume that some of these people did not receive the invitation because they had moved to another address. Between 1993 and 1997 the annual mean removal rate varied be-
TABLE 3. Comparison of associations within total study population (sample frame) and among respondents to the HLEQ a,b Odds ratios current smokers
Total study population
Responders
Socioeconomic status 1993 to 1997 Low Medium High
N 21,445
N 12,039
1.66 (1.54–1.78) 1.29 (1.19–1.40) 1*
1.55 (1.41–1.71) 1.29 (1.16–1.43) 1*
N 8,799
N 4,805
1* 1.29 (1.12–1.48) 1.70 (1.46–1.97) 1.82 (1.47–2.25) 1.96 (1.34–2.88) N 12,499
1* 1.37 (1.14–1.64) 1.79 (1.46–2.20) 2.03 (1.48–2.81) 1.42 (0.75–2.69) N 7,229
1* 1.11 (0.95–1.29) 1.49 (1.30–1.70) 2.09 (1.78–2.45) 3.38 (2.40–4.77)
1* 1.15 (0.95–1.40) 1.45 (1.22–1.72) 2.18 (1.76–2.69) 3.95 (2.44–6.39)
Subjective Health 1993 to 1994 Excellent Good Moderate Fair Bad Subjective Health 1995 to 1997 Excellent Very good Good Fair Bad
* Reference category. Health And Life Experiences Questionnaire. b With adjustment for sex. a
109
tween 98 and 163 per 1,000 in Amsterdam, Doetinchem and Maastricht (23). The major reasons for non-response (lack of time and interest and had already had a medical check up) were also reported in other studies (6, 13). Response rates in MORGEN were found lower among men and younger people, comparable with findings from other studies (4, 5, 7–9). In comparison with the general population of the Netherlands within the same age range (24), the percentage of male current smokers in the MORGEN-project (36%) was lower than the general population (42%) while among women this was reversed (MORGEN, 36%; general population 32%) (24). The percentages of current drinkers of alcohol were found somewhat higher in MORGEN (91% for men and 81% for women) as compared with the general population (88% for men and 73% for women) (24). Since response rates, smoking habits and alcohol consumption were all associated with socioeconomic status, these differences may be due to lower response rates in the lower socioeconomic status categories. The response percentage to the HLEQ was 56%. The main reasons given for non-response were “questionnaire was too personal” or “too lengthy”. There were no substantial differences in response rates among people with different sex or age. Main differences were found in socioeconomic status and occupational status, which was comparable with findings obtained in other studies (1–4). Respondents to the HLEQ tended to report better lifestyle behaviors and subjective health, also reported in other studies (1, 2, 5, 7, 11–13). Weighted prevalence estimates were calculated to evaluate and correct for indirect selective non-response and our results suggested no bias in prevalence estimation. However, an important assumption inherent in this method was that the prevalence of lifestyle risk factors among respondents is comparable with that among non-respondents. Based on a small non-response survey in the MORGENproject (25), this seems—at least for smoking—not a valid assumption. Moreover, we found a higher proportion current drinkers and a higher proportion practicing sports among respondents on the HLEQ as compared with those who gave only limited information. Therefore, bias in these prevalence estimates should be considered likely. When studying associations between smoking status and either socioeconomic status or subjective health for respondents and non-respondents separately, it was found that these associations did not vary according to response status. This is comparable with results from a study on estimates of health care utilization, where estimations based on respondents only were slightly different from those based on the entire target population (10). Therefore, when using a twophase sampling design, non-response in the second phase might introduce bias in prevalence estimates (especially in estimations of unhealthy lifestyles), although non-response bias in associations is less likely.
110
van Loon et al. SURVEY NON-RESPONSE IN THE NETHERLANDS
In conclusion, prevalence estimates of current smoking, low physical activity or bad subjective health based upon respondents to the HLEQ are likely to be underestimates of the rates found in the general population of the Netherlands. However, no differences were found among respondents and non-respondents in the association between smoking and socioeconomic status or subjective health. Therefore, we conclude that non-response did not cause bias in the examined associations. The MORGEN project was financially supported by the Ministry of Public Health, Welfare and Sports of The Netherlands and the National Institute of Public Health and the Environment. The authors thank the epidemiologists and field workers of the Municipal Health Services in Amsterdam, Doetinchem and Maastricht for their important contribution to the data collection. The project steering committee consisted of Dr. H.B. Bueno de Mesquita, Dr. H.A. Smit, Dr. W.M.M. Verschuren and J.C. Seidell (project leader). Data management was provided by A. Blokstra, P. Steinberger and A. van Kessel. The study on psychosocial factors and cancer risk is supported by a grant from the Dutch Cancer Society. Use of the questionnaire on lifestyle factors was in part made possible by the financial support received from the Europe against Cancer Program of the Commission of the European Communities in relation to the European Prospective Investigation into Cancer and Nutrition. The psychosocial component of the EPIC in Norfolk is funded in the UK by the Medical Research Council.
REFERENCES 1. Boeing H, Korfmann A, Bergmann MM. Recruitment procedures of EPIC-Germany. Ann Nutr Metab. 1999;43:205–215. 2. Boström G, Hallqvist J, Haglund BJA, Romelsjö A, Svanström L, Diderichsen F. Socioeconomic differences in smoking in an urban Swedish population. Scan J Soc Med. 1993;21:77–81. 3. Bergstrand R, Vedin A, Wiehelmsson C, Wilhelmsen L. Bias due to non-participation and heterogenous sub-groups in population surveys. J Chron Dis. 1983;36:725–728. 4. Rodes A, Sans S, Balana LL, Paluzie G, Aguilera R, Balaguer-Vintro I. Recruitment methods and differences in early, late and non-respondents in the first MONICA-Catalonia population survey. Rev Epidém et Santé Publ. 1990;38:447–453. 5. Jacobsen BK, Thelle DS. The Tromsø Heart Study: Responders and non-responders to a health questionnaire. Do they differ? Scand J Soc Med. 1988;16:101–104. 6. Janzon L, Hanson BS, Isacsson S-O, Lindell S-E, Steen B. Factors influencing participation in health surveys. Results from prospective population study ‘men born in 1914’ in Malmö. Sweden. J Epidemiol Comm Health. 1986;40:174–177.
AEP Vol. 13, No. 2 February 2003: 105–110
7. Macera CA, Jackson KL, Davis DR, Kronenfeld JJ, Blair SN. Patterns of non-response to a mail survey. J Clin Epidemiol. 1990;43:1427–1430. 8. Etter J-F, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol. 1997;50:1123–1128. 9. Eastwood BJ, Gregor RD, Maclean DR, Wolf HK. Effects of recruitment strategy on response rates and risk factor profile in two cardiovascular surveys. Int J Epidemiol. 1996;25:763–769. 10. Reijneveld SA, Stronks K. The impact of response bias on estimates of health care utilization in a metropolitan area: the use of administrative data. Int J Epidemiol. 1999;28:1134–1140. 11. Hill A, Roberts J, Ewings P, Gunnell D. Non-response bias in a lifestyle survey. J Public Health Med. 1997;19:203–207. 12. O’Neill TW, Marsden D, Silman AJ. and the European Vertebral Osteoporosis Study Group. Differences in the characteristics of responders and non-responders in a prevalence survey of vertebral osteoporosis. Osteoporosis Int. 1995;5:327–334. 13. Criqui MH. Response bias and risk ratios in epidemiologic studies. Am J Epidemiol. 1979;109:394–399. 14. Criqui MH, Barrett-Connor E, Austin M. Differences between respondents and non-respondents in a population-based cardiovascular disease study. Am J Epidemiol. 1978;108:367–372. 15. Surtees PG, Wainwright NWJ, Brayne C. Psychosocial aetiology of chronic disease: a pragmatic approach to the assessment of lifetime affective morbidity in an EPIC component study. J Epidemiol Comm Health. 2000;54:114–122. 16. Ocké M, Bueno de Mesquita B, Goddijn E, Ocké M, Bueno de Mesquita B, Goddyn E, Jansen A, Pols M, van Staveren W, Kromhout D. The Dutch EPIC food frequency questionnaire. I. Description of the questionnaire and relative validity and reproducibility for food groups. Int J Epidemiol. 1997;26:s37–s48. 17. Van der Zee KI, Sanderman H, Heyink J. A comparison of two multidimensional measures of health status: the Nottingham health profile and the RAND 36-item health survey 1.0. Qual Life Res. 1996;5:165–174. 18. Ormel J. Neuroticism and well-being inventories: measuring traits or states? Psychol Med. 1983;13:165–176. 19. Van Oostrom MA, Tijhuis MAR, De Haes JCJM, Tempelaar R, Kromhout D. A measurement of social support in epidemiological research: the social experiences checklist tested in a general population in the Netherlands. J Epidemiol Comm Health. 1995;49:518–524. 20. Kalfs N, Kool E. Experiences with non-response. Rapport NIMMO Nederlands Instituut voor Maatschappij en Markt-onderzoek. 1994. 21. Hupkens CLH, Van den Berg J, Van der Zee J. National health interview surveys in Europe: an overview. Health Policy. 1999;47:145–168. 22. De Heer WF, Israëls AZ, eds. Response trends in Europe. American Statistical Association; Proceedings of the section on Survey Research Methods 1992; 92–101. 23. Central Bureau of Statistics. Statistisch Bestand Nederlandse Gemeenten 1999. Voorburg/Heerlen; 1999. 24. Central Bureau of Statistics. Netherlands Health Interview Survey. 1981–1995. The Hague, The Netherlands. 1996. 25. Steenbrink-Van Woerden J. Non-response pilotstudy MORGENproject 1993. The Netherlands; RIVM; 1994.