- Email: [email protected]
In response to the 2004, Vol. 27, Issue 3, article entitled
Letters to the Editor Kazim Sheikh, MD U.S. Department of Health and Human Services Centers for Medicare & Medicaid Services Kansas City, Missouri E-mail: [email protected]
In Response to the 2004, Vol. 27, Issue 3, article entitled
Consequences of Declining Survey Response Rates for Smoking Prevalance Estimates
I
n their two-state study of survey response and smoking prevalence, Biener et al.1 concluded that declining response rates did not result in biased prevalence of smoking. This misleading statement was based on their comparison of the estimated smoking prevalence from state surveys with those from the Current Population Surveys (the “gold standard”) for three time periods. The authors found that even though the survey response rates declined 12% to 27% over time, the smoking prevalence estimates from the two sources remained comparable. The gold-standard survey itself could have been unreliable because its response rates also declined from 85% to 76%, and nonresponse bias had not been excluded. Nonresponse bias can exist even when the response rate is as high as 83%,2 and there is no ceiling above which bias cannot exist. If the prevalence of smoking from the state surveys with declining response rates was comparable with the gold-standard prevalence, both sets of prevalence rates could have been biased. Response bias is known to be inversely correlated with the survey response rate,3–5 and the authors also cited a paper6 that supports this inverse correlation. Furthermore, the prevalence of the risk factors for an outcome (smoking in this case) and the incidence of the outcome among nonrespondents are often different from that in respondents.2,7 However, nonresponse bias may exist even when the risk factor prevalence and outcome incidence among nonrespondents are the same as those among respondents.8,9 The only reliable way to investigate nonresponse bias is to conduct a special study of the nonrespondents or their representative sample, and determine the prevalence of smoking or established risk factors for smoking among the nonrespondents and compare them with those among the respondents.10 Such investigations have found that smokers are less likely than nonsmokers to respond to surveys.7,11–13 In addition to lower socioeconomic status (SES), many other factors increase the risk of smoking. These factors include genetic predisposition to chemical dependency, acquired illicit drug dependency, clinical depression, other psychological aberrations, alcohol consumption, family history, adverse home environment, and stress. Regular physical exercise, participation in athletic sports, and appropriate health education at a young age are protective. If there were no differences between respondents and nonrespondents with respect to the prevalence of these unmeasured risk factors for smoking, only then the authors could claim “no evidence of biased estimates of smoking behavior.” Similar differences between the survey samples and the parent population with respect to age and SES from survey to survey provide only false security against biased smoking prevalence.
The views expressed in this letter do not represent the views and policies of the Centers for Medicare & Medicaid Services or the United States.
References 1. Biener L, Garrett CA, Gilpin EA, Roman AM, Currivan DB. Consequences of declining survey response rates for smoking prevalence estimates. Am J Prev Med 2004;27:254 –7. 2. Sheikh K. Predicting risk among non-respondents in prospective studies. Eur J Epidemiol 1986;2:39 – 43. 3. Sheikh K, Mattingly S. Investigating non-response bias in mail surveys. J Epidemiol Community Health 1981;35:293– 6. 4. Kreiger N, Nishri ED. The effect of non-response on estimation of relative risk in a case-control study. Ann Epidemiol 1997;7:194 –9. 5. Seltzer CC, Bosse R, Garvey AJ. Mail survey response by smoking status. Am J Epidemiol 1975;100:453–7. 6. Mariolis P. Data accuracy: how good are our usual indicators? Paper presented at Statistics Canada Symposium 2001, Ottawa, Canada. Available at: www. statcan.ca/english/conferences/symposium2001/session2/s2b.pdf. Accessed October 6, 2004. 7. Criqui MH, Barrett-Connor E, Austin M. Differences between respondents and non-respondents in population-based cardiovascular disease study. Am J Epidemiol 1978;108:367–72. 8. Greenland S. Response and follow-up bias in cohort studies. Am J Epidemiol 1977;106:184 –7. 9. Criqui MH. Response bias and risk ratios in epidemiologic studies. Am J Epidemiol 1979;109:394 –9. 10. Austin MA, Criqui MH, Barrett-Connor E, Holdbrook MJ. The effect of response bias on the odds ratio. Am J Epidemiol 1981;114:137– 43. 11. Oakes TW, Friedman GD, Seltzer CC. Mail survey response by health status of smokers, non-smokers, and ex-smokers. Am J Epidemiol 1973;98:50 –5. 12. Seltzer CC, Bosse R, Garvey AJ. Mail survey response by smoking status. Am J Epidemiol 1975;100:453–7. 13. Van Loon AJM, Tijhuis M, Picavet HS, Surtees PG, Ormel J. Survey non-response in the Netherlands: effects on prevalence estimates and associations. Ann Epidemiol 2003;13:105–10.
Author’s Response To the Editor:
W
e have no argument with Dr. Sheikh’s view that estimates derived from all sample surveys are subject to nonresponse bias, and we did not imply that the Massachusetts and California surveys produce unbiased estimates of smoking prevalence. The goal of our analysis was to investigate whether estimates of smoking prevalence had become increasingly biased as a consequence of decreasing response rates. The results suggested that they had not, since (1) the estimates did not grow more discrepant from those obtained by the Current Population Survey, and (2) comparing the respondents’ demographic profiles with that of recent U.S. Census reports demonstrated that subgroups were overor under-represented in a similar fashion, regardless of the
Am J Prev Med 2005;28(2) © 2005 American Journal of Preventive Medicine • Published by Elsevier Inc.
0749-3797/05/$–see front matter doi:10.1016/j.amepre.2004.10.019
241
In Response to the 2004, Vol. 27, Issue 3, article entitled
Consequences of Declining Survey Response Rates for Smoking Prevalance Estimates
I
n their two-state study of survey response and smoking prevalence, Biener et al.1 concluded that declining response rates did not result in biased prevalence of smoking. This misleading statement was based on their comparison of the estimated smoking prevalence from state surveys with those from the Current Population Surveys (the “gold standard”) for three time periods. The authors found that even though the survey response rates declined 12% to 27% over time, the smoking prevalence estimates from the two sources remained comparable. The gold-standard survey itself could have been unreliable because its response rates also declined from 85% to 76%, and nonresponse bias had not been excluded. Nonresponse bias can exist even when the response rate is as high as 83%,2 and there is no ceiling above which bias cannot exist. If the prevalence of smoking from the state surveys with declining response rates was comparable with the gold-standard prevalence, both sets of prevalence rates could have been biased. Response bias is known to be inversely correlated with the survey response rate,3–5 and the authors also cited a paper6 that supports this inverse correlation. Furthermore, the prevalence of the risk factors for an outcome (smoking in this case) and the incidence of the outcome among nonrespondents are often different from that in respondents.2,7 However, nonresponse bias may exist even when the risk factor prevalence and outcome incidence among nonrespondents are the same as those among respondents.8,9 The only reliable way to investigate nonresponse bias is to conduct a special study of the nonrespondents or their representative sample, and determine the prevalence of smoking or established risk factors for smoking among the nonrespondents and compare them with those among the respondents.10 Such investigations have found that smokers are less likely than nonsmokers to respond to surveys.7,11–13 In addition to lower socioeconomic status (SES), many other factors increase the risk of smoking. These factors include genetic predisposition to chemical dependency, acquired illicit drug dependency, clinical depression, other psychological aberrations, alcohol consumption, family history, adverse home environment, and stress. Regular physical exercise, participation in athletic sports, and appropriate health education at a young age are protective. If there were no differences between respondents and nonrespondents with respect to the prevalence of these unmeasured risk factors for smoking, only then the authors could claim “no evidence of biased estimates of smoking behavior.” Similar differences between the survey samples and the parent population with respect to age and SES from survey to survey provide only false security against biased smoking prevalence.
The views expressed in this letter do not represent the views and policies of the Centers for Medicare & Medicaid Services or the United States.
References 1. Biener L, Garrett CA, Gilpin EA, Roman AM, Currivan DB. Consequences of declining survey response rates for smoking prevalence estimates. Am J Prev Med 2004;27:254 –7. 2. Sheikh K. Predicting risk among non-respondents in prospective studies. Eur J Epidemiol 1986;2:39 – 43. 3. Sheikh K, Mattingly S. Investigating non-response bias in mail surveys. J Epidemiol Community Health 1981;35:293– 6. 4. Kreiger N, Nishri ED. The effect of non-response on estimation of relative risk in a case-control study. Ann Epidemiol 1997;7:194 –9. 5. Seltzer CC, Bosse R, Garvey AJ. Mail survey response by smoking status. Am J Epidemiol 1975;100:453–7. 6. Mariolis P. Data accuracy: how good are our usual indicators? Paper presented at Statistics Canada Symposium 2001, Ottawa, Canada. Available at: www. statcan.ca/english/conferences/symposium2001/session2/s2b.pdf. Accessed October 6, 2004. 7. Criqui MH, Barrett-Connor E, Austin M. Differences between respondents and non-respondents in population-based cardiovascular disease study. Am J Epidemiol 1978;108:367–72. 8. Greenland S. Response and follow-up bias in cohort studies. Am J Epidemiol 1977;106:184 –7. 9. Criqui MH. Response bias and risk ratios in epidemiologic studies. Am J Epidemiol 1979;109:394 –9. 10. Austin MA, Criqui MH, Barrett-Connor E, Holdbrook MJ. The effect of response bias on the odds ratio. Am J Epidemiol 1981;114:137– 43. 11. Oakes TW, Friedman GD, Seltzer CC. Mail survey response by health status of smokers, non-smokers, and ex-smokers. Am J Epidemiol 1973;98:50 –5. 12. Seltzer CC, Bosse R, Garvey AJ. Mail survey response by smoking status. Am J Epidemiol 1975;100:453–7. 13. Van Loon AJM, Tijhuis M, Picavet HS, Surtees PG, Ormel J. Survey non-response in the Netherlands: effects on prevalence estimates and associations. Ann Epidemiol 2003;13:105–10.
Author’s Response To the Editor:
W
e have no argument with Dr. Sheikh’s view that estimates derived from all sample surveys are subject to nonresponse bias, and we did not imply that the Massachusetts and California surveys produce unbiased estimates of smoking prevalence. The goal of our analysis was to investigate whether estimates of smoking prevalence had become increasingly biased as a consequence of decreasing response rates. The results suggested that they had not, since (1) the estimates did not grow more discrepant from those obtained by the Current Population Survey, and (2) comparing the respondents’ demographic profiles with that of recent U.S. Census reports demonstrated that subgroups were overor under-represented in a similar fashion, regardless of the
Am J Prev Med 2005;28(2) © 2005 American Journal of Preventive Medicine • Published by Elsevier Inc.
0749-3797/05/$–see front matter doi:10.1016/j.amepre.2004.10.019
241