A two-phase sampling survey for nonresponse and its paradata to correct nonresponse bias in a health surveillance survey

Available online at

ScienceDirect www.sciencedirect.com Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

Methodological note

A two-phase sampling survey for nonresponse and its paradata to correct nonresponse bias in a health surveillance survey Une enqueˆte en deux phases pour non-re´ponse et ses paradonne´es pour corriger les biais de non-re´ponse dans une enqueˆte de surveillance e´pide´miologique G. Santin a,b,*, L. Be´ne´zet a, B. Geoffroy-Perez a, J. Bouyer c, A. Gue´guen b a Direction sante´ travail, Sante´ publique France, 94415 Saint-Maurice, France UMS 011, Inserm-UVSQ, unite´ « Cohortes e´pide´miologiques en population », 94807 Villejuif cedex, France c Inserm, CESP centre de recherche en e´pide´miologie et sante´ des populations, e´quipe 4 : VIH/Pe´diatrie, 94270 Le Kremlin-Biceˆtre, France b

Received 26 November 2015; accepted 6 October 2016 Available online 17 January 2017

Abstract Background. – The decline in participation rates in surveys, including epidemiological surveillance surveys, has become a real concern since it may increase nonresponse bias. The aim of this study is to estimate the contribution of a complementary survey among a subsample of nonrespondents, and the additional contribution of paradata in correcting for nonresponse bias in an occupational health surveillance survey. Methods. – In 2010, 10,000 workers were randomly selected and sent a postal questionnaire. Sociodemographic data were available for the whole sample. After data collection of the questionnaires, a complementary survey among a random subsample of 500 nonrespondents was performed using a questionnaire administered by an interviewer. Paradata were collected for the complete subsample of the complementary survey. Nonresponse bias in the initial sample and in the combined samples were assessed using variables from administrative databases available for the whole sample, not subject to differential measurement errors. Corrected prevalences by reweighting technique were estimated by first using the initial survey alone and then the initial and complementary surveys combined, under several assumptions regarding the missing data process. Results were compared by computing relative errors. Results. – The response rates of the initial and complementary surveys were 23.6% and 62.6%, respectively. For the initial and the combined surveys, the relative errors decreased after correction for nonresponse on sociodemographic variables. For the combined surveys without paradata, relative errors decreased compared with the initial survey. The contribution of the paradata was weak. Conclusion. – When a complex descriptive survey has a low response rate, a short complementary survey among nonrespondents with a protocol which aims to maximize the response rates, is useful. The contribution of sociodemographic variables in correcting for nonresponse bias is important whereas the additional contribution of paradata in correcting for nonresponse bias is questionable. # 2016 Elsevier Masson SAS. All rights reserved. Keywords: Unit nonresponse; Selection bias; Two-phase sampling for nonresponse; Paradata; Public health survey

Re´sume´ Position du proble`me. – La diminution des taux de participation dans les enqueˆtes, y compris dans les enqueˆtes de surveillance e´pide´miologique, est potentiellement un vrai proble`me puisqu’il peut entraıˆner une augmentation du biais de non-re´ponse. L’objectif de l’e´tude est d’estimer la contribution d’une enqueˆte comple´mentaire aupre`s d’un sous-e´chantillon de non-re´pondants et la contribution additionnelle des paradonne´es pour corriger la non-re´ponse dans une enqueˆte de surveillance e´pide´miologique des risques professionnels.

* Corresponding author at: UMS 011, Inserm-UVSQ, unite´ « Cohortes e´pide´miologiques en population », hoˆpital Paul-Brousse, baˆtiment 15/16, 16, avenue PaulVaillant-Couturier, 94807 Villejuif cedex, France. E-mail address: [email protected] (G. Santin). http://dx.doi.org/10.1016/j.respe.2016.10.059 0398-7620/# 2016 Elsevier Masson SAS. All rights reserved.

72

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

Me´thodes. – En 2010, 10 000 travailleurs ont e´te´ tire´s au sort et ont rec¸u un questionnaire postal. Des donne´es sociode´mographiques e´taient disponibles pour l’ensemble de l’e´chantillon. Apre`s la collecte des donne´es de questionnaire, une enqueˆte comple´mentaire aupre`s d’un souse´chantillon ale´atoire de 500 non-re´pondants a e´te´ conduite par questionnaire administre´ par un enqueˆteur. Des paradonne´es ont e´te´ collecte´es pour l’ensemble du sous-e´chantillon de l’enqueˆte comple´mentaire. Les biais de non-re´ponse a` l’enqueˆte initiale et aux enqueˆtes combine´es ont e´te´ e´value´s en utilisant des variables issues de bases me´dico-administratives, disponibles pour l’ensemble de l’e´chantillon et non-sujettes a` des erreurs de mesure diffe´rentielles. Des pre´valences corrige´es par reponde´ration ont e´te´ estime´es en utilisant d’abord seulement les re´pondants de l’enqueˆte initiale puis en combinant les re´pondants a` l’enqueˆte initiale et a` l’enqueˆte comple´mentaire, sous diffe´rentes hypothe`ses sur le processus de nonre´ponse. Les re´sultats ont e´te´ compare´s en estimant des erreurs relatives. Re´sultats. – Les taux de re´ponse a` l’enqueˆte initiale et a` l’enqueˆte comple´mentaire e´taient respectivement de 23,6 % et 62,6 %. Pour l’enqueˆte initiale et les enqueˆtes combine´es, les erreurs relatives diminuent apre`s correction de la non-re´ponse sur les variables sociode´mographiques. Pour les enqueˆtes combine´es sans l’utilisation des paradonne´es, les erreurs relatives diminuent en comparaison de celles estime´es via l’enqueˆte initiale. La contribution des paradonne´es est faible. Conclusion. – Lorsqu’une enqueˆte descriptive a un faible taux de re´ponse, une enqueˆte comple´mentaire courte parmi les non-re´pondants avec un protocole cherchant a` maximiser le taux de re´ponse, est utile. La contribution des variables sociode´mographiques pour corriger la non-re´ponse est importante, alors que la contribution additionnelle des paradonne´es pour corriger les biais de non-re´ponse est discutable. # 2016 Elsevier Masson SAS. Tous droits re´serve´s. Mots cle´s : Non-re´ponse totale ; Biais de se´lection ; Enqueˆte en deux phases pour non-re´ponse ; Paradonne´es ; Enqueˆte de sante´ publique

1. Introduction In recent years, the decline in participation rates in surveys, including epidemiological surveillance surveys, has become a real concern since it may increase nonresponse bias [1,2]. A nonresponse bias occurs when the response probability and the outcome variable are correlated. Three types of nonresponse are generally considered, using Little and Rubin terminology of nonresponse classification [3]: missing completely at random (MCAR), missing at random (MAR), missing not at random (MNAR). In the case of a MCAR process, there is no association between the response probability and the outcome variable. In the case of a MAR process, associations between the response probability and the outcome variable are completely explained by available auxiliary data. In the case of a MNAR process, associations between the response probability and the outcome variable cannot be completely explained by available auxiliary data. Since nonresponse bias is a function of both the response rate and the covariance between the probability of response and outcome variables [4], two options are available to reduce the effect of nonresponse. The first is to increase the response rate by developing designs that make it possible to contact hard-to-reach persons, sometimes called ‘‘responsive design’’ protocols [5,6]. As Groves and Heeringa [5] pointed out, despite being developed several decades before the use of the term ‘‘responsive design’’, the oldest responsive design protocol in survey methodology is the use of two-phase sampling for nonresponse [7]. Briefly, this design consists in first carrying out a classic random survey that leads to the division of the original sample into two strata: respondents and nonrespondents. Second, a subsample of nonrespondents is randomly selected and surveyed on a subset of variables collected in the initial survey with a protocol designed to obtain a maximal response rate. This second survey, called a follow-up survey in the survey methodology literature, will be called a complementary survey in the present paper to

avoid confusion with cohort studies aimed at following subjects over time. A 100% response rate to the complementary survey makes it possible to estimate unbiased prevalences by combining the responses of the first and the complementary surveys. Nevertheless, such a response rate is unrealistic, and therefore two-phase sampling cannot guarantee to reach an unbiased estimate (and even a reduction of nonresponse bias) [8]. The second option to reduce nonresponse bias is to use reweighting techniques [9]. The major difficulty in effectively reweighting correction is identifying available relevant auxiliary data that are correlated both with probability of response and with outcome variables. One example of auxiliary data is known population totals [10]. In this case, calibration techniques can be used [11]. If auxiliary data are individually available for respondents and nonrespondents, they can be used to estimate response probability, and the inverse probability weighting (IPW) technique can be applied. Often, the sampling frame provides such auxiliary data (for instance sociodemographic data such as age or gender). However, these auxiliary data may be unavailable or insufficient [12]. In this context, one source increasingly used in survey statistics is paradata [13– 16]. These data are mainly collected in surveys using interviewers, and were originally defined as data related to the interview process, such as the frequency or timing of phone calls. In recent years, their definition has been extended to include information directly recorded by the interviewer during face-to-face interviews, for instance observations that characterize the interviewee’s neighborhood, such as the density of stores [16]. Paradata are easily collected for respondents and nonrespondents and are generally strongly correlated with participation [16]. In practice, the options to reduce the effects of nonresponse are not mutually exclusive. The objective of this study was first to evaluate the contribution of a complementary survey in decreasing nonresponse bias in order to estimate prevalences, and second, to assess the additional contribution of complementary survey’s

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

paradata in reducing this bias. We used the data of the CosetMSA pilot study, a French occupational health surveillance survey. 2. Methods 2.1. The Coset-MSA study The pilot study included workers aged between 18 and 65 years on December 31, 2008, who had worked at least 90 days in a workplace affiliated to the Mutualite´ Sociale Agricole (MSA) insurance fund in 2008, in one of five French administrative areas (Bouches-du-Rhoˆne, Pas-de-Calais,

[(Fig._1)TD$IG]

73

Pyre´ne´es-Atlantiques, Saoˆne-et-Loire, Finiste`re). The survey design featured two-phase sampling for nonresponse (hereafter ‘‘combined surveys’’) (Fig. 1) [5,7]. In the first phase, 10,000 people were randomly selected with stratification on gender, age, employment status (salaried vs. non-salaried worker) and geographical area (2000 persons by area); this sample will be called thereafter the ‘‘whole sample’’. They received a 40-page self-administered postal questionnaire about working conditions and health. A postal reminder was mailed one month later [17]. This first phase of the survey (hereafter the ‘‘initial survey’’) was conducted in February 2010, the response rate being 23.6%. In the second phase (hereafter ‘‘complementary survey’’), a subsample of 100 nonrespondents in each

Sampling frame from MSA database N=115,572

IS sample ( =10,000 1st phase After data collection

IS respondents ( =2,363

IS nonrespondents ( =7,637

2nd phase

IS respondents ( =2,363

CS sample ( =500

After data collection δCS,i

IS respondents ( =2,363

CS respondents ( =313

Questionnaire data Fig. 1. Two-phase sampling for nonrespondents. N: population size; IS: initial survey; CS: complementary survey; sIS: random sample of IS (size: nIS); sCS: random sample of CS (size nCS); pIS;i : known probability sampling weight of the individual i for the initial survey; p2nd phase=sIS;nr ;i : known probability sampling weight of the individual i for the second phase conditionally on the respondents sample coming from the initial survey; pCS=sIS;nr ;i : known probability sampling weight of the individual i for CS conditionally on the nonrespondents sample coming from the initial survey; ssurvey,r: the respondent sample from ssurvey and nsurvey,r its size where survey is equal to IS or CS; sCS,i: the unknown response probability for the survey CS that generates sCS,r.

74

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

area was then randomly selected and questioned by an interviewer in November 2010. Data collection was designed so as to achieve a maximum response rate [5,9]. All the persons in this subsample received an information letter. A shorter questionnaire was administered by interviewer, which consisted of a selected number of questions from the questionnaire of the initial survey. To maximize the response rate to the complementary survey, our initial intention was to collect data by phone (Computed Assisted Telephone Interviewing) or by face-to-face (Computed Assisted Personal Interviewing) interview if telephone contact was impossible. However, this kind of design can lead to problems interpreting results as differences in responses could be linked to the difference in data collection modes. We therefore created two random groups, as follows: in each geographical area, 70 and 30 nonrespondents were randomly selected and designated to have phone and face-toface interviews, respectively. If the designated type of interview was impossible (for instance, no phone number was available, or no one answered the telephone call), the interviewer switched to the alternative type. Up to 20 attempts for phone interviews and up to 3 visits for face-to-face interviews were made, at different times and days of the week, including Saturdays. After excluding from the whole sample persons who could not be contacted by post (n = 406) and refusals (n = 236), data from health and occupational administrative databases were available for 9358 persons. The study protocol was approved by the French institutional review committee (CNIL number 909091 and DR-2010-148). 2.2. Data 2.2.1. Outcome variables We studied outcome variables from health and occupational administrative databases for two reasons:  they were available for respondents and nonrespondents to the postal questionnaire and so they enabled us to define a gold standard prevalence;  they allowed us to specifically study nonresponse bias, independently of measurement bias. This is an important point given the possibility of differential measurement bias being induced because of the three modes of data collection used in the survey (postal mail, phone and face-to-face interviews) [18]. Two types of outcome variables were used: health-related data (hospitalization and reimbursement claims for medical services), extracted from the French Health Insurance Information System database (SNIIRAM) and work-related data, extracted from the MSA databases. In the present study, the variables used were:  more than 100 reimbursement claims for medical services (including consultations, prescribed medicines, etc.) between 2008 and 2010;  hospitalization between 2008 and 2010;  working in a primary economic activity;

 job duration less than 10 years in 2010;  sickness absence in 2010 (at work);  worker compensation for accident at work or occupational disease between 2002 and 2008. 2.2.2. Data used to correct for nonresponse (auxiliary variables) 2.2.2.1. Data from the MSA administrative database. Sociodemographic data (available for all individuals in the MSA database, so for both respondents and nonrespondents to the initial survey) were collected in 2008: gender, age, employment status (salaried vs. non-salaried worker) and geographical area. 2.2.2.2. Paradata from the interview process. Paradata (available for both respondents and nonrespondents to the complementary survey) were recorded just before data collection and at the same time as the questionnaire data collection. Paradata available before the questionnaire data collection reflected the quality of the information available to contact the persons: accuracy of the postal mailing address and of phone number. A specific postal service was used to verify and update the postal mailing addresses that were given by the MSA one year before the beginning of the study. If verification concluded that no change of address had occurred, the accuracy of the address provided by the MSA was considered good, otherwise it was considered poor. Phone numbers were checked in the phone directory using each individual’s name (first name and last name) and the updated mailing address. If a phone number corresponding to the same name and mailing address was found, its accuracy was considered good. If a phone number corresponded to the same name but not the same address, its accuracy was considered bad. The third modality of this variable was when no phone number was found. At the moment of data collection, the following paradata were recorded: frequency of phone calls, frequency of phone calls after 5 p.m., frequency of phone calls on Saturdays, frequency of interviewer visits, and visits of an interviewer on Saturdays. A final variable corresponded the two planned modes of data collection (face-to-face and phone) and the switch of the mode of data collection. 2.3. Statistical analysis 2.3.1. Response rates A respondent was defined as a person who had answered completely or partially a Coset-MSA’s questionnaire. For the initial and complementary surveys, two types of unweighted response rates were computed: in the first one, the denominator included the selected sample and in the second one, the denominator excluded the non-contacted persons of all the selected sample. The contacted persons were, for the initial survey, the persons who received the questionnaire and for the complementary survey, the persons who received the information letter. For the combined surveys, a weighted response rate c combined surveys ) was estimated [The American Association (RR

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

for Public Opinion Research. Standard definitions: final dispositions of case codes and outcome rates for surveys. 7th edition. AAPOR. 2011]. c combined RR

surveys

P 1 P 1 sIS pIS;i rSI;i þ sCS pIS;i pCS=sIS;nr ;i rSC;i P 1 ¼ sIS pIS;i

where: ssurvey random sample of IS (initial survey) or CS (complementary survey); pIS;i known probability sampling weight of individual i for the initial survey; pCS=SIS;nr ;i known probability sampling weight of individual i for the complementary survey conditionally on the nonrespondents sample from the initial survey;  r survey;i ¼

1 if i responds to the survey IS or CS 0 otherwise

To understand this point, we have to keep in mind that surveys among nonrespondents are conducted to maximize the frequency of respondents for the survey; to do so, methodologists may prefer to assign high inclusion probability for a group of persons who is more likely to respond to the complementary survey. In this case, estimating unweighted response rate would lead to an artificially high response rate; this is the reason why the recommendation is to estimate a weighted response rate. 2.3.2. Determination of the response probability models Three backward logistic regressions were computed: one for the probability of response to the initial survey, and two for the probability of response to the complementary survey according whether paradata were used or not. In each case, variables with a P-value less than 5% were kept in the final model. 2.3.3. Contribution of complementary survey and of paradata in reducing nonresponse bias The study of the nonresponse bias was performed using variables derived from health and occupational databases. First, six estimations of the prevalence were computed using several sets of weights (see supplementary material part 1):  with the whole sample to provide a gold standard prevalence: bGS ; P  with data from the initial survey: b1 ,  without correction for nonresponse: P  with correction for nonresponse using the sociodemograb2 ; phic variables: P  with data of the combined surveys: b3 ,  without correction for nonresponse: P  with correction for nonresponse using the sociodemograb4 , phic variables: P  with correction for nonresponse using the sociodemograb5 . phic variables and paradata: P b2 The validity of the correction for nonresponse to estimate P b 4 implicitly assumed that the nonresponse processes were or P

75

missing at random (MAR) conditionally on the sociodemob5, sociodemographic variables and graphic variables (or, for P paradata). To correct for nonresponse, we used a reweighting technique called the equal-quantile score method [19–21]; briefly it consisted in estimating by a logistic regression the response probability according to auxiliary variables, ordering the estimated probabilities in ascending order and partitioning the sample in classes of approximately the same size and finally computing and using the response rate observed within the same class as the weight correction for nonresponse. The confidence intervals took into account the sampling design and the variance due to the nonresponse [Brion, P. and Gros, E. Macro Calker. Insee. 2013, Santin G. Non-re´ponse totale dans les enqueˆtes de surveillance e´pide´miologique–Annexe I.9. 2015]. Then, the relative nonresponse errors were then computed: c ¼ bPi bPGS 100 for i ¼ 1;. . .; 5 RE bPGS A relative error of less than 10% was considered acceptable [22]. In line with Olson [23], these relative errors were assumed to correspond to nonresponse bias. The analyses were conducted with SAS 9.4. 3. Results 3.1. Response rate to the initial and complementary surveys The response rate to the initial survey was 23.6%; 96.0% of questionnaires were successfully delivered to the address provided and among the contacted persons, 24.6% of them answered the questionnaire (see supplementary material part 5). The response rate to the complementary survey was 62.6%; 97% of the information letter was successfully delivered to the address provided and among the contacted persons, 64.5% of them answered the questionnaire (see supplementary material part 5). Among respondents, 57% were interviewed by phone vs 43% by face-to-face interview. The weighted response rate to combined surveys was estimated at 70.4%. 3.2. Response probability models The non-contacted persons were considered as nonrespondents since their percentage was low. 3.2.1. Initial survey The response to the initial survey was significantly higher in the following groups: women (odds ratio [OR], 1.3; 95% confidence interval [CI]: [1.2, 1.5]), older persons (OR, 1.6; 95% CI: [1.4, 1.8] for people aged between 50 and 65 years vs people aged between 18 and 34 years), salaried workers (OR, 0.8; 95% CI: [0.7, 0.8] for non-salaried vs salaried workers) and persons living in the Saoˆne-et-Loire area (OR, 1.7; 95% CI: [1.4, 1.9] for people living in Saoˆne-et-Loire vs people living in Bouches-du-Rhoˆne) (Table 1). No significant interaction was found.

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

76

Table 1 Response to the surveys according to sociodemographic variables (final multiple logistic regression). Initial survey

Gender Male Female Age (yr) 18–24 25–49 50–65 Employment status Salaried Non-salaried Geographic area Bouches-du-Rhoˆne Finiste`re Pas-de-Calais Pyre´ne´es-Atlantiques Saoˆne-et-Loire

Complementary survey

n

OR

6775 3225

1 1.3

2677 4323 3000

1 1.4 1.6

5845 4155

1 0.8

95% CI

P

n

OR

95% CI

281 219

1 2.5

1.7, 3.7

100 100 100 100 100

1 1.6 1.4 2.3 2.5

0.9, 0.8, 1.3, 1.3,

P

< 0.001 1.2, 1.5 < 0.001 1.2, 1.6 1.4, 1.8 < 0.001

< 0.001

0.7, 0.8 < 0.001

2000 2000 2000 2000 2000

1 1.4 1.2 1.3 1.7

1.2, 1.0, 1.1, 1.4,

1.7 1.4 1.5 1.9

3.2.2. Complementary survey In the final model including only sociodemographic variables, response to the complementary survey was significantly greater in the following groups: non-salaried workers (unlike the initial survey) (OR, 2.5; 95% CI: [1.7, 3.7] for nonsalaried vs salaried workers), and persons living in the Saoˆneet-Loire area (OR, 2.5; 95% CI: [1.3, 4.5] for people living in Saoˆne-et-Loire vs people living in Bouches-du-Rhoˆne) (Table 1). Most of the paradata were significantly associated with the response to the complementary survey (see supplementary material part 2). After taking into account paradata and sociodemographic data in a multivariate analysis (Table 2), the response to the complementary survey was higher in the following groups: non-salaried workers (OR, 2.4; 95% CI: [1.6, 3.6] for non-salaried vs salaried workers), for persons who were not visited by an interviewer visit on a Saturday (OR, 2.2; 95% CI: [1.0, 5.2]) or for whom planned data collection mode was the phone mode and there were no switch compared with other persons (for example OR = 0.2, 95% CI [0.1, 0.3] for telephone mode planned and switch to face-to-face). No significant interaction was found.

0.02 2.9 2.5 4.3 4.5

3.3. Contribution of complementary survey and of paradata in reducing nonresponse bias 3.3.1. Contribution of the complementary survey For the six outcome variables from administrative databases the relative errors (Table 3) using the data from the respondents to the initial survey decreased after correcting for nonresponse using sociodemographic variables. The same results were observed for the combined surveys, except for the variable ‘‘hospitalizations’’. After nonresponse correction by sociodemographic variables, the differences between the prevalence values estimated with the initial survey and with the combination of the initial and complementary surveys differed according to the dependent variables considered (Table 3). Relative errors for ‘‘over 100 reimbursement claims for medical services’’ were high (RE = 22.3 for the initial survey; RE = 14.5 for the combined surveys), moderate for ‘‘hospitalizations’’ (RE = 9.6 for the initial survey; RE = 6.5 for the combined surveys), and ‘‘sickness absence’’ (RE = 5.3 for the initial survey; RE = 3.4 for the combined surveys) and low for the other variables. Nevertheless, irrespective of the variable

Table 2 Response to the complementary survey according to sociodemographic variables and paradata (final multiple logistic regression).

Employment status Salaried Non-salaried Interviewer visit for face-to-face interview on Saturdays Yes No Data collection evolution Phone at the beginning, phone at the end Phone at the beginning, face-to-face at the end Face-to-face at the beginning, face-to-face at the end Face-to-face at the beginning, phone at the end

n

OR

281 219

1 2.4

31 469

1 2.2

199 137 139 25

1 0.2 0.2 0.1

95% CI

P < 0.001

1.6, 3.6 0.045 1.0, 5.2 < 0.001 0.1, 0.3 0.1, 0.4 0.0, 0.2

10.7

3.2

5.3

3.3.2. Contribution of the paradata Irrespective of the outcome variable considered, the prevalences estimated from the combined survey when taking and not taking paradata into account (Table 3) were slightly different.

1.6 4.7

6.0

5.5

6.3

12.0

studied, the prevalence estimated for the combined surveys was always closer to the gold standard prevalence than that estimated for the initial survey. Furthermore, the gold standard prevalence was always included in the confidence intervals of the prevalence estimated in the combined surveys. It is worth noting that the confidence intervals for the combined surveys were at least twice as wide as those estimated for the initial survey.

1.7

3.4

10.1 4.8

1.5

0.6

1.2 2.9

6.2 3.8

1.1

6.5 4.3 9.6

5.1

8.6

14.5 15.7 22.3 29.4

(a-f) /f

32.0

39.4

62.9

68.9

24.2

33.0

37.5

60.0

74.9

26.0

28.4

37.9

55.9

61.0

24.7

27.4

38.0

55.4

59.3

25.2

Worker compensation for accident at work or occupational disease between 2002 and 2008

Sickness absence at work

Job duration less than 10 years

Primary economic activity

Hospitalization between 2008 and 2010

X: set of sociodemographic variables; Z: set of paradata; MCAR: missing completely at random; MAR: missing at random.

32.1

40.2

72.2

23.7

64.5

19.9 19.1, 20.7 20.9 20.1, 21.7 61.1 60.1,62.1 56.4 55.4, 57.4 33.4 33.0, 34.8 26.9 25.9, 27.9 26.9

22.0 17.1, 19.1 14.5, 64.2 56.2, 55.7 46.9, 33.2 26.2, 26.5 20.9, 27.9

22.8 17.7, 19.5 14.9, 61.8 54.7, 54.8 46.7, 32.8 26.2, 26.4 20.8, 29.6

23.1 16.5, 20.0 13.9, 64.9 54.9, 50.8 41.6, 30.3 23.1, 26.0 18.9, 26.2

24.3 22.5, 22.9 21.1, 58.8 56.7, 53.7 51.5, 35.8 33.6, 26.4 24.5, 27.6

25.8 23.9, 23.4 21.6, 57.2 55.1, 53.4 51.3, 36.0 34.0, 25.6 23.8, More than 100 reimbursement claims for medical services

% 95% CI

% 95% CI

% 95% CI % 95% CI % 95% CI

% 95% CI

(f) MAR(X,Z) (e) MCAR (c) MAR(X) (b) MCAR (a)

MAR(X) (d)

Respondents to the initial survey and the complementary survey (n = 2,618)

Complete sample (gold standard) (n = 9,358)

77

4. Discussion

Respondents to the initial survey (n = 2,321)

Table 3 Gold standard prevalence and estimated prevalences of administrative database variables under several assumptions on nonresponse processes.

Relative errors (%)

(d-f) /f (c-f) /f (b-f) /f

11.5

(e-f) /f

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

Our results suggest that although sociodemographic variables were associated with survey response, and despite the fact that numerous studies report that such variables are also related to health and occupation [1,24,25], they were not sufficient to correct for nonresponse bias in our initial survey. This is particularly true for the variable ‘‘over 100 reimbursement claims for medical services’’. Our results also showed that a complementary survey, with a protocol which aims to maximize the response rates, is useful to correct for nonresponse bias. Moreover, although paradata were strongly associated with response to the complementary survey, they were less useful in correcting for nonresponse after taking sociodemographic variables into account. The sociodemographic variables were collected in 2008, two years before the survey; this was not a limit for sex and age, and we considered that geographic area and employment status in 2008 were good markers of the situation of the person in 2010. The use of outcome variables coming from administrative databases is a real strength of our study at two levels: they enabled to define a prevalence gold standard and they allowed to specifically study nonresponse bias, independently of measurement bias. Administrative databases were already used in previous studies for this purpose [23,26]. Further analyses were conducted by using outcome variables coming from the questionnaire variables (see supplementary material part 3); results were globally the same as those coming from administrative data. Nevertheless, in the case of the outcome variables coming from questionnaire, gold standard prevalence were not available and the differences observed on the prevalences estimated with several assumptions on the nonresponse process could be due to nonresponse bias and/ or measurement bias. The response rate to the complementary survey (62.6%) was three times higher than that to the initial survey. This was to be expected since the response rate to the initial survey was particularly low [27,28] and demonstrates the efficacy of the complementary study’s protocol, which was designed to maximize the response rate. Even though the response rate to the complementary survey was much higher than that to the initial survey, this does not

78

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79

imply that the complementary survey may have been useful in correcting for nonresponse bias if the respondents of the complementary survey shared the same profile as their initial survey counterparts. The results showed that these groups shared certain sociodemographic characteristics but not all of them. For example, for the initial survey salaried workers were more likely to respond than non-salaried workers; given the size of the questionnaire (40 pages) and the data collection procedure (self-administered questionnaire), we can hypothesize that people with more time available (salaried workers in service company) were more likely to respond. For the complementary survey, non-salaried workers were more likely to respond than salaried workers; we can hypothesize that nonsalaried workers (farmers in the majority) move less often than salaried workers or that surveys with interviewers are more comfortable for them than self-administered questionnaires. However, respondents to the initial and the complementary surveys were probably different for others characteristics which we did not measure. This can be one of the reasons why some differences between the estimated prevalences remained, even after correction for nonresponse bias using sociodemographic variables. The contribution of the complementary survey must also be discussed in terms of its costs and its range for the confidence intervals estimated from the combined survey. In the CosetMSA study, the cost of collecting data from the 500 persons in the complementary survey and in the initial postal survey of 10,000 persons was the same, even though fewer variables were collected in the former. Concerning the range of the confidence intervals estimated from the combined survey, they were at least as twice as wide as those estimated in the initial survey. The major part of the loss in precision came from the sampling design, which led to very different weights for the respondents to the initial and complementary surveys, respectively (range 7 to 220). To reduce this range of weights, it would have been necessary to select a larger sample for the complementary survey, and the cost of the survey would have increased. Nevertheless, the loss of precision, which was already found in a similar epidemiological study [12], may be considered acceptable. As mentioned above, paradata were strongly associated with nonresponse in the present study. This unsurprising result has already been found in numerous studies [14,16]. It is important to remember that the variable ‘‘interviewer visits for face-toface interviews on a Saturday’’ is included in our final model. Our results support the recommendation of including indicators of time and day of interview in paradata [14]. The result of the univariate analysis also supports the recommendation of including the availability of a phone number [29]. However, to be really relevant, paradata also have to be strongly correlated with outcome variables but it is generally not the case [16]. In our study, the contribution of the paradata in correcting for the prevalence of the variables coming from health and occupational databases was weak, but this contribution was studied after a relevant correction for nonresponse bias on sociodemographic variables. If the prevalences estimated from the combined surveys had been

corrected for solely using paradata, their contribution would have been greater for certain variables, for example ‘‘over 100 reimbursement claims for medical services’’ and ‘‘job duration less than 10 years’’ (see supplementary material part 4). The use of paradata to correct for nonresponse is relatively recent and numerous questions about their exploitation remain [15]. Even though they were not relevant in the present study, paradata are nevertheless potentially useful auxiliary data to consider when correcting for nonresponse bias [16,30]. Collecting paradata is worthwhile especially as those paradata related to the data collection process have the advantage of usually being inexpensive once the system is set up. Even though our analysis showed the usefulness of the complementary survey, the MAR assumption, where the link between the variables of interest and nonresponse is completely explained by both sociodemographic variables and paradata, was not verified for all outcome variables coming from administrative databases. Consequently, a residual bias remained for some outcomes and probably also for questionnaire variables. Funding This study received funding only from Sante´ publique France, the national public health agency. Disclosure of interest The authors declare that they have no competing interest. Acknowledgements The authors thank the Mutualite´ Sociale Agricole personnel (Alain Pelc, Nicolas Viarouge, Florian Bre´maud, Yves Cosset) for their fruitful collaboration on the Coset-MSA project, Marie Zins and her Constances team for exchanges during the study, Marcel Goldberg and Ellen Imbernon for their fruitful comments on the manuscript and the Ipsos company which conducted the phone and face-to-face interviews. The authors thank Sante´ publique France, the national public health agency, which funds the study. The authors also thank Diane Cyr, Nina Crowte and Jude Sweeney for the editing and reviewing of the English version of the manuscript. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j. respe.2016.10.059. References [1] Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol 2007;17(9):643–53. [2] Stang A. Nonresponse research–an underdeveloped field in epidemiology. Eur J Epidemiol 2003;18(10):929–31. [3] Little RJA, Rubin DB. Statistical analysis with missing data. New York: Wiley; 1987.

G. Santin et al. / Revue d’E´pide´miologie et de Sante´ Publique 65 (2017) 71–79 [4] Bethlehem J. Weighting nonresponse adjustments based on auxiliary information. In: Groves RM, Dillman DA, Eltinge JL, Little RJA, editors. Survey nonresponse. New-York: Wiley; 2002. p. 275–88. [5] Groves RM, Heeringa SG. Responsive design for household surveys: tools for actively controlling survey errors and costs. J R Statist Soc A 2006;169:439–57. [6] Wagner J, West BT, Kirgis N, Lepkowski JM, Axinn WG, Kruger-Ndiaye S. Use of paradata in a responsive design framework to manage a field data collection. J Official Statistics 2012;28(4):477–99. [7] Hansen M, Hurwitz W. The problem of nonresponse in sample surveys. JASA 1946;41:517–29. [8] Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Q 2008;72(2):167–89. [9] Sa¨rndal CE, Swensson B, Wretman J. Nonresponse. Model assisted survey sampling. New-York: Springer; 1992: 556–600. [10] Deville JC, Sarndal CE. Calibration estimators in survey sampling. JASA 1992;87:376–82. [11] Sa¨rndal CE, Lundstro¨m S. Estimation in surveys with nonresponse. Chichester: Wiley-Blackwell; 2005. [12] Jenkins P, Scheim C, Wang JT, Reed R, Green A. Assessment of coverage rates and bias using double sampling methodology. J Clin Epidemiol 2004;57(2):123–30. [13] Beaumont JF. On the use of data collection process information for the treatment of unit nonresponse through weight adjustment. Survey Methodol 2005;31(2):227–31. [14] Kreuter F, Kohler U. Analysing contact sequences in call record data. Potential and limitations of sequence indicators for nonresponse adjustments in the European Social Survey. J Official Statistics 2009;25(2):203–26. [15] Kreuter F. Improving surveys with paradata: analytic uses of process information. Hoboken: Wiley; 2013. [16] Olson K. Paradata for nonresponse adjustment. Ann Am Acad Political Soc Sc 2013;645:142–70. [17] Santin G, Geoffroy B, Be´ne´zet L, Dele´zire P, Chatelot J, Sitta R, et al. In an occupational health surveillance study, auxiliary data from administrative health and occupational databases effectively correct for nonresponse. J Clin Epidemiol 2014;67:722–30.

79

[18] Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health 2005;27(3):281–91. [19] Eltinge JL, Yansaneh IS. Diagnostics for formation of nonresponse adjustment cells, with an application to income nonresponse in the U.S. consumer expenditure survey. Survey Methodol 1997;23:33–40. [20] Haziza D, Beaumont JF. On the construction of imputation classes in surveys. Int Stat Rev 2007;75(1):25–43. [21] Little RJA. Survey nonresponse adjustments for estimates of means. Int Stat Rev 1986;54:139–57. [22] Vercambre MN, Gilbert F. Respondents in an epidemiologic survey had fewer psychotropic prescriptions than nonrespondents: an insight into health-related selection bias using routine health insurance data. J Clin Epidemiol 2012;65(11):1181–9. [23] Olson K. Survey participation, nonresponse bias. Measurement error bias, and total bias. Public Opin Q 2006;70(5):737–58. [24] Goldberg M, Chastang JF, Leclerc A, Zins M, Bonenfant S, Bugel I, et al. Socioeconomic, demographic, occupational, and health factors associated with participation in a long-term epidemiologic survey: a prospective study of the French GAZEL cohort and its target population. Am J Epidemiol 2001;154(4):373–84. [25] Goldberg M, Luce D. Selection effects in epidemiological cohorts: nature, causes and consequences. Rev Epidemiol Sante Publ 2001;49(5): 477–92. [26] Kreuter F, Muller G, Trappmann M. Nonresponse and measurement error in employment research. Making use of administrative data. Public Opin Q 2010;74(5):880–906. [27] Levy PS, Lemeshow S. Sampling of populations, methods and applications. 2nd ed, New York: Wiley & Sons; 1991. [28] Stoop IAL, Billiet J, Koch A, Fitzgerald R. Improving survey response. Lessons learned from the European Social Survey. Chichester: Wiley; 2010. [29] Bethlehem JG, Cobben F, Schouten B. Handbook of nonresponse in household surveys. Hoboken: Wiley; 2011. [30] Maitland A, Casas Cordero C, Kreuter F. An exploration into the use of paradata for nonresponse adjustment in a health survey; 2008.