- Email: [email protected]
A conditional probability approach to surveillance system sensitivity assessment
ARTICLE IN PRESS Public Health (2008) 122, 53–60
www.elsevierhealth.com/journals/pubh
Original Research
A conditional probability approach to surveillance system sensitivity assessment R. Majdzadeh, F. Pourmalek Epidemiology and Biostatistics Department, School of Public Health and Institute of Public Health Research, Tehran University of Medical Sciences, Tehran, Iran Available online 24 July 2007
KEYWORDS Epidemiological studies; Health services research; Iran; Population surveillance; Sensitivity
Summary Objective: To determine the sexually transmitted diseases (STD) surveillance system sensitivity with a conditional probability approach at district level in Darregaz, a frontier town in the north of Iran. Study design: A cross-sectional survey. Methods: We used a sample survey of sexually active inhabitants for proxy measurement of the medical service utilization pattern for STD, and interviews with all practitioners to determine their knowledge of STD diagnosis and attitude towards STD reporting as proxy measures of actual STD diagnosis and reporting, respectively. Point estimates of the STD surveillance system sensitivity for each of the health service sectors were derived from multiplying the three proxy measures of sensitivity determinants, i.e., utilization, diagnosis, and reporting, as conditional probabilities. Estimates of sensitivity for all health service sectors were summed to obtain the overall sensitivity. Results: The sensitivity of the surveillance system was 21.2% (95% confidence interval (CI) 15.5–25.3%) for detecting symptomatic STD. Of the sexually active inhabitants, 8.9% (95% CI 5.5–14.2%) did not use health services if they contracted STDs. The public health sector’s contribution to overall sensitivity (59.6%) was greater than its proportion of service utilization for STD (45.3%). Conclusions: The strengths of the conditional probability approach are feasibility of conducting necessary surveys, decomposing sensitivity into its determinants, and providing evidence for intervention at different points for planning purposes. This approach tends to overestimate the overall sensitivity. & 2007 The Royal Institute of Public Health. Published by Elsevier Ltd. All rights reserved.
Abbreviations: CDC, Centers for Disease Control and Prevention; CI, confidence interval; PCA, principal components analysis; STD, sexually transmitted diseases Corresponding author. Tel.: +98 21 66491070; fax: +98 21 66419537. E-mail address: [email protected] (R. Majdzadeh). 0033-3506/$ - see front matter & 2007 The Royal Institute of Public Health. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.puhe.2007.04.011
ARTICLE IN PRESS 54
R. Majdzadeh, F. Pourmalek
Introduction
Methods
In an ideal surveillance system, all cases in the population would be detected. In practice, depending on several factors (which will be addressed later), some of those who have the disease will not be detected as cases. This is due to the lack of full sensitivity of surveillance systems. Assessing the sensitivity of a surveillance system illustrates the degree of under-detection. The Centers for Disease Control and Prevention (CDC) publication Guidelines for evaluating surveillance systems suggests describing the public health importance of health events under surveillance, describing the system to be evaluated, indicating the utilization of the system, addressing the attributes of the system, describing the resources used to operate the system, and drawing conclusions and recommendations. Among the system attributes to be addressed is sensitivity – the proportion of all health events of interest that are to be captured by the system.1 The estimation of surveillance sensitivity can be complicated by the absence of an appropriate gold standard. A year 2000 review of 31 articles showed that gold standards were medical charts, records, or logs (35%); registries (29%); estimations of total cases (19%); other reporting systems, including vital statistics (10%); unduplicated databases from multiple sources (6%); telephone interviews (6%); enhanced surveillance, such as follow-up of cases (6%); and findings from physical examination (3%).2 The capture–recapture method3 and modelling4 have also been used for the same purpose. Both of these methods are based on assumptions. The capture– recapture method is rarely used for peripheral-level health systems. Modelling methods are more suitable for research purposes rather than for fieldwork. The sensitivity of a disease surveillance system depends on whether people with the condition seek medical care, the disease is correctly diagnosed, and finally, the diagnosed cases are reported.5 The main idea of this study was to use the survey data on these factors – the three conditional probabilities – for computing the sensitivity of a surveillance system. Estimation of system sensitivity by this method can easily be undertaken at the field level. The surveillance of sexually transmitted diseases (STD) was selected for the application of this methodology, since under-reporting in STD surveillance systems is a well-known problem.6 Darregaz, a frontier town adjacent to Turkmenistan in the north of Iran, was selected because of the movement of young people across the border and the probable consequences of such movement on STD.
For determining STD surveillance sensitivity, three components of sensitivity were estimated: (1) health services utilization by sexually active inhabitants, (2) the probability of a correct diagnosis among patients who utilize health services, and (3) the probability of the diagnosed cases being reported. These three components were approximated by three proxy measurements: (1) people’s survey: the response of sexually active inhabitants of the town to the question ‘‘Where do you think people would go to seek care if they contracted an STD and/or for their main complaints in Darregaz?’’; (2) practitioners’ census: the mean percent of practitioners’ knowledge about the diagnosis of STD; and (3) the mean percent of practitioners’ positive attitude towards reporting the diagnosed STD cases.
People’s survey In July 2004, a proportional stratified random sample (from different municipality zones in Darregaz) of inhabitants aged 15–50 years was surveyed to find the service utilization pattern. Within the 190 intended persons in the sample, non-respondents were substituted from the same sampling scheme. STD and their complications were described during the interviews. Test–retest reliability of the people’s questionnaire was assessed with 112 peer respondents in Bandar Abbas, a port in the south of Iran. The retest phase was conducted 1 week after the initial test phase.
Practitioners’ census Practitioners’ knowledge of STD and their attitude towards reporting STD cases were assessed with a pre-tested self-administered questionnaire. The census frame included all the general physicians, all the specialists to whom a STD patient would refer, and all the midwives, practicing in Darregaz. Internal consistency, reliability of knowledge and attitude questions were assessed separately by Cronbach’s a before and after pre-testing. Pre-test questions whose omission would increase the a were dropped until the internal consistency reliability was maximized. Principal components analysis (PCA) was used to derive binary summary variables of knowledge of STD diagnosis and attitude towards reporting cases. Two groups of questions on practitioners’ knowledge of STD diagnosis and questions on their
ARTICLE IN PRESS Surveillance system sensitivity assessment attitude towards reporting STD cases were treated separately to construct binary summary variables of knowledge and attitude. Before PCA in each group, questions whose deletion would increase Cronbach’s a were dropped. Then, factors that equalled to the remaining questions in number were extracted with PCA. The percent of variance explained by each extracted factor was multiplied by the factor value and the results were summed to form the summary numerical variable for knowledge or attitude. Values of the knowledge summary numerical variable for all respondents were then summed with the absolute value of the lowest figure (for the same variable) to make all the values positive. The same was done for the attitude summary numerical variable. The values of these summary numerical variables for a hypothetical respondent practitioner who would give correct answers to all the knowledge questions and with a positive attitude were taken as full knowledge and completely positive attitude, respectively. The knowledge summary numerical variable of each respondent divided by the maximum attainable value for the knowledge summary numerical variable, gave each respondent’s knowledge as a percentage. The mean value for this percentage in each health sector gave the proxy measure of correct diagnosis for practitioners in that sector. The percent of practitioners reporting STD cases was derived similarly from the attitude summary variable.
Point estimation of sensitivity Point estimates of sensitivity of the STD surveillance system within each of the three health sectors were calculated by multiplying the percentage of each sector’s service utilization (from the people’s survey), the probability of correct diagnosis by the sector’s practitioners (from the practitioners’ census and PCA), and the probability of case reporting by the sector’s practitioners (from the practitioners’ census and PCA). These point estimates of sensitivity within each health sector were then summed to obtain the total sensitivity: Total sensitivity X X X ¼ Si ¼ ½ðUi = Ui ÞDi Ri , where i ¼ different health sectors (including no use of health services, e.g., self-treatment, use of local herbal remedies, etc.); Si ¼ sensitivity provided by a sector (i.e., proportion of patients captured in a sector among all patients in the community), P Si ¼ (Ui/ Ui) Di Ri; Ui ¼ probability of health care utilization in sector i; Di ¼ probability of
55 correct diagnosis in sector i; Ri ¼ probability of reporting patients, conditional on correct diagnosis in sector i. In this formula, Di and Ri probabilities in each sector are relatively independent from respective Di and Ri in other sectors. For example, D1 can range from 0 to 1, relatively independent of the probability of correct diagnosis in other sectors (D2 or D3). The same holds true for Ri, but not for Ui. The sum of utilization of different sectors cannot exceed one. This is the reason P why the related term in the general formula is (Ui/ Ui) and not Ui.
Uncertainty, sensitivity, and counterfactual analyses Interval estimates of the three sensitivity components give the confidence intervals (CI) for the point estimate of sensitivity. In order to assess the sensitivity of the main findings to the probable overestimation information bias in proxy measurement of the determining components of surveillance sensitivity (discussed below), calculations were repeated with values of the three components of surveillance sensitivity being set 5–20% lower than the observed values in all health sectors. Then the calculated new values for the surveillance sensitivity were compared with their original values. For planning purposes, a ‘what-if’ scenario was developed. For this counterfactual scenario, a 5% improvement was assumed in each of the three components of surveillance sensitivity. Then the resultant changes in ‘sector sensitivity’ and ‘total sensitivity’ were determined.
Ethics The study protocol was approved by the institutional ethics review board of Tehran University of the Medical Sciences.
Results Of 49 midwives and general and specialist physicians in Darregaz, 35 practitioners (71.4%) participated in the study; eight (22.9%) worked primarily in the public sector, 16 (45.7%) worked primarily in the private sector and 11 (31.4%) worked in both sectors. Of the 49 intended practitioners, 14 (28.6%) did not respond after three follow-ups (four specialists, four general physicians and six midwives).
ARTICLE IN PRESS 56
R. Majdzadeh, F. Pourmalek
Test–retest reliability of the people’s questionnaire: intra-cluster correlation of test and retest values of the single question in this questionnaire was 0.98 (95% CI 0.97–0.99). Internal consistency reliability of the practitioners’ questionnaire: for seven questions on the knowledge of practitioners about the correct diagnosis of STD, Cronbach’s a was 0.46, which increased to 0.60 after dropping one of them. Further dropping of questions did not increase a; and thus, six knowledge questions were further analysed. Cronbach’s a for six questions on the attitude of practitioners towards reporting STD cases was 0.22; this rose to 0.42 after the deletion of one question. Further dropping of questions did not increase a, and five attitude questions were further analysed.
Table 1
Determinants of sensitivity The health service utilization pattern of people aged 15–50 years in Darregaz is shown in Table 1. Those who did not use health services went directly to pharmacies, consulted with acquaintances, or used local herbal remedies, and therefore, did not come to the notice of the surveillance system. The mixed private–public sector in the utilization pattern relates to the responses of those people who mentioned they would go and see a physician or midwife, but they did not have a preference for utilizing private and/or public health services. The assessment of practitioners’ knowledge about the correct diagnosis of STD, and their attitude towards reporting the diagnosed STD cases are shown in Table 2.
Health service utilization pattern of people aged 15–50 years in cases of STD in Darregaz in July 2004.
Utilization pattern
Frequency
Proportion (%)
95% CI
Public sector Private sector Mixed private–public No. use of health services
86 44 43 17
45.3 23.2 22.6 8.9
38.1–52.6 17.5–29.9 17.0–29.4 5.5–14.2
Total
190
100.0
CI, confidence interval.
Table 2 Point estimates and 95% confidence intervals for STD surveillance system sensitivity in Darregaz in July 2004 by different health sectors (all numbers are percentages). Estimate
Health sector
Utilization (Ui)
Diagnosis (Di)
Reporting (Ri)y
Sector sensitivity (Si)
Contribution of sector to total sensitivity
Point estimate
Public Private sector Mixed private–public
45.3 23.2 22.6
50.9 33.5 45.1
54.9 50.2 45.7
12.6 3.9 4.7
59.6 18.4 22.0
Lower 95% CI
Public Private sector Mixed private–public
38.1 17.5 17.0
39.8 22.0 33.3
44.1 19.2 36.2
6.7 0.7 2.0
70.5 7.8 21.6
Upper 95% CI
Public Private sector Mixed private–public
52.6 29.9 29.4
62.0 45.2 56.9
65.7 41.2 55.2
21.4 5.6 9.2
59.2 15.4 25.4
CI, confidence interval.
Using knowledge of practitioners about correct diagnosis of STD as a proxy. y
Using attitude of practitioners towards reporting of STD cases as a proxy.
ARTICLE IN PRESS Surveillance system sensitivity assessment
Surveillance sensitivity Table 2 also gives the calculations of surveillance sensitivity point estimates and 95% CI by different sectors with the formula described earlier. The right-hand column in Table 2 illustrates the percentage of contribution of private and public sectors in STD case detection in Darregaz in 2004. The ‘total sensitivity’ of the surveillance system was 21.2% (95% CI 15.5–25.3) for detecting symptomatic STD – the sum of percentages of patients found in the population by different health sectors. Figure 1 illustrates the template for computing the three conditional probabilities and the final point estimate of STD surveillance sensitivity in Darregaz in July 2004.
Sensitivity analysis Results of the sensitivity analysis for the main study findings with regard to the probable information bias in proxy measurement of the determining components of surveillance sensitivity (discussed below) showed that when calculations were repeated with values of the three components of surveillance sensitivity set 5% and 20% lower than the observed values in all health sectors, the new surveillance sensitivity values dropped by 0.9 and
57 0.5 of their original values, respectively (18.2% and 10.9% total sensitivity).
Counterfactual analysis Table 3 shows the results of the counterfactual scenario of 5% improvement in each component of utilization, diagnosis, and reporting. All computations are tabulated for further clarification, and the notations are identical with those for the aforementioned formulae. The 5% improvement in each of the three components of sensitivity, within the public sector for example, would result in a 1.9% increase in the total sensitivity. This 1.9% was computed from the difference in values of the total sensitivity from the actual present situation (12.7%) and from the scenario of counterfactual 5% increase in each of the three components (14.6%).
Discussion The 21.2% total sensitivity (95% CI 15.5–25.3%) of the STD surveillance system in Darregaz means that to estimate the actual number of symptomatic STD cases in general, the number of cases detected by the surveillance system should be multiplied by a correction factor of 4.7 (95% CI 3.9–5.9%). Based on
Figure 1 Template for computing the three conditional probabilities and final sensitivity estimate for the STD surveillance system in Darregaz in July 2004.
ARTICLE IN PRESS
1.9 0.6 0.7 3.2 14.6 4.5 5.4 24.4y 12.7 3.9 4.7 21.2y 0.6 0.2 0.2 1.06
Total sensitivity calculated with the formula described in the text. y
45.3 23.2 22.6 91.0 Public Private Mixed Total
Weighted mean. Weights are the proportion of practitioners in each sector among all practitioners.
57.6 52.7 48.0 52.4 54.9 50.2 45.7 49.9a 0.6 0.2 0.2 1.06 53.4 35.2 47.4 43.2 50.9 33.5 45.1 41.1 0.6 0.2 0.2 1.06
Present value Health sector
47.6 24.4 23.7 95.7
Present value Present value Present value
Utilization Sensitivity determinants
5% increase in utilization
Resultant increase in sensitivity
Diagnosis
5% increase in diagnosis
Resultant increase in sensitivity
Reporting
5% increase in reporting
Resultant increase in sensitivity
Sensitivity
Improved value of sensitivity
Resultant increase in sensitivity
R. Majdzadeh, F. Pourmalek Table 3 Impact of a 5% improvement in each determinant factor on STD surveillance sensitivity in Darregaz (baseline sensitivity is 21.2%; all numbers are percentages; differences at the level of 0.1 are the result of rounding).
58
the results illustrated in Figure 1, 8.9% of people did not get appropriate care for their STD and subsequently did not have a chance of being detected and reported by the surveillance system. Of course, it is expected that a small fraction of those going directly to pharmacies are diagnosed (i.e., Di40) and receive antibiotics and may be (partially) treated, but again will escape the surveillance system. The situation for the private sector practitioners was worse than for the public sector. Although the former group comprised 45.7% of all responding practitioners in the town and 23.2% of people said that they would solely use the private sector services, their contribution to case finding was only 18.4%. The major strength of our conditional probability approach to the assessment of surveillance sensitivity is manifest in the straightforward decomposition of sensitivity to its three determining components that provides the health policy makers and managers with the necessary information for intervention planning. Results of the counterfactual scenario of a given 5% improvement in each of service utilization, diagnosis, and reporting components, show that investing in such improvements in the public health sector would result in an increase in the total surveillance sensitivity (1.9%), which would be more than three times the result for the private sector (0.6%). Of course, such information is not sufficient for choosing the most appropriate interventions, and decisions must be based on the economic evaluation of each intervention as well as other due considerations, such as feasibility, policy, equity, and ethical issues, etc. The obtained information is quite useful for the purpose of counterfactual analysis. In this study, a greater contribution of the public sector to case finding (the right-hand column in Table 2) resulted in better subsequent provisional total sensitivity (the right-hand column in Table 3) with the same percentage of improvement. The other advantage of the ‘conditional probability’ method presented in this paper is its applicability in the field setting, even for rare events. The accuracy of the conditional probability method is undeterminable due to the lack of a gold standard for comparison. In an article that reviewed 33 studies in the USA,7 two main distinct methods were described. These two approaches were named ‘uncorrected’ and ‘under-ascertainment corrected’ methods. The authors described the former method as dividing the reported cases by the total number of cases detected by active case finding and the use of supplementary data from other sources. This approach does not account for the cases that are left undetected in all
ARTICLE IN PRESS Surveillance system sensitivity assessment sources. That is why the method is called ‘uncorrected’; it is prone to underestimating the number of real cases in the community and, conversely, overestimating the sensitivity of the surveillance system. The second method that had been used in one-third of the papers reviewed was capture– recapture, which is an effort to correct for this under-reporting (hence the ‘under-ascertainment corrected’ method).7 The main assumption in this method is the independency of data sources.8 The former approach can be applied if this assumption is violated, but it makes the computations more complex.9–11 However, in contrast to the approach that has been used in our study, neither of these two methods is applicable to the decomposing of surveillance sensitivity into its determining factors. The end result of both ‘uncorrected’ and ‘underascertainment corrected’ methods is just the value of surveillance sensitivity per se, without decomposing it into its determining factors. The conditional probability method is prone to two kinds of systematic error in the estimation of sensitivity of the surveillance system. The first one is the selection bias due to choosing a sample of the general population instead of ill people who actually suffered from the condition. If the affecting condition, STD in this study, were dependent on the service utilization pattern, then the result from the general population would be different from that for real patients. Using three proxy measurements instead of direct measures makes this method prone to information bias. Asking people about their preference in seeking medical care as well as the knowledge and attitude of practitioners regarding STD could lead to overestimation of real values of interest. The sensitivity analysis performed shows that the main finding of ‘21.2% total sensitivity’ is modestly sensitive to these upward measurement biases (116–195% overestimation of the total sensitivity, if the components of sensitivity are 5–20% overestimated). As far as these measurement biases affect the three components in the three health sectors to similar degrees, the proportion of each sector’s contribution to the total sensitivity does not change. The same holds true for the proportions of gain in total sensitivity subsequent to improvements in the utilization, diagnosis, and reporting in different sectors. But if the biases affect the three components very differently in the three sectors, these two groups of proportions will be less valid. Another important issue is the unsuitability of this method for asymptomatic conditions. The result of this study can be used for symptomatic
59 STD, e.g., gonorrhoea, but is not appropriate for the estimation of asymptomatic or sub-clinical STD cases, like chlamydia, or, more importantly, HIV positivity. Of course, if the condition is asymptomatic, then people do not feel the need to seek medical care. However, the latter two parts of sensitivity calculations (i.e., health care providers’ knowledge and attitude), would still be applicable. In this approach, the absolute value of sensitivity is not as important as the expected effect of different points of interventions for improving it, and comparisons can be made for the pertinent costs of each intervention. In this way, one of the responsibilities of public health agencies at the peripheral levels (in districts and in provinces – based on the administrative health structure of the country) can be elaborated: evaluation of surveillance and planning for its improvement. Few studies have described the factors that influence the reporting behaviour of health care providers. These factors are described as lack of awareness of the legal requirement for reporting, lack of knowledge of which diseases are reportable, lack of understanding of how or to whom to report, assuming that someone else will report the case, intentional failure to report to protect patient privacy, insufficient rewards for reporting,7,12 and interference of the reporting task with the daily clinical practice.13 One article reports that a lack of appropriate feedback and a belief that no useful action was taken after reporting could be important factors in the failure to report notifiable diseases.14 Interventional studies have had limited success in decreasing these problems in the final reporting performance of providers,7,15 but some have shown that even assigning a nurse for the task and the introduction of a ward notification register could greatly improve notification rates.14 Although the CDC published the Guidelines for evaluating surveillance systems in 1988,1 only 31 studies have been covered in a 2000 review article,2 even though the search strategy for identifying the relevant literature in this review article was reasonable. Of course, not all evaluations are published in peer-reviewed journals, but an alternative explanation for this paucity of reports is that the assessment of surveillance systems sensitivity is not a frequent practice among public health professionals. On the other hand, the definition of essential functions of public health (assessment of community health, policy development based on community diagnosis, and assurance of coverage and quality of services) by the Institute of Medicine in 1998 emphasizes the role of surveillance in health practice.16 On the whole, it seems that sensitivity assessment is not quite
ARTICLE IN PRESS 60 feasible as a usual task in public health practice. Surveillance systems require ongoing evaluation if the information they provide is to accurately portray the conditions in the community for further action. The result of most complicated methods of surveillance sensitivity estimation is a number that is not decomposable to its constituents, and thus is not helpful in finding evidence on which to base appropriate intervention(s). These facts, along with the ease of conducting necessary surveys and the final decomposing of sensitivity constituents, make the present method relatively applicable at the field level. Although there is some overestimation of sensitivity, it is valid for prioritizing the points for intervention, and reliable for tracking changes over time.
R. Majdzadeh, F. Pourmalek
4.
5.
6.
7.
8.
9.
Acknowledgements This study was financially supported by the Institute of Public Health Research, Tehran University of Medical Sciences. We thank Dr. Ebrahim Khajeh and Dr. Ali Beheshtian for valuable support provided during data collection.
10.
11.
12.
Competing interests The authors declare that they have no competing interests. 13.
References 1. Centers for Disease Control and Prevention. Guidelines for evaluating surveillance systems. MMWR Morb Mortal Wkly Rep 1988;37(Suppl 5):1–18. 2. German RR. Sensitivity and predictive value positive measurements for public health surveillance systems. Epidemiology 2000;11:720–7. 3. Gallay A, Nardone A, Vaillant V, Desenclos JC. [The capture–recapture applied to epidemiology: principles,
14.
15.
16.
´pide limits and application]. Rev E ´miol Sante ´ Publique 2002;50:219–32. Bailey TC, Carvalho MS, Lapa TM, Souza WV, Brewer MJ. Modeling of under-detection of cases in disease surveillance. Ann Epidemiol 2005;15:335–43. Romaguera RA, German RR, Klaucke DN. Evaluating public health surveillance. In: Teusch S, Churchill E, editors. Principles and practice of public health surveillance. New York: Oxford University Press; 1994. p. 176–92. Reintjes R, Termorshuizen F, van de Laar MJ. Assessing the sensitivity of STD surveillance in the Netherlands: an application of the capture–recapture method. Epidemiol Infect 1999;122:97–102. Doyle TJ, Glynn MK, Groseclose SL. Completeness of notifiable infectious disease reporting in the United States: an analytical literature review. Am J Epidemiol 2002; 155:866–74. Hook EB, Regal RR. Capture–recapture methods in epidemiology: methods and limitations. Epidemiol Rev 1995;17: 243–64. Nardone A, Decludt B, Jarraud S, Etienne J, Hubert B, Infuso A, Gallay A, Desenclos JC. Repeat capture–recapture studies as part of the evaluation of the surveillance of Legionnaires’ disease in France. Epidemiol Infect 2003;131:647–54. Hook EB, Regal RR. Accuracy of alternative approaches to capture–recapture estimates of disease frequency: internal validity analysis of data from five sources. Am J Epidemiol 2000;152:771–9. Brenner H. Application of capture–recapture methods for disease monitoring: potential effects of imperfect record linkage. Methods Inf Med 1994;33:502–6. St. Lawrence JS, Montano DE, Kasprzyk D, Phillips WR, Armstrong K, Leichliter JS. STD screening, testing, case reporting, and clinical and partner notification practices: a national survey of US physicians. Am J Public Health 2002; 92:1784–8. Figueiras A, Lado E, Fernandez S, Hervada X. Influence of physicians’ attitude on under-notifying infectious diseases: a longitudinal study. Public Health 2004;118:521–6. Seneviratne SL, Gunatilake SB, de Silva HJ. Reporting notifiable diseases: methods for improvement, attitude and community outcome. Trans R Soc Trop Med Hyg 1997; 91:135–7. Weiss BP, Strassburg MA, Fannin SL. Improving disease reporting in Los Angeles County: trial and results. Public Health Rep 1988;103:415–21. Buehler JW. Surveillance. In: Rothman KJ, Greenland S, editors. Modern epidemiology. 2nd ed. Philadelphia: Lippincott-Raven Publishers; 1998. p. 435–57.
www.elsevierhealth.com/journals/pubh
Original Research
A conditional probability approach to surveillance system sensitivity assessment R. Majdzadeh, F. Pourmalek Epidemiology and Biostatistics Department, School of Public Health and Institute of Public Health Research, Tehran University of Medical Sciences, Tehran, Iran Available online 24 July 2007
KEYWORDS Epidemiological studies; Health services research; Iran; Population surveillance; Sensitivity
Summary Objective: To determine the sexually transmitted diseases (STD) surveillance system sensitivity with a conditional probability approach at district level in Darregaz, a frontier town in the north of Iran. Study design: A cross-sectional survey. Methods: We used a sample survey of sexually active inhabitants for proxy measurement of the medical service utilization pattern for STD, and interviews with all practitioners to determine their knowledge of STD diagnosis and attitude towards STD reporting as proxy measures of actual STD diagnosis and reporting, respectively. Point estimates of the STD surveillance system sensitivity for each of the health service sectors were derived from multiplying the three proxy measures of sensitivity determinants, i.e., utilization, diagnosis, and reporting, as conditional probabilities. Estimates of sensitivity for all health service sectors were summed to obtain the overall sensitivity. Results: The sensitivity of the surveillance system was 21.2% (95% confidence interval (CI) 15.5–25.3%) for detecting symptomatic STD. Of the sexually active inhabitants, 8.9% (95% CI 5.5–14.2%) did not use health services if they contracted STDs. The public health sector’s contribution to overall sensitivity (59.6%) was greater than its proportion of service utilization for STD (45.3%). Conclusions: The strengths of the conditional probability approach are feasibility of conducting necessary surveys, decomposing sensitivity into its determinants, and providing evidence for intervention at different points for planning purposes. This approach tends to overestimate the overall sensitivity. & 2007 The Royal Institute of Public Health. Published by Elsevier Ltd. All rights reserved.
Abbreviations: CDC, Centers for Disease Control and Prevention; CI, confidence interval; PCA, principal components analysis; STD, sexually transmitted diseases Corresponding author. Tel.: +98 21 66491070; fax: +98 21 66419537. E-mail address: [email protected] (R. Majdzadeh). 0033-3506/$ - see front matter & 2007 The Royal Institute of Public Health. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.puhe.2007.04.011
ARTICLE IN PRESS 54
R. Majdzadeh, F. Pourmalek
Introduction
Methods
In an ideal surveillance system, all cases in the population would be detected. In practice, depending on several factors (which will be addressed later), some of those who have the disease will not be detected as cases. This is due to the lack of full sensitivity of surveillance systems. Assessing the sensitivity of a surveillance system illustrates the degree of under-detection. The Centers for Disease Control and Prevention (CDC) publication Guidelines for evaluating surveillance systems suggests describing the public health importance of health events under surveillance, describing the system to be evaluated, indicating the utilization of the system, addressing the attributes of the system, describing the resources used to operate the system, and drawing conclusions and recommendations. Among the system attributes to be addressed is sensitivity – the proportion of all health events of interest that are to be captured by the system.1 The estimation of surveillance sensitivity can be complicated by the absence of an appropriate gold standard. A year 2000 review of 31 articles showed that gold standards were medical charts, records, or logs (35%); registries (29%); estimations of total cases (19%); other reporting systems, including vital statistics (10%); unduplicated databases from multiple sources (6%); telephone interviews (6%); enhanced surveillance, such as follow-up of cases (6%); and findings from physical examination (3%).2 The capture–recapture method3 and modelling4 have also been used for the same purpose. Both of these methods are based on assumptions. The capture– recapture method is rarely used for peripheral-level health systems. Modelling methods are more suitable for research purposes rather than for fieldwork. The sensitivity of a disease surveillance system depends on whether people with the condition seek medical care, the disease is correctly diagnosed, and finally, the diagnosed cases are reported.5 The main idea of this study was to use the survey data on these factors – the three conditional probabilities – for computing the sensitivity of a surveillance system. Estimation of system sensitivity by this method can easily be undertaken at the field level. The surveillance of sexually transmitted diseases (STD) was selected for the application of this methodology, since under-reporting in STD surveillance systems is a well-known problem.6 Darregaz, a frontier town adjacent to Turkmenistan in the north of Iran, was selected because of the movement of young people across the border and the probable consequences of such movement on STD.
For determining STD surveillance sensitivity, three components of sensitivity were estimated: (1) health services utilization by sexually active inhabitants, (2) the probability of a correct diagnosis among patients who utilize health services, and (3) the probability of the diagnosed cases being reported. These three components were approximated by three proxy measurements: (1) people’s survey: the response of sexually active inhabitants of the town to the question ‘‘Where do you think people would go to seek care if they contracted an STD and/or for their main complaints in Darregaz?’’; (2) practitioners’ census: the mean percent of practitioners’ knowledge about the diagnosis of STD; and (3) the mean percent of practitioners’ positive attitude towards reporting the diagnosed STD cases.
People’s survey In July 2004, a proportional stratified random sample (from different municipality zones in Darregaz) of inhabitants aged 15–50 years was surveyed to find the service utilization pattern. Within the 190 intended persons in the sample, non-respondents were substituted from the same sampling scheme. STD and their complications were described during the interviews. Test–retest reliability of the people’s questionnaire was assessed with 112 peer respondents in Bandar Abbas, a port in the south of Iran. The retest phase was conducted 1 week after the initial test phase.
Practitioners’ census Practitioners’ knowledge of STD and their attitude towards reporting STD cases were assessed with a pre-tested self-administered questionnaire. The census frame included all the general physicians, all the specialists to whom a STD patient would refer, and all the midwives, practicing in Darregaz. Internal consistency, reliability of knowledge and attitude questions were assessed separately by Cronbach’s a before and after pre-testing. Pre-test questions whose omission would increase the a were dropped until the internal consistency reliability was maximized. Principal components analysis (PCA) was used to derive binary summary variables of knowledge of STD diagnosis and attitude towards reporting cases. Two groups of questions on practitioners’ knowledge of STD diagnosis and questions on their
ARTICLE IN PRESS Surveillance system sensitivity assessment attitude towards reporting STD cases were treated separately to construct binary summary variables of knowledge and attitude. Before PCA in each group, questions whose deletion would increase Cronbach’s a were dropped. Then, factors that equalled to the remaining questions in number were extracted with PCA. The percent of variance explained by each extracted factor was multiplied by the factor value and the results were summed to form the summary numerical variable for knowledge or attitude. Values of the knowledge summary numerical variable for all respondents were then summed with the absolute value of the lowest figure (for the same variable) to make all the values positive. The same was done for the attitude summary numerical variable. The values of these summary numerical variables for a hypothetical respondent practitioner who would give correct answers to all the knowledge questions and with a positive attitude were taken as full knowledge and completely positive attitude, respectively. The knowledge summary numerical variable of each respondent divided by the maximum attainable value for the knowledge summary numerical variable, gave each respondent’s knowledge as a percentage. The mean value for this percentage in each health sector gave the proxy measure of correct diagnosis for practitioners in that sector. The percent of practitioners reporting STD cases was derived similarly from the attitude summary variable.
Point estimation of sensitivity Point estimates of sensitivity of the STD surveillance system within each of the three health sectors were calculated by multiplying the percentage of each sector’s service utilization (from the people’s survey), the probability of correct diagnosis by the sector’s practitioners (from the practitioners’ census and PCA), and the probability of case reporting by the sector’s practitioners (from the practitioners’ census and PCA). These point estimates of sensitivity within each health sector were then summed to obtain the total sensitivity: Total sensitivity X X X ¼ Si ¼ ½ðUi = Ui ÞDi Ri , where i ¼ different health sectors (including no use of health services, e.g., self-treatment, use of local herbal remedies, etc.); Si ¼ sensitivity provided by a sector (i.e., proportion of patients captured in a sector among all patients in the community), P Si ¼ (Ui/ Ui) Di Ri; Ui ¼ probability of health care utilization in sector i; Di ¼ probability of
55 correct diagnosis in sector i; Ri ¼ probability of reporting patients, conditional on correct diagnosis in sector i. In this formula, Di and Ri probabilities in each sector are relatively independent from respective Di and Ri in other sectors. For example, D1 can range from 0 to 1, relatively independent of the probability of correct diagnosis in other sectors (D2 or D3). The same holds true for Ri, but not for Ui. The sum of utilization of different sectors cannot exceed one. This is the reason P why the related term in the general formula is (Ui/ Ui) and not Ui.
Uncertainty, sensitivity, and counterfactual analyses Interval estimates of the three sensitivity components give the confidence intervals (CI) for the point estimate of sensitivity. In order to assess the sensitivity of the main findings to the probable overestimation information bias in proxy measurement of the determining components of surveillance sensitivity (discussed below), calculations were repeated with values of the three components of surveillance sensitivity being set 5–20% lower than the observed values in all health sectors. Then the calculated new values for the surveillance sensitivity were compared with their original values. For planning purposes, a ‘what-if’ scenario was developed. For this counterfactual scenario, a 5% improvement was assumed in each of the three components of surveillance sensitivity. Then the resultant changes in ‘sector sensitivity’ and ‘total sensitivity’ were determined.
Ethics The study protocol was approved by the institutional ethics review board of Tehran University of the Medical Sciences.
Results Of 49 midwives and general and specialist physicians in Darregaz, 35 practitioners (71.4%) participated in the study; eight (22.9%) worked primarily in the public sector, 16 (45.7%) worked primarily in the private sector and 11 (31.4%) worked in both sectors. Of the 49 intended practitioners, 14 (28.6%) did not respond after three follow-ups (four specialists, four general physicians and six midwives).
ARTICLE IN PRESS 56
R. Majdzadeh, F. Pourmalek
Test–retest reliability of the people’s questionnaire: intra-cluster correlation of test and retest values of the single question in this questionnaire was 0.98 (95% CI 0.97–0.99). Internal consistency reliability of the practitioners’ questionnaire: for seven questions on the knowledge of practitioners about the correct diagnosis of STD, Cronbach’s a was 0.46, which increased to 0.60 after dropping one of them. Further dropping of questions did not increase a; and thus, six knowledge questions were further analysed. Cronbach’s a for six questions on the attitude of practitioners towards reporting STD cases was 0.22; this rose to 0.42 after the deletion of one question. Further dropping of questions did not increase a, and five attitude questions were further analysed.
Table 1
Determinants of sensitivity The health service utilization pattern of people aged 15–50 years in Darregaz is shown in Table 1. Those who did not use health services went directly to pharmacies, consulted with acquaintances, or used local herbal remedies, and therefore, did not come to the notice of the surveillance system. The mixed private–public sector in the utilization pattern relates to the responses of those people who mentioned they would go and see a physician or midwife, but they did not have a preference for utilizing private and/or public health services. The assessment of practitioners’ knowledge about the correct diagnosis of STD, and their attitude towards reporting the diagnosed STD cases are shown in Table 2.
Health service utilization pattern of people aged 15–50 years in cases of STD in Darregaz in July 2004.
Utilization pattern
Frequency
Proportion (%)
95% CI
Public sector Private sector Mixed private–public No. use of health services
86 44 43 17
45.3 23.2 22.6 8.9
38.1–52.6 17.5–29.9 17.0–29.4 5.5–14.2
Total
190
100.0
CI, confidence interval.
Table 2 Point estimates and 95% confidence intervals for STD surveillance system sensitivity in Darregaz in July 2004 by different health sectors (all numbers are percentages). Estimate
Health sector
Utilization (Ui)
Diagnosis (Di)
Reporting (Ri)y
Sector sensitivity (Si)
Contribution of sector to total sensitivity
Point estimate
Public Private sector Mixed private–public
45.3 23.2 22.6
50.9 33.5 45.1
54.9 50.2 45.7
12.6 3.9 4.7
59.6 18.4 22.0
Lower 95% CI
Public Private sector Mixed private–public
38.1 17.5 17.0
39.8 22.0 33.3
44.1 19.2 36.2
6.7 0.7 2.0
70.5 7.8 21.6
Upper 95% CI
Public Private sector Mixed private–public
52.6 29.9 29.4
62.0 45.2 56.9
65.7 41.2 55.2
21.4 5.6 9.2
59.2 15.4 25.4
CI, confidence interval.
Using knowledge of practitioners about correct diagnosis of STD as a proxy. y
Using attitude of practitioners towards reporting of STD cases as a proxy.
ARTICLE IN PRESS Surveillance system sensitivity assessment
Surveillance sensitivity Table 2 also gives the calculations of surveillance sensitivity point estimates and 95% CI by different sectors with the formula described earlier. The right-hand column in Table 2 illustrates the percentage of contribution of private and public sectors in STD case detection in Darregaz in 2004. The ‘total sensitivity’ of the surveillance system was 21.2% (95% CI 15.5–25.3) for detecting symptomatic STD – the sum of percentages of patients found in the population by different health sectors. Figure 1 illustrates the template for computing the three conditional probabilities and the final point estimate of STD surveillance sensitivity in Darregaz in July 2004.
Sensitivity analysis Results of the sensitivity analysis for the main study findings with regard to the probable information bias in proxy measurement of the determining components of surveillance sensitivity (discussed below) showed that when calculations were repeated with values of the three components of surveillance sensitivity set 5% and 20% lower than the observed values in all health sectors, the new surveillance sensitivity values dropped by 0.9 and
57 0.5 of their original values, respectively (18.2% and 10.9% total sensitivity).
Counterfactual analysis Table 3 shows the results of the counterfactual scenario of 5% improvement in each component of utilization, diagnosis, and reporting. All computations are tabulated for further clarification, and the notations are identical with those for the aforementioned formulae. The 5% improvement in each of the three components of sensitivity, within the public sector for example, would result in a 1.9% increase in the total sensitivity. This 1.9% was computed from the difference in values of the total sensitivity from the actual present situation (12.7%) and from the scenario of counterfactual 5% increase in each of the three components (14.6%).
Discussion The 21.2% total sensitivity (95% CI 15.5–25.3%) of the STD surveillance system in Darregaz means that to estimate the actual number of symptomatic STD cases in general, the number of cases detected by the surveillance system should be multiplied by a correction factor of 4.7 (95% CI 3.9–5.9%). Based on
Figure 1 Template for computing the three conditional probabilities and final sensitivity estimate for the STD surveillance system in Darregaz in July 2004.
ARTICLE IN PRESS
1.9 0.6 0.7 3.2 14.6 4.5 5.4 24.4y 12.7 3.9 4.7 21.2y 0.6 0.2 0.2 1.06
Total sensitivity calculated with the formula described in the text. y
45.3 23.2 22.6 91.0 Public Private Mixed Total
Weighted mean. Weights are the proportion of practitioners in each sector among all practitioners.
57.6 52.7 48.0 52.4 54.9 50.2 45.7 49.9a 0.6 0.2 0.2 1.06 53.4 35.2 47.4 43.2 50.9 33.5 45.1 41.1 0.6 0.2 0.2 1.06
Present value Health sector
47.6 24.4 23.7 95.7
Present value Present value Present value
Utilization Sensitivity determinants
5% increase in utilization
Resultant increase in sensitivity
Diagnosis
5% increase in diagnosis
Resultant increase in sensitivity
Reporting
5% increase in reporting
Resultant increase in sensitivity
Sensitivity
Improved value of sensitivity
Resultant increase in sensitivity
R. Majdzadeh, F. Pourmalek Table 3 Impact of a 5% improvement in each determinant factor on STD surveillance sensitivity in Darregaz (baseline sensitivity is 21.2%; all numbers are percentages; differences at the level of 0.1 are the result of rounding).
58
the results illustrated in Figure 1, 8.9% of people did not get appropriate care for their STD and subsequently did not have a chance of being detected and reported by the surveillance system. Of course, it is expected that a small fraction of those going directly to pharmacies are diagnosed (i.e., Di40) and receive antibiotics and may be (partially) treated, but again will escape the surveillance system. The situation for the private sector practitioners was worse than for the public sector. Although the former group comprised 45.7% of all responding practitioners in the town and 23.2% of people said that they would solely use the private sector services, their contribution to case finding was only 18.4%. The major strength of our conditional probability approach to the assessment of surveillance sensitivity is manifest in the straightforward decomposition of sensitivity to its three determining components that provides the health policy makers and managers with the necessary information for intervention planning. Results of the counterfactual scenario of a given 5% improvement in each of service utilization, diagnosis, and reporting components, show that investing in such improvements in the public health sector would result in an increase in the total surveillance sensitivity (1.9%), which would be more than three times the result for the private sector (0.6%). Of course, such information is not sufficient for choosing the most appropriate interventions, and decisions must be based on the economic evaluation of each intervention as well as other due considerations, such as feasibility, policy, equity, and ethical issues, etc. The obtained information is quite useful for the purpose of counterfactual analysis. In this study, a greater contribution of the public sector to case finding (the right-hand column in Table 2) resulted in better subsequent provisional total sensitivity (the right-hand column in Table 3) with the same percentage of improvement. The other advantage of the ‘conditional probability’ method presented in this paper is its applicability in the field setting, even for rare events. The accuracy of the conditional probability method is undeterminable due to the lack of a gold standard for comparison. In an article that reviewed 33 studies in the USA,7 two main distinct methods were described. These two approaches were named ‘uncorrected’ and ‘under-ascertainment corrected’ methods. The authors described the former method as dividing the reported cases by the total number of cases detected by active case finding and the use of supplementary data from other sources. This approach does not account for the cases that are left undetected in all
ARTICLE IN PRESS Surveillance system sensitivity assessment sources. That is why the method is called ‘uncorrected’; it is prone to underestimating the number of real cases in the community and, conversely, overestimating the sensitivity of the surveillance system. The second method that had been used in one-third of the papers reviewed was capture– recapture, which is an effort to correct for this under-reporting (hence the ‘under-ascertainment corrected’ method).7 The main assumption in this method is the independency of data sources.8 The former approach can be applied if this assumption is violated, but it makes the computations more complex.9–11 However, in contrast to the approach that has been used in our study, neither of these two methods is applicable to the decomposing of surveillance sensitivity into its determining factors. The end result of both ‘uncorrected’ and ‘underascertainment corrected’ methods is just the value of surveillance sensitivity per se, without decomposing it into its determining factors. The conditional probability method is prone to two kinds of systematic error in the estimation of sensitivity of the surveillance system. The first one is the selection bias due to choosing a sample of the general population instead of ill people who actually suffered from the condition. If the affecting condition, STD in this study, were dependent on the service utilization pattern, then the result from the general population would be different from that for real patients. Using three proxy measurements instead of direct measures makes this method prone to information bias. Asking people about their preference in seeking medical care as well as the knowledge and attitude of practitioners regarding STD could lead to overestimation of real values of interest. The sensitivity analysis performed shows that the main finding of ‘21.2% total sensitivity’ is modestly sensitive to these upward measurement biases (116–195% overestimation of the total sensitivity, if the components of sensitivity are 5–20% overestimated). As far as these measurement biases affect the three components in the three health sectors to similar degrees, the proportion of each sector’s contribution to the total sensitivity does not change. The same holds true for the proportions of gain in total sensitivity subsequent to improvements in the utilization, diagnosis, and reporting in different sectors. But if the biases affect the three components very differently in the three sectors, these two groups of proportions will be less valid. Another important issue is the unsuitability of this method for asymptomatic conditions. The result of this study can be used for symptomatic
59 STD, e.g., gonorrhoea, but is not appropriate for the estimation of asymptomatic or sub-clinical STD cases, like chlamydia, or, more importantly, HIV positivity. Of course, if the condition is asymptomatic, then people do not feel the need to seek medical care. However, the latter two parts of sensitivity calculations (i.e., health care providers’ knowledge and attitude), would still be applicable. In this approach, the absolute value of sensitivity is not as important as the expected effect of different points of interventions for improving it, and comparisons can be made for the pertinent costs of each intervention. In this way, one of the responsibilities of public health agencies at the peripheral levels (in districts and in provinces – based on the administrative health structure of the country) can be elaborated: evaluation of surveillance and planning for its improvement. Few studies have described the factors that influence the reporting behaviour of health care providers. These factors are described as lack of awareness of the legal requirement for reporting, lack of knowledge of which diseases are reportable, lack of understanding of how or to whom to report, assuming that someone else will report the case, intentional failure to report to protect patient privacy, insufficient rewards for reporting,7,12 and interference of the reporting task with the daily clinical practice.13 One article reports that a lack of appropriate feedback and a belief that no useful action was taken after reporting could be important factors in the failure to report notifiable diseases.14 Interventional studies have had limited success in decreasing these problems in the final reporting performance of providers,7,15 but some have shown that even assigning a nurse for the task and the introduction of a ward notification register could greatly improve notification rates.14 Although the CDC published the Guidelines for evaluating surveillance systems in 1988,1 only 31 studies have been covered in a 2000 review article,2 even though the search strategy for identifying the relevant literature in this review article was reasonable. Of course, not all evaluations are published in peer-reviewed journals, but an alternative explanation for this paucity of reports is that the assessment of surveillance systems sensitivity is not a frequent practice among public health professionals. On the other hand, the definition of essential functions of public health (assessment of community health, policy development based on community diagnosis, and assurance of coverage and quality of services) by the Institute of Medicine in 1998 emphasizes the role of surveillance in health practice.16 On the whole, it seems that sensitivity assessment is not quite
ARTICLE IN PRESS 60 feasible as a usual task in public health practice. Surveillance systems require ongoing evaluation if the information they provide is to accurately portray the conditions in the community for further action. The result of most complicated methods of surveillance sensitivity estimation is a number that is not decomposable to its constituents, and thus is not helpful in finding evidence on which to base appropriate intervention(s). These facts, along with the ease of conducting necessary surveys and the final decomposing of sensitivity constituents, make the present method relatively applicable at the field level. Although there is some overestimation of sensitivity, it is valid for prioritizing the points for intervention, and reliable for tracking changes over time.
R. Majdzadeh, F. Pourmalek
4.
5.
6.
7.
8.
9.
Acknowledgements This study was financially supported by the Institute of Public Health Research, Tehran University of Medical Sciences. We thank Dr. Ebrahim Khajeh and Dr. Ali Beheshtian for valuable support provided during data collection.
10.
11.
12.
Competing interests The authors declare that they have no competing interests. 13.
References 1. Centers for Disease Control and Prevention. Guidelines for evaluating surveillance systems. MMWR Morb Mortal Wkly Rep 1988;37(Suppl 5):1–18. 2. German RR. Sensitivity and predictive value positive measurements for public health surveillance systems. Epidemiology 2000;11:720–7. 3. Gallay A, Nardone A, Vaillant V, Desenclos JC. [The capture–recapture applied to epidemiology: principles,
14.
15.
16.
´pide limits and application]. Rev E ´miol Sante ´ Publique 2002;50:219–32. Bailey TC, Carvalho MS, Lapa TM, Souza WV, Brewer MJ. Modeling of under-detection of cases in disease surveillance. Ann Epidemiol 2005;15:335–43. Romaguera RA, German RR, Klaucke DN. Evaluating public health surveillance. In: Teusch S, Churchill E, editors. Principles and practice of public health surveillance. New York: Oxford University Press; 1994. p. 176–92. Reintjes R, Termorshuizen F, van de Laar MJ. Assessing the sensitivity of STD surveillance in the Netherlands: an application of the capture–recapture method. Epidemiol Infect 1999;122:97–102. Doyle TJ, Glynn MK, Groseclose SL. Completeness of notifiable infectious disease reporting in the United States: an analytical literature review. Am J Epidemiol 2002; 155:866–74. Hook EB, Regal RR. Capture–recapture methods in epidemiology: methods and limitations. Epidemiol Rev 1995;17: 243–64. Nardone A, Decludt B, Jarraud S, Etienne J, Hubert B, Infuso A, Gallay A, Desenclos JC. Repeat capture–recapture studies as part of the evaluation of the surveillance of Legionnaires’ disease in France. Epidemiol Infect 2003;131:647–54. Hook EB, Regal RR. Accuracy of alternative approaches to capture–recapture estimates of disease frequency: internal validity analysis of data from five sources. Am J Epidemiol 2000;152:771–9. Brenner H. Application of capture–recapture methods for disease monitoring: potential effects of imperfect record linkage. Methods Inf Med 1994;33:502–6. St. Lawrence JS, Montano DE, Kasprzyk D, Phillips WR, Armstrong K, Leichliter JS. STD screening, testing, case reporting, and clinical and partner notification practices: a national survey of US physicians. Am J Public Health 2002; 92:1784–8. Figueiras A, Lado E, Fernandez S, Hervada X. Influence of physicians’ attitude on under-notifying infectious diseases: a longitudinal study. Public Health 2004;118:521–6. Seneviratne SL, Gunatilake SB, de Silva HJ. Reporting notifiable diseases: methods for improvement, attitude and community outcome. Trans R Soc Trop Med Hyg 1997; 91:135–7. Weiss BP, Strassburg MA, Fannin SL. Improving disease reporting in Los Angeles County: trial and results. Public Health Rep 1988;103:415–21. Buehler JW. Surveillance. In: Rothman KJ, Greenland S, editors. Modern epidemiology. 2nd ed. Philadelphia: Lippincott-Raven Publishers; 1998. p. 435–57.