- Email: [email protected]
Validating ICD coding algorithms for diabetes mellitus from administrative data
diabetes research and clinical practice 89 (2010) 189–195
Contents lists available at ScienceDirect
Diabetes Research and Clinical Practice journ al h omepage: www .elsevier.co m/lo cate/diabres
Validating ICD coding algorithms for diabetes mellitus from administrative data Guanmin Chen a,*, Nadia Khan b, Robin Walker a, Hude Quan a a b
Department of Community Health Sciences, University of Calgary, Canada Department of Medicine, University of British Columbia, Canada
article info
abstract
Article history:
Aim: To assess validity of diabetes International Classification of Disease (ICD) 9 and 10
Received 1 December 2009
coding algorithms from administrative data using physicians’ charts as the ‘gold standard’
Received in revised form
across time periods and geographic regions.
1 March 2010
Methods: From 48 urban and 16 rural general practitioners’ clinics in Alberta and British
Accepted 8 March 2010
Columbia, Canada, we randomly selected 50 patient charts/clinic for those who visited the
Published on line 2 April 2010
clinic in either 2001 or 2004. Reviewed chart data were linked with inpatient discharge abstract and physician claims administrative data. We identified patients with diabetes in
Keywords:
the administrative databases using ICD-9 code 250.xx and ICD-10 codes E10.x–E14.x.
Validity
Results: The prevalence of diabetes was 8.1% among clinic charts. The coding algorithm of
Administrative data
‘‘2 physician claims within 2 years or 1 hospitalization with the relevant diabetes ICD codes’’
Diabetes, ICD, Surveillance
had higher validity than other 7 algorithms assessed (sensitivity 92.3%, specificity 96.9%, positive predictive value 77.2%, and negative predictive value 99.3%). After adjustment for age, sex, and comorbid conditions, sensitivity and positive predictive values were not significantly different between time periods and regions. Conclusion: Diabetes could be accurately identified in administrative data using the following case definition ‘‘2 physician claims within 2 years or 1 hospital discharge abstract record with diagnosis codes 250.xx or E10.x–E14.x’’. # 2010 Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
Diabetes prevalence has increased substantially in the past two decades, and is a major cause of morbidity and mortality [1,2]. As a leading cause of blindness, end-stage renal disease and cardiovascular disease, diabetes poses a major challenge to the healthcare system, and weighs heavily on patients and their families [3,4]. Therefore, diabetes has been widely studied for projecting population incidence, identifying high-risk groups and evaluating prevention and control initiatives for reducing the disease and its complications [5,6].
In recent years administrative data have been widely used to conduct large-scale diabetes studies as the data are considerably less expensive and ready to access than conducting population based surveys. As the results derived from administrative data depend on the data quality, validity of administrative data in recording diabetes has been evaluated [7–9]. However, these validation studies have been mainly conducted in metropolitan areas, at a fixed time period, and from one diagnosis coding system, such as the International Classification of Disease (ICD) 9 database. Completeness of administrative data depends on healthcare system utilization because the data records patients who
* Corresponding author at: Department of Community Health Sciences, University of Calgary, 3330 Hospital Dr. NW, Calgary, Alberta, Canada T2N 4N1. Tel.: +1 403 210 7696; fax: +1 403 210 3818. E-mail address: [email protected] (G. Chen). 0168-8227/$ – see front matter # 2010 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.diabres.2010.03.007
190
diabetes research and clinical practice 89 (2010) 189–195
have used the system. Health services utilization is determined by predisposing factors of demographic characteristics, enabling factors such as geographic location and health insurance and need factors such as the presence of chronic diseases [10]. Thus, the validity of diabetes ICD coding algorithms may vary by these factors. We conducted this study to investigate the validation of diabetes case definitions for ICD-9/ICD-10 administrative data across time periods and geographic areas after adjustment for patient demographic characteristics and chronic diseases.
2.
Methods
2.1.
Chart review data
Charts of general practitioners (GPs) were employed as the reference standard. GP’s charts were selected through a two stage sampling process. The first stage was GP recruitment. A list of GPs in the provinces of Alberta and British Columbia, Canada was obtained from the provincial licensing physician directories. GPs were randomly selected from urban (i.e. Calgary with approximately 1.1 million population and Vancouver with approximately 2.1 million population), and rural (defined as less than 10,000 population) areas. GPs were contacted by fax or telephone to determine their eligibility and to invite them to participate in the study. We included fee for service GPs who practiced >2 days per week at their current location between 1999 and 2001 or between 2002 and 2004. We excluded locum physicians and those whose primary practice was at walk-in clinics, community health centers, and hospitals or emergency rooms. The second stage was selection of charts at GP clinics. At the clinics where computerized patient registry was available, we generated a random list of patients using the registry. At those without computerized patient registry, the chart shelves were equally divided into 50 sections. Trained chart reviewers consecutively assessed the charts in a section until an eligible chart was identified. The eligibility criteria were those who (1) were aged 35 years or older; (2) were alive and did not migrate out of province in the 2-year period prior to the study, and (3) had at least 2 visits to a GP for each period of 1999 to 2001 and 2002 to 2004. We identified approximately 50 eligible charts at each clinic and extracted data from charts using a standardized data collection form on demographic factors, medications, laboratory results, and comorbid conditions including hypertension, stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis. The number of patients per GP (i.e. 50 charts) was chosen to represent patients’ demographic and clinical characteristics. Ethics approval was obtained from the University of Calgary and University of British Columbia Ethics Committees. The presence of diabetes was determined by documented information on the use of either insulin or an oral hypoglycemic agent, an indication of diabetes in any of the text notes from the chart, such as diabetes admission history, diabetic retinopathy history or other diabetes-specific complications or
fasting blood sugar 7.0 mmol/L or 2 h blood sugar in the 75 g oral glucose tolerance test 11.1 mmol/L. Patients without the above documented information were assigned ‘‘absence of diabetes’’.
2.2.
Administrative data
The GP patients were linked with physician claims and hospital discharge abstract administrative data from 1999 to 2004 using personal unique identifier (i.e. personal health number). Fee-for-service physicians in Canada submit billing claims to their respective provincial government insurance program to reimburse for all services they provided. These claims contain patients’ personal health number, type of service provided as well as one ICD-9 diagnosis code in British Columbia and up to 3 ICD-9 codes in Alberta. Hospital discharge abstract data included up to 16 ICD-9 diagnosis codes prior to April 1, 2002, and up to 25 ICD-10 diagnoses codes after March 31, 2002. We determined the number of the physician claims and hospital discharge records with ICD-9 codes of 250.xx and ICD10 codes of E10.x–E14.x in the administrative data and then defined the presence of diabetes using the following algorithms: (1) 2 physician claims or 1 hospitalization with the ICD code; (2) 2 physician claims with the ICD code; (3) 1 physician claim or 1 hospitalization with the ICD code and (4) 1 physician claim with the ICD code. We further specified the gap between 2 claims by 1, 2, and 3 years.
2.3.
Statistical methods
We calculated sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and kappa values for each diabetes algorithm described above, and compared it with the chart review data (reference standard). These statistics were stratified by sex, age groups, comorbid conditions, study period, rural and urban areas, and by province. Difference in sensitivities and PPV between study periods and regions were calculated, while controlling for sex, age group, and comorbid conditions in a log-linear regression model [11]. All statistical analyses were performed with the statistical software SAS (version 9.2).
3.
Results
From the chart review, 36.3% patients were male, and the mean age was 52.8 years. A total of 1642 (48.8%) individuals had at least one comorbid condition other than diabetes. Prevalence of diabetes was 8.1% in the chart review data (see Table 1). The 8 ICD algorithms for defining diabetes (see Table 2) had a high sensitivity (ranging from 91.2% to 95.6%), specificity (ranging from 92.8% to 97.6%), and NPV (ranging from 99.2% to 99.6%) when 3-year data were analyzed. The algorithm of ‘‘2 physician claims within 2 years or 1 hospitalization’’ allowed for maximal PPV of 77.2% while maintaining a sensitivity of 92.3%. The algorithm of ‘‘1 physician claim or 1 hospitalization’’ had a kappa value of 0.65 and the other 7 algorithms had kappa values 0.79. However, when these algorithms were
191
diabetes research and clinical practice 89 (2010) 189–195
Table 1 – Characteristics of study samples in chart review data. Year
Number of patients Age (mean SD, years)
Provincea
Region
Total
2001
2004
Urban
Rural
AB
BC
1482 53.2 12.5
1880 52.4 12.8
2372 52.2 12.6
990 54.2 12.7
1565 52.3 12.9
1797 53.2 12.5
3362 52.8 12.7
Sex (%) Male Female
506 (34.3) 973 (65.7)
711 (37.8) 1169 (62.2)
796 (33.6) 1576 (66.4)
424 (42.8) 566 (57.2)
641 (34.6) 1024 (65.4)
679 (37.8) 1118 (62.2)
1220 (36.3) 2142 (63.7)
Comorbid conditionsa (%) Yes No
632 (42.6) 850 (57.4)
1010 (53.7) 870 (46.3)
1134 (47.8) 1238 (52.2)
508 (51.3) 482 (48.7)
760 (48.6) 805 (51.4)
882 (49.1) 915 (50.9)
1642 (48.8) 1720 (51.2)
105 (7.1) 1377 (92.9)
168 (8.9) 1712 (91.1)
186 (7.8) 2186 (92.8)
87 (8.8) 903 (91.2)
122 (7.8) 1443 (92.2)
151 (8.4) 1646 (91.6)
273 (8.1) 3089 (91.9)
Diabetes (%) Yes No
a AB = province of Alberta and BC = province of British Columbia. Comorbidity includes stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis.
applied in the 2-year data, validity became lower than that in the 3-year data. The stratified validity indexes for the algorithm of ‘‘2 physician claims within 2 years or 1 hospitalization’’ are shown in Table 3. These indexes varied little by region (i.e. rural and urban, and provinces), year (2001 and 2004), sex and age. However, PPV was higher for patients with comorbid conditions compared to those without (see Table 4, relative PPV: 1.15, 95% confidence interval: 1.03–1.28).
4.
Discussion
We validated diabetes recorded in administration data and found that the algorithm of ‘‘2 physician claims within 2 years or 1 hospitalization with ICD-9 code of 250.xx or ICD-10 code of E10.x–E14.x’’ had high validity. The validity remained similar across time periods and regions. To optimize the validity, at least 3-year data was needed. Our findings were consistent with some previous studies’ reports. Saydah et al. [7] reviewed validation studies of administrative data in recording diabetes, which were published from 1966 to 2002. Of the 16 studies reviewed, the sensitivity ranged from 46% to 97%; the PPV ranged from 60% to 97% and the kappa value ranged from 0.67 to 0.96. We updated the literature (see Table 5) to February 2010. Of 6 studies reviewed, the sensitivity ranged from 59% to 86%; the PPV ranged from 63% to 99% and the kappa value ranged from 0.60 to 0.83. The variation among these studies may reflect the fact that validity of administrative data is inconsistent across regions or that validity is influenced by inconsistency in study methods, such as reference standards, type of administrative data, study population and data collection mode. Hux et al. [12] reviewed GP charts and linked the chart data with physician claims and hospital discharge abstract administrative data in the province of Ontario, Canada. Compared to that study we found higher sensitivity (86.0% versus 92.3%) and slightly lower PPV (80.0% versus 77.2%). Lix et al. [13] validated diabetes recorded in physician claims and hospital discharge data using self-reported diabetes in a community survey in the province of Manitoba, Canada as a reference
standard. Lix reported sensitivity of 79.5% and PPV of 87.9%. All three Canadian studies (including this study) demonstrated a similar kappa value (ranging from 0.79 to 0.82). Quan et al. [14] validated hospital discharge abstract administrative data in the province of Alberta, Canada and found relatively low sensitivity of 59.1% and PPV of 63.1% for identifying diabetes with complications. This indicates that diabetes could be accurately identified using both the physician claims and hospital discharge abstract administrative data. The strength of our study is that we validated diabetes in administrative data across time periods and regions. After adjustment for potential confounding of age, sex and comorbid conditions there was no significant variation by time and region. However, PPV was higher for patients with comorbidities than those without comorbidities. That may be related to the nature of administrative data. The data captures patients who visited physicians or were hospitalized and missed those who did not see physicians during the study period. Diabetic patients with comorbidities may more frequently visit physicians compared to patients without comorbidities, consequently physicians may be more likely to submit claims with diabetes for those with comorbidities. In contrast, when the competing comorbid conditions are severe or potentially fatal, physicians may be more likely to claim these conditions rather than diabetes. Quan et al. [14] found that validity of hospital discharge abstract administrative data was higher for identifying diabetes without complications than that with complications. Diabetes validity may be influenced by the features and sources of administrative data. There is 1 ICD-9 diagnosis code in British Columbia, and up to 3 ICD-9 diagnosis codes recorded in Alberta physician claims data. The diabetes code might not be coded for diabetic patients if their main purpose of the visit was not for diabetes, such as a cold or other medical examination. In addition, when the competing comorbid conditions are severe or potentially fatal, physicians may be more likely to claim these conditions rather than diabetes [9]. This may potentially bias the validity for identifying diabetes cases in administrative data. In Alberta, up to 3 ICD-9 diagnosis-coding fields are available for each claim. However, the majority of physicians
192
Table 2 – Validity of administrative data diabetes case definition compared with chart data. 3 years observation period data PPV (%)
*
NPV (%)
Kappa
96.8 (96.2–97.4)
71.8 (67.1–76.5)
99.3 (99.0–100.0)
0.79 (0.75–0.82)
96.9 (96.2–97.5)
77.2 (72.5–81.7)
99.3 (99.0–99.6)
0.79 (0.75–0.83)
77.3 (72.3–82.3)
99.0 (98.6–99.4)
87.2 (82.9–91.4)
98.0 (97.5–98.5)
97.2 (87.9–94.6)
73.8 (68.7–78.4)
99.2 (99.0–100.0)
0.80 (0.76–0.83)
75.5 (70.4–80.6)
99.1 (98.7–99.4)
87.7 (83.4–91.9)
97.9 (97.4–98.4)
91.9 (88.7–95.2)
97.3 (96.7–97.9)
74.9 (70.2–80.0)
99.3 (99.0–100.0)
0.81 (0.77–0.84)
91.9 (89.0–94.9)
97.4 (96.7–97.9)
75.4 (70.8–80.0)
99.3 (99.0–99.5)
0.81 (0.78–0.85)
76.6 (71.5–81.6)
99.3 (99.0–99.6)
90.9 (87.2–94.6)
98.0 (97.5–98.4)
91.2 (87.9–94.6)
97.6 (97.1–98.1)
72.1 (67.5–76.9)
99.2 (99.9–99.8)
0.82 (0.79–0.85)
74.4 (69.2–79.5)
99.4 (99.1–99.7)
91.4 (87.8–95.1)
97.8 (97.3–98.3)
95.6 (92.5–97.7)
92.8 (91.9–93.7)
54.0 (49.6–58.5)
99.6 (99.4–99.8)
0.65 (0.61–0.69)
86.4 (82.4–90.5)
97.1 (96.5–97.7)
72.4 (67.5–77.3)
98.8 (98.4–99.2)
91.2 (87.9–94.6)
97.6 (97.1–98.1)
72.1 (67.5–76.9)
99.2 (98.9–99.5)
0.82 (0.78–0.85)
76.6 (71.5–81.6)
99.3 (99.0–99.6)
90.9 (87.2–94.6)
98.0 (97.5–98.4)
Sensitivity (%) Specificity (%) 2 claims or 1 hospitalization 2 claims 92.3 (89.2–95.5) gap 3 years 2 claims 92.3 (89.2–95.5) gap 2 years 2 claims 91.6 (88.3–94.6) gap 1 year 2 claims only 2 claims gap 3 years 2 claims gap 2 years 2 claims gap 1 year 1 claim or 1 hospitalization 1 claim only
*
2 years observation period data
*
PPV = positive predictive value, NPV = negative predictive value.
PPV* (%)
Sensitivity (%) Specificity (%) –
–
–
–
–
–
NPV* (%) –
–
Kappa – 0.80 (0.77–0.84) 0.80 (0.76–0.84)
– 0.82 (0.78–0.86) 0.80 (0.77–0.85) 0.77 (0.73–0.81) 0.82 (78.0–85.5)
diabetes research and clinical practice 89 (2010) 189–195
Algorithm
193
diabetes research and clinical practice 89 (2010) 189–195
Table 3 – The validity index for the coding algorithm of 2 physician claims within 2 years or 1 hospitalization by region, year, province and demographic factorsa. N
Prevalence (%)
Kappa value
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%)
Region Rural Urban
990 2372
9.2 10.4
0.83 0.82
92.0 90.9
97.5 97.7
76.8 77.7
99.3 99.2
Year 2001 2004
1482 1880
9.3 9.8
0.76 0.86
90.5 91.7
96.9 98.2
78.8 83.2
99.3 99.2
Province AB BC
1565 1797
8.0 11.0
0.88 0.78
90.2 92.1
99.0 96.4
88.0 78.2
99.2 99.3
Sex Female Male
2142 1220
9.0 10.8
0.81 0.83
90.5 91.9
98.0 96.9
75.6 78.6
99.3 99.0
Age (years) <65 65
2709 653
7.1 20.2
0.82 0.81
96.1 88.0
94.0 98.4
75.0 78.5
99.2 99.2
Comorbid presencea Yes 1642 No 1720
14.4 5.6
0.83 0.77
90.1 95.0
96.9 98.1
81.4 65.5
98.5 99.8
a PPV = positive predictive value, NPV = negative predictive value, AB = province of Alberta, and BC = province of British Columbia. Comorbidity includes hypertension, stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis.
submit only 1 diagnosis. In our data, 74% of claims had only 1 diagnosis recorded, while 4% recorded 2 diagnoses, 2% recorded 3 diagnoses and 20% were missing diagnoses (services for diagnostics and laboratory tests are not required to record diagnoses). The validity of the diabetes definition had little change when more than one coding field was utilized to define diabetes. Similarly, adding emergency department visits to the diabetes definition contributed little to the validity of the case definition. Comprehensiveness of administrative data is related to the healthcare system and data collection system. Canada has a government-owned universal healthcare insurance system. Traditional fee-for-service programs remunerate physicians for each medical service provided, as determined by details from the physicians’ claims submissions. If the physician does not provide a claim, he/she will not be paid for the service provided. The physician submission is entered into the claims
database regardless of where the service was provided. Hospital discharge abstract data from all hospitals are centralized and stored in the provincial health ministries. Therefore, both databases (physician claims and inpatient) capture relatively complete and comprehensive information on inpatient and outpatient physician services for all specialties, including general medicine. Application of our study findings depends on the availability of administrative data and study purposes. Surveillance is generally aimed to capture all prevalent and incidence diabetes cases. Health services researchers utilize administrative data for evaluation of outcomes and resources utilization. Utilization of hospital discharge abstract data alone in surveillance and health services research misses a large number of diabetes cases because many diabetes patients are managed at outpatient clinics. In addition, severity of disease between inpatients and outpatients may
Table 4 – Relative sensitivity and positive predictive value (95% confidence interval) for diabetes defined using 2 physician claims within 2 years or 1 hospitalization case definition in administrative data. Variable Comorbidity (yes versus no) Age (65 years versus <65 years) Sex (female versus male) Region (urban versus rural) Year (2004 versus 2001) Province (BC versus AB) a
Relative sensitivity (95% confidence interval) 0.88 0.91 0.99 1.04 1.02 1.04
(0.76–1.02) (0.83–1.00) (0.93–1.06) (0.94–1.16) (0.94–1.10) (0.96–1.12)
P-value 0.10 0.06 0.84 0.42 0.69 0.34
Relative PPVa (95% confidence interval) 1.15 1.08 1.05 1.06 1.09 0.92
(1.03–1.28) (0.99–1.16) (0.95–1.15) (0.96–1.17) (0.99–1.21) (0.83–1.01)
P-value 0.01 0.07 0.35 0.26 0.07 0.70
PPV = positive predictive value, AB = province of Alberta, and BC = province of British Columbia. Comorbidity includes hypertension, stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis.
194
diabetes research and clinical practice 89 (2010) 189–195
Table 5 – Previous studies of validating administrative data in identifying diabetes. NPV* (%)
96.9
77.2
99.3
0.79
75.8
98.7
88.5
96.8
0.79
59.1
99.0
63.1
98.9
0.6
82.8
98.9
99.4
71.2
0.74
74.6
99.7
93.4
98.7
0.82
86.1
97.1
80.0
98.1
0.80
79.5
99.3
87.9
98.7
0.82
70.6
84.7
99.4
–
Reference, population
This study, Canada, 2001 and 2004
Physician charts, 35 years old
ICD-9 and ICD-10, 2 physician claims within 2 years or 1 hospitalization
92.3
Quan et al. [14], Canada, 2003
Hospital chart, 18 years old
ICD-10 hospital data for diabetes without complications ICD-10 hospital data for diabetes with complications ICD-9 and ICD-10, 4 physician claims within 2 years ICD-10 hospital or birth data
Guttmann et al. [8], Canada, 1994–2003 Bell et al. [15], Australia, 2002 Hux et al. [12], Canada, 1991–1999
Hospital chart, <19 years old Medical chart, pregnant women Physician charts, 20 years old
Lix et al. [13], Canada, 2001
Community Health Survey, 19 years old
Zgibor et al. [9], USA, 2001–2003
Physician chart, 18 years old
Administrative data and definition
PPV* (%)
Author, study location and year
ICD-9 and ICD-10, 2 physician claims within 2 years or 1 hospitalization ICD-9 and ICD-10, 2 physician claims within 2 years or 1 hospitalization Electric health records, any 3 indicators (including physician visits, hospitalization, ER, medication using ICD-9, any record of A1c measures, glucose > 200 mg/dl and ER)
Sensitivity Specificity (%) (%)
Kappa
–
Note: Saydah et al. [7] reviewed literature published from 1966 to mid-2002. We updated the literature to February 2010. PPV = positive predictive value, NPV = negative predictive value.
*
be significantly different and outcomes evaluation using hospital discharge data alone is likely to be biased. Therefore, our validated algorithm is useful for identifying diabetic inpatients and outpatients. Our study has limitations. Firstly, the GP’ chart data were created as the ‘‘reference standard’’. This ‘‘reference standard’’ reflects the portion of diabetes diagnosed by a physician. Persons who did not see GPs or who were not diagnosed by the GPs might be missed. For example, a patient may be hospitalized and diagnosed with diabetes by a specialist but the diagnosis information was not forwarded to their GP. To improve the reference standard, inpatient charts and laboratory test data should be utilized to establish a more reliable standard than chart data. Secondly, we selected GP clinics in urban and rural areas but missed clinics in remote or suburban areas. Under the Canadian universal healthcare system, it is expected that GP billing practice may be similar across regions. However, further study is needed. Thirdly, validity may vary by age group but we only included patients aged 35 or older. However Guttmann et al. [8] validated diabetes in administrative data for children and reported similar results to this study. In conclusion, our study is unique as, to the best of our knowledge, it is the first to compare the validation of diabetes from both inpatient and outpatient administrative data between time periods and regions. The validation of the diabetes case definition ‘‘2 physician claims within 2 years or 1 hospitalization’’ was robust between age groups, sex, rural and urban areas, provinces, and years. But patients with comorbidities had a slightly higher PPV than those without.
Acknowledgements The data collection was funded by Canadian Institutes of Health Research. Dr. Quan is supported by Alberta InnovateHealth Solution salary award. Dr. Khan is supported by a New Investigator Award from the Canadian Institutes of Health Research.
Conflict of interest All authors of this paper have no conflicts of interest to declare.
references
[1] C.C. Cowie, K.F. Rust, E.S. Ford, M.S. Eberhardt, D.D. ByrdHolt, C. Li, et al., Full accounting of diabetes and prediabetes in the U.S. population in 1988–1994 and 2005–2006, Diabetes Care 32 (2009) 287–294. [2] L.L. Lipscombe, J.E. Hux, Trends in diabetes prevalence, incidence, and mortality in Ontario, Canada 1995–2005: a population-based study, Lancet 369 (2007) 750–756. [3] American Diabetes Association, Economic costs of diabetes in the U.S. in 2007, Diabetes Care 31 (2008) 596–615. [4] S. Wild, G. Roglic, A. Green, R. Sicree, H. King, Global prevalence of diabetes: estimates for the year 2000 and projections for 2030, Diabetes Care 27 (2004) 1047–1053. [5] J. Goldman, S. Kinnear, J. Chung, D.J. Rothman, New York City’s initiatives on diabetes and HIV/AIDS: implications for
diabetes research and clinical practice 89 (2010) 189–195
[6] [7]
[8]
[9]
[10]
patient care, public health, and medical professionalism, Am. J. Public Health 98 (2008) 807–813. A.L. Fairchild, Public health. Diabetes and disease surveillance, Science 313 (2006) 175–176. S.H. Saydah, L.S. Geiss, E. Tierney, S.M. Benjamin, M. Engelgau, F. Brancati, Review of the performance of methods to identify diabetes cases among vital statistics, administrative, and survey data, Ann. Epidemiol. 14 (2004) 507–516. A. Guttmann, M. Nakhla, M. Henderson, T. To, D. Daneman, K. Cauch-Dudek, et al., Validation of a health administrative data algorithm for assessing the epidemiology of diabetes in Canadian children, Pediatr. Diabetes 11 (2009) 122–128. J.C. Zgibor, T.J. Orchard, M. Saul, G. Piatt, K. Ruppert, A. Stewart, et al., Developing and validating a diabetes database in a large health system, Diabetes Res. Clin. Pract. 75 (2007) 313–319. R.M. Andersen, Revisiting the behavioral model and access to medical care: does it matter? J. Health Soc. Behav. 36 (1995) 1–10.
195
[11] M.S. Pepe, The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press, Oxford; New York, 2003. [12] J.E. Hux, F. Ivis, V. Flintoft, A. Bica, Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm, Diabetes Care 25 (2002) 512–516. [13] L. Lix, M. Yogendran, C. Burchill, C. Metge, N. McKeen, D. Moore, et al., Defining and validating chronic diseases: an administrative data approach, in: Policy MCfH, Manitoba, 2004. [14] H. Quan, B. Li, L.D. Saunders, G.A. Parsons, C.I. Nilsson, A. Alibhai, et al., Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database, Health Serv. Res. 43 (2008) 1424–1441. [15] J.C. Bell, J.B. Ford, C.A. Cameron, C.L. Roberts, The accuracy of population health data for monitoring trends and outcomes among women with diabetes in pregnancy, Diabetes Res. Clin. Pract. 81 (2008) 105–109.
Contents lists available at ScienceDirect
Diabetes Research and Clinical Practice journ al h omepage: www .elsevier.co m/lo cate/diabres
Validating ICD coding algorithms for diabetes mellitus from administrative data Guanmin Chen a,*, Nadia Khan b, Robin Walker a, Hude Quan a a b
Department of Community Health Sciences, University of Calgary, Canada Department of Medicine, University of British Columbia, Canada
article info
abstract
Article history:
Aim: To assess validity of diabetes International Classification of Disease (ICD) 9 and 10
Received 1 December 2009
coding algorithms from administrative data using physicians’ charts as the ‘gold standard’
Received in revised form
across time periods and geographic regions.
1 March 2010
Methods: From 48 urban and 16 rural general practitioners’ clinics in Alberta and British
Accepted 8 March 2010
Columbia, Canada, we randomly selected 50 patient charts/clinic for those who visited the
Published on line 2 April 2010
clinic in either 2001 or 2004. Reviewed chart data were linked with inpatient discharge abstract and physician claims administrative data. We identified patients with diabetes in
Keywords:
the administrative databases using ICD-9 code 250.xx and ICD-10 codes E10.x–E14.x.
Validity
Results: The prevalence of diabetes was 8.1% among clinic charts. The coding algorithm of
Administrative data
‘‘2 physician claims within 2 years or 1 hospitalization with the relevant diabetes ICD codes’’
Diabetes, ICD, Surveillance
had higher validity than other 7 algorithms assessed (sensitivity 92.3%, specificity 96.9%, positive predictive value 77.2%, and negative predictive value 99.3%). After adjustment for age, sex, and comorbid conditions, sensitivity and positive predictive values were not significantly different between time periods and regions. Conclusion: Diabetes could be accurately identified in administrative data using the following case definition ‘‘2 physician claims within 2 years or 1 hospital discharge abstract record with diagnosis codes 250.xx or E10.x–E14.x’’. # 2010 Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
Diabetes prevalence has increased substantially in the past two decades, and is a major cause of morbidity and mortality [1,2]. As a leading cause of blindness, end-stage renal disease and cardiovascular disease, diabetes poses a major challenge to the healthcare system, and weighs heavily on patients and their families [3,4]. Therefore, diabetes has been widely studied for projecting population incidence, identifying high-risk groups and evaluating prevention and control initiatives for reducing the disease and its complications [5,6].
In recent years administrative data have been widely used to conduct large-scale diabetes studies as the data are considerably less expensive and ready to access than conducting population based surveys. As the results derived from administrative data depend on the data quality, validity of administrative data in recording diabetes has been evaluated [7–9]. However, these validation studies have been mainly conducted in metropolitan areas, at a fixed time period, and from one diagnosis coding system, such as the International Classification of Disease (ICD) 9 database. Completeness of administrative data depends on healthcare system utilization because the data records patients who
* Corresponding author at: Department of Community Health Sciences, University of Calgary, 3330 Hospital Dr. NW, Calgary, Alberta, Canada T2N 4N1. Tel.: +1 403 210 7696; fax: +1 403 210 3818. E-mail address: [email protected] (G. Chen). 0168-8227/$ – see front matter # 2010 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.diabres.2010.03.007
190
diabetes research and clinical practice 89 (2010) 189–195
have used the system. Health services utilization is determined by predisposing factors of demographic characteristics, enabling factors such as geographic location and health insurance and need factors such as the presence of chronic diseases [10]. Thus, the validity of diabetes ICD coding algorithms may vary by these factors. We conducted this study to investigate the validation of diabetes case definitions for ICD-9/ICD-10 administrative data across time periods and geographic areas after adjustment for patient demographic characteristics and chronic diseases.
2.
Methods
2.1.
Chart review data
Charts of general practitioners (GPs) were employed as the reference standard. GP’s charts were selected through a two stage sampling process. The first stage was GP recruitment. A list of GPs in the provinces of Alberta and British Columbia, Canada was obtained from the provincial licensing physician directories. GPs were randomly selected from urban (i.e. Calgary with approximately 1.1 million population and Vancouver with approximately 2.1 million population), and rural (defined as less than 10,000 population) areas. GPs were contacted by fax or telephone to determine their eligibility and to invite them to participate in the study. We included fee for service GPs who practiced >2 days per week at their current location between 1999 and 2001 or between 2002 and 2004. We excluded locum physicians and those whose primary practice was at walk-in clinics, community health centers, and hospitals or emergency rooms. The second stage was selection of charts at GP clinics. At the clinics where computerized patient registry was available, we generated a random list of patients using the registry. At those without computerized patient registry, the chart shelves were equally divided into 50 sections. Trained chart reviewers consecutively assessed the charts in a section until an eligible chart was identified. The eligibility criteria were those who (1) were aged 35 years or older; (2) were alive and did not migrate out of province in the 2-year period prior to the study, and (3) had at least 2 visits to a GP for each period of 1999 to 2001 and 2002 to 2004. We identified approximately 50 eligible charts at each clinic and extracted data from charts using a standardized data collection form on demographic factors, medications, laboratory results, and comorbid conditions including hypertension, stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis. The number of patients per GP (i.e. 50 charts) was chosen to represent patients’ demographic and clinical characteristics. Ethics approval was obtained from the University of Calgary and University of British Columbia Ethics Committees. The presence of diabetes was determined by documented information on the use of either insulin or an oral hypoglycemic agent, an indication of diabetes in any of the text notes from the chart, such as diabetes admission history, diabetic retinopathy history or other diabetes-specific complications or
fasting blood sugar 7.0 mmol/L or 2 h blood sugar in the 75 g oral glucose tolerance test 11.1 mmol/L. Patients without the above documented information were assigned ‘‘absence of diabetes’’.
2.2.
Administrative data
The GP patients were linked with physician claims and hospital discharge abstract administrative data from 1999 to 2004 using personal unique identifier (i.e. personal health number). Fee-for-service physicians in Canada submit billing claims to their respective provincial government insurance program to reimburse for all services they provided. These claims contain patients’ personal health number, type of service provided as well as one ICD-9 diagnosis code in British Columbia and up to 3 ICD-9 codes in Alberta. Hospital discharge abstract data included up to 16 ICD-9 diagnosis codes prior to April 1, 2002, and up to 25 ICD-10 diagnoses codes after March 31, 2002. We determined the number of the physician claims and hospital discharge records with ICD-9 codes of 250.xx and ICD10 codes of E10.x–E14.x in the administrative data and then defined the presence of diabetes using the following algorithms: (1) 2 physician claims or 1 hospitalization with the ICD code; (2) 2 physician claims with the ICD code; (3) 1 physician claim or 1 hospitalization with the ICD code and (4) 1 physician claim with the ICD code. We further specified the gap between 2 claims by 1, 2, and 3 years.
2.3.
Statistical methods
We calculated sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and kappa values for each diabetes algorithm described above, and compared it with the chart review data (reference standard). These statistics were stratified by sex, age groups, comorbid conditions, study period, rural and urban areas, and by province. Difference in sensitivities and PPV between study periods and regions were calculated, while controlling for sex, age group, and comorbid conditions in a log-linear regression model [11]. All statistical analyses were performed with the statistical software SAS (version 9.2).
3.
Results
From the chart review, 36.3% patients were male, and the mean age was 52.8 years. A total of 1642 (48.8%) individuals had at least one comorbid condition other than diabetes. Prevalence of diabetes was 8.1% in the chart review data (see Table 1). The 8 ICD algorithms for defining diabetes (see Table 2) had a high sensitivity (ranging from 91.2% to 95.6%), specificity (ranging from 92.8% to 97.6%), and NPV (ranging from 99.2% to 99.6%) when 3-year data were analyzed. The algorithm of ‘‘2 physician claims within 2 years or 1 hospitalization’’ allowed for maximal PPV of 77.2% while maintaining a sensitivity of 92.3%. The algorithm of ‘‘1 physician claim or 1 hospitalization’’ had a kappa value of 0.65 and the other 7 algorithms had kappa values 0.79. However, when these algorithms were
191
diabetes research and clinical practice 89 (2010) 189–195
Table 1 – Characteristics of study samples in chart review data. Year
Number of patients Age (mean SD, years)
Provincea
Region
Total
2001
2004
Urban
Rural
AB
BC
1482 53.2 12.5
1880 52.4 12.8
2372 52.2 12.6
990 54.2 12.7
1565 52.3 12.9
1797 53.2 12.5
3362 52.8 12.7
Sex (%) Male Female
506 (34.3) 973 (65.7)
711 (37.8) 1169 (62.2)
796 (33.6) 1576 (66.4)
424 (42.8) 566 (57.2)
641 (34.6) 1024 (65.4)
679 (37.8) 1118 (62.2)
1220 (36.3) 2142 (63.7)
Comorbid conditionsa (%) Yes No
632 (42.6) 850 (57.4)
1010 (53.7) 870 (46.3)
1134 (47.8) 1238 (52.2)
508 (51.3) 482 (48.7)
760 (48.6) 805 (51.4)
882 (49.1) 915 (50.9)
1642 (48.8) 1720 (51.2)
105 (7.1) 1377 (92.9)
168 (8.9) 1712 (91.1)
186 (7.8) 2186 (92.8)
87 (8.8) 903 (91.2)
122 (7.8) 1443 (92.2)
151 (8.4) 1646 (91.6)
273 (8.1) 3089 (91.9)
Diabetes (%) Yes No
a AB = province of Alberta and BC = province of British Columbia. Comorbidity includes stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis.
applied in the 2-year data, validity became lower than that in the 3-year data. The stratified validity indexes for the algorithm of ‘‘2 physician claims within 2 years or 1 hospitalization’’ are shown in Table 3. These indexes varied little by region (i.e. rural and urban, and provinces), year (2001 and 2004), sex and age. However, PPV was higher for patients with comorbid conditions compared to those without (see Table 4, relative PPV: 1.15, 95% confidence interval: 1.03–1.28).
4.
Discussion
We validated diabetes recorded in administration data and found that the algorithm of ‘‘2 physician claims within 2 years or 1 hospitalization with ICD-9 code of 250.xx or ICD-10 code of E10.x–E14.x’’ had high validity. The validity remained similar across time periods and regions. To optimize the validity, at least 3-year data was needed. Our findings were consistent with some previous studies’ reports. Saydah et al. [7] reviewed validation studies of administrative data in recording diabetes, which were published from 1966 to 2002. Of the 16 studies reviewed, the sensitivity ranged from 46% to 97%; the PPV ranged from 60% to 97% and the kappa value ranged from 0.67 to 0.96. We updated the literature (see Table 5) to February 2010. Of 6 studies reviewed, the sensitivity ranged from 59% to 86%; the PPV ranged from 63% to 99% and the kappa value ranged from 0.60 to 0.83. The variation among these studies may reflect the fact that validity of administrative data is inconsistent across regions or that validity is influenced by inconsistency in study methods, such as reference standards, type of administrative data, study population and data collection mode. Hux et al. [12] reviewed GP charts and linked the chart data with physician claims and hospital discharge abstract administrative data in the province of Ontario, Canada. Compared to that study we found higher sensitivity (86.0% versus 92.3%) and slightly lower PPV (80.0% versus 77.2%). Lix et al. [13] validated diabetes recorded in physician claims and hospital discharge data using self-reported diabetes in a community survey in the province of Manitoba, Canada as a reference
standard. Lix reported sensitivity of 79.5% and PPV of 87.9%. All three Canadian studies (including this study) demonstrated a similar kappa value (ranging from 0.79 to 0.82). Quan et al. [14] validated hospital discharge abstract administrative data in the province of Alberta, Canada and found relatively low sensitivity of 59.1% and PPV of 63.1% for identifying diabetes with complications. This indicates that diabetes could be accurately identified using both the physician claims and hospital discharge abstract administrative data. The strength of our study is that we validated diabetes in administrative data across time periods and regions. After adjustment for potential confounding of age, sex and comorbid conditions there was no significant variation by time and region. However, PPV was higher for patients with comorbidities than those without comorbidities. That may be related to the nature of administrative data. The data captures patients who visited physicians or were hospitalized and missed those who did not see physicians during the study period. Diabetic patients with comorbidities may more frequently visit physicians compared to patients without comorbidities, consequently physicians may be more likely to submit claims with diabetes for those with comorbidities. In contrast, when the competing comorbid conditions are severe or potentially fatal, physicians may be more likely to claim these conditions rather than diabetes. Quan et al. [14] found that validity of hospital discharge abstract administrative data was higher for identifying diabetes without complications than that with complications. Diabetes validity may be influenced by the features and sources of administrative data. There is 1 ICD-9 diagnosis code in British Columbia, and up to 3 ICD-9 diagnosis codes recorded in Alberta physician claims data. The diabetes code might not be coded for diabetic patients if their main purpose of the visit was not for diabetes, such as a cold or other medical examination. In addition, when the competing comorbid conditions are severe or potentially fatal, physicians may be more likely to claim these conditions rather than diabetes [9]. This may potentially bias the validity for identifying diabetes cases in administrative data. In Alberta, up to 3 ICD-9 diagnosis-coding fields are available for each claim. However, the majority of physicians
192
Table 2 – Validity of administrative data diabetes case definition compared with chart data. 3 years observation period data PPV (%)
*
NPV (%)
Kappa
96.8 (96.2–97.4)
71.8 (67.1–76.5)
99.3 (99.0–100.0)
0.79 (0.75–0.82)
96.9 (96.2–97.5)
77.2 (72.5–81.7)
99.3 (99.0–99.6)
0.79 (0.75–0.83)
77.3 (72.3–82.3)
99.0 (98.6–99.4)
87.2 (82.9–91.4)
98.0 (97.5–98.5)
97.2 (87.9–94.6)
73.8 (68.7–78.4)
99.2 (99.0–100.0)
0.80 (0.76–0.83)
75.5 (70.4–80.6)
99.1 (98.7–99.4)
87.7 (83.4–91.9)
97.9 (97.4–98.4)
91.9 (88.7–95.2)
97.3 (96.7–97.9)
74.9 (70.2–80.0)
99.3 (99.0–100.0)
0.81 (0.77–0.84)
91.9 (89.0–94.9)
97.4 (96.7–97.9)
75.4 (70.8–80.0)
99.3 (99.0–99.5)
0.81 (0.78–0.85)
76.6 (71.5–81.6)
99.3 (99.0–99.6)
90.9 (87.2–94.6)
98.0 (97.5–98.4)
91.2 (87.9–94.6)
97.6 (97.1–98.1)
72.1 (67.5–76.9)
99.2 (99.9–99.8)
0.82 (0.79–0.85)
74.4 (69.2–79.5)
99.4 (99.1–99.7)
91.4 (87.8–95.1)
97.8 (97.3–98.3)
95.6 (92.5–97.7)
92.8 (91.9–93.7)
54.0 (49.6–58.5)
99.6 (99.4–99.8)
0.65 (0.61–0.69)
86.4 (82.4–90.5)
97.1 (96.5–97.7)
72.4 (67.5–77.3)
98.8 (98.4–99.2)
91.2 (87.9–94.6)
97.6 (97.1–98.1)
72.1 (67.5–76.9)
99.2 (98.9–99.5)
0.82 (0.78–0.85)
76.6 (71.5–81.6)
99.3 (99.0–99.6)
90.9 (87.2–94.6)
98.0 (97.5–98.4)
Sensitivity (%) Specificity (%) 2 claims or 1 hospitalization 2 claims 92.3 (89.2–95.5) gap 3 years 2 claims 92.3 (89.2–95.5) gap 2 years 2 claims 91.6 (88.3–94.6) gap 1 year 2 claims only 2 claims gap 3 years 2 claims gap 2 years 2 claims gap 1 year 1 claim or 1 hospitalization 1 claim only
*
2 years observation period data
*
PPV = positive predictive value, NPV = negative predictive value.
PPV* (%)
Sensitivity (%) Specificity (%) –
–
–
–
–
–
NPV* (%) –
–
Kappa – 0.80 (0.77–0.84) 0.80 (0.76–0.84)
– 0.82 (0.78–0.86) 0.80 (0.77–0.85) 0.77 (0.73–0.81) 0.82 (78.0–85.5)
diabetes research and clinical practice 89 (2010) 189–195
Algorithm
193
diabetes research and clinical practice 89 (2010) 189–195
Table 3 – The validity index for the coding algorithm of 2 physician claims within 2 years or 1 hospitalization by region, year, province and demographic factorsa. N
Prevalence (%)
Kappa value
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%)
Region Rural Urban
990 2372
9.2 10.4
0.83 0.82
92.0 90.9
97.5 97.7
76.8 77.7
99.3 99.2
Year 2001 2004
1482 1880
9.3 9.8
0.76 0.86
90.5 91.7
96.9 98.2
78.8 83.2
99.3 99.2
Province AB BC
1565 1797
8.0 11.0
0.88 0.78
90.2 92.1
99.0 96.4
88.0 78.2
99.2 99.3
Sex Female Male
2142 1220
9.0 10.8
0.81 0.83
90.5 91.9
98.0 96.9
75.6 78.6
99.3 99.0
Age (years) <65 65
2709 653
7.1 20.2
0.82 0.81
96.1 88.0
94.0 98.4
75.0 78.5
99.2 99.2
Comorbid presencea Yes 1642 No 1720
14.4 5.6
0.83 0.77
90.1 95.0
96.9 98.1
81.4 65.5
98.5 99.8
a PPV = positive predictive value, NPV = negative predictive value, AB = province of Alberta, and BC = province of British Columbia. Comorbidity includes hypertension, stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis.
submit only 1 diagnosis. In our data, 74% of claims had only 1 diagnosis recorded, while 4% recorded 2 diagnoses, 2% recorded 3 diagnoses and 20% were missing diagnoses (services for diagnostics and laboratory tests are not required to record diagnoses). The validity of the diabetes definition had little change when more than one coding field was utilized to define diabetes. Similarly, adding emergency department visits to the diabetes definition contributed little to the validity of the case definition. Comprehensiveness of administrative data is related to the healthcare system and data collection system. Canada has a government-owned universal healthcare insurance system. Traditional fee-for-service programs remunerate physicians for each medical service provided, as determined by details from the physicians’ claims submissions. If the physician does not provide a claim, he/she will not be paid for the service provided. The physician submission is entered into the claims
database regardless of where the service was provided. Hospital discharge abstract data from all hospitals are centralized and stored in the provincial health ministries. Therefore, both databases (physician claims and inpatient) capture relatively complete and comprehensive information on inpatient and outpatient physician services for all specialties, including general medicine. Application of our study findings depends on the availability of administrative data and study purposes. Surveillance is generally aimed to capture all prevalent and incidence diabetes cases. Health services researchers utilize administrative data for evaluation of outcomes and resources utilization. Utilization of hospital discharge abstract data alone in surveillance and health services research misses a large number of diabetes cases because many diabetes patients are managed at outpatient clinics. In addition, severity of disease between inpatients and outpatients may
Table 4 – Relative sensitivity and positive predictive value (95% confidence interval) for diabetes defined using 2 physician claims within 2 years or 1 hospitalization case definition in administrative data. Variable Comorbidity (yes versus no) Age (65 years versus <65 years) Sex (female versus male) Region (urban versus rural) Year (2004 versus 2001) Province (BC versus AB) a
Relative sensitivity (95% confidence interval) 0.88 0.91 0.99 1.04 1.02 1.04
(0.76–1.02) (0.83–1.00) (0.93–1.06) (0.94–1.16) (0.94–1.10) (0.96–1.12)
P-value 0.10 0.06 0.84 0.42 0.69 0.34
Relative PPVa (95% confidence interval) 1.15 1.08 1.05 1.06 1.09 0.92
(1.03–1.28) (0.99–1.16) (0.95–1.15) (0.96–1.17) (0.99–1.21) (0.83–1.01)
P-value 0.01 0.07 0.35 0.26 0.07 0.70
PPV = positive predictive value, AB = province of Alberta, and BC = province of British Columbia. Comorbidity includes hypertension, stroke, dementia, dyslipidemia, coronary artery disease, peripheral vascular disease, congestive heart failure, chronic pulmonary disease, asthma, cancer, depression, chronic kidney disease, and dialysis.
194
diabetes research and clinical practice 89 (2010) 189–195
Table 5 – Previous studies of validating administrative data in identifying diabetes. NPV* (%)
96.9
77.2
99.3
0.79
75.8
98.7
88.5
96.8
0.79
59.1
99.0
63.1
98.9
0.6
82.8
98.9
99.4
71.2
0.74
74.6
99.7
93.4
98.7
0.82
86.1
97.1
80.0
98.1
0.80
79.5
99.3
87.9
98.7
0.82
70.6
84.7
99.4
–
Reference, population
This study, Canada, 2001 and 2004
Physician charts, 35 years old
ICD-9 and ICD-10, 2 physician claims within 2 years or 1 hospitalization
92.3
Quan et al. [14], Canada, 2003
Hospital chart, 18 years old
ICD-10 hospital data for diabetes without complications ICD-10 hospital data for diabetes with complications ICD-9 and ICD-10, 4 physician claims within 2 years ICD-10 hospital or birth data
Guttmann et al. [8], Canada, 1994–2003 Bell et al. [15], Australia, 2002 Hux et al. [12], Canada, 1991–1999
Hospital chart, <19 years old Medical chart, pregnant women Physician charts, 20 years old
Lix et al. [13], Canada, 2001
Community Health Survey, 19 years old
Zgibor et al. [9], USA, 2001–2003
Physician chart, 18 years old
Administrative data and definition
PPV* (%)
Author, study location and year
ICD-9 and ICD-10, 2 physician claims within 2 years or 1 hospitalization ICD-9 and ICD-10, 2 physician claims within 2 years or 1 hospitalization Electric health records, any 3 indicators (including physician visits, hospitalization, ER, medication using ICD-9, any record of A1c measures, glucose > 200 mg/dl and ER)
Sensitivity Specificity (%) (%)
Kappa
–
Note: Saydah et al. [7] reviewed literature published from 1966 to mid-2002. We updated the literature to February 2010. PPV = positive predictive value, NPV = negative predictive value.
*
be significantly different and outcomes evaluation using hospital discharge data alone is likely to be biased. Therefore, our validated algorithm is useful for identifying diabetic inpatients and outpatients. Our study has limitations. Firstly, the GP’ chart data were created as the ‘‘reference standard’’. This ‘‘reference standard’’ reflects the portion of diabetes diagnosed by a physician. Persons who did not see GPs or who were not diagnosed by the GPs might be missed. For example, a patient may be hospitalized and diagnosed with diabetes by a specialist but the diagnosis information was not forwarded to their GP. To improve the reference standard, inpatient charts and laboratory test data should be utilized to establish a more reliable standard than chart data. Secondly, we selected GP clinics in urban and rural areas but missed clinics in remote or suburban areas. Under the Canadian universal healthcare system, it is expected that GP billing practice may be similar across regions. However, further study is needed. Thirdly, validity may vary by age group but we only included patients aged 35 or older. However Guttmann et al. [8] validated diabetes in administrative data for children and reported similar results to this study. In conclusion, our study is unique as, to the best of our knowledge, it is the first to compare the validation of diabetes from both inpatient and outpatient administrative data between time periods and regions. The validation of the diabetes case definition ‘‘2 physician claims within 2 years or 1 hospitalization’’ was robust between age groups, sex, rural and urban areas, provinces, and years. But patients with comorbidities had a slightly higher PPV than those without.
Acknowledgements The data collection was funded by Canadian Institutes of Health Research. Dr. Quan is supported by Alberta InnovateHealth Solution salary award. Dr. Khan is supported by a New Investigator Award from the Canadian Institutes of Health Research.
Conflict of interest All authors of this paper have no conflicts of interest to declare.
references
[1] C.C. Cowie, K.F. Rust, E.S. Ford, M.S. Eberhardt, D.D. ByrdHolt, C. Li, et al., Full accounting of diabetes and prediabetes in the U.S. population in 1988–1994 and 2005–2006, Diabetes Care 32 (2009) 287–294. [2] L.L. Lipscombe, J.E. Hux, Trends in diabetes prevalence, incidence, and mortality in Ontario, Canada 1995–2005: a population-based study, Lancet 369 (2007) 750–756. [3] American Diabetes Association, Economic costs of diabetes in the U.S. in 2007, Diabetes Care 31 (2008) 596–615. [4] S. Wild, G. Roglic, A. Green, R. Sicree, H. King, Global prevalence of diabetes: estimates for the year 2000 and projections for 2030, Diabetes Care 27 (2004) 1047–1053. [5] J. Goldman, S. Kinnear, J. Chung, D.J. Rothman, New York City’s initiatives on diabetes and HIV/AIDS: implications for
diabetes research and clinical practice 89 (2010) 189–195
[6] [7]
[8]
[9]
[10]
patient care, public health, and medical professionalism, Am. J. Public Health 98 (2008) 807–813. A.L. Fairchild, Public health. Diabetes and disease surveillance, Science 313 (2006) 175–176. S.H. Saydah, L.S. Geiss, E. Tierney, S.M. Benjamin, M. Engelgau, F. Brancati, Review of the performance of methods to identify diabetes cases among vital statistics, administrative, and survey data, Ann. Epidemiol. 14 (2004) 507–516. A. Guttmann, M. Nakhla, M. Henderson, T. To, D. Daneman, K. Cauch-Dudek, et al., Validation of a health administrative data algorithm for assessing the epidemiology of diabetes in Canadian children, Pediatr. Diabetes 11 (2009) 122–128. J.C. Zgibor, T.J. Orchard, M. Saul, G. Piatt, K. Ruppert, A. Stewart, et al., Developing and validating a diabetes database in a large health system, Diabetes Res. Clin. Pract. 75 (2007) 313–319. R.M. Andersen, Revisiting the behavioral model and access to medical care: does it matter? J. Health Soc. Behav. 36 (1995) 1–10.
195
[11] M.S. Pepe, The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press, Oxford; New York, 2003. [12] J.E. Hux, F. Ivis, V. Flintoft, A. Bica, Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm, Diabetes Care 25 (2002) 512–516. [13] L. Lix, M. Yogendran, C. Burchill, C. Metge, N. McKeen, D. Moore, et al., Defining and validating chronic diseases: an administrative data approach, in: Policy MCfH, Manitoba, 2004. [14] H. Quan, B. Li, L.D. Saunders, G.A. Parsons, C.I. Nilsson, A. Alibhai, et al., Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database, Health Serv. Res. 43 (2008) 1424–1441. [15] J.C. Bell, J.B. Ford, C.A. Cameron, C.L. Roberts, The accuracy of population health data for monitoring trends and outcomes among women with diabetes in pregnancy, Diabetes Res. Clin. Pract. 81 (2008) 105–109.