- Email: [email protected]
The cost to providers of participating in an immunization registry
The Cost to Providers of Participating in an Immunization Registry Kimberly J. Rask, MD, PhD, Kristen J. Wells, BA, Susan A. Kohler, RN, MPH, Cynthia T. Rust, MD, Charles B. Cangialose, PhD Introduction: The medical and public health communities advocate immunization registries as one tool to achieve national immunization goals. Although substantial resources have been expended to establish registries across the nation, minimal research has been conducted to evaluate provider participation costs. Methods:
The objective of this study was to identify the direct costs to participate in an immunization registry. To estimate labor and equipment costs, we conducted interviews and direct observation at four sites that were participating in one of two immunization registries. We calculated mean data-entry times from direct observation of clinic personnel.
Results:
The annual cost of participating in a registry varied extremely, ranging from $6083 to $24,246, with the annual cost per patient ranging from $0.65 to $7.74. Annual per-patient costs were lowest in the site that used an automated data-entry interface. Of the sites requiring a separate data-entry step, costs were lowest for the site participating in the registry that provided more intensive training and had a higher proportion of the target population entered into the registry.
Conclusions: Ease of registry interface, data-entry times, and target population coverage affect provider participation costs. Designing the registry to accept electronic transfers of records and to avoid duplicative data-entry tasks may decrease provider costs. Medical Subject Headings (MeSH): costs and cost analysis, health personnel, immunization, registries, vaccination (Am J Prev Med 2000;19(2):99 –103) © 2000 American Journal of Preventive Medicine
Introduction
H
1
ealthy People 2000 established a national goal for the year 2000 to completely immunize 90% of infants by 24 months of age. Immunization registries are one tool to achieve these goals.2–5 Participating providers can use registries to consolidate scattered records6; to provide an immunization needs assessment for each patient; to provide current immunization recommendations; to promote automated recall of underimmunized children; to document immunizations for schools, preschools, and camps; to help manage vaccine inventories; to provide practice-based immunization coverage assessments; and to calculate Health Plan Employer Data Information Set (HEDIS)7
From the Division of General Medicine, Emory University School of Medicine (Rask, Kohler), Department of Health Policy and Management, Rollins School of Public Health, Emory University (Rask, Wells), Department of Pediatrics, Emory University School of Medicine (Rust), Atlanta, Georgia; and Health Funding Authority (Cangialose), Wellington, New Zealand Address correspondence and reprint requests to: Kimberly J. Rask, MD, PhD, Department of Health Policy and Management, Rollins School of Public Health, 1518 Clifton Road, NE, Room 636, Atlanta, Georgia, 30322. E-mail: [email protected].
reporting requirements for managed care plans.5 The public and private sectors have funded the development and maintenance of immunization registries.8 Although substantial effort has been involved in establishing registries across the nation, only six of the 64 national immunization projects report active participation from a significant proportion of private providers.8 Only two registries have 100% of community private providers participating, and only four registries have more than 50% of private providers participating.9 Focus groups have identified several challenges to registry implementation in private practices. These include staff concerns about double entry of data, slowing of patient flow, staff time required for data entry, disproportionately high costs for small practices with limited staffs, and high staff turnover.10,11 The direct costs to a physician practice for connecting to the registry can include hardware costs, software costs, and staff time for training and data entry. However, little work has been done to quantify these costs. The purpose of this study was to measure the direct costs incurred by a sample of health care providers who currently participate in local immunization registries.
Am J Prev Med 2000;19(2) 0749-3797/00/$–see front matter © 2000 American Journal of Preventive Medicine • Published by Elsevier Science Inc. PII S0749-3797(00)00183-5
99
The study examined the following questions: (1) Are participation costs affected by the method the provider uses to enter and retrieve data from the registry? and (2) Do participation costs vary across registries?
Methods We studied provider sites that were participating in either the Metro Atlanta Team for Child Health (MATCH) registry or the Chatham County All Kids Count registry. The MATCH registry is a community-based partnership among two county health agencies, local nonprofit agencies, and community health centers. The services that MATCH provides include record look-ups, real-time data interface or batch data-entry interface, and the capability to generate reminder and recall notices either through postcards or an autodialer. Among the pediatric practices participating in the registry are public health clinics, community health centers, and hospital-based pediatric clinics. The Chatham County (Georgia) All Kids Count registry is a component of the Health Outcomes and Services Tracking (HOST) system that was developed by the Center for Health Information through the Division of Public Health in the State of Georgia.11 HOST is an integrated children’s health care system that includes health check; Special Supplemental Nutrition Program for Women, Infants, and Children (WIC); and immunization information. The services this registry provides include record look-ups; manual data entry; generation of reminder and recall notices through postcards, letters, or an autodialer; and practice assessment of immunization coverage. The HOST system has been implemented in all three birth hospitals, all public health sites, all community health center sites, a hospital pediatric clinic, and 98% of the private childhood immunization providers practicing in Chatham County. This study focused on the specific resources (e.g., personnel and equipment) expended to maintain an interface with the registry. The four provider sites evaluated in the study were selected because they (1) consistently participated in one of the registries as reported by registry activity logs, and (2) used different registry interfaces. We conducted interviews with provider site personnel to define the registryrelated processes, evaluate patient flow in the clinic, and determine the method of entering and retrieving data from the registry. Following the interviews and visits to each provider site, we created flow charts to depict the process of obtaining and distributing registry information at each provider site. Two different types of registry interfaces were examined—manual and automated entry. For the purpose of this study, an automated entry interface (Site A) was defined as having data downloaded directly into the registry from a billing or patient-management system. A manual-entry interface (Sites B, C, D) was defined as having registry data entry performed by clinic personnel.
Settings Site A was an urban county health department that processes data for five public health clinics, which provided approximately 50,800 immunizations to 18,500 children and adolescents in 1998. Each clinic enters data into a patient-manage-
100
ment database. An automated process retrieves data from the patient-management database and downloads the data into the registry. Site B was an urban hospital-based pediatric clinic, staffed primarily by physician assistants and nurse practitioners, and annually provides approximately 5600 immunizations to 2300 children. Site C was a pediatric clinic at the same hospital that is staffed by pediatric residents and provides approximately 10,900 immunizations to 4400 children each year. Sites B and C had two different registry interfaces during the study period. Initially, Sites B and C shared a dedicated employee who was responsible for entering and updating immunization records from both clinics into the registry; the cost estimates for this model are identified as Site BC. Both clinics subsequently transitioned to a model in which clinic assistants and technicians assumed the responsibility for entering and editing demographic records, whereas the nurse seeing the patient at the visit in which an immunization was given was responsible for updating and accessing registry records. The cost estimates for this model are designated as Site B and Site C, respectively. Site D was an urban community health center that provides primary care to children, adolescents, and adults. A nurse practitioner and a pediatrician staff the pediatric department of the clinic, which annually provides approximately 2400 immunizations to 1400 children. Personnel at Site D manually enter data into the registry. All of the sites provide medical care to a low-income population, the majority of whom are uninsured, underinsured, or Medicaid recipients. Sites A, B, and C participate in the same registry.
Cost Enumeration We used direct observation and interviews to calculate the cost of personnel effort used to perform registry-related functions. We interviewed full-time personnel dedicated to registry activities at Sites A and BC to calculate the amount of effort used for registry activities. Estimates of the effort expended to perform registry activities at Sites B, C, and D were made from direct observation because personnel at these sites were not dedicated to registry activities alone. We gathered observation data using standardized forms. The information gathered during observation included the amount of time that personnel spent using the registry, the reason that the registry was accessed, the result of each transaction, the number of patients examined during a clinic session, the number of patients immunized during a clinic session, whether the registry was accessed during each clinical encounter, and the type of personnel who accessed the registry. A transaction was defined as any type of interaction with the registry terminal and included the following activities: (1) verification of a patient record; (2) verification of patient’s immunization status; (3) entry of demographic data; (4) editing of an existing record; (5) entry of immunization data; (6) printing an immunization record; (7) printing a school, preschool, or camp certificate; (8) logging on to the computer; and (9) logging off of the computer. To estimate data entry and editing time, we calculated the mean time for demographic data entry, immunization data entry, and editing existing records from observations made at each provider site. We performed t tests for independent samples to compare the differences in observed data-entry times. Statistical analyses were conducted using SPSS威 for Windows.12
American Journal of Preventive Medicine, Volume 19, Number 2
Table 1. Annual provider cost estimates (1998 dollars)
Provider site Site A Site B Site C Site BC Site D
Type of provider County health department Hospital-based primary care clinic Hospital-based primary care clinic Hospital-based primary care clinic Community health center
Method of data entry Automated interface Manual data entry by clinic staff Manual data entry by clinic staff Manual data entry by a data-entry specialist Manual data entry by clinic staff
Data Entry Validation We queried the immunization registry 8 months following the last observation period at Sites B and C to ascertain the proportion of shots given in the clinic that were entered into the registry database. The immunization information recorded on the Vaccines for Children pharmacy log at the clinic site was considered the gold standard.
Cost Analyses To allow comparability across sites, we estimated salaries for clinic personnel from public labor statistics13 and adjusted them to 1998 dollars. Where salary information was not available from public labor statistics, we made estimates by using the mean of the salary range for the same job description at the same provider site. We added a fixed benefits rate of 22.8% to all salaries. We also used interviews to identify the specific equipment required to participate in the registry. The estimated cost for computer equipment was based on acquisition costs for comparable systems at a local university. Hardware costs were amortized over 5 years at a 5% discount rate with 10% scrap value.14 All cost analyses were performed using Microsoft Excel威.
Results The average number of patients seen during a half-day clinic session at the observed sites ranged from 11 to 21, whereas the average number of immunizations administered ranged from 8 to 14. When clinic staff attempted to access patient records from the registry at the time of the patient encounter, they found a majority of records (83%, n⫽29) at Site D, but found only 56 (45%) and 14 (29%) of the patient records at Sites B and C, respectively. When the registry was queried to assess completeness of data entry, only 56 of the 94 patients (60%) immunized at Sites B and C had immunizations recorded correctly.
Participation Cost Estimates We used patient volume estimates, data-entry times, and participation rates to calculate the total annual cost for registry participation (e.g., data entry, data verifica-
Patients vaccinated per year and entered
Shots per year entered
Annual cost of Annual participating cost per in registry patient
Annual cost per shot
18,500
50,800
$12,065
$0.65
$0.24
1,800
4,300
13,938
7.74
3.24
4,000
9,900
16,377
4.09
1.65
6,600
16,500
24,246
3.67
1.47
1,400
2,400
6,083
4.35
2.53
tion, producing shot records, logging on and off). Because Site A used an automated data dump from the patient-management system to enter records into the registry, we assumed that 100% of shots given would be entered into the registry. At Site D, the volume estimates were based on a registry report that lists the number of actual shots entered. At Sites B and C, we based volume estimates on the observation periods. Table 1 gives a summary of provider site characteristics and registry participation costs. Participation costs per child and per immunization varied widely across the sites. As expected, we found the lowest costs at the site that did not require any manual data entry by clinic personnel. Automated data entry remained the least costly interface, even assuming that the proportion of records entered was substantially less than 100%. A combination of personnel time and fixed equipment expenditures drove participation costs at clinics with manual data entry. Although Site D had the lowest labor costs, its smaller number of patients (and shots) resulted in higher equipment costs per patient (and per shot entered). Costs were lower in Site B and Site C when a dedicated employee (Site BC, 0.8 full-time equivalent [FTE]) rather than usual clinic staff (Site B, cumulative 0.42 FTE; and site C, cumulative 0.48 FTE) was used to enter immunization records into the registry.
Data Entry To evaluate whether the registry interface affected participation costs, data-entry times for Clinics B and C (Registry 1) were compared to data-entry times for Clinic D (Registry 2). In all three clinics, usual clinic staff performed immunization record entry into the registry. As seen in Table 2, the time required to enter or to edit a record in the registry was extremely variable. At Sites B and C, 16 different staff members were observed interfacing with the registry, whereas at Site D five staff members, who accessed the registry for every patient seen during the clinic session, entered the immunizations. Clinic personnel at Sites B and C spent Am J Prev Med 2000;19(2)
101
Table 2. Comparison of data entry observation time Data entry times (in seconds)
Mean
Table 3. Estimated yearly data entry cost (1998) dollars)
Range
SD
Immunization record Sites B and C 147.92 Site D 57.00
20–420 30–90
113.71 22.25
3.60
.001
Demographic record Sites B and C 306.43 Site D 87.50
30–600 45–150
159.69 40.71
3.27*
.004
t
p
*Unequal variance t-test SD, standard deviation
on average 2.47 minutes entering new shots and 5.10 minutes entering a demographic record, with some patient entries requiring more than 10 minutes. The immunization data entered included the date and type of immunization. The required demographic information included the patient name, address, telephone number, medical record number, date of birth, race, gender, parents’ names, parents’ address, and parents’ telephone number. Personnel at Site D spent approximately 57 seconds entering new shots and 1.47 minutes entering a demographic record. The immunization data entered included immunization dose, date of immunization, type of immunization, manufacturer, lot number, site of immunization, route of immunization, expiration date, and the provider who administered the immunization. Despite entering substantially more information, clinic personnel spent less time on data entry, and the difference was statistically significant. Because most patient records were already entered in the registry, the entry of a demographic record at Site D typically consisted of entering only a change in the last point of contact, the last provider to see the patient, or the date the next immunization was due. The difference between mean demographic data-entry times was also statistically significant.
Data Entry Cost Estimates We used the data-entry time estimates to estimate the annual cost of data entry at the three sites that used general clinical staff to manually enter records into the registry (Table 3). The annual data-entry cost per patient at Sites B and C was higher than the annual data-entry cost per patient at Site D because of the longer time required for data entry and retrieval. These clinics’ ability to spread fixed equipment costs over a larger number of patients tempered the higher labor costs.
Discussion This study was a unique attempt to determine the direct costs incurred by providers who participate in an immunization registry. In addition, this study points out 102
Type of cost
Number of patients
Sites B and C Labor Equipment Total
6600
Site D Labor Equipment Total
1400
Annual cost
Annual cost per patient
$7951 $2030 $9981
$1.21 $0.31 $1.51
$ 688 $1015 $1703
$0.49 $0.72 $1.22
some of the operational challenges that need to be solved for immunization registries to achieve their potential. Annual participation costs per patient and per immunization were the least expensive for the public health department that used an automated data-entry interface. We noted similar findings in an evaluation of All Kids Count registries.15 Although less costly, the automated interface had disadvantages at the clinic level because personnel in the five clinics were unable to look up recently updated registry data without contacting the central health department administrative staff. One cause of cost variation in sites with manual data entry is the amount of clinic personnel time required to enter or to edit records in the registry. Ease of the registry interface, the amount and quality of training that staff receive, and registry software that minimizes keystrokes are likely to significantly lower participation costs. Clinic patient volume also affects cost. Higher patient volumes in larger clinics decrease the cost per patient for fixed expenses, such as a dedicated registry computer. Smaller clinics may find it more difficult to finance registry participation. We found it difficult to determine whether the degree of registry integration into the patient flow increased or decreased costs. The site with the lowest labor requirement for data entry also had the most complete integration of registry activities into patient flow. The ease of the interface may have promoted regular use by clinic staff. However, in another clinic, the per-patient costs of registry participation were lower when a dedicated employee performed data entry than when clinic staff performed data entry, in part because the staff was less familiar with the computer program. One very important disadvantage of batch data entry was that records were rarely accessed during the clinical encounter. If registries are not used at every encounter, the opportunity to reduce “missed opportunities” for immunization cannot be realized. Finally, data-entry labor costs will be higher for clinics participating in a registry with incomplete capture of the target population. As most of the target population is included in one of the study registries at birth, demographic data-entry transactions by participating clinics consist of updates with minimal data-
American Journal of Preventive Medicine, Volume 19, Number 2
entry requirements. The disappointing accuracy of registry-immunization records from two of the study clinics confirms previous findings and highlights the importance of ongoing quality assurance efforts by registries.16,17 Several limitations to our study exist that may affect generalizability. Because registry development is idiosyncratic and our sample of clinics participating in two registries is small, it is difficult to generalize to different registries. These are preliminary findings that need to be confirmed in other settings. All of the study clinics serve urban, predominantly disadvantaged populations. Turnover in clinic personnel makes ease of registry interface and ongoing training efforts particularly important. To avoid seasonal visit shifts, project staff observed clinics on multiple occasions during different times of the year. We also revisited clinics after major personnel changes. In addition, we reviewed the Vaccines for Children logs for non-observation periods to verify that the number of patients immunized and the number of immunizations administered were similar to periods of observation. Because the study used a cross-sectional design, we could not measure whether, for example, the ability to produce shot records has offset some of the labor costs of participation. However, none of the study clinics has eliminated existing immunization paperwork; therefore, registry functions are performed in addition to usual paper documentation. This study is an important first step, and more work is necessary to develop reliable estimates of provider costs and savings attributable to registry participation. Costs are and will be an important determinant of community providers’ willingness to participate in registries. Provider costs can be decreased by designing the registry to accept electronic transfers of immunization records, designing user-friendly interfaces, investing in initial and ongoing training of provider-site personnel, and avoiding duplicative data-entry tasks. Greater automation of data transfer would be expected to reduce both participation costs and retraining costs associated with staff turnover. Immunization registries offer opportunities to improve health as well as the delivery of appropriate health care services. To realize this opportunity, however, registries need to pursue implementation strategies that ensure that the benefits of participation exceed the costs. This research was supported by Grant Number U1W/ CCU414706-01 from the National Immunization Program of
the Centers for Disease Control and Prevention (CDC). The contents are solely the responsibility of the authors and do not necessarily represent the official views of the CDC. Approval for this study was obtained from the Emory University Human Investigations Committee. The authors wish to thank Gene Downing, Marianne Pappas, and David Shields for their invaluable assistance in the implementation of this study.
References 1. U.S. Department of Health and Human Services. Healthy People 2000: national health promotion and disease prevention objectives. Pub. No. PHS91–50212. Washington, DC: U.S. Government Printing Office, 1991. 2. Gostin LO, Lazzarini Z. Childhood immunization registries: a national review of public health information systems and the protection of privacy. JAMA 1995;274:1793–9. 3. Codero JF, Orenstein WA. The future of immunization registries. Am J Prev Med 1997;13(suppl 1):122– 4. 4. Sinn JS, Kronenburg MA, Morrow AL. The purpose and functions of immunization information systems within health care organizations. Arch Pediatr Adolesc Med 1997;151:615–20. 5. Linkins RW, Feikema SM. Immunization registries: the cornerstone of childhood immunization in the 21st century. Pediatr Ann 1998;27:349 –54. 6. Yawn BP, Edmonson L, Huber L, Poland GA, Jacobson RM, Jacobsen SJ. The impact of a simulated immunization registry on perceived childhood immunization status. Am J Managed Care 1998;4:185–92. 7. National Committee for Quality Assurance. HEDIS 3.0: Updated specifications for HEDIS 3.0. Washington, DC: National Committee for Quality Assurance, 1997. 8. National Vaccine Advisory Committee. Development of community- and state-based immunization registries: findings and recommendations for action. 1999. Available at: www.cdc.gov/nip/registry/i_recs.htm. 9. Wood D, Saarlas KN, Inkelas M, Matyas BT. Immunization registries in the United States: implications for the practice of public health in a changing health care system. Annu Rev Public Health 1999;20:231–55. 10. Bordley WC, Dempsey-Tanner T, Freed GL, Lister ME. Challenges to private provider participation in immunization registries. Am J Prev Med 1997;13(suppl 1):66 –70. 11. Pappas M, Baker BJ, Appelbaum D, Hinds W, Alexander ER, Korolak K. Immunization registries: provider recruitment and participation. Am J Prev Med 1997;13(suppl 1):71– 6. 12. SPSS Inc. SPSS for Windows. Release 7.5.1. Chicago: SPSS Inc., 1996. 13. Georgia Department of Labor. Georgia Wage Survey, 1996. Atlanta, GA: Georgia Department of Labor, 1997. 14. Haddix AC, Teutsch SM, Shaffer PA, Dun ˜ et DO. Prevention effectiveness: a guide to decision analysis and economic evaluation, 1st ed. New York: Oxford University Press, 1996. 15. Slifkin RT, Freeman VA, Biddle A, et al. The cost of immunization registries: four case studies. A report prepared for the Robert Wood Johnson Foundation. Chapel Hill, NC: Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, 1998. 16. Wilton R, Pennisi AJ. Evaluating the accuracy of transcribed computerstored immunization data. Pediatrics 1994;94:902– 6. 17. Payne T, Kanvik S, Seward R, et al. Development and validation of an immunization tracking system in a large health maintenance organization. Am J Prev Med 1993;9:96 –100.
Am J Prev Med 2000;19(2)
103
The objective of this study was to identify the direct costs to participate in an immunization registry. To estimate labor and equipment costs, we conducted interviews and direct observation at four sites that were participating in one of two immunization registries. We calculated mean data-entry times from direct observation of clinic personnel.
Results:
The annual cost of participating in a registry varied extremely, ranging from $6083 to $24,246, with the annual cost per patient ranging from $0.65 to $7.74. Annual per-patient costs were lowest in the site that used an automated data-entry interface. Of the sites requiring a separate data-entry step, costs were lowest for the site participating in the registry that provided more intensive training and had a higher proportion of the target population entered into the registry.
Conclusions: Ease of registry interface, data-entry times, and target population coverage affect provider participation costs. Designing the registry to accept electronic transfers of records and to avoid duplicative data-entry tasks may decrease provider costs. Medical Subject Headings (MeSH): costs and cost analysis, health personnel, immunization, registries, vaccination (Am J Prev Med 2000;19(2):99 –103) © 2000 American Journal of Preventive Medicine
Introduction
H
1
ealthy People 2000 established a national goal for the year 2000 to completely immunize 90% of infants by 24 months of age. Immunization registries are one tool to achieve these goals.2–5 Participating providers can use registries to consolidate scattered records6; to provide an immunization needs assessment for each patient; to provide current immunization recommendations; to promote automated recall of underimmunized children; to document immunizations for schools, preschools, and camps; to help manage vaccine inventories; to provide practice-based immunization coverage assessments; and to calculate Health Plan Employer Data Information Set (HEDIS)7
From the Division of General Medicine, Emory University School of Medicine (Rask, Kohler), Department of Health Policy and Management, Rollins School of Public Health, Emory University (Rask, Wells), Department of Pediatrics, Emory University School of Medicine (Rust), Atlanta, Georgia; and Health Funding Authority (Cangialose), Wellington, New Zealand Address correspondence and reprint requests to: Kimberly J. Rask, MD, PhD, Department of Health Policy and Management, Rollins School of Public Health, 1518 Clifton Road, NE, Room 636, Atlanta, Georgia, 30322. E-mail: [email protected].
reporting requirements for managed care plans.5 The public and private sectors have funded the development and maintenance of immunization registries.8 Although substantial effort has been involved in establishing registries across the nation, only six of the 64 national immunization projects report active participation from a significant proportion of private providers.8 Only two registries have 100% of community private providers participating, and only four registries have more than 50% of private providers participating.9 Focus groups have identified several challenges to registry implementation in private practices. These include staff concerns about double entry of data, slowing of patient flow, staff time required for data entry, disproportionately high costs for small practices with limited staffs, and high staff turnover.10,11 The direct costs to a physician practice for connecting to the registry can include hardware costs, software costs, and staff time for training and data entry. However, little work has been done to quantify these costs. The purpose of this study was to measure the direct costs incurred by a sample of health care providers who currently participate in local immunization registries.
Am J Prev Med 2000;19(2) 0749-3797/00/$–see front matter © 2000 American Journal of Preventive Medicine • Published by Elsevier Science Inc. PII S0749-3797(00)00183-5
99
The study examined the following questions: (1) Are participation costs affected by the method the provider uses to enter and retrieve data from the registry? and (2) Do participation costs vary across registries?
Methods We studied provider sites that were participating in either the Metro Atlanta Team for Child Health (MATCH) registry or the Chatham County All Kids Count registry. The MATCH registry is a community-based partnership among two county health agencies, local nonprofit agencies, and community health centers. The services that MATCH provides include record look-ups, real-time data interface or batch data-entry interface, and the capability to generate reminder and recall notices either through postcards or an autodialer. Among the pediatric practices participating in the registry are public health clinics, community health centers, and hospital-based pediatric clinics. The Chatham County (Georgia) All Kids Count registry is a component of the Health Outcomes and Services Tracking (HOST) system that was developed by the Center for Health Information through the Division of Public Health in the State of Georgia.11 HOST is an integrated children’s health care system that includes health check; Special Supplemental Nutrition Program for Women, Infants, and Children (WIC); and immunization information. The services this registry provides include record look-ups; manual data entry; generation of reminder and recall notices through postcards, letters, or an autodialer; and practice assessment of immunization coverage. The HOST system has been implemented in all three birth hospitals, all public health sites, all community health center sites, a hospital pediatric clinic, and 98% of the private childhood immunization providers practicing in Chatham County. This study focused on the specific resources (e.g., personnel and equipment) expended to maintain an interface with the registry. The four provider sites evaluated in the study were selected because they (1) consistently participated in one of the registries as reported by registry activity logs, and (2) used different registry interfaces. We conducted interviews with provider site personnel to define the registryrelated processes, evaluate patient flow in the clinic, and determine the method of entering and retrieving data from the registry. Following the interviews and visits to each provider site, we created flow charts to depict the process of obtaining and distributing registry information at each provider site. Two different types of registry interfaces were examined—manual and automated entry. For the purpose of this study, an automated entry interface (Site A) was defined as having data downloaded directly into the registry from a billing or patient-management system. A manual-entry interface (Sites B, C, D) was defined as having registry data entry performed by clinic personnel.
Settings Site A was an urban county health department that processes data for five public health clinics, which provided approximately 50,800 immunizations to 18,500 children and adolescents in 1998. Each clinic enters data into a patient-manage-
100
ment database. An automated process retrieves data from the patient-management database and downloads the data into the registry. Site B was an urban hospital-based pediatric clinic, staffed primarily by physician assistants and nurse practitioners, and annually provides approximately 5600 immunizations to 2300 children. Site C was a pediatric clinic at the same hospital that is staffed by pediatric residents and provides approximately 10,900 immunizations to 4400 children each year. Sites B and C had two different registry interfaces during the study period. Initially, Sites B and C shared a dedicated employee who was responsible for entering and updating immunization records from both clinics into the registry; the cost estimates for this model are identified as Site BC. Both clinics subsequently transitioned to a model in which clinic assistants and technicians assumed the responsibility for entering and editing demographic records, whereas the nurse seeing the patient at the visit in which an immunization was given was responsible for updating and accessing registry records. The cost estimates for this model are designated as Site B and Site C, respectively. Site D was an urban community health center that provides primary care to children, adolescents, and adults. A nurse practitioner and a pediatrician staff the pediatric department of the clinic, which annually provides approximately 2400 immunizations to 1400 children. Personnel at Site D manually enter data into the registry. All of the sites provide medical care to a low-income population, the majority of whom are uninsured, underinsured, or Medicaid recipients. Sites A, B, and C participate in the same registry.
Cost Enumeration We used direct observation and interviews to calculate the cost of personnel effort used to perform registry-related functions. We interviewed full-time personnel dedicated to registry activities at Sites A and BC to calculate the amount of effort used for registry activities. Estimates of the effort expended to perform registry activities at Sites B, C, and D were made from direct observation because personnel at these sites were not dedicated to registry activities alone. We gathered observation data using standardized forms. The information gathered during observation included the amount of time that personnel spent using the registry, the reason that the registry was accessed, the result of each transaction, the number of patients examined during a clinic session, the number of patients immunized during a clinic session, whether the registry was accessed during each clinical encounter, and the type of personnel who accessed the registry. A transaction was defined as any type of interaction with the registry terminal and included the following activities: (1) verification of a patient record; (2) verification of patient’s immunization status; (3) entry of demographic data; (4) editing of an existing record; (5) entry of immunization data; (6) printing an immunization record; (7) printing a school, preschool, or camp certificate; (8) logging on to the computer; and (9) logging off of the computer. To estimate data entry and editing time, we calculated the mean time for demographic data entry, immunization data entry, and editing existing records from observations made at each provider site. We performed t tests for independent samples to compare the differences in observed data-entry times. Statistical analyses were conducted using SPSS威 for Windows.12
American Journal of Preventive Medicine, Volume 19, Number 2
Table 1. Annual provider cost estimates (1998 dollars)
Provider site Site A Site B Site C Site BC Site D
Type of provider County health department Hospital-based primary care clinic Hospital-based primary care clinic Hospital-based primary care clinic Community health center
Method of data entry Automated interface Manual data entry by clinic staff Manual data entry by clinic staff Manual data entry by a data-entry specialist Manual data entry by clinic staff
Data Entry Validation We queried the immunization registry 8 months following the last observation period at Sites B and C to ascertain the proportion of shots given in the clinic that were entered into the registry database. The immunization information recorded on the Vaccines for Children pharmacy log at the clinic site was considered the gold standard.
Cost Analyses To allow comparability across sites, we estimated salaries for clinic personnel from public labor statistics13 and adjusted them to 1998 dollars. Where salary information was not available from public labor statistics, we made estimates by using the mean of the salary range for the same job description at the same provider site. We added a fixed benefits rate of 22.8% to all salaries. We also used interviews to identify the specific equipment required to participate in the registry. The estimated cost for computer equipment was based on acquisition costs for comparable systems at a local university. Hardware costs were amortized over 5 years at a 5% discount rate with 10% scrap value.14 All cost analyses were performed using Microsoft Excel威.
Results The average number of patients seen during a half-day clinic session at the observed sites ranged from 11 to 21, whereas the average number of immunizations administered ranged from 8 to 14. When clinic staff attempted to access patient records from the registry at the time of the patient encounter, they found a majority of records (83%, n⫽29) at Site D, but found only 56 (45%) and 14 (29%) of the patient records at Sites B and C, respectively. When the registry was queried to assess completeness of data entry, only 56 of the 94 patients (60%) immunized at Sites B and C had immunizations recorded correctly.
Participation Cost Estimates We used patient volume estimates, data-entry times, and participation rates to calculate the total annual cost for registry participation (e.g., data entry, data verifica-
Patients vaccinated per year and entered
Shots per year entered
Annual cost of Annual participating cost per in registry patient
Annual cost per shot
18,500
50,800
$12,065
$0.65
$0.24
1,800
4,300
13,938
7.74
3.24
4,000
9,900
16,377
4.09
1.65
6,600
16,500
24,246
3.67
1.47
1,400
2,400
6,083
4.35
2.53
tion, producing shot records, logging on and off). Because Site A used an automated data dump from the patient-management system to enter records into the registry, we assumed that 100% of shots given would be entered into the registry. At Site D, the volume estimates were based on a registry report that lists the number of actual shots entered. At Sites B and C, we based volume estimates on the observation periods. Table 1 gives a summary of provider site characteristics and registry participation costs. Participation costs per child and per immunization varied widely across the sites. As expected, we found the lowest costs at the site that did not require any manual data entry by clinic personnel. Automated data entry remained the least costly interface, even assuming that the proportion of records entered was substantially less than 100%. A combination of personnel time and fixed equipment expenditures drove participation costs at clinics with manual data entry. Although Site D had the lowest labor costs, its smaller number of patients (and shots) resulted in higher equipment costs per patient (and per shot entered). Costs were lower in Site B and Site C when a dedicated employee (Site BC, 0.8 full-time equivalent [FTE]) rather than usual clinic staff (Site B, cumulative 0.42 FTE; and site C, cumulative 0.48 FTE) was used to enter immunization records into the registry.
Data Entry To evaluate whether the registry interface affected participation costs, data-entry times for Clinics B and C (Registry 1) were compared to data-entry times for Clinic D (Registry 2). In all three clinics, usual clinic staff performed immunization record entry into the registry. As seen in Table 2, the time required to enter or to edit a record in the registry was extremely variable. At Sites B and C, 16 different staff members were observed interfacing with the registry, whereas at Site D five staff members, who accessed the registry for every patient seen during the clinic session, entered the immunizations. Clinic personnel at Sites B and C spent Am J Prev Med 2000;19(2)
101
Table 2. Comparison of data entry observation time Data entry times (in seconds)
Mean
Table 3. Estimated yearly data entry cost (1998) dollars)
Range
SD
Immunization record Sites B and C 147.92 Site D 57.00
20–420 30–90
113.71 22.25
3.60
.001
Demographic record Sites B and C 306.43 Site D 87.50
30–600 45–150
159.69 40.71
3.27*
.004
t
p
*Unequal variance t-test SD, standard deviation
on average 2.47 minutes entering new shots and 5.10 minutes entering a demographic record, with some patient entries requiring more than 10 minutes. The immunization data entered included the date and type of immunization. The required demographic information included the patient name, address, telephone number, medical record number, date of birth, race, gender, parents’ names, parents’ address, and parents’ telephone number. Personnel at Site D spent approximately 57 seconds entering new shots and 1.47 minutes entering a demographic record. The immunization data entered included immunization dose, date of immunization, type of immunization, manufacturer, lot number, site of immunization, route of immunization, expiration date, and the provider who administered the immunization. Despite entering substantially more information, clinic personnel spent less time on data entry, and the difference was statistically significant. Because most patient records were already entered in the registry, the entry of a demographic record at Site D typically consisted of entering only a change in the last point of contact, the last provider to see the patient, or the date the next immunization was due. The difference between mean demographic data-entry times was also statistically significant.
Data Entry Cost Estimates We used the data-entry time estimates to estimate the annual cost of data entry at the three sites that used general clinical staff to manually enter records into the registry (Table 3). The annual data-entry cost per patient at Sites B and C was higher than the annual data-entry cost per patient at Site D because of the longer time required for data entry and retrieval. These clinics’ ability to spread fixed equipment costs over a larger number of patients tempered the higher labor costs.
Discussion This study was a unique attempt to determine the direct costs incurred by providers who participate in an immunization registry. In addition, this study points out 102
Type of cost
Number of patients
Sites B and C Labor Equipment Total
6600
Site D Labor Equipment Total
1400
Annual cost
Annual cost per patient
$7951 $2030 $9981
$1.21 $0.31 $1.51
$ 688 $1015 $1703
$0.49 $0.72 $1.22
some of the operational challenges that need to be solved for immunization registries to achieve their potential. Annual participation costs per patient and per immunization were the least expensive for the public health department that used an automated data-entry interface. We noted similar findings in an evaluation of All Kids Count registries.15 Although less costly, the automated interface had disadvantages at the clinic level because personnel in the five clinics were unable to look up recently updated registry data without contacting the central health department administrative staff. One cause of cost variation in sites with manual data entry is the amount of clinic personnel time required to enter or to edit records in the registry. Ease of the registry interface, the amount and quality of training that staff receive, and registry software that minimizes keystrokes are likely to significantly lower participation costs. Clinic patient volume also affects cost. Higher patient volumes in larger clinics decrease the cost per patient for fixed expenses, such as a dedicated registry computer. Smaller clinics may find it more difficult to finance registry participation. We found it difficult to determine whether the degree of registry integration into the patient flow increased or decreased costs. The site with the lowest labor requirement for data entry also had the most complete integration of registry activities into patient flow. The ease of the interface may have promoted regular use by clinic staff. However, in another clinic, the per-patient costs of registry participation were lower when a dedicated employee performed data entry than when clinic staff performed data entry, in part because the staff was less familiar with the computer program. One very important disadvantage of batch data entry was that records were rarely accessed during the clinical encounter. If registries are not used at every encounter, the opportunity to reduce “missed opportunities” for immunization cannot be realized. Finally, data-entry labor costs will be higher for clinics participating in a registry with incomplete capture of the target population. As most of the target population is included in one of the study registries at birth, demographic data-entry transactions by participating clinics consist of updates with minimal data-
American Journal of Preventive Medicine, Volume 19, Number 2
entry requirements. The disappointing accuracy of registry-immunization records from two of the study clinics confirms previous findings and highlights the importance of ongoing quality assurance efforts by registries.16,17 Several limitations to our study exist that may affect generalizability. Because registry development is idiosyncratic and our sample of clinics participating in two registries is small, it is difficult to generalize to different registries. These are preliminary findings that need to be confirmed in other settings. All of the study clinics serve urban, predominantly disadvantaged populations. Turnover in clinic personnel makes ease of registry interface and ongoing training efforts particularly important. To avoid seasonal visit shifts, project staff observed clinics on multiple occasions during different times of the year. We also revisited clinics after major personnel changes. In addition, we reviewed the Vaccines for Children logs for non-observation periods to verify that the number of patients immunized and the number of immunizations administered were similar to periods of observation. Because the study used a cross-sectional design, we could not measure whether, for example, the ability to produce shot records has offset some of the labor costs of participation. However, none of the study clinics has eliminated existing immunization paperwork; therefore, registry functions are performed in addition to usual paper documentation. This study is an important first step, and more work is necessary to develop reliable estimates of provider costs and savings attributable to registry participation. Costs are and will be an important determinant of community providers’ willingness to participate in registries. Provider costs can be decreased by designing the registry to accept electronic transfers of immunization records, designing user-friendly interfaces, investing in initial and ongoing training of provider-site personnel, and avoiding duplicative data-entry tasks. Greater automation of data transfer would be expected to reduce both participation costs and retraining costs associated with staff turnover. Immunization registries offer opportunities to improve health as well as the delivery of appropriate health care services. To realize this opportunity, however, registries need to pursue implementation strategies that ensure that the benefits of participation exceed the costs. This research was supported by Grant Number U1W/ CCU414706-01 from the National Immunization Program of
the Centers for Disease Control and Prevention (CDC). The contents are solely the responsibility of the authors and do not necessarily represent the official views of the CDC. Approval for this study was obtained from the Emory University Human Investigations Committee. The authors wish to thank Gene Downing, Marianne Pappas, and David Shields for their invaluable assistance in the implementation of this study.
References 1. U.S. Department of Health and Human Services. Healthy People 2000: national health promotion and disease prevention objectives. Pub. No. PHS91–50212. Washington, DC: U.S. Government Printing Office, 1991. 2. Gostin LO, Lazzarini Z. Childhood immunization registries: a national review of public health information systems and the protection of privacy. JAMA 1995;274:1793–9. 3. Codero JF, Orenstein WA. The future of immunization registries. Am J Prev Med 1997;13(suppl 1):122– 4. 4. Sinn JS, Kronenburg MA, Morrow AL. The purpose and functions of immunization information systems within health care organizations. Arch Pediatr Adolesc Med 1997;151:615–20. 5. Linkins RW, Feikema SM. Immunization registries: the cornerstone of childhood immunization in the 21st century. Pediatr Ann 1998;27:349 –54. 6. Yawn BP, Edmonson L, Huber L, Poland GA, Jacobson RM, Jacobsen SJ. The impact of a simulated immunization registry on perceived childhood immunization status. Am J Managed Care 1998;4:185–92. 7. National Committee for Quality Assurance. HEDIS 3.0: Updated specifications for HEDIS 3.0. Washington, DC: National Committee for Quality Assurance, 1997. 8. National Vaccine Advisory Committee. Development of community- and state-based immunization registries: findings and recommendations for action. 1999. Available at: www.cdc.gov/nip/registry/i_recs.htm. 9. Wood D, Saarlas KN, Inkelas M, Matyas BT. Immunization registries in the United States: implications for the practice of public health in a changing health care system. Annu Rev Public Health 1999;20:231–55. 10. Bordley WC, Dempsey-Tanner T, Freed GL, Lister ME. Challenges to private provider participation in immunization registries. Am J Prev Med 1997;13(suppl 1):66 –70. 11. Pappas M, Baker BJ, Appelbaum D, Hinds W, Alexander ER, Korolak K. Immunization registries: provider recruitment and participation. Am J Prev Med 1997;13(suppl 1):71– 6. 12. SPSS Inc. SPSS for Windows. Release 7.5.1. Chicago: SPSS Inc., 1996. 13. Georgia Department of Labor. Georgia Wage Survey, 1996. Atlanta, GA: Georgia Department of Labor, 1997. 14. Haddix AC, Teutsch SM, Shaffer PA, Dun ˜ et DO. Prevention effectiveness: a guide to decision analysis and economic evaluation, 1st ed. New York: Oxford University Press, 1996. 15. Slifkin RT, Freeman VA, Biddle A, et al. The cost of immunization registries: four case studies. A report prepared for the Robert Wood Johnson Foundation. Chapel Hill, NC: Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, 1998. 16. Wilton R, Pennisi AJ. Evaluating the accuracy of transcribed computerstored immunization data. Pediatrics 1994;94:902– 6. 17. Payne T, Kanvik S, Seward R, et al. Development and validation of an immunization tracking system in a large health maintenance organization. Am J Prev Med 1993;9:96 –100.
Am J Prev Med 2000;19(2)
103