- Email: [email protected]
Impact of an Internet-Based Emergency Department Appointment System to Access Primary Care at Safety Net Community Clinics
HEALTH POLICY AND CLINICAL PRACTICE/BRIEF RESEARCH REPORT
Impact of an Internet-Based Emergency Department Appointment System to Access Primary Care at Safety Net Community Clinics Theodore C. Chan, MD James P. Killeen, MD Edward M. Castillo, PhD, MPH Gary M. Vilke, MD David A. Guss, MD Roberta Feinberg, MS Lawrence Friedman, MD
From the Department of Emergency Medicine (Chan, Killeen, Castillo, Vilke, Guss) and Department of Pediatrics (Friedman), University of California San Diego Medical Center and School of Medicine, San Diego, CA; and San Diego Family Care, San Diego, CA (Feinberg).
Study objective: We evaluate the effect of an Internet-based, electronic referral system (termed IMPACT-ED for Improving Medical home and Primary care Access to the Community clinics Through the ED) on access and follow-up at primary care community clinics for safety net emergency department (ED) patients. Methods: We conducted a nonblinded interventional trial at an urban, safety net, hospital ED with a census of 39,000 annually. IMPACT-ED identified patients who had no source of regular care and lived in a 15-ZIP-code low-income area served by 3 community clinics. Emergency physicians received an automated notification through the electronic medical record to access an imbedded software program for scheduling follow-up clinic appointments. Patients who would benefit from a follow-up clinic visit within 2 weeks as determined by the emergency physician received a computer-generated appointment time and clinic map with bus routes as part of their discharge instructions, and the clinics received an electronic notification of the appointment. We compared frequency of follow-up for a 6-month period before implementation when patients received written instructions to call the clinic on their own (pre-IMPACT) and 6 months after implementation (post-IMPACT). Statistical analysis was conducted with 2 testing, and corresponding 95% confidence intervals are presented. Results: There were 326 patients who received an appointment (post-IMPACT), of whom 81 followed up at the clinic as directed (24.8%), compared with 399 patients who received a referral (pre-IMPACT), of whom 4 followed up as directed (1.0%), for an absolute improvement of 23.8% (95% confidence interval 19.1% to 28.6%). Conclusion: Although most patients still failed to follow up at the community clinics as directed, the use of an Internet-based scheduling program linking a safety net ED with local community clinics significantly improved the frequency of follow-up for patients without primary care. [Ann Emerg Med. 2009;54:279-284.] 0196-0644/$-see front matter Copyright © 2008 by the American College of Emergency Physicians. doi:10.1016/j.annemergmed.2008.10.030
INTRODUCTION Background In the past 2 decades, emergency department (ED) use in the United States has increased dramatically to more than 110 million visits annually.1 More than 80% of these patients will be discharged from the ED with a recommendation for follow-up care, and more than half of these with specific instructions to follow up at an outpatient clinic or with an ambulatory physician.2 However, access to care remains a pervasive problem in the US health care system, and obtaining follow-up care after an ED visit can be problematic for patients regardless of insurance status.2,3 Volume , . : August
Importance Access is an even greater challenge for safety net hospital EDs providing care to the underserved. Low-income, uninsured, and underinsured patients often do not have an established relationship with a primary care physician and lack access to a primary care clinic.4 To address this need, we developed an Internet-based secure referral system (termed IMPACT-ED for Improving Medical home and Primary care Access to the Community clinics Through the ED) between our safety net hospital ED and local area community clinics to improve primary care access for underserved patients. The system accesses the clinic walk-in Annals of Emergency Medicine 279
Impact of an Internet-Based Emergency Department Appointment System
Editor’s Capsule Summary
What is already known on this topic For patients without a regular care provider, followup general outpatient care after an emergency department (ED) visit often fails for many reasons. What question this study addressed Can an electronic real-time referral system that gives clear instructions and appointment specifics improve post-ED discharge follow-up rates for those without antecedent ongoing care? What this study adds to our knowledge At this single urban ED, introducing the electronic system improved follow-up rates, yet most patients (75%) still did not adhere to the suggested plan. How this might change clinical practice Tailored ED-to-community clinic linkage may help improve compliance with follow-up care but leaves much room for improvement.
appointment availabilities at the ED encounter and allows ED providers to give designated patients follow-up appointment times at the clinics. Goals of This Investigation We sought to examine the effect of the IMPACT-ED referral appointment scheduling system on patient access and adherence with follow-up care instructions after an ED visit at a safety net hospital.
MATERIALS AND METHODS Study Design We conducted a nonrandomized, nonblinded interventional trial to determine community clinic access and follow-up adherence for select patients after implementation of the Internet-based referral system. This project was approved and informed consent was waived by our University’s institutional review board. Setting and Selection of Participants IMPACT-ED was implemented in the ED of an academic, urban hospital (annual census 39,000), which is a major safety net provider and disproportionate share hospital providing nearly 45% of the unfunded and underfunded (Medicaid and county indigent funding) hospital-based indigent care in a city of 1.7 million people. Discharged patients who stated they had no primary care physician or clinic and resided in a 15-ZIPcode region served by 3 local area community clinics were eligible regardless of health care insurance coverage or primary language. If the ED provider determined the patient would 280 Annals of Emergency Medicine
Chan et al
benefit from follow-up at a local community clinic within 2 weeks of the ED visit, the patient was eligible for a referral appointment. Interventions IMPACT-ED is an Internet-based, Health Insurance Portability and Accountability Act-compliant, linkage and referral system between the ED and the 3 local area federally qualified health center community clinics. The referral system creates a secure Web interface between both the ED electronic medical record and community clinic appointment systems. ED providers received an automatic computer notification through the existing ED electronic medical record when a patient met criteria according to registration information (home address within the 15-ZIP-code area and stated they had no primary physician or clinic). Available clinic appointment times were imbedded within the electronic medical record and electronic ED discharge instructions program. Physicians were taught to access the clinic appointment system when they thought an eligible patient would benefit from follow-up at the clinic within 2 weeks of the index ED visit. Providers were advised to discuss the referral appointment with the patient to determine their availability before selecting a specific appointment time for the patient. The patient then received the appointment time, clinical contact information, and location (including map and bus routes) as part of their standard ED written aftercare instructions on discharge. The community clinics received an automated electronic notification of the selected appointment time through the system interface, as well as a secure e-mail with the patient’s demographic, registration, and contact information. Before IMPACT-ED, a discharged patient was provided community clinic contact information, including telephone number, at the discretion of the emergency physician according to whether the patient would benefit from follow-up at the community clinic within 2 weeks. This contact information was accessible through the ED’s standard computerized discharge instructions, for which the provider would select the appropriate neighborhood community clinic and the patient would receive instructions to call the clinic on the printed discharge instructions. Data Collection and Processing We collected data from discharged ED patients who were referred to the 3 local area community clinics by using the standard method for the 6-month period before the implementation of IMPACT-ED (from September 2006 through February 2007) and from patients who were referred and provided appointment times to the clinics for the 6-month period after the implementation of IMPACT-ED (after a 1month implementation washout period, April 2007 through October 2007). Data collected for each group included patient age, sex, race/ethnicity, marital status, health insurance coverage, and triage acuity during the ED visit (3-tiered score). Data were also collected on follow-up at the 3 community Volume , . : August
Chan et al
Impact of an Internet-Based Emergency Department Appointment System
Figure. Flow diagram of patients eligible for a referral (pre-period) or an appointment (post-period) from a 15-ZIP-code area.
clinics, as well as at repeated visits to the ED within the 2 weeks after the index ED visit. For both periods, patient data were collected through a computerized search and data extraction from the ED electronic medical record for all discharged patients who met eligibility requirements and received either standard referral information (pre-IMPACT) or an actual appointment for follow-up (postIMPACT) to one of the 3 community clinics according to physician discretion. Data on subsequent repeated visits to the ED were also collected through electronic query of the ED electronic medical record. Data on follow-up at the community clinics were obtained by a computerized search of the IMPACTED database for appointment “show” status (post-IMPACT) and through an electronic query of the practice management systems for all 3 clinics (pre-IMPACT). Primary Data Analysis Our primary outcome measure for this study was frequency of follow-up at the community clinics by patients discharged from the ED. The primary analysis was to compare clinic adherence frequency between those referred in the preIMPACT period with those provided appointment times in the post-IMPACT period. We also compared these 2 groups in terms of repeated ED visit frequency in the 2-week period after the index visit. Proportions and differences between patient characteristics are presented with corresponding 95% confidence intervals (CIs). Multivariate logistic regression was used to identify variables associated with the pre- or postIMPACT periods and to identify which factors were associated with adherence with follow-up at the community clinics during the post period. Data analysis was conducted with SPSS version 14.0 (SPSS, Inc., Chicago, IL).
RESULTS In the pre-IMPACT period, there were 2,753 discharged ED patients who lived in the 15-ZIP-code area and indicated they had no primary care clinic or provider. Of these, 399 (14.5%) (95% CI 13.2% to 15.8%) were provided a standard referral to the community clinic according to emergency physician Volume , . : August
discretion and determination that the patient would benefit from follow-up within 2 weeks. In the post-IMPACT period, there were 2,466 discharged ED patients who lived in the designated area, of whom 326 (13.2%) (95% CI 11.9% to 14.5%) were provided actual clinic follow-up appointment times through IMPACT-ED, again according to emergency physician discretion and determination that the patient would benefit from close follow-up at the clinics (Figure). There was a higher frequency of men in the pre-IMPACT compared with the post-IMPACT group (difference⫽8.2%; 95% CI 1.0% to 15.3%) (Table 1). On multivariate logistic regression analysis, there were no independent associations among patient characteristics (age, sex, race/ethnicity, marital status, ED visit acuity, and health coverage insurance status) and period. Adherence with follow-up at the clinics within 2 weeks of the ED visit was significantly higher in the post-IMPACT appointment group, with 81 of 326 following up at the clinic at the appointed time (24.8%) compared with the pre-IMPACT referral group, of which 4 of 399 patients followed up and were treated in the clinic within 2 weeks of the ED visit (1.0%). There was an absolute improvement in clinic follow-up with the appointment system of 23.8% (95% CI 19.1% to 28.6%). Those who kept their scheduled appointment were slightly older than those who did not, with a difference of 4.9 years of age (95% CI 1.7 to 8.2 years) (Table 2). In the logistic regression analysis, only age was mildly associated with appointment adherence among those given an appointment while controlling for the other patient characteristics (odds ratio⫽1.03; 95% CI 1.01 to 1.05). There were fewer patients in the post-IMPACT appointment group who had return visits to the ED within 2 weeks of the index visit compared with the pre-IMPACT referral group. Thirty-eight of 326 patients (11.7% [95% CI 8.2% to 15.2%]) were treated again in the ED within 2 weeks in the postIMPACT group compared with 59 of 399 patients (14.8% [95% CI 11.3% to 18.3%]), though this difference did not reach statistical significance. Annals of Emergency Medicine 281
Impact of an Internet-Based Emergency Department Appointment System
Chan et al
Table 1. Demographic data, triage acuity, and insurance coverage status between pre-IMPACT and post-IMPACT patients. Patient characteristic
Pre-IMPACT (nⴝ399)
Post-IMPACT (nⴝ326)
Age, y
Mean (SD) 39.4 (13.1)
Mean (SD) 39.2 (13.6)
(95% CI) ⫺0.2 (⫺2.2 to 1.8)
Number (%)
Number (%)
Difference, % (95% CI)
242 (60.7)
171 (52.5)
⫺8.2 (⫺15.4 to ⫺1.0)
45 (11.3) 304 (76.2) 50 (12.5)
42 (12.9) 236 (72.4) 48 (14.7)
1.6 (⫺3.2 to 6.4) ⫺3.8 (⫺10.2 to 2.6) 2.2 (⫺2.8 to 7.2)
24 (6.0) 225 (56.4) 150 (37.6)
10 (3.1) 176 (54.0) 140 (42.9)
⫺2.9 (⫺5.9 to 0.0) ⫺2.4 (⫺9.7 to 4.9) 5.3 (⫺1.8 to 12.5)
204 (51.1) 82 (20.6) 104 (26.1) 9 (2.3)
173 (53.1) 67 (20.6) 63 (19.3) 23 (7.1)
2.0 (⫺5.4 to 9.2) 0.0 (⫺5.9 to 5.9) ⫺6.8 (⫺12.8 to ⫺0.7) 4.8 (1.7 to 7.9)
Sex Male Marital status Married Single Other† Triage Emergent (1) Acute (2) Urgent (3) Insurance Self/none Private/commercial County indigent care Medical/Medicare
Difference, Means
Table 2. Demographic data, triage acuity, and insurance coverage status between patients who kept and did not keep appointment. Patient characteristic Age, y, mean (SD)
Sex Male Marital status Married Single Other Triage Emergency (1) Acute (2) Urgent (3) Insurance Self/none Private/commercial County indigent care Medical/Medicare
Kept Appointment (nⴝ81)
Did Not Keep Appointment (nⴝ245)
Difference, Means
Mean (SD) 42.9 (12.5)
Mean (SD) 38.0 (13.8)
(95% CI) ⫺4.9 (⫺8.2 to ⫺1.7)
Number (%)
Number (%)
Difference to % (95% CI)
40 (49.4)
130 (53.1)
3.7 (⫺8.9 to 16.2)
11 (13.6) 57 (70.4) 13 (16.0)
31 (12.7) 179 (73.1) 35 (14.3)
⫺0.9 (⫺9.5 to 7.6) 2.7 (⫺8.7 to 14.1) ⫺1.7 (⫺10.9 to 7.4)
2 (2.5) 43 (53.1) 36 (44.4)
8 (3.3) 133 (54.3) 104 (42.4)
0.8 (⫺3.3 to 4.8) 1.2 (⫺11.3 to 13.7) ⫺2.0 (⫺14.5 to 10.5)
44 (54.3) 16 (19.8) 16 (19.8) 5 (6.2)
129 (52.7) 51 (20.8) 47 (19.2) 18 (7.3)
⫺1.6 (⫺14.2 to 10.9) 1.0 (⫺9.0 to 11.1) ⫺0.6 (⫺10.5 to 9.4) 1.1 (⫺5.0 to 7.3)
LIMITATIONS Our study has a number of limitations. First, IMPACT-ED was conducted at a single safety net hospital ED with 3 community clinics as an initial pilot project, and our results may not be reproducible in other settings. Second, the periods studied were relatively short, and it is possible that the improvement observed in follow-up would have decreased during a longer period. In addition, the 2 periods studied involved different calendar months, which may have affected the groups in terms of the presenting conditions of the patients, likelihood they would seek follow-up care, and the potential for other changes in the ED that may have resulted in differences 282 Annals of Emergency Medicine
that would not have occurred had a concurrent control group been included. Third, this study was a nonblinded, nonrandomized intervention trial with historical controls and not a randomized controlled experimental trial. The intervention was implemented as a quality improvement and care coordination effort at our institution. Although the subject groups were similar on a number of demographic, health coverage, and triage acuity measures on multivariate analysis, and a similar proportion of patients were given pre-IMPACT referrals or post-IMPACT appointments, we cannot directly compare the 2 groups of patients. Furthermore, although the decision for Volume , . : August
Chan et al
Impact of an Internet-Based Emergency Department Appointment System
referral or appointment was determined by the physician, we did not collect specific information on the exact reason or medical indication for the short-term follow-up. Fourth, the study groups represented only a small fraction of patients who were treated in our ED. Although the criteria of home address and lack of primary care excluded some patients, the most important limiting factor was physician discretion. ED providers were advised to provide patients either referrals or appointments to those who they believed would benefit from follow-up at the community clinics within a 2-week period. Furthermore, although the proportion of patients referred or given appointments was similar in both the pre- and postIMPACT periods, it is possible that reliance on physician discretion introduced bias into our nonblinded study. Awareness of the IMPACT-ED project during the postIMPACT period may have resulted in physicians selecting only those patients with greater likelihood of following up at the clinic. In addition, although we did examine a number of factors that can affect health care behavior (such as insurance coverage, acuity, and patient demographics), we did not study many factors that have been identified as important predictors of adherence, including income, occupation, transportation availability, patient perspective, and underlying health status and comorbid conditions.5,6 Finally, although we evaluated the frequency of short-term return visits to our ED between the groups, we did not specifically investigate whether patients were treated subsequently in other health care settings or the overall effect on the health outcomes of our patients.
DISCUSSION For a variety of reasons, the uninsured and underinsured face greater challenges in accessing primary health care.4,7 Nationally, capacity and expansion of community health centers and clinics have been proposed as a way to meet the needs of this population. Linkages between community health clinics and local area EDs have been suggested as a means to improve communication and care coordination for the safety net population.4 We implemented an Internet-based referral and appointment system to the community clinics for patients treated in our ED according to home residence and lack of a regular source of primary care, as defined by the patient stating he or she had no primary care physician or clinic. Emergency physicians were able to schedule patients who they believed would benefit with follow-up in the clinics within 2 weeks of their ED visit. We found that patients who received an appointment, along with a computer-generated map and bus routes to the clinics at ED discharge, were significantly more likely to follow up as directed compared with those who only received instructions to contact the clinic after discharge. Previous studies have demonstrated the effect of providing actual appointment times on follow-up adherence frequency after ED discharge.7-10 In a study of 250 ED patients, Kyriacou et al8 reported that follow-up frequency was significantly higher (59% versus 37%) for ED patients randomized to receive Volume , . : August
outpatient appointment times with their primary care or referral physician before discharge. In another study involving 189 ED patients, O’Brien et al9 found that follow-up adherence improved from 14% to 30% for patients provided appointments during the ED visit. In many of these studies, however, access to the clinic schedules and the ability to provide appointment times for ED patients was limited to daytime hours8,9 or required significant staff time and even case managers to schedule the appointments.7,10 In contrast, the IMPACT-ED system was available 24 hours a day, 7 days a week for appointment scheduling and designed to automate the identification, referral, and appointment processes to reduce obstacles and burdens for both providers and patients in accessing primary care. Despite our efforts to use informatics tools to automate the referral and appointment system, providers did not refer or provide appointments to eligible patients at any higher rate than referrals had occurred before the IMPACT-ED program. Though specific concerns were not raised by staff, it is possible that the actual process was unduly burdensome on providers, which may have led to concerns about ED efficiency and patient flow. However, the similar rates in the pre and post periods indicate that the process did not create a great enough burden to actually reduce referrals to the clinics. In addition, although there was a lower rate of short-term repeated visits to the ED by patients in the post-IMPACT group, this difference was not statistically significant. In fact, previous research has reported mixed results about whether improved primary care availability for ED patients actually reduces subsequent ED visits.9-11 Our program involved linking a safety net hospital ED with 3 community clinics located in low-income neighborhoods with large indigent and underserved populations. Perhaps because of our target population, baseline frequency of clinic follow-up in the pre-IMPACT period (0.8%) was remarkably below that reported in other studies. Even with our appointment system in place, clinic follow-up, although much improved, remained poor. However, our local area community health center clinics observed that their regular clinic appointment “show rates” were low at baseline and that our results were in line with primary care clinic show frequency for other safety net and public hospitals.12 In addition, there were concerns raised that the low adherence might negatively affect the clinics in terms of lost appointment times and availabilities for other clinic patients. However, by utilizing “walk-in” clinic time slots for ED patient appointments, as opposed to standard clinic scheduled appointments, and releasing these times within 24 hours if not filled (or as soon as the patient failed to show at the appointment time) back to walk-in clinic availability, the clinics believed the negative effect was minimal. All 3 clinics have expressed a strong desire to continue participation in the program. We implemented an Internet-based referral and follow-up appointment system linking a safety hospital ED and 3 local Annals of Emergency Medicine 283
Impact of an Internet-Based Emergency Department Appointment System area community clinics located in an underserved community. Providing patients without a regular source of primary care residing in the select community a follow-up community clinic appointment before ED discharge significantly improved follow-up adherence for this population. However, the majority of patients still did not follow up at the community clinic despite this intervention. The authors gratefully acknowledge the support of the Emergency Department and Community Clinics staff in implementing the IMPACT-ED project. The authors also thank Scott Kennedy of San Diego Family Care for his assistance with data collection for this study. Supervising editor: Donald M. Yealy, MD Author contributions: TCC, JPK, and LF conceived of the study and obtained grant funding for the project. TCC, JPK, and GMV designed the study. JPK, EMC, and RF collected the data and interpreted the results. DAG provided oversight for the project and study. EMC provided statistical advice and analyzed the study. TCC and EMC drafted the article, and all authors contributed substantially to its revision. TCC takes responsibility for the paper as a whole. Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article, that might create any potential conflict of interest. See the Manuscript Submission Agreement in this issue for examples of specific conflicts covered by this statement. This study was funded by a grant from the Alliance Healthcare Foundation, San Diego, CA. Publication dates: Received for publication July 22, 2008. Revisions received September 26, 2008, and October 21, 2008. Accepted for publication October 31, 2008. Available online December 13, 2008. Presented as an abstract at the Society of Academic Medicine 2008 annual meeting, Washington, DC, May 2008. Reprints not available from the authors.
284 Annals of Emergency Medicine
Chan et al
Address for correspondence: Theodore C. Chan, MD, UCSD Emergency Department, 200 West Arbor Drive #8676, San Diego, CA 92130; 619-543-6463, fax 619-543-3115; E-mail [email protected].
REFERENCES 1. Nawar EW, Niska RW, Xu J. National Hospital Ambulatory Medical Care Survey: 2005 Emergency Department Summary. Advance Data From Vital and Health Statistics No. 386. Hyattsville, MD: National Center for Health Statistics; 2007. 2. Asplin BR, Rhodes KV, Levy H, et al. Insurance status and access to urgent ambulatory care follow-up appointments. JAMA. 2005; 294:1248-1254. 3. Handel DA, McConnnel KJ, Allen H, et al. Outpatient follow-up in today’s health care environment. Ann Emerg Med. 2007;49:288292. 4. Felland LE, Hurley RE, Kemper NM. Safety Net Hospital Emergency Departments: Creating Safety Valves for Non-urgent Care. Washington, DC: Center for Studying Health System Change; 2008. Issue Brief No. 120. 5. Aday LA, Andersen R. A framework for the study of access to medical care. Health Serv Res. 1974;9:208-220. 6. Cooper RJ, Schriger DL. Evidence-based emergency department discharge planning: how do we get there? Ann Emerg Med. 1999; 34:667-669. 7. McCarthy ML, Hirshon JM, Ruggles RL, et al. Referral of medically uninsured emergency department patients to primary care. Acad Emerg Med. 2002;9:639-642. 8. Kyriacou DN, Handel D, Stein AC, et al. Brief report: factors affecting outpatient follow-up compliance of emergency department patients. J Gen Intern Med. 2005;20:938-942. 9. O’Brien GM, Stein MD, Fagan MJ, et al. Enhanced emergency department referral improves primary care access. Am J Manag Care. 1999;5:1265-1269. 10. Horwitz SM, Busch SH, Balestracci KMB, et al. Intensive intervention improves primary care follow-up for uninsured emergency department patients. Acad Emerg Med. 2005;12:647652. 11. Murnik M, Randal F, Guevara M, et al. Web-based primary care referral program associated with reduced emergency department utilization. Fam Med. 2006;38:185-189. 12. Rask KJ, Williams MV, McNagny SE, et al. Ambulatory health care use by patients in a public hospital emergency department. J Gen Intern Med. 1998;13:614-620.
Volume , . : August
Impact of an Internet-Based Emergency Department Appointment System to Access Primary Care at Safety Net Community Clinics Theodore C. Chan, MD James P. Killeen, MD Edward M. Castillo, PhD, MPH Gary M. Vilke, MD David A. Guss, MD Roberta Feinberg, MS Lawrence Friedman, MD
From the Department of Emergency Medicine (Chan, Killeen, Castillo, Vilke, Guss) and Department of Pediatrics (Friedman), University of California San Diego Medical Center and School of Medicine, San Diego, CA; and San Diego Family Care, San Diego, CA (Feinberg).
Study objective: We evaluate the effect of an Internet-based, electronic referral system (termed IMPACT-ED for Improving Medical home and Primary care Access to the Community clinics Through the ED) on access and follow-up at primary care community clinics for safety net emergency department (ED) patients. Methods: We conducted a nonblinded interventional trial at an urban, safety net, hospital ED with a census of 39,000 annually. IMPACT-ED identified patients who had no source of regular care and lived in a 15-ZIP-code low-income area served by 3 community clinics. Emergency physicians received an automated notification through the electronic medical record to access an imbedded software program for scheduling follow-up clinic appointments. Patients who would benefit from a follow-up clinic visit within 2 weeks as determined by the emergency physician received a computer-generated appointment time and clinic map with bus routes as part of their discharge instructions, and the clinics received an electronic notification of the appointment. We compared frequency of follow-up for a 6-month period before implementation when patients received written instructions to call the clinic on their own (pre-IMPACT) and 6 months after implementation (post-IMPACT). Statistical analysis was conducted with 2 testing, and corresponding 95% confidence intervals are presented. Results: There were 326 patients who received an appointment (post-IMPACT), of whom 81 followed up at the clinic as directed (24.8%), compared with 399 patients who received a referral (pre-IMPACT), of whom 4 followed up as directed (1.0%), for an absolute improvement of 23.8% (95% confidence interval 19.1% to 28.6%). Conclusion: Although most patients still failed to follow up at the community clinics as directed, the use of an Internet-based scheduling program linking a safety net ED with local community clinics significantly improved the frequency of follow-up for patients without primary care. [Ann Emerg Med. 2009;54:279-284.] 0196-0644/$-see front matter Copyright © 2008 by the American College of Emergency Physicians. doi:10.1016/j.annemergmed.2008.10.030
INTRODUCTION Background In the past 2 decades, emergency department (ED) use in the United States has increased dramatically to more than 110 million visits annually.1 More than 80% of these patients will be discharged from the ED with a recommendation for follow-up care, and more than half of these with specific instructions to follow up at an outpatient clinic or with an ambulatory physician.2 However, access to care remains a pervasive problem in the US health care system, and obtaining follow-up care after an ED visit can be problematic for patients regardless of insurance status.2,3 Volume , . : August
Importance Access is an even greater challenge for safety net hospital EDs providing care to the underserved. Low-income, uninsured, and underinsured patients often do not have an established relationship with a primary care physician and lack access to a primary care clinic.4 To address this need, we developed an Internet-based secure referral system (termed IMPACT-ED for Improving Medical home and Primary care Access to the Community clinics Through the ED) between our safety net hospital ED and local area community clinics to improve primary care access for underserved patients. The system accesses the clinic walk-in Annals of Emergency Medicine 279
Impact of an Internet-Based Emergency Department Appointment System
Editor’s Capsule Summary
What is already known on this topic For patients without a regular care provider, followup general outpatient care after an emergency department (ED) visit often fails for many reasons. What question this study addressed Can an electronic real-time referral system that gives clear instructions and appointment specifics improve post-ED discharge follow-up rates for those without antecedent ongoing care? What this study adds to our knowledge At this single urban ED, introducing the electronic system improved follow-up rates, yet most patients (75%) still did not adhere to the suggested plan. How this might change clinical practice Tailored ED-to-community clinic linkage may help improve compliance with follow-up care but leaves much room for improvement.
appointment availabilities at the ED encounter and allows ED providers to give designated patients follow-up appointment times at the clinics. Goals of This Investigation We sought to examine the effect of the IMPACT-ED referral appointment scheduling system on patient access and adherence with follow-up care instructions after an ED visit at a safety net hospital.
MATERIALS AND METHODS Study Design We conducted a nonrandomized, nonblinded interventional trial to determine community clinic access and follow-up adherence for select patients after implementation of the Internet-based referral system. This project was approved and informed consent was waived by our University’s institutional review board. Setting and Selection of Participants IMPACT-ED was implemented in the ED of an academic, urban hospital (annual census 39,000), which is a major safety net provider and disproportionate share hospital providing nearly 45% of the unfunded and underfunded (Medicaid and county indigent funding) hospital-based indigent care in a city of 1.7 million people. Discharged patients who stated they had no primary care physician or clinic and resided in a 15-ZIPcode region served by 3 local area community clinics were eligible regardless of health care insurance coverage or primary language. If the ED provider determined the patient would 280 Annals of Emergency Medicine
Chan et al
benefit from follow-up at a local community clinic within 2 weeks of the ED visit, the patient was eligible for a referral appointment. Interventions IMPACT-ED is an Internet-based, Health Insurance Portability and Accountability Act-compliant, linkage and referral system between the ED and the 3 local area federally qualified health center community clinics. The referral system creates a secure Web interface between both the ED electronic medical record and community clinic appointment systems. ED providers received an automatic computer notification through the existing ED electronic medical record when a patient met criteria according to registration information (home address within the 15-ZIP-code area and stated they had no primary physician or clinic). Available clinic appointment times were imbedded within the electronic medical record and electronic ED discharge instructions program. Physicians were taught to access the clinic appointment system when they thought an eligible patient would benefit from follow-up at the clinic within 2 weeks of the index ED visit. Providers were advised to discuss the referral appointment with the patient to determine their availability before selecting a specific appointment time for the patient. The patient then received the appointment time, clinical contact information, and location (including map and bus routes) as part of their standard ED written aftercare instructions on discharge. The community clinics received an automated electronic notification of the selected appointment time through the system interface, as well as a secure e-mail with the patient’s demographic, registration, and contact information. Before IMPACT-ED, a discharged patient was provided community clinic contact information, including telephone number, at the discretion of the emergency physician according to whether the patient would benefit from follow-up at the community clinic within 2 weeks. This contact information was accessible through the ED’s standard computerized discharge instructions, for which the provider would select the appropriate neighborhood community clinic and the patient would receive instructions to call the clinic on the printed discharge instructions. Data Collection and Processing We collected data from discharged ED patients who were referred to the 3 local area community clinics by using the standard method for the 6-month period before the implementation of IMPACT-ED (from September 2006 through February 2007) and from patients who were referred and provided appointment times to the clinics for the 6-month period after the implementation of IMPACT-ED (after a 1month implementation washout period, April 2007 through October 2007). Data collected for each group included patient age, sex, race/ethnicity, marital status, health insurance coverage, and triage acuity during the ED visit (3-tiered score). Data were also collected on follow-up at the 3 community Volume , . : August
Chan et al
Impact of an Internet-Based Emergency Department Appointment System
Figure. Flow diagram of patients eligible for a referral (pre-period) or an appointment (post-period) from a 15-ZIP-code area.
clinics, as well as at repeated visits to the ED within the 2 weeks after the index ED visit. For both periods, patient data were collected through a computerized search and data extraction from the ED electronic medical record for all discharged patients who met eligibility requirements and received either standard referral information (pre-IMPACT) or an actual appointment for follow-up (postIMPACT) to one of the 3 community clinics according to physician discretion. Data on subsequent repeated visits to the ED were also collected through electronic query of the ED electronic medical record. Data on follow-up at the community clinics were obtained by a computerized search of the IMPACTED database for appointment “show” status (post-IMPACT) and through an electronic query of the practice management systems for all 3 clinics (pre-IMPACT). Primary Data Analysis Our primary outcome measure for this study was frequency of follow-up at the community clinics by patients discharged from the ED. The primary analysis was to compare clinic adherence frequency between those referred in the preIMPACT period with those provided appointment times in the post-IMPACT period. We also compared these 2 groups in terms of repeated ED visit frequency in the 2-week period after the index visit. Proportions and differences between patient characteristics are presented with corresponding 95% confidence intervals (CIs). Multivariate logistic regression was used to identify variables associated with the pre- or postIMPACT periods and to identify which factors were associated with adherence with follow-up at the community clinics during the post period. Data analysis was conducted with SPSS version 14.0 (SPSS, Inc., Chicago, IL).
RESULTS In the pre-IMPACT period, there were 2,753 discharged ED patients who lived in the 15-ZIP-code area and indicated they had no primary care clinic or provider. Of these, 399 (14.5%) (95% CI 13.2% to 15.8%) were provided a standard referral to the community clinic according to emergency physician Volume , . : August
discretion and determination that the patient would benefit from follow-up within 2 weeks. In the post-IMPACT period, there were 2,466 discharged ED patients who lived in the designated area, of whom 326 (13.2%) (95% CI 11.9% to 14.5%) were provided actual clinic follow-up appointment times through IMPACT-ED, again according to emergency physician discretion and determination that the patient would benefit from close follow-up at the clinics (Figure). There was a higher frequency of men in the pre-IMPACT compared with the post-IMPACT group (difference⫽8.2%; 95% CI 1.0% to 15.3%) (Table 1). On multivariate logistic regression analysis, there were no independent associations among patient characteristics (age, sex, race/ethnicity, marital status, ED visit acuity, and health coverage insurance status) and period. Adherence with follow-up at the clinics within 2 weeks of the ED visit was significantly higher in the post-IMPACT appointment group, with 81 of 326 following up at the clinic at the appointed time (24.8%) compared with the pre-IMPACT referral group, of which 4 of 399 patients followed up and were treated in the clinic within 2 weeks of the ED visit (1.0%). There was an absolute improvement in clinic follow-up with the appointment system of 23.8% (95% CI 19.1% to 28.6%). Those who kept their scheduled appointment were slightly older than those who did not, with a difference of 4.9 years of age (95% CI 1.7 to 8.2 years) (Table 2). In the logistic regression analysis, only age was mildly associated with appointment adherence among those given an appointment while controlling for the other patient characteristics (odds ratio⫽1.03; 95% CI 1.01 to 1.05). There were fewer patients in the post-IMPACT appointment group who had return visits to the ED within 2 weeks of the index visit compared with the pre-IMPACT referral group. Thirty-eight of 326 patients (11.7% [95% CI 8.2% to 15.2%]) were treated again in the ED within 2 weeks in the postIMPACT group compared with 59 of 399 patients (14.8% [95% CI 11.3% to 18.3%]), though this difference did not reach statistical significance. Annals of Emergency Medicine 281
Impact of an Internet-Based Emergency Department Appointment System
Chan et al
Table 1. Demographic data, triage acuity, and insurance coverage status between pre-IMPACT and post-IMPACT patients. Patient characteristic
Pre-IMPACT (nⴝ399)
Post-IMPACT (nⴝ326)
Age, y
Mean (SD) 39.4 (13.1)
Mean (SD) 39.2 (13.6)
(95% CI) ⫺0.2 (⫺2.2 to 1.8)
Number (%)
Number (%)
Difference, % (95% CI)
242 (60.7)
171 (52.5)
⫺8.2 (⫺15.4 to ⫺1.0)
45 (11.3) 304 (76.2) 50 (12.5)
42 (12.9) 236 (72.4) 48 (14.7)
1.6 (⫺3.2 to 6.4) ⫺3.8 (⫺10.2 to 2.6) 2.2 (⫺2.8 to 7.2)
24 (6.0) 225 (56.4) 150 (37.6)
10 (3.1) 176 (54.0) 140 (42.9)
⫺2.9 (⫺5.9 to 0.0) ⫺2.4 (⫺9.7 to 4.9) 5.3 (⫺1.8 to 12.5)
204 (51.1) 82 (20.6) 104 (26.1) 9 (2.3)
173 (53.1) 67 (20.6) 63 (19.3) 23 (7.1)
2.0 (⫺5.4 to 9.2) 0.0 (⫺5.9 to 5.9) ⫺6.8 (⫺12.8 to ⫺0.7) 4.8 (1.7 to 7.9)
Sex Male Marital status Married Single Other† Triage Emergent (1) Acute (2) Urgent (3) Insurance Self/none Private/commercial County indigent care Medical/Medicare
Difference, Means
Table 2. Demographic data, triage acuity, and insurance coverage status between patients who kept and did not keep appointment. Patient characteristic Age, y, mean (SD)
Sex Male Marital status Married Single Other Triage Emergency (1) Acute (2) Urgent (3) Insurance Self/none Private/commercial County indigent care Medical/Medicare
Kept Appointment (nⴝ81)
Did Not Keep Appointment (nⴝ245)
Difference, Means
Mean (SD) 42.9 (12.5)
Mean (SD) 38.0 (13.8)
(95% CI) ⫺4.9 (⫺8.2 to ⫺1.7)
Number (%)
Number (%)
Difference to % (95% CI)
40 (49.4)
130 (53.1)
3.7 (⫺8.9 to 16.2)
11 (13.6) 57 (70.4) 13 (16.0)
31 (12.7) 179 (73.1) 35 (14.3)
⫺0.9 (⫺9.5 to 7.6) 2.7 (⫺8.7 to 14.1) ⫺1.7 (⫺10.9 to 7.4)
2 (2.5) 43 (53.1) 36 (44.4)
8 (3.3) 133 (54.3) 104 (42.4)
0.8 (⫺3.3 to 4.8) 1.2 (⫺11.3 to 13.7) ⫺2.0 (⫺14.5 to 10.5)
44 (54.3) 16 (19.8) 16 (19.8) 5 (6.2)
129 (52.7) 51 (20.8) 47 (19.2) 18 (7.3)
⫺1.6 (⫺14.2 to 10.9) 1.0 (⫺9.0 to 11.1) ⫺0.6 (⫺10.5 to 9.4) 1.1 (⫺5.0 to 7.3)
LIMITATIONS Our study has a number of limitations. First, IMPACT-ED was conducted at a single safety net hospital ED with 3 community clinics as an initial pilot project, and our results may not be reproducible in other settings. Second, the periods studied were relatively short, and it is possible that the improvement observed in follow-up would have decreased during a longer period. In addition, the 2 periods studied involved different calendar months, which may have affected the groups in terms of the presenting conditions of the patients, likelihood they would seek follow-up care, and the potential for other changes in the ED that may have resulted in differences 282 Annals of Emergency Medicine
that would not have occurred had a concurrent control group been included. Third, this study was a nonblinded, nonrandomized intervention trial with historical controls and not a randomized controlled experimental trial. The intervention was implemented as a quality improvement and care coordination effort at our institution. Although the subject groups were similar on a number of demographic, health coverage, and triage acuity measures on multivariate analysis, and a similar proportion of patients were given pre-IMPACT referrals or post-IMPACT appointments, we cannot directly compare the 2 groups of patients. Furthermore, although the decision for Volume , . : August
Chan et al
Impact of an Internet-Based Emergency Department Appointment System
referral or appointment was determined by the physician, we did not collect specific information on the exact reason or medical indication for the short-term follow-up. Fourth, the study groups represented only a small fraction of patients who were treated in our ED. Although the criteria of home address and lack of primary care excluded some patients, the most important limiting factor was physician discretion. ED providers were advised to provide patients either referrals or appointments to those who they believed would benefit from follow-up at the community clinics within a 2-week period. Furthermore, although the proportion of patients referred or given appointments was similar in both the pre- and postIMPACT periods, it is possible that reliance on physician discretion introduced bias into our nonblinded study. Awareness of the IMPACT-ED project during the postIMPACT period may have resulted in physicians selecting only those patients with greater likelihood of following up at the clinic. In addition, although we did examine a number of factors that can affect health care behavior (such as insurance coverage, acuity, and patient demographics), we did not study many factors that have been identified as important predictors of adherence, including income, occupation, transportation availability, patient perspective, and underlying health status and comorbid conditions.5,6 Finally, although we evaluated the frequency of short-term return visits to our ED between the groups, we did not specifically investigate whether patients were treated subsequently in other health care settings or the overall effect on the health outcomes of our patients.
DISCUSSION For a variety of reasons, the uninsured and underinsured face greater challenges in accessing primary health care.4,7 Nationally, capacity and expansion of community health centers and clinics have been proposed as a way to meet the needs of this population. Linkages between community health clinics and local area EDs have been suggested as a means to improve communication and care coordination for the safety net population.4 We implemented an Internet-based referral and appointment system to the community clinics for patients treated in our ED according to home residence and lack of a regular source of primary care, as defined by the patient stating he or she had no primary care physician or clinic. Emergency physicians were able to schedule patients who they believed would benefit with follow-up in the clinics within 2 weeks of their ED visit. We found that patients who received an appointment, along with a computer-generated map and bus routes to the clinics at ED discharge, were significantly more likely to follow up as directed compared with those who only received instructions to contact the clinic after discharge. Previous studies have demonstrated the effect of providing actual appointment times on follow-up adherence frequency after ED discharge.7-10 In a study of 250 ED patients, Kyriacou et al8 reported that follow-up frequency was significantly higher (59% versus 37%) for ED patients randomized to receive Volume , . : August
outpatient appointment times with their primary care or referral physician before discharge. In another study involving 189 ED patients, O’Brien et al9 found that follow-up adherence improved from 14% to 30% for patients provided appointments during the ED visit. In many of these studies, however, access to the clinic schedules and the ability to provide appointment times for ED patients was limited to daytime hours8,9 or required significant staff time and even case managers to schedule the appointments.7,10 In contrast, the IMPACT-ED system was available 24 hours a day, 7 days a week for appointment scheduling and designed to automate the identification, referral, and appointment processes to reduce obstacles and burdens for both providers and patients in accessing primary care. Despite our efforts to use informatics tools to automate the referral and appointment system, providers did not refer or provide appointments to eligible patients at any higher rate than referrals had occurred before the IMPACT-ED program. Though specific concerns were not raised by staff, it is possible that the actual process was unduly burdensome on providers, which may have led to concerns about ED efficiency and patient flow. However, the similar rates in the pre and post periods indicate that the process did not create a great enough burden to actually reduce referrals to the clinics. In addition, although there was a lower rate of short-term repeated visits to the ED by patients in the post-IMPACT group, this difference was not statistically significant. In fact, previous research has reported mixed results about whether improved primary care availability for ED patients actually reduces subsequent ED visits.9-11 Our program involved linking a safety net hospital ED with 3 community clinics located in low-income neighborhoods with large indigent and underserved populations. Perhaps because of our target population, baseline frequency of clinic follow-up in the pre-IMPACT period (0.8%) was remarkably below that reported in other studies. Even with our appointment system in place, clinic follow-up, although much improved, remained poor. However, our local area community health center clinics observed that their regular clinic appointment “show rates” were low at baseline and that our results were in line with primary care clinic show frequency for other safety net and public hospitals.12 In addition, there were concerns raised that the low adherence might negatively affect the clinics in terms of lost appointment times and availabilities for other clinic patients. However, by utilizing “walk-in” clinic time slots for ED patient appointments, as opposed to standard clinic scheduled appointments, and releasing these times within 24 hours if not filled (or as soon as the patient failed to show at the appointment time) back to walk-in clinic availability, the clinics believed the negative effect was minimal. All 3 clinics have expressed a strong desire to continue participation in the program. We implemented an Internet-based referral and follow-up appointment system linking a safety hospital ED and 3 local Annals of Emergency Medicine 283
Impact of an Internet-Based Emergency Department Appointment System area community clinics located in an underserved community. Providing patients without a regular source of primary care residing in the select community a follow-up community clinic appointment before ED discharge significantly improved follow-up adherence for this population. However, the majority of patients still did not follow up at the community clinic despite this intervention. The authors gratefully acknowledge the support of the Emergency Department and Community Clinics staff in implementing the IMPACT-ED project. The authors also thank Scott Kennedy of San Diego Family Care for his assistance with data collection for this study. Supervising editor: Donald M. Yealy, MD Author contributions: TCC, JPK, and LF conceived of the study and obtained grant funding for the project. TCC, JPK, and GMV designed the study. JPK, EMC, and RF collected the data and interpreted the results. DAG provided oversight for the project and study. EMC provided statistical advice and analyzed the study. TCC and EMC drafted the article, and all authors contributed substantially to its revision. TCC takes responsibility for the paper as a whole. Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article, that might create any potential conflict of interest. See the Manuscript Submission Agreement in this issue for examples of specific conflicts covered by this statement. This study was funded by a grant from the Alliance Healthcare Foundation, San Diego, CA. Publication dates: Received for publication July 22, 2008. Revisions received September 26, 2008, and October 21, 2008. Accepted for publication October 31, 2008. Available online December 13, 2008. Presented as an abstract at the Society of Academic Medicine 2008 annual meeting, Washington, DC, May 2008. Reprints not available from the authors.
284 Annals of Emergency Medicine
Chan et al
Address for correspondence: Theodore C. Chan, MD, UCSD Emergency Department, 200 West Arbor Drive #8676, San Diego, CA 92130; 619-543-6463, fax 619-543-3115; E-mail [email protected].
REFERENCES 1. Nawar EW, Niska RW, Xu J. National Hospital Ambulatory Medical Care Survey: 2005 Emergency Department Summary. Advance Data From Vital and Health Statistics No. 386. Hyattsville, MD: National Center for Health Statistics; 2007. 2. Asplin BR, Rhodes KV, Levy H, et al. Insurance status and access to urgent ambulatory care follow-up appointments. JAMA. 2005; 294:1248-1254. 3. Handel DA, McConnnel KJ, Allen H, et al. Outpatient follow-up in today’s health care environment. Ann Emerg Med. 2007;49:288292. 4. Felland LE, Hurley RE, Kemper NM. Safety Net Hospital Emergency Departments: Creating Safety Valves for Non-urgent Care. Washington, DC: Center for Studying Health System Change; 2008. Issue Brief No. 120. 5. Aday LA, Andersen R. A framework for the study of access to medical care. Health Serv Res. 1974;9:208-220. 6. Cooper RJ, Schriger DL. Evidence-based emergency department discharge planning: how do we get there? Ann Emerg Med. 1999; 34:667-669. 7. McCarthy ML, Hirshon JM, Ruggles RL, et al. Referral of medically uninsured emergency department patients to primary care. Acad Emerg Med. 2002;9:639-642. 8. Kyriacou DN, Handel D, Stein AC, et al. Brief report: factors affecting outpatient follow-up compliance of emergency department patients. J Gen Intern Med. 2005;20:938-942. 9. O’Brien GM, Stein MD, Fagan MJ, et al. Enhanced emergency department referral improves primary care access. Am J Manag Care. 1999;5:1265-1269. 10. Horwitz SM, Busch SH, Balestracci KMB, et al. Intensive intervention improves primary care follow-up for uninsured emergency department patients. Acad Emerg Med. 2005;12:647652. 11. Murnik M, Randal F, Guevara M, et al. Web-based primary care referral program associated with reduced emergency department utilization. Fam Med. 2006;38:185-189. 12. Rask KJ, Williams MV, McNagny SE, et al. Ambulatory health care use by patients in a public hospital emergency department. J Gen Intern Med. 1998;13:614-620.
Volume , . : August