- Email: [email protected]
Computerized recruiting for clinical trials in real time
CONDUCTING CLINICAL RESEARCH/BRIEF RESEARCH REPORT
Computerized Recruiting for Clinical Trials in Real Time
Debra L. Weiner, MD, PhD Atul J. Butte, MD Patricia L. Hibberd, MD, PhD Gary R. Fleisher, MD From the Division of Emergency Medicine (Weiner, Fleisher), the Children’s Hospital Informatics Program, Division of Endocrinology (Butte), and the Clinical Research Program, Division of Infectious Disease (Hibberd), Children’s Hospital Boston, Boston, MA, and Harvard Medical School, Boston, MA (Weiner, Butte, Hibberd, Fleisher).
Study objective: Success of prospective studies, particularly in the emergency department, often depends on immediate identification of eligible patients to ensure timely sample collection and initiation of study interventions. We report use of a realtime automated notification system to identify potential patients for a clinical trial at the time of ED registration on the basis of information routinely collected. We hypothesize that the automated notification system improves the rate of investigator notification. Methods: We performed a prospective comparison of the notification rate by the automated notification system compared with that by ED clinicians. Results: In the 11 months before use of the automated notification system, the investigator was notified by ED staff for 56% of 61 potentially eligible patients. During 10 months of using the automated notification system, the investigator was paged by the automated notification system for 84% of 49 potentially eligible patients. Conclusion: The automated notification system improves study investigator notification. Use requires online linked registration, a database, and paging systems. The automated notification system is a potentially valuable tool in the recruitment of patients for clinical trials. [Ann Emerg Med. 2003;41:242-246.]
Copyright © 2003 by the American College of Emergency Physicians. 0196-0644/2003/$30.00 + 0 doi:10.1067/mem.2003.52
2 4 2
ANNALS OF EMERGENCY MEDICINE
41:2
FEBRUARY 2003
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
INTRODUCTION
The success of prospective clinical research studies, particularly in the emergency department, often depends on immediate identification of potentially eligible patients to ensure timely collection of critical samples, initiation of study interventions, or both. Large numbers of clinicians not directly involved in research must be aware of active studies, know their eligibility criteria, and take the time either to notify the study investigators or to obtain consent from and enroll patients themselves. Especially in a busy ED, consistent and timely study investigator notification often does not occur. A system that maximizes enrollment without disrupting clinical care in the ED is needed. Computerized, real-time event-monitoring systems that use alphanumeric pages to notify clinicians have been used in clinical care to assist with triage decisions; identify clinical practice guidelines for which patients might be eligible; provide alerts for physiologic data, laboratory results, and medication order errors; and offer diagnostic decision support and treatment recommendations.1-3 Computer software has also been developed to improve clinical trial processes, including recruitment, enrollment, randomization, data collection, and analysis.4-7 In general, these systems have been dependent on entry of patient clinical data generated by a clinician. Particularly in the ED, where patient care often precedes clinical data entry, a point-of-entry patient identification and recruitment system would likely be advantageous, particularly for interventional trials. A computer-to-pager automated notification system (RealTime Recruiting, Atul Butte, MD, Children’s Hospital Boston, Boston, MA) has previously been used by us in our ED to identify potential study patients and to notify a study investigator in real time on the basis of critical laboratory values.8,9 We hypothesized that this software system could be modified to identify potential study participants at the time of registration in the ED. The specific aim of this pilot study was to compare the rate of study investigator pages by the automated notification system with the rate from ED staff for potential study patients for a prospective clinical trial. The clinical trial for which the automated notification system was being used evaluated the use of inhaled nitric oxide in the
FEBRUARY 2003
41:2
ANNALS OF EMERGENCY MEDICINE
treatment of acute vaso-occlusive crisis in pediatric patients with sickle cell disease. M AT E R I A L S A N D M E T H O D S
The Children’s Hospital Boston ED is an urban, academic tertiary-care center. Thirty attending physicians supervise 15 fellows and approximately 200 rotating residents/students annually. Nursing care is provided by 35 permanent and 15 per diem nurses. Patients with possible sickle cell vaso-occlusive crisis are triaged by a nurse directly to a patient-care room, where they are registered and seen promptly by the emergency physician, who initiates diagnostic evaluation and treatment. There are approximately 75 ED visits per year by patients potentially eligible for the clinical trial. Potential eligibility criteria for the clinical trial included sickle cell disease, age 10 to 21 years, pain likely caused by vaso-occlusion, and temperature of less than 38.5°C (<101.3°F). The study investigator determined final eligibility on the basis of more detailed criteria. Study inhalation therapy had to be initiated within 90 minutes of the first dose of the opioid given in the ED. The clinical trial, including testing of the effectiveness of the automated notification system, was approved by the hospital’s institutional review board. Informed consent was obtained for participation in the clinical trial. Emergency physicians and nurses were informed about the study through seminars and handouts and were reminded on a routine basis with signs in the ED, verbal communications several times a week, and e-mail messages every few months. In addition, the charting area of the ED has a bulletin board for information about each active study in the ED that includes eligibility criteria, instructions, and beeper numbers to page the study investigator. Physicians and nurses were requested to page the study investigator 24 hours a day for all patients meeting potential eligibility criteria. Paging by ED staff was the only notification system for the first 11 months of the clinical trial. The automated notification system, once initiated, was monitored for rate of pages for 10 months. The automated notification system (Figure) identifies potentially eligible patients on the basis of information
2 4 3
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
routinely entered into the hospital’s computerized registration system (Allegra; IntraNexus, Virginia Beach, VA) and includes registration site (ED), patient name, medical record number, date of birth, temperature, and chief complaint. Registration information is automatically transferred to the hospital central data repository (Oracle, Redwood City, CA). The automated notification system software connects to and scans the Oracle database by using Java Database Connectivity every 2 minutes and tracks all patients for up to 8 hours after their ED registration. The automated notification system automatically pages the study investigator with patient name, medical record number, and date and time of registration when an ED patient meeting the specified criteria is registered. A log of patients detected and paged by the automated notification system is generated and available to the investigator on a passwordprotected Intranet Web site. In addition to the information provided by page, the Web site includes the date and time that the automated notification system initiated the page. For this clinical trial, the patients that the automated notification system was programmed to recognize were those in an existing Division of Hematology electronic database of patients with sickle cell disease modified to include only patients between the ages of 10 and 21 years. This database was used by the automated notification system to search the central data repository. Clinician-initiated notification was continued
even once the automated notification system began thus providing a dual-notification system. Notification rates were compared by using the χ2 statistic to test the hypothesis that the automated notification system would improve study investigator notification of potential study patients compared with notification by ED staff. This analysis was performed with StatXact 4.0.1 software (Cytel Software Corporation, Cambridge, MA). To evaluate the effectiveness of the automated notification system, only patients in the database submitted to the automated notification system were considered potentially eligible. The number of ED visits by potentially eligible patients was ascertained retrospectively by using multiple approaches. The ED log generated by the registration system of all patients seen in the ED was reviewed by a hematology nurse practitioner familiar with all patients with sickle cell disease. In addition, 2 separate chart queries of the ED electronic medical record system were performed: one by names of patients in the hematology database used by the automated notification system and the other by whole-chart search for the keywords “sickle” and “HbS.” Once the automated notification system was initiated, patient visits were also identified by means of review of the automated notification system–generated Web-based log. The goal was to increase notification by 20% or greater, which, for a 3-year study with a 30-month enrollment period, could translate into a 6-month time savings.
Figure.
Schematic of flow of information with the automated notification system. Registration information is automatically transferred to the hospital central data repository (Oracle). The automated notification system software scans the Oracle database and automatically pages the study investigator when a patient meeting criteria specified in the automated notification system database is registered. An Intranet log of pages initiated by the automated notification system is generated.
2 4 4
ED registration Registration site Date, time Name, MR#, DOB Temperature, chief complaint
Sickle cell patient database
“Allegra”
Name, MR# DOB (a. 10-21 y) HbSS, HbSC, HbSBthal
Central data repository “Oracle”
Real time recruiting “RTR”
“RTR” database and statistics
Beeper
Intranet
“Sickle” Name, MR#, age Registration date/time
Name, MR#, age Registration date/time Page date/time
ANNALS OF EMERGENCY MEDICINE
41:2
FEBRUARY 2003
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
R E S U LT S
In the 11 months before use of the automated notification system, the investigator was paged for 34 (56%) of 61 potentially eligible patients. In the 10 months with use of the automated notification system, the investigator was paged for 41 (84%) of 49 potentially eligible patients, a significant increase in notification rate (χ2=9.77; 95% confidence interval 12% to 44%). The study investigator was paged by the automated notification system within 15 minutes for 86% and within 30 minutes for 93% of these patients (median 9 minutes; range 5 minutes to 7.8 hours). The interval between scans was less than 5 minutes for 99.95% of scans. Six patients were missed by the automated notification system despite their inclusion in the automated notification system database: 2 because the automated notification system was offline, 3 because ED registration was offline, and 1 because the paging system was not operational. The automated notification system pages for an additional 2 patients were delayed beyond the time period that would allow study enrollment because of treatment already received in the ED (2.4 and 7.8 hours, respectively). The automated notification system was offline for more than 1 hour 18 times during the 10month study period (median 1.65 days; range 1.9 hours to 11.5 days), a total of 15% of the time or the equivalent of 54.8 days. The investigator was not notified by the automated notification system about an additional 5 patients not in the hematology database who had a total of 7 ED visits after initiation of the automated notification system. The investigator was paged by the automated notification system for 44 patients with sickle cell disease who were not eligible because they had a chief complaint other than vaso-occlusive crisis, including 16 with a temperature of greater than 38.5°C (101.3°F). DISCUSSION
To our knowledge, this is the first report to describe use of a computerized, real-time recruitment system designed to automatically identify and notify study investigators about potential patients for clinical trials on the basis of information obtained at ED patient registra-
FEBRUARY 2003
41:2
ANNALS OF EMERGENCY MEDICINE
tion. The automated notification system significantly improved investigator notification of potential study patients for an actual clinical trial of inhaled nitric oxide for treatment of acute vaso-occlusive crisis in children with sickle cell disease. We have identified and are addressing issues of patient identification and investigator notification related directly or indirectly to the automated notification system that will improve its efficiency. The median time to page of 9 minutes was greater than expected, given the every-2-minute scan time of the automated notification system. This might in part be explained by the fact that, after data collection was complete, it was recognized that ED registration and automated notification system clocks were asynchronous when spot checks revealed that the registration clock is consistently behind the automated notification system by up to 11 minutes. As a first attempt at using the automated notification system, we identified patients on the basis of only a database of appropriately aged patients with sickle cell disease. The automated notification system can be optimized by using multiple search strategies to increase the page rate for potentially eligible patients and decrease the page rate for patients who do not meet potential eligibility criteria. The automated notification system can be programmed to also recognize patients on the basis of a chief complaint that contains the words “sickle,” “Hb,” “pain,” and/or “crisis” entered at registration, to calculate age on the basis of date of birth, and to exclude those with temperatures of 38.5°C (101.3°F) or greater. Arden Syntax, the standard of the American National Standards Institute, would be useful for this approach.10 A limitation of the automated notification system is that although the sensitivity of patient identification is potentially very high, the specificity is limited to clinical or laboratory information that can be entered into the registration system and retrieved on the hospital system. This might result in more pages for patients who do not meet eligibility criteria on the basis of information not entered as part of registration than paging initiated by ED personnel with complete knowledge of criteria for study eligibility. The patient database used by the automated notification system could easily be modified on an ongoing basis to eliminate the names of
2 4 5
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
patients for whom permanent ineligibility is established on the basis of information derived by the investigator during automated notification system–initiated patient evaluation or after enrollment in the study to decrease the number of pages for ineligible patients. Use of the automated notification system will depend, in part, on the quantity of pages acceptable to the investigator, particularly for ineligible patients. We were willing to accept pages for ineligible patients because we were targeting a rare disease and rare event; rate of enrollment of potentially eligible patients was projected to be low; we required clinical determinations and biologic specimens be obtained before initiation of treatment; and treatment needed to be standardized and initiated rapidly. Another limitation of this automated notification system is that it does not function when ED registration, the central hospital computer system, the automated notification system, or the paging system is offline. This limits its use to research or noncritical clinical applications. Moving the automated notification system to a dedicated server has decreased its offline time. Warning systems now notify the investigator of computer outages. At present, patient confidentiality is at least a theoretic concern with the automated notification system. A paging system is not point-to-point secure, and pages could be intercepted. Although receiving information that includes patient identifiers is advantageous, if necessary, the investigator could get this information by calling the ED once paged or checking the passwordprotected Intranet automated notification system log. Alternatively, a security system that scrambles information could potentially be developed. In addition to identifying patients for clinical trials, the automated notification system can also be used to identify patients on the basis of registration information, laboratory values for other clinical studies evaluating diagnostic or treatment options, or both; to identify patients eligible for clinical practice guidelines; and to track ED visits for patients with particular diseases, chief complaints, or both. Therefore, use of the automated notification system is not limited to research but also offers the opportunity to enhance routine patient care.
2 4 6
Author contributions: AJB developed the automated notification system and wrote this application. DLW and AJB conceived of adapting the automated notification system for patient identification on the basis of registration information. DLW, AJB, and GRF conceived the study. DLW and AJB were responsible for data collection. DLW and PLH were responsible for study design and statistical analysis. DLW drafted the manuscript, and all authors contributed substantially to critical review and revision. DLW takes responsibility for the paper as a whole. Received for publication July 5, 2001. Revision received August 1, 2002. Accepted for publication September 3, 2002. Presented at the American Pediatric Societies, Baltimore, MD, May 2001. Dr. Weiner is supported by US Food and Drug Administration Orphan Drug grant RD-R-001686 and General Clinical Research Center, National Center for Research Resources, National Institutes of Health grant RR02172. Dr. Butte is supported by National Library of Medicine grant 5T15 LM07092-07. Nitric oxide for the clinical trial was donated by Pulmonox Medical Corporation. Dr. Butte is the developer of the RealTime Recruiting software and has filed for a patent. Address for reprints: Debra L. Weiner, MD, PhD, Division of Emergency Medicine, Children’s Hospital, 300 Longwood Avenue, Boston, MA 02115; 617-355-4144, fax 617-731-3279; E-mail [email protected]. We thank Susan Kurth, PNP, Division of Hematology for making data from her review of ED logs available to us.
REFERENCES 1. Kuperman GJ, Teich JM, Bates DW, et al. Detecting alerts, notifying the physician and offering action items: a comprehensive alerting system. Proc AMIA Symp. 1996:704-708. 2. Shabot MM, LoBue M, Chen J. Wireless clinical alerts for physiologic, laboratory and medication data. Proc AMIA Symp. 2000:789-793. 3. Aronsky D, Huang PJ. Automatic identification of patients eligible for a pneumonia guideline. Proc AMIA Symp. 2000:12-16. 4. Johnson N, Lilford RJ. Computerized automation of clinical trials. Ballieres Clin Obstet Gynecol. 1990;4:771-786. 5. Silva J, Wittes R. Role of clinical trials informatics in the NCI’s cancer informatics infrastructure. Proc AMIA Symp. 1999:950-953. 6. Ohno-Machado L, Wang SJ, Mar P, et al. Decision support for clinical trial eligibility determination in breast cancer. Proc AMIA Symp. 1999:340-344. 7. Papaconstantinou C, Theocharous G, Mahadevan S. An expert system for assigning patients into clinical trials based on Bayesian networks. J Med Syst. 1998;22:189-202. 8. Butte AJ, Weinstein DA, Kohane IS. Enrolling patients into clinical trials faster using RealTime Recruiting. Proc AMIA Symp. 2000:111-115. 9. Weinstein DA, Butte AJ, Raymond K, et al. High incidence of unrecognized metabolic and endocrine disorders in acutely ill children with previously unrecognized hypoglycemia [abstract]. Pediatr Res. 2001;49:103. 10. Hripcsak G, Ludemann P, Pryor TA, et al. Rationale for the Arden Syntax. Comput Biomed Res. 1994;27:291-324.
ANNALS OF EMERGENCY MEDICINE
41:2
FEBRUARY 2003
Computerized Recruiting for Clinical Trials in Real Time
Debra L. Weiner, MD, PhD Atul J. Butte, MD Patricia L. Hibberd, MD, PhD Gary R. Fleisher, MD From the Division of Emergency Medicine (Weiner, Fleisher), the Children’s Hospital Informatics Program, Division of Endocrinology (Butte), and the Clinical Research Program, Division of Infectious Disease (Hibberd), Children’s Hospital Boston, Boston, MA, and Harvard Medical School, Boston, MA (Weiner, Butte, Hibberd, Fleisher).
Study objective: Success of prospective studies, particularly in the emergency department, often depends on immediate identification of eligible patients to ensure timely sample collection and initiation of study interventions. We report use of a realtime automated notification system to identify potential patients for a clinical trial at the time of ED registration on the basis of information routinely collected. We hypothesize that the automated notification system improves the rate of investigator notification. Methods: We performed a prospective comparison of the notification rate by the automated notification system compared with that by ED clinicians. Results: In the 11 months before use of the automated notification system, the investigator was notified by ED staff for 56% of 61 potentially eligible patients. During 10 months of using the automated notification system, the investigator was paged by the automated notification system for 84% of 49 potentially eligible patients. Conclusion: The automated notification system improves study investigator notification. Use requires online linked registration, a database, and paging systems. The automated notification system is a potentially valuable tool in the recruitment of patients for clinical trials. [Ann Emerg Med. 2003;41:242-246.]
Copyright © 2003 by the American College of Emergency Physicians. 0196-0644/2003/$30.00 + 0 doi:10.1067/mem.2003.52
2 4 2
ANNALS OF EMERGENCY MEDICINE
41:2
FEBRUARY 2003
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
INTRODUCTION
The success of prospective clinical research studies, particularly in the emergency department, often depends on immediate identification of potentially eligible patients to ensure timely collection of critical samples, initiation of study interventions, or both. Large numbers of clinicians not directly involved in research must be aware of active studies, know their eligibility criteria, and take the time either to notify the study investigators or to obtain consent from and enroll patients themselves. Especially in a busy ED, consistent and timely study investigator notification often does not occur. A system that maximizes enrollment without disrupting clinical care in the ED is needed. Computerized, real-time event-monitoring systems that use alphanumeric pages to notify clinicians have been used in clinical care to assist with triage decisions; identify clinical practice guidelines for which patients might be eligible; provide alerts for physiologic data, laboratory results, and medication order errors; and offer diagnostic decision support and treatment recommendations.1-3 Computer software has also been developed to improve clinical trial processes, including recruitment, enrollment, randomization, data collection, and analysis.4-7 In general, these systems have been dependent on entry of patient clinical data generated by a clinician. Particularly in the ED, where patient care often precedes clinical data entry, a point-of-entry patient identification and recruitment system would likely be advantageous, particularly for interventional trials. A computer-to-pager automated notification system (RealTime Recruiting, Atul Butte, MD, Children’s Hospital Boston, Boston, MA) has previously been used by us in our ED to identify potential study patients and to notify a study investigator in real time on the basis of critical laboratory values.8,9 We hypothesized that this software system could be modified to identify potential study participants at the time of registration in the ED. The specific aim of this pilot study was to compare the rate of study investigator pages by the automated notification system with the rate from ED staff for potential study patients for a prospective clinical trial. The clinical trial for which the automated notification system was being used evaluated the use of inhaled nitric oxide in the
FEBRUARY 2003
41:2
ANNALS OF EMERGENCY MEDICINE
treatment of acute vaso-occlusive crisis in pediatric patients with sickle cell disease. M AT E R I A L S A N D M E T H O D S
The Children’s Hospital Boston ED is an urban, academic tertiary-care center. Thirty attending physicians supervise 15 fellows and approximately 200 rotating residents/students annually. Nursing care is provided by 35 permanent and 15 per diem nurses. Patients with possible sickle cell vaso-occlusive crisis are triaged by a nurse directly to a patient-care room, where they are registered and seen promptly by the emergency physician, who initiates diagnostic evaluation and treatment. There are approximately 75 ED visits per year by patients potentially eligible for the clinical trial. Potential eligibility criteria for the clinical trial included sickle cell disease, age 10 to 21 years, pain likely caused by vaso-occlusion, and temperature of less than 38.5°C (<101.3°F). The study investigator determined final eligibility on the basis of more detailed criteria. Study inhalation therapy had to be initiated within 90 minutes of the first dose of the opioid given in the ED. The clinical trial, including testing of the effectiveness of the automated notification system, was approved by the hospital’s institutional review board. Informed consent was obtained for participation in the clinical trial. Emergency physicians and nurses were informed about the study through seminars and handouts and were reminded on a routine basis with signs in the ED, verbal communications several times a week, and e-mail messages every few months. In addition, the charting area of the ED has a bulletin board for information about each active study in the ED that includes eligibility criteria, instructions, and beeper numbers to page the study investigator. Physicians and nurses were requested to page the study investigator 24 hours a day for all patients meeting potential eligibility criteria. Paging by ED staff was the only notification system for the first 11 months of the clinical trial. The automated notification system, once initiated, was monitored for rate of pages for 10 months. The automated notification system (Figure) identifies potentially eligible patients on the basis of information
2 4 3
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
routinely entered into the hospital’s computerized registration system (Allegra; IntraNexus, Virginia Beach, VA) and includes registration site (ED), patient name, medical record number, date of birth, temperature, and chief complaint. Registration information is automatically transferred to the hospital central data repository (Oracle, Redwood City, CA). The automated notification system software connects to and scans the Oracle database by using Java Database Connectivity every 2 minutes and tracks all patients for up to 8 hours after their ED registration. The automated notification system automatically pages the study investigator with patient name, medical record number, and date and time of registration when an ED patient meeting the specified criteria is registered. A log of patients detected and paged by the automated notification system is generated and available to the investigator on a passwordprotected Intranet Web site. In addition to the information provided by page, the Web site includes the date and time that the automated notification system initiated the page. For this clinical trial, the patients that the automated notification system was programmed to recognize were those in an existing Division of Hematology electronic database of patients with sickle cell disease modified to include only patients between the ages of 10 and 21 years. This database was used by the automated notification system to search the central data repository. Clinician-initiated notification was continued
even once the automated notification system began thus providing a dual-notification system. Notification rates were compared by using the χ2 statistic to test the hypothesis that the automated notification system would improve study investigator notification of potential study patients compared with notification by ED staff. This analysis was performed with StatXact 4.0.1 software (Cytel Software Corporation, Cambridge, MA). To evaluate the effectiveness of the automated notification system, only patients in the database submitted to the automated notification system were considered potentially eligible. The number of ED visits by potentially eligible patients was ascertained retrospectively by using multiple approaches. The ED log generated by the registration system of all patients seen in the ED was reviewed by a hematology nurse practitioner familiar with all patients with sickle cell disease. In addition, 2 separate chart queries of the ED electronic medical record system were performed: one by names of patients in the hematology database used by the automated notification system and the other by whole-chart search for the keywords “sickle” and “HbS.” Once the automated notification system was initiated, patient visits were also identified by means of review of the automated notification system–generated Web-based log. The goal was to increase notification by 20% or greater, which, for a 3-year study with a 30-month enrollment period, could translate into a 6-month time savings.
Figure.
Schematic of flow of information with the automated notification system. Registration information is automatically transferred to the hospital central data repository (Oracle). The automated notification system software scans the Oracle database and automatically pages the study investigator when a patient meeting criteria specified in the automated notification system database is registered. An Intranet log of pages initiated by the automated notification system is generated.
2 4 4
ED registration Registration site Date, time Name, MR#, DOB Temperature, chief complaint
Sickle cell patient database
“Allegra”
Name, MR# DOB (a. 10-21 y) HbSS, HbSC, HbSBthal
Central data repository “Oracle”
Real time recruiting “RTR”
“RTR” database and statistics
Beeper
Intranet
“Sickle” Name, MR#, age Registration date/time
Name, MR#, age Registration date/time Page date/time
ANNALS OF EMERGENCY MEDICINE
41:2
FEBRUARY 2003
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
R E S U LT S
In the 11 months before use of the automated notification system, the investigator was paged for 34 (56%) of 61 potentially eligible patients. In the 10 months with use of the automated notification system, the investigator was paged for 41 (84%) of 49 potentially eligible patients, a significant increase in notification rate (χ2=9.77; 95% confidence interval 12% to 44%). The study investigator was paged by the automated notification system within 15 minutes for 86% and within 30 minutes for 93% of these patients (median 9 minutes; range 5 minutes to 7.8 hours). The interval between scans was less than 5 minutes for 99.95% of scans. Six patients were missed by the automated notification system despite their inclusion in the automated notification system database: 2 because the automated notification system was offline, 3 because ED registration was offline, and 1 because the paging system was not operational. The automated notification system pages for an additional 2 patients were delayed beyond the time period that would allow study enrollment because of treatment already received in the ED (2.4 and 7.8 hours, respectively). The automated notification system was offline for more than 1 hour 18 times during the 10month study period (median 1.65 days; range 1.9 hours to 11.5 days), a total of 15% of the time or the equivalent of 54.8 days. The investigator was not notified by the automated notification system about an additional 5 patients not in the hematology database who had a total of 7 ED visits after initiation of the automated notification system. The investigator was paged by the automated notification system for 44 patients with sickle cell disease who were not eligible because they had a chief complaint other than vaso-occlusive crisis, including 16 with a temperature of greater than 38.5°C (101.3°F). DISCUSSION
To our knowledge, this is the first report to describe use of a computerized, real-time recruitment system designed to automatically identify and notify study investigators about potential patients for clinical trials on the basis of information obtained at ED patient registra-
FEBRUARY 2003
41:2
ANNALS OF EMERGENCY MEDICINE
tion. The automated notification system significantly improved investigator notification of potential study patients for an actual clinical trial of inhaled nitric oxide for treatment of acute vaso-occlusive crisis in children with sickle cell disease. We have identified and are addressing issues of patient identification and investigator notification related directly or indirectly to the automated notification system that will improve its efficiency. The median time to page of 9 minutes was greater than expected, given the every-2-minute scan time of the automated notification system. This might in part be explained by the fact that, after data collection was complete, it was recognized that ED registration and automated notification system clocks were asynchronous when spot checks revealed that the registration clock is consistently behind the automated notification system by up to 11 minutes. As a first attempt at using the automated notification system, we identified patients on the basis of only a database of appropriately aged patients with sickle cell disease. The automated notification system can be optimized by using multiple search strategies to increase the page rate for potentially eligible patients and decrease the page rate for patients who do not meet potential eligibility criteria. The automated notification system can be programmed to also recognize patients on the basis of a chief complaint that contains the words “sickle,” “Hb,” “pain,” and/or “crisis” entered at registration, to calculate age on the basis of date of birth, and to exclude those with temperatures of 38.5°C (101.3°F) or greater. Arden Syntax, the standard of the American National Standards Institute, would be useful for this approach.10 A limitation of the automated notification system is that although the sensitivity of patient identification is potentially very high, the specificity is limited to clinical or laboratory information that can be entered into the registration system and retrieved on the hospital system. This might result in more pages for patients who do not meet eligibility criteria on the basis of information not entered as part of registration than paging initiated by ED personnel with complete knowledge of criteria for study eligibility. The patient database used by the automated notification system could easily be modified on an ongoing basis to eliminate the names of
2 4 5
COMPUTERIZED STUDY RECRUITING REAL TIME Weiner et al
patients for whom permanent ineligibility is established on the basis of information derived by the investigator during automated notification system–initiated patient evaluation or after enrollment in the study to decrease the number of pages for ineligible patients. Use of the automated notification system will depend, in part, on the quantity of pages acceptable to the investigator, particularly for ineligible patients. We were willing to accept pages for ineligible patients because we were targeting a rare disease and rare event; rate of enrollment of potentially eligible patients was projected to be low; we required clinical determinations and biologic specimens be obtained before initiation of treatment; and treatment needed to be standardized and initiated rapidly. Another limitation of this automated notification system is that it does not function when ED registration, the central hospital computer system, the automated notification system, or the paging system is offline. This limits its use to research or noncritical clinical applications. Moving the automated notification system to a dedicated server has decreased its offline time. Warning systems now notify the investigator of computer outages. At present, patient confidentiality is at least a theoretic concern with the automated notification system. A paging system is not point-to-point secure, and pages could be intercepted. Although receiving information that includes patient identifiers is advantageous, if necessary, the investigator could get this information by calling the ED once paged or checking the passwordprotected Intranet automated notification system log. Alternatively, a security system that scrambles information could potentially be developed. In addition to identifying patients for clinical trials, the automated notification system can also be used to identify patients on the basis of registration information, laboratory values for other clinical studies evaluating diagnostic or treatment options, or both; to identify patients eligible for clinical practice guidelines; and to track ED visits for patients with particular diseases, chief complaints, or both. Therefore, use of the automated notification system is not limited to research but also offers the opportunity to enhance routine patient care.
2 4 6
Author contributions: AJB developed the automated notification system and wrote this application. DLW and AJB conceived of adapting the automated notification system for patient identification on the basis of registration information. DLW, AJB, and GRF conceived the study. DLW and AJB were responsible for data collection. DLW and PLH were responsible for study design and statistical analysis. DLW drafted the manuscript, and all authors contributed substantially to critical review and revision. DLW takes responsibility for the paper as a whole. Received for publication July 5, 2001. Revision received August 1, 2002. Accepted for publication September 3, 2002. Presented at the American Pediatric Societies, Baltimore, MD, May 2001. Dr. Weiner is supported by US Food and Drug Administration Orphan Drug grant RD-R-001686 and General Clinical Research Center, National Center for Research Resources, National Institutes of Health grant RR02172. Dr. Butte is supported by National Library of Medicine grant 5T15 LM07092-07. Nitric oxide for the clinical trial was donated by Pulmonox Medical Corporation. Dr. Butte is the developer of the RealTime Recruiting software and has filed for a patent. Address for reprints: Debra L. Weiner, MD, PhD, Division of Emergency Medicine, Children’s Hospital, 300 Longwood Avenue, Boston, MA 02115; 617-355-4144, fax 617-731-3279; E-mail [email protected]. We thank Susan Kurth, PNP, Division of Hematology for making data from her review of ED logs available to us.
REFERENCES 1. Kuperman GJ, Teich JM, Bates DW, et al. Detecting alerts, notifying the physician and offering action items: a comprehensive alerting system. Proc AMIA Symp. 1996:704-708. 2. Shabot MM, LoBue M, Chen J. Wireless clinical alerts for physiologic, laboratory and medication data. Proc AMIA Symp. 2000:789-793. 3. Aronsky D, Huang PJ. Automatic identification of patients eligible for a pneumonia guideline. Proc AMIA Symp. 2000:12-16. 4. Johnson N, Lilford RJ. Computerized automation of clinical trials. Ballieres Clin Obstet Gynecol. 1990;4:771-786. 5. Silva J, Wittes R. Role of clinical trials informatics in the NCI’s cancer informatics infrastructure. Proc AMIA Symp. 1999:950-953. 6. Ohno-Machado L, Wang SJ, Mar P, et al. Decision support for clinical trial eligibility determination in breast cancer. Proc AMIA Symp. 1999:340-344. 7. Papaconstantinou C, Theocharous G, Mahadevan S. An expert system for assigning patients into clinical trials based on Bayesian networks. J Med Syst. 1998;22:189-202. 8. Butte AJ, Weinstein DA, Kohane IS. Enrolling patients into clinical trials faster using RealTime Recruiting. Proc AMIA Symp. 2000:111-115. 9. Weinstein DA, Butte AJ, Raymond K, et al. High incidence of unrecognized metabolic and endocrine disorders in acutely ill children with previously unrecognized hypoglycemia [abstract]. Pediatr Res. 2001;49:103. 10. Hripcsak G, Ludemann P, Pryor TA, et al. Rationale for the Arden Syntax. Comput Biomed Res. 1994;27:291-324.
ANNALS OF EMERGENCY MEDICINE
41:2
FEBRUARY 2003