- Email: [email protected]
Implementing an obstetric triage acuity scale: interrater reliability and patient flow analysis
Expert Reviews
www.AJOG .org
PATIENT SAFETY SERIES
Implementing an obstetric triage acuity scale: interrater reliability and patient flow analysis David S. Smithson, MD; Rachel Twohey, MET; Tim Rice, BScN; Nancy Watts, MN; Christopher M. Fernandes, MD; Robert J. Gratton, MD
M
ost emergency departments in North America use triage tools to ensure that patients requiring acute care receive priority treatment and to determine which patients can safely wait.1,2 The Canadian Triage and Acuity Scale (CTAS) was introduced in 1999 and revised in 2006 and 2008. It has been studied extensively and has high degrees of both reliability and validity.2-5 However, CTAS includes only a small number of high acuity obstetric determinants that do not reflect the diversity of patients assessed in an obstetric triage unit. Obstetric triage units face many of the same challenges that led to the development and implementation of triage scales in emergency departments. The need to address access to care and long wait times, to assess acuity and workloads, and to increase accountability for limited resources led the Canadian Association of Emergency Physicians to develop a clear system that triages patients consistently within an institution and allows for comparison between institutions.6 The ability to triage, evaluate, and treat patients has been shown to
From the London Health Sciences Centre, Western University, Department of Obstetrics and Gynaecology, London, Ontario, Canada. Received Jan. 15, 2013; revised March 5, 2013; accepted March 21, 2013. Financial support for the study was provided by the Academic Medical Organization of Southwestern Ontario’s Innovation Fund. The authors report no conflict of interest. Reprints: Robert J. Gratton, MD, Chief of Obstetrics, London Health Sciences Centre, Associate Professor, Department of Obstetrics and Gynaecology, 800 Commissioners Rd., London, Ontario, N6A 4V2. Rob.gratton@ lhsc.on.ca. 0002-9378/$36.00 ª 2013 Mosby, Inc. All rights reserved. http://dx.doi.org/10.1016/j.ajog.2013.03.031
A 5-category Obstetric Triage Acuity Scale (OTAS) was developed with a comprehensive set of obstetrical determinants. The objectives of this study were as follows: (1) to test the interrater reliability of OTAS and (2) to determine the distribution of patient acuity and flow by OTAS level. To test the interrater reliability, 110 triage charts were used to generate vignettes and the consistency of the OTAS level assigned by 8 triage nurses was measured. OTAS performed with substantial (Kappa, 0.61 e 0.77, OTAS 1-4) and near perfect correlation (0.87, OTAS 5). To assess patient flow, the times to primary and secondary health care provider assessments and lengths of stay stratified by acuity were abstracted from the patient management system. Two-thirds of triage visits were low acuity (OTAS 4, 5). There was a decrease in length of stay (median [interquartile range], minutes) as acuity decreased from OTAS 1 (120.0 [156.0] minutes) to OTAS 3 (75.0 [120.8]). The major contributor to length of stay was time to secondary health care provider assessment and this did not change with acuity. The percentage of patients admitted to the antenatal or birthing unit decreased from 80% (OTAS 1) to 12% (OTAS 5). OTAS provides a reliable assessment of acuity and its implementation has allowed for triaging of obstetric patients based on acuity, and a more in-depth assessment of the patient flow. By standardizing assessment, OTAS allows for opportunities to improve performance and make comparisons of patient care and flow across organizations. Key words: acuity, obstetric triage, patient flow
reduce length of stay and increase patient satisfaction scores.7 At London Health Sciences Centre (LHSC), obstetric triage provides 24 hour a day urgent and emergent care for all pregnant women beyond 20 weeks’ gestation from the London area and for tertiary referrals from the southwest region of Ontario. Before restructuring of the London hospitals, obstetric triage services were provided at 2 sites. In June 2011, all obstetric services in London were amalgamated at LHSC and approximately 11,300 patients were seen in the new obstetric triage unit in the first year. In planning for this amalgamation, we sought to better understand the volume and acuity in triage and to look for opportunities to improve the quality of care and patient flow. To facilitate this, the perinatal program at LHSC developed and implemented a 5-level acuity classification scale that reflected the variety of patients seen in the
obstetric triage unit. The purposes of this study were to (1) measure the interrater reliability and validity of the Obstetric Triage Acuity Scale (OTAS) and (2) assess the distribution of acuity and patient flow by OTAS level.
Materials and methods This study (17702E) was approved by the Western University Research Ethics Board on Feb. 10, 2011. Interrater reliability OTAS was modeled on the 5-category (1-Resusitative, 2-Emergent, 3-Urgent, 4-Less Urgent, 5-Nonurgent) CTAS tool.2,3 The acuity color coding and goals for time to assessment were replicated. A comprehensive set of obstetric modifiers was developed to reflect the variety of presentations and indications for referral to obstetric triage (Figure 1). An expert review panel comprised of physicians and nurses reviewed the classification
OCTOBER 2013 American Journal of Obstetrics & Gynecology
287
Expert Reviews
Patient Safety Series
www.AJOG.org
FIGURE 1
Obstetric Triage Acuity Scale (OTAS)
Triage tool used in the primary nursing assessment to stratify patients and provide care based on acuity. BPP, biophysical profile; FHR, fetal heart rate; MVC, motor vehicle collision; NST, nonstress test; PPROM, preterm premature rupture of membranes; RUQ, right upper quadrant; SROM, spontaneous rupture of membranes. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
system for accuracy and completeness of obstetric modifiers. To assess the interrater reliability (IRR), an educational program was provided to all the triage nurses. Eight nurses were randomly selected and assigned triage levels to clinical scenarios based on actual patient visits. These scenarios were from randomly selected 4 hour time blocks (2 per day) from June 1, 2011, to Jan. 31, 2012. The short vignettes containing the initial set of facts presented to a triage nurse were incorporated into an online questionnaire using
Survey Monkey. We measured interrater agreement using the weighted Kappa to account for multiple raters, multiple categories, and for similar classification by chance alone. The calculated sample size of 110 scenarios was based on a Kappa correlation level of 0.8, a confidence interval of 95%, and 8 raters.8 Validity As an initial assessment of the validity of OTAS, admission to the birthing and antenatal units was determined from the chart review (Jan. 1, 2009 to Dec. 31,
288 American Journal of Obstetrics & Gynecology OCTOBER 2013
2010) and used as a surrogate measure of resource use. The proportion of patients requiring admission was stratified by OTAS level to examine the relationship between the acuity level and hospital resource use. Patient flow and acuity analysis Before the 2011 merger, current state value stream maps of patient flow at St. Joseph’s Health Care Centre (SJHC) and LHSC were developed. The maps outlined every task performed during a triage visit including all resources used
Patient Safety Series
www.AJOG.org (physical space, human), as well as the length of time required at each step. Leveraging best practices in patient care, we created a future state Value Stream Map for the new obstetric triage unit.9 In planning for the new unit, we assessed the (1) volume, (2) length of stay (LOS), and (3) distribution of acuity at each site premerger and combined the data to create a citywide dataset. First, to assess patient volumes, we performed a retrospective analysis (Jan. 1, 2009 to Dec. 31, 2010) using electronic data from our local patient registration and management systems. The patient volumes were analyzed by time of day and day of week. Second, to assess LOS (as discharge time was not initially captured by our electronic system), the time to key tasks (initial nursing assessment, and secondary health care provider assessment [physician, resident or midwife]) and acuity, a paper chart review was performed on a representative sample of triage visits (Jan. 1, 2009 to Dec. 31, 2010) at both sites. Calculations of the sample sizes to capture a representative cohort (95% confidence level) and make inferences to the entire population given annual triage visit volumes were performed (LHSCetotal visits 9829, sample size 566 visits, SJHCetotal visits 12754, sample size 573 visits). Acuity was assigned based on the OTAS tool by 2 independent reviewers. The distribution of acuity was determined for both SJHC and LHSC before the merger based on OTAS level as percent of the total. The median times to registration, to primary nursing assessment, to secondary health care provider assessment and the LOS were stratified by acuity. Overall LOS and time to assessment were compared using the KruskeleWallis test and the Mann Whitney was used to compare times at each OTAS level. A preliminary assessment of patient flow (volumes and LOS [June 1, 2011e Jan. 31, 2012]) and distribution by acuity (April 1eAug. 31, 2012) was performed postmerger as data was available electronically. Compliance of acuity assignment by nursing staff was also assessed.
Results Interrater reliability IRR was calculated for each category using both direct correlation and the weighted Kappa correlation (Table 1). Based on Kappa references outlined by Landis and Koch (<0 ¼ poor, 0-0.20 ¼ slight, 0.21-0.40 ¼ fair, 0.410.60 ¼ moderate, 0.61-0.80 ¼ substantial, and 0.81-1.0 ¼ almost perfect) overall the OTAS tool showed (Kappa 0.71) “substantial reliability.”10 We found “near perfect” reliability for OTAS-5 and “substantial reliability” at the OTAS 1-4 levels. Validity The proportion of patients admitted to the antenatal or birthing unit was stratified by OTAS level (Figure 2). Admission to the antenatal unit correlated well with acuity. Birthing unit admission correlated with acuity for OTAS 3 to 5. The greatest contributor to birthing room admission in OTAS 3 was term labor. The admission rates to the birthing unit for OTAS 1 and 2 were less than OTAS 3. The combined admission rate correlated better across all the OTAS levels.
Expert Reviews
TABLE 1
Interrater reliability measures of OTAS Measure
Weighted Kappa
Direct correlation coefficient
OTAS 1
0.7685
0.8750
OTAS 2
0.7283
0.8382
OTAS 3
0.6104
0.7500
OTAS 4
0.6482
0.7532
OTAS 5
0.8664
0.8347
Overall
0.7147
0.7898
Kappa references: 0.61-0.80 ¼ substantial correlation, 0.81-1.0 ¼ near perfect correlation. OTAS, obstetric triage acuity scale. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
Patient flow and acuity analysis Figure 3 shows the triage volumes by time of day (combined LHSC and SJHC, Jan. 1, 2010eDec. 31, 2010) over 1 year. The distribution of volume postmerger was similar with peaks at 1000 and 1900 hours, and two-thirds of the 24-hour volume presented between 0700 and 1900 hours.
FIGURE 2
Percentage of patients admitted from obstetric triage
Proportion of triage patients admitted to the birthing unit or antenatal unit by Obstetric Triage Acuity Scale classification. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
OCTOBER 2013 American Journal of Obstetrics & Gynecology
289
Expert Reviews
Patient Safety Series
www.AJOG.org
FIGURE 3
Presentation of patients by time of day to obstetric triage
Total obstetric triage volumes by hour of day from Jan. 1 to Dec. 31, 2010. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
There were some differences in the distribution of acuity (Table 2) at the premerger sites (likely because SJHC was the tertiary care center premerger). The citywide premerger distributions for OTAS 1-3 match very closely to those captured electronically from postmerger. Combining OTAS 4 and 5 patients, 68% (premerger) and 67% (postmerger) of the patients seen were of lower acuity. The median LOS and interquartile ranges (IQR) by site were 95.0 (IQR, 103.0) minutes at LHSC and 90.0 (IQR, 135.0) minutes at SJHC. Comparing the 2 sites, the overall LOS was not significantly different (P > .05). The median
LOS was 100.8 (IQR, 126.0) minutes at LHSC postmerger. We combined the citywide premerger data to assess the LOS by acuity and the contributors to LOS. The LOS by acuity was as follows: OTAS 1 ¼ 120.0 (IQR, 156.0), 2 ¼ 110.0 (IQR, 175.5), 3 ¼ 75.0 (IQR, 120.8), 4 ¼ 84.0 (IQR, 116.3), 5 ¼ 100.0 (IQR, 104.5) minutes. There was an overall significant difference in the LOS based on acuity (P < .05), with the LOS for OTAS 3 and 4 significantly less than the other OTAS levels. Components of LOS were determined for all OTAS levels. In Figure 4, for OTAS 5 patients these components were: time to primary
TABLE 2
Overall distribution by acuity classification Variable OTAS 1
LHSC 09-10 1%
SJHC 09-10 2%
Citywide 09-10
LHSC Apr-Aug 2012
2%
1%
OTAS 2
3%
17%
11%
11%
OTAS 3
10%
26%
19%
21%
OTAS 4
39%
22%
29%
40%
OTAS 5
48%
32%
39%
27%
LHSC, London Health Sciences Centre; OTAS, obstetric triage acuity scale; SJHC, St Joseph’s Health Care Centre. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
290 American Journal of Obstetrics & Gynecology OCTOBER 2013
nursing assessment of 5.0 (IQR, 14.0) minutes, time to secondary health care provider assessment of 59.0 (IQR, 67.0) minutes, and LOS of 100.0 (IQR, 104.5) minutes. The times to nursing triage assessment were OTAS 1 ¼ 0.0 (IQR, 7.5), 2 ¼ 5.0 (IQR 14.5), 3 ¼ 5.0 (IQR 10.0), 4 ¼ 3.0 (IQR, 6.0), and 5 ¼ 5.0 (IQR, 14.0) minutes. Despite the differences in acuity, the times to secondary health care provider were not significantly different (P >.05) (OTAS 1 ¼ 67.0 [IQR, 70.0], 2 ¼ 60.0 [IQR, 96.0], 3 ¼ 46.0 [IQR, 72.0], 4 ¼ 50.0 [IQR, 59.0], 5 ¼ 59.0 [IQR, 67.0] minutes). Lastly, Figure 5 shows successful implementation of acuity tracking using OTAS in over 90% of cases.
Comment We have developed and successfully implemented a comprehensive 5 category obstetric acuity scale. OTAS has substantial IRR and the acuity level correlates with admission rates to the antenatal and birthing units. We have produced an “acuity profile” for the triage unit, and shown that two-thirds of the visits are lower acuity (OTAS 4 and 5). Implementation of OTAS enabled the measurement of patient flow stratified by acuity. The LOS was variable with the shortest LOS for OTAS 3 and 4 patients. The time to nursing triage assessment was less that 5 minutes; however, the time to secondary health care provider was a major determinant to LOS and did not change with acuity. The concept of obstetric triage over the last 10 to 15 years has largely focused on the establishment of triage units (physical space) and the training of advanced practice nurses and midwives.11 Very little research has addressed an understanding of the acuity or patient flow in obstetric triage units. There are 2 reports of classification systems that were used to determine staffing models.12,13 Paisley et al14 published a descriptive study of the implementation of the Florida Hospital Obstetric Triage Acuity Tool in 4 hospitals. They did not report any performance measures of the acuity scale. To our knowledge, the OTAS is the first comprehensive acuity classification tool for
www.AJOG.org
Patient Safety Series
Expert Reviews
FIGURE 4
Contributors to length of stay
Time taken to complete each of the principal steps in the care pathway and cumulative times to primary nursing assessment, secondary health care provider assessment and overall length of stay stratified by acuity. OTAS, obstetric triage acuity scale; SHCP, supplemental health care program. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
OCTOBER 2013 American Journal of Obstetrics & Gynecology
291
Expert Reviews
Patient Safety Series
FIGURE 5
Compliance in assigning OTAS level
Proportion of patients assigned an OTAS score from April 1 to Aug. 31, 2012. OTAS, Obstetric Triage Acuity Scale classification. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
obstetric patients with established reliability and validity. The Canadian Triage and Acuity Scale (CTAS) has a high degree of reliability and validity.4,5 It has been widely accepted2-5,15,16 and implemented in emergency departments in Canada.6 A Canadian Triage and Acuity Scale for Children has more recently been developed17 to address the unique aspects of this patient population. Obstetric triage units, like that at LHSC, provide urgent care to a large volume of patients with unique presentations. The use of CTAS in obstetric triage is limited as it only includes a small number of higher acuity modifers.3 OTAS is based on the 5-category (1-Resusitative, 2-Emergent, 3-Urgent, 4-Less Urgent, 5-Nonurgent) CTAS scale but developed with a comprehensive list of obstetric modifiers. The IRR of CTAS has been studied using clinical vignettes. The overall reliability of OTAS (Kappa .71) is similar to that of triage scales used in emergency departments.4,5,17 Various severity indicators and measurements of resource utilization have been used to study validity. Admission to hospital1,4,18 and LOS1,19 have been used as surrogate
markers of resource use. We demonstrated an increase in the proportion of patients requiring admission to the antenatal unit (2-27%) and the birthing unit (10-53%) as acuity increased. Admission to the antenatal unit correlated well with acuity. Birthing unit admission correlated with acuity for OTAS 3 to 5 as the greatest contributor to admission in the lower acuity groups was term labor. The admission rates to the birthing unit for OTAS 1 and 2 were less than OTAS 3 as the pregnancy complications may require antenatal admission, observation, stabilization or medical management before delivery. The combined admission rate correlated better across all the OTAS levels. In addition, there was a significant decrease in median LOS from OTAS 1-3 (120.0 to 75.0 minutes). Analysis of patient volumes demonstrated peaks at 10 AM and 7 PM. We believe these peaks correlate with referral from clinics and booked induction events, respectively. Two-thirds of the daily volume was seen between 7 AM and 7 PM. These findings have had significant implications for care planning, resource allocation, and strategic optimization of
292 American Journal of Obstetrics & Gynecology OCTOBER 2013
www.AJOG.org staffing levels. There were small, but predictable differences in the acuity profiles based on the level of care the hospitals provided, as SJHC saw a greater percentage of high acuity patients premerger (OTAS 1-3). The citywide premerger distributions for OTAS 1-3 very closely matched those postmerger. The lower proportion of OTAS 5 visits postmerger is likely a result of low acuity booked events that were moved out of triage in the planning process. Combining OTAS 4 and 5 patients, approximately two thirds of our visits were of lower acuity. The distribution of acuity using the Florida Hospital Obstetric Triage Acuity Tool was very different, as 83% of their visits were classified at higher acuity (Levels 2 and 3). This difference may reflect differences in the obstetric determinants and/or differences in the patient populations. The primary outcome measure following implementation of the Florida Hospital Obstetric Triage Acuity Tool was time to initial nursing assessment. Ninety-eight percent of patients were assessed within 10 minutes.14 We observed the median time to triage assessment was 0e5.0 minutes. We further assessed patient flow by measuring the time to secondary health care provider assessment and overall LOS. There was no decrease in time to secondary health care provider assessment with increased acuity and time was longest in OTAS 1 patients. This concerning finding may be related to the small number of OTAS 1 patients and times skewed by a few outliers. We believe that some OTAS 1 patients were admitted directly to the delivery room and bypassed obstetric triage. These cases of immediate assessment would not be reflected in the triage data. There was a decrease in LOS as acuity decreased (OTAS 1-3) but the overall differences across OTAS levels were variable. To improve the LOS and the time to secondary health care provider assessment for more acute patients, we are investigating the feasibility of a “fast-track” pathway to see less acute patients (OTAS 4 and 5) in “acuity adaptable space.”19,20 “Fast track” units and pathways have been adopted in
Patient Safety Series
www.AJOG.org emergency rooms and have been shown to decrease the LOS for low acuity patients and to decrease the length of time to secondary health care provider assessment for higher acuity patients. Computerized simulation software will be used to study the effect of proposed changes in staffing, physical space, and assessment pathways on patient flow. One potential limitation of our study is that the performance of OTAS was studied using “paper based” scenarios of actual patient visits. Although it is practically very difficult to study, the IRR of CTAS and the Emergency Severity Index (ESI) has been compared in “live” and “paper cases” with no significant differences.1,21 We used hospital admission and LOS as surrogate markers of resource use but the validity should be confirmed by assessing use of laboratory and ultrasound investigations. The reliability and validity tested at a tertiary center should be confirmed in a community hospital environment. The study of contributors to patient flow was based on retrospective assessment of charts as there is no way to capture this data in our current patient management system. In the future, we plan to study patient flow and the contributors to delays in “real time” particularly after changes in staffing, secondary health care provider availability and implementation of acuity-based pathways. Although the implementation was very successful, the compliance in assignment of OTAS plateaued near 90%. This practice change in the obstetric triage needs ongoing reinforcement to experienced nurses (who may still rely on their own informal triaging assessments) and in the orientation of midlevel and junior nurses to work in obstetric triage. Ongoing education on the principles and care benefits of triaging, auditing of compliance and setting targets are vital to successful implementation. The OTAS is the first comprehensive obstetric acuity classification tool with established reliability and validity. With implementation of the OTAS tool at
LHSC, obstetric patients are being triaged in a standardized manner. We believe that OTAS will have wide application to other obstetric triage units and emergency departments that provide care to a significant number of pregnant women. In addition, the OTAS scoring system has provided an understanding of our acuity distribution and enabled the measurement of flow and patient care based on acuity. We are now focusing on improving patient care and flow based on acuity using computerized simulation modeling of changes in staffing and physical space. Specific care pathways for the most common presentations within each OTAS level are being developed. We speculate that the introduction of a “fast track pathway” for lower acuity patients in obstetric triage will result in a decrease in LOS for low acuity cases and an improvement in times to secondary assessment in higher acuity patients. REFERENCES 1. Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The Emergency Severity Index Triage Algorithm version 2 is reliable and valid. Acad Emerg Med 2003;10:1070-80. 2. Murray M, Bullard M, Grafstein E, CTAS National Working Group. Revisions to the Canadian Emergency Department Triage and Acuity Scale Implementation guidelines. CJEM 2004;14:421-7. 3. Bullard MJ, Unger B, Spence J, Grafstein E, CTAS National Working Group. Revisions to the Canadian Emergency Department Triage and Acuity Scale Implementation Guidelines. CJEM 2008;10:136-42. 4. Worster A, Fernandes CM, Eva K, Upadhye S. Predictive validity comparison of two five-level triage acuity scales. Eur J Emerg Med 2007;14:188-92. 5. Worster A, Gillboy N, Fernandes CM, et al. Assessment of inter-observer reliability of two five-level triage and acuity scales: a randomized controlled trial. CJEM 2004;6: 240-5. 6. Beveridge R. CAEP issues: the Canadian triage and acuity scale: a new and critical element in health care reform. Canadian Association of Emergency Physicians. J Emerg Med 1998;16:507-11. 7. Yancer DA, Foshee D, Cole H, Beauchamp R, de la Pena W, Keefe T, et al. Managing capacity
Expert Reviews
to reduce emergency department overcrowding and ambulance diversions. J Qual Patient Safety 2006;32:239-45. 8. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use, 4th ed. Oxford University Press; 2008. 9. Ng D, Vail G, Thomas S, Schmidt N. Applying the Lean principles of the Toyota Production System to reduce wait times in the emergency department. CJEM 2010;12:50-7. 10. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74. 11. Angelini DJ. Obstetric triage and advanced practice nursing. J Perinat Neonatal Nurs 2000; 13:1-12. 12. Freitas CA, Helmer FT, Cousins N. The development and management uses of a patient classification for a high risk perinatal center. J Obstet Gynecol Neontal Nurs 1987;16: 330-8. 13. Loper D, Ham E. Creating a patient classification system: one centers experience in the triage process. J Perinat Neonatal Nurs 2000;13:1-49. 14. Paisley KS, Wallace R, DuRant PG. The development of an obstetrical triage acuity tool. MCN Am J Matern Child Nurs 2011;36: 290-6. 15. Jimenez JG, Murray MJ, Beveridge R, et al. Implementation of the Canadian Triage and Acuity Scale in the principality of Andorra: can triage parameters serve as emergency department quality indicators? CJEM 2003;5: 315-22. 16. Devkaran S, Parsons H, Van Dyke M. The impact of a fast track area on quality and effectiveness outcomes: a Middle Eastern emergency department perspective. BMC Emeg Med 2009;9:11-7. 17. Warren DW, Jarvis A, LeBlanc L, et al. Revisions to the Canadian Triage and acuity scale paediatric guidelines (PeadCTAS). CMEJ 2008;10:224-43. 18. Gravel J, Fitzpatrick E, Gouin S, et al. Performance of the Canadian triage and acuity scale for children: a multicentre database study. Ann Emerg Med 2012;61: 27-32.e3. 19. Hendrich AL, Fay J, Sorrells AK. Effects of acuity-adaptable rooms on flow of patients and delivery of care. Am J Crit Care 2004;13: 35-45. 20. Al-Darrab AA, Fan J, Fernandes CM, et al. How does fast track affect quality of care in the emergency department? Eur J Emerg Med 2005;13:32-5. 21. Worster A, Sardo A, Eva K, Fernandes CM, Upadhye S. Triage tool inter-rater reliability: a comparison of live versus paper case scenarios. J Emerg Nurs 2007;33:319-23.
OCTOBER 2013 American Journal of Obstetrics & Gynecology
293
www.AJOG .org
PATIENT SAFETY SERIES
Implementing an obstetric triage acuity scale: interrater reliability and patient flow analysis David S. Smithson, MD; Rachel Twohey, MET; Tim Rice, BScN; Nancy Watts, MN; Christopher M. Fernandes, MD; Robert J. Gratton, MD
M
ost emergency departments in North America use triage tools to ensure that patients requiring acute care receive priority treatment and to determine which patients can safely wait.1,2 The Canadian Triage and Acuity Scale (CTAS) was introduced in 1999 and revised in 2006 and 2008. It has been studied extensively and has high degrees of both reliability and validity.2-5 However, CTAS includes only a small number of high acuity obstetric determinants that do not reflect the diversity of patients assessed in an obstetric triage unit. Obstetric triage units face many of the same challenges that led to the development and implementation of triage scales in emergency departments. The need to address access to care and long wait times, to assess acuity and workloads, and to increase accountability for limited resources led the Canadian Association of Emergency Physicians to develop a clear system that triages patients consistently within an institution and allows for comparison between institutions.6 The ability to triage, evaluate, and treat patients has been shown to
From the London Health Sciences Centre, Western University, Department of Obstetrics and Gynaecology, London, Ontario, Canada. Received Jan. 15, 2013; revised March 5, 2013; accepted March 21, 2013. Financial support for the study was provided by the Academic Medical Organization of Southwestern Ontario’s Innovation Fund. The authors report no conflict of interest. Reprints: Robert J. Gratton, MD, Chief of Obstetrics, London Health Sciences Centre, Associate Professor, Department of Obstetrics and Gynaecology, 800 Commissioners Rd., London, Ontario, N6A 4V2. Rob.gratton@ lhsc.on.ca. 0002-9378/$36.00 ª 2013 Mosby, Inc. All rights reserved. http://dx.doi.org/10.1016/j.ajog.2013.03.031
A 5-category Obstetric Triage Acuity Scale (OTAS) was developed with a comprehensive set of obstetrical determinants. The objectives of this study were as follows: (1) to test the interrater reliability of OTAS and (2) to determine the distribution of patient acuity and flow by OTAS level. To test the interrater reliability, 110 triage charts were used to generate vignettes and the consistency of the OTAS level assigned by 8 triage nurses was measured. OTAS performed with substantial (Kappa, 0.61 e 0.77, OTAS 1-4) and near perfect correlation (0.87, OTAS 5). To assess patient flow, the times to primary and secondary health care provider assessments and lengths of stay stratified by acuity were abstracted from the patient management system. Two-thirds of triage visits were low acuity (OTAS 4, 5). There was a decrease in length of stay (median [interquartile range], minutes) as acuity decreased from OTAS 1 (120.0 [156.0] minutes) to OTAS 3 (75.0 [120.8]). The major contributor to length of stay was time to secondary health care provider assessment and this did not change with acuity. The percentage of patients admitted to the antenatal or birthing unit decreased from 80% (OTAS 1) to 12% (OTAS 5). OTAS provides a reliable assessment of acuity and its implementation has allowed for triaging of obstetric patients based on acuity, and a more in-depth assessment of the patient flow. By standardizing assessment, OTAS allows for opportunities to improve performance and make comparisons of patient care and flow across organizations. Key words: acuity, obstetric triage, patient flow
reduce length of stay and increase patient satisfaction scores.7 At London Health Sciences Centre (LHSC), obstetric triage provides 24 hour a day urgent and emergent care for all pregnant women beyond 20 weeks’ gestation from the London area and for tertiary referrals from the southwest region of Ontario. Before restructuring of the London hospitals, obstetric triage services were provided at 2 sites. In June 2011, all obstetric services in London were amalgamated at LHSC and approximately 11,300 patients were seen in the new obstetric triage unit in the first year. In planning for this amalgamation, we sought to better understand the volume and acuity in triage and to look for opportunities to improve the quality of care and patient flow. To facilitate this, the perinatal program at LHSC developed and implemented a 5-level acuity classification scale that reflected the variety of patients seen in the
obstetric triage unit. The purposes of this study were to (1) measure the interrater reliability and validity of the Obstetric Triage Acuity Scale (OTAS) and (2) assess the distribution of acuity and patient flow by OTAS level.
Materials and methods This study (17702E) was approved by the Western University Research Ethics Board on Feb. 10, 2011. Interrater reliability OTAS was modeled on the 5-category (1-Resusitative, 2-Emergent, 3-Urgent, 4-Less Urgent, 5-Nonurgent) CTAS tool.2,3 The acuity color coding and goals for time to assessment were replicated. A comprehensive set of obstetric modifiers was developed to reflect the variety of presentations and indications for referral to obstetric triage (Figure 1). An expert review panel comprised of physicians and nurses reviewed the classification
OCTOBER 2013 American Journal of Obstetrics & Gynecology
287
Expert Reviews
Patient Safety Series
www.AJOG.org
FIGURE 1
Obstetric Triage Acuity Scale (OTAS)
Triage tool used in the primary nursing assessment to stratify patients and provide care based on acuity. BPP, biophysical profile; FHR, fetal heart rate; MVC, motor vehicle collision; NST, nonstress test; PPROM, preterm premature rupture of membranes; RUQ, right upper quadrant; SROM, spontaneous rupture of membranes. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
system for accuracy and completeness of obstetric modifiers. To assess the interrater reliability (IRR), an educational program was provided to all the triage nurses. Eight nurses were randomly selected and assigned triage levels to clinical scenarios based on actual patient visits. These scenarios were from randomly selected 4 hour time blocks (2 per day) from June 1, 2011, to Jan. 31, 2012. The short vignettes containing the initial set of facts presented to a triage nurse were incorporated into an online questionnaire using
Survey Monkey. We measured interrater agreement using the weighted Kappa to account for multiple raters, multiple categories, and for similar classification by chance alone. The calculated sample size of 110 scenarios was based on a Kappa correlation level of 0.8, a confidence interval of 95%, and 8 raters.8 Validity As an initial assessment of the validity of OTAS, admission to the birthing and antenatal units was determined from the chart review (Jan. 1, 2009 to Dec. 31,
288 American Journal of Obstetrics & Gynecology OCTOBER 2013
2010) and used as a surrogate measure of resource use. The proportion of patients requiring admission was stratified by OTAS level to examine the relationship between the acuity level and hospital resource use. Patient flow and acuity analysis Before the 2011 merger, current state value stream maps of patient flow at St. Joseph’s Health Care Centre (SJHC) and LHSC were developed. The maps outlined every task performed during a triage visit including all resources used
Patient Safety Series
www.AJOG.org (physical space, human), as well as the length of time required at each step. Leveraging best practices in patient care, we created a future state Value Stream Map for the new obstetric triage unit.9 In planning for the new unit, we assessed the (1) volume, (2) length of stay (LOS), and (3) distribution of acuity at each site premerger and combined the data to create a citywide dataset. First, to assess patient volumes, we performed a retrospective analysis (Jan. 1, 2009 to Dec. 31, 2010) using electronic data from our local patient registration and management systems. The patient volumes were analyzed by time of day and day of week. Second, to assess LOS (as discharge time was not initially captured by our electronic system), the time to key tasks (initial nursing assessment, and secondary health care provider assessment [physician, resident or midwife]) and acuity, a paper chart review was performed on a representative sample of triage visits (Jan. 1, 2009 to Dec. 31, 2010) at both sites. Calculations of the sample sizes to capture a representative cohort (95% confidence level) and make inferences to the entire population given annual triage visit volumes were performed (LHSCetotal visits 9829, sample size 566 visits, SJHCetotal visits 12754, sample size 573 visits). Acuity was assigned based on the OTAS tool by 2 independent reviewers. The distribution of acuity was determined for both SJHC and LHSC before the merger based on OTAS level as percent of the total. The median times to registration, to primary nursing assessment, to secondary health care provider assessment and the LOS were stratified by acuity. Overall LOS and time to assessment were compared using the KruskeleWallis test and the Mann Whitney was used to compare times at each OTAS level. A preliminary assessment of patient flow (volumes and LOS [June 1, 2011e Jan. 31, 2012]) and distribution by acuity (April 1eAug. 31, 2012) was performed postmerger as data was available electronically. Compliance of acuity assignment by nursing staff was also assessed.
Results Interrater reliability IRR was calculated for each category using both direct correlation and the weighted Kappa correlation (Table 1). Based on Kappa references outlined by Landis and Koch (<0 ¼ poor, 0-0.20 ¼ slight, 0.21-0.40 ¼ fair, 0.410.60 ¼ moderate, 0.61-0.80 ¼ substantial, and 0.81-1.0 ¼ almost perfect) overall the OTAS tool showed (Kappa 0.71) “substantial reliability.”10 We found “near perfect” reliability for OTAS-5 and “substantial reliability” at the OTAS 1-4 levels. Validity The proportion of patients admitted to the antenatal or birthing unit was stratified by OTAS level (Figure 2). Admission to the antenatal unit correlated well with acuity. Birthing unit admission correlated with acuity for OTAS 3 to 5. The greatest contributor to birthing room admission in OTAS 3 was term labor. The admission rates to the birthing unit for OTAS 1 and 2 were less than OTAS 3. The combined admission rate correlated better across all the OTAS levels.
Expert Reviews
TABLE 1
Interrater reliability measures of OTAS Measure
Weighted Kappa
Direct correlation coefficient
OTAS 1
0.7685
0.8750
OTAS 2
0.7283
0.8382
OTAS 3
0.6104
0.7500
OTAS 4
0.6482
0.7532
OTAS 5
0.8664
0.8347
Overall
0.7147
0.7898
Kappa references: 0.61-0.80 ¼ substantial correlation, 0.81-1.0 ¼ near perfect correlation. OTAS, obstetric triage acuity scale. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
Patient flow and acuity analysis Figure 3 shows the triage volumes by time of day (combined LHSC and SJHC, Jan. 1, 2010eDec. 31, 2010) over 1 year. The distribution of volume postmerger was similar with peaks at 1000 and 1900 hours, and two-thirds of the 24-hour volume presented between 0700 and 1900 hours.
FIGURE 2
Percentage of patients admitted from obstetric triage
Proportion of triage patients admitted to the birthing unit or antenatal unit by Obstetric Triage Acuity Scale classification. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
OCTOBER 2013 American Journal of Obstetrics & Gynecology
289
Expert Reviews
Patient Safety Series
www.AJOG.org
FIGURE 3
Presentation of patients by time of day to obstetric triage
Total obstetric triage volumes by hour of day from Jan. 1 to Dec. 31, 2010. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
There were some differences in the distribution of acuity (Table 2) at the premerger sites (likely because SJHC was the tertiary care center premerger). The citywide premerger distributions for OTAS 1-3 match very closely to those captured electronically from postmerger. Combining OTAS 4 and 5 patients, 68% (premerger) and 67% (postmerger) of the patients seen were of lower acuity. The median LOS and interquartile ranges (IQR) by site were 95.0 (IQR, 103.0) minutes at LHSC and 90.0 (IQR, 135.0) minutes at SJHC. Comparing the 2 sites, the overall LOS was not significantly different (P > .05). The median
LOS was 100.8 (IQR, 126.0) minutes at LHSC postmerger. We combined the citywide premerger data to assess the LOS by acuity and the contributors to LOS. The LOS by acuity was as follows: OTAS 1 ¼ 120.0 (IQR, 156.0), 2 ¼ 110.0 (IQR, 175.5), 3 ¼ 75.0 (IQR, 120.8), 4 ¼ 84.0 (IQR, 116.3), 5 ¼ 100.0 (IQR, 104.5) minutes. There was an overall significant difference in the LOS based on acuity (P < .05), with the LOS for OTAS 3 and 4 significantly less than the other OTAS levels. Components of LOS were determined for all OTAS levels. In Figure 4, for OTAS 5 patients these components were: time to primary
TABLE 2
Overall distribution by acuity classification Variable OTAS 1
LHSC 09-10 1%
SJHC 09-10 2%
Citywide 09-10
LHSC Apr-Aug 2012
2%
1%
OTAS 2
3%
17%
11%
11%
OTAS 3
10%
26%
19%
21%
OTAS 4
39%
22%
29%
40%
OTAS 5
48%
32%
39%
27%
LHSC, London Health Sciences Centre; OTAS, obstetric triage acuity scale; SJHC, St Joseph’s Health Care Centre. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
290 American Journal of Obstetrics & Gynecology OCTOBER 2013
nursing assessment of 5.0 (IQR, 14.0) minutes, time to secondary health care provider assessment of 59.0 (IQR, 67.0) minutes, and LOS of 100.0 (IQR, 104.5) minutes. The times to nursing triage assessment were OTAS 1 ¼ 0.0 (IQR, 7.5), 2 ¼ 5.0 (IQR 14.5), 3 ¼ 5.0 (IQR 10.0), 4 ¼ 3.0 (IQR, 6.0), and 5 ¼ 5.0 (IQR, 14.0) minutes. Despite the differences in acuity, the times to secondary health care provider were not significantly different (P >.05) (OTAS 1 ¼ 67.0 [IQR, 70.0], 2 ¼ 60.0 [IQR, 96.0], 3 ¼ 46.0 [IQR, 72.0], 4 ¼ 50.0 [IQR, 59.0], 5 ¼ 59.0 [IQR, 67.0] minutes). Lastly, Figure 5 shows successful implementation of acuity tracking using OTAS in over 90% of cases.
Comment We have developed and successfully implemented a comprehensive 5 category obstetric acuity scale. OTAS has substantial IRR and the acuity level correlates with admission rates to the antenatal and birthing units. We have produced an “acuity profile” for the triage unit, and shown that two-thirds of the visits are lower acuity (OTAS 4 and 5). Implementation of OTAS enabled the measurement of patient flow stratified by acuity. The LOS was variable with the shortest LOS for OTAS 3 and 4 patients. The time to nursing triage assessment was less that 5 minutes; however, the time to secondary health care provider was a major determinant to LOS and did not change with acuity. The concept of obstetric triage over the last 10 to 15 years has largely focused on the establishment of triage units (physical space) and the training of advanced practice nurses and midwives.11 Very little research has addressed an understanding of the acuity or patient flow in obstetric triage units. There are 2 reports of classification systems that were used to determine staffing models.12,13 Paisley et al14 published a descriptive study of the implementation of the Florida Hospital Obstetric Triage Acuity Tool in 4 hospitals. They did not report any performance measures of the acuity scale. To our knowledge, the OTAS is the first comprehensive acuity classification tool for
www.AJOG.org
Patient Safety Series
Expert Reviews
FIGURE 4
Contributors to length of stay
Time taken to complete each of the principal steps in the care pathway and cumulative times to primary nursing assessment, secondary health care provider assessment and overall length of stay stratified by acuity. OTAS, obstetric triage acuity scale; SHCP, supplemental health care program. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
OCTOBER 2013 American Journal of Obstetrics & Gynecology
291
Expert Reviews
Patient Safety Series
FIGURE 5
Compliance in assigning OTAS level
Proportion of patients assigned an OTAS score from April 1 to Aug. 31, 2012. OTAS, Obstetric Triage Acuity Scale classification. Smithson. Implementing an obstetric triage acuity scale. Am J Obstet Gynecol 2013.
obstetric patients with established reliability and validity. The Canadian Triage and Acuity Scale (CTAS) has a high degree of reliability and validity.4,5 It has been widely accepted2-5,15,16 and implemented in emergency departments in Canada.6 A Canadian Triage and Acuity Scale for Children has more recently been developed17 to address the unique aspects of this patient population. Obstetric triage units, like that at LHSC, provide urgent care to a large volume of patients with unique presentations. The use of CTAS in obstetric triage is limited as it only includes a small number of higher acuity modifers.3 OTAS is based on the 5-category (1-Resusitative, 2-Emergent, 3-Urgent, 4-Less Urgent, 5-Nonurgent) CTAS scale but developed with a comprehensive list of obstetric modifiers. The IRR of CTAS has been studied using clinical vignettes. The overall reliability of OTAS (Kappa .71) is similar to that of triage scales used in emergency departments.4,5,17 Various severity indicators and measurements of resource utilization have been used to study validity. Admission to hospital1,4,18 and LOS1,19 have been used as surrogate
markers of resource use. We demonstrated an increase in the proportion of patients requiring admission to the antenatal unit (2-27%) and the birthing unit (10-53%) as acuity increased. Admission to the antenatal unit correlated well with acuity. Birthing unit admission correlated with acuity for OTAS 3 to 5 as the greatest contributor to admission in the lower acuity groups was term labor. The admission rates to the birthing unit for OTAS 1 and 2 were less than OTAS 3 as the pregnancy complications may require antenatal admission, observation, stabilization or medical management before delivery. The combined admission rate correlated better across all the OTAS levels. In addition, there was a significant decrease in median LOS from OTAS 1-3 (120.0 to 75.0 minutes). Analysis of patient volumes demonstrated peaks at 10 AM and 7 PM. We believe these peaks correlate with referral from clinics and booked induction events, respectively. Two-thirds of the daily volume was seen between 7 AM and 7 PM. These findings have had significant implications for care planning, resource allocation, and strategic optimization of
292 American Journal of Obstetrics & Gynecology OCTOBER 2013
www.AJOG.org staffing levels. There were small, but predictable differences in the acuity profiles based on the level of care the hospitals provided, as SJHC saw a greater percentage of high acuity patients premerger (OTAS 1-3). The citywide premerger distributions for OTAS 1-3 very closely matched those postmerger. The lower proportion of OTAS 5 visits postmerger is likely a result of low acuity booked events that were moved out of triage in the planning process. Combining OTAS 4 and 5 patients, approximately two thirds of our visits were of lower acuity. The distribution of acuity using the Florida Hospital Obstetric Triage Acuity Tool was very different, as 83% of their visits were classified at higher acuity (Levels 2 and 3). This difference may reflect differences in the obstetric determinants and/or differences in the patient populations. The primary outcome measure following implementation of the Florida Hospital Obstetric Triage Acuity Tool was time to initial nursing assessment. Ninety-eight percent of patients were assessed within 10 minutes.14 We observed the median time to triage assessment was 0e5.0 minutes. We further assessed patient flow by measuring the time to secondary health care provider assessment and overall LOS. There was no decrease in time to secondary health care provider assessment with increased acuity and time was longest in OTAS 1 patients. This concerning finding may be related to the small number of OTAS 1 patients and times skewed by a few outliers. We believe that some OTAS 1 patients were admitted directly to the delivery room and bypassed obstetric triage. These cases of immediate assessment would not be reflected in the triage data. There was a decrease in LOS as acuity decreased (OTAS 1-3) but the overall differences across OTAS levels were variable. To improve the LOS and the time to secondary health care provider assessment for more acute patients, we are investigating the feasibility of a “fast-track” pathway to see less acute patients (OTAS 4 and 5) in “acuity adaptable space.”19,20 “Fast track” units and pathways have been adopted in
Patient Safety Series
www.AJOG.org emergency rooms and have been shown to decrease the LOS for low acuity patients and to decrease the length of time to secondary health care provider assessment for higher acuity patients. Computerized simulation software will be used to study the effect of proposed changes in staffing, physical space, and assessment pathways on patient flow. One potential limitation of our study is that the performance of OTAS was studied using “paper based” scenarios of actual patient visits. Although it is practically very difficult to study, the IRR of CTAS and the Emergency Severity Index (ESI) has been compared in “live” and “paper cases” with no significant differences.1,21 We used hospital admission and LOS as surrogate markers of resource use but the validity should be confirmed by assessing use of laboratory and ultrasound investigations. The reliability and validity tested at a tertiary center should be confirmed in a community hospital environment. The study of contributors to patient flow was based on retrospective assessment of charts as there is no way to capture this data in our current patient management system. In the future, we plan to study patient flow and the contributors to delays in “real time” particularly after changes in staffing, secondary health care provider availability and implementation of acuity-based pathways. Although the implementation was very successful, the compliance in assignment of OTAS plateaued near 90%. This practice change in the obstetric triage needs ongoing reinforcement to experienced nurses (who may still rely on their own informal triaging assessments) and in the orientation of midlevel and junior nurses to work in obstetric triage. Ongoing education on the principles and care benefits of triaging, auditing of compliance and setting targets are vital to successful implementation. The OTAS is the first comprehensive obstetric acuity classification tool with established reliability and validity. With implementation of the OTAS tool at
LHSC, obstetric patients are being triaged in a standardized manner. We believe that OTAS will have wide application to other obstetric triage units and emergency departments that provide care to a significant number of pregnant women. In addition, the OTAS scoring system has provided an understanding of our acuity distribution and enabled the measurement of flow and patient care based on acuity. We are now focusing on improving patient care and flow based on acuity using computerized simulation modeling of changes in staffing and physical space. Specific care pathways for the most common presentations within each OTAS level are being developed. We speculate that the introduction of a “fast track pathway” for lower acuity patients in obstetric triage will result in a decrease in LOS for low acuity cases and an improvement in times to secondary assessment in higher acuity patients. REFERENCES 1. Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The Emergency Severity Index Triage Algorithm version 2 is reliable and valid. Acad Emerg Med 2003;10:1070-80. 2. Murray M, Bullard M, Grafstein E, CTAS National Working Group. Revisions to the Canadian Emergency Department Triage and Acuity Scale Implementation guidelines. CJEM 2004;14:421-7. 3. Bullard MJ, Unger B, Spence J, Grafstein E, CTAS National Working Group. Revisions to the Canadian Emergency Department Triage and Acuity Scale Implementation Guidelines. CJEM 2008;10:136-42. 4. Worster A, Fernandes CM, Eva K, Upadhye S. Predictive validity comparison of two five-level triage acuity scales. Eur J Emerg Med 2007;14:188-92. 5. Worster A, Gillboy N, Fernandes CM, et al. Assessment of inter-observer reliability of two five-level triage and acuity scales: a randomized controlled trial. CJEM 2004;6: 240-5. 6. Beveridge R. CAEP issues: the Canadian triage and acuity scale: a new and critical element in health care reform. Canadian Association of Emergency Physicians. J Emerg Med 1998;16:507-11. 7. Yancer DA, Foshee D, Cole H, Beauchamp R, de la Pena W, Keefe T, et al. Managing capacity
Expert Reviews
to reduce emergency department overcrowding and ambulance diversions. J Qual Patient Safety 2006;32:239-45. 8. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use, 4th ed. Oxford University Press; 2008. 9. Ng D, Vail G, Thomas S, Schmidt N. Applying the Lean principles of the Toyota Production System to reduce wait times in the emergency department. CJEM 2010;12:50-7. 10. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74. 11. Angelini DJ. Obstetric triage and advanced practice nursing. J Perinat Neonatal Nurs 2000; 13:1-12. 12. Freitas CA, Helmer FT, Cousins N. The development and management uses of a patient classification for a high risk perinatal center. J Obstet Gynecol Neontal Nurs 1987;16: 330-8. 13. Loper D, Ham E. Creating a patient classification system: one centers experience in the triage process. J Perinat Neonatal Nurs 2000;13:1-49. 14. Paisley KS, Wallace R, DuRant PG. The development of an obstetrical triage acuity tool. MCN Am J Matern Child Nurs 2011;36: 290-6. 15. Jimenez JG, Murray MJ, Beveridge R, et al. Implementation of the Canadian Triage and Acuity Scale in the principality of Andorra: can triage parameters serve as emergency department quality indicators? CJEM 2003;5: 315-22. 16. Devkaran S, Parsons H, Van Dyke M. The impact of a fast track area on quality and effectiveness outcomes: a Middle Eastern emergency department perspective. BMC Emeg Med 2009;9:11-7. 17. Warren DW, Jarvis A, LeBlanc L, et al. Revisions to the Canadian Triage and acuity scale paediatric guidelines (PeadCTAS). CMEJ 2008;10:224-43. 18. Gravel J, Fitzpatrick E, Gouin S, et al. Performance of the Canadian triage and acuity scale for children: a multicentre database study. Ann Emerg Med 2012;61: 27-32.e3. 19. Hendrich AL, Fay J, Sorrells AK. Effects of acuity-adaptable rooms on flow of patients and delivery of care. Am J Crit Care 2004;13: 35-45. 20. Al-Darrab AA, Fan J, Fernandes CM, et al. How does fast track affect quality of care in the emergency department? Eur J Emerg Med 2005;13:32-5. 21. Worster A, Sardo A, Eva K, Fernandes CM, Upadhye S. Triage tool inter-rater reliability: a comparison of live versus paper case scenarios. J Emerg Nurs 2007;33:319-23.
OCTOBER 2013 American Journal of Obstetrics & Gynecology
293