- Email: [email protected]
Compliance with national guidelines for stroke in radiology
Operations Research for Health Care 6 (2015) 33–39
Contents lists available at ScienceDirect
Operations Research for Health Care journal homepage: www.elsevier.com/locate/orhc
Compliance with national guidelines for stroke in radiology Izabela Komenda ∗ , Vincent Knight, Hannah Mary Williams School of Mathematics, Cardiff University, Cardiff, Wales, United Kingdom
article
info
Article history: Received 19 June 2014 Accepted 6 September 2015 Available online 14 September 2015 Keywords: Simulation modelling Stroke Radiology
abstract Stroke is a medical emergency, and if patient outcomes are to be optimised there should be no delays in accessing treatment. This project focuses on the application of Operational Research methodology to investigate how a hospital can comply with the revised computerised tomography (CT) scanning guidelines for stroke. Such guidelines, released by the Royal College of Physicians recommend a 50% reduction in time from hospital admission to report of a CT head scan to just 12 hours. The results of statistical analyses of historical data were used to populate a discrete event simulation model of patient flow through the CT scanning unit. The model was then used to explore a number of operational modifications to the CT scanning system through a series of scenario analyses. The results of this investigation presented evidence of a number of strategies to support operational improvements in relation to revised stroke guidelines. Crown Copyright © 2015 Published by Elsevier Ltd. All rights reserved.
1. Introduction Stroke is a serious medical illness with a mortality rate higher than most forms of cancer (Davis et al. [1]). Each year, approximately 15 million people worldwide and 152,000 people in the United Kingdom suffer a stroke (Townsend et al. [2]). Recent publications from the British Heart Foundation report that during 2010, stroke was the fourth largest cause of mortality in the UK (Townsend et al. [2]). If outcomes for stroke patients are to be optimised, accessing treatment should not be delayed (Jauch et al. [3]). Previously, guidelines provided by the Royal College of Physicians (RCP) have suggested that a computerised tomography (CT) brain-imaging scan has to be delivered within a maximum of 24 h of hospital admission for all patients with symptoms of stroke. However, the latest clinical guidelines released in December 2012 (RCP [4]) have recommended a 50% reduction in the aforementioned time window to just 12 h. The main aim of this paper is to investigate the effect on various aspects of the system and therefore evaluate the likely benefit, or otherwise, in relation to specific targets for stroke. This has been achieved by incorporating various modifications to a simulation model that models current CT scanning activity. The key outputs of interest were the overall time in the system for stroke patients and the percentage compliance under guideline time frames. The model predicted that by necessitating that all routine stroke patients are seen for a CT head scan
∗
Corresponding author. E-mail address: [email protected] (I. Komenda).
http://dx.doi.org/10.1016/j.orhc.2015.09.001 2211-6923/Crown Copyright © 2015 Published by Elsevier Ltd. All rights reserved.
on the day it is requested, an improvement in compliance with revised 12 h stroke guidelines could be achieved. Moreover, additional benefits were projected when routine stroke patients were re-prioritised further, ahead of patients with non-urgent, unscheduled CT scan requests. This project was conducted within the Radiology Department of the Royal Gwent Hospital, Newport, Aneurin Bevan University Health Board (ABUHB). ABUHB is one of the seven local health boards in Wales which serves an estimated population of over 639,000, approximately 21% of the total Welsh population. The existing Operational Research (OR) literature surrounding planning for CT scanning processes focuses mainly on scheduling (Boland [5]; Patrick and Puterman [6]; Vermeulen et al. [7]; Van Lent et al. [8]). Discrete event simulation is employed as the most appropriate modelling technique for the identified decision support problem. Several studies have been conducted where use of a simulation model facilitated improvement in the overall acute stroke pathway (Bayer et al. [9], Churilov and Donnan [10], Churilov et al. [11], Cordeaux et al. [12], Lahr et al. [13,14], Mar et al. [15], Monks et al. [16,17] and Pitt et al. [18]); however there is a distinct lack of research concerning the specific use of discrete event simulation in the CT scanning environment. The main contribution of this paper to the existing literature is a casestudy applied to the CT unit. It is hoped that this area will greatly benefit from this application. The paper is organised as follows. In Section 2 statistical analysis of CT request data is described. In Section 3 the model is introduced, which is then validated and verified in Section 4. In Section 5 results from ‘what if...?’ scenarios are discussed before summarising the findings in Section 6.
34
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
Table 1 CT scan request categories. Patient origin
RGH Ward
Pathology
Urgency of scan
Description
Request category
Thrombolysis
Immediate
Inpatient Thrombolysis
Routine Stroke Not Stroke Not Stroke
By the end of the day following the request Immediate By the end of the day the request is made By the end of the day following the request
Ward patients who qualify for an assessment of their suitability for thrombolysis treatment Ward patients with suspected stroke for which thrombolysis treatment is known not to be appropriate Ward patients who require a CT scan immediately Ward patients who require a CT scan on the day of request Ward patients who require a CT scan by the end of the day following its request
Inpatient Tomorrow
A&E patients who qualify for assessment of their suitability for thrombolysis treatment A&E patients with suspected stroke for which thrombolysis treatment is known not to be appropriate A&E patients who require a CT scan immediately A&E patients who require a CT scan on the day of request
A&E Thrombolysis
GP referrals with pre-scheduled appointments
Outpatient
Not Stroke
A&E
Outpatient
Thrombolysis
Immediate
Routine Stroke Not Stroke Not Stroke
By the end of the day the request is made Immediate By the end of the day the request is made Must be seen on day of appointment
Not Stroke
2. Statistical analysis of CT request data The data set was extracted from the radiology information system, RadIS II, which contains a fairly complete record of CT activity at the RGH. An additional data set contains all A&E activity. Both data sets relate to the same two-year time frame, from the 1st June 2011 to 31st May 2013. CT scans are requested at different levels of urgency for a number of different patient types. This information is summarised in Table 1. A total of 1,869 CT scan requests were stroke related and from these, 295 necessary to assess whether thrombolysis treatment was appropriate. Thrombolysis is a treatment to dissolve blood clots by pharmacological means. It works by injecting clot-busting drug into blood vessels. This mode of treatment is only suitable for victims of ischaemic stroke and must be delivered within four and a half hours of symptom onset (Emberson et al. [19]). Monthly, weekly and hourly time-dependency analysis was completed in an attempt to establish patterns and trends in the frequency of unscheduled CT scans requested with respect to time. Kruskal Wallis analysis was applied to investigate whether the frequency of CT scan requests differed significantly across month of the year and day of the week. Pairwise Wilcoxon testing has been applied to group patient categories by weekday. Classification and Regression Tree (CART) analysis was applied to partition a full data set into smaller, homogeneous subsets for each weekday grouping and for each request category. For each request according to month of the year, a p-value is reported: A&E Thrombolysis (p-value = 0.7570), A&E Routine Stroke (p-value = 0.5201), A&E Immediate (p-value = 0.999), A&E Today (p-value = 0.722), Inpatient Thrombolysis (p-value = 0.366), Inpatient Routine Stroke (p-value = 0.955), Inpatient Immediate (p-value = 0.966), Inpatient Today (p-value = 0.463) and Inpatient Tomorrow (p-value = 0.853). P-value greater than 0.05 for each category suggests no seasonal trends. The frequency of CT requests was not shown to differ significantly across day of the week for the Inpatient Thrombolysis (p-value = 0.082), Inpatient Immediate (p-value = 0.061), A&E Thrombolysis (p-value = 0.074) and A&E Routine Stroke (p-value = 0.077) groups. For each of the remaining categories, an observed p-value < 0.05 evidenced that frequency of CT scan requests placed did in fact differ significantly according to day of the week. The following weekday groupings were proposed:
• A&E Today, Inpatient Today and Inpatient Routine Stroke: Monday–Friday and Saturday–Sunday.
Inpatient Routine Stroke Inpatient Immediate Inpatient Today
A&E Routine Stroke A&E Immediate A&E Today
• A&E Immediate: Monday–Thursday and Friday–Sunday. • Inpatient Tomorrow: Monday–Thursday, Friday, and Saturday–Sunday. A summary of the results conducted for each request group and for each weekday partition is presented in Table 2. Note that for stroke patients from A&E time of request was considered as admission to A&E, not the actual request for a CT scan as the request time for this patient category was unavailable. 3. Development of a patient flow model The CT scanning system can be conceptualised by a series of activities, queues and waiting lists. Patients arrive into the system at the request for a CT scan. Then patients either join a physical queue outside of the CT unit or are placed on a waiting list. Following the administration of a CT scan, medical images are uploaded electronically onto the database, to be reported by a team of radiologists. An DES model was produced in SIMUL8 and the CT flow diagram is presented in Fig. 1. The first aspect of the system considered was the arrival of patients, initiated by either the request for a CT scan or admission into A&E. Recall that CT scan requests fall into 10 categories. Nine of these categories relate to unscheduled requests from inpatient wards and A&E and for all the 31 h groups described in Table 2 a negative exponential inter-arrival distribution was found as the best fit. The AIC (Akaike Information Criterion) goodness-of-fit test was used to select the most appropriate probability density functions and estimated parameters are presented in Table 2. Time check code was applied within the model to update interarrival distributional parameters according to hour of the day and day of the week. The arrivals of outpatients with pre-scheduled CT scans were modelled by incorporating a front end scheduling system directly within the model. The most appropriate service time distributions are presented in Table 3. A fundamental aspect of the system is the time stroke patients spend in A&E, prior to a CT scan request. In direct consideration of the CT scanning targets for stroke, this aspect of care was addressed separately for patients assessed for thrombolysis and those considered to be routine stroke. A priority system based on existing protocols at the RGH was integrated into the model to characterise queueing conduct and the behaviour upon various waiting lists. Patients considered for thrombolysis treatment are given highest priority for CT scanning, closely followed by those with immediate requests. Patients with pre-scheduled appointments take the next priority level. Capacity constraints within the Radiology department necessitate that
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
35
Table 2 Results from CART analysis and estimated mean inter-arrival times for negative exponential distributions. Urgency
Origin
Day-of-week group
Hour group
Mean inter-arrival time (mins)
Fri–Sun
00:00–23:00
428.70
Mon–Thurs
00:00–8:00 9:00–23:00
3582.86 481.69
Sun–Sat
10:00–16:00 17:00–20:00 21:00–9:00
496.92 1012.72 3697.40
Sat–Sun
8:00–17:00 18:00–7:00
600 100000
Mon–Fri
8:00–14:00 15:00–7:00
192.49 629.36
Sat–Sun
8:00–17:00 18:00–7:00
448.92 100000
Mon–Fri
8:00–14:00 15:00–7:00
40.65 1279.90
Sun–Sat
8:00–17:00 18:00–7:00
2151.72 100000
Mon–Thurs
9:00–16:00 17:00–8:00
49.94 2134.47
Fri
9:00–16:00 17:00–8:00
80.65 3220.65
A&E
Sun–Sat
9:00–17:00 18:00–8:00
3551.35 10770.49
Inpatient
Sun–Sat
9:00–11:00 12:00–8:00
2154.10 14835.48
A&E
Sun–Sat
0:00–8:00 9:00–16:00 17:00–23:00
32850 1990.91 10950
Sat–Sun
9:00–16:00 17:00–8:00
818.36 8681.74
Mon–Fri
9:00–16:00 17:00–8:00
224.72 5113.47
A&E Immediate Inpatient
A&E Today Inpatient
Tomorrow
Inpatient
Thrombolysis
Routine Stroke Inpatient
Fig. 1. CT flow diagram.
these patients are imaged ahead of those on hospital wards or in A&E. Due to pressures of meeting 4 h A&E targets, A&E patients with non-urgent requests take priority ahead of those on hospital wards. It was important to acknowledge that scan urgency changes over time. For instance, ‘scan tomorrow’ patients must be considered as ‘scan today’ cases the next morning. Clinical guideline durations for CT scanning propose a segregation of CT procedures into 6 groups; each allocated an estimate time frame ranging from 10 min to 120 min. The CT head scan delivered to stroke patients defines a 10 min procedure. For each remaining request category the proportion of CT scan requests
are presented in Table 4. For patients with pre-scheduled CT scan requests, procedure requested was determined according to the booking matrix of advanced appointments. Time to administer a CT scan is not recorded directly in the data set, the most appropriate distributions listed in Table 3 were chosen after discussion with radiography staff and manual data collection. For some CT procedures preparation in the form of liquid, which patients are asked to drink 3 h before their CT scan is administered to enhance the quality of medical images. The proportion of requests for which preparation would be required was calculated
36
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39 Table 3 Service time distributions. Request category
Activity
PDF
A&E Thrombolysis A&E Routine Stroke All except for outpatient All except for outpatient All except for outpatient All except for outpatient All except for outpatient All except for outpatient Outpatient Outpatient Outpatient Outpatient Outpatient Outpatient All (if required) All Thrombolysis All Thrombolysis All Thrombolysis—OoH All Routine Stroke All Routine Stroke All Routine Stroke
A&E A&E CT 10 mins procedure CT 15 mins procedure CT 20 mins procedure CT 30 mins procedure CT 40 mins procedure CT 120 mins procedure CT 10 mins procedure CT 15 mins procedure CT 20 mins procedure CT 30 mins procedure CT 40 mins procedure CT 120 mins procedure CT preparation CT reporting—interactive CT reporting—batch CT reporting—manual CT reporting—interactive CT reporting—batch CT reporting—manual
Neg. Exp. Weibull Normal Normal Normal Normal Normal Uniform Triangular Triangular Triangular Triangular Triangular Uniform Fixed Shifted Neg. Exp. Triangular Shifted Neg. Exp. Pearson5 Pearson5 Pearson5
Parameters 138.48
α = 1.1622, β = 252.11 µ = 10, σ = 1.665 µ = 15, σ = 1.665 µ = 20, σ = 2.665 µ = 30, σ = 2.665 µ = 40, σ = 2.665 LB = 30, UB = 150 LB = 5, Mode = 10, UB = 13 LB = 10, Mode = 15, UB = 18 LB = 15, Mode = 20, UB = 23 LB = 25, Mode = 30, UB = 33 LB = 35, Mode = 40, UB = 43 LB = 30, UB = 150 180
µ = 2.571, shift = 10.597 LB = 11, Mode = 11, UB = 186.41 µ = 37.58, shift = 9 α = 2.1439, β = 60.431 α = 1.3914, β = 49.024 α = 2.841, β = 154.48
Table 4 Proportion of CT scan requests by procedure group. Request category
10 min
15 min
20 min
30 min
40 min
120 min
A&E Immediate A&E Today Inpatient Immediate Inpatient Today Inpatient Tomorrow
0.93 0.78 0.69 0.44 0.20
0 0 0 0 0
0.07 0.22 0.31 0.56 0.77
0 0 0 0 0.01
0 0 0 0 0.01
0 0 0 0 0.01
Table 5 Radiographer shift patterns. Day
Time
Details
Monday–Friday
8:00–17:59
In-hours radiography team conducts in-hours CT scanning.
Monday–Friday
00:00–7:59 18:00–23:59
On-call radiographer administers urgent non-CT head scans. A&E radiographers administer urgent CT head scans.
Saturday–Sunday
9:00–16:59
On-call radiographer administers all non-CT head scans which must be delivered within the day. A&E radiographers administer all CT head scans which must be delivered within the day.
Saturday–Sunday
00:00–8:59 17:00–23:59
On-call radiographer administers urgent non-CT head scans. A&E radiographers administer urgent CT head scans.
and then applied within the model. The 3 h time delay following administration of the contrast medium was modelled using a fixed preparation service time of 180 min. Data contained within the RadIS II set described only CT reporting aspects for stroke patients. During normal working hours two methods of reporting are available: interactive and batch. Interactive reporting entails the use of voice recognition to input, validate and finalise a report directly. Batch reporting involves the production of a voice recording of a report to be typed up by a radiology secretary. Reporting method varies according to the preference of a consultant radiologist. The proportion of reports produced for thrombolysis were 75.6% interactive and 24.4% batch and for routine stroke 57.4% interactive and 42.6% batch. The data demonstrated shortest average reporting time using the interactive method in both cases. The A&E radiography team deliver all CT head scans outside of normal working hours. In any other case, the on-call radiographer is utilised. It is important to differentiate between practices employed throughout normal working hours and on-call. Delays as a result of a transportation requirement for on-call radiographers are commonly experienced. Moreover, time to administer a CT procedure can be significantly longer outside of working hours. A paperbased record of on-call CT activity over the 3 month period from
November 2012 to February 2013 was analysed to obtain times of procedures undertaken by the A&E radiography team and on-call radiographers. Also, delays outside of normal working hours were investigated and the results used in the model. Shift patterns were applied directly within the model to drive behaviour of the system according to hour of the day and day of the week. This allowed workforce availability and patient routing to be updated according to in-hours and out-of-hours practices. Workforce details are given in Tables 5 and 6. 4. Model validation and verification There are various procedures and techniques available to test model credibility (Sargent [20]). To represent that the CT scanners can be used at all times the simulation clock was set-up to run for 24 h a day, 7 days a week and a warm-up period of one week was incorporated. A time frame of one year was considered sensible for analysis. The simulation was run for increasing numbers of iterations until the confidence intervals (CI) around a selected key output variable were contained within a 5% level of precision (Hoad et al. [21]). It was found that a steady state is achieved at around 200 iterations.
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
37
Table 6 Radiologist shift patterns. Day
Time
Details
Monday–Friday
9:00–16:59
In-hours consultant radiologists report CT scans.
00:00–8:59 17:00–23:59 00:00–23:59
On-call consultant radiologists report CT scans.
Monday–Friday Saturday–Sunday
Table 7 Comparison of model output with real system data—patient weekly arrivals. Data
Model
A&E
Thrombolysis Routine Stroke Immediate Today
1.65 2.07 17.97 20.82
1.66 2.12 18.09 21.21
Inpatient
Thrombolysis Routine Stroke Immediate Today Tomorrow
1.16 12.91 8.96 58.14 46.90
1.12 12.90 9.06 59.21 47.88
Table 8 Comparison of model output with real system data—percentage compliance with guidelines.
Thrombolysis Routine Stroke
Data
Model
Mean
Mean
95% CI
85.87% 90.7%
85.5% 91.93%
[84.87%, 86.13%] [91.78%, 92.07%]
Verification is the process of ensuring that a simulation model performs in the way that it is intended (Law and Kelton [22]). That is, that the proposed conceptual model has been transformed into a computer model with accuracy. A preliminary method of verification was achieved through static testing, whereby each individual aspect of the model was examined in turn. The code utilised to drive behaviour of the model was examined closely it was confirmed that the process undertaken is accurate and is behaving as expected. Validation is the process of ensuring that a simulation model is an accurate representation of the system under study (Law and Kelton [22]). Detailed discussion with radiology staff and observation within the CT unit confirmed that the conceptual model accurately reflects real-life practice. To validate the model, comparisons between model output and real-system data was produced. It was confirmed that annual arrival output from the simulation model is comparable to patient request frequencies observed in practice. Comparative values for the average weekly patient volumes are presented in Table 7. The average model output does not differ from real-system data at a 5% level of significance. A fundamental measure of interest is the percentage of stroke patients passing through the system within guideline target time frames, comparative values are presented in Table 8. For patients assessed for thrombolysis treatment, the measure given refers to the percentage of patients who spent less than 4.5 h in the system. For routine stroke, the percentage of patients passing through the system within 24 h is shown. The output was not calculated to differ significantly for patients considered for thrombolysis treatment. In the case of routine stroke, the model was shown to slightly overestimate the percentage compliance measure. The results of this comparison however suggested that for the purpose of this project, the overall time measure is modelled sufficiently. Additional validation tests were undertaken in relation to other various subsections of the model such as the overall percentage
of patients requiring preparation and the proportion of requests reported using each of the two methods. In each case the output was not shown to differ significantly. To quantify the uncertainty of our outputs, a sensitivity analysis of compliance using a series of distributions for CT reporting phase has been performed. The sensitivity analyses have minimal effect on outcomes. The model had been successfully verified and validated and could be used to investigate a number of changes to operational practice. 5. Model investigation The primary aim of this paper was to investigate a range of operational changes to estimate the effect on various aspects of the system and therefore evaluate the likely benefit, or otherwise, in relation to specific targets for stroke. Various operational modifications to the CT scanning process were explored through a series of ‘what if..?’ scenario analyses. Each run compares a baseline against a what if scenario using the same random number generator seed value. The first aspect considered was patient priority for CT scanning. In the first scenario all routine stroke patients were seen for a CT head scan on the day it is requested. Moreover, routine stroke patients were re-prioritised further, ahead of patients with non-urgent, unscheduled CT scan requests. The second aspect considered was increased capacity in the form of extended working hours of the CT unit. The final phase of analysis was conducted with regard to reporting CT scans. The key outputs of interest were the overall time in the system for stroke patients and the percentage compliance under guideline time frames. 5.1. What if 1: Re-prioritising routine stroke patients The new twelve hour target necessitated a revision of CT scanning protocols for routine stroke patients from hospital wards. The model was adapted to force the condition that all stroke patients are seen for a CT head scan on the day it is requested (Scenario 1). A further re-prioritisation of routine stroke patients was additionally considered, where routine stroke patients were given priority ahead of any non-urgent, unscheduled ‘scan today’ patients (Scenario 1a). Incorporating CT scanning on the day of request for all routine stroke patients, the model estimated reduction of 6 h and 18 min in average time in the system. As expected, this was mainly a result of a decrease in the time measure for inpatients, however small but significant reductions were also projected for those from A&E. The current estimate of compliance with revised 12 h guidelines (63.13%, CI: [62.66%, 63.60%]) was increased by over 32% under the initial revision of scanning protocols. When CT scanning for routine stroke patients was given priority ahead of that for any other non-urgent unscheduled case, the model predicted a further reduction of 1 h 40 min in average time in the system, from that already achieved under same-day CT scanning. This revision of priority sourced a notable compliance estimate of 97.95%, CI: [97.85%, 98.06%] as shown in Table 9. In addition to the effect of modified CT scanning rules on stroke patients, any consequence upon other aspects of the system were considered. Firstly, no significant difference in the expected waiting time for an urgent or pre-scheduled CT scan was estimated
38
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
Table 9 Comparison of model output with real system data for What if 1. Baseline
What if Scenario 1
Time in the system for stroke patients % compliance with 12 h guideline
Scenario 2
Mean
95% CI
Mean
95% CI
Mean
95% CI
623.13 63.13%
[617.65, 628.62] [62.66%, 63.60%]
245.89 95.87%
[244.06, 247.71] [95.74%, 96.01%]
144.12 97.95%
[142.84, 145.41] [97.85%, 98.06%]
under either of the two modifications. Moreover, no notable increased demand on the out of hours service was estimated in either case. The observed average weekly distribution of procedure frequencies administered by each domain was obtained. Small increases in demand imparted upon the A&E radiography team were not statistically significant. 5.2. What if 2: Extended working hours Discussion with radiography staff revealed that an extension to working hours of the CT unit was one option being considered for application. Specifically, two ways of extending the present working day by 2 h were proposed for further analysis—an early shift at the beginning of the current practice to define revised normal working hours of 6 am–6 pm, and a late shift at the end of the existing day from 8 am to 8 pm. 5.2.1. What if 2a: Comparison of shift patterns The model was run to investigate the scale of effect on various aspects of the system under modified CT scanning rules established in Scenario 1a. The model predicted a statistically significant reduction in average time in the system for routine stroke and thrombolysis patients with application of each of the extended shifts. The average time in the system for thrombolysis patients was reduced by 1.71 min and 1.87 min and for routine stroke patients by 2.58 min and 9.47 min for an early and a late shift correspondingly compared with the results established for Scenario 1a. The results of this analysis did not however provide a conclusive argument surrounding the most efficient option for increased hours. Whereas shorter average throughput time was predicted with the late shift, shorter maximum expected time in the system was projected with the earlier alternative. Moreover, estimated compliance with 12 h stroke guidelines was higher for the late shift, but did not differ significantly between the two options. No notable benefit was predicted for patients assessed for thrombolysis treatment. It was important to evaluate the effect of the extended shifts upon various other aspects of the system. Most fundamentally, waiting time for patients with urgent and pre-scheduled requests was a key measure. The model predicted statistically significant shorter average waiting times for an urgent CT procedure with either extension. The most notable result from this analysis projected that the introduction of a dedicated 6 am–8 am service would in fact have an adverse effect upon waiting time for prescheduled patients. Demand for out of hours services was also carefully measured for each scenario. As expected, significant reductions in requirement for both on-call radiographers and the A&E team were projected with extensions to the working day. Output from the model determined that a late shift would make the most efficient use of two additional hours. Incorporating dedicated CT scanning from 8 am to 8 pm, the model estimated reduced demand for each of the out of hours services by over 3 call outs per week. 5.2.2. What if 2b: Increased outpatient appointments It was suggested that an extension to the current working day would be likely joined with the requirement to deliver more
outpatient CT scanning. The revised models were therefore adapted to consider increasing numbers (from 0 to 27) of additional 10 min outpatient appointments booked in during extended hours. The results did not show statistically significant impact upon waiting time for urgent requests under each increase in outpatient appointment allocations. For patients with pre-scheduled appointments, significantly longer maximum expected delays prior to CT delivery were projected under application of the early shift. The expected frequency of CT procedures required out of hours with the varying degrees of additional outpatient appointments booked during extensions to the current day were analysed. The model estimated that the incorporation of a late shift to deliver dedicated CT scanning from 8 am–8 pm would make the most efficient use of extended in-hours time. With the addition of this shift, under each increase in outpatient appointments scheduled, a smaller frequency of out of hours CT delivery was predicted. Moreover, the model projected that up to 21 additional outpatient appointments could be scheduled between the hours of 6 pm–8 pm each day, without increasing demand on out of hours services. 5.3. What If 3: Modified reporting protocols 5.3.1. What If 3a: Routine stroke reports marked urgent Currently, routine stroke reports are not considered with great urgency. The model was therefore adapted to investigate the effect of marking them with higher priority. Initially, this was investigated by associating routine stroke reports with the same level of urgency as thrombolysis cases. The distribution of reporting time applied for routine stroke patients was modified to represent that of patients assessed for thrombolysis treatment, and the model rerun under both current and revised CT scanning rules (Scenario 1a). The model estimated a significant reduction of 21 min in average throughput time for routine stroke patients under current CT scanning rules. When the initial CT scanning of routine stroke patients was given higher priority (under conditions of Scenario 1a), a similar reduction in average throughput time was projected. Each of these results was calculated as statistically significant. The compliance with guidelines for routine stroke patients were increased, but no statistically significant impact was however predicted. 5.3.2. What If 3b: Reduced time to report Reductions in the time between delivery of a CT scan and the finalisation of its report of 10%–50% were investigated. The percentage reduction in time in the system estimated by the model, against the percentage reduction in reporting time applied were calculated. The most notable result of this analysis established that the estimated relative reduction in mean time in the system for routine stroke patients was significantly greater under the revised scanning rules of Scenario 1a. This output therefore suggested that efforts in making reductions in the delay to finalise a report would be put to most efficient use under an initial modification of initial scanning rules, to deliver a CT head scan to all routine stroke patients on the day that it is requested, and in addition to give such patients priority ahead of those with non-urgent, unscheduled CT requests.
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
5.3.3. What If 3c: Dedicated in-hours reporting method Recall that reporting is undertaken via two alternative methods during normal working hours: interactive and batch. The probability of in-hours reporting method was an input variable derived from historical CT data, estimated for patients assessed for thrombolysis treatment and those considered routine stroke separately. The effect of adapting such parameters was therefore investigated during analysis of the CT reporting process. This was initially undertaken by evaluating the effect of a dedicated in-hours reporting method. The model was modified to force the condition of 100% interactive reporting and 100% batch reporting in turn. The model estimated slightly increased time in the system under the condition of 100% batch reporting. A more variable set of results was produced in evaluation of a dedicated use of the interactive method. For thrombolysis patients, no significant difference in the overall estimated time measure was calculated. An average reduction of over 11 min in overall throughput time of routine stroke patients was however shown to be statistically significant. Overall, under the deliberation of a dedicated in-hours reporting method, interactive reporting would be the most efficient choice. 6. Conclusions The main objective of this paper was to explore how the Royal Gwent Hospital can comply with revised CT scanning guidelines for stroke. A DES model describing the CT scanning system was designed and then used to investigate a number of operational modifications. The project commissioners felt that the project not only satisfied the original aims but explored a number of other areas. As a result of this work some changes have already been implemented, such as improvement in the reporting phase, other implementations are still under discussion. The model provided evidence of a number of strategies to support progression in performance towards revised CT scanning guideline targets for stroke. Recommendations based on this analysis ultimately depend upon the degree of priority that can be given to routine stroke patients. Clearly, the considerable benefit of ensuring delivery of a CT head scan to all stroke patients on the day of its request identifies this as an efficient initial scheme for application. A further revision of priority for such patients ahead of non-urgent, unscheduled cases is also recommended to achieve a greater improvement against the revised guideline targets. In light of an extension to the current working day, the application of a late shift to deliver 8 am–8 pm dedicated CT scanning is endorsed as a resourceful approach. Finally, if reductions in delays to finalise routine stroke reports are believed to be achievable in practice, a revision of CT scanning priority to place all stroke patients ahead of those with non-urgent, unscheduled requests is recommended as an accompanying policy. A number of assumptions were made; for example, CT scanners do not break down or require maintenance, there are no travel delays of patients to the CT room, time to administer a CT scan is independent of radiographer, patients with pre-scheduled CT scans always attend their appointment and on time. The main limitation of this work relates to the reporting phase of CT scans. As the data available described only CT reporting aspects of radiologist activity, it was not possible to model reporting as a dedicated service. For this reason, reporting was considered for stroke patients alone. A further limitation is identified with regard to service time in A&E. In reality, it is intuitive that various time dependencies may be related to delays experienced in A&E. This characteristic was not investigated further. Finally, factors such as the way in which a patient arrives at the CT unit (e.g. bed, wheelchair, foot), age of a patient and experience of a radiographer were not incorporated directly within the model.
39
Acknowledgements We would like to acknowledge the contribution of the Radiology and Radiography clinical staff for their time, advice, for providing the principal data sets and for sharing their invaluable knowledge and providing us with an array of resources required to complete this work. References [1] S. Davis, K. Lees, G. Donnan, Treating the acute stroke patient as an emergency: current practices and future opportunities, Int. J. Clin. Pract. 60 (4) (2006) 399–407. [2] N. Townsend, K. Wickramasinge, P. Bhatnagar, K. Smolina, M. Nichols, J. Leal, R. Luengo-Fernandez, M. Rayner, Coronary Heart Disease Statistics: 2012 Edition, 2012. [3] E.C. Jauch, J.L. Saver, H.P. Adams, A. Bruno, J.J.B. Connors, B.M. Demaerschalk, P. Khatri, P.W. McMullan, A.I. Qureshi, K. Rosenfield, P.A. Scott, D.R. Summers, D.Z. Wang, M. Wintermark, H. Yonas, Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association, Stroke 44 (3) (2013) 870–947. [4] Royal College of Physicians. Intercollegiate Stroke Working Party. National clinical guideline for stroke: fourth ed., 2012. [5] G.W.L. Boland, Stakeholder expectations for radiologists: obstacles or opportunities? J. Am. Coll. Radiol. 3 (3) (2006) 156–163. [6] J. Patrick, M.L. Puterman, Improving resource utilization for diagnostic services through flexible inpatient scheduling: A method for improving resource utilization, J. Oper. Res. Soc. 58 (2007) 235–245. [7] I.B. Vermeulen, S.M. Bohte, S.G. Elkhuizen, H. Lameris, P.J.M. Bakker, H. La Poutré, Adaptive resource allocation for efficient patient scheduling, Artif. Intell. Med. 46 (1) (2009) 67–80. [8] W.A.M. van Lent, J.W. Deetman, H.J. Teertstra, S.H. Muller, E.W. Hans, W.H. van Harten, Reducing the throughput time of the diagnostic track involving CT scanning with computer simulation, Eur. J. Radiol. 81 (11) (2012) 3131–3140. [9] Steffen Bayer, Christina Petsoulas, Benita Cox, Alasdair Honeyman, James Barlow, Facilitating stroke care planning through simulation modelling, Health Inform. J. 16 (2010) 129–143. [10] Leonid Churilov, Geoffrey a. Donnan, Operations Research for stroke care systems: An opportunity for The Science of Better to do much better, Oper. Res. Health Care 1 (1) (2012) 6–15. [11] Leonid Churilov, Audur Fridriksdottir, Mahsa Keshtkaran, Ian Mosley, Andrew Flitman, Helen M. Dewey, Decision support in pre-hospital stroke care operations: A case of using simulation to improve eligibility of acute stroke patients for thrombolysis treatment, Comput. Oper. Res. 40 (9) (2013) 2208–2218. [12] Claire Cordeaux, Andrew Hughes, Mark Elder, Simulating the impact of change: Implementing best practice in stroke care, Lond. J. Prim. Care (2011) 33–37. [13] Maarten M.H. Lahr, Durk-Jouke van der Zee, Gert-jan Luijckx, Patrick C.a.J. Vroomen, Erik Buskens, A simulation-based approach for improving utilization of thrombolysis in acute brain infarction, Med. Care 51 (2013) 1101–1105. [14] Maarten M.H. Lahr, Durk Jouke Van Der Zee, Patrick C.a.J. Vroomen, Gert Jan Luijckx, Erik Buskens, Thrombolysis in acute ischemic stroke: A simulation study to improve pre- and in-hospital delays in community hospitals, PLoS ONE 8 (11) (2013) 1–6. [15] Javier Mar, Arantzazu Arrospide, Mercè Comas, Budget impact analysis of thrombolysis for stroke in Spain: a discrete event simulation model, Value Health 13 (1) (2010) 69–76. [16] Thomas Monks, Martin Pitt, Ken Stein, Martin James, Maximizing the population benefit from thrombolysis in acute ischemic stroke: A modeling study of in-hospital delays, Stroke 43 (2012) 2706–2711. [17] Thomas Monks, Martin Pitt, Ken Stein, Martin a. James, Hyperacute stroke care and NHS England’s business. 3049 (May 2014), 2015, pp. 10–11. [18] Martin Pitt, Thomas Monks, Paritosh Agarwal, David Worthington, Gary a. Ford, Kennedy R. Lees, Ken Stein, Martin a. James, Will delays in treatment jeopardize the population benefit from extending the time window for stroke thrombolysis? Stroke 43 (2012) 2992–2997. [19] J. Emberson, K.R. Lees, P. Lyden, L. Blackwell, G. Albers, E. Bluhmki, T. Brott, G. Cohen, S. Davis, G. Donnan, J. Grotta, G. Howard, M. Kaste, M. Koga, R. von Kummer, M. Lansberg, R.I. Lindley, G. Murray, J.M. Olivot, M. Parsons, B. Tilley, D. Toni, K. Toyoda, N. Wahlgren, J. Wardlaw, W. Whiteley, G.J. Del Zoppo, C. Baigent, P. Sandercock, W. Hacke, Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials, Lancet 384 (2014) 4–10. [20] R.G. Sargent, Verification and validation of simulation models, in: Proceedings of the 2010 Winter Simulation Conference B. Johansson, S. Jain, J. MontoyaTorres, J. Hugan, and E. Yücesan (Eds.), 2010, pp. 166–183. [21] K. Hoad, S. Robinson, R. Davies, M. Elder, Automating DES output analysis: How many replications to run, 2008. [22] A.M. Law, W.D. Kelton, Simulation Modeling and Analysis, second ed., McGraw-Hill Inc., New York, 1991.
Contents lists available at ScienceDirect
Operations Research for Health Care journal homepage: www.elsevier.com/locate/orhc
Compliance with national guidelines for stroke in radiology Izabela Komenda ∗ , Vincent Knight, Hannah Mary Williams School of Mathematics, Cardiff University, Cardiff, Wales, United Kingdom
article
info
Article history: Received 19 June 2014 Accepted 6 September 2015 Available online 14 September 2015 Keywords: Simulation modelling Stroke Radiology
abstract Stroke is a medical emergency, and if patient outcomes are to be optimised there should be no delays in accessing treatment. This project focuses on the application of Operational Research methodology to investigate how a hospital can comply with the revised computerised tomography (CT) scanning guidelines for stroke. Such guidelines, released by the Royal College of Physicians recommend a 50% reduction in time from hospital admission to report of a CT head scan to just 12 hours. The results of statistical analyses of historical data were used to populate a discrete event simulation model of patient flow through the CT scanning unit. The model was then used to explore a number of operational modifications to the CT scanning system through a series of scenario analyses. The results of this investigation presented evidence of a number of strategies to support operational improvements in relation to revised stroke guidelines. Crown Copyright © 2015 Published by Elsevier Ltd. All rights reserved.
1. Introduction Stroke is a serious medical illness with a mortality rate higher than most forms of cancer (Davis et al. [1]). Each year, approximately 15 million people worldwide and 152,000 people in the United Kingdom suffer a stroke (Townsend et al. [2]). Recent publications from the British Heart Foundation report that during 2010, stroke was the fourth largest cause of mortality in the UK (Townsend et al. [2]). If outcomes for stroke patients are to be optimised, accessing treatment should not be delayed (Jauch et al. [3]). Previously, guidelines provided by the Royal College of Physicians (RCP) have suggested that a computerised tomography (CT) brain-imaging scan has to be delivered within a maximum of 24 h of hospital admission for all patients with symptoms of stroke. However, the latest clinical guidelines released in December 2012 (RCP [4]) have recommended a 50% reduction in the aforementioned time window to just 12 h. The main aim of this paper is to investigate the effect on various aspects of the system and therefore evaluate the likely benefit, or otherwise, in relation to specific targets for stroke. This has been achieved by incorporating various modifications to a simulation model that models current CT scanning activity. The key outputs of interest were the overall time in the system for stroke patients and the percentage compliance under guideline time frames. The model predicted that by necessitating that all routine stroke patients are seen for a CT head scan
∗
Corresponding author. E-mail address: [email protected] (I. Komenda).
http://dx.doi.org/10.1016/j.orhc.2015.09.001 2211-6923/Crown Copyright © 2015 Published by Elsevier Ltd. All rights reserved.
on the day it is requested, an improvement in compliance with revised 12 h stroke guidelines could be achieved. Moreover, additional benefits were projected when routine stroke patients were re-prioritised further, ahead of patients with non-urgent, unscheduled CT scan requests. This project was conducted within the Radiology Department of the Royal Gwent Hospital, Newport, Aneurin Bevan University Health Board (ABUHB). ABUHB is one of the seven local health boards in Wales which serves an estimated population of over 639,000, approximately 21% of the total Welsh population. The existing Operational Research (OR) literature surrounding planning for CT scanning processes focuses mainly on scheduling (Boland [5]; Patrick and Puterman [6]; Vermeulen et al. [7]; Van Lent et al. [8]). Discrete event simulation is employed as the most appropriate modelling technique for the identified decision support problem. Several studies have been conducted where use of a simulation model facilitated improvement in the overall acute stroke pathway (Bayer et al. [9], Churilov and Donnan [10], Churilov et al. [11], Cordeaux et al. [12], Lahr et al. [13,14], Mar et al. [15], Monks et al. [16,17] and Pitt et al. [18]); however there is a distinct lack of research concerning the specific use of discrete event simulation in the CT scanning environment. The main contribution of this paper to the existing literature is a casestudy applied to the CT unit. It is hoped that this area will greatly benefit from this application. The paper is organised as follows. In Section 2 statistical analysis of CT request data is described. In Section 3 the model is introduced, which is then validated and verified in Section 4. In Section 5 results from ‘what if...?’ scenarios are discussed before summarising the findings in Section 6.
34
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
Table 1 CT scan request categories. Patient origin
RGH Ward
Pathology
Urgency of scan
Description
Request category
Thrombolysis
Immediate
Inpatient Thrombolysis
Routine Stroke Not Stroke Not Stroke
By the end of the day following the request Immediate By the end of the day the request is made By the end of the day following the request
Ward patients who qualify for an assessment of their suitability for thrombolysis treatment Ward patients with suspected stroke for which thrombolysis treatment is known not to be appropriate Ward patients who require a CT scan immediately Ward patients who require a CT scan on the day of request Ward patients who require a CT scan by the end of the day following its request
Inpatient Tomorrow
A&E patients who qualify for assessment of their suitability for thrombolysis treatment A&E patients with suspected stroke for which thrombolysis treatment is known not to be appropriate A&E patients who require a CT scan immediately A&E patients who require a CT scan on the day of request
A&E Thrombolysis
GP referrals with pre-scheduled appointments
Outpatient
Not Stroke
A&E
Outpatient
Thrombolysis
Immediate
Routine Stroke Not Stroke Not Stroke
By the end of the day the request is made Immediate By the end of the day the request is made Must be seen on day of appointment
Not Stroke
2. Statistical analysis of CT request data The data set was extracted from the radiology information system, RadIS II, which contains a fairly complete record of CT activity at the RGH. An additional data set contains all A&E activity. Both data sets relate to the same two-year time frame, from the 1st June 2011 to 31st May 2013. CT scans are requested at different levels of urgency for a number of different patient types. This information is summarised in Table 1. A total of 1,869 CT scan requests were stroke related and from these, 295 necessary to assess whether thrombolysis treatment was appropriate. Thrombolysis is a treatment to dissolve blood clots by pharmacological means. It works by injecting clot-busting drug into blood vessels. This mode of treatment is only suitable for victims of ischaemic stroke and must be delivered within four and a half hours of symptom onset (Emberson et al. [19]). Monthly, weekly and hourly time-dependency analysis was completed in an attempt to establish patterns and trends in the frequency of unscheduled CT scans requested with respect to time. Kruskal Wallis analysis was applied to investigate whether the frequency of CT scan requests differed significantly across month of the year and day of the week. Pairwise Wilcoxon testing has been applied to group patient categories by weekday. Classification and Regression Tree (CART) analysis was applied to partition a full data set into smaller, homogeneous subsets for each weekday grouping and for each request category. For each request according to month of the year, a p-value is reported: A&E Thrombolysis (p-value = 0.7570), A&E Routine Stroke (p-value = 0.5201), A&E Immediate (p-value = 0.999), A&E Today (p-value = 0.722), Inpatient Thrombolysis (p-value = 0.366), Inpatient Routine Stroke (p-value = 0.955), Inpatient Immediate (p-value = 0.966), Inpatient Today (p-value = 0.463) and Inpatient Tomorrow (p-value = 0.853). P-value greater than 0.05 for each category suggests no seasonal trends. The frequency of CT requests was not shown to differ significantly across day of the week for the Inpatient Thrombolysis (p-value = 0.082), Inpatient Immediate (p-value = 0.061), A&E Thrombolysis (p-value = 0.074) and A&E Routine Stroke (p-value = 0.077) groups. For each of the remaining categories, an observed p-value < 0.05 evidenced that frequency of CT scan requests placed did in fact differ significantly according to day of the week. The following weekday groupings were proposed:
• A&E Today, Inpatient Today and Inpatient Routine Stroke: Monday–Friday and Saturday–Sunday.
Inpatient Routine Stroke Inpatient Immediate Inpatient Today
A&E Routine Stroke A&E Immediate A&E Today
• A&E Immediate: Monday–Thursday and Friday–Sunday. • Inpatient Tomorrow: Monday–Thursday, Friday, and Saturday–Sunday. A summary of the results conducted for each request group and for each weekday partition is presented in Table 2. Note that for stroke patients from A&E time of request was considered as admission to A&E, not the actual request for a CT scan as the request time for this patient category was unavailable. 3. Development of a patient flow model The CT scanning system can be conceptualised by a series of activities, queues and waiting lists. Patients arrive into the system at the request for a CT scan. Then patients either join a physical queue outside of the CT unit or are placed on a waiting list. Following the administration of a CT scan, medical images are uploaded electronically onto the database, to be reported by a team of radiologists. An DES model was produced in SIMUL8 and the CT flow diagram is presented in Fig. 1. The first aspect of the system considered was the arrival of patients, initiated by either the request for a CT scan or admission into A&E. Recall that CT scan requests fall into 10 categories. Nine of these categories relate to unscheduled requests from inpatient wards and A&E and for all the 31 h groups described in Table 2 a negative exponential inter-arrival distribution was found as the best fit. The AIC (Akaike Information Criterion) goodness-of-fit test was used to select the most appropriate probability density functions and estimated parameters are presented in Table 2. Time check code was applied within the model to update interarrival distributional parameters according to hour of the day and day of the week. The arrivals of outpatients with pre-scheduled CT scans were modelled by incorporating a front end scheduling system directly within the model. The most appropriate service time distributions are presented in Table 3. A fundamental aspect of the system is the time stroke patients spend in A&E, prior to a CT scan request. In direct consideration of the CT scanning targets for stroke, this aspect of care was addressed separately for patients assessed for thrombolysis and those considered to be routine stroke. A priority system based on existing protocols at the RGH was integrated into the model to characterise queueing conduct and the behaviour upon various waiting lists. Patients considered for thrombolysis treatment are given highest priority for CT scanning, closely followed by those with immediate requests. Patients with pre-scheduled appointments take the next priority level. Capacity constraints within the Radiology department necessitate that
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
35
Table 2 Results from CART analysis and estimated mean inter-arrival times for negative exponential distributions. Urgency
Origin
Day-of-week group
Hour group
Mean inter-arrival time (mins)
Fri–Sun
00:00–23:00
428.70
Mon–Thurs
00:00–8:00 9:00–23:00
3582.86 481.69
Sun–Sat
10:00–16:00 17:00–20:00 21:00–9:00
496.92 1012.72 3697.40
Sat–Sun
8:00–17:00 18:00–7:00
600 100000
Mon–Fri
8:00–14:00 15:00–7:00
192.49 629.36
Sat–Sun
8:00–17:00 18:00–7:00
448.92 100000
Mon–Fri
8:00–14:00 15:00–7:00
40.65 1279.90
Sun–Sat
8:00–17:00 18:00–7:00
2151.72 100000
Mon–Thurs
9:00–16:00 17:00–8:00
49.94 2134.47
Fri
9:00–16:00 17:00–8:00
80.65 3220.65
A&E
Sun–Sat
9:00–17:00 18:00–8:00
3551.35 10770.49
Inpatient
Sun–Sat
9:00–11:00 12:00–8:00
2154.10 14835.48
A&E
Sun–Sat
0:00–8:00 9:00–16:00 17:00–23:00
32850 1990.91 10950
Sat–Sun
9:00–16:00 17:00–8:00
818.36 8681.74
Mon–Fri
9:00–16:00 17:00–8:00
224.72 5113.47
A&E Immediate Inpatient
A&E Today Inpatient
Tomorrow
Inpatient
Thrombolysis
Routine Stroke Inpatient
Fig. 1. CT flow diagram.
these patients are imaged ahead of those on hospital wards or in A&E. Due to pressures of meeting 4 h A&E targets, A&E patients with non-urgent requests take priority ahead of those on hospital wards. It was important to acknowledge that scan urgency changes over time. For instance, ‘scan tomorrow’ patients must be considered as ‘scan today’ cases the next morning. Clinical guideline durations for CT scanning propose a segregation of CT procedures into 6 groups; each allocated an estimate time frame ranging from 10 min to 120 min. The CT head scan delivered to stroke patients defines a 10 min procedure. For each remaining request category the proportion of CT scan requests
are presented in Table 4. For patients with pre-scheduled CT scan requests, procedure requested was determined according to the booking matrix of advanced appointments. Time to administer a CT scan is not recorded directly in the data set, the most appropriate distributions listed in Table 3 were chosen after discussion with radiography staff and manual data collection. For some CT procedures preparation in the form of liquid, which patients are asked to drink 3 h before their CT scan is administered to enhance the quality of medical images. The proportion of requests for which preparation would be required was calculated
36
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39 Table 3 Service time distributions. Request category
Activity
A&E Thrombolysis A&E Routine Stroke All except for outpatient All except for outpatient All except for outpatient All except for outpatient All except for outpatient All except for outpatient Outpatient Outpatient Outpatient Outpatient Outpatient Outpatient All (if required) All Thrombolysis All Thrombolysis All Thrombolysis—OoH All Routine Stroke All Routine Stroke All Routine Stroke
A&E A&E CT 10 mins procedure CT 15 mins procedure CT 20 mins procedure CT 30 mins procedure CT 40 mins procedure CT 120 mins procedure CT 10 mins procedure CT 15 mins procedure CT 20 mins procedure CT 30 mins procedure CT 40 mins procedure CT 120 mins procedure CT preparation CT reporting—interactive CT reporting—batch CT reporting—manual CT reporting—interactive CT reporting—batch CT reporting—manual
Neg. Exp. Weibull Normal Normal Normal Normal Normal Uniform Triangular Triangular Triangular Triangular Triangular Uniform Fixed Shifted Neg. Exp. Triangular Shifted Neg. Exp. Pearson5 Pearson5 Pearson5
Parameters 138.48
α = 1.1622, β = 252.11 µ = 10, σ = 1.665 µ = 15, σ = 1.665 µ = 20, σ = 2.665 µ = 30, σ = 2.665 µ = 40, σ = 2.665 LB = 30, UB = 150 LB = 5, Mode = 10, UB = 13 LB = 10, Mode = 15, UB = 18 LB = 15, Mode = 20, UB = 23 LB = 25, Mode = 30, UB = 33 LB = 35, Mode = 40, UB = 43 LB = 30, UB = 150 180
µ = 2.571, shift = 10.597 LB = 11, Mode = 11, UB = 186.41 µ = 37.58, shift = 9 α = 2.1439, β = 60.431 α = 1.3914, β = 49.024 α = 2.841, β = 154.48
Table 4 Proportion of CT scan requests by procedure group. Request category
10 min
15 min
20 min
30 min
40 min
120 min
A&E Immediate A&E Today Inpatient Immediate Inpatient Today Inpatient Tomorrow
0.93 0.78 0.69 0.44 0.20
0 0 0 0 0
0.07 0.22 0.31 0.56 0.77
0 0 0 0 0.01
0 0 0 0 0.01
0 0 0 0 0.01
Table 5 Radiographer shift patterns. Day
Time
Details
Monday–Friday
8:00–17:59
In-hours radiography team conducts in-hours CT scanning.
Monday–Friday
00:00–7:59 18:00–23:59
On-call radiographer administers urgent non-CT head scans. A&E radiographers administer urgent CT head scans.
Saturday–Sunday
9:00–16:59
On-call radiographer administers all non-CT head scans which must be delivered within the day. A&E radiographers administer all CT head scans which must be delivered within the day.
Saturday–Sunday
00:00–8:59 17:00–23:59
On-call radiographer administers urgent non-CT head scans. A&E radiographers administer urgent CT head scans.
and then applied within the model. The 3 h time delay following administration of the contrast medium was modelled using a fixed preparation service time of 180 min. Data contained within the RadIS II set described only CT reporting aspects for stroke patients. During normal working hours two methods of reporting are available: interactive and batch. Interactive reporting entails the use of voice recognition to input, validate and finalise a report directly. Batch reporting involves the production of a voice recording of a report to be typed up by a radiology secretary. Reporting method varies according to the preference of a consultant radiologist. The proportion of reports produced for thrombolysis were 75.6% interactive and 24.4% batch and for routine stroke 57.4% interactive and 42.6% batch. The data demonstrated shortest average reporting time using the interactive method in both cases. The A&E radiography team deliver all CT head scans outside of normal working hours. In any other case, the on-call radiographer is utilised. It is important to differentiate between practices employed throughout normal working hours and on-call. Delays as a result of a transportation requirement for on-call radiographers are commonly experienced. Moreover, time to administer a CT procedure can be significantly longer outside of working hours. A paperbased record of on-call CT activity over the 3 month period from
November 2012 to February 2013 was analysed to obtain times of procedures undertaken by the A&E radiography team and on-call radiographers. Also, delays outside of normal working hours were investigated and the results used in the model. Shift patterns were applied directly within the model to drive behaviour of the system according to hour of the day and day of the week. This allowed workforce availability and patient routing to be updated according to in-hours and out-of-hours practices. Workforce details are given in Tables 5 and 6. 4. Model validation and verification There are various procedures and techniques available to test model credibility (Sargent [20]). To represent that the CT scanners can be used at all times the simulation clock was set-up to run for 24 h a day, 7 days a week and a warm-up period of one week was incorporated. A time frame of one year was considered sensible for analysis. The simulation was run for increasing numbers of iterations until the confidence intervals (CI) around a selected key output variable were contained within a 5% level of precision (Hoad et al. [21]). It was found that a steady state is achieved at around 200 iterations.
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
37
Table 6 Radiologist shift patterns. Day
Time
Details
Monday–Friday
9:00–16:59
In-hours consultant radiologists report CT scans.
00:00–8:59 17:00–23:59 00:00–23:59
On-call consultant radiologists report CT scans.
Monday–Friday Saturday–Sunday
Table 7 Comparison of model output with real system data—patient weekly arrivals. Data
Model
A&E
Thrombolysis Routine Stroke Immediate Today
1.65 2.07 17.97 20.82
1.66 2.12 18.09 21.21
Inpatient
Thrombolysis Routine Stroke Immediate Today Tomorrow
1.16 12.91 8.96 58.14 46.90
1.12 12.90 9.06 59.21 47.88
Table 8 Comparison of model output with real system data—percentage compliance with guidelines.
Thrombolysis Routine Stroke
Data
Model
Mean
Mean
95% CI
85.87% 90.7%
85.5% 91.93%
[84.87%, 86.13%] [91.78%, 92.07%]
Verification is the process of ensuring that a simulation model performs in the way that it is intended (Law and Kelton [22]). That is, that the proposed conceptual model has been transformed into a computer model with accuracy. A preliminary method of verification was achieved through static testing, whereby each individual aspect of the model was examined in turn. The code utilised to drive behaviour of the model was examined closely it was confirmed that the process undertaken is accurate and is behaving as expected. Validation is the process of ensuring that a simulation model is an accurate representation of the system under study (Law and Kelton [22]). Detailed discussion with radiology staff and observation within the CT unit confirmed that the conceptual model accurately reflects real-life practice. To validate the model, comparisons between model output and real-system data was produced. It was confirmed that annual arrival output from the simulation model is comparable to patient request frequencies observed in practice. Comparative values for the average weekly patient volumes are presented in Table 7. The average model output does not differ from real-system data at a 5% level of significance. A fundamental measure of interest is the percentage of stroke patients passing through the system within guideline target time frames, comparative values are presented in Table 8. For patients assessed for thrombolysis treatment, the measure given refers to the percentage of patients who spent less than 4.5 h in the system. For routine stroke, the percentage of patients passing through the system within 24 h is shown. The output was not calculated to differ significantly for patients considered for thrombolysis treatment. In the case of routine stroke, the model was shown to slightly overestimate the percentage compliance measure. The results of this comparison however suggested that for the purpose of this project, the overall time measure is modelled sufficiently. Additional validation tests were undertaken in relation to other various subsections of the model such as the overall percentage
of patients requiring preparation and the proportion of requests reported using each of the two methods. In each case the output was not shown to differ significantly. To quantify the uncertainty of our outputs, a sensitivity analysis of compliance using a series of distributions for CT reporting phase has been performed. The sensitivity analyses have minimal effect on outcomes. The model had been successfully verified and validated and could be used to investigate a number of changes to operational practice. 5. Model investigation The primary aim of this paper was to investigate a range of operational changes to estimate the effect on various aspects of the system and therefore evaluate the likely benefit, or otherwise, in relation to specific targets for stroke. Various operational modifications to the CT scanning process were explored through a series of ‘what if..?’ scenario analyses. Each run compares a baseline against a what if scenario using the same random number generator seed value. The first aspect considered was patient priority for CT scanning. In the first scenario all routine stroke patients were seen for a CT head scan on the day it is requested. Moreover, routine stroke patients were re-prioritised further, ahead of patients with non-urgent, unscheduled CT scan requests. The second aspect considered was increased capacity in the form of extended working hours of the CT unit. The final phase of analysis was conducted with regard to reporting CT scans. The key outputs of interest were the overall time in the system for stroke patients and the percentage compliance under guideline time frames. 5.1. What if 1: Re-prioritising routine stroke patients The new twelve hour target necessitated a revision of CT scanning protocols for routine stroke patients from hospital wards. The model was adapted to force the condition that all stroke patients are seen for a CT head scan on the day it is requested (Scenario 1). A further re-prioritisation of routine stroke patients was additionally considered, where routine stroke patients were given priority ahead of any non-urgent, unscheduled ‘scan today’ patients (Scenario 1a). Incorporating CT scanning on the day of request for all routine stroke patients, the model estimated reduction of 6 h and 18 min in average time in the system. As expected, this was mainly a result of a decrease in the time measure for inpatients, however small but significant reductions were also projected for those from A&E. The current estimate of compliance with revised 12 h guidelines (63.13%, CI: [62.66%, 63.60%]) was increased by over 32% under the initial revision of scanning protocols. When CT scanning for routine stroke patients was given priority ahead of that for any other non-urgent unscheduled case, the model predicted a further reduction of 1 h 40 min in average time in the system, from that already achieved under same-day CT scanning. This revision of priority sourced a notable compliance estimate of 97.95%, CI: [97.85%, 98.06%] as shown in Table 9. In addition to the effect of modified CT scanning rules on stroke patients, any consequence upon other aspects of the system were considered. Firstly, no significant difference in the expected waiting time for an urgent or pre-scheduled CT scan was estimated
38
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
Table 9 Comparison of model output with real system data for What if 1. Baseline
What if Scenario 1
Time in the system for stroke patients % compliance with 12 h guideline
Scenario 2
Mean
95% CI
Mean
95% CI
Mean
95% CI
623.13 63.13%
[617.65, 628.62] [62.66%, 63.60%]
245.89 95.87%
[244.06, 247.71] [95.74%, 96.01%]
144.12 97.95%
[142.84, 145.41] [97.85%, 98.06%]
under either of the two modifications. Moreover, no notable increased demand on the out of hours service was estimated in either case. The observed average weekly distribution of procedure frequencies administered by each domain was obtained. Small increases in demand imparted upon the A&E radiography team were not statistically significant. 5.2. What if 2: Extended working hours Discussion with radiography staff revealed that an extension to working hours of the CT unit was one option being considered for application. Specifically, two ways of extending the present working day by 2 h were proposed for further analysis—an early shift at the beginning of the current practice to define revised normal working hours of 6 am–6 pm, and a late shift at the end of the existing day from 8 am to 8 pm. 5.2.1. What if 2a: Comparison of shift patterns The model was run to investigate the scale of effect on various aspects of the system under modified CT scanning rules established in Scenario 1a. The model predicted a statistically significant reduction in average time in the system for routine stroke and thrombolysis patients with application of each of the extended shifts. The average time in the system for thrombolysis patients was reduced by 1.71 min and 1.87 min and for routine stroke patients by 2.58 min and 9.47 min for an early and a late shift correspondingly compared with the results established for Scenario 1a. The results of this analysis did not however provide a conclusive argument surrounding the most efficient option for increased hours. Whereas shorter average throughput time was predicted with the late shift, shorter maximum expected time in the system was projected with the earlier alternative. Moreover, estimated compliance with 12 h stroke guidelines was higher for the late shift, but did not differ significantly between the two options. No notable benefit was predicted for patients assessed for thrombolysis treatment. It was important to evaluate the effect of the extended shifts upon various other aspects of the system. Most fundamentally, waiting time for patients with urgent and pre-scheduled requests was a key measure. The model predicted statistically significant shorter average waiting times for an urgent CT procedure with either extension. The most notable result from this analysis projected that the introduction of a dedicated 6 am–8 am service would in fact have an adverse effect upon waiting time for prescheduled patients. Demand for out of hours services was also carefully measured for each scenario. As expected, significant reductions in requirement for both on-call radiographers and the A&E team were projected with extensions to the working day. Output from the model determined that a late shift would make the most efficient use of two additional hours. Incorporating dedicated CT scanning from 8 am to 8 pm, the model estimated reduced demand for each of the out of hours services by over 3 call outs per week. 5.2.2. What if 2b: Increased outpatient appointments It was suggested that an extension to the current working day would be likely joined with the requirement to deliver more
outpatient CT scanning. The revised models were therefore adapted to consider increasing numbers (from 0 to 27) of additional 10 min outpatient appointments booked in during extended hours. The results did not show statistically significant impact upon waiting time for urgent requests under each increase in outpatient appointment allocations. For patients with pre-scheduled appointments, significantly longer maximum expected delays prior to CT delivery were projected under application of the early shift. The expected frequency of CT procedures required out of hours with the varying degrees of additional outpatient appointments booked during extensions to the current day were analysed. The model estimated that the incorporation of a late shift to deliver dedicated CT scanning from 8 am–8 pm would make the most efficient use of extended in-hours time. With the addition of this shift, under each increase in outpatient appointments scheduled, a smaller frequency of out of hours CT delivery was predicted. Moreover, the model projected that up to 21 additional outpatient appointments could be scheduled between the hours of 6 pm–8 pm each day, without increasing demand on out of hours services. 5.3. What If 3: Modified reporting protocols 5.3.1. What If 3a: Routine stroke reports marked urgent Currently, routine stroke reports are not considered with great urgency. The model was therefore adapted to investigate the effect of marking them with higher priority. Initially, this was investigated by associating routine stroke reports with the same level of urgency as thrombolysis cases. The distribution of reporting time applied for routine stroke patients was modified to represent that of patients assessed for thrombolysis treatment, and the model rerun under both current and revised CT scanning rules (Scenario 1a). The model estimated a significant reduction of 21 min in average throughput time for routine stroke patients under current CT scanning rules. When the initial CT scanning of routine stroke patients was given higher priority (under conditions of Scenario 1a), a similar reduction in average throughput time was projected. Each of these results was calculated as statistically significant. The compliance with guidelines for routine stroke patients were increased, but no statistically significant impact was however predicted. 5.3.2. What If 3b: Reduced time to report Reductions in the time between delivery of a CT scan and the finalisation of its report of 10%–50% were investigated. The percentage reduction in time in the system estimated by the model, against the percentage reduction in reporting time applied were calculated. The most notable result of this analysis established that the estimated relative reduction in mean time in the system for routine stroke patients was significantly greater under the revised scanning rules of Scenario 1a. This output therefore suggested that efforts in making reductions in the delay to finalise a report would be put to most efficient use under an initial modification of initial scanning rules, to deliver a CT head scan to all routine stroke patients on the day that it is requested, and in addition to give such patients priority ahead of those with non-urgent, unscheduled CT requests.
I. Komenda et al. / Operations Research for Health Care 6 (2015) 33–39
5.3.3. What If 3c: Dedicated in-hours reporting method Recall that reporting is undertaken via two alternative methods during normal working hours: interactive and batch. The probability of in-hours reporting method was an input variable derived from historical CT data, estimated for patients assessed for thrombolysis treatment and those considered routine stroke separately. The effect of adapting such parameters was therefore investigated during analysis of the CT reporting process. This was initially undertaken by evaluating the effect of a dedicated in-hours reporting method. The model was modified to force the condition of 100% interactive reporting and 100% batch reporting in turn. The model estimated slightly increased time in the system under the condition of 100% batch reporting. A more variable set of results was produced in evaluation of a dedicated use of the interactive method. For thrombolysis patients, no significant difference in the overall estimated time measure was calculated. An average reduction of over 11 min in overall throughput time of routine stroke patients was however shown to be statistically significant. Overall, under the deliberation of a dedicated in-hours reporting method, interactive reporting would be the most efficient choice. 6. Conclusions The main objective of this paper was to explore how the Royal Gwent Hospital can comply with revised CT scanning guidelines for stroke. A DES model describing the CT scanning system was designed and then used to investigate a number of operational modifications. The project commissioners felt that the project not only satisfied the original aims but explored a number of other areas. As a result of this work some changes have already been implemented, such as improvement in the reporting phase, other implementations are still under discussion. The model provided evidence of a number of strategies to support progression in performance towards revised CT scanning guideline targets for stroke. Recommendations based on this analysis ultimately depend upon the degree of priority that can be given to routine stroke patients. Clearly, the considerable benefit of ensuring delivery of a CT head scan to all stroke patients on the day of its request identifies this as an efficient initial scheme for application. A further revision of priority for such patients ahead of non-urgent, unscheduled cases is also recommended to achieve a greater improvement against the revised guideline targets. In light of an extension to the current working day, the application of a late shift to deliver 8 am–8 pm dedicated CT scanning is endorsed as a resourceful approach. Finally, if reductions in delays to finalise routine stroke reports are believed to be achievable in practice, a revision of CT scanning priority to place all stroke patients ahead of those with non-urgent, unscheduled requests is recommended as an accompanying policy. A number of assumptions were made; for example, CT scanners do not break down or require maintenance, there are no travel delays of patients to the CT room, time to administer a CT scan is independent of radiographer, patients with pre-scheduled CT scans always attend their appointment and on time. The main limitation of this work relates to the reporting phase of CT scans. As the data available described only CT reporting aspects of radiologist activity, it was not possible to model reporting as a dedicated service. For this reason, reporting was considered for stroke patients alone. A further limitation is identified with regard to service time in A&E. In reality, it is intuitive that various time dependencies may be related to delays experienced in A&E. This characteristic was not investigated further. Finally, factors such as the way in which a patient arrives at the CT unit (e.g. bed, wheelchair, foot), age of a patient and experience of a radiographer were not incorporated directly within the model.
39
Acknowledgements We would like to acknowledge the contribution of the Radiology and Radiography clinical staff for their time, advice, for providing the principal data sets and for sharing their invaluable knowledge and providing us with an array of resources required to complete this work. References [1] S. Davis, K. Lees, G. Donnan, Treating the acute stroke patient as an emergency: current practices and future opportunities, Int. J. Clin. Pract. 60 (4) (2006) 399–407. [2] N. Townsend, K. Wickramasinge, P. Bhatnagar, K. Smolina, M. Nichols, J. Leal, R. Luengo-Fernandez, M. Rayner, Coronary Heart Disease Statistics: 2012 Edition, 2012. [3] E.C. Jauch, J.L. Saver, H.P. Adams, A. Bruno, J.J.B. Connors, B.M. Demaerschalk, P. Khatri, P.W. McMullan, A.I. Qureshi, K. Rosenfield, P.A. Scott, D.R. Summers, D.Z. Wang, M. Wintermark, H. Yonas, Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association, Stroke 44 (3) (2013) 870–947. [4] Royal College of Physicians. Intercollegiate Stroke Working Party. National clinical guideline for stroke: fourth ed., 2012. [5] G.W.L. Boland, Stakeholder expectations for radiologists: obstacles or opportunities? J. Am. Coll. Radiol. 3 (3) (2006) 156–163. [6] J. Patrick, M.L. Puterman, Improving resource utilization for diagnostic services through flexible inpatient scheduling: A method for improving resource utilization, J. Oper. Res. Soc. 58 (2007) 235–245. [7] I.B. Vermeulen, S.M. Bohte, S.G. Elkhuizen, H. Lameris, P.J.M. Bakker, H. La Poutré, Adaptive resource allocation for efficient patient scheduling, Artif. Intell. Med. 46 (1) (2009) 67–80. [8] W.A.M. van Lent, J.W. Deetman, H.J. Teertstra, S.H. Muller, E.W. Hans, W.H. van Harten, Reducing the throughput time of the diagnostic track involving CT scanning with computer simulation, Eur. J. Radiol. 81 (11) (2012) 3131–3140. [9] Steffen Bayer, Christina Petsoulas, Benita Cox, Alasdair Honeyman, James Barlow, Facilitating stroke care planning through simulation modelling, Health Inform. J. 16 (2010) 129–143. [10] Leonid Churilov, Geoffrey a. Donnan, Operations Research for stroke care systems: An opportunity for The Science of Better to do much better, Oper. Res. Health Care 1 (1) (2012) 6–15. [11] Leonid Churilov, Audur Fridriksdottir, Mahsa Keshtkaran, Ian Mosley, Andrew Flitman, Helen M. Dewey, Decision support in pre-hospital stroke care operations: A case of using simulation to improve eligibility of acute stroke patients for thrombolysis treatment, Comput. Oper. Res. 40 (9) (2013) 2208–2218. [12] Claire Cordeaux, Andrew Hughes, Mark Elder, Simulating the impact of change: Implementing best practice in stroke care, Lond. J. Prim. Care (2011) 33–37. [13] Maarten M.H. Lahr, Durk-Jouke van der Zee, Gert-jan Luijckx, Patrick C.a.J. Vroomen, Erik Buskens, A simulation-based approach for improving utilization of thrombolysis in acute brain infarction, Med. Care 51 (2013) 1101–1105. [14] Maarten M.H. Lahr, Durk Jouke Van Der Zee, Patrick C.a.J. Vroomen, Gert Jan Luijckx, Erik Buskens, Thrombolysis in acute ischemic stroke: A simulation study to improve pre- and in-hospital delays in community hospitals, PLoS ONE 8 (11) (2013) 1–6. [15] Javier Mar, Arantzazu Arrospide, Mercè Comas, Budget impact analysis of thrombolysis for stroke in Spain: a discrete event simulation model, Value Health 13 (1) (2010) 69–76. [16] Thomas Monks, Martin Pitt, Ken Stein, Martin James, Maximizing the population benefit from thrombolysis in acute ischemic stroke: A modeling study of in-hospital delays, Stroke 43 (2012) 2706–2711. [17] Thomas Monks, Martin Pitt, Ken Stein, Martin a. James, Hyperacute stroke care and NHS England’s business. 3049 (May 2014), 2015, pp. 10–11. [18] Martin Pitt, Thomas Monks, Paritosh Agarwal, David Worthington, Gary a. Ford, Kennedy R. Lees, Ken Stein, Martin a. James, Will delays in treatment jeopardize the population benefit from extending the time window for stroke thrombolysis? Stroke 43 (2012) 2992–2997. [19] J. Emberson, K.R. Lees, P. Lyden, L. Blackwell, G. Albers, E. Bluhmki, T. Brott, G. Cohen, S. Davis, G. Donnan, J. Grotta, G. Howard, M. Kaste, M. Koga, R. von Kummer, M. Lansberg, R.I. Lindley, G. Murray, J.M. Olivot, M. Parsons, B. Tilley, D. Toni, K. Toyoda, N. Wahlgren, J. Wardlaw, W. Whiteley, G.J. Del Zoppo, C. Baigent, P. Sandercock, W. Hacke, Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials, Lancet 384 (2014) 4–10. [20] R.G. Sargent, Verification and validation of simulation models, in: Proceedings of the 2010 Winter Simulation Conference B. Johansson, S. Jain, J. MontoyaTorres, J. Hugan, and E. Yücesan (Eds.), 2010, pp. 166–183. [21] K. Hoad, S. Robinson, R. Davies, M. Elder, Automating DES output analysis: How many replications to run, 2008. [22] A.M. Law, W.D. Kelton, Simulation Modeling and Analysis, second ed., McGraw-Hill Inc., New York, 1991.