- Email: [email protected]
The Use of the Australian Basic Treatment Equivalent (BTE) Workload Measure for Linear Accelerators in Canada
Clinical Oncology (2001)13:8–13 # The Royal College of Radiologists
Clinical Oncology
Original Article The Use of the Australian Basic Treatment Equivalent (BTE) Workload Measure for Linear Accelerators in Canada P. Craighead1, C. Herring1, C. Hillier2, D. Guo1, J. Budden3 and K. Rans1 1
Tom Baker Cancer Centre, Calgary, Alberta; 2Cape Breton Cancer Centre Sydney, Nova Scotia and Northeastern Ontario Cancer Centre, Sudbury, Ontario, Canada.
3
Abstract. The Inter Society of Radiation Oncologists of North America (ISRON) workload standard for linear accelerators is the one most widely used; it regards the treatment of 250 or more patients per year as an acceptable limit. Nevertheless, there is concern that this standard does not represent the current workload of linear accelerators, given that the complexity of techniques and equipment has increased significantly since the ISRON model was developed in the late 1980s. Delaney et al. recently validated a workload indicator for Australian (AUS) centres, known as the basic treatment equivalent (BTE). They showed that this was a better predictor of workload and that there was less variation between centres using this model than there would have been by using fields/hour. This centre attempted to validate this model for use in a Canadian centre, by collecting treatment data on all linear accelerator-treated patients during February 1998. The linear accelerators at this centre delivered 2295 fractions (6928 fields) in 662 hours during February 1998. When 15 minutes was used as a denominator, the BTE model functioned as a better workload indicator than simple measures such as fields/hour. It also had better performance in reducing variability between machines. A BTE of 3403 was calculated for these machines. The mean value for fields/hour, BTE/hour and BTE/fraction for this centre fell within the range of values quoted by AUS centres. The BTE/fraction value for this centre was relatively high compared with the AUS mean, indicating this centre’s reliance on the use of a high number of complex techniques. We recommend that the model should be further refined for the Canadian context by developing BTE values with the use of local time and motion studies, including factors such as multileaf collimators and enhanced dynamic wedges. Keywords: Linear accelerator; Productivity; Radiation oncology; Workload
Introduction Simple guidelines for the expected productivity of radiotherapy machines have been developed by several national bodies, the most widely used of which was conceived by the Inter Society of Radiation Oncologists of North America (ISRON). This algorithm is used widely by various funding agencies, including many Canadian provincial health departments, as a guide for staffing or the purchase of equipment. It utilizes numbers of patients per treatment machine (250 per unit annually) as an index of workload [1].
Correspondence and offprint requests to: Dr P. Craighead, Director of Radiation Oncology, Tom Baker Cancer Centre, 1331 29th St NW, Calgary, AB, Canada T2N 4N2. E-mail: [email protected]
Radiotherapy services have been a point of stress in many government-funded health care systems throughout the 1990s. Publications from Germany, Australia, Sweden, the United Kingdom, Canada and The Netherlands have shown that radiotherapy services are chronically underfunded, and that plans for the future are insufficient for the expected demand [2–9]. More recently, funding concerns also have surfaced in the USA, where managed care by health maintenance organizations (HMOs) is having an impact on the clinical practice of radiation Oncology [10,11]. This suggests that the models being used to fund radiotherapy services are poor predictors of actual workload, or that governments/HMOs do not fund appropriately, despite accurate models. Canadian radiotherapy services have had excessively long waiting times for treatment in the 1990s [12]. Despite the increased complexity of radiotherapy over this time, with more time needed to treat
a large proportion of the patients, Canadian provincial cancer agencies continue to use patients per machine as their measure of radiotherapy workload. Unless a workload indicator that accommodates complexity can be developed in a timely manner, it is felt that centres attempting to offer a wider range of complex treatment techniques will be underfunded for staffing and technical resources [13]. In 1991 ISRON proposed a new model, which attempted to take the complexity of techniques into account, known as the equivalent simple treatment visit (ESTV) [9]. This model proposed that treatment techniques should be weighted according to complexity into simple, intermediate and complex categories, with relative values of 1.0, 1.1 and 1.25 respectively. An Australian (AUS) ESTV study carried out by Delaney et al. showed that this model could not be applied in their centres because the intermediate techniques for AUS centres were measured as significantly longer than complex techniques within the ISRON ESTV model [14]. This raises the question of whether the ESTV model applies equally to countries outside the USA. Delaney et al. have proposed a new AUS measure of linear accelerator workload, called the basic treatment equivalent (BTE) [15]. This was developed by using a time and motion study to determine the time required for all treatment techniques and scoring them all relative to a baseline ‘simple’ technique by using a statistical regression model. The baseline simple technique was treatment with two simple fields, measured to be within a 10-minute time slot. This baseline was scored as one BTE unit, with more complex treatments being given higher values according to the increase in average time beyond 10 minutes taken to treat patients (e.g. if a technique took 30 minutes to execute, this would imply a BTE of 3). The model accommodates technical factors such as the number of fields, and the use of beam shielding or portal films, as well as patient factors such as first visit, the administration of an anaesthetic, and performance status [15]. Delaney et al. reasoned that, as a basic treatment BTE was equivalent to two simple fields and also equivalent to 10 minutes, then a direct comparison between these two could be made by comparing BTE per time unit and field per time unit, with 10 minutes being equivalent to 1 time unit. This was based on the AUS federal guideline of regarding any two fields treated in 10 minutes as one unit [16]. They hypothesized that whichever of these two measures (BTE or two fields) per time unit was closer to 1 should be regarded as the better indicator. Several AUS centres proceeded to show that BTE/time performed better as an indicator, with reduced variability between centres compared with fields 2/ time [16], with the conclusion that BTE/time was more likely to take measures of complexity into account for AUS centres.
The Department of Radiation Oncology at the Tom Baker Cancer Centre (TBCC) is a tertiary centre, which treats approximately 2400 patients annually. In early 1998 the Department operated five linear accelerators, two cobalt units, a single orthovoltage unit and two simulators. As in most North American facilities, the scheduling of patients is generally applied by using 15-minute aliquots, with an average of 7.5 hours/day being available for treatment. This centre uses a staffing model of three radiation therapists per therapy unit in order to have two qualified people present at any time for set-up and treatment. We performed this study to evaluate the BTE model for this Canadian centre, surmising that the regulated slot times for AUS (10 min) and TBCC treatments (15 min) are both unable properly to accommodate the true time requirements for all patients. The BTE model, which integrates several factors with the number of fields, was felt to have a better chance of representing radiotherapy workload measurement than the numbers of fields, even though the AUS model is based on scheduling with 10minute aliquots.
Materials and Methods The information necessary to calculate fields/hour, total BTE, BTE/hour, BTE/fraction and patients/hour was collected on all patient fractions treated for a 4week period in this department, during February 1998 (Tables 1 and 2). We used the data for the five linear accelerators for the scope of this study. Two radiation therapists were assigned to collect these data from the units every day, and to collate this in the necessary format at the completion of the study. Table 1. TBCC workload: February 1998 Linac unit no.
BTE
Total fields
1 2 3 4 5 Total
627.82 628.17 525.35 788.68 832.96 3402.98
1181 1215 1064 1748 1720 6928
No. patients during period 467 462 440 465 461 2295
Operating time (min) .8700 .8040 .8010 .7500 .7500 39.750
Table 2. TBCC workload: February 1998 Linac unit no.
Fields/ fraction
BTEp (mean)
Fields/ hour
Fields2 (mean)
BTE/ hour
1 2 3 4 5
2.52 2.63 2.41 3.76 3.73
1.28 1.30 1.16 1.62 1.61
8.14 9.07 7.98 14.03 13.63
1.22 1.28 1.20 1.83 1.83
4.33 4.69 3.95 6.33 6.62
Australian BTE Workload Measure for Linear Accelerators in Canada
9
Primary Objectives 1. To assess whether the AUS BTE model is more accurate than fields/hour for this centre 2. To compare indicators (BTE/hour, fields/hour, BTE/fraction) between this and AUS centres 3. To determine whether the BTE model needs modification for use in this centre
Hypothesis This was that linear accelerator workload for this centre would be better measured by using BTE than fields/hour.
Methodology: Comparison of Simple Workload Indicators and BTE for this Centre A comparison was made between two fields per 15 minutes (one unit in Canadian provincial terms) and BTE per 15 minutes (1 BTE at the TBCC) as representative of simple and comprehensive indicators respectively. These units were called BTEp (BTE per 15 minutes) and fields2 (number of fields/2 per 15 minutes) respectively. This study used Delaney’s hypothesis that whichever value (BTEp or fields2) was closest to 1 should be considered the more accurate predictor of workload. These values were calculated by dividing total BTE by total time/15, and total fields/2 divided by total time/15. The squared distances of individual observations from mean BTEp and fields2 values were calculated. The paired t test was used to compare the variance, with 95% confidence intervals. The one-sample Wilcoxon signed rank test was used to test the ability of BTEp or fields2 to be closer to 1. The difference between BTEp and fields2 for total workload was tested for statistical significance by using the paired Wilcoxon signed rank test. The difference between machines in BTEp and fields2 was tested by using the Kruskal– Wallis rank sum test.
Comparison of Workload Indicators in AUS Centres and TBCC We compared the mean values of BTE/hour, BTE/ fraction, and fields/hour for the TBCC with the published results of AUS centres, in order to assess the differences. BTE/fraction, BTE/hour and fields/ hour for the TBCC were compared with the AUS values for descriptive purposes.
The Use of the AUS Model at the TBCC: Does This Need Modification or Not? We postulated that the AUS BTE model could only be considered for use without a local time and motion study to develop our own values if the correlation between the amount of time needed for treatment with increasing BTE was significantly better than that between time and increasing numbers of fields. Spearman’s rank correlation test was used to test the significance of an association between the number of fields, increasing BTE values, and the time needed to treat. A comparison was also made between the actual BTE numbers and those expected by dividing total time by 10 minutes [16]. The actual BTE should compensate for any loss in throughput because of complexity; therefore, hypothetically, the difference between the actual and the predicted BTE should be minimal. It was proposed that modifications should be made to the AUS model if the difference between the actual and the predicted BTE for this centre was greater than 5%.
Results During the 4 weeks of February 1998, the linear accelerators within the Department treated 2295 fractions using 6928 fields over 662 machine hours (Table 1). This translated into an average linear accelerator utilization rate of 7.28 hours/day, and 10.5 fields/hour (Table 2). Although linear accelerator units 1, 2 and 3 each handle a distinct spectrum of patient diagnoses, units 4 and 5 have been used to
Table 3. Analysis of casemix by machine and fields/fraction (n = total no. patients/unit)
10
% fields/fraction
% Unit 1 (n = 467)
% Unit 2 (n = 462)
% Unit 3 (n = 440)
% Unit 4 (n = 465)
% Unit 5 (n = 461)
% Patients
1 field 2 fields 3 fields 4 fields 5 fields 6 fields Total % patients treated
3.74 70.49 7.73 11.48 0 6.56 100 21
0 56.83 25.33 15.86 1.98 0 100 20
1.95 73.59 1.52 19.26 3.68 0 100 19
2.54 8.67 1.95 84.99 0 2.05 100 20
0 14.82 2.82 80.24 0 1.4 100 20
1.84 44.75 7.85 42.5 1.16 1.9 100
P. Craighead et al.
treat mainly pelvic cancers (Table 3). Table 3 also shows that the number of patients treated per machine during that month was equally distributed between the treatment machines, despite the significant differences in the number of fields treated per patient for some units. Units 4 and 5 were mainly used for four-field techniques; the other three units were predominantly used to deliver two fields.
Comparison of BTEp and Fields2 as a Workload Indicator Using 15 Minutes as the Denominator Wilcoxon rank sum tests for BTEp and fields2 showed that they are both significantly greater than 1 (P<0.001). The paired Wilcoxon signed rank test showed that BTEp is significantly closer to 1 than fields2 (P<0.001). There is a significant difference between the various machines for both fields2 and BTEp (P<0.001). This increased predictive ability for BTEp is significant for the combined values of units 4 and 5 (P<0.001), but not for the joint values of units 1, 2 and 3 (P = 0.320). The difference between the squared distances from the mean of fields2 and BTEp was 0.114, which implies less variation for BTEp (P<0.00; paired t test, confidence intervals 0.078 to 0.149). When 10 minutes is substituted in the BTEp and fields2 formulae, fields2 is closer to 1 and therefore a better indicator (P = 0.00, Wilcoxon signed rank test).
Comparison between TBCC and Combined AUS Mean Values
Fig. 1. Productivity: fields/hour (A–J = individual AUS centres).
Fig. 2. Productivity: BTE/hours (A–J = individual AUS centres).
Fields/hour (Fig. 1): This centre’s mean value (10.5 fields/hour) falls within the range for AUS centres (9.2–14.1; median 10.88; mean 11.26). BTE/hour (Fig. 2): The TBCC mean value (5.16 BTE/ hour) falls within the range of the AUS values of 4.7– 6.9 (median 5.25; mean 5.64). BTE/fraction (Fig. 3): This centre’s mean value (1.49 BTE/fraction) falls within the range of the AUS values of 1.2–1.7 (median 1.3; mean 1.37).
Validation of the AUS BTE Model for Use in Canadian Conditions There is a significant correlation between increasing BTE and increased time needed to treat (P<0.001). However, there is no evidence for a statistically significant correlation between increasing numbers of fields and an increase in the time needed to treat (P = 0.198). This suggests that the AUS BTE model may be a more comprehensive indicator in the Canadian setting than fields or patients per machine. However, there was a difference of 16.8% between
Fig. 3. Productivity: BTE/fractions (A–J = individual AUS centres).
the actual (3403) and the predicted (3975) BTE, suggesting that the AUS model could not fully accommodate local workload factors that might influence throughput. Australian BTE Workload Measure for Linear Accelerators in Canada
11
Discussion Statistics for 1997 show that the linear accelerators at this centre treated an average of 245 patients during that year, which the ISRON model would regard as suboptimal because it is less than the 250 recommended per unit per year. With the increasing number of complex techniques, it is considered that these simple indicators probably do not reflect the true workload. Productivity within radiotherapy services is dependent on several factors, including: actual treatment time, transit time for patients to get in and out of the treatment vault, complexity of technique, performance status of patients, scheduled and unscheduled maintenance, training time, and others. Most departments have developed average ‘treatment slots’, which are artificial constructs established to assist patient scheduling and the prediction of workload. These slots are sometimes too short or too long for the treatment being applied. For example, if there are excessive slots without enough time to treat using complex techniques (or poor performance cases), managers end up having to reduce the throughput on that machine to accommodate patients and staff. If the treatment slots are too long for a particular technique, the staff generally use the full time to treat their patients, or they will use the small time gains to accommodate other patients requiring longer treatment times.
Measurement of Workload This present study shows that BTE/hour is a better measure of combined linear accelerator workload than fields2/hour, which can be implied from the better correlation between the time needed to treat and BTE compared with fields2. The ability to show that BTEp using a 15-minute denominator is a better indicator at this centre confirms the potential for this model outside of Australia. This study could be criticized because it used a different denominator to that in the original BTE model, which was 10 minutes. When we used 10 minutes as the denominator, fields2 was a better indicator than BTEp. This reversal of indicator performance is related to the failure of BTEp to function as an indicator for treatment units 1, 2 and 3, which was amplified when using a 10-minute denominator. The latter had enough effect on the combined value to make fields2 a better indicator. Of interest is the finding that BTEp is a better indicator than fields2 in units 4 and 5, regardless of whether 10 or 15 minutes is used as a denominator. The authors propose that this can be explained by these units being able to treat complex techniques using close to 10 minutes per patient, thus making its BTE findings closer to the AUS model. These advantages over fields2 are that significant for these treatment units that they are not reversed by a change 12
P. Craighead et al.
in the denominator. A further conclusion from this finding is that the centre needs to carry out its own time and motion study to be able to modify the factors influencing BTE scores at the TBCC. The increase in fields/hour and BTE/hour for units 4 and 5 would lead us to speculate that multileaf collimation and enhanced dynamic wedges enable a more efficient throughput of complex techniques. This is illustrated by the significantly improved predictive ability of BTEp for units 4 and 5. The increased efficiencies created by these beam accessories indicate the need to include this as a factor in any statistical regression for future BTE models. That there is less variation in BTEp than in fields2 between the different machines might explain the discrepancy between the proportion of patients and the number of fields per patient per machine (Table 3). This is probably due to BTE being able to represent the workload of machines more fairly by including factors such as complexity and performance status, which results in techniques (whether simple, associated with many fields, etc.) being credited for the actual time needed to treat patients.
Comparison of AUS and TBCC Values Fields/hour, BTE/time and BTE/fraction at this centre fell within the range of values for AUS centres (Figs 1–3). These authors do not have access to the core data from AUS centres, so it is difficult to show statistically whether this centre’s values are dissimilar or not. The lower BTE/hour and fields/hour values further support our concern that scheduling practices need to be refined after a time and motion study. The high BTE/fraction values for the TBCC, and the correlation of increased BTE with increased time to treat, imply that the lower actual BTE is probably related to reduced efficiency relative to AUS departments. Preliminary work has already allowed us to reduce scheduling times to 12 minutes for patients with breast and prostate cancers.
Need for Modifications of the AUS Model We suggest that staff at centres outside Australia who wish to use this model should consider performing time and motion studies. The large difference between the actual and the expected BTE for the TBCC confirms the need to modify the BTE model for this centre before it can be of real use to us. The authors hypothesize that a time and motion study will enable both refinement of the scheduling practices (making us more efficient) and modification of the BTE factors. The differences in booking practices between this and AUS centres should be kept in context. The authors believed that it was important to use a pragmatic approach to compare the adequacy of the AUS model for this centre. Booking times represent
ony an average of the time needed to treat all patients; it is therefore the utilization of the full day that is far more critical to any model. We have shown that, although the BTE model is a better predictor than fields2 for the TBCC using 15 minutes, this was reversed when 10 minutes was used. This study indicates that the guidelines for calculating the AUS BTE model are easy to use. Once a refined model has been developed here, this could assist us in the following ways: to motivate the use of a standard indicator of machine productivity in a provincial/ national jurisdiction; to measure the productivity of treatment machines; to assess how to integrate newer technology; and to encourage a reasonable level of funding.
Conclusion As radiation oncology advances over the next decade, it will be critical to develop workload indicators that accommodate factors such as treatment complexity, the performance status of the patients, and newer technologies. This is especially true of universally funded health care systems, where these indicators are used by health departments to provide funding for cancer treatment. This study has shown that the BTE model can be applied to centres outside Australia, but that these jurisdictions would be best advised to modify the values to integrate the local treatment environment. Although it might be tempting to think that the development of an objective workload indicator for radiation oncology productivity has little relevance to private and semi-privately funded systems, this is very far from being the truth [17]. The emergence of a strongly managed care movement (i.e. HMOs) in the USA is expected to threaten the ability of some radiation oncology departments to provide such a wide range of technologies. For HMOs to be reassured that funding is appropriate they will need workload indicators that are objective and accommodate complexity of technique.
References 1. Report of the Inter Society of Radiation Oncologists of North America (ISRON) In: Integrated cancer management. Philadelphia, Pennsylvania: American College of Radiation Oncology, 1991. 2. Van Daal WA, Bos MA. Infrastructure for radiotherapy in the Netherlands: development from 1970 to 2010. Int J Radiat Oncol Biol Phys 1997;37:411–5. 3. Stevens G, Firth I. Clinical audit in radiation oncology: results from one centre. Aust Radiol 1996;40:47–54. 4. Theil HJ, Oeschler WA, Kastura B, et al. Costs–proceeds calculation for radiotherapy: comment. Strahlenther Onkol 1996;172:128–47; discussion 154. 5. Owen JB, Coia LR, Hanks GE. The structure of radiation oncology in the United States in 1994. Int J Radiat Oncol Biol Phys 1997;39:179–85. 6. Richards MA, Parrott JC. Tertiary cancer services in Britain: benchmarking study of activity and facilities at 12 specialist centres. BMJ 1996;313:347–9. 7. Van Daal WA, Bos MA. Infrastructure for radiotherapy in the Netherlands: development from 1970 to 2001. Int J Radiation Oncol Biol and Physics 1997;37(2):411–5. 8. Frodin IE, Jonson E, Moller T, et al. Radiotherapy in Sweden. Acta Oncol 1996;35:967–79. 9. Wade Report. Cancer manpower committee – Projections of equipment and personnel needs for radiotherapy in Ontario. Toronto, Ontario: Ontario Government, 1991. 10. Peters LJ. Through a glass darkly: predicting the future of radiation oncology. Int J Radiat Oncol Biol Phys 1995;31:219–25. 11. Burkhardt JH, Sunshine JH. Utilization of radiologic services in different payment systems and patient populations. Radiology 1996;200:201–7. 12. Fields A. Cancer treatment under pressure: strategies in the face of changing demographics and economics. Can J Oncol 1992;3 Suppl 3. 13. Abraham C, Fine S. Canadian cancer system slips to second best. Globe and Mail (Toronto) 1998 July 25; Sect A1: 8–9. 14. Delaney GP, Gebski V, Rus M, et al. The equivalent simple treatment visit (ESTV) model does not measure radiation oncology productivity under Australian conditions. Clin Oncol 1997;9:245–7. 15. Delaney GP, Gebski V, Lunn AD, et al. Basic treatment equivalent (BTE): a new measure of linear accelerator workload. Clin Oncol 1997;9:234–9. 16. Delaney GP, Gebsku V, Lunn AD, et al. An assessment of the basic treatment equivalent (BTE) model as a measure of radiotherapy workload. Clin Oncol 1997;9:240–4. 17. Lazar G, Desch E. Performance measurements in cancer care. Cancer 1998;82 (8 Suppl): 2016–21.
Received for publication October 1998 Accepted following revision June 2000
Australian BTE Workload Measure for Linear Accelerators in Canada
13
Clinical Oncology
Original Article The Use of the Australian Basic Treatment Equivalent (BTE) Workload Measure for Linear Accelerators in Canada P. Craighead1, C. Herring1, C. Hillier2, D. Guo1, J. Budden3 and K. Rans1 1
Tom Baker Cancer Centre, Calgary, Alberta; 2Cape Breton Cancer Centre Sydney, Nova Scotia and Northeastern Ontario Cancer Centre, Sudbury, Ontario, Canada.
3
Abstract. The Inter Society of Radiation Oncologists of North America (ISRON) workload standard for linear accelerators is the one most widely used; it regards the treatment of 250 or more patients per year as an acceptable limit. Nevertheless, there is concern that this standard does not represent the current workload of linear accelerators, given that the complexity of techniques and equipment has increased significantly since the ISRON model was developed in the late 1980s. Delaney et al. recently validated a workload indicator for Australian (AUS) centres, known as the basic treatment equivalent (BTE). They showed that this was a better predictor of workload and that there was less variation between centres using this model than there would have been by using fields/hour. This centre attempted to validate this model for use in a Canadian centre, by collecting treatment data on all linear accelerator-treated patients during February 1998. The linear accelerators at this centre delivered 2295 fractions (6928 fields) in 662 hours during February 1998. When 15 minutes was used as a denominator, the BTE model functioned as a better workload indicator than simple measures such as fields/hour. It also had better performance in reducing variability between machines. A BTE of 3403 was calculated for these machines. The mean value for fields/hour, BTE/hour and BTE/fraction for this centre fell within the range of values quoted by AUS centres. The BTE/fraction value for this centre was relatively high compared with the AUS mean, indicating this centre’s reliance on the use of a high number of complex techniques. We recommend that the model should be further refined for the Canadian context by developing BTE values with the use of local time and motion studies, including factors such as multileaf collimators and enhanced dynamic wedges. Keywords: Linear accelerator; Productivity; Radiation oncology; Workload
Introduction Simple guidelines for the expected productivity of radiotherapy machines have been developed by several national bodies, the most widely used of which was conceived by the Inter Society of Radiation Oncologists of North America (ISRON). This algorithm is used widely by various funding agencies, including many Canadian provincial health departments, as a guide for staffing or the purchase of equipment. It utilizes numbers of patients per treatment machine (250 per unit annually) as an index of workload [1].
Correspondence and offprint requests to: Dr P. Craighead, Director of Radiation Oncology, Tom Baker Cancer Centre, 1331 29th St NW, Calgary, AB, Canada T2N 4N2. E-mail: [email protected]
Radiotherapy services have been a point of stress in many government-funded health care systems throughout the 1990s. Publications from Germany, Australia, Sweden, the United Kingdom, Canada and The Netherlands have shown that radiotherapy services are chronically underfunded, and that plans for the future are insufficient for the expected demand [2–9]. More recently, funding concerns also have surfaced in the USA, where managed care by health maintenance organizations (HMOs) is having an impact on the clinical practice of radiation Oncology [10,11]. This suggests that the models being used to fund radiotherapy services are poor predictors of actual workload, or that governments/HMOs do not fund appropriately, despite accurate models. Canadian radiotherapy services have had excessively long waiting times for treatment in the 1990s [12]. Despite the increased complexity of radiotherapy over this time, with more time needed to treat
a large proportion of the patients, Canadian provincial cancer agencies continue to use patients per machine as their measure of radiotherapy workload. Unless a workload indicator that accommodates complexity can be developed in a timely manner, it is felt that centres attempting to offer a wider range of complex treatment techniques will be underfunded for staffing and technical resources [13]. In 1991 ISRON proposed a new model, which attempted to take the complexity of techniques into account, known as the equivalent simple treatment visit (ESTV) [9]. This model proposed that treatment techniques should be weighted according to complexity into simple, intermediate and complex categories, with relative values of 1.0, 1.1 and 1.25 respectively. An Australian (AUS) ESTV study carried out by Delaney et al. showed that this model could not be applied in their centres because the intermediate techniques for AUS centres were measured as significantly longer than complex techniques within the ISRON ESTV model [14]. This raises the question of whether the ESTV model applies equally to countries outside the USA. Delaney et al. have proposed a new AUS measure of linear accelerator workload, called the basic treatment equivalent (BTE) [15]. This was developed by using a time and motion study to determine the time required for all treatment techniques and scoring them all relative to a baseline ‘simple’ technique by using a statistical regression model. The baseline simple technique was treatment with two simple fields, measured to be within a 10-minute time slot. This baseline was scored as one BTE unit, with more complex treatments being given higher values according to the increase in average time beyond 10 minutes taken to treat patients (e.g. if a technique took 30 minutes to execute, this would imply a BTE of 3). The model accommodates technical factors such as the number of fields, and the use of beam shielding or portal films, as well as patient factors such as first visit, the administration of an anaesthetic, and performance status [15]. Delaney et al. reasoned that, as a basic treatment BTE was equivalent to two simple fields and also equivalent to 10 minutes, then a direct comparison between these two could be made by comparing BTE per time unit and field per time unit, with 10 minutes being equivalent to 1 time unit. This was based on the AUS federal guideline of regarding any two fields treated in 10 minutes as one unit [16]. They hypothesized that whichever of these two measures (BTE or two fields) per time unit was closer to 1 should be regarded as the better indicator. Several AUS centres proceeded to show that BTE/time performed better as an indicator, with reduced variability between centres compared with fields 2/ time [16], with the conclusion that BTE/time was more likely to take measures of complexity into account for AUS centres.
The Department of Radiation Oncology at the Tom Baker Cancer Centre (TBCC) is a tertiary centre, which treats approximately 2400 patients annually. In early 1998 the Department operated five linear accelerators, two cobalt units, a single orthovoltage unit and two simulators. As in most North American facilities, the scheduling of patients is generally applied by using 15-minute aliquots, with an average of 7.5 hours/day being available for treatment. This centre uses a staffing model of three radiation therapists per therapy unit in order to have two qualified people present at any time for set-up and treatment. We performed this study to evaluate the BTE model for this Canadian centre, surmising that the regulated slot times for AUS (10 min) and TBCC treatments (15 min) are both unable properly to accommodate the true time requirements for all patients. The BTE model, which integrates several factors with the number of fields, was felt to have a better chance of representing radiotherapy workload measurement than the numbers of fields, even though the AUS model is based on scheduling with 10minute aliquots.
Materials and Methods The information necessary to calculate fields/hour, total BTE, BTE/hour, BTE/fraction and patients/hour was collected on all patient fractions treated for a 4week period in this department, during February 1998 (Tables 1 and 2). We used the data for the five linear accelerators for the scope of this study. Two radiation therapists were assigned to collect these data from the units every day, and to collate this in the necessary format at the completion of the study. Table 1. TBCC workload: February 1998 Linac unit no.
BTE
Total fields
1 2 3 4 5 Total
627.82 628.17 525.35 788.68 832.96 3402.98
1181 1215 1064 1748 1720 6928
No. patients during period 467 462 440 465 461 2295
Operating time (min) .8700 .8040 .8010 .7500 .7500 39.750
Table 2. TBCC workload: February 1998 Linac unit no.
Fields/ fraction
BTEp (mean)
Fields/ hour
Fields2 (mean)
BTE/ hour
1 2 3 4 5
2.52 2.63 2.41 3.76 3.73
1.28 1.30 1.16 1.62 1.61
8.14 9.07 7.98 14.03 13.63
1.22 1.28 1.20 1.83 1.83
4.33 4.69 3.95 6.33 6.62
Australian BTE Workload Measure for Linear Accelerators in Canada
9
Primary Objectives 1. To assess whether the AUS BTE model is more accurate than fields/hour for this centre 2. To compare indicators (BTE/hour, fields/hour, BTE/fraction) between this and AUS centres 3. To determine whether the BTE model needs modification for use in this centre
Hypothesis This was that linear accelerator workload for this centre would be better measured by using BTE than fields/hour.
Methodology: Comparison of Simple Workload Indicators and BTE for this Centre A comparison was made between two fields per 15 minutes (one unit in Canadian provincial terms) and BTE per 15 minutes (1 BTE at the TBCC) as representative of simple and comprehensive indicators respectively. These units were called BTEp (BTE per 15 minutes) and fields2 (number of fields/2 per 15 minutes) respectively. This study used Delaney’s hypothesis that whichever value (BTEp or fields2) was closest to 1 should be considered the more accurate predictor of workload. These values were calculated by dividing total BTE by total time/15, and total fields/2 divided by total time/15. The squared distances of individual observations from mean BTEp and fields2 values were calculated. The paired t test was used to compare the variance, with 95% confidence intervals. The one-sample Wilcoxon signed rank test was used to test the ability of BTEp or fields2 to be closer to 1. The difference between BTEp and fields2 for total workload was tested for statistical significance by using the paired Wilcoxon signed rank test. The difference between machines in BTEp and fields2 was tested by using the Kruskal– Wallis rank sum test.
Comparison of Workload Indicators in AUS Centres and TBCC We compared the mean values of BTE/hour, BTE/ fraction, and fields/hour for the TBCC with the published results of AUS centres, in order to assess the differences. BTE/fraction, BTE/hour and fields/ hour for the TBCC were compared with the AUS values for descriptive purposes.
The Use of the AUS Model at the TBCC: Does This Need Modification or Not? We postulated that the AUS BTE model could only be considered for use without a local time and motion study to develop our own values if the correlation between the amount of time needed for treatment with increasing BTE was significantly better than that between time and increasing numbers of fields. Spearman’s rank correlation test was used to test the significance of an association between the number of fields, increasing BTE values, and the time needed to treat. A comparison was also made between the actual BTE numbers and those expected by dividing total time by 10 minutes [16]. The actual BTE should compensate for any loss in throughput because of complexity; therefore, hypothetically, the difference between the actual and the predicted BTE should be minimal. It was proposed that modifications should be made to the AUS model if the difference between the actual and the predicted BTE for this centre was greater than 5%.
Results During the 4 weeks of February 1998, the linear accelerators within the Department treated 2295 fractions using 6928 fields over 662 machine hours (Table 1). This translated into an average linear accelerator utilization rate of 7.28 hours/day, and 10.5 fields/hour (Table 2). Although linear accelerator units 1, 2 and 3 each handle a distinct spectrum of patient diagnoses, units 4 and 5 have been used to
Table 3. Analysis of casemix by machine and fields/fraction (n = total no. patients/unit)
10
% fields/fraction
% Unit 1 (n = 467)
% Unit 2 (n = 462)
% Unit 3 (n = 440)
% Unit 4 (n = 465)
% Unit 5 (n = 461)
% Patients
1 field 2 fields 3 fields 4 fields 5 fields 6 fields Total % patients treated
3.74 70.49 7.73 11.48 0 6.56 100 21
0 56.83 25.33 15.86 1.98 0 100 20
1.95 73.59 1.52 19.26 3.68 0 100 19
2.54 8.67 1.95 84.99 0 2.05 100 20
0 14.82 2.82 80.24 0 1.4 100 20
1.84 44.75 7.85 42.5 1.16 1.9 100
P. Craighead et al.
treat mainly pelvic cancers (Table 3). Table 3 also shows that the number of patients treated per machine during that month was equally distributed between the treatment machines, despite the significant differences in the number of fields treated per patient for some units. Units 4 and 5 were mainly used for four-field techniques; the other three units were predominantly used to deliver two fields.
Comparison of BTEp and Fields2 as a Workload Indicator Using 15 Minutes as the Denominator Wilcoxon rank sum tests for BTEp and fields2 showed that they are both significantly greater than 1 (P<0.001). The paired Wilcoxon signed rank test showed that BTEp is significantly closer to 1 than fields2 (P<0.001). There is a significant difference between the various machines for both fields2 and BTEp (P<0.001). This increased predictive ability for BTEp is significant for the combined values of units 4 and 5 (P<0.001), but not for the joint values of units 1, 2 and 3 (P = 0.320). The difference between the squared distances from the mean of fields2 and BTEp was 0.114, which implies less variation for BTEp (P<0.00; paired t test, confidence intervals 0.078 to 0.149). When 10 minutes is substituted in the BTEp and fields2 formulae, fields2 is closer to 1 and therefore a better indicator (P = 0.00, Wilcoxon signed rank test).
Comparison between TBCC and Combined AUS Mean Values
Fig. 1. Productivity: fields/hour (A–J = individual AUS centres).
Fig. 2. Productivity: BTE/hours (A–J = individual AUS centres).
Fields/hour (Fig. 1): This centre’s mean value (10.5 fields/hour) falls within the range for AUS centres (9.2–14.1; median 10.88; mean 11.26). BTE/hour (Fig. 2): The TBCC mean value (5.16 BTE/ hour) falls within the range of the AUS values of 4.7– 6.9 (median 5.25; mean 5.64). BTE/fraction (Fig. 3): This centre’s mean value (1.49 BTE/fraction) falls within the range of the AUS values of 1.2–1.7 (median 1.3; mean 1.37).
Validation of the AUS BTE Model for Use in Canadian Conditions There is a significant correlation between increasing BTE and increased time needed to treat (P<0.001). However, there is no evidence for a statistically significant correlation between increasing numbers of fields and an increase in the time needed to treat (P = 0.198). This suggests that the AUS BTE model may be a more comprehensive indicator in the Canadian setting than fields or patients per machine. However, there was a difference of 16.8% between
Fig. 3. Productivity: BTE/fractions (A–J = individual AUS centres).
the actual (3403) and the predicted (3975) BTE, suggesting that the AUS model could not fully accommodate local workload factors that might influence throughput. Australian BTE Workload Measure for Linear Accelerators in Canada
11
Discussion Statistics for 1997 show that the linear accelerators at this centre treated an average of 245 patients during that year, which the ISRON model would regard as suboptimal because it is less than the 250 recommended per unit per year. With the increasing number of complex techniques, it is considered that these simple indicators probably do not reflect the true workload. Productivity within radiotherapy services is dependent on several factors, including: actual treatment time, transit time for patients to get in and out of the treatment vault, complexity of technique, performance status of patients, scheduled and unscheduled maintenance, training time, and others. Most departments have developed average ‘treatment slots’, which are artificial constructs established to assist patient scheduling and the prediction of workload. These slots are sometimes too short or too long for the treatment being applied. For example, if there are excessive slots without enough time to treat using complex techniques (or poor performance cases), managers end up having to reduce the throughput on that machine to accommodate patients and staff. If the treatment slots are too long for a particular technique, the staff generally use the full time to treat their patients, or they will use the small time gains to accommodate other patients requiring longer treatment times.
Measurement of Workload This present study shows that BTE/hour is a better measure of combined linear accelerator workload than fields2/hour, which can be implied from the better correlation between the time needed to treat and BTE compared with fields2. The ability to show that BTEp using a 15-minute denominator is a better indicator at this centre confirms the potential for this model outside of Australia. This study could be criticized because it used a different denominator to that in the original BTE model, which was 10 minutes. When we used 10 minutes as the denominator, fields2 was a better indicator than BTEp. This reversal of indicator performance is related to the failure of BTEp to function as an indicator for treatment units 1, 2 and 3, which was amplified when using a 10-minute denominator. The latter had enough effect on the combined value to make fields2 a better indicator. Of interest is the finding that BTEp is a better indicator than fields2 in units 4 and 5, regardless of whether 10 or 15 minutes is used as a denominator. The authors propose that this can be explained by these units being able to treat complex techniques using close to 10 minutes per patient, thus making its BTE findings closer to the AUS model. These advantages over fields2 are that significant for these treatment units that they are not reversed by a change 12
P. Craighead et al.
in the denominator. A further conclusion from this finding is that the centre needs to carry out its own time and motion study to be able to modify the factors influencing BTE scores at the TBCC. The increase in fields/hour and BTE/hour for units 4 and 5 would lead us to speculate that multileaf collimation and enhanced dynamic wedges enable a more efficient throughput of complex techniques. This is illustrated by the significantly improved predictive ability of BTEp for units 4 and 5. The increased efficiencies created by these beam accessories indicate the need to include this as a factor in any statistical regression for future BTE models. That there is less variation in BTEp than in fields2 between the different machines might explain the discrepancy between the proportion of patients and the number of fields per patient per machine (Table 3). This is probably due to BTE being able to represent the workload of machines more fairly by including factors such as complexity and performance status, which results in techniques (whether simple, associated with many fields, etc.) being credited for the actual time needed to treat patients.
Comparison of AUS and TBCC Values Fields/hour, BTE/time and BTE/fraction at this centre fell within the range of values for AUS centres (Figs 1–3). These authors do not have access to the core data from AUS centres, so it is difficult to show statistically whether this centre’s values are dissimilar or not. The lower BTE/hour and fields/hour values further support our concern that scheduling practices need to be refined after a time and motion study. The high BTE/fraction values for the TBCC, and the correlation of increased BTE with increased time to treat, imply that the lower actual BTE is probably related to reduced efficiency relative to AUS departments. Preliminary work has already allowed us to reduce scheduling times to 12 minutes for patients with breast and prostate cancers.
Need for Modifications of the AUS Model We suggest that staff at centres outside Australia who wish to use this model should consider performing time and motion studies. The large difference between the actual and the expected BTE for the TBCC confirms the need to modify the BTE model for this centre before it can be of real use to us. The authors hypothesize that a time and motion study will enable both refinement of the scheduling practices (making us more efficient) and modification of the BTE factors. The differences in booking practices between this and AUS centres should be kept in context. The authors believed that it was important to use a pragmatic approach to compare the adequacy of the AUS model for this centre. Booking times represent
ony an average of the time needed to treat all patients; it is therefore the utilization of the full day that is far more critical to any model. We have shown that, although the BTE model is a better predictor than fields2 for the TBCC using 15 minutes, this was reversed when 10 minutes was used. This study indicates that the guidelines for calculating the AUS BTE model are easy to use. Once a refined model has been developed here, this could assist us in the following ways: to motivate the use of a standard indicator of machine productivity in a provincial/ national jurisdiction; to measure the productivity of treatment machines; to assess how to integrate newer technology; and to encourage a reasonable level of funding.
Conclusion As radiation oncology advances over the next decade, it will be critical to develop workload indicators that accommodate factors such as treatment complexity, the performance status of the patients, and newer technologies. This is especially true of universally funded health care systems, where these indicators are used by health departments to provide funding for cancer treatment. This study has shown that the BTE model can be applied to centres outside Australia, but that these jurisdictions would be best advised to modify the values to integrate the local treatment environment. Although it might be tempting to think that the development of an objective workload indicator for radiation oncology productivity has little relevance to private and semi-privately funded systems, this is very far from being the truth [17]. The emergence of a strongly managed care movement (i.e. HMOs) in the USA is expected to threaten the ability of some radiation oncology departments to provide such a wide range of technologies. For HMOs to be reassured that funding is appropriate they will need workload indicators that are objective and accommodate complexity of technique.
References 1. Report of the Inter Society of Radiation Oncologists of North America (ISRON) In: Integrated cancer management. Philadelphia, Pennsylvania: American College of Radiation Oncology, 1991. 2. Van Daal WA, Bos MA. Infrastructure for radiotherapy in the Netherlands: development from 1970 to 2010. Int J Radiat Oncol Biol Phys 1997;37:411–5. 3. Stevens G, Firth I. Clinical audit in radiation oncology: results from one centre. Aust Radiol 1996;40:47–54. 4. Theil HJ, Oeschler WA, Kastura B, et al. Costs–proceeds calculation for radiotherapy: comment. Strahlenther Onkol 1996;172:128–47; discussion 154. 5. Owen JB, Coia LR, Hanks GE. The structure of radiation oncology in the United States in 1994. Int J Radiat Oncol Biol Phys 1997;39:179–85. 6. Richards MA, Parrott JC. Tertiary cancer services in Britain: benchmarking study of activity and facilities at 12 specialist centres. BMJ 1996;313:347–9. 7. Van Daal WA, Bos MA. Infrastructure for radiotherapy in the Netherlands: development from 1970 to 2001. Int J Radiation Oncol Biol and Physics 1997;37(2):411–5. 8. Frodin IE, Jonson E, Moller T, et al. Radiotherapy in Sweden. Acta Oncol 1996;35:967–79. 9. Wade Report. Cancer manpower committee – Projections of equipment and personnel needs for radiotherapy in Ontario. Toronto, Ontario: Ontario Government, 1991. 10. Peters LJ. Through a glass darkly: predicting the future of radiation oncology. Int J Radiat Oncol Biol Phys 1995;31:219–25. 11. Burkhardt JH, Sunshine JH. Utilization of radiologic services in different payment systems and patient populations. Radiology 1996;200:201–7. 12. Fields A. Cancer treatment under pressure: strategies in the face of changing demographics and economics. Can J Oncol 1992;3 Suppl 3. 13. Abraham C, Fine S. Canadian cancer system slips to second best. Globe and Mail (Toronto) 1998 July 25; Sect A1: 8–9. 14. Delaney GP, Gebski V, Rus M, et al. The equivalent simple treatment visit (ESTV) model does not measure radiation oncology productivity under Australian conditions. Clin Oncol 1997;9:245–7. 15. Delaney GP, Gebski V, Lunn AD, et al. Basic treatment equivalent (BTE): a new measure of linear accelerator workload. Clin Oncol 1997;9:234–9. 16. Delaney GP, Gebsku V, Lunn AD, et al. An assessment of the basic treatment equivalent (BTE) model as a measure of radiotherapy workload. Clin Oncol 1997;9:240–4. 17. Lazar G, Desch E. Performance measurements in cancer care. Cancer 1998;82 (8 Suppl): 2016–21.
Received for publication October 1998 Accepted following revision June 2000
Australian BTE Workload Measure for Linear Accelerators in Canada
13