- Email: [email protected]
Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection
Clinical Oncology xxx (2016) 1e7 Contents lists available at ScienceDirect
Clinical Oncology journal homepage: www.clinicaloncologyonline.net
Original Article
Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection J. Mackenzie *yz, G. Graham *, I.A. Olivotto *y * Tom
Baker Cancer Centre, Calgary, Alberta, Canada Department of Oncology, University of Calgary, Alberta, Canada z Edinburgh Cancer Centre, Western General Hospital, Edinburgh, UK y
Received 13 February 2016; received in revised form 20 June 2016; accepted 20 July 2016
Abstract Aims: The Canadian Partnership for Quality Radiotherapy quality assurance guidelines recommend that radiation oncologist peer review of curative radiotherapy plans takes place ideally before the first fraction of treatment is delivered. This study documented and evaluated the outcomes of weekly, disease sitespecific, radiotherapy peer review, quality assurance rounds at the Tom Baker Cancer Centre in Calgary, Canada with a view to making recommendations about the optimal timing and documentation of peer review during the radiotherapy planning processes. Materials and Methods: Outcomes of each case reviewed at (i) breast, (ii) head and neck (including thyroid and cutaneous cases) and (iii) lung team quality assurance rounds from 6 January to 5 May 2015 were recorded prospectively. Each radiotherapy plan was assigned an outcome: A for plans with no suggested changes; B for satisfactory, but where issues were raised to consider for future patients; or C when a change was recommended before the first or next fraction. The B outcomes were further subdivided into B1 for a case-specific concern and B2 for a policy gap. Plans were assessed after contour definition and before the plan was formulated (post-contouring reviews) and/or assessed when the plan was complete (post-planning reviews). Results: 209 radiotherapy plans prescribed by 20 radiation oncologists were peer reviewed at 43 quality assurance meetings. 93% were curative-intent and 7% were palliative. 83% of plans were reviewed before delivery of the first treatment fraction. There were a total of 257 case reviews: 60 at the post-contouring stage, 197 at the post-planning stage, including 46 patients reviewed at both time points. Overall rates of A, B1, B2 and C outcomes were 78%, 9%, 4% and 9%, respectively. The most common reason for a B or C outcome was related to target volume definition. Only 56% of C outcomes at the post-planning stage would have been detected at the post-contouring stage. Results varied between tumour site groups. Conclusions: 9% of radiotherapy plans reviewed had changes suggested before delivery to the patient. Review at the post-planning stage after plan completion was necessary to detect all suggested changes, but for head and neck cases, all C outcomes could have been detected at the post-contouring stage. More widespread implementation of radiotherapy peer review in the UK is recommended. Ó 2016 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Key words: Peer review of radiotherapy planning; radiotherapy quality assurance
Introduction The ‘Towards Safer Radiotherapy’ report [1] detailed a framework for improving patient safety and reducing error in the process of radiotherapy delivery. The report highlighted that radiotherapy indications are generally reviewed by multidisciplinary teams but the details of target volume and dose prescription are not usually
Author for correspondence: J. Mackenzie, Department of Clinical Oncology, Edinburgh Cancer Centre, Western General Hospital, Crewe Road South, Edinburgh EH4 2XU, UK. Tel: þ44-131-537-2209. E-mail address: [email protected] (J. Mackenzie).
evaluated by a separate clinician. Given that variation in physician contouring is well documented [2], peer review of the professional decisions made during radiotherapy planning should be a standard of care. Deviations from trial radiotherapy standards have been shown to have a detrimental effect on patient outcomes [3]. Quality assurance of physician contouring and radiotherapy plan quality are now expected in multicentre radiotherapy trials [4]. Peer review of clinical radiotherapy plans is not yet standard in UK practice, however, the recent Royal College of Radiologists publication on consultant job planning advocates the use of peer review as an important component of radiotherapy quality assurance [5]. As
http://dx.doi.org/10.1016/j.clon.2016.08.012 0936-6555/Ó 2016 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
2
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
radiotherapy practice evolves to deliver higher, targeted doses to smaller volumes, target definition and dosimetry review becomes increasingly important. Elements of a radiotherapy plan that could be evaluated include indications for treatment, target volume and organs at risk (OAR) definition, choice of dose/fractionation, plan design elements, such as number of fields or use of bolus, and compliance with policy guidelines. Most North American radiotherapy centres incorporate aspects of peer review into routine practice [6,7]. An American Society for Radiation Oncology (ASTRO) white paper provides a framework for peer review activities [8]. The Canadian Partnership for Quality Radiotherapy issued quality assurance guidelines first published in 2011, which included the recommendation that curative-intent treatment volumes and dosimetry be peer reviewed by a radiation oncologist, ideally before the first fraction or within the first 25% of a prescribed course [9]. Peer review quality assurance meetings can optimise radiotherapy plans for individual patients, streamline physician practice, improve interdisciplinary communication and collaboration and be an excellent teaching opportunity for oncology, physics and radiography trainees [8,10]. In most Canadian radiotherapy centres, regular peer review quality assurance meetings occur [6] but the frequency, structure and documentation processes differ. Some sites aim to review all curative-intent and complex palliative plans, others review a random sample of plans. Standards have not been agreed for documentation or the timing of review during the planning process. This is a report of a project undertaken in a mid-sized Canadian radiotherapy centre, to standardise the nomenclature and documentation of peer review quality assurance meetings and to define the optimal time to peer review radiotherapy plans. Prior to this project, seven weekly one hour tumour sitespecific peer review meetings took place at the Tom Baker Cancer Centre (TBCC). Review outcomes were fed back informally within each meeting but with no documentation of the discussions and decisions taking place, and no standardised terminology to measure the outcomes of peer review. There was therefore an opportunity to standardise the nomenclature used and to document and quantify the work taking place within these meetings and the subsequent impact on patient care within the radiotherapy planning pathway. Approaches to peer review within the seven meetings were variable and therefore a selection of three of the seven meetings (breast, lung and head and neck groups) was chosen to pilot recording of outcomes in the first instance. At the beginning of the study we adapted the A, B, C classification system designed by Lefresne et al. [11] to document the outcomes from the three meetings.
Materials and Methods At TBCC, Calgary, Canada, data collection was undertaken prospectively from 6 January to 5 May 2015 from cases reviewed at the 1 hour/week quality assurance meetings of
the breast, head and neck (including thyroid and cutaneous cases) and lung tumour sites. Case Review In terms of choice of cases, the head and neck and lung groups aimed to assess all curatively treated patients, but due to curative case volume, the breast group prioritised locoregionally treated patients over breast-alone treatments. No targets existed for the number of patients to be reviewed per week, with a principal focus on the quality of review. Cases were identified as either ‘curative’ or ‘palliative’. The case review was conducted in a seminar room equipped with two computers, one to access the electronic medical records for clinical details and the other to project the radiotherapy contours and/or plan for group review. The seminar room enabled video-conference with radiation oncologists from two neighbouring, small radiotherapy centres. Following a clinical synopsis by the treating radiation oncologist or rounds chair, indications for treatment, target and OAR contours, dose/fractionation and dose volume histograms were reviewed. Review took place regardless of the treating radiation oncologist’s attendance. A record of the number of radiation oncologists in attendance was made, principally to ensure that at least two radiation oncologists were in attendance in order to constitute a peer review. In accordance with Canadian Partnership for Quality Radiotherapy recommendation, it was documented whether peer review took place prior to delivery of the first treatment fraction. Outcomes The assembled radiation oncologists agreed to a consensus A, B or C outcome as previously described [11]. A plans were considered satisfactory with no suggested changes, a B outcome meant the plan was satisfactory but case discussion had identified issues to consider for future patients and a C meant an unsatisfactory plan, with a change recommended before the first or next fraction. The B outcomes were further subdivided into B1 for a casespecific concern and B2 for a policy gap for that clinical circumstance that should be developed for future patients. The rate of C outcomes could be considered to be the ‘error detection rate’ in the quality assurance process. We recorded whether the treating physician changed the plan as a consequence of a C outcome, although within this study there was no obligation to do so. Timing of Peer Review Across the three groups there were different approaches to the timing of radiotherapy peer review. In order to assess the optimal timing, we categorised these approaches according to the time point of peer review. Some cases were reviewed after the plan had been approved by the treating physician. This provided an opportunity for a ‘global review’ of the case, including target volume definition, OARs,
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
indications for treatment, choice of dose/fractionation and overall plan construction and dosimetry. These were referred to as post-planning (PP) reviews. Some plans were reviewed after contouring, with a focus on target volume definition, OARs, indications for treatment and choice of dose/fractionation. These were referred to as post-contouring (PC) reviews. The breast and lung groups preferentially reviewed radical plans at the post-planning stage. The head and neck group routinely discussed patients post-contouring. That timing was initially implemented to reduce replanning work. For the study interval, the head and neck group agreed to review plans at both the post-contouring and the post-planning stages. In some circumstances this meant two case reviews of the same patient (post-contouring and post-planning) but for others the plan had already been prepared before the next meeting so they had a post-planning review despite not yet having had a postcontouring review. This change in the head and neck team approach was made to determine if additional changes were suggested as a consequence of peer review at a later stage in the planning process, i.e. was anything missed by reviewing at this earlier stage (post-contouring) in the planning process. For post-planning B and C outcomes it was therefore also documented whether all the changes suggested could have been identified at the postcontouring stage. Analysis of B and C Outcomes After completion of the above prospective data collection, outcomes were analysed and classified according to the types of suggested change. The common themes for B and C outcomes were then used to develop a classification system. Using the A Project Ethics Community Consensus Initiative (ARECCI) Ethics Screening Tool [12] this project was identified as posing ‘minimal risk’ to participants and was exempt from institutional research ethics board review.
Results Outcomes There were 257 case reviews among 209 patients planned by 20 radiation oncologists discussed in 43 quality assurance meetings during the study interval (17 radiation oncologists from TBCC and 3 from smaller centres who participated by video-conferencing). During the study period, 511 radiotherapy courses of the 3 index tumour sites and 969 total radiotherapy courses were started at TBCC. Therefore, 209/511 (41%) of potentially eligible cases underwent peer review during the study period. Of the reviewed plans, 93% were curativeintent and 7% were palliative. A median of 4 radiation oncologists (range 2e8) reviewed each case. 83% of cases were reviewed before delivery of the first fraction of treatment.
3
The overall rate of A, B1, B2 and C outcomes was 78%, 9%, 4% and 9%, respectively, across the 257 case reviews. Table 1 details outcomes overall and by tumour site group. Timing of Peer Review 60 cases were reviewed at the post-contouring stage and 197 at the post-planning stage. In total, 46 patients had peer review at both the post-contouring and the post-planning stage e 40 in the head and neck group and 6 in the breast group. The rates of A, B1, B2 and C outcomes according to the timing of peer review are shown in Table 1. In 56% of C outcomes at the post-planning stage, all of the suggested changes could have been detected after contouring and before plan development (post-contouring stage). When evaluated by tumour site this was 53%, 100% and 57%, respectively, for breast, head and neck and lung disease sites. Analysis of B and C Outcomes B and C outcomes were grouped together for the purposes of providing sufficient numbers to identify common themes. In total, 56 B or C outcomes were given to 55 patients. One patient was assigned a C at the post-contouring review, then a B2 at the post-planning review. In 16 of the 55 patients, two reasons were identified for the B or C outcome, but in most patients (n ¼ 39) only one reason was defined for the B or C outcome. The reasons identified for B or C outcomes were grouped as follows: U, unusual patient anatomy/pathology or comorbidity; T, choice of target volume (including definition of OARs); D, plan design issue; P, lack of policy for specific clinical circumstance; F, dose/fractionation issue; I, indication for treatment. Examples are given in Table 2. The most common reason for B or C outcomes was the choice of the target volume (T) but varied by tumour site (Figure 1). 16 of 18 C outcomes were translated into plan changes following quality assurance review. Of the remaining 2 case reviews receiving a C outcome, in one case the plan was not changed and in the other case it was not clear whether the plan had been changed due to difficulty accessing the planning system at the peripheral centre where the patient was treated.
Discussion Peer review quality assurance across the three selected tumour sites resulted in suggested radiotherapy plan changes (C outcomes) in 9% of the case reviews. These cases could be considered ‘near misses’ where deviation from best practice was detected before having consequences for the patient. This supports peer review contributing towards safer radiotherapy. The purpose of radiotherapy plan peer review is twofold: (i) to reduce ‘errors’ for individual patients and (ii) to create a forum for group discussion, consensus building, communication and treatment policy improvement [13].
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
4
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
Table 1 Number of cases, timing and outcomes of peer review; in total and by tumour site group
Number of patients PC reviews PP reviews Number of patients reviewed at both PC and PP stage Total number of reviews (PCþPP) Percentage of cases reviewed before first fraction delivered Outcomes e PC reviews A B1 B2 C Outcomes e PP reviews A B1 B2 C All outcomes (sum of PCþPP outcomes) A B1 B2 C
Total
Breast
Head and neck
Lung
209 58 197 46 257 80% n ¼ 60 75% 13% 3% 8% n ¼ 197 79% 8% 4% 9% n ¼ 257 78% 9% 4% 9%
85 12* 81y 6 93 66% n ¼ 12 33% 17% 17% 33% n ¼ 81 72% 7% 7% 14% n ¼ 91 67% 9% 9% 16%
73 43 70 40 113 97% n ¼ 43 88% 12% 0% 0% n ¼ 70 90% 7% 1.5% 1.5% n ¼ 113 89% 9% 1% 1%
51 5 46 0 51 92% n¼5 60% 20% 0% 20% n ¼ 46 76% 9% 2% 13% n ¼ 51 75% 10% 2% 14%
PC, post-contouring; PP, post-planning. * Two patients are included who physicians chose to discuss before contouring, both of whom had C outcomes as the overall treatment plan was changed as a consequence of peer review. y Two patients had two PP reviews. One patient had their main plan and then boost plan reviewed on separate occasions. A second patient was reviewed a second time in error but on the second occasion a policy gap was identified (B2).
Table 2 Categorisation of B and C peer review outcomes, including examples for each category Category B/C outcome
% of total B/C outcomes
Example
Unusual anatomy/ pathology/comorbidity (U) Target volume (T) Plan design (D) Policy (P) Dose/fractionation (F)
10
Breast patient with severe pectus excavatum
35 20 16 15
Indication for radiotherapy (I)
4
Coverage of an additional nodal group suggested Choice of number of fields to give better dose distribution Definition of GTV following neoadjuvant chemotherapy for SCLC Lung SBRT case changed from 60 Gy in 8# to 60 Gy in 15# as not meeting heart DVH Treatment strategy changed from radiation to systemic therapy
GTV, gross tumour volume; SCLC, small cell lung cancer; SBRT, stereotactic body radiotherapy; DVH, dose volume histogram.
Individual patients benefit from a ‘team-based’ review of radiotherapy planning decisions, similar to the process adopted by multidisciplinary teams to review treatment decisions for new or challenging cases [14,15]. Peer review can identify policy gaps for specific clinical circumstances. In the current study, there were 10 instances that identified a lack of departmental policy (B2 outcomes). Such cases can flag an opportunity for tumour group consensus building. Outcomes The outcomes classification used was adapted from Lefresne et al. [11] with a subdivision of B outcomes to differentiate policy gap as compared with individual patient
variation issues. Lefresne et al. reported outcomes from a once-weekly, ‘generic’, departmental quality assurance meeting where cases were selected randomly from patients starting in the previous week. The B and C outcome rates were 6% and 1%, respectively. Their process may be a good educational and communication forum and method to introduce regular peer review into a department’s function, but lacks the specificity of in-depth review of most curative or complex plans by site-specific colleagues. Lymberiou et al. [16] also used the Lefresne et al. classification, applied to breast quality assurance rounds only, with 2223 cases reviewed over 2 years. They found rates of A, B and C outcomes of 96%, 2% and 2%, respectively. Multivariate analysis showed a higher likelihood of C outcome for patients undergoing regional nodal irradiation.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
5
Fig 1. B and C outcome categorisation by tumour site group and in total, shown as a percentage of the total number of suggested changes by that group.
Similar to our TBCC group, the authors prioritised discussion of locoregional treatments rather than two field tangents as these were considered most complex and more likely to require changes. Despite the group’s description of ‘well-established treatment protocols’ there were still high proportions of B (55%) and C (44%) outcomes attributed to clinical treatment decisions such as the use of boost and indications for regional nodal irradiation. In our study, there was considerable variation in B and C outcome rates between tumour site groups. This could be related to several factors. The head and neck and lung teams attempted to review all curative-intent cases, whereas the breast team prioritised inclusion of only more complex, locoregional or unusual cases. The number of changes within the head and neck plans was low compared with the other tumour sites. This correlated with well-documented departmental consensus (four radiation oncologists) on normal tissue, target volume and dosimetry constraints. In addition, most head and neck patients were treated with intensity-modulated radiotherapy whereas a wider range of radiotherapy techniques was used in the breast and lung sites, with attendant greater scope for practice variation. There is an opportunity to improve consensus and potentially reduce inconsistencies if the breast and lung groups were to adopt similar contouring and planning constraint policies. Other groups have reported on peer review outcomes in larger populations across all tumour sites. Brundage et al. [13] reviewed 3054 cases and reported a 4.1% ‘not approved’ rate. Ballo et al. [17] reviewed 2988 cases and reported a change recommendation in 12.2% of cases. The number of changes varied by disease site and treating physician [17]. They also found that not only did groups increase uptake of peer review over time but that the number of plan changes reduced over time, possibly related to team standardisation evolving from the peer review discussions. We therefore plan to repeat the study
1 year after adopting the documentation process across all seven tumour site quality assurance groups (this widespread rollout occurred in June 2015). Timing of Peer Review The ideal timing of peer review may be different depending on the disease site. For the duration of the study, the head and neck group changed their practice from reviewing at the post-contouring stage to reviewing at both the post-contouring and the post-planning stage. However, if the plan was already complete before an opportunity for post-contouring review then the whole case would be reviewed in one sitting (a post-planning review). 7 B and C outcomes were defined at the post-planning reviews for head and neck patients and in all cases the suggested changes could have been detected at the post-contouring stage. The head and neck group had a detailed protocol describing acceptable plan quality parameters that could explain the absence of B or C outcomes occurring later in the planning process, and the low rate of B and C outcomes overall compared with the other disease sites. There were 12 B and C outcomes from 113 case reviews of head and neck patients and 11 of 12 (92%) were classified as related to target volume or OAR definition. The only post-planning C outcome in the head and neck group suggested changes that could have been identified at the post-contouring stage. These data support the timing of head and neck case reviews at the post-contouring stage to have the highest yield to detect practice inconsistencies. Given the small numbers, however, it cannot be assumed that a postplanning review would never detect inconsistencies that could not have been identified at the post-contouring stage. The safest approach overall therefore is to review at the post-planning stage or to review at both the postcontouring and the post-planning stage to minimise replanning.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
6
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
Contrary to the head and neck cases, a high percentage of B and C outcomes would have been missed if the breast and lung groups reviewed only at the post-contouring stage. If the desire was to review each case only once, optimal timing for those sites would be at the post-planning stage. No other reported peer review studies have commented on the timing of peer review, with most opting for review at the postplanning stage to encompass all aspects of radiotherapy planning decision making within one review. Increasing Peer Review in Practice A survey of Canadian cancer centres showed widespread adoption of weekly peer review meetings [10]. However, of the 14 centres surveyed by Brundage et al., only 29% conducted peer review in more than 80% of cases treated with curative-intent and there is therefore considerable room for improvement. In a separate survey, Canadian radiation oncologists felt peer review was beneficial and expressed enthusiasm for such processes, recognising their potential to improve patient care [6]. They listed various resource limitations that they felt impacted on the ability to perform peer review, including protected time for radiation oncologists to attend meetings. Implementing routine peer review in UK centres also presents challenges for clinical oncologists who generally have broader commitments in both systemic and radiation therapy. Despite this, efforts should be made to address this ‘weakest link’ in the planning process [2]. The importance of job planning to accommodate time for peer review has been recently recognised by the Royal College of Radiologists [5]. UK centres have published successful projects implementing regular peer review [18]. In Wolverhampton, weekly quality assurance meetings of non-trial, curative-intent cases resulted in a reduction in ‘major alterations’ to treatment plans (i.e. C outcomes) from 10% to 4.2% over two audits separated by 18 months [14]. In Belfast, radiotherapy plans for radically treated lung cancer patients were reviewed in weekly peer review meetings. Of 122 cases reviewed, a change in a component of the treatment plan took place in 27% of cases as a consequence of peer review [19]. The current study contributed to a cultural change in peer review quality assurance meetings at the TBCC. Documentation of peer review decisions was formalised and greater discipline was introduced into defining a concrete (recordable) outcome of the peer review discussion. The documentation used for the study was adopted in June 2015, by all seven tumour site quality assurance groups for recording their outcomes. Successful peer review quality assurance requires widespread acceptance and a non-threatening, non-judgemental group culture. The focus needs to be on improving care as compared with finding errors. Success requires oncologists becoming comfortable with public review of their own and colleagues’ treatment decisions with the goal to improve care for patients. As a strategy to introduce regular peer review, weekly, department-wide meetings could review a random sample of cases across tumour sites. Once comfort and evidence of
value have been developed, more labour-intensive but detailed disease site-specific rounds could be introduced. Either option can be applied across hospital sites using video-conferencing to enable peer support and shared practice between geographically separate groups [17].
Conclusions This prospective audit of peer review quality assurance outcomes in a Canadian cancer centre resulted in suggested plan changes before the next fraction in 9% of cases reviewed and consistent documentation of outcomes. Outcomes varied between tumour sites. Review at the plan completion time point is suggested as being the optimal timing for peer review. Widespread implementation of such processes in the UK could be guided by existing experience and guidelines. The yield from investment in this area should be improved patient safety and team functioning.
Acknowledgements J. Mackenzie received a clinical research fellowship at the Tom Baker Cancer Centre funded by the Alberta Cancer Foundation, through a bequest from Mrs Helen Thomson. The authors are grateful to the members of the Tom Baker Cancer Centre head and neck, lung and breast quality assurance meetings for their support and cooperation during this project.
References [1] The Royal College of Radiologists, Society and College of Radiographers, Institute of Physics and Engineering in Medicine, National Patient Safety Agency, British Institute of Radiology. Towards safer radiotherapy 2008. https://www.ipem.ac.uk/ Portals/0/Images/Towards%20Safer%20Radiotherapy.pdf. [2] Roques TW. Patient selection and radiotherapy volume definition - can we improve the weakest links in the treatment chain? Clin Oncol (R Coll Radiol) 2014;26(6):353e355. [3] Peters LJ, O’Sullivan B, Giralt J, et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol 2010;28(18):2996e3001. [4] Gwynne S, Spezi E, Sebag-Montefiore D, et al. Improving radiotherapy quality assurance in clinical trials: assessment of target volume delineation of the pre-accrual benchmark case. Br J Radiol 2013;86(1024):20120398. [5] The Royal College of Radiologists. Guide to job planning in clinical oncology 2015. https://www.rcr.ac.uk/system/files/ publication/field_publication_files/bfco153_jobplanning.pdf. [6] Hamilton SN, Hasan H, Parsons C, et al. Canadian radiation oncologists’ opinions regarding peer review: a national survey. Pract Radiat Oncol 2015;5(2):120e126. [7] Lawrence YR, Whiton MA, Symon Z, et al. Quality assurance peer review chart rounds in 2011: a survey of academic institutions in the United States. Int J Radiat Oncol Biol Phys 2012;84(3):590e595. [8] Marks LB, Adams RD, Pawlicki T, et al. Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: executive summary. Pract Radiat Oncol 2013;3(3):149e156.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7 [9] Canadian Partnership for Quality Radiotherapy. Quality assurance guidelines for Canadian radiation treatment programs 2013. http://www.cpqr.ca/wp-content/uploads/2013/09/ QRT2015-12-03.pdf. [10] Brundage M, Foxcroft S, McGowan T, Gutierrez E, Sharpe M, Warde P. A survey of radiation treatment planning peerreview activities in a provincial radiation oncology programme: current practice and future directions. BMJ Open 2013;3(7). http://dx.doi.org/10.1136/bmjopen-2013-003241. [11] Lefresne S, Olivotto IA, Joe H, Blood PA, Olson RA. Impact of quality assurance rounds in a Canadian radiation therapy department. Int J Radiat Oncol Biol Phys 2013;85(3):e117ee121. [12] Hagen B, O’Beirne M, Desai S, Stingl M, Pachnowski CA, Hayward S. Innovations in the ethical review of health-related quality improvement and research: the Alberta Research Ethics Community Consensus Initiative (ARECCI). Healthcare Policy 2007;2(4):e164ee177. [13] Brundage MD, Dixon PF, Mackillop WJ, et al. A real-time audit of radiation therapy in a regional cancer center. Int J Radiat Oncol Biol Phys 1999;43(1):115e124.
7
[14] Taplin SH, Weaver S, Salas E, et al. Reviewing cancer care team effectiveness. J Oncol Pract 2015;11(3):239e246. [15] Lamb BW, Brown KF, Nagpal K, Vincent C, Green JS, Sevdalis N. Quality of care management decisions by multidisciplinary cancer teams: a systematic review. Ann Surg Oncol 2011;18(8):2116e2125. [16] Lymberiou T, Galuszka S, Lee G, et al. Predictors of breast radiotherapy plan modifications: quality assurance rounds in a large cancer centre. Radiother Oncol 2015;114(1):17e21. [17] Ballo MT, Chronowski GM, Schlembach PJ, Bloom ES, Arzu IY, Kuban DA. Prospective peer review quality assurance for outpatient radiation therapy. Pract Radiat Oncol 2014;4(5):279e284. [18] Brammer CV, Pettit L, Allerton R, et al. Impact of the introduction of weekly radiotherapy quality assurance meetings at one UK cancer centre. Br J Radiol 2014;87(1043):20140422. [19] Rooney KP, McAleese J, Crockett C, et al. The impact of colleague peer review on the radiotherapy treatment planning process in the radical treatment of lung cancer. Clin Oncol (R Coll Radiol) 2015;27(9):514e518.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
Clinical Oncology journal homepage: www.clinicaloncologyonline.net
Original Article
Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection J. Mackenzie *yz, G. Graham *, I.A. Olivotto *y * Tom
Baker Cancer Centre, Calgary, Alberta, Canada Department of Oncology, University of Calgary, Alberta, Canada z Edinburgh Cancer Centre, Western General Hospital, Edinburgh, UK y
Received 13 February 2016; received in revised form 20 June 2016; accepted 20 July 2016
Abstract Aims: The Canadian Partnership for Quality Radiotherapy quality assurance guidelines recommend that radiation oncologist peer review of curative radiotherapy plans takes place ideally before the first fraction of treatment is delivered. This study documented and evaluated the outcomes of weekly, disease sitespecific, radiotherapy peer review, quality assurance rounds at the Tom Baker Cancer Centre in Calgary, Canada with a view to making recommendations about the optimal timing and documentation of peer review during the radiotherapy planning processes. Materials and Methods: Outcomes of each case reviewed at (i) breast, (ii) head and neck (including thyroid and cutaneous cases) and (iii) lung team quality assurance rounds from 6 January to 5 May 2015 were recorded prospectively. Each radiotherapy plan was assigned an outcome: A for plans with no suggested changes; B for satisfactory, but where issues were raised to consider for future patients; or C when a change was recommended before the first or next fraction. The B outcomes were further subdivided into B1 for a case-specific concern and B2 for a policy gap. Plans were assessed after contour definition and before the plan was formulated (post-contouring reviews) and/or assessed when the plan was complete (post-planning reviews). Results: 209 radiotherapy plans prescribed by 20 radiation oncologists were peer reviewed at 43 quality assurance meetings. 93% were curative-intent and 7% were palliative. 83% of plans were reviewed before delivery of the first treatment fraction. There were a total of 257 case reviews: 60 at the post-contouring stage, 197 at the post-planning stage, including 46 patients reviewed at both time points. Overall rates of A, B1, B2 and C outcomes were 78%, 9%, 4% and 9%, respectively. The most common reason for a B or C outcome was related to target volume definition. Only 56% of C outcomes at the post-planning stage would have been detected at the post-contouring stage. Results varied between tumour site groups. Conclusions: 9% of radiotherapy plans reviewed had changes suggested before delivery to the patient. Review at the post-planning stage after plan completion was necessary to detect all suggested changes, but for head and neck cases, all C outcomes could have been detected at the post-contouring stage. More widespread implementation of radiotherapy peer review in the UK is recommended. Ó 2016 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Key words: Peer review of radiotherapy planning; radiotherapy quality assurance
Introduction The ‘Towards Safer Radiotherapy’ report [1] detailed a framework for improving patient safety and reducing error in the process of radiotherapy delivery. The report highlighted that radiotherapy indications are generally reviewed by multidisciplinary teams but the details of target volume and dose prescription are not usually
Author for correspondence: J. Mackenzie, Department of Clinical Oncology, Edinburgh Cancer Centre, Western General Hospital, Crewe Road South, Edinburgh EH4 2XU, UK. Tel: þ44-131-537-2209. E-mail address: [email protected] (J. Mackenzie).
evaluated by a separate clinician. Given that variation in physician contouring is well documented [2], peer review of the professional decisions made during radiotherapy planning should be a standard of care. Deviations from trial radiotherapy standards have been shown to have a detrimental effect on patient outcomes [3]. Quality assurance of physician contouring and radiotherapy plan quality are now expected in multicentre radiotherapy trials [4]. Peer review of clinical radiotherapy plans is not yet standard in UK practice, however, the recent Royal College of Radiologists publication on consultant job planning advocates the use of peer review as an important component of radiotherapy quality assurance [5]. As
http://dx.doi.org/10.1016/j.clon.2016.08.012 0936-6555/Ó 2016 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
2
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
radiotherapy practice evolves to deliver higher, targeted doses to smaller volumes, target definition and dosimetry review becomes increasingly important. Elements of a radiotherapy plan that could be evaluated include indications for treatment, target volume and organs at risk (OAR) definition, choice of dose/fractionation, plan design elements, such as number of fields or use of bolus, and compliance with policy guidelines. Most North American radiotherapy centres incorporate aspects of peer review into routine practice [6,7]. An American Society for Radiation Oncology (ASTRO) white paper provides a framework for peer review activities [8]. The Canadian Partnership for Quality Radiotherapy issued quality assurance guidelines first published in 2011, which included the recommendation that curative-intent treatment volumes and dosimetry be peer reviewed by a radiation oncologist, ideally before the first fraction or within the first 25% of a prescribed course [9]. Peer review quality assurance meetings can optimise radiotherapy plans for individual patients, streamline physician practice, improve interdisciplinary communication and collaboration and be an excellent teaching opportunity for oncology, physics and radiography trainees [8,10]. In most Canadian radiotherapy centres, regular peer review quality assurance meetings occur [6] but the frequency, structure and documentation processes differ. Some sites aim to review all curative-intent and complex palliative plans, others review a random sample of plans. Standards have not been agreed for documentation or the timing of review during the planning process. This is a report of a project undertaken in a mid-sized Canadian radiotherapy centre, to standardise the nomenclature and documentation of peer review quality assurance meetings and to define the optimal time to peer review radiotherapy plans. Prior to this project, seven weekly one hour tumour sitespecific peer review meetings took place at the Tom Baker Cancer Centre (TBCC). Review outcomes were fed back informally within each meeting but with no documentation of the discussions and decisions taking place, and no standardised terminology to measure the outcomes of peer review. There was therefore an opportunity to standardise the nomenclature used and to document and quantify the work taking place within these meetings and the subsequent impact on patient care within the radiotherapy planning pathway. Approaches to peer review within the seven meetings were variable and therefore a selection of three of the seven meetings (breast, lung and head and neck groups) was chosen to pilot recording of outcomes in the first instance. At the beginning of the study we adapted the A, B, C classification system designed by Lefresne et al. [11] to document the outcomes from the three meetings.
Materials and Methods At TBCC, Calgary, Canada, data collection was undertaken prospectively from 6 January to 5 May 2015 from cases reviewed at the 1 hour/week quality assurance meetings of
the breast, head and neck (including thyroid and cutaneous cases) and lung tumour sites. Case Review In terms of choice of cases, the head and neck and lung groups aimed to assess all curatively treated patients, but due to curative case volume, the breast group prioritised locoregionally treated patients over breast-alone treatments. No targets existed for the number of patients to be reviewed per week, with a principal focus on the quality of review. Cases were identified as either ‘curative’ or ‘palliative’. The case review was conducted in a seminar room equipped with two computers, one to access the electronic medical records for clinical details and the other to project the radiotherapy contours and/or plan for group review. The seminar room enabled video-conference with radiation oncologists from two neighbouring, small radiotherapy centres. Following a clinical synopsis by the treating radiation oncologist or rounds chair, indications for treatment, target and OAR contours, dose/fractionation and dose volume histograms were reviewed. Review took place regardless of the treating radiation oncologist’s attendance. A record of the number of radiation oncologists in attendance was made, principally to ensure that at least two radiation oncologists were in attendance in order to constitute a peer review. In accordance with Canadian Partnership for Quality Radiotherapy recommendation, it was documented whether peer review took place prior to delivery of the first treatment fraction. Outcomes The assembled radiation oncologists agreed to a consensus A, B or C outcome as previously described [11]. A plans were considered satisfactory with no suggested changes, a B outcome meant the plan was satisfactory but case discussion had identified issues to consider for future patients and a C meant an unsatisfactory plan, with a change recommended before the first or next fraction. The B outcomes were further subdivided into B1 for a casespecific concern and B2 for a policy gap for that clinical circumstance that should be developed for future patients. The rate of C outcomes could be considered to be the ‘error detection rate’ in the quality assurance process. We recorded whether the treating physician changed the plan as a consequence of a C outcome, although within this study there was no obligation to do so. Timing of Peer Review Across the three groups there were different approaches to the timing of radiotherapy peer review. In order to assess the optimal timing, we categorised these approaches according to the time point of peer review. Some cases were reviewed after the plan had been approved by the treating physician. This provided an opportunity for a ‘global review’ of the case, including target volume definition, OARs,
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
indications for treatment, choice of dose/fractionation and overall plan construction and dosimetry. These were referred to as post-planning (PP) reviews. Some plans were reviewed after contouring, with a focus on target volume definition, OARs, indications for treatment and choice of dose/fractionation. These were referred to as post-contouring (PC) reviews. The breast and lung groups preferentially reviewed radical plans at the post-planning stage. The head and neck group routinely discussed patients post-contouring. That timing was initially implemented to reduce replanning work. For the study interval, the head and neck group agreed to review plans at both the post-contouring and the post-planning stages. In some circumstances this meant two case reviews of the same patient (post-contouring and post-planning) but for others the plan had already been prepared before the next meeting so they had a post-planning review despite not yet having had a postcontouring review. This change in the head and neck team approach was made to determine if additional changes were suggested as a consequence of peer review at a later stage in the planning process, i.e. was anything missed by reviewing at this earlier stage (post-contouring) in the planning process. For post-planning B and C outcomes it was therefore also documented whether all the changes suggested could have been identified at the postcontouring stage. Analysis of B and C Outcomes After completion of the above prospective data collection, outcomes were analysed and classified according to the types of suggested change. The common themes for B and C outcomes were then used to develop a classification system. Using the A Project Ethics Community Consensus Initiative (ARECCI) Ethics Screening Tool [12] this project was identified as posing ‘minimal risk’ to participants and was exempt from institutional research ethics board review.
Results Outcomes There were 257 case reviews among 209 patients planned by 20 radiation oncologists discussed in 43 quality assurance meetings during the study interval (17 radiation oncologists from TBCC and 3 from smaller centres who participated by video-conferencing). During the study period, 511 radiotherapy courses of the 3 index tumour sites and 969 total radiotherapy courses were started at TBCC. Therefore, 209/511 (41%) of potentially eligible cases underwent peer review during the study period. Of the reviewed plans, 93% were curativeintent and 7% were palliative. A median of 4 radiation oncologists (range 2e8) reviewed each case. 83% of cases were reviewed before delivery of the first fraction of treatment.
3
The overall rate of A, B1, B2 and C outcomes was 78%, 9%, 4% and 9%, respectively, across the 257 case reviews. Table 1 details outcomes overall and by tumour site group. Timing of Peer Review 60 cases were reviewed at the post-contouring stage and 197 at the post-planning stage. In total, 46 patients had peer review at both the post-contouring and the post-planning stage e 40 in the head and neck group and 6 in the breast group. The rates of A, B1, B2 and C outcomes according to the timing of peer review are shown in Table 1. In 56% of C outcomes at the post-planning stage, all of the suggested changes could have been detected after contouring and before plan development (post-contouring stage). When evaluated by tumour site this was 53%, 100% and 57%, respectively, for breast, head and neck and lung disease sites. Analysis of B and C Outcomes B and C outcomes were grouped together for the purposes of providing sufficient numbers to identify common themes. In total, 56 B or C outcomes were given to 55 patients. One patient was assigned a C at the post-contouring review, then a B2 at the post-planning review. In 16 of the 55 patients, two reasons were identified for the B or C outcome, but in most patients (n ¼ 39) only one reason was defined for the B or C outcome. The reasons identified for B or C outcomes were grouped as follows: U, unusual patient anatomy/pathology or comorbidity; T, choice of target volume (including definition of OARs); D, plan design issue; P, lack of policy for specific clinical circumstance; F, dose/fractionation issue; I, indication for treatment. Examples are given in Table 2. The most common reason for B or C outcomes was the choice of the target volume (T) but varied by tumour site (Figure 1). 16 of 18 C outcomes were translated into plan changes following quality assurance review. Of the remaining 2 case reviews receiving a C outcome, in one case the plan was not changed and in the other case it was not clear whether the plan had been changed due to difficulty accessing the planning system at the peripheral centre where the patient was treated.
Discussion Peer review quality assurance across the three selected tumour sites resulted in suggested radiotherapy plan changes (C outcomes) in 9% of the case reviews. These cases could be considered ‘near misses’ where deviation from best practice was detected before having consequences for the patient. This supports peer review contributing towards safer radiotherapy. The purpose of radiotherapy plan peer review is twofold: (i) to reduce ‘errors’ for individual patients and (ii) to create a forum for group discussion, consensus building, communication and treatment policy improvement [13].
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
4
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
Table 1 Number of cases, timing and outcomes of peer review; in total and by tumour site group
Number of patients PC reviews PP reviews Number of patients reviewed at both PC and PP stage Total number of reviews (PCþPP) Percentage of cases reviewed before first fraction delivered Outcomes e PC reviews A B1 B2 C Outcomes e PP reviews A B1 B2 C All outcomes (sum of PCþPP outcomes) A B1 B2 C
Total
Breast
Head and neck
Lung
209 58 197 46 257 80% n ¼ 60 75% 13% 3% 8% n ¼ 197 79% 8% 4% 9% n ¼ 257 78% 9% 4% 9%
85 12* 81y 6 93 66% n ¼ 12 33% 17% 17% 33% n ¼ 81 72% 7% 7% 14% n ¼ 91 67% 9% 9% 16%
73 43 70 40 113 97% n ¼ 43 88% 12% 0% 0% n ¼ 70 90% 7% 1.5% 1.5% n ¼ 113 89% 9% 1% 1%
51 5 46 0 51 92% n¼5 60% 20% 0% 20% n ¼ 46 76% 9% 2% 13% n ¼ 51 75% 10% 2% 14%
PC, post-contouring; PP, post-planning. * Two patients are included who physicians chose to discuss before contouring, both of whom had C outcomes as the overall treatment plan was changed as a consequence of peer review. y Two patients had two PP reviews. One patient had their main plan and then boost plan reviewed on separate occasions. A second patient was reviewed a second time in error but on the second occasion a policy gap was identified (B2).
Table 2 Categorisation of B and C peer review outcomes, including examples for each category Category B/C outcome
% of total B/C outcomes
Example
Unusual anatomy/ pathology/comorbidity (U) Target volume (T) Plan design (D) Policy (P) Dose/fractionation (F)
10
Breast patient with severe pectus excavatum
35 20 16 15
Indication for radiotherapy (I)
4
Coverage of an additional nodal group suggested Choice of number of fields to give better dose distribution Definition of GTV following neoadjuvant chemotherapy for SCLC Lung SBRT case changed from 60 Gy in 8# to 60 Gy in 15# as not meeting heart DVH Treatment strategy changed from radiation to systemic therapy
GTV, gross tumour volume; SCLC, small cell lung cancer; SBRT, stereotactic body radiotherapy; DVH, dose volume histogram.
Individual patients benefit from a ‘team-based’ review of radiotherapy planning decisions, similar to the process adopted by multidisciplinary teams to review treatment decisions for new or challenging cases [14,15]. Peer review can identify policy gaps for specific clinical circumstances. In the current study, there were 10 instances that identified a lack of departmental policy (B2 outcomes). Such cases can flag an opportunity for tumour group consensus building. Outcomes The outcomes classification used was adapted from Lefresne et al. [11] with a subdivision of B outcomes to differentiate policy gap as compared with individual patient
variation issues. Lefresne et al. reported outcomes from a once-weekly, ‘generic’, departmental quality assurance meeting where cases were selected randomly from patients starting in the previous week. The B and C outcome rates were 6% and 1%, respectively. Their process may be a good educational and communication forum and method to introduce regular peer review into a department’s function, but lacks the specificity of in-depth review of most curative or complex plans by site-specific colleagues. Lymberiou et al. [16] also used the Lefresne et al. classification, applied to breast quality assurance rounds only, with 2223 cases reviewed over 2 years. They found rates of A, B and C outcomes of 96%, 2% and 2%, respectively. Multivariate analysis showed a higher likelihood of C outcome for patients undergoing regional nodal irradiation.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
5
Fig 1. B and C outcome categorisation by tumour site group and in total, shown as a percentage of the total number of suggested changes by that group.
Similar to our TBCC group, the authors prioritised discussion of locoregional treatments rather than two field tangents as these were considered most complex and more likely to require changes. Despite the group’s description of ‘well-established treatment protocols’ there were still high proportions of B (55%) and C (44%) outcomes attributed to clinical treatment decisions such as the use of boost and indications for regional nodal irradiation. In our study, there was considerable variation in B and C outcome rates between tumour site groups. This could be related to several factors. The head and neck and lung teams attempted to review all curative-intent cases, whereas the breast team prioritised inclusion of only more complex, locoregional or unusual cases. The number of changes within the head and neck plans was low compared with the other tumour sites. This correlated with well-documented departmental consensus (four radiation oncologists) on normal tissue, target volume and dosimetry constraints. In addition, most head and neck patients were treated with intensity-modulated radiotherapy whereas a wider range of radiotherapy techniques was used in the breast and lung sites, with attendant greater scope for practice variation. There is an opportunity to improve consensus and potentially reduce inconsistencies if the breast and lung groups were to adopt similar contouring and planning constraint policies. Other groups have reported on peer review outcomes in larger populations across all tumour sites. Brundage et al. [13] reviewed 3054 cases and reported a 4.1% ‘not approved’ rate. Ballo et al. [17] reviewed 2988 cases and reported a change recommendation in 12.2% of cases. The number of changes varied by disease site and treating physician [17]. They also found that not only did groups increase uptake of peer review over time but that the number of plan changes reduced over time, possibly related to team standardisation evolving from the peer review discussions. We therefore plan to repeat the study
1 year after adopting the documentation process across all seven tumour site quality assurance groups (this widespread rollout occurred in June 2015). Timing of Peer Review The ideal timing of peer review may be different depending on the disease site. For the duration of the study, the head and neck group changed their practice from reviewing at the post-contouring stage to reviewing at both the post-contouring and the post-planning stage. However, if the plan was already complete before an opportunity for post-contouring review then the whole case would be reviewed in one sitting (a post-planning review). 7 B and C outcomes were defined at the post-planning reviews for head and neck patients and in all cases the suggested changes could have been detected at the post-contouring stage. The head and neck group had a detailed protocol describing acceptable plan quality parameters that could explain the absence of B or C outcomes occurring later in the planning process, and the low rate of B and C outcomes overall compared with the other disease sites. There were 12 B and C outcomes from 113 case reviews of head and neck patients and 11 of 12 (92%) were classified as related to target volume or OAR definition. The only post-planning C outcome in the head and neck group suggested changes that could have been identified at the post-contouring stage. These data support the timing of head and neck case reviews at the post-contouring stage to have the highest yield to detect practice inconsistencies. Given the small numbers, however, it cannot be assumed that a postplanning review would never detect inconsistencies that could not have been identified at the post-contouring stage. The safest approach overall therefore is to review at the post-planning stage or to review at both the postcontouring and the post-planning stage to minimise replanning.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
6
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7
Contrary to the head and neck cases, a high percentage of B and C outcomes would have been missed if the breast and lung groups reviewed only at the post-contouring stage. If the desire was to review each case only once, optimal timing for those sites would be at the post-planning stage. No other reported peer review studies have commented on the timing of peer review, with most opting for review at the postplanning stage to encompass all aspects of radiotherapy planning decision making within one review. Increasing Peer Review in Practice A survey of Canadian cancer centres showed widespread adoption of weekly peer review meetings [10]. However, of the 14 centres surveyed by Brundage et al., only 29% conducted peer review in more than 80% of cases treated with curative-intent and there is therefore considerable room for improvement. In a separate survey, Canadian radiation oncologists felt peer review was beneficial and expressed enthusiasm for such processes, recognising their potential to improve patient care [6]. They listed various resource limitations that they felt impacted on the ability to perform peer review, including protected time for radiation oncologists to attend meetings. Implementing routine peer review in UK centres also presents challenges for clinical oncologists who generally have broader commitments in both systemic and radiation therapy. Despite this, efforts should be made to address this ‘weakest link’ in the planning process [2]. The importance of job planning to accommodate time for peer review has been recently recognised by the Royal College of Radiologists [5]. UK centres have published successful projects implementing regular peer review [18]. In Wolverhampton, weekly quality assurance meetings of non-trial, curative-intent cases resulted in a reduction in ‘major alterations’ to treatment plans (i.e. C outcomes) from 10% to 4.2% over two audits separated by 18 months [14]. In Belfast, radiotherapy plans for radically treated lung cancer patients were reviewed in weekly peer review meetings. Of 122 cases reviewed, a change in a component of the treatment plan took place in 27% of cases as a consequence of peer review [19]. The current study contributed to a cultural change in peer review quality assurance meetings at the TBCC. Documentation of peer review decisions was formalised and greater discipline was introduced into defining a concrete (recordable) outcome of the peer review discussion. The documentation used for the study was adopted in June 2015, by all seven tumour site quality assurance groups for recording their outcomes. Successful peer review quality assurance requires widespread acceptance and a non-threatening, non-judgemental group culture. The focus needs to be on improving care as compared with finding errors. Success requires oncologists becoming comfortable with public review of their own and colleagues’ treatment decisions with the goal to improve care for patients. As a strategy to introduce regular peer review, weekly, department-wide meetings could review a random sample of cases across tumour sites. Once comfort and evidence of
value have been developed, more labour-intensive but detailed disease site-specific rounds could be introduced. Either option can be applied across hospital sites using video-conferencing to enable peer support and shared practice between geographically separate groups [17].
Conclusions This prospective audit of peer review quality assurance outcomes in a Canadian cancer centre resulted in suggested plan changes before the next fraction in 9% of cases reviewed and consistent documentation of outcomes. Outcomes varied between tumour sites. Review at the plan completion time point is suggested as being the optimal timing for peer review. Widespread implementation of such processes in the UK could be guided by existing experience and guidelines. The yield from investment in this area should be improved patient safety and team functioning.
Acknowledgements J. Mackenzie received a clinical research fellowship at the Tom Baker Cancer Centre funded by the Alberta Cancer Foundation, through a bequest from Mrs Helen Thomson. The authors are grateful to the members of the Tom Baker Cancer Centre head and neck, lung and breast quality assurance meetings for their support and cooperation during this project.
References [1] The Royal College of Radiologists, Society and College of Radiographers, Institute of Physics and Engineering in Medicine, National Patient Safety Agency, British Institute of Radiology. Towards safer radiotherapy 2008. https://www.ipem.ac.uk/ Portals/0/Images/Towards%20Safer%20Radiotherapy.pdf. [2] Roques TW. Patient selection and radiotherapy volume definition - can we improve the weakest links in the treatment chain? Clin Oncol (R Coll Radiol) 2014;26(6):353e355. [3] Peters LJ, O’Sullivan B, Giralt J, et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol 2010;28(18):2996e3001. [4] Gwynne S, Spezi E, Sebag-Montefiore D, et al. Improving radiotherapy quality assurance in clinical trials: assessment of target volume delineation of the pre-accrual benchmark case. Br J Radiol 2013;86(1024):20120398. [5] The Royal College of Radiologists. Guide to job planning in clinical oncology 2015. https://www.rcr.ac.uk/system/files/ publication/field_publication_files/bfco153_jobplanning.pdf. [6] Hamilton SN, Hasan H, Parsons C, et al. Canadian radiation oncologists’ opinions regarding peer review: a national survey. Pract Radiat Oncol 2015;5(2):120e126. [7] Lawrence YR, Whiton MA, Symon Z, et al. Quality assurance peer review chart rounds in 2011: a survey of academic institutions in the United States. Int J Radiat Oncol Biol Phys 2012;84(3):590e595. [8] Marks LB, Adams RD, Pawlicki T, et al. Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: executive summary. Pract Radiat Oncol 2013;3(3):149e156.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012
J. Mackenzie et al. / Clinical Oncology xxx (2016) 1e7 [9] Canadian Partnership for Quality Radiotherapy. Quality assurance guidelines for Canadian radiation treatment programs 2013. http://www.cpqr.ca/wp-content/uploads/2013/09/ QRT2015-12-03.pdf. [10] Brundage M, Foxcroft S, McGowan T, Gutierrez E, Sharpe M, Warde P. A survey of radiation treatment planning peerreview activities in a provincial radiation oncology programme: current practice and future directions. BMJ Open 2013;3(7). http://dx.doi.org/10.1136/bmjopen-2013-003241. [11] Lefresne S, Olivotto IA, Joe H, Blood PA, Olson RA. Impact of quality assurance rounds in a Canadian radiation therapy department. Int J Radiat Oncol Biol Phys 2013;85(3):e117ee121. [12] Hagen B, O’Beirne M, Desai S, Stingl M, Pachnowski CA, Hayward S. Innovations in the ethical review of health-related quality improvement and research: the Alberta Research Ethics Community Consensus Initiative (ARECCI). Healthcare Policy 2007;2(4):e164ee177. [13] Brundage MD, Dixon PF, Mackillop WJ, et al. A real-time audit of radiation therapy in a regional cancer center. Int J Radiat Oncol Biol Phys 1999;43(1):115e124.
7
[14] Taplin SH, Weaver S, Salas E, et al. Reviewing cancer care team effectiveness. J Oncol Pract 2015;11(3):239e246. [15] Lamb BW, Brown KF, Nagpal K, Vincent C, Green JS, Sevdalis N. Quality of care management decisions by multidisciplinary cancer teams: a systematic review. Ann Surg Oncol 2011;18(8):2116e2125. [16] Lymberiou T, Galuszka S, Lee G, et al. Predictors of breast radiotherapy plan modifications: quality assurance rounds in a large cancer centre. Radiother Oncol 2015;114(1):17e21. [17] Ballo MT, Chronowski GM, Schlembach PJ, Bloom ES, Arzu IY, Kuban DA. Prospective peer review quality assurance for outpatient radiation therapy. Pract Radiat Oncol 2014;4(5):279e284. [18] Brammer CV, Pettit L, Allerton R, et al. Impact of the introduction of weekly radiotherapy quality assurance meetings at one UK cancer centre. Br J Radiol 2014;87(1043):20140422. [19] Rooney KP, McAleese J, Crockett C, et al. The impact of colleague peer review on the radiotherapy treatment planning process in the radical treatment of lung cancer. Clin Oncol (R Coll Radiol) 2015;27(9):514e518.
Please cite this article in press as: Mackenzie J, et al., Peer Review of Radiotherapy Planning: Quantifying Outcomes and a Proposal for Prospective Data Collection, Clinical Oncology (2016), http://dx.doi.org/10.1016/j.clon.2016.08.012