Improving patient safety and workflow efficiency with standardized pretreatment radiation therapist chart reviews

    Improving patient safety and workflow efficiency with standardized pretreatment radiation therapist chart reviews Kelly Cooper Younge, Katherine Woch Naheedy, Joel Wilkinson, Joumana Dekmak, Elizabeth Covington, Bonnie Durbin, Eric Nelson, Stephanie Filpansick, Jean M Moran PII: DOI: Reference:

S1879-8500(17)30041-3 doi: 10.1016/j.prro.2017.01.015 PRRO 731

To appear in:

Practical Radiation Oncology

Received date: Revised date: Accepted date:

9 December 2016 23 January 2017 30 January 2017

Please cite this article as: Younge Kelly Cooper, Naheedy Katherine Woch, Wilkinson Joel, Dekmak Joumana, Covington Elizabeth, Durbin Bonnie, Nelson Eric, Filpansick Stephanie, Moran Jean M, Improving patient safety and workflow efficiency with standardized pre-treatment radiation therapist chart reviews, Practical Radiation Oncology (2017), doi: 10.1016/j.prro.2017.01.015

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT Improving patient safety and workflow efficiency with standardized pre-treatment radiation therapist chart reviews

RI P

T

Running Title: Implementation of therapist pre-treatment QA

2

Department of Radiation Oncology, University of Michigan Health System, Ann Arbor, MI Department of Radiation Oncology, University of Alabama, Birmingham, AL

NU

1

SC

Kelly Cooper Younge, PhD1*, Katherine Woch Naheedy, MS1, Joel Wilkinson, MS1, Joumana Dekmak, MS1, Elizabeth Covington, PhD2, Bonnie Durbin, BS RT(T) 1, Eric Nelson, BS RT(T) 1, Stephanie Filpansick, BS RT(T) 1, and Jean M Moran, PhD1

CE

PT

ED

Kelly Cooper Younge, PhD Department of Radiation Oncology University of Michigan Health System 1500 East Medical Center Dr. Ann Arbor, MI 48109 Phone: (734) 936-4309 Fax: (734) 936-7859 [email protected]

MA

*Corresponding Author:

AC

Conflict of interest: none

ACCEPTED MANUSCRIPT Abstract Purpose: Therapists play a critical role in ensuring patient safety, however, they are sometimes given

RI P

T

insufficient time to perform quality assurance of a patient’s treatment chart and documentation prior to the start of treatment. In this work, we show the benefits of introducing a formal therapist pre-start QA

SC

checklist, completed in a quiet space well in advance of treatment, into our workflow. Materials and Methods: A therapist pre-start QA checklist was created by analyzing in-house variance

NU

reports and treatment unit delays over a period of six months. Therapists were then given dedicated

MA

time and a workspace to perform their checks within the dosimetry office of our department. The effectiveness of the checklist was quantified by recording the percentage of charts QA’ed prior to

ED

treatment, the percentage of charts with errors needing intervention, and treatment unit delays over a nearly two year time period. The frequency and types of errors found by the pre-start QA were also

PT

recorded.

CE

Results: Through the use of therapist pre-start QA, instances of treatment unit delays were reduced by up to a factor of nine over the first year of the program. At the outset of this new initiative, nearly 40%

AC

of charts had errors requiring intervention, with the majority being scheduling-related. With upstream workflow changes and automation, this was reduced over the period of a year to about 10%. Conclusion: The number of treatment unit delays was dramatically reduced through the use of a formal therapist pre-start QA checklist completed well in advance of treatment. The data collected via the checklist continues to be used for further quality improvement efforts. Introduction Peer review in radiation oncology has emerged as a vital component of ensuring patient safety and accurate treatment delivery (1-3). As the complexity of radiation treatments continues to increase, so

ACCEPTED MANUSCRIPT too does the breadth of knowledge and expertise required to correctly treat a patient. Radiation therapists play a critical role in ensuring safe treatment delivery (4-6), however, the role of therapists in

T

peer review activities has been little discussed in the literature and has primarily focused on therapist

RI P

peer review of themselves at the treatment unit. In this work, we discuss the role of therapist peer review of the radiotherapy treatment chart and activities upstream of patient treatment in the form of a

SC

formal pre-start quality assurance checklist with the goal of improving patient safety and reducing

NU

delays experienced by patients.

A recent ASTRO Safety White Paper discussed the concept of peer review in a radiation oncology

MA

department (2). This paper highlighted the idea that peer review includes the entire radiation oncology team working together and not simply select individuals. Treatment therapists are normally expected to

ED

review a host of parameters and documents before treatment in order to prevent errors introduced

PT

upstream from reaching the patient. However, it has been demonstrated that relying on memory alone, especially with respect to policies, is one of the least effective safety barriers, especially as treatment

CE

complexity increases (7).

AC

In this work, we describe the implementation of a therapist pre-start quality assurance (QA) program applied to all new plans approved for treatment. Prior to the implementation of this new initiative, the therapists at our institution performed quality assurance checks of the patient’s plan and setup documentation on day one immediately prior to treatment. This workflow limited the effectiveness of the therapists’ checks due to the distractions of preparing the patient for treatment. Additionally, if errors or mistakes were detected prior to treatment, the threshold for fixing them was very high because the result was often a delay in treatment for the current patient and potentially future patients. In this work, we use the definition of error from Ford et al.: a “failure to complete a planned action as intended or the use of an incorrect plan of action to achieve a given aim” (8). Some errors studied here

ACCEPTED MANUSCRIPT affected patients due to treatment unit delays, however, none of these errors led to adverse events (i.e. suboptimal clinical outcome). We define a treatment unit delay is any period of downtime at the

T

treatment unit caused by an upstream error, during which the patient and staff must wait for a

RI P

correction to the treatment chart or plan.

SC

Methods

In a previous work (9), we described the use of a semi-automated Eclipse API (application programming

NU

interface) (Varian Medical Systems, Palo Alto, CA) script called Plan Checker used by dosimetrists and

MA

physicists to evaluate treatment plans. This script was designed using departmental treatment unit delays and near-miss and variance reports to tailor the script to our institutional-specific needs. A similar

ED

method was used in the design of the therapist pre-start QA checklist. In-house variance reports were evaluated for errors that were still frequently occurring and causing treatment unit delays after the

PT

upstream workflow change of adopting the semi-automated Plan Checker script. Items chosen for the

CE

therapist pre-start QA checklist were items that the therapists had expertise in reviewing and either those errors that were occurring frequently or those that had a potentially high severity.

AC

Figure 1 shows the checklist that is used by the therapists for pre-start QA. The two columns on the far right labeled “S” for severity and “O” for occurrence have been added to the figure and are not part of the actual checklist. Note that in this analysis, occurrence also inherently includes an error’s detectability as we only included errors that were not caught before reaching the treatment unit in our analysis of in-house variance reports. Each of the items on the checklist is meant to catch a variety of potential errors that could reach this point in the workflow. The severity column breaks down each check into low, medium, and high potential severity of these errors. The second column shows the actual occurrence rate of errors that caused treatment unit delays, based on data recording for six months prior to the initiation of the therapist pre-start QA. Low corresponds to 4 or fewer delays,

ACCEPTED MANUSCRIPT medium to 5-12, and high to greater than 12 (out of a total of 121 reported delays). The combination of severity and occurrence gives the “risk” of a certain error (8). The checklist is ordered for the most

RI P

T

efficient navigation through the workspaces needed in the treatment management system. All of the checks on the therapist pre-start QA are also part of the physics plan check, although some of

SC

the checks are more effective when performed by the therapists. As examples: the therapists are the best judges of setup instruction clarity, and machine scheduling is often not yet completed at the time

NU

of the physics check. Additionally, where feasible, some checks were kept exclusively part of the physics

MA

check to best use our resources allotted to QA tasks.

The department uses the vendor-supplied workflow tool known as the CarePath (Varian Medical

ED

Systems) to define different workflows which are part of the treatment planning and delivery process (9). At the outset of the therapist pre-start QA initiative, a QA task assigned to the Therapist user class

PT

was added to the majority of CarePaths. It was intentionally left out of a few specialized treatment types

CE

(e.g. stereotactic radiosurgery) which have alternative QA methods. The therapist pre-start QA task becomes available as soon as the physics chart review task is completed. Therapists are assigned by the

AC

chief therapist to complete the checks throughout the treatment day. Every effort is made to allow the therapists to check the upcoming patients for their own treatment machine within a few days of the first treatment, so that they can become familiar with patients they will be treating in the near future. To achieve these goals, the therapists use an in-house script that displays all upcoming new start patients, their treatment machine, and color-coded QA status (patient names will appear a different color depending on whether the plan is in dosimetry, physics, ready for pre-start QA, or ready for treatment). Therapist pre-start QA was piloted in April 2015 with the check being recommended but not required. During the initial rollout, a small subset of therapists was trained to complete the QA checklist. A detailed instruction document explaining how to perform each check was provided to improve

ACCEPTED MANUSCRIPT consistency of the checks by different therapists. At monthly intervals, additional groups of therapists were trained to complete the checklist until eventually 18/22 therapists were trained. Based on the

T

effectiveness of the check, the therapist pre-start QA became mandatory prior to the start of treatment

RI P

one year after the initial rollout in April 2016.

SC

At initial rollout, the therapist pre-start QA checklist was completed by the therapists at the treatment unit on an available computer, or in the chief therapist’s office. One year later, we transitioned to having

NU

the therapist pre-start QA performed in the dosimetry office to facilitate communication between the therapists and the dosimetrists. Near the end of 2016, we transitioned from using a word document in

MA

ARIA to a “questionnaire”. This data format in ARIA is mineable and facilitated more efficient collection

ED

of data.

Throughout the first 18 months of the therapists using the pre-start QA checklist, . details on the type

PT

and frequency of errors found during the therapist pre-start QA were recorded. We also tracked

CE

treatment unit delays due to upstream errors before and after the implementation of therapist pre-start QA, and the frequency and types of errors that were not caught prior to treatment.

AC

All errors found by the therapist pre-start QA continue to be analyzed by our departmental multidisciplinary variance team with the goal of catching the error earlier in the workflow or eliminating the error at the source. Results Figure 2 shows the number of therapist pre-start QA checks that were performed monthly during the first 17 months of the program. During this timeframe, the therapist pre-start QA was performed 2,876 times. After the check became mandatory in April 2016, there were only a few sporadic occasions where the QA was missed. The chief therapist restructured how therapists were assigned to complete the

ACCEPTED MANUSCRIPT checklists in order to limit these occurrences. When plans were missed after April 2016 (total of four from May – August), it was most often because the QA task was not available to the therapists until

T

shortly before the patient’s treatment time and no therapist was available to perform the check prior to

RI P

treatment.

SC

Figure 3 shows the percentage of plans that had at least one error requiring an intervention by the therapist performing the therapist pre-start QA. The distribution of the types of these errors is shown in

NU

Figures 4 (scheduling versus plan preparation issues) and 5 (distribution of plan preparation errors). Of note, many of the plan preparation error classes shown in Figure 5 are manual entry or transcription

MA

errors that are followed by a manual (i.e. non-automated) QA check. The majority of all errors found are scheduling related. These errors vary from low severity, such as an eligible treatment machine being

ED

omitted from the scheduling note in our appointment scheduling application, to higher severity, such as

PT

having the patient scheduled on an incompatible machine. An incorrect schedule may lead to a patient missing treatment or a disruption to the schedule for other patients and is therefore an important

CE

metric for us to monitor.

AC

Figure 6 illustrates how delays at the treatment unit were affected by the implementation of the therapist pre-start QA. This graph only includes delays that were caused by upstream errors (i.e. those that had the potential to be caught by the therapist pre-start QA check), not those that originated at the treatment unit. A six to nine-fold decrease in delays at the treatment unit was seen from when the prestart QA checklist was launched until the time when it became mandatory. Although it is difficult to quantify, in the best case scenario, delays to patient treatments could be estimated to lead to a 10 minute delay. In the worst case scenario, delays could be up to a day depending on the type of change required. From October 2014 to November 2016, this represents a reduction from 4.3 hours to 0.5 hours of delay per month in the best case and a reduction from 624 hours to 72 hours per month in the worst

ACCEPTED MANUSCRIPT case. Additionally, in some cases eliminating a delay for one patient also prevents delays that may have propagated to other patients throughout the day should the treatment unit continue to run behind.

T

These reductions in delays are even more impressive considering that on-treatment delays typically

RI P

result in a waste of time experienced by the patient, a team of three therapists, plus any other team

SC

members required to make the correction (such as a dosimetrist, physicist, physician, or scheduler). Discussion

NU

Treatment unit delays were dramatically reduced through the implementation of a pre-start quality

MA

assurance check performed by the radiation therapists (Figure 6). Errors that were previously causing delays were instead more often caught earlier, reducing total treatment unit delay time by up to a factor

ED

of 9 every month. Though additional time is required to be spent in fixing errors no matter at which point in the workflow they are caught, finding the errors earlier allows time for the team to come up

PT

with an acceptable solution in a unrushed environment and avoids wasted patient time and idle

CE

machine time.

The therapist pre-start QA checklist described here was designed based on incident learning reports.

AC

The Safety Profile Assessment developed by AAPM and first released in 2013 highlighted the importance of recording, analyzing, and responding to error and near-miss reports (10,11). Kalapurakal et al. studied voluntarily reported errors from a ten-year time period. Through the use of checklists and time-outs, a significant reduction in many error categories was seen and errors related to wrong patient, wrong site, and wrong dose were eliminated (12). Mazur et al. showed a significant benefit of continuous quality improvement using incident learning and noted that in their study, 24% of errors were caught by therapists (13). Clark et al. also demonstrated the effectiveness of workflow changes designed to address events discovered through incident learning, noting a reduction in both the number of events occurring and the severity of recorded events over the time period of their study (14). As we have in this

ACCEPTED MANUSCRIPT work, Arnold et al. highlighted the importance of tracking incident rates over time to assess the effectiveness of workflow initiatives and process changes (15).

RI P

T

During the initial rollout of this new QA program, ensuring all plans were checked prior the patients’ scheduled treatment time was difficult and required a long ramp-up time during which the scheduling of

SC

the therapist pre-start QA was adjusted multiple times. Once the majority of therapists were trained to perform the pre-start QA, we transitioned to having therapists from each treatment unit review their

NU

own upcoming patient starts and perform the QA on these patients throughout the day during scheduled time slots. This not only increased the compliance with completing the QA on time but also

MA

increased the utility of the QA for the therapists. However, even with this improved scheduling, some treatment charts are still being checked within an hour or two of the treatment start. In the future, we

ED

may need to institute a time requirement to try to limit these types of situations.

PT

Employment of the therapist pre-start QA constituted a significant workflow change for the therapists.

CE

The therapists already have a tremendous number of responsibilities and this checklist requires them to step away from the treatment machine for an extended amount of time to perform the QA. Our

AC

therapists have reported that they have noticed a reduced need to call for urgent help related to plan preparation or scheduling issues. We have found it important to keep the entire team well-informed regarding ongoing efforts to reduce upstream errors to maintain motivation for this program. Providing space in the dosimetry workroom for the therapists to complete their checks has also increased the incentive to continue with the QA. The therapists and dosimetrists have an increased awareness of one another’s workflows and processes, and can easily reach out for help when needed. Dosimetrists have reported value in more frequent communication with therapists while treatment planning. Each member of the radiation oncology team is uniquely qualified to catch certain types of errors. Once equipped with a list of these errors that were the most common causes of treatment unit delays in our

ACCEPTED MANUSCRIPT department, our team worked together to determine the workflow location that would be the most appropriate to check each particular item. The pre-start QA checklist consists only of items that the

T

treatment therapists have a unique ability to check (e.g. whether or not setup instructions are clear),

RI P

cannot be caught reliably upstream (e.g. scheduling that occurs after the physics check), and/or are of

SC

potential high severity (e.g. the treatment plan does not match the prescription). We recognize that adding QA checks for all possible failure modes is not a practical long term solution.

NU

The ultimate goal of any QA program will always be to protect the patient from the consequences of an upstream error. This fact may mean that we must implement downstream error-checking while we look

MA

for ways of improving the upstream workflow. We continue to study our processes looking for ways to eliminate the causes of these errors or to catch them closer to their point of entry. Once this is

ED

accomplished, the number of checks required late in the workflow can be reduced. Help is needed from

PT

treatment planning and management system vendors to make the planning and workflow process more robust to reduce the chances of introducing errors. In treatment planning systems that allow writable

CE

scripting, these data serve as a guide to prioritizing the type of information to automate in the

AC

treatment planning and treatment management systems. Until more checks can be automated, some redundancy between the physics and therapist checks is important in helping to eliminate errors before they reach the patient. For example, Gopan et al. studied the effectiveness of pretreatment physics plan review and found that, during the timeframe that was studied, only 38% of errors that could have been caught by the review were actually detected (16). In fact, our in-house variance reporting system also indicates that both the pretreatment physics plan review and the therapist pre-start QA are prone to missing errors that are later found at the treatment unit. The authors of Gopan et al. suggested automation and standardization of the review process as key elements in helping to improve the checking process. We have automated a number of upstream QA

ACCEPTED MANUSCRIPT checks using our Plan Checker script such as a comparison of the treatment plan to the prescription, reference point doses and dose limits, field naming, and field clearance. This automation has greatly

T

reduced or even eliminated entire categories of errors, and we continue to use data like that shown in

RI P

Figures 4 and 5 to prioritize future improvements to our software tools (17). In particular, scheduling patients correctly is a challenge in our clinic because each treatment unit is unique and not all treatment

NU

increase the accuracy of our patient treatment schedules.

SC

accessories are available on all machines. We are currently working on a software tool that will help us

We note that there are limitations of this study. In almost any voluntary reporting system, not all errors

MA

or delays will be reported. Additionally, the distribution of delay causes will vary between institutions, even though some similarities are likely. This highlights the importance of recording data on delays and

ED

other variances such that QA programs can be individualized where necessary (8).

PT

Conclusion

CE

Adding a formal therapist pre-start QA to our workflow process has dramatically reduced the number of reported treatment delays in our department. Data collected from this checklist will be used to tailor

AC

future QA procedures. We encourage further investigation and reports on this type of QA by other members of the radiation oncology community. Bibliography 1. Zeitman A, Palta J, and Steinberg M, Safety is no accident: A framework for quality radiation oncology and care. Am. Soc. Radiat. Oncol. 2012;1–52, see https://www.astro.org/ProductCatalog/Products/Publications/SafetyisNoAccident/.

ACCEPTED MANUSCRIPT 2. Marks LB, Adams RD, Pawlicki T, et al. Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: Executive summary. Pract Radiat Oncol

T

2013;3:149-156.

RI P

3. Adams RD, Marks LB, Pawlicki T, et al. The new radiation therapy clinical practice: The emerging role of clinical peer review for radiation therapists and medical dosimetrists. Medical Dosimetry

SC

2010;35(4):320-323.

White Paper, 2012. Available Online:

NU

4. Odle TG and Rosier N. Radiation Therapy Safety: The Criticial Role of the Radiation Therapist,

MA

https://foundation.asrt.org/docs/librariesprovider3/pdfs/asrt12_whitepaper.pdf. 5. Watson L and Odle TG. Patient Safety and Quality in Medical Imaging: The Radiologic

ED

Technologist’s Roles. White Paper, 2013. Available Online: https://www.asrt.org/docs/librariesprovider3/pdfs/asrt13_whitepaper.pdf.

PT

6. ASRT Foundation: Commitment to Succeed, 2013. Available Online:

CE

https://www.asrt.org/docs/librariesprovider3/pdfs/cr13_comtosucceed_radther.pdf. 7. Marks LB, Jackson M, Xie L, et al. The challenge of maximizing safety in radiation oncology. Pract

AC

Radiat Oncol. 2011 Jan-Mar;1(1):2-14. 8. Ford EC, Fong de Los Santos L, Pawlicki T, et al. Consensus recommendations for incident learning database structures in radiation oncology. Med. Phys. 2012;39(12):7272-7290. 9. Redacted prior to submission. 10. Dunscombe, P, Brown, D, Donaldson, H, et al. Safety profile assessment: An online tool to gauge safety-critical performance in radiation oncology. Pract. Radiat. Oncol. 2015;5:127–134. 11. Ford E, Brown D, Donaldson, H, et al. Patterns of practice for safety critical processes in radiation oncology in the US from the AAPM safety profile assessment survey. Pract. Radiat. Oncol. 2015;5:e423–e429.

ACCEPTED MANUSCRIPT 12. Kalapurakal JA, Zafirovski A, Smith J et al., A Comprehensive Quality Assurance Program for Personnel and Procedures in Radiation Oncology: Value of Voluntary Error Reporting and

T

Checklists. Int J Radiat Oncol Biol Phys. 2013;86(2):241-248.

RI P

13. Mazur L, Chera B, Mosaly P, et al. The association between event learning and continuous quality improvement programs and culture of patient safety. Pract. Radiat. Oncol. 2015;5:286–

SC

294.

NU

14. Clark BG, Brown RJ, Ploquin J, and Dunscombe P, Patient safety improvements in radiation treatment through 5 years of incident learning. Pract. Radiat. Oncol. 2013;3:157–163.

MA

15. Arnold A, Delaney GP, Cassapi L, and Barton M. The use of categorized time-trend reporting of radiation oncology incidents: A proactive analytical approach to improving quality and safety

ED

over time. Int. J. Radiat. Oncol.,Biol., Phys. 2010;87:1548–1554. 16. Gopan O, Zeng, J, Novak A, et al. The effectiveness of pretreatment physics plan review for

PT

detecting errors in radiation therapy. Med. Phys. 2016;43:5181.

Figure Legends

AC

CE

17. Redacted prior to submission.

Figure 1: Checklist used for therapist pre-start QA. The “S” and “O” columns have been added for the figure only. The first column labeled S represents the severity of such an error not being caught prior to treatment, and the column labeled O represents the actual occurrence rate of the error between July 2014 and February 2015 prior to checklist implementation (see text). The first row of the checklist allows the therapist performing the check to notate to which plan or plans the checklist corresponds, and the remaining rows are the checklist items.

ACCEPTED MANUSCRIPT Figure 2: Number of therapist pre-start QA’s completed and missed per month since the initial pilot of the program. Shaded area represents time period where the QA was mandatory.

RI P

T

Figure 3: Percentage of charts needing intervention due to an error found at the time of therapist prestart QA.

SC

Figure 4: Number of errors found during therapist pre-start QA, separated into scheduling versus plan

NU

preparation issues.

MA

Figure 5: Distribution of plan preparations errors found via therapist pre-start QA. Figure 6: Number of events causing a treatment delay per month before and after the implementation

AC

CE

PT

ED

of therapist pre-start QA.

AC

CE

PT

ED

MA

NU

SC

RI P

T

ACCEPTED MANUSCRIPT

Fig. 1

AC

CE

PT

ED

MA

NU

SC

RI P

T

ACCEPTED MANUSCRIPT

Fig. 2

AC

CE

PT

ED

MA

NU

SC

RI P

T

ACCEPTED MANUSCRIPT

Fig. 3

CE AC

Fig. 4

PT

ED

MA

NU

SC

RI P

T

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

MA

NU

SC

RI P

T

ACCEPTED MANUSCRIPT

Fig. 5

AC

CE

PT

ED

MA

NU

SC

RI P

T

ACCEPTED MANUSCRIPT

Fig. 6