Competency in Quality: Defining the Scope and Nature of Quality Competencies for Radiation Oncology Residency Programs

Journal of Medical Imaging and Radiation Sciences

Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146

Journal de l’imagerie médicale et des sciences de la radiation

www.elsevier.com/locate/jmir

Research Article

Competency in Quality: Defining the Scope and Nature of Quality Competencies for Radiation Oncology Residency Programs Caitlin Gillan, MRT(T), BSc, MEd, FCAMRTab*, Kathy Wan Ting Yip, BSc, MRT(T) (MR)a, Jenna Adleman, MD, MScab and Meredith Giuliani, MBBS, MEd, FRCPCab a

Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Canada b Department of Radiation Oncology, University of Toronto, Toronto, Canada

ABSTRACT Introduction: Quality and safety in radiation treatment have garnered increasing attention in recent years. With the introduction of the CanMEDS 2015 Physician Competency Framework, incorporation of patient safety and quality improvement will be required across all seven established physician roles for radiation oncology residents. An appreciation for the competency areas relevant to radiation oncologists (ROs) in the quality and safety domain is thus needed to inform training in this area. Methods: Semistructured interprofessional focus groups were held with ROs, medical physicists, and radiation therapists to ascertain the scope of quality principles required of newly certified ROs, to identify current teaching best practices, and to define required competencies in this area. Audio recordings were transcribed verbatim and data analyzed iteratively and coded using a constant comparison method. Results: Three focus groups were held with 20 participants overall, and an average duration of 68 minutes (range 47–81 minutes). Participants found it difficult to define quality but noted that for residents it might encompass competencies in peer review, incident and change management, and quality culture. Although addressed in various ways in current residency programs, it was thought that explicit acknowledgment of relevant ‘‘nonmedical expert’’ quality competencies would ensure adequate attention in residency. Conclusions: Quality and safety are important concepts in radiation oncology, warranting attention in residency training to develop the knowledge, skills, and behaviour necessary in practice.

RESUM E Introduction : La qualit
e et la s
ecurit
e des traitements de radiation ont suscit
e une attention croissante au cours des dernieres ann
ees.

Avec le d
evoilement du Cadre de comp
e tences CanMEDS 2015 pour les m
e decins, l’incorporation de la s
ecurit
e du patient et de l’am
elioration de la qualit
e devient une exigence dans les sept r^oles du m
edecin pour les r
esidents en oncologie. Il est donc n
ecessaire de comprendre les champs de comp
etence dans le domaine de la s
ecurit
e et de la qualit
e qui son pertinents pour les radiooncologues afin d’
eclairer la formation dans ce domaine. M
ethodologie : Des groupes de discussion interprofessionnels semistructur
es ont
et
e tenus avec des radio-oncologues, des physiciens m
edicaux et des radioth
erapeutes afin de recenser les pratiques exemplaires actuelles en enseignement et d
efinir les comp
etences requises dans ce domaine. Les enregistrements ont
et
e transcrits mot a mot et les donn
ees ont
et
e analys
ees de fac¸on it
erative et cod
ees au moyen d’une m
ethode de comparaison constante. R
esultats : Trois groupes de discussion ont
et
e tenus avec 20 participants au total, pour une dur
ee moyenne de 68 minutes (plage de 47 a 81 minutes). Les participants ont trouv
e difficile de d
efinir la qualit
e, amis note que pour les r
esidents cela pouvait comprendre des comp
etences en r
evision par les pairs, en gestion des incidents et du changement et en culture de qualit
e. Bien que ce soit abord
e de diff
erentes fac¸ons dans les programmes actuels de r
esidence en m
edecine, les participants croient qu’une reconnaissance explicite des comp
etences en qualit
e « autres que celles d’un expert m
edical » permettrait d’accorder a cette question une attention ad
equate dans le programme de r
esidence. Conclusions : La qualit
e et la s
ecurit
e sont des notions importantes en radio-oncologie et m
eritent qu’on y accorde suffisamment d’attention dans les programmes de r
esidence pour permettre le d
eveloppement des connaissances, des comp
etences et des comportements n
ecessaires dans la pratique.

Keywords: Quality and safety; competency; training; radiation oncology residency

* Corresponding author: Caitlin Gillan, MRT(T), BSc, MEd, FCAMRT, Radiation Medicine Program, Princess Margaret Cancer Centre, 610 University Avenue, Toronto, Ontario, M5G 2M9, Canada.

E-mail address: [email protected] (C. Gillan).

1939-8654/$ - see front matter Ó 2016 Canadian Association of Medical Radiation Technologists. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jmir.2016.01.004

Introduction Quality and safety in radiation treatment have garnered increasing attention in recent years. Highly publicized incidents [1, 2], the overall complexity of cancer care [3], and a focus on personalized, but evidence-based practice [4] have prompted a number of valuable initiatives to optimize the quality and safety of radiation treatment for all Canadians. There is growing recognition that poor quality radiotherapy can lead to poor quality outcomes [5] and that integration of quality programs, metrics, and standards into everyday practice may be as important as other advancements in the treatment of cancer. Although many initiatives relate directly to the delivery of care, such as the activities of the Canadian Partnership for Quality Radiotherapy and efforts of individual and regional cancer programs to standardize care and formalize peer review, there has been little attention to these considerations in precertification education. Lessons learned in other industries with a more established mechanism for training in quality and safety, such as the airline industry [6], are increasingly suggesting that this topic warrants consideration in radiation oncology. With the introduction of the CanMEDS 2015 Physician Competency Framework, the Royal College of Physicians and Surgeons of Canada articulates the need to incorporate patient safety and quality improvement across all seven established physician roles [7]. This charges each individual specialty with the responsibility to revise its own competency profiles and program curricula to ensure this new focus is addressed. Using an approach that proved successful in developing an imaging competency profile for newly certified radiation oncologists (ROs) [8, 9], a collaboration was undertaken between educators in radiation medicine disciplines and the Canadian Partnership for Quality Radiotherapy to define the elements of competency inherent to quality in radiation oncology. The first step in this process was to seek an appreciation for the competency areas relevant to ROs in the quality and safety domain, through consultation of experts in this area. This article reports on phase I of a larger study and serves as a needs assessment. Resultant themes will be included in a broader environmental scan for potential competencies, and then reviewed in a consensus-building exercise to develop a final comprehensive competency profile. Methods Qualitative Method Interprofessional focus groups were employed to generate a broad preliminary picture of the competencies required of ROs. As a need assessment, intended to identify an inclusive set of topics to be considered in subsequent phases of this study, no qualitative framework for theory generation or testing was used. Focus groups were selected over interviews as the qualitative tool of choice because they afforded the opportunity to engage a diverse set of viewpoints, highlight 140

areas of agreement and dispute regarding professional responsibilities in quality, and allow emergent themes to be explored and elaborated with minimal involvement from the moderator, relying instead on participants’ ideas inspiring contributions from others. Focus groups are deemed an appropriate choice when the subject matter is not particularly sensitive in nature, and participants are not likely to feel intimidated or unduly influenced within the broader group structure [10, 11]. Study Population A purposive sample of radiation medicine professionals involved in the technical delivery of radiation therapy care was considered for inclusion. These included ROs, radiation therapists (RTTs), and medical physicists (MPs). As per professional designations in Canada, RTTs included treatment planning (dosimetry) responsibilities and roles. Inclusion was limited to these professional groups as the three main professions involved in the technical delivery of radiotherapy care. Although nursing, social work, and a spectrum of other professional groups are integral to the broader cancer patient journey, quality and safety in the planning and delivery of radiotherapy presents a unique set of considerations, which were the desired focus of this investigation. The inclusion criterion was that participants either have unique insight or roles in quality management or leadership, or in trainee education. Select residents and fellows were chosen based on an interest in quality issues or perceived insight into their academic programs. Potential participants were chosen from among those within a single radiation treatment program at an academically affiliated institution (group A) or those scheduled to attend the Canadian Organization of Medical Physicists’ Canadian Winter School (www.comp-ocpm.ca/winter-school), focused on quality issues in radiation medicine (groups B and C). The context of the latter two groups maximized the opportunity to ensure broad geographical representation and participants from both major urban academic health centres and smaller regional cancer centres. Data Collection Institutional Research Ethics Board approval was granted before recruitment. An e-mail invitation to participate in the study was sent to potential participants. Efforts were made to accommodate all who responded. Representation of profession was considered in the composition of scheduled focus groups and, as per optimal focus group guidelines, a maximum of eight participants were scheduled in each group. Focus groups were held at a mutually agreed on time in the case of group A, or at predetermined times during scheduled breaks in the conference (groups B and C). Each focus group was led by the same trained investigator, an RTT. Focus groups were semistructured and consisted of guiding questions designed to: ascertain the scope of quality principles required of newly certified ROs, identify current teaching best practices, and define required competencies in this area. Although an interview script was used that articulated those

C. Gillan et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146

questions (Table 1), clarification was provided as necessary and there was flexibility in facilitation and direction pursued by the facilitator, depending on the natural flow of each session. Questions and probes were modified for subsequent interviews based on emergent themes. Field notes were not taken by interviewers. Audio recordings were transcribed verbatim by one of the investigators. Data Coding and Analysis Data were analyzed iteratively by two investigators and discussed regularly with the broader research team. Coding of transcripts and identification of themes were done manually using highlighting within a standard word processor. Provisional thematic categories were identified throughout data collection by way of the constant comparison method with each new focus group as it was transcribed (aligning newly emerging data with categorization of earlier data, and iterative modification of groupings based on saturation of individual themes) [12]. Further refinement of themes and specific competencies was based on concurrence between investigators, which was achieved through independent review of aggregated data and group meetings. Assurance of Trustworthiness Measures were taken to reduce bias and ensure trustworthiness in qualitative data. A single, trained moderator facilitated all three sessions. Transferability of data was ensured through representation within focus groups of multiple professions, Canadian jurisdictions, and cancer centre sizes. Transcripts were reviewed independently by the two primary investigators with notes kept and compared on decisions made regarding initial thematic analysis. The fact that the primary reviewers were both RTTs is a consideration with regard to reflexivity, because data were interpreted initially through the lens of this professional group. Review by the broader research team (two Table 1 Guiding Interview Script

ROs), one of whom had been present as an observer at two focus groups and the other who had not been present at any focus group, as well as by four study participants – one RO, two RTTs, one MP (representing two focus groups) contributed to the credibility and reflexivity of data. Confirmability, or objectivity, of data analysis, while addressed through independent and iterative data review, was not considered a major potential for bias in this investigation, since the intention was to bring to light themes for debate in a subsequent consensus-building exercise, not to assign meaning to participants’ contributions to the focus groups. Results Three focus groups were conducted during a two-week period. The composition of each and convention for coding participants is summarized in Table 2. There was a total of 20 participants (seven ROs, one RO fellow, six RTTs, five MPs, and one MP resident), with a balanced representation of male and female (11 and nine, respectively), as well as distribution from across Canada (representing 10 centres and five provinces). Focus groups were an average of 68 minutes (range, 47–81 minutes) in duration, with an average of w7 participants (range, 3–9). Although initial composition of focus groups was more balanced than what eventually took place, last minute scheduling conflicts for participants led to a less optimal distribution of numbers and professional groups, primarily noted in group 3. Saturation of themes was achieved nonetheless, and fluid participation of all participants in all groups was deemed sufficient to justify acceptability of resultant group composition for the purposes of this investigation. Participants discussed topics that fell under two overarching areas, the definition and scope of the term ‘‘quality’’ with respect to competencies in radiation oncology residency and specific quality-related content areas that might be addressed in a competency profile. Scope of Quality

What would you consider to be encompassed in the scope of ‘‘quality and safety’’ for a newly certified radiation oncologist? Probes (based on emerging discussion): - What might better fit in another profession’s scope of practice? - What is better suited to subsequent continuing medical education training, postcertification? Can you identify and define specific competencies that might fall within this scope? Probes (based on emerging discussion): - At what level must each of these be explored within residency? - Should any specific quality-related principles, frameworks, or tools be explicitly referenced in a quality and safety competency profile? Are quality and safety currently being considered formally in training programs, and how is competency being assessed? Probes (based on emerging discussion): - Is the necessary formal and specific attention being paid to these competencies? - Does jurisdiction or size of a radiation treatment program impact on the ability to address certain competencies? - What considerations are involved in effectively teaching and assessing these nonmedical expert competencies?

Participants found it difficult to establish a concise definition of the nature and scope of ‘‘quality’’ in the practical setting of radiation oncology residency but felt it was important to do so to appreciate the competencies necessary to be proficient as ROs. As noted by one MP, Table 2 Focus Group Composition and Coding Focus Group

RO

MP

RTT

1

RO0102 RO0101

MP0101 MP0102

8

2

RO0201 RO0202 RO0203 RO0204 RO0301 RO0302 8

MP0201 MP0202 MP0203

RTT0101 RTT0102 RTT0103 RTT0201 RTT0202

RTT0301

3

3

5

C. Gillan et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146

6

Total

9

20

141

‘‘everybody can use the word ‘quality’ . and everyone starts talking about it, but nobody stops and wonders whether we’re talking about the same thing or not’’ (MP0101)

But. there’s no appreciation of quality beyond my personal work and maybe that’s what’s not in anybody’s curriculum’’ (RTT0101)

In one focus group, familiarity with the work of Edwards Deming and the Institute of Medicine’s To Err is Human [13] and Crossing the Quality Chasm [14] led to an academic discourse on the term. Quality was considered to encompass care being free of error, consistent, evidence-based, equitable, effective, efficient, and patient-centred. An example of the debate that allowed the group to flesh out the appreciation of the term is included here:

Overall, the expectation of residents was that they recognize the potential impact of poor quality in aspects of radiation medicine practice, such as inaccurate calibration of a linear accelerator, inaccurate delineation of targets, lack of a robust system for peer review of treatment plans, and ‘‘how variations in that individual quality will impact care at the end’’ (MP0103). This would necessitate ‘‘being respectful of the process.. Because whatever you are doing will (have an) impact down the line’’ (MP0202). As noted by an RO, ‘‘. the ability to identify nonquality or quality states is probably what we want to be able to do..’’ (RO0101). The desire for residents to perceive quality measures as important to practice was acknowledged insofar that a newly hired staff RO at a new centre could recognize and address a lack of quality. Although an appreciation of quality considerations was believed to be fundamental to competent radiation oncology practice, concern was raised by some ROs in the groups about the scope of its inclusion in formal residency programs. This related primarily to time limitations and concerns regarding ‘‘introduc(ing) them to too many obligatory competencies’’ (RO0202). It was believed that some topics raised as being relevant to quality, such as change management practices, would be better left for leadership programs beyond residency. Lack of consistency of quality management practice across Canada would also pose a challenge in standardizing a competency profile as, in some programs, ‘‘we want the exposure, and yet at the same time it is tough to get it’’ (R0203). It should be noted that the concept of safety was alluded to as a related but distinct element of quality in that safety would fit under the broad umbrella of ensuring quality practice. Many specific concepts were thought to be more generic safety considerations, such as patient identification and fall risks, whereas some were felt to be unique to radiation therapy, such as pacemaker and pregnancy status. Although acknowledged tangentially in all groups, discussion specific to ‘‘safety’’ was minimal, and the focus remained on the concept of quality as it related to radiation medicine.

MP0102dTaking the context as a whole, if you focus solely on making the patient experience a pleasant one, perhaps you would incur costs elsewhere in the system that may tip the balance. MP0101dKind of like ALARA, right? RTT0101dIt’s just that the word ‘‘quality’’, to me, doesn’t. MP0101dIf you go to the Institute of Medicine, under consistency–they have the evidence based stuff.. executiondit’s effective, it’s efficient, subject to constants of the system. RTT0103d. Like a service sector type definition of quality. There is a big emphasis, you know – quality is defined by the customer. But there is a couple of big differences. with public healthcare. they can’t actually judge quality the same way they can, like buying an umbrella. If they buy an umbrella, you can decide if you got a good price etc, it functions well and you like the colour.but when you get a course of radiation treatment, how do you know that we made a good treatment plan or that we follow up with your side effects well, or that we coordinated things behind the scenes in an efficient way? You can’t. RO0101dBut it’s becoming much more common. RTT0103dBut by and large a patient comes through here and has a course of breast treatment, and is going to judge that experience based on their perception of what that experience should be like, which is largely informed by us. MP0101dNo, it’s informed by all the healthcare they have had leading up to this point. RO0101dAnd their friends’ healthcare and family healthcare and anyone who has seen healthcare. It’s the same process.but like you’re buying a version of radiation is not like you’re buying an umbrella – I agree.

All groups believed quality-related topics were important to address in residency, but also that they were not currently given attention as topics unto themselves. It was felt important to highlight quality as being separate from being medically competent. As noted by an RTT, ‘‘I think that competencies are designed by nature to describe what an individual is doing. how an individual performs. In each of those competencies is an inference of quality that it meets some standard of quality against the gold standard .

142

Competency Categories Most of the time in all focus groups was spent discussing particular aspects of quality in radiation medicine that might be relevant to a residency competency profile. The four main emergent themes, identified and explored by all three groups independently were: peer review, incident management, change management, and quality culture. Peer Review Peer review of radiation treatment plans was the most often-addressed quality topic in each focus group, especially by RO participants, as it was seen to fit squarely within their scope.

C. Gillan et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146

‘‘Residents need to have some sort of competency in how to participate in peer review. they need to understand foundational principles about what are the steps to doing effective peer review–is the coverage right? is the dose right? . The other piece of this is what is the importance of peer review? Why is it a key component of an effective radiotherapy program? That would be, in my mind, another one of the pillars’’ (RO0201)

Two specific considerations were mentioned that extended beyond the technical clinical aspects of peer review: an understanding of the variability in format for effective peer review and the appropriate professional conduct when engaging in peer review. Much of the discussion was framed around what might constitute ineffective peer review, and how the establishment of competencies for ROs might mitigate this in future practice. It was believed to be important to be ‘‘aware of the spectrum of peer review that is out there, and then. if you are in a place that you do not think is meeting the bar, then you need to make a change, right?’’ (RO0102). Specifically, ‘‘How does the peer review process occur? Who is choosing the cases that are being selected for peer review? Is it one person who is delegated? Does that person rotate? Is it only a certain person’s cases that are being reviewed all the time? You know, if there’s implications about that, and is that the right format to do that? . Oftentimes times are limited, and so you have an hour for lunch rounds to get through the cases and it’s like how many cases per person, is it going to be radical, is it every radical? . How does that process get selected?’’ (RO0301)

Communication skills were thought critical, especially in the provision and response to feedback. ‘‘What do they do when a physician offers them suggestions on a plan? If they disagree, what do they do in that instance?’’ (RTT0202). Being constructive and collegial and valuing the role of peer review in improving quality were seen as being as important as the technical skills relating to identifying poor plans. One RO, a residency program director, noted that participation in peer review rounds was expected of residents in his centre, and was evaluated based on appropriate engagement; ‘‘We try to evaluate our resident participation in rounds by saying can this resident . recognize when there is reasonable variation? . Can they come up with constructive ways of modifying the plan or suggesting change in such a way that it can be received effectively, or they can actively communicate that task or that message in a non-threatening way or a productive way?’’ (RO0204).

Incident Management Exposure to the management of treatment-related incidents was thought by all participants to be important. Discussion focused on two related elements: management of the individually-affected patient and broader organizational incident investigation and learning.

Disclosure of an incident to the patient was seen to sit squarely within the scope of practice of the RO, and as one participant noted, ‘‘if you’ve not been inculcated as a trainee, then you’ve not developed the relevant skills that you’ll need to do it’’ (RO0202). Given the infrequent and adhoc but sensitive nature of incident disclosure meetings with patients, the RO participants made mention of the lack of opportunity to be a part of such events as trainees. Decisions around the need for disclosure in a given case, and also the nuance of delivering such news without assigning blame, are also important. An RTT noted the considerations relating to ‘‘When do you disclose? Do you disclose when somebody leaves a tray in the machine and that’s 1%. would you tell the patient? That type of thing’’ (RTT0202). Also within the management of a given patient, it was felt that trainees required exposure to considerations regarding specific harm to a patient and changes in their management or care plan as a result of an incident. ‘‘Because it would be something they’d be expected, in their first job, in their first clinic. to identify whether it was, you know, acute harm, latent. no one else can tick that box or make that call except the physician, and it shouldn’t be the first time that they see it, or the first time they have to ponder that idea.’’ (RTT0301)

A broader appreciation of the process of investigation was noted by all groups to be something residents could learn through observation or participation in an incident investigation. One RO observed that ‘‘as trainees we’re often protected from these kind of incident management situations. protected from being kind of at the front line’’ (RO0202) despite the possibility of including a trainee in a formal incident investigation, even as an observer, as a learning exercise. This relates both to the steps in an investigation process, and also the hallmarks of a collaborative and just culture. ‘‘I think they need to know, first of all, the importance of incident reporting. That it’s important to identify not only incidents but the near misses. It’s a learning opportunity, so they need to grasp the importance of it. the culture of safety, and being able to report an incident in a blame-free environment.’’ (RO0302) ‘‘Knowing how to report the incident to your centre. I think it’s amazing to come to a root cause analysis – that may not be possible, but it’s a great process to go through. I think it would be a fantastic aspect to have in the training, to understand the whole entire process that we’re going through’’ (RTT0202) ‘‘Who are the people you go to and what are the steps. some general understanding. of the sort of algorithm of people you need to involve at certain points’’ (RO0301) ‘‘I’m wondering if there’s a distinction here. You know, if we’re talking about errors, residents probably do need to have some fundamental knowledge of what are the basic steps in an error report’’ (RO0201).

C. Gillan et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146

143

Some conversation related to gaps in communication of incident investigations at different centers, and the importance of engaging trainees as well as staff: MP0201dWhat is the structure for the group to be reviewing incidents, and how is that going to be communicated back? We’re doing this in [major urban centre], but I never hear the results of one of these things. And I’m aware of some of the incidents that occur, so. how do I go back and find out? MP0202dI don’t know whether that is institution dependent, but I am aware that at my cancer centre – we’re a small centre–and we follow the [Canadian Partnership for Quality Radiotherapy] recommendations and have a QC committee which. handles these noncompliance cases and these investigations of incidents. MP0201dI’m aware that we have this committee, but I’ve never seen the breakdown of a failure mode, or communication and the formalized structure for this to actually occur I think would ideally be included in your training so that you would be able to set one of these up. MP0202dAnd that feedback, to speak to your point there– trainees need to be exposed to that. RO0202dAnd there’s also some fact-based knowledge in there, which I think when we’re talking about this in terms of what we need to put into curriculum, and potentially something that you examine on. in the UK we were examined on–you [are presented] an incident and [asked] what you would report on..

Change Management Change management is a topic that was touched on in all groups. Although few participants framed it as a specific term or topic area, there was some debate regarding the appropriateness of devoting time to it in a residency program. At its most basic level, an appreciation for how to address shortcomings in the system or introduce novel treatment techniques was felt to be valuable. ‘‘Change management . how do I work this system? My program, how do I capitalize on the resources available to me, or get them available to me, to be able to do IMRT for head and neck when we’re not doing it, or be able to do this, or to improve the peer review system?’’ (RO0102)

At least one participant in each group referred to a situation where a newly certified RO might be hired in an unfamiliar centre, and immediately recognize a need for change, be it an outdated or unsafe practice, or a lack of a formal peer review framework. Possessing the skills to contribute to change in such a situation was highlighted in each case. From the perspective of one RO, ‘‘we teach them how to recognize the aspects of quality, such as peer review, but if they recognize that it’s not there but they don’t know how to implement the change we’ve only created frustration.[they need to be] able to advocate for that change and know. how to do it in a way that’s professional’’ (RO0302) 144

In one of the groups, there was a suggestion that such skills extend beyond the expectations of the Manager role (referring to CanMEDS). Although there was support for this statement in that more technical medical competencies need to be prioritized within a time-bound training period, it was also noted that such skills were in increasingly greater need among more junior ROs. ‘‘I see it in my colleagues who don’t have that. at what point are they going to adopt quality management activities or participation in peer review, or buying into the development of these protocols? .so if we had some way of exposing our residents to it, at least it may break down some of the barriers to adopting it’’ (RO0204)

In a point raised by an RTT in that same group, ‘‘there are many centres that are not large, massive centres, and so we’re seeing.people move into being head of department within five years of practice. or who are moving to program director positions, which is still a leadership position, in the first few years of practice. And if you’re not exposed to these concepts, but will be using them in your leadership role..?’’ (RTT0201)

Quality Culture Although also a nebulous topic within the focus groups, the concept of a quality culture was a theme interwoven in all discussions. This encompassed interprofessional collaboration, lifelong learning, and collectively striving for a high level of quality in practice. Much of the conversation related directly to other themes, such as approaching peer review and incident management in a professional manner, as ‘‘just having a pleasant and friendly work environment reflects on the way we deliver care’’ (MP0102). One RO mentioned palpable tension in a room where peer review was not executed well, and it was not that the appropriate technical elements were not discussed, but that clinicians were not giving or receiving feedback in a constructive manner. Engaging trainees in the development of necessary communication skills for that type of context would facilitate a less hostile environment. As she noted, ‘‘both sets of rounds addressed useful issues and quality issues, but one functioned well and one was so bad I left’’ (RO0201). The need was expressed for residents to develop skills and insight required to adapt to evolving practice after certification and to seek out the necessary competencies to practice safely and effectively. An RTT articulated well the relationship between quality and possessing such lifelong learning skills. ‘‘It’s like building the culture of continuous quality improvement, right? It’s kind of like continuing medical education; you’re constantly improving yourself. And so my understanding is that for physicians and for others, you do that as a selfregulating profession – you take it on yourself’’ (RTT0201)

To be able to do this, an element of self-awareness was considered desirable.

C. Gillan et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146

‘‘Awareness of the gaps in one’s knowledge. Or awareness of a need for CME [continuing medical education]. That is quality. To recognize that you’re going to be 10, 20 years out in practice and realizing that there are new techniques that you’re unfamiliar with. Quality means that you will seek the means to learn about it.’’ (RO0302)

Discussion Quality and safety are important areas to address in a radiation oncology residency, and extend beyond technical competency in medical tasks and roles. Although quality care relies heavily on technical proficiency, it cannot be separated from the broader appreciation for individual, organizational, and systems assurance of high standard, safe, and evidence-based care. Relevant formal curricular elements will be mandatory with CanMEDS 2015 [7]. Peer review and incident management emerged as clearly definable areas of potential focus for residents, whereas other areas, such as change management and quality culture, were less readily articulated as individual topics. Much of the discussion in the focus groups in this study devolved into examples of poor performance in quality activities rather than the definition of desirable competencies to equip residents to engage in these activities effectively. Peer review and incident management are areas of practice that are attracting increasing attention in the radiation oncology community [15, 16], and the value placed on them in the focus groups is thus not unexpected. Exposure of trainees to these rounds varies, with one 2013 study in Ontario, Canada noting that of the 14 radiation treatment centres in the province, only 3 (21%) self-reported that trainees regularly attended peer review rounds [17]. It is expected that this number is increasing given the increasing attention in the community to the value of peer review. However, any expectation of competency during a clinical rotation tends to be around the clinical presentation of an individual case rather than the nuances of effective peer review, constructive feedback, and communication skills. Formally requiring of residents an appreciation and proficiency in the complete peer review experience can support broader and stronger embedding of this important quality measure in future radiation oncology practice. By definition, treatment incidents are ad hoc and often time sensitive events; and the resident would rarely be the first point of contact in such an instance. For that reason, and perhaps also because of intentional exclusion of unnecessary individuals, it could be expected that many residents might never be exposed to incident management in the course of their training, either in the initial management of the individual patient, or in any formal investigation that might follow. As noted in this study, however, considerations around disclosure, treatment plan modification, and broader systems repercussions will fall under the purview of even the most junior RO, and thus warrant attention in training. Initiatives in other fields have focused on standardized activities rather than integration into actual incident investigation and

management, often through simulation or reflective practice [18, 19]. Formal acknowledgement in a competency profile or residency curriculum might prompt programs to similarly develop relevant clinical simulation scenarios, encourage resident inclusion in disclosure consultations, and even create the expectation that residents be members of incident investigation teams. Change management and quality culture were less concrete concepts that arose in this study, but despite potential difficulties in assessment, competence in some of the softer skills encompassed in these areas can be seen as necessary in certifying ROs [20]. Interprofessional and collaborative practice, communication, and the need to recognize and address poor practice to establish evidence-based care were all acknowledged to be relevant to the overarching themes of quality and safety, and many of the terms and concepts raised in this investigation mirror calls to action made in the radiation oncology literature within the last decade [21]. Implementation of continuing medical education initiatives for staff ROs related to quality improvement have been shown to lead to stronger adherence to protocols and improved patient management, stemming from a focused appreciation for the relevance of current evidence to practice improvement [22]. Tess et al (2015) note that ‘‘having a culture that prioritizes quality and safety and that values the role of the trainee in the continuous improvement process’’ is critical to the ongoing delivery of quality care [23]. Participation in formal quality-related activities, such as the aforementioned root cause analyses and disclosure were seen to be one approach to indoctrinating trainees to a quality culture. Interprofessional engagement in quality improvement activities and practice change were also acknowledged, although specific competencies in these areas were not defined. Just as the CanMEDS 2015 Physician Competency Framework weaves quality and safety within the established roles of medical expert, collaborator, leader, communicator, health advocate, scholar, and professional [7], related competencies should not be considered independently of existing residency requirements. Explicit reference to quality and safety competencies can nonetheless ensure these areas are sufficiently explored as part of achieving broader competency in these roles. For example, full participation in peer review rounds should involve communicator and professional roles as well as the medical expert role. Effective participation in disclosure and incident management can attest to health advocate and communicator roles. A 2011 report by the Future of Medical Education in Canada Postgraduate Project noted that at the time of publication there existed approximately 40 published patient safety and quality improvement training curricula in the literature, and that half of these were published in the preceding 3 years [24]. Although this suggests the increasing value placed on these concepts, many involved only single didactic qualityrelated lectures, and none addressed clinical training interventions specific to radiation oncology. This study has laid the

C. Gillan et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146

145

groundwork for further investigation of the nature of quality and safety competencies for ROs and can also provide valuable insight for similar work in the related professions in radiation treatment, namely radiation therapy and medical physics. Conclusions Quality and safety are important concepts in radiation oncology, warranting attention in residency training. Although not adequately addressed formally in current curricula, integration of competencies relating to peer review, incident management, change management, and quality culture are necessary for proficient and quality radiation oncology practice, and can be accomplished within the broader assessment of established CanMEDS roles. Related nonmedical expert competencies can prove difficult to define in an objective manner conducive to teaching and assessment but are important for a certified RO. The degree to which trainees should be exposed to such concepts during a timebound residency program with competing priorities remains to be defined. There was nonetheless a general consensus that it be expected that residents should, at minimum, be able to identify areas of poor quality in their practice. The knowledge and skills necessary to do so must thus be acquired during training. Explicit definition of the areas of competency identified in this investigation will be explored further through a consensus-building exercise with subject matter experts. Developing a quality and safety competency profile can provide guidance to residency programs in facilitating the necessary exposure to quality-related clinical situations to develop the knowledge, skills, and behaviour necessary in practice. Acknowledgments The authors appreciate the funding and support provided by the Canadian Partnership for Quality Radiotherapy, through the Canadian Partnership Against Cancer, to complete this work. References [1] Bogdanich, W. (2010). Radiation offers new cures, and ways to do harm. New York: New York Times. [2] Bogdanich, W., & Rebelo, K. (2010). The radiation boom: a pinpoint beam strays invisibly, harming instead of healing. New York: The New York Times. [3] Institute of Medicine. Patient safety: achieving a new standard for care. www.iom.edu/reports/2003/patient-safety-achieving-a-new-standardfor-care.aspx. Accessed December 3, 2015. [4] Jaffray, D. A. (2012). Image-guided radiotherapy: from current concept to future perspectives. Nature Reviews Clinical Oncology 9(12), 688–699. [5] Peters, L. J., O’Sullivan, B., & Giralt, J., et al. (2010). Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol 28(18), 2996–3001.

146

[6] Logan, T. J. (2008). Error prevention as developed in airlines. Int J Radiat Oncol Biol Phys 71(1 Suppl), S178–S181. [7] Frank, J. R., Snell, L. S., & Sherbino, J. (2014). The Draft CanMEDS 2015 Milestone Guide. Ottawa: The Royal College of Physicians and Surgeons of Canada. [8] Giuliani, M., Gillan, C., Milne, R., Uchine, M., Millar, B. A., & Catton, P. (2014). Determining an imaging literacy curriculum for radiation oncologists; an international Delphi study. Int J Radiat Oncol Biol Phys 88(4), 961–966. [9] Gillan, C., Uchino, M., Giuliani, M., Millar, B. A., & Catton, P. (2013). Defining imaging literacy in Radiation Oncology interprofessionally: towards a competency profile for Canadian residency programs. Journal of Medical Imaging & Radiation Science 44(3), 150–156. [10] Kitzinger, J. (1995). Qualitative research: introducing focus groups. BMJ 311(7000), 299–302. [11] Palmer, C., & Bolderston, A. (2006). A brief introduction to qualitative research. Can J Med Radiat Technol 37(1), 16–19. [12] Glasser, B. (1965)‘The Constant Comparative Method of Qualitative Analysis’ in Social Problems Vol. 12 No. 4. California: University of California Press. [13] Committee on Quality Health Care in America, Institute of Medicine (2000). To Err is Human: building a safer health system. Washington: National Academies Press. [14] Committee on Quality Health Care in America, Institute of Medicine (2001). Crossing the Quality Chasm: a new healthcare system for the 21st century. Washington: National Academies Press. [15] Marks, L. B., Adams, R. D., & Pawlicki, T., et al. (2013). Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: executive summary. Practical Radiation Oncology 3(3), 149–156. [16] Lawrence, Y. R., Whiton, M. A., & Symon, Z., et al. (2012). Quality assurance peer review chart rounds in 2011: a survey of academic institutions in the United States. Int J Radiat Oncol Biol Phys 84(3), 590–595. [17] Brundage, M., Foxcroft, S., McGowan, T., Gutierrez, E., Sharpe, M., & Warde, P. (2013). A survey of radiation treatment planning peer review activities in a provincial radiation oncology programme: current practice and future directions. British Medical Journal Open 3, e003241. [18] Datta, R., Upadhyay, K. K., & Jaideep, C. N. (2012). Simulation and its role in medical education. Medical Journal of Armed Forces India 68(2), 167–172. [19] Kruger, A., Gillmann, B., Hardt, C., Doring, R., Beckers, S. K., & Rossaint, R. (2009). Teaching non-technical skills for critical incidents: crisis resource management training for medical students. Anaesthetist 58(6), 582–588. [20] Wong, B. M., Hollenberg, E., Etchells, E., Kuper, A., Levinson, W., & Shojania, K. G. (2011). Integrating Patient Safety and Quality Improvement with Postgraduate Medical Education. Members of the FMEC PG Consortium. [21] Silvey, A. B., & Warrick, L. H. (2008). Linking quality assurance to performance improvement to produce a high-reliability organization. Int J Radiat Oncol Biol Phys 71(1 Suppl), S195–S199. [22] Leong, C. N., Shakespeare, T. P., & Mukherjee, R. K., et al. (2006). Efficacy of an integrated continuing medical education (CME) and quality improvement (QI) program on radiation oncologist (RO) clinical practice. International Journal for Radiation Oncology Biology Physics 66(5), 1457–1460. [23] Tess, A., Vidyarthi, A., Yang, J., & Myers, J. S. (2015). Bridging the gap: a framework and strategies for integrating the quality and safety mission of teaching hospitals and graduate medical education. Acad Med 90(9), 1251–1257. [24] Kirch, D. G., & Boysen, P. G. (2010). Changing the culture in medical education to teach patient safety. Health Aff 29(9), 1600–1604.

C. Gillan et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 139-146