- Email: [email protected]
A qualitative study to identify opportunities for improving trauma quality improvement
Journal of Critical Care (2012) 27, 738.e1–738.e7
A qualitative study to identify opportunities for improving trauma quality improvement☆,☆☆ Maria Jose Santana PhD a,⁎, Sharon Straus MD, MSc b , Russell Gruen MD, PhD c , Henry T. Stelfox MD, PhD d a
Department of Community Health Sciences, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada Saint Michael's Hospital, Department of Medicine, University of Toronto, Toronto, Ontario, Canada c The National Trauma Research Institute, The Alfred, Monash University, Melbourne, Australia d Departments of Critical Care Medicine, Medicine and Community Health Sciences, Institute for Public Health, University of Calgary, Calgary, Alberta, Canada b
Keywords: Injury; Trauma care; Quality improvement activities
Abstract Background: Quality improvement (QI) is a central tenant of trauma center accreditation in most countries, but its effectiveness is largely unknown. We sought to explore opportunities for improving trauma QI. Methods: We performed a qualitative research study using grounded theory analyses of interviews with medical directors and program managers from 75 verified trauma centers sampled from the United States (n = 51), Canada (n = 14), and Australasia (Australia and New Zealand [n = 10]) to explore experiences with trauma QI activities and identify opportunities for improvement. Results: Most trauma centers indicated that they perceived trauma QI to be important and devoted personnel for QI (data entry, data analyst, educator, nurse practitioner). Programs identified 5 principal opportunities to improve trauma QI: (1) ensure resource adequacy (human resources, registry maintenance, financial support, institutional support), (2) encourage stakeholder participation (engagement, communication, coordination), (3) ensure clinical relevance, (4) incorporate evidence-based tools, and (5) require provider and QI program accountability.
☆
Disclosure: The project was supported by Partnerships in Health System Improvement Grant (PHE-91429) from the Canadian Institutes of Health Research and Alberta Innovates. Dr Stelfox is supported by a New Investigator Award from the Canadian Institutes of Health Research and a Population Health Investigator Award from Alberta Innovates. Dr Straus is supported by a Tier 1, Canada Research Chair. Dr Gruen is supported by a Practitioner Fellowship from the Australian National Health and Medical Research Council. Funding sources had no role in the design, conduct, or reporting of this study, and we are unaware of any conflicts of interest. None of the authors have financial or professional conflicts of interest that would influence the conduct or reporting of this study. Drs Santana and Stelfox had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. ☆☆ Authorship: Dr Maria Jose Santana was involved in data interpretation, drafting of the article, and approving of the final version to be published. Dr Sharon Straus was involved in drafting of the article and in approving the final version to be published. Dr Russell Gruen provided guidance with drafting the article and in approving the final version to be published. Dr Henry T. Stelfox was involved in the design of the study, acquisition of data, data interpretation and analysis, drafting of the article, and approving the final version to be published. ⁎ Corresponding author. W21C Research and Innovation Center, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada T2N 4Z6. Tel.: +1 403 210 9257; fax: +1 403 210 9850. E-mail address: [email protected]. 0883-9441/$ – see front matter © 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jcrc.2012.07.010
738.e2
M.J. Santana et al. Conclusions: Quality improvement programs exist as accreditation requirements in most centers. However, trauma QI practices depend on a range of local and regional factors, and concrete opportunities for improvement that address impact and sustainability exist. © 2012 Elsevier Inc. All rights reserved.
1. Background The Institute of Medicine through its reports To Err Is Human [1] and Crossing the Quality Chasm [2] has revealed that medical care often falls short of established standards. Publicity for these shortfalls has made health care quality an international public health policy issue [3]. Efforts to identify opportunities for improvement have demonstrated that the problem of substandard health care extends throughout health care systems including treatment for injured patients [4]. Injury care has a long history of quality improvement (QI), beginning with surgical audits in the nineteenth century [5] and evolving to include morbidity and mortality conferences, audits, and audit filters [4]. More recently, national and international organizations have developed policies to encourage organizations providing injury care to engage in QI. For example, The Joint Commission on Accreditation of Healthcare Organizations has proposed “the continuous evaluation of a trauma system and trauma providers through structured review of the process of care as well as the outcome.” [1] The American College of Surgeons Committee on Trauma [4] and the World Health Organization [5] have developed guidelines to help trauma centers develop and manage QI programs [4,6,7]. Despite these efforts, little is known about the nature and effectiveness of trauma QI. Preliminary evidence suggests that most accredited trauma centers engage in QI, but significant variation exists in program organization and activities [8]. Therefore, we conducted interviews of leaders from verified trauma centers in the United States, Canada, Australia, and New Zealand, 4 high-income countries with similar systems of trauma care [6,7], to explore experiences with trauma QI [8]. We describe factors affecting trauma QI practices and opportunities for improvement.
2. Materials and methods 2.1. Study design We performed a mixed-methods study comprising surveys and interviews of leaders from trauma centers verified by national professional trauma organizations in the United States (trauma centers verified by the American College of Surgeons, n = 263) [9], Canada (trauma centers verified by the Trauma Association of Canada, n = 46) [10], and Australia and New Zealand (trauma centers verified by
the Royal Australasian College of Surgeons, n = 21) [11]. Leaders who indicated in the survey (251/330 trauma centers responded to the survey) that their program measured quality of care (243/251 survey responses) were invited to participate in telephone interviews to learn about trauma center experiences with QI. Results of the survey have been previously reported [8,12]. This article summarizes the qualitative analyses of the interviews performed.
2.2. Data collection Sampling of trauma centers for interviews was performed across 6 baseline strata identified in the survey (country, accreditation/verification for treatment for adult vs pediatric patients, accreditation/verification as trauma center vs trauma system, level of designation—levels 1-4, urban/rural location, and academic status) until saturation was achieved and no new themes were identified. Interviews were conducted between June 4, 2009, and February 9, 2010. Audiotape recordings of the interviews were transcribed verbatim.
2.3. Data analysis The interviews were analyzed using qualitative content analysis drawing on grounded theory using a process of open, axial, and selective coding [14,15]. All interviews were assigned a unique identifier and imported into MAXQDA2 (Verbi Software, Marburg, Germany) a computer program for qualitative data storage, indexing, and theorizing. One investigator (H.T.S.) and a research assistant independently read each transcript and coded the raw data, line by line, generating categories and themes from the data. We grouped codes into categories, and axial coding was done to look at the interrelationship of categories including consideration of context, intervening conditions, and consequences. We used written memos to provide a record of the analytic process. The validity of the qualitative analyses was examined 4 ways. We evaluated the intrarater (median agreement for observed codes, 100%; interquartile range [IQR], 87.5%100%) and interrater (median agreement for observed codes, 87.5%; IQR, 87.5%-100%) reliability of coding for a random sample of 8 transcripts using proportions and exact binomial confidence intervals. Second, the emerging findings were presented to an interdisciplinary research team who questioned the analysts and reviewed the memos created by the coders. A rigorous record of the data analysis and methodology was documented to ensure critical appraisal of the methodology. We presented the themes to the participants to ensure accurate interpretation of the findings.
Opportunities for improving trauma QI The study was approved by the Conjoint Health Research Ethics Board at the University of Calgary. Participants provided written consent.
3. Results Seventy-five trauma centers participated in the interviews (Table 1). We identified themes related to factors affecting trauma QI practices and opportunities for improvement (see Table 2). Verbatim quotes are included to illustrate key ideas.
3.1. Resource adequacy The availability of adequate resources was identified by almost all trauma centers as an important factor influencing QI efforts. Most centers suggested that effective QI is dependent on adequate human resources, with many centers perceiving current resource allocation as insufficient. A common challenge described by centers was ensuring that staff with multiple responsibilities have sufficient protected time to perform QI. For example, centers described that clinical work frequently encroached on time designated for QI or other activities (eg, education). Two suggestions were provided to manage this challenge. First, QI activities should be assigned to staff with no other responsibilities to avoid conflict. This approach was perceived to be more feasible for large centers as compared with small centers where low patient volumes require staff to have multiple responsibilities. Second, some centers suggested hiring additional clinicians as a mechanism to protect staff with QI responsibilities from being diverted to help with clinical patient care. “Quality improvement takes a lot of time. It would be really helpful to have a dedicated quality improvement person. We all wear multiple hats and the work goes on the back burner when we are needed clinically.”
Additional resources consisting of equipment and finances were also identified to be relevant. For example, some centers identified the importance of basic equipment such as laptop computers for medical record abstraction. Institutional support (eg, priority program designation) was reported as a key factor in resource allocation. Centers that perceived having limited institutional support reported significant resource deficits. Finally, many centers reported that QI resources were heavily focused on accreditation efforts with fewer resources available to support other activities. A small minority of centers suggested the potential value of linking trauma center performance to trauma center remuneration and developing independent quality and safety teams to audit performance. “We would like participate in TQIP but I will have to convince my administration that they would have to spend nine thousand dollars a year for that”.
738.e3 Table 1
Characteristics of participating trauma centers a
Characteristics
Trauma center (n = 75), n (%)
Countries United States 51 (68) Canada 14 (19) Australia and New Zealand 10 (13) Last year program verified, median (IQR) ACS 2007 (2007, 2008) TAC 2005.5 (2004, 2006) RACS 2006.5 (2004, 2008.5) Accredited or verified as Trauma center 55 (73) Trauma system 4 (5) Trauma center and system 5 (7) Accredited or verified for treatment for Adult patients 41 (55) Pediatric patients 10 (13) Adult and pediatric patients 13 (17) Level of designated care b Level 1 58 (77) Level 2 10 (13) Level 3 7 (9) Level 4 0 (0) Geographical location Urban 56 (75) Suburban 10 (13) Rural 9 (12) Academic status University based 21 (28) University affiliated 39 (52) Nonteaching 15 (20) Median household income of surrounding neighborhoods c Top tertile 21 (28) Middle tertile 24 (32) Bottom tertile 20 (27) Interview participant Medical director (n = 13) USA 3 (4) Canada 4 (5) Australasia 6 (8) Program manager (n = 62) USA 48 (64) Canada 10 (13) Australasia 4 (5) ACS, American College of Surgeons; TAC, Trauma Association of Canada; RACS, Royal Australasian College of Surgeons. a Data are expressed as number (percentage), unless otherwise indicated. b Level of designated care based on the American College of Surgeons criteria: level 1, regional resource and tertiary care; level 2, initial definitive trauma care regardless of severity of injury; level 3, assessment, resuscitation, emergency general surgery and stabilization prior to transport and; and level 4, advanced trauma life support prior to transport. c Median household income was obtained from the 2000 US census data, 2006 Canadian census data, 2006 Australian census data, and 2006 New Zealand census data with neighborhoods, respectively, defined by zip code tabulation area, forward sortation area (first 3 digits of the postal code), postal area, and ward. Distribution of median household incomes for neighborhoods surrounding trauma centers are presented as tertiles within each country's census data.
738.e4 Table 2
M.J. Santana et al. Opportunities for improving trauma QI activities
Opportunities for improvement Resource adequacy Human resources “It would be really helpful to have a dedicated quality improvement person.” Equipment “Computers will facilitate the collection and data manangement.” Financial support “…if remuneration for trauma care was attached to proven quality performance as measured at national level that would be fantastic.” Institution support “I think that if our national organization … would be more willing to step up and say these are the performance indicators that we feel are necessary for every trauma center, I think you should follow and do them for these reasons it would be a lot easier as well for me to have that external support.” Stakeholder “The engagement of key stakeholders can engagement be challenging and I think that is universal with my discussion with colleagues who work in other centres.” Stakeholder “I would truthfully like to see an evolution communication of communication where it's not just between departments but everybody has a better sort of flattening the hierarchy by creating an easier way to communicate between services and with each other…” Stakeholder “… it is really important to [have] the right coordination people in the room. I see so many trauma programs around our country that say they do [quality improvement] but when you talk to the program trauma manager, they say the only real exposure to their medical director and [other stakeholders] is running down the hall with them.” Clinical relevance Timely “My dream would be that we track our indicators in real time, review charts within a day or two of them being flagged and fix problems while still active.” Current “My goal is to be concurrent, to track patients within a couple hours of admission, to be identifying the issues, to be following through with the issues and fixing them before the patients left.” Evidence based “We are data rich but information poor, we need more evidence-based measures.” Allows for “In an ideal world I would like to be benchmaking benchmarking outcomes … but the big problem is that people do not use the same data dictionary and it is hard to benchmark against when people are not comparing apples to apples.” Accountability “There is no point in collecting data if you are not going to link it with quality and actually go back and look at the issue.”
3.2. Stakeholder participation Trauma centers highlighted stakeholder participation as a critical factor in establishing effective QI. Stakeholder participation encompassed 3 separate themes: stakeholder engagement, effective communication, and stakeholder coordination. Trauma centers highlighted the importance of stakeholder commitment and indicated that strategies are needed to facilitate stakeholder engagement. Five suggestions for improving stakeholder engagement were obtained: (i) ensuring clarity of purpose (ie, having simple, clear, and specific QI goals that are understood by all stakeholders), (ii) inviting participation (ie, ensuring that all relevant stakeholders are invited and periodically reinvited to participate), (iii) establishing contractual obligations (eg, making assignment of clinical weeks of trauma service contingent on participation in QI activities), (iv) engaging clinicians with feedback (eg, clinician report cards to identify areas of strong performance and opportunities for improvement), and (v) providing nonmonetary incentives (eg, continuing medical education credit for QI participation). “You can't run an effective trauma quality improvement program if the surgeons don't show up. Engagement of key stakeholders can be challenging and I think that is universal with my discussion with colleagues who work in other centers.”
Effective communication among stakeholders was identified as an essential element of stakeholder participation. For example, many smaller centers indicated that their limited size greatly facilitated communication as they were “…small enough so that you all know each other, run into each other in the hall and can have offline informal chats….” Conversely, several larger centers indicated that they use structured tools ranging from teleconferences to monthly newsletters to facilitate communication. Conflict between health providers within institutions and between institutions was particularly challenging for stakeholder communication. For example, several centers indicated that identifying who is responsible for the overall management of the patient with multiple injuries can be difficult but is key. Is it the trauma surgeon, trauma nurse coordinator, intensive care physician, or the subspecialty service managing most of a patient's injuries or most severe injuries? Determining responsibility and owndership for each aspect of patient care is as much a concern of QI as it is of coordination of a trauma service managing multipe injured patients. “… it is really important to [have] the right people in the room. I see so many trauma programs around our country that say they do [quality improvement] but when you talk to the program trauma manager, they say the only real exposure to their medical director and [other stakeholders] is running down the hall with them.”
Finally, trauma centers highlighted the complexity and multidisciplinary nature of injury care. By extension, centers
Opportunities for improving trauma QI stressed the importance that QI efforts be coordinated. These challenges varied somewhat between centers but included both stakeholder coordination and coordination of QI activities. Many centers indicated that coordination of stakeholders and QI activities is a long-term process that builds upon good communication, trust, and long-term relationships. Programs also identified that standardization of both clinical processes of care and QI activities was essential to facilitate coordination. In addition, several trauma centers in Canada and Australasia highted the challenges of coordinating QI activities for their large geographical patient referral areas. They indicated that these circumstances required an increased focus of QI strategies toward prehospital care and transportation issues. “I think it is critical that the trauma program has a relationship with partners within our geographical areas… so if they are in a position where they are not sure or not to transfer then they are not reluctant to call.”
3.3. Clinical relevance Trauma centers highlighted that many clinicians perceive trauma QI to be an administrative activity necessary to satisfy accreditation requirements, but of limited relevance to patient care. They indicated that for trauma QI to be effective, it needs to be clinically relevant. Three criteria were proposed to accomplish this. First, they should directly target patient care activities. Quality improvement efforts are often organized using well-accepted frameworks such as those proposed by Donabedian [16] or the Institutes of Medicine [2]. However, for clinicians to value QI, there needs to be a direct and visible link to patient care. Second, they should be practical, so that individual clinicians can easily implement QI into their clinical practice. Furthermore, organizations with different resources available should be able to perform similar activities without feeling burdened. Third, they should be timely, so that performance measurement and feedback are performed while the patient is still in the hospital. The participants indicated that timely QI facilitates “loop closure” and directly links QI to patient care highlighting its clinical relevance. “Ideally, [quality improvement] needs to be patient focused, clinically relevant, feasible, easy to report and timely… There is no point in collecting data if you are not going to link it with quality and actually go back and look at the issue”.
3.4. Evidence based A minority of centers indicated that an important strategy for improving QI is to make it evidence based. Participants indicated that there was limited evidence about the reliability, validity, and effectiveness of quality indicators.
738.e5 They reported that the absence of evidence made selection of quality indicators for measuring performance arbitrary and difficult. In addition, centers indicated that with the exception of previously published American College of Surgeons audit filters, there was limited standardization of quality indicators used by trauma centers. For example, several trauma centers highlighted that centers used indicators with different definitions (eg, intubation for patients with decreased level of conscious vs intubation for patients with Glasgow Coma Score b9, etc) or thresholds (eg, time to computed tomographic scan b30 minutes vs 1 hour vs b 2 hours, etc). This lack of standardization was reported as being a major impairment to implementing benchmarking across institutions. Finally, trauma centers highlighted the importance of using clinically relevant and evidence-based performance measures to benchmark performance. “We are data rich, but information poor. We need more evidence based measures.”“In an ideal world I would like to be benchmarking outcomes … but the big problem is that people do not use the same data dictionary and it is hard to benchmark against when people are not comparing apples to apples.”
3.5. Accountability Centers highlighted the importance of linking QI to clinical accountability. The participants stated that accountability was important for both quality of patient care and quality of QI processes. None of the respondents suggested that QI should be associated with recrimination. However, they felt that bringing back identified problems and solutions to individual providers was essential for “closing the loop.” In addition, some respondents indicated that QI programs needed to be accountable for the quality of their work. “Having an outside reviewer, someone not part of the system and not involved with accreditation, come in and review cases with our group a couple of times a year, who can ask questions would be tremendously beneficial.”
4. Discussion Our article provides a description of factors affecting trauma QI and strategies to address these challenges. The results are derived from qualitative analyses of telephone interviews with trauma center leaders in the United States, Canada, Australia, and New Zealand. Factors reported to affect trauma QI included resource adequacy and stakeholder participation. Opportunities to improve trauma QI activities include focusing on clinically relevant activities, developing evidence-based tools, and ensuring accountability for patient care and QI efforts. Our study adds to a growing body of QI literature. In a recent scoping review, White et al [17] identified 99 research
738.e6 studies describing the development and evaluation of QI teams (groups of individuals brought together to undertake specific initiatives to improve the quality of care). The authors identified that during the past 20 years, there has been substantial growth in QI efforts, important variation in both participants and the context in which initiatives were developed, and limited evidence about the attributes of successful and unsuccessful team initiatives. Previous studies [18-20] have highlighted the challenges of implementing QI activities. Bradley et al [18] described the role of senior managers in conducting QI activities leading to successful implementation. The authors identified similar barriers to those in our study including a lack of resources (absence of administration support) and difficulty engaging stakeholders that included poor physician leadership. These finding were confirmed by Wolfson et al [20] in a series of semistructured interviews of 39 primary care practices where they identified commonly reported problems with QI to be information technology management and lack of interest from staff. Despite the rich history of trauma QI, less is known about the development and evaluation of QI activities for injured patients [4]. From our work, it appears that most centers in developed trauma systems engage in QI. These programs are largely local in nature and use diverse performance measures and improvement strategies [8]. A small (but growing) number of centers have engaged in standardized QI measurement processes such as those offered through the American College of Surgeons' Trauma Quality Improvement Program [9]. Our study adds to this existing knowledge by highlighting some of the challenges associated with trauma QI, including resource adequacy, stakeholder participation, clinical relevance, limited evidence-based tools, and accountability. Despite large differences in available resources, similar challenges appear to exist for trauma QI in low- and middleincome countries. For example, a recent workshop of stakeholders from 7 countries across the Asia-Pacific region convened at the Royal Australasian College of Surgeons identified a lack of integrated trauma QI activities, limited engagement of leaders, and the absence of standards of care as important gaps and barriers to effective trauma QI [21,22]. In response, the workshop participants (regional injury care champions) developed a QI network (Asia-Pacific Trauma Quality Improvement Network) to support local QI efforts by providing a forum for knowledge and experience exchange, increasing QI capacity through workshops and training, and coordinating the development and dissemination of trauma QI tools. Trauma centers in high-income countries may benefit from a similar strategy. The results of this study need to be interpreted within the context of its limitations. First of all, we recorded what trauma center leaders perceived as factors affecting trauma QI practices and opportunities for improvement, although we did not directly observe how they practice. Second, we interviewed trauma centers verified by professional trauma
M.J. Santana et al. societies in 4 high-income countries with similar systems of trauma care. Quality improvement practices at other trauma centers within these countries (nonverified centers) or in other countries may have very different practices. Nevertheless, by limiting our evaluation to those institutions verified by national professional trauma organizations, we have focused on those programs most likely to provide the most valuable performance measurement and QI data [15]. In summary, this qualitative study provides the first description of factors affecting trauma center QI practices and opportunities for improvent in 4 high-income countries. Our data indicate that most trauma centers are engaged in QI activities and have a demonstrated interest in improving their programs. Common opportunities to improve trauma QI include ensuring resource adequacy, encouraging stakeholder participation, making sure of clinical relevance, developing evidence-based tools, and requiring provider and QI program accountability. Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.jcrc.2012.07.010.
Acknowledgments The authors would like to thank Nancy Clayden for help with interviews and trauma center leaders in the United States, Canada, Australia, and New Zealand for their support and collaboration.
References [1] Institute of Medicine. To err is human. Washington DC: National Academy Press; 1999. [2] Institute of Medicine Committee on the Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academy Press; 2001. [3] Stelfox HT, Palmisani S, Scurlock C, Orav EJ, Bates DW. The “To Err is Human” report and the patient safety literature. Qual Saf Health Care 2006;15(3):174-8. [4] Maier RV, Rhodes M. Trauma performance improvement. In: Rivara FP, Cummings P, Koepsell TD, Grossman DC, Maier RV, editors. Injury control. Cambridge: Cambridge University Press; 2001. p. 236-49. [5] Wright JE. The history of surgical audit. J Qual Clin Pract 1995;15(2): 81-8. [6] Mock C. WHO releases guidelines for trauma quality improvement programmes. Inj Prev 2009;15(5):359. [7] American College of Surgeons. Resources for optimal care of the injured patient: 2006. Chicago: American College of Surgeons; 2007. [8] Stelfox HT, Straus SE, Nathens A, Gruen RL, Hameed M, Kirkpatrick AW. Trauma center quality improvement programs in the United States. Annals of Surgery: Canada and Australasia; 2012 [Epub ahead of print]. [9] American College of Surgeons Committee on Trauma. Verified trauma centers: American College of Surgeons. http://inspiringquality.facs. org/about/committee-on-trauma/. [10] Trauma Association of Canada. Accredited/verified hospital: Trauma Association of Canada. www.traumacanada.org/.
Opportunities for improving trauma QI [11] Royal Australasian College of Surgeons. Trauma verification: hospitals who have undergone trauma verification: Royal Australasian College of Surgeons. www.surgeons.org/for-hospitals/trauma-verification/. [12] Santana MJ, Stelfox HT. Quality indicators used by trauma centers for performance measurement. J Trauma 2012 In press. [13] Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. Quality indicators for evaluating trauma care: a scoping review. Arch Surg 2010;145(3):286-95. [14] Qualitative research in health care. London: BMJ Books; 1999. [15] Strauss AL. Qualitative analysis for social scientists. Cambridge: Cambridge University Press; 2003. [16] Donabedian A. Evaluating the quality of medical care. Milbank Q 2005;83:691-729. [17] White D, Straus SE, Stelfox HT, Holroyd-Leduc JM, Bell CM, Jackson K, et al. What is the value and impact of quality and safety teams? A scoping review. Implem Sci 2011;6:97, http://dx.doi.org/10.1186/ 1748-5908-6-97.
738.e7 [18] Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, et al. The roles of senior management in quality improvement efforts: what are the key components? J Healthc Manag 2003;48(1):1529. [19] Bradley EH, Herrin J, Mattera JA, Holmboe ES, et al. Quality improvement efforts and hospital performance. Med Care 2005;43: 282-92. [20] Wolfson D, Bernabeo E, Leas B, Sofaer S, Pawlson G, Pillittere D. Quality improvement in small office settings: an examination of successful practices. BMC Fam Pract 2009;10:14. [21] Stelfox HT, Joshipura M, Chadbunchachai W, Ellawala RN, O'Reilly G, Nguyen TS, et al. Trauma quality improvement in low and middle income countries of the Asia-Pacific Region: a mixed methods study. World J Surg 2012 [Epub ahead of print] PMID:22526038 [PubMed as supplied by publisher]. [22] Arnold JL. International emergency medicine and the recent development of emergency medicine worldwide. Ann Emerg Med 1999;33(1):97-103.
A qualitative study to identify opportunities for improving trauma quality improvement☆,☆☆ Maria Jose Santana PhD a,⁎, Sharon Straus MD, MSc b , Russell Gruen MD, PhD c , Henry T. Stelfox MD, PhD d a
Department of Community Health Sciences, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada Saint Michael's Hospital, Department of Medicine, University of Toronto, Toronto, Ontario, Canada c The National Trauma Research Institute, The Alfred, Monash University, Melbourne, Australia d Departments of Critical Care Medicine, Medicine and Community Health Sciences, Institute for Public Health, University of Calgary, Calgary, Alberta, Canada b
Keywords: Injury; Trauma care; Quality improvement activities
Abstract Background: Quality improvement (QI) is a central tenant of trauma center accreditation in most countries, but its effectiveness is largely unknown. We sought to explore opportunities for improving trauma QI. Methods: We performed a qualitative research study using grounded theory analyses of interviews with medical directors and program managers from 75 verified trauma centers sampled from the United States (n = 51), Canada (n = 14), and Australasia (Australia and New Zealand [n = 10]) to explore experiences with trauma QI activities and identify opportunities for improvement. Results: Most trauma centers indicated that they perceived trauma QI to be important and devoted personnel for QI (data entry, data analyst, educator, nurse practitioner). Programs identified 5 principal opportunities to improve trauma QI: (1) ensure resource adequacy (human resources, registry maintenance, financial support, institutional support), (2) encourage stakeholder participation (engagement, communication, coordination), (3) ensure clinical relevance, (4) incorporate evidence-based tools, and (5) require provider and QI program accountability.
☆
Disclosure: The project was supported by Partnerships in Health System Improvement Grant (PHE-91429) from the Canadian Institutes of Health Research and Alberta Innovates. Dr Stelfox is supported by a New Investigator Award from the Canadian Institutes of Health Research and a Population Health Investigator Award from Alberta Innovates. Dr Straus is supported by a Tier 1, Canada Research Chair. Dr Gruen is supported by a Practitioner Fellowship from the Australian National Health and Medical Research Council. Funding sources had no role in the design, conduct, or reporting of this study, and we are unaware of any conflicts of interest. None of the authors have financial or professional conflicts of interest that would influence the conduct or reporting of this study. Drs Santana and Stelfox had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. ☆☆ Authorship: Dr Maria Jose Santana was involved in data interpretation, drafting of the article, and approving of the final version to be published. Dr Sharon Straus was involved in drafting of the article and in approving the final version to be published. Dr Russell Gruen provided guidance with drafting the article and in approving the final version to be published. Dr Henry T. Stelfox was involved in the design of the study, acquisition of data, data interpretation and analysis, drafting of the article, and approving the final version to be published. ⁎ Corresponding author. W21C Research and Innovation Center, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada T2N 4Z6. Tel.: +1 403 210 9257; fax: +1 403 210 9850. E-mail address: [email protected]. 0883-9441/$ – see front matter © 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jcrc.2012.07.010
738.e2
M.J. Santana et al. Conclusions: Quality improvement programs exist as accreditation requirements in most centers. However, trauma QI practices depend on a range of local and regional factors, and concrete opportunities for improvement that address impact and sustainability exist. © 2012 Elsevier Inc. All rights reserved.
1. Background The Institute of Medicine through its reports To Err Is Human [1] and Crossing the Quality Chasm [2] has revealed that medical care often falls short of established standards. Publicity for these shortfalls has made health care quality an international public health policy issue [3]. Efforts to identify opportunities for improvement have demonstrated that the problem of substandard health care extends throughout health care systems including treatment for injured patients [4]. Injury care has a long history of quality improvement (QI), beginning with surgical audits in the nineteenth century [5] and evolving to include morbidity and mortality conferences, audits, and audit filters [4]. More recently, national and international organizations have developed policies to encourage organizations providing injury care to engage in QI. For example, The Joint Commission on Accreditation of Healthcare Organizations has proposed “the continuous evaluation of a trauma system and trauma providers through structured review of the process of care as well as the outcome.” [1] The American College of Surgeons Committee on Trauma [4] and the World Health Organization [5] have developed guidelines to help trauma centers develop and manage QI programs [4,6,7]. Despite these efforts, little is known about the nature and effectiveness of trauma QI. Preliminary evidence suggests that most accredited trauma centers engage in QI, but significant variation exists in program organization and activities [8]. Therefore, we conducted interviews of leaders from verified trauma centers in the United States, Canada, Australia, and New Zealand, 4 high-income countries with similar systems of trauma care [6,7], to explore experiences with trauma QI [8]. We describe factors affecting trauma QI practices and opportunities for improvement.
2. Materials and methods 2.1. Study design We performed a mixed-methods study comprising surveys and interviews of leaders from trauma centers verified by national professional trauma organizations in the United States (trauma centers verified by the American College of Surgeons, n = 263) [9], Canada (trauma centers verified by the Trauma Association of Canada, n = 46) [10], and Australia and New Zealand (trauma centers verified by
the Royal Australasian College of Surgeons, n = 21) [11]. Leaders who indicated in the survey (251/330 trauma centers responded to the survey) that their program measured quality of care (243/251 survey responses) were invited to participate in telephone interviews to learn about trauma center experiences with QI. Results of the survey have been previously reported [8,12]. This article summarizes the qualitative analyses of the interviews performed.
2.2. Data collection Sampling of trauma centers for interviews was performed across 6 baseline strata identified in the survey (country, accreditation/verification for treatment for adult vs pediatric patients, accreditation/verification as trauma center vs trauma system, level of designation—levels 1-4, urban/rural location, and academic status) until saturation was achieved and no new themes were identified. Interviews were conducted between June 4, 2009, and February 9, 2010. Audiotape recordings of the interviews were transcribed verbatim.
2.3. Data analysis The interviews were analyzed using qualitative content analysis drawing on grounded theory using a process of open, axial, and selective coding [14,15]. All interviews were assigned a unique identifier and imported into MAXQDA2 (Verbi Software, Marburg, Germany) a computer program for qualitative data storage, indexing, and theorizing. One investigator (H.T.S.) and a research assistant independently read each transcript and coded the raw data, line by line, generating categories and themes from the data. We grouped codes into categories, and axial coding was done to look at the interrelationship of categories including consideration of context, intervening conditions, and consequences. We used written memos to provide a record of the analytic process. The validity of the qualitative analyses was examined 4 ways. We evaluated the intrarater (median agreement for observed codes, 100%; interquartile range [IQR], 87.5%100%) and interrater (median agreement for observed codes, 87.5%; IQR, 87.5%-100%) reliability of coding for a random sample of 8 transcripts using proportions and exact binomial confidence intervals. Second, the emerging findings were presented to an interdisciplinary research team who questioned the analysts and reviewed the memos created by the coders. A rigorous record of the data analysis and methodology was documented to ensure critical appraisal of the methodology. We presented the themes to the participants to ensure accurate interpretation of the findings.
Opportunities for improving trauma QI The study was approved by the Conjoint Health Research Ethics Board at the University of Calgary. Participants provided written consent.
3. Results Seventy-five trauma centers participated in the interviews (Table 1). We identified themes related to factors affecting trauma QI practices and opportunities for improvement (see Table 2). Verbatim quotes are included to illustrate key ideas.
3.1. Resource adequacy The availability of adequate resources was identified by almost all trauma centers as an important factor influencing QI efforts. Most centers suggested that effective QI is dependent on adequate human resources, with many centers perceiving current resource allocation as insufficient. A common challenge described by centers was ensuring that staff with multiple responsibilities have sufficient protected time to perform QI. For example, centers described that clinical work frequently encroached on time designated for QI or other activities (eg, education). Two suggestions were provided to manage this challenge. First, QI activities should be assigned to staff with no other responsibilities to avoid conflict. This approach was perceived to be more feasible for large centers as compared with small centers where low patient volumes require staff to have multiple responsibilities. Second, some centers suggested hiring additional clinicians as a mechanism to protect staff with QI responsibilities from being diverted to help with clinical patient care. “Quality improvement takes a lot of time. It would be really helpful to have a dedicated quality improvement person. We all wear multiple hats and the work goes on the back burner when we are needed clinically.”
Additional resources consisting of equipment and finances were also identified to be relevant. For example, some centers identified the importance of basic equipment such as laptop computers for medical record abstraction. Institutional support (eg, priority program designation) was reported as a key factor in resource allocation. Centers that perceived having limited institutional support reported significant resource deficits. Finally, many centers reported that QI resources were heavily focused on accreditation efforts with fewer resources available to support other activities. A small minority of centers suggested the potential value of linking trauma center performance to trauma center remuneration and developing independent quality and safety teams to audit performance. “We would like participate in TQIP but I will have to convince my administration that they would have to spend nine thousand dollars a year for that”.
738.e3 Table 1
Characteristics of participating trauma centers a
Characteristics
Trauma center (n = 75), n (%)
Countries United States 51 (68) Canada 14 (19) Australia and New Zealand 10 (13) Last year program verified, median (IQR) ACS 2007 (2007, 2008) TAC 2005.5 (2004, 2006) RACS 2006.5 (2004, 2008.5) Accredited or verified as Trauma center 55 (73) Trauma system 4 (5) Trauma center and system 5 (7) Accredited or verified for treatment for Adult patients 41 (55) Pediatric patients 10 (13) Adult and pediatric patients 13 (17) Level of designated care b Level 1 58 (77) Level 2 10 (13) Level 3 7 (9) Level 4 0 (0) Geographical location Urban 56 (75) Suburban 10 (13) Rural 9 (12) Academic status University based 21 (28) University affiliated 39 (52) Nonteaching 15 (20) Median household income of surrounding neighborhoods c Top tertile 21 (28) Middle tertile 24 (32) Bottom tertile 20 (27) Interview participant Medical director (n = 13) USA 3 (4) Canada 4 (5) Australasia 6 (8) Program manager (n = 62) USA 48 (64) Canada 10 (13) Australasia 4 (5) ACS, American College of Surgeons; TAC, Trauma Association of Canada; RACS, Royal Australasian College of Surgeons. a Data are expressed as number (percentage), unless otherwise indicated. b Level of designated care based on the American College of Surgeons criteria: level 1, regional resource and tertiary care; level 2, initial definitive trauma care regardless of severity of injury; level 3, assessment, resuscitation, emergency general surgery and stabilization prior to transport and; and level 4, advanced trauma life support prior to transport. c Median household income was obtained from the 2000 US census data, 2006 Canadian census data, 2006 Australian census data, and 2006 New Zealand census data with neighborhoods, respectively, defined by zip code tabulation area, forward sortation area (first 3 digits of the postal code), postal area, and ward. Distribution of median household incomes for neighborhoods surrounding trauma centers are presented as tertiles within each country's census data.
738.e4 Table 2
M.J. Santana et al. Opportunities for improving trauma QI activities
Opportunities for improvement Resource adequacy Human resources “It would be really helpful to have a dedicated quality improvement person.” Equipment “Computers will facilitate the collection and data manangement.” Financial support “…if remuneration for trauma care was attached to proven quality performance as measured at national level that would be fantastic.” Institution support “I think that if our national organization … would be more willing to step up and say these are the performance indicators that we feel are necessary for every trauma center, I think you should follow and do them for these reasons it would be a lot easier as well for me to have that external support.” Stakeholder “The engagement of key stakeholders can engagement be challenging and I think that is universal with my discussion with colleagues who work in other centres.” Stakeholder “I would truthfully like to see an evolution communication of communication where it's not just between departments but everybody has a better sort of flattening the hierarchy by creating an easier way to communicate between services and with each other…” Stakeholder “… it is really important to [have] the right coordination people in the room. I see so many trauma programs around our country that say they do [quality improvement] but when you talk to the program trauma manager, they say the only real exposure to their medical director and [other stakeholders] is running down the hall with them.” Clinical relevance Timely “My dream would be that we track our indicators in real time, review charts within a day or two of them being flagged and fix problems while still active.” Current “My goal is to be concurrent, to track patients within a couple hours of admission, to be identifying the issues, to be following through with the issues and fixing them before the patients left.” Evidence based “We are data rich but information poor, we need more evidence-based measures.” Allows for “In an ideal world I would like to be benchmaking benchmarking outcomes … but the big problem is that people do not use the same data dictionary and it is hard to benchmark against when people are not comparing apples to apples.” Accountability “There is no point in collecting data if you are not going to link it with quality and actually go back and look at the issue.”
3.2. Stakeholder participation Trauma centers highlighted stakeholder participation as a critical factor in establishing effective QI. Stakeholder participation encompassed 3 separate themes: stakeholder engagement, effective communication, and stakeholder coordination. Trauma centers highlighted the importance of stakeholder commitment and indicated that strategies are needed to facilitate stakeholder engagement. Five suggestions for improving stakeholder engagement were obtained: (i) ensuring clarity of purpose (ie, having simple, clear, and specific QI goals that are understood by all stakeholders), (ii) inviting participation (ie, ensuring that all relevant stakeholders are invited and periodically reinvited to participate), (iii) establishing contractual obligations (eg, making assignment of clinical weeks of trauma service contingent on participation in QI activities), (iv) engaging clinicians with feedback (eg, clinician report cards to identify areas of strong performance and opportunities for improvement), and (v) providing nonmonetary incentives (eg, continuing medical education credit for QI participation). “You can't run an effective trauma quality improvement program if the surgeons don't show up. Engagement of key stakeholders can be challenging and I think that is universal with my discussion with colleagues who work in other centers.”
Effective communication among stakeholders was identified as an essential element of stakeholder participation. For example, many smaller centers indicated that their limited size greatly facilitated communication as they were “…small enough so that you all know each other, run into each other in the hall and can have offline informal chats….” Conversely, several larger centers indicated that they use structured tools ranging from teleconferences to monthly newsletters to facilitate communication. Conflict between health providers within institutions and between institutions was particularly challenging for stakeholder communication. For example, several centers indicated that identifying who is responsible for the overall management of the patient with multiple injuries can be difficult but is key. Is it the trauma surgeon, trauma nurse coordinator, intensive care physician, or the subspecialty service managing most of a patient's injuries or most severe injuries? Determining responsibility and owndership for each aspect of patient care is as much a concern of QI as it is of coordination of a trauma service managing multipe injured patients. “… it is really important to [have] the right people in the room. I see so many trauma programs around our country that say they do [quality improvement] but when you talk to the program trauma manager, they say the only real exposure to their medical director and [other stakeholders] is running down the hall with them.”
Finally, trauma centers highlighted the complexity and multidisciplinary nature of injury care. By extension, centers
Opportunities for improving trauma QI stressed the importance that QI efforts be coordinated. These challenges varied somewhat between centers but included both stakeholder coordination and coordination of QI activities. Many centers indicated that coordination of stakeholders and QI activities is a long-term process that builds upon good communication, trust, and long-term relationships. Programs also identified that standardization of both clinical processes of care and QI activities was essential to facilitate coordination. In addition, several trauma centers in Canada and Australasia highted the challenges of coordinating QI activities for their large geographical patient referral areas. They indicated that these circumstances required an increased focus of QI strategies toward prehospital care and transportation issues. “I think it is critical that the trauma program has a relationship with partners within our geographical areas… so if they are in a position where they are not sure or not to transfer then they are not reluctant to call.”
3.3. Clinical relevance Trauma centers highlighted that many clinicians perceive trauma QI to be an administrative activity necessary to satisfy accreditation requirements, but of limited relevance to patient care. They indicated that for trauma QI to be effective, it needs to be clinically relevant. Three criteria were proposed to accomplish this. First, they should directly target patient care activities. Quality improvement efforts are often organized using well-accepted frameworks such as those proposed by Donabedian [16] or the Institutes of Medicine [2]. However, for clinicians to value QI, there needs to be a direct and visible link to patient care. Second, they should be practical, so that individual clinicians can easily implement QI into their clinical practice. Furthermore, organizations with different resources available should be able to perform similar activities without feeling burdened. Third, they should be timely, so that performance measurement and feedback are performed while the patient is still in the hospital. The participants indicated that timely QI facilitates “loop closure” and directly links QI to patient care highlighting its clinical relevance. “Ideally, [quality improvement] needs to be patient focused, clinically relevant, feasible, easy to report and timely… There is no point in collecting data if you are not going to link it with quality and actually go back and look at the issue”.
3.4. Evidence based A minority of centers indicated that an important strategy for improving QI is to make it evidence based. Participants indicated that there was limited evidence about the reliability, validity, and effectiveness of quality indicators.
738.e5 They reported that the absence of evidence made selection of quality indicators for measuring performance arbitrary and difficult. In addition, centers indicated that with the exception of previously published American College of Surgeons audit filters, there was limited standardization of quality indicators used by trauma centers. For example, several trauma centers highlighted that centers used indicators with different definitions (eg, intubation for patients with decreased level of conscious vs intubation for patients with Glasgow Coma Score b9, etc) or thresholds (eg, time to computed tomographic scan b30 minutes vs 1 hour vs b 2 hours, etc). This lack of standardization was reported as being a major impairment to implementing benchmarking across institutions. Finally, trauma centers highlighted the importance of using clinically relevant and evidence-based performance measures to benchmark performance. “We are data rich, but information poor. We need more evidence based measures.”“In an ideal world I would like to be benchmarking outcomes … but the big problem is that people do not use the same data dictionary and it is hard to benchmark against when people are not comparing apples to apples.”
3.5. Accountability Centers highlighted the importance of linking QI to clinical accountability. The participants stated that accountability was important for both quality of patient care and quality of QI processes. None of the respondents suggested that QI should be associated with recrimination. However, they felt that bringing back identified problems and solutions to individual providers was essential for “closing the loop.” In addition, some respondents indicated that QI programs needed to be accountable for the quality of their work. “Having an outside reviewer, someone not part of the system and not involved with accreditation, come in and review cases with our group a couple of times a year, who can ask questions would be tremendously beneficial.”
4. Discussion Our article provides a description of factors affecting trauma QI and strategies to address these challenges. The results are derived from qualitative analyses of telephone interviews with trauma center leaders in the United States, Canada, Australia, and New Zealand. Factors reported to affect trauma QI included resource adequacy and stakeholder participation. Opportunities to improve trauma QI activities include focusing on clinically relevant activities, developing evidence-based tools, and ensuring accountability for patient care and QI efforts. Our study adds to a growing body of QI literature. In a recent scoping review, White et al [17] identified 99 research
738.e6 studies describing the development and evaluation of QI teams (groups of individuals brought together to undertake specific initiatives to improve the quality of care). The authors identified that during the past 20 years, there has been substantial growth in QI efforts, important variation in both participants and the context in which initiatives were developed, and limited evidence about the attributes of successful and unsuccessful team initiatives. Previous studies [18-20] have highlighted the challenges of implementing QI activities. Bradley et al [18] described the role of senior managers in conducting QI activities leading to successful implementation. The authors identified similar barriers to those in our study including a lack of resources (absence of administration support) and difficulty engaging stakeholders that included poor physician leadership. These finding were confirmed by Wolfson et al [20] in a series of semistructured interviews of 39 primary care practices where they identified commonly reported problems with QI to be information technology management and lack of interest from staff. Despite the rich history of trauma QI, less is known about the development and evaluation of QI activities for injured patients [4]. From our work, it appears that most centers in developed trauma systems engage in QI. These programs are largely local in nature and use diverse performance measures and improvement strategies [8]. A small (but growing) number of centers have engaged in standardized QI measurement processes such as those offered through the American College of Surgeons' Trauma Quality Improvement Program [9]. Our study adds to this existing knowledge by highlighting some of the challenges associated with trauma QI, including resource adequacy, stakeholder participation, clinical relevance, limited evidence-based tools, and accountability. Despite large differences in available resources, similar challenges appear to exist for trauma QI in low- and middleincome countries. For example, a recent workshop of stakeholders from 7 countries across the Asia-Pacific region convened at the Royal Australasian College of Surgeons identified a lack of integrated trauma QI activities, limited engagement of leaders, and the absence of standards of care as important gaps and barriers to effective trauma QI [21,22]. In response, the workshop participants (regional injury care champions) developed a QI network (Asia-Pacific Trauma Quality Improvement Network) to support local QI efforts by providing a forum for knowledge and experience exchange, increasing QI capacity through workshops and training, and coordinating the development and dissemination of trauma QI tools. Trauma centers in high-income countries may benefit from a similar strategy. The results of this study need to be interpreted within the context of its limitations. First of all, we recorded what trauma center leaders perceived as factors affecting trauma QI practices and opportunities for improvement, although we did not directly observe how they practice. Second, we interviewed trauma centers verified by professional trauma
M.J. Santana et al. societies in 4 high-income countries with similar systems of trauma care. Quality improvement practices at other trauma centers within these countries (nonverified centers) or in other countries may have very different practices. Nevertheless, by limiting our evaluation to those institutions verified by national professional trauma organizations, we have focused on those programs most likely to provide the most valuable performance measurement and QI data [15]. In summary, this qualitative study provides the first description of factors affecting trauma center QI practices and opportunities for improvent in 4 high-income countries. Our data indicate that most trauma centers are engaged in QI activities and have a demonstrated interest in improving their programs. Common opportunities to improve trauma QI include ensuring resource adequacy, encouraging stakeholder participation, making sure of clinical relevance, developing evidence-based tools, and requiring provider and QI program accountability. Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.jcrc.2012.07.010.
Acknowledgments The authors would like to thank Nancy Clayden for help with interviews and trauma center leaders in the United States, Canada, Australia, and New Zealand for their support and collaboration.
References [1] Institute of Medicine. To err is human. Washington DC: National Academy Press; 1999. [2] Institute of Medicine Committee on the Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academy Press; 2001. [3] Stelfox HT, Palmisani S, Scurlock C, Orav EJ, Bates DW. The “To Err is Human” report and the patient safety literature. Qual Saf Health Care 2006;15(3):174-8. [4] Maier RV, Rhodes M. Trauma performance improvement. In: Rivara FP, Cummings P, Koepsell TD, Grossman DC, Maier RV, editors. Injury control. Cambridge: Cambridge University Press; 2001. p. 236-49. [5] Wright JE. The history of surgical audit. J Qual Clin Pract 1995;15(2): 81-8. [6] Mock C. WHO releases guidelines for trauma quality improvement programmes. Inj Prev 2009;15(5):359. [7] American College of Surgeons. Resources for optimal care of the injured patient: 2006. Chicago: American College of Surgeons; 2007. [8] Stelfox HT, Straus SE, Nathens A, Gruen RL, Hameed M, Kirkpatrick AW. Trauma center quality improvement programs in the United States. Annals of Surgery: Canada and Australasia; 2012 [Epub ahead of print]. [9] American College of Surgeons Committee on Trauma. Verified trauma centers: American College of Surgeons. http://inspiringquality.facs. org/about/committee-on-trauma/. [10] Trauma Association of Canada. Accredited/verified hospital: Trauma Association of Canada. www.traumacanada.org/.
Opportunities for improving trauma QI [11] Royal Australasian College of Surgeons. Trauma verification: hospitals who have undergone trauma verification: Royal Australasian College of Surgeons. www.surgeons.org/for-hospitals/trauma-verification/. [12] Santana MJ, Stelfox HT. Quality indicators used by trauma centers for performance measurement. J Trauma 2012 In press. [13] Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. Quality indicators for evaluating trauma care: a scoping review. Arch Surg 2010;145(3):286-95. [14] Qualitative research in health care. London: BMJ Books; 1999. [15] Strauss AL. Qualitative analysis for social scientists. Cambridge: Cambridge University Press; 2003. [16] Donabedian A. Evaluating the quality of medical care. Milbank Q 2005;83:691-729. [17] White D, Straus SE, Stelfox HT, Holroyd-Leduc JM, Bell CM, Jackson K, et al. What is the value and impact of quality and safety teams? A scoping review. Implem Sci 2011;6:97, http://dx.doi.org/10.1186/ 1748-5908-6-97.
738.e7 [18] Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, et al. The roles of senior management in quality improvement efforts: what are the key components? J Healthc Manag 2003;48(1):1529. [19] Bradley EH, Herrin J, Mattera JA, Holmboe ES, et al. Quality improvement efforts and hospital performance. Med Care 2005;43: 282-92. [20] Wolfson D, Bernabeo E, Leas B, Sofaer S, Pawlson G, Pillittere D. Quality improvement in small office settings: an examination of successful practices. BMC Fam Pract 2009;10:14. [21] Stelfox HT, Joshipura M, Chadbunchachai W, Ellawala RN, O'Reilly G, Nguyen TS, et al. Trauma quality improvement in low and middle income countries of the Asia-Pacific Region: a mixed methods study. World J Surg 2012 [Epub ahead of print] PMID:22526038 [PubMed as supplied by publisher]. [22] Arnold JL. International emergency medicine and the recent development of emergency medicine worldwide. Ann Emerg Med 1999;33(1):97-103.