- Email: [email protected]
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework
GYNAECOLOGY
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework Innie Chen, MD, MPH,1,2 Deborah Money, MD,3 Paul Yong, MD, PhD,3 Christina Williams, MD,3 Catherine Allaire, MD3 Department of Obstetrics and Gynecology, University of Ottawa and the Ottawa Hospital Research Institute, Ottawa ON
1
School of Population and Public Health, University of British Columbia, Vancouver BC
2
Department of Obstetrics and Gynaecology, University of British Columbia, Vancouver BC
3
Abstract
Résumé
Objective: Chronic pelvic pain (CPP) is a prevalent, debilitating, and costly condition. Although national guidelines and empiric evidence support the use of a multidisciplinary model of care for such patients, such clinics are uncommon in Canada. The BC Women’s Centre for Pelvic Pain and Endometriosis was created to respond to this need, and there is interest in this model of care’s impact on the burden of disease in British Columbia.
Objectif : La douleur pelvienne chronique (DPC) est un trouble prévalent, débilitant et coûteux. Bien que des lignes directrices nationales et des données empiriques soutiennent l’utilisation d’un modèle de soins multidisciplinaires pour les patientes qui présentent des DPC, les cliniques canadiennes offrant un tel modèle sont peu nombreuses. Le BC Women’s Centre for Pelvic Pain and Endometriosis a été créé pour répondre à ce besoin et un intérêt a été manifesté à l’endroit des effets de ce modèle de soins sur le fardeau de la maladie en Colombie-Britannique.
We sought to create an approach to its evaluation using the RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) evaluation framework to assess the impact of the care model and to guide clinical decision-making and policy.
Methods: The RE-AIM evaluation framework was applied to consider the different dimensions of impact of the BC Centre. The proposed measures, data sources, and data management strategies for this mixed-methods approach were identified. Results: The five dimensions of impact were considered at individual and organizational levels, and corresponding indicators were proposed to enable integration into existing data infrastructure to facilitate collection and early program evaluation. Conclusion: The RE-AIM framework can be applied to the evaluation of a multidisciplinary chronic pelvic pain clinic. This will allow better assessment of the impact of innovative models of care for women with chronic pelvic pain.
Nous avons cherché à créer une approche envers son évaluation en utilisant le cadre de référence RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) pour déterminer les effets de ce modèle de soins et pour orienter le processus décisionnel clinique et la formulation de politiques en la matière.
Méthodes : Le cadre d’évaluation RE-AIM a été appliqué pour nous permettre d’envisager les diverses dimensions des effets du BC Centre. Les mesures proposées, les sources de données et les stratégies de gestion des données pour cette approche à méthodes mixtes ont été identifiées. Résultats : Les cinq dimensions des effets ont été prises en considération aux niveaux personnel et organisationnel, et des indicateurs correspondants ont été proposés pour en permettre l’intégration dans l’infrastructure de données existante, en vue de faciliter la collecte des données et l’évaluation précoce du programme. Conclusion : Le cadre RE-AIM peut être appliqué à l’évaluation d’une clinique multidisciplinaire visant la douleur pelvienne chronique. Cela permettra la tenue d’une meilleure évaluation des effets de modèles novateurs de soins chez les femmes qui connaissent des douleurs pelviennes chroniques.
Key Words: Chronic pelvic pain, multidisciplinary, RE-AIM, program evaluation Competing Interests: None declared. Received on November 8, 2014 Accepted on March 16, 2015
804 l SEPTEMBER JOGC SEPTEMBRE 2015
J Obstet Gynaecol Can 2015;37(9):804–809
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework
INTRODUCTION
C
hronic pelvic pain (CPP) is a prevalent, debilitating, and costly condition that affects nearly 15% of the general population.1,2 The burden of this disease is reflected in the associated reduction in quality of life, loss of productivity, and increased use of health care services,3 with an estimated annual cost of $1.8 billion in Canada from endometriosis alone.4 Chronic pelvic pain is a complex condition that often affects multiple body systems, and this can lead to challenges in the care of affected individuals.5 Given the complex nature of CPP, Canadian and other national guidelines have recommended use of a multidisciplinary model of care for the treatment of chronic pelvic pain,5–8 and the limited available evidence points to the efficacy of such a model.9,10 Multidisciplinary models of care that offer the services and support of psychotherapy and physiotherapy in addition to medical and surgical therapy within the same clinic are uncommon in Canada, although they are present in some areas. The result is a delay in diagnosis and suboptimal treatment for many patients with CPP. The BC Women’s Centre for Pelvic Pain and Endometriosis is a multidisciplinary program offering the expertise of a pelvic floor physiotherapist, a sexual and reproductive health counsellor, a nurse educator, and an anaesthetist, as well as gynaecologists trained in advanced laparoscopic surgery. Complex CPP patients also attend an education session before consultation with a physician, and are offered group education to learn about lifestyle and behavioural techniques to deal with CPP. This program was established with provincial health authority funding for the purpose of reducing the current burden of disease. Provincial needs assessment, national guidelines, and expert opinion all supported the development, implementation, and maintenance of the program. Whether the model is adopted in other areas of Canada will depend on its assessed impact. The RE-AIM framework was initially developed and introduced by Glasgow et al. in 1999 for the purpose of systematically evaluating management interventions for chronic illness.11 While evaluations of health interventions are generally limited to assessing efficacy within controlled settings, the RE-AIM framework acknowledges the importance of other factors that influence the value of an intervention in real-world settings. By considering the five dimensions of Reach, Efficacy, Adoption, Implementation, and Maintenance, the framework provides measures of longer term effect on individuals and populations.12 The dimensions of RE-AIM follow a logical sequence, and have been applied to the evaluation of health interventions
in a variety of ways.13 RE-AIM has been widely adopted in the evaluation of physical activity programs,14–17 obesity prevention programs,18,19 and interventions for the management of diabetes14,17,20–22 and other chronic diseases,11,13 but its use in the evaluation of chronic pelvic pain programs has not yet been described. We describe here an adaption of the RE-AIM framework for the evaluation of a multidisciplinary chronic pelvic pain program. METHODS
Using a mixed-methods approach, we considered the five dimensions of the RE-AIM model at individual and organizational levels, and developed an evaluation approach specific to this clinic to assess the impact of this program on the burden of chronic pelvic pain and endometriosis in British Columbia. The overarching evaluation question guiding this framework was whether women were receiving accessible, effective, and sustainable treatments through the BC Women’s Centre for Pelvic Pain and Endometriosis. The specific questions guiding the evaluation were: 1. Reach—whether the intervention is reaching the target population of women with chronic pelvic pain; 2. Effectiveness—what the effect of the multidisciplinary model is on the treatment of chronic pelvic pain; 3. Adoption—whether the multidisciplinary model has been espoused by other health care professionals or other centres; 4. Implementation—whether the model of care has been implemented as intended; and 5. Maintenance—whether the implementation is costeffective. RESULTS
Proposed indicators for each of the five dimensions are listed in the Table. Reach
Reach refers to the proportion of the population that is potentially served by the program, and is a measure of program access and participation. In this case, the denominator is women of reproductive age affected by CPP who required specialist referral in British Columbia, and the numerator is women who have accessed this centre. Concrete barriers to access include geographic distance and logistical concerns such as taking time off work and SEPTEMBER JOGC SEPTEMBRE 2015 l 805
Proportion of women of reproductive age affected by CPP requiring specialist referral in British Columbia who have access to the clinic
Behavioural, quality of life, and patient satisfaction outcomes from multidisciplinary model of care
Ability of the physician to recognize the need to address the psychosocial aspect of pain in the diagnosis, education, and treatment of a patient
Adoption of the current multidisciplinary model of care at other cities within Canada
Participant adherence to treatment
Application of the biopsychosocial multidisciplinary framework for all patients requiring treatment for CPP and commitment to adopting and helping create most current and best practices for the treatment of this chronic condition
Reach
Efficacy
Adoption individual
Adoption institutional
Implementation individual
Implementation institutional
Program-specific definition
806 l SEPTEMBER JOGC SEPTEMBRE 2015 Occurrence of and attendance at regular interdisciplinary meetings to discuss complex patients
Member participating in related CME, such as International Pelvic Pain Society Meeting Scientific presentations and publications in pelvic pain Breadth of web-based, paper, and inperson learning tools.
Ongoing commitment to current and best practices Knowledge dissemination Knowledge translation
Use of urine drug screen and opioid risk tool for patients on opioid therapy
Appropriate use of opioids Multidisciplinary collaboration
Attendance at education sessions, physician, physiotherapy, and counsellor visits
Number of multidisciplinary care models in Canada
Patient participation
National dissemination
Number of CME courses offered and physicians attended Number of physicians and fellows trained
Physician outreach and mentoring
Endometriosis Health Profile-30
Quality of life Number of physician support telephone calls
Survey item
Patient satisfaction with treatment
Physician support
Brief Pain Inventory
Targets program participants Continued
Targets other health care providers
Reflects need for pelvic pain specialists to congregate and educate each other
Measures member engagement and multidisciplinary dialogue
Monitoring for addiction risk associated with opioids
Direct measure of participation
Identification of potential collaborators
Measure of physician interest in the program, recognition of the program model, and dissemination of knowledge
Measure of collaboration in caring for patients
Specifically developed for endometriosis Easily modified for chronic pelvic pain
Standardized and ease of administration
Standardized questionnaire addressing both pain symptoms and ability to function Commonly used research tool
Measure of use and access
Monitors use of this access program
Use of transportation vouchers Waitlist for initial consultation appointment Waitlist for surgical procedure
Captured with patient demographics Enables mapping to geographic regions
Rationale
Patient postal code
Measurement tool
Pelvic pain symptoms
Access to clinical services
Geographic reach
Indicator
Program-specific definitions and indicators for each of the five RE-AIM dimensions in the evaluation of a multidisciplinary chronic pelvic pain clinic
Gynaecology
Reflects ongoing improvement in patient outcomes
Narcotic use is an indirect measure of disability
Standardized data collection by hospital and administrative databases
Efficacy indicators at one year follow-up post-discharge Proportion discharged from clinic Proportion rereferred to clinic
Proportion on disability and social assistance Return to work
Physician document Pharmacy database system
Physician billings for pelvic pain
Ongoing improvement in symptoms and quality of life
Productivity
Cessation of narcotic medicine
Health care use
Individual’s ability to be pain free, to have an acceptable quality of life, and to engage in society and meaningful work
Ability to demonstrate overall cost savings from decreased health care use of program
Maintenance individual
Maintenance institutional
Discharge abstract database for hospital admissions Emergency department data for emergency visits
Rationale Measurement tool Indicator Program-specific definition
Continued
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework
childcare. Some interventions to help reduce these barriers include reimbursement of travel costs for patients from outside the Lower Mainland of British Columbia, the option of viewing online education modules and arranging for follow-up by telephone, and the ability to schedule physician visits, allied health professional visits, and group education sessions on the same day. Other barriers include wait times for consultation and surgery. Interventions to help alleviate this include physician triaging of referrals and the availability of telephone support for physicians in the community. The geographic reach of the program can be measured by mapping patients’ residential postal codes and tracking the use of transportation vouchers, while access to clinical services can be assessed by monitoring the length of the wait lists for consultations and surgical procedures. Efficacy
Efficacy refers to behavioural, quality of life, and patient satisfaction outcomes of a given intervention. With the use of patient questionnaires and the introduction of electronic medical records, a process is already in place for evaluation of clinic efficacy. Patients seen for CPP complete questionnaires on self-reported pain and the effect of pain on quality of life to determine the attributable level of pain and debilitation. Results are tracked through time to ascertain treatment effect. Physical examinations and treatments are also standardized and monitored to determine the efficacy of specific interventions for specific subpopulations. Measures of pelvic pain symptoms can be assessed with the Brief Pain Inventory,23,24 and quality of life can be measured using the Endometriosis Health Profile-30,25 which was specifically developed for endometriosis and also subsequently tailored for pelvic pain. In addition, all program participants can be surveyed to determine their satisfaction with treatment provided by the Centre. We anticipate that patients in the pre-contemplative stage of readiness for change may be resistant to the multidisciplinary model, and a Pain Stages of Change Questionnaire may be helpful in identifying these individuals to permit a separate analysis.26 Adoption
Adoption refers to the proportion of institutions that assume a policy or program. In this case, it refers to the uptake of the biopsychosocial model of care on two levels. The first level of adoption refers to the approach to patient care by individual physicians, and the ability of the physician to recognize the need to address the psychosocial aspect of pain and incorporate education, lifestyle, psychological, SEPTEMBER JOGC SEPTEMBRE 2015 l 807
Gynaecology
and physical therapies into a comprehensive treatment plan. At our Centre, this is enabled by providing physician education and support through dissemination of clinic resources via the Centre’s website, organization of CME courses, provision of physician telephone consultations, and incorporation of physician mentorship and training as part of the program. The Centre will monitor the involvement in such activities as measures of physician support, outreach, and monitoring. The second level of adoption refers to the adoption of the current multidisciplinary model of care at other locations within Canada. This is a longer term goal, and its realization will depend on the Centre’s ability to demonstrate efficacy through ongoing evaluation and to train physicians to replicate this model in other centres through the existing fellowship program. The Centre will identify other multidisciplinary care models in Canada as potential partners for collaboration. Implementation
At the individual level, implementation is a measure of a participant’s adherence to treatment, and is necessary for interpreting efficacy. At the program level, it refers to adherence to the principles and goals of the Centre; that is, the extent to which the program is delivered as it was intended. It refers to the clinic’s consistency in applying the biopsychosocial multidisciplinary framework for all patients requiring treatment for CPP, and its commitment to adopting and creating the best practices for treatment of this chronic condition. At present, the Centre conducts weekly multidisciplinary meetings to review complex CPP patients, develop patient care pathways, and review overall program functions. The level of these activities will be measured by holding and attending these multidisciplinary meetings, while commitment to best practices will be assessed by attendance at related CME events. The program also has a strong clinical and translational research mandate, and activity in this area is quantified by scientific presentations and publications and the development of patient education materials. Maintenance
At the individual level, this refers to the ability of the individual to maintain behavioural change. In the treatment of CPP, it refers to the individual’s ability to be pain free, to have an acceptable quality of life, and to engage in society and in meaningful work. This will be measured by the proportion of patients discharged from the Centre, the proportion requiring referral to the Centre, and efficacy indicators measured at one year follow-up after discharge. 808 l SEPTEMBER JOGC SEPTEMBRE 2015
At the institution level, it is the extent to which an intervention becomes part of the everyday culture and norms of the organization. The main determinant of sustainability is the ability to demonstrate overall cost savings from decreased use of health care services by program participants, as the program is directly tied to provincial support. Measures of economic burden of disease include those related to productivity, such as the proportion of patients on disability insurance and social assistance, as well as those able to return to work following treatment in the program. Use of narcotic medication tracked through physician documentation and pharmacy databases are an indirect measure of disability. Health care use is monitored through physician billings, emergency department visits, and hospital admissions. The execution of this evaluative model will involve the collection and management of data from a variety of sources. Clinical outcomes, patient-reported outcomes, and survey data can be entered by patients and clinicians into centralized, web-accessible databases housed on secure servers within the hospital. Provincial health care administrative databases can also be accessed to determine measures of health care use, including physician billings, emergency department visits, and hospital admissions. DISCUSSION
The use of the RE-AIM evaluation framework for a multidisciplinary chronic pelvic pain clinic represents a novel application of this framework to assess the reach, efficacy, adoption, implementation and maintenance aspects of this unique model of care in Canada. As a possible prototypic model for the treatment of chronic pelvic pain in Canada, the potential for institutional adoption, implementation and maintenance of this model in other Canadian centres will depend on the ongoing results of systematic evaluation. The strength of applying the RE-AIM framework for this purpose is that it takes into consideration other dimensions of impact rather than efficacy alone. Although the REAIM framework was initially developed for public health interventions, the utility of this approach for evaluation has been successfully demonstrated for other health care interventions.12 Glasgow et al. demonstrated how the RE-AIM framework can be used to evaluate components of care for chronic illness management, and showed the importance of considering multiple dimensions rather than efficacy alone in program planning, implementation, and funding allocation.11 In our search of the scientific literature, we were unable to identify any published reports
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework
on an evaluative framework for multidisciplinary chronic pelvic pain models. As chronic disease becomes a greater health care priority, and the need for multidisciplinary models of care becomes increasingly recognized, the development of a framework for evaluation of these types of health care interventions becomes increasingly necessary. CONCLUSION
The RE-AIM framework can be successfully applied to the evaluation of a multidisciplinary chronic pelvic pain program. Such evaluation can inform key stakeholders and allow better assessment of the impact of innovative models of care for women with chronic pelvic pain. REFERENCES 1. Latthe P, Latthe M, Say L, Gülmezoglu M, Khan KS. WHO systematic review of prevalence of chronic pelvic pain: a neglected reproductive health morbidity. BMC Public Health 2006;6:177. 2. Ahangari A. Prevalence of chronic pelvic pain among women: an updated review. Pain Physician 2014;17(2):E141–7. 3. Mathias SD, Kuppermann M, Liberman RF, Lipschutz RC, Steege JF. Chronic pelvic pain: prevalence, health-related quality of life, and economic correlates. Obstet Gynecol 1996;87(3):321–7. 4. Levy AR, Osenenko KM, Lozano-Ortega G, Sambrook R, Jeddi M, Bélisle S, et al. Economic burden of surgically confirmed endometriosis in Canada. J Obstet Gynaecol Can 2011;33(8):830–7. 5. Jarrell JF, Vilos GA; SOGC Chronic Pelvic Pain Committee. Consensus guidelines for the management of chronic pelvic pain. SOGC Clinical Practice Guideline no. 164, part 1, August 2005. J Obstet Gynaecol Can 2005;27(8):781–826. 6. Jarrell JF, Vilos GA; SOGC Chronic Pelvic Pain Committee. Consensus guidelines for the management of chronic pelvic pain. SOGC Clinical Practice Guideline no. 164, part 2, September 2005. J Obstet Gynaecol Can 2005;27(9):869–910. 7. Leyland N, Casper R, Laberge P, Singh SS; SOGC Clinical Practice Gynaecology Committee. Endometriosis: diagnosis and management. SOGC Clinical Practice Guideline no. 244, July 2010. J Obstet Gynaecol Can 2010;32(7):S1–S32. 8. ACOG Committee on Practice Bulletins–Gynecology. ACOG practice bulletin no. 51. Chronic pelvic pain. Obstet Gynecol 2004;103(3):589–605. 9. Lamvu G, Williams R, Zolnoun D, Wechter ME, Shortliffe A, Fulton G, et al. Long-term outcomes after surgical and nonsurgical management of chronic pelvic pain: one year after evaluation in a pelvic pain specialty clinic. Am J Obstet Gynecol 2006;195(2):591–8. 10. Kames LD, Rapkin AJ, Naliboff BD, Afifi S, Ferrer-Brechner T. Effectiveness of an interdisciplinary pain management program for the treatment of chronic pelvic pain. Pain 1990;41(1):41–6.
11. Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns 2001;44(2):119–27. 12. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health 1999;89(9):1322–7. 13. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health 2013;103(6):e38–46. 14. Antikainen I, Ellis R. A RE-AIM evaluation of theory-based physical activity interventions. J Sport Exerc Psychol 2011;33(2):198–214. 15. Austin G, Bell T, Caperchione C, Mummery WK. Translating research to practice: using the RE-AIM framework to examine an evidence-based physical activity intervention in primary school settings. Health Promot Pract 2011;12(6):932–41. 16. De Meij JS, Chinapaw MJ, Kremers SP, Van der Wal MF, Jurg ME, Van Mechelen W. Promoting physical activity in children: the stepwise development of the primary school-based JUMP-in intervention applying the RE-AIM evaluation framework. Br J Sports Med 2010;44(12):879–87. 17. Matthews L, Kirk A, Macmillan F, Mutrie N. Can physical activity interventions for adults with type 2 diabetes be translated into practice settings? A systematic review using the RE-AIM framework. Transl Behav Med 2014;4(1):60–78. 18. Estabrook B, Zapka J, Lemon SC. Evaluating the implementation of a hospital work-site obesity prevention intervention: applying the RE-AIM framework. Health Promot Pract 2012;13(2):190–7. 19. Glasgow RE, Askew S, Purcell P, Levine E, Warner ET, Stange KC, et al. Use of RE-AIM to address health inequities: application in a low-income community health center based weight loss and hypertension selfmanagement program. Transl Behav Med 2013;3(2):200–10. 20. Almeida FA, Pardo KA, Seidel RW, Davy BM, You W, Wall SS, et al. Design and methods of “diaBEAT-it!”: a hybrid preference/randomized control trial design using the RE-AIM framework. Contemp Clin Trials 2014;38(2):383–96. 21. Glasgow RE, Nelson CC, Strycker LA, King DK. Using RE-AIM metrics to evaluate diabetes self-management support interventions. Am J Prev Med 2006;30(1):67–73. 22. Wozniak L, Rees S, Soprovich A, Al Sayah F, Johnson ST, Majumdar SR, et al. Applying the RE-AIM framework to the Alberta’s caring for diabetes project: a protocol for a comprehensive evaluation of primary care quality improvement interventions. BMJ Open 2012;(5). pii: e002099. doi: 10.1136/bmjopen-2012-002099. 23. Cleeland CS, Ryan KM. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singapore 1994;23(2):129–38. 24. Bourdel N, Alves J, Pickering G, Ramilo I, Roman H, Canis M. Systematic review of endometriosis pain assessment: how to choose a scale? Hum Reprod Update 2015;21(1):136–52. doi: 10.1093/humupd/dmu046. Epub 2014 Sep 1. 25. Jones G1, Jenkinson C, Kennedy S. Evaluating the responsiveness of the Endometriosis Health Profile Questionnaire: the EHP-30. Qual Life Res 2004;13(3):705–13. 26. Burns JW, Glenn B, Lofland K, Bruehl S, Harden RN. Stages of change in readiness to adopt a self-management approach to chronic pain: the moderating role of early-treatment stage progression in predicting outcome. Pain 2005;115(3):322–31. Epub 2005 Apr 19.
SEPTEMBER JOGC SEPTEMBRE 2015 l 809
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework Innie Chen, MD, MPH,1,2 Deborah Money, MD,3 Paul Yong, MD, PhD,3 Christina Williams, MD,3 Catherine Allaire, MD3 Department of Obstetrics and Gynecology, University of Ottawa and the Ottawa Hospital Research Institute, Ottawa ON
1
School of Population and Public Health, University of British Columbia, Vancouver BC
2
Department of Obstetrics and Gynaecology, University of British Columbia, Vancouver BC
3
Abstract
Résumé
Objective: Chronic pelvic pain (CPP) is a prevalent, debilitating, and costly condition. Although national guidelines and empiric evidence support the use of a multidisciplinary model of care for such patients, such clinics are uncommon in Canada. The BC Women’s Centre for Pelvic Pain and Endometriosis was created to respond to this need, and there is interest in this model of care’s impact on the burden of disease in British Columbia.
Objectif : La douleur pelvienne chronique (DPC) est un trouble prévalent, débilitant et coûteux. Bien que des lignes directrices nationales et des données empiriques soutiennent l’utilisation d’un modèle de soins multidisciplinaires pour les patientes qui présentent des DPC, les cliniques canadiennes offrant un tel modèle sont peu nombreuses. Le BC Women’s Centre for Pelvic Pain and Endometriosis a été créé pour répondre à ce besoin et un intérêt a été manifesté à l’endroit des effets de ce modèle de soins sur le fardeau de la maladie en Colombie-Britannique.
We sought to create an approach to its evaluation using the RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) evaluation framework to assess the impact of the care model and to guide clinical decision-making and policy.
Methods: The RE-AIM evaluation framework was applied to consider the different dimensions of impact of the BC Centre. The proposed measures, data sources, and data management strategies for this mixed-methods approach were identified. Results: The five dimensions of impact were considered at individual and organizational levels, and corresponding indicators were proposed to enable integration into existing data infrastructure to facilitate collection and early program evaluation. Conclusion: The RE-AIM framework can be applied to the evaluation of a multidisciplinary chronic pelvic pain clinic. This will allow better assessment of the impact of innovative models of care for women with chronic pelvic pain.
Nous avons cherché à créer une approche envers son évaluation en utilisant le cadre de référence RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) pour déterminer les effets de ce modèle de soins et pour orienter le processus décisionnel clinique et la formulation de politiques en la matière.
Méthodes : Le cadre d’évaluation RE-AIM a été appliqué pour nous permettre d’envisager les diverses dimensions des effets du BC Centre. Les mesures proposées, les sources de données et les stratégies de gestion des données pour cette approche à méthodes mixtes ont été identifiées. Résultats : Les cinq dimensions des effets ont été prises en considération aux niveaux personnel et organisationnel, et des indicateurs correspondants ont été proposés pour en permettre l’intégration dans l’infrastructure de données existante, en vue de faciliter la collecte des données et l’évaluation précoce du programme. Conclusion : Le cadre RE-AIM peut être appliqué à l’évaluation d’une clinique multidisciplinaire visant la douleur pelvienne chronique. Cela permettra la tenue d’une meilleure évaluation des effets de modèles novateurs de soins chez les femmes qui connaissent des douleurs pelviennes chroniques.
Key Words: Chronic pelvic pain, multidisciplinary, RE-AIM, program evaluation Competing Interests: None declared. Received on November 8, 2014 Accepted on March 16, 2015
804 l SEPTEMBER JOGC SEPTEMBRE 2015
J Obstet Gynaecol Can 2015;37(9):804–809
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework
INTRODUCTION
C
hronic pelvic pain (CPP) is a prevalent, debilitating, and costly condition that affects nearly 15% of the general population.1,2 The burden of this disease is reflected in the associated reduction in quality of life, loss of productivity, and increased use of health care services,3 with an estimated annual cost of $1.8 billion in Canada from endometriosis alone.4 Chronic pelvic pain is a complex condition that often affects multiple body systems, and this can lead to challenges in the care of affected individuals.5 Given the complex nature of CPP, Canadian and other national guidelines have recommended use of a multidisciplinary model of care for the treatment of chronic pelvic pain,5–8 and the limited available evidence points to the efficacy of such a model.9,10 Multidisciplinary models of care that offer the services and support of psychotherapy and physiotherapy in addition to medical and surgical therapy within the same clinic are uncommon in Canada, although they are present in some areas. The result is a delay in diagnosis and suboptimal treatment for many patients with CPP. The BC Women’s Centre for Pelvic Pain and Endometriosis is a multidisciplinary program offering the expertise of a pelvic floor physiotherapist, a sexual and reproductive health counsellor, a nurse educator, and an anaesthetist, as well as gynaecologists trained in advanced laparoscopic surgery. Complex CPP patients also attend an education session before consultation with a physician, and are offered group education to learn about lifestyle and behavioural techniques to deal with CPP. This program was established with provincial health authority funding for the purpose of reducing the current burden of disease. Provincial needs assessment, national guidelines, and expert opinion all supported the development, implementation, and maintenance of the program. Whether the model is adopted in other areas of Canada will depend on its assessed impact. The RE-AIM framework was initially developed and introduced by Glasgow et al. in 1999 for the purpose of systematically evaluating management interventions for chronic illness.11 While evaluations of health interventions are generally limited to assessing efficacy within controlled settings, the RE-AIM framework acknowledges the importance of other factors that influence the value of an intervention in real-world settings. By considering the five dimensions of Reach, Efficacy, Adoption, Implementation, and Maintenance, the framework provides measures of longer term effect on individuals and populations.12 The dimensions of RE-AIM follow a logical sequence, and have been applied to the evaluation of health interventions
in a variety of ways.13 RE-AIM has been widely adopted in the evaluation of physical activity programs,14–17 obesity prevention programs,18,19 and interventions for the management of diabetes14,17,20–22 and other chronic diseases,11,13 but its use in the evaluation of chronic pelvic pain programs has not yet been described. We describe here an adaption of the RE-AIM framework for the evaluation of a multidisciplinary chronic pelvic pain program. METHODS
Using a mixed-methods approach, we considered the five dimensions of the RE-AIM model at individual and organizational levels, and developed an evaluation approach specific to this clinic to assess the impact of this program on the burden of chronic pelvic pain and endometriosis in British Columbia. The overarching evaluation question guiding this framework was whether women were receiving accessible, effective, and sustainable treatments through the BC Women’s Centre for Pelvic Pain and Endometriosis. The specific questions guiding the evaluation were: 1. Reach—whether the intervention is reaching the target population of women with chronic pelvic pain; 2. Effectiveness—what the effect of the multidisciplinary model is on the treatment of chronic pelvic pain; 3. Adoption—whether the multidisciplinary model has been espoused by other health care professionals or other centres; 4. Implementation—whether the model of care has been implemented as intended; and 5. Maintenance—whether the implementation is costeffective. RESULTS
Proposed indicators for each of the five dimensions are listed in the Table. Reach
Reach refers to the proportion of the population that is potentially served by the program, and is a measure of program access and participation. In this case, the denominator is women of reproductive age affected by CPP who required specialist referral in British Columbia, and the numerator is women who have accessed this centre. Concrete barriers to access include geographic distance and logistical concerns such as taking time off work and SEPTEMBER JOGC SEPTEMBRE 2015 l 805
Proportion of women of reproductive age affected by CPP requiring specialist referral in British Columbia who have access to the clinic
Behavioural, quality of life, and patient satisfaction outcomes from multidisciplinary model of care
Ability of the physician to recognize the need to address the psychosocial aspect of pain in the diagnosis, education, and treatment of a patient
Adoption of the current multidisciplinary model of care at other cities within Canada
Participant adherence to treatment
Application of the biopsychosocial multidisciplinary framework for all patients requiring treatment for CPP and commitment to adopting and helping create most current and best practices for the treatment of this chronic condition
Reach
Efficacy
Adoption individual
Adoption institutional
Implementation individual
Implementation institutional
Program-specific definition
806 l SEPTEMBER JOGC SEPTEMBRE 2015 Occurrence of and attendance at regular interdisciplinary meetings to discuss complex patients
Member participating in related CME, such as International Pelvic Pain Society Meeting Scientific presentations and publications in pelvic pain Breadth of web-based, paper, and inperson learning tools.
Ongoing commitment to current and best practices Knowledge dissemination Knowledge translation
Use of urine drug screen and opioid risk tool for patients on opioid therapy
Appropriate use of opioids Multidisciplinary collaboration
Attendance at education sessions, physician, physiotherapy, and counsellor visits
Number of multidisciplinary care models in Canada
Patient participation
National dissemination
Number of CME courses offered and physicians attended Number of physicians and fellows trained
Physician outreach and mentoring
Endometriosis Health Profile-30
Quality of life Number of physician support telephone calls
Survey item
Patient satisfaction with treatment
Physician support
Brief Pain Inventory
Targets program participants Continued
Targets other health care providers
Reflects need for pelvic pain specialists to congregate and educate each other
Measures member engagement and multidisciplinary dialogue
Monitoring for addiction risk associated with opioids
Direct measure of participation
Identification of potential collaborators
Measure of physician interest in the program, recognition of the program model, and dissemination of knowledge
Measure of collaboration in caring for patients
Specifically developed for endometriosis Easily modified for chronic pelvic pain
Standardized and ease of administration
Standardized questionnaire addressing both pain symptoms and ability to function Commonly used research tool
Measure of use and access
Monitors use of this access program
Use of transportation vouchers Waitlist for initial consultation appointment Waitlist for surgical procedure
Captured with patient demographics Enables mapping to geographic regions
Rationale
Patient postal code
Measurement tool
Pelvic pain symptoms
Access to clinical services
Geographic reach
Indicator
Program-specific definitions and indicators for each of the five RE-AIM dimensions in the evaluation of a multidisciplinary chronic pelvic pain clinic
Gynaecology
Reflects ongoing improvement in patient outcomes
Narcotic use is an indirect measure of disability
Standardized data collection by hospital and administrative databases
Efficacy indicators at one year follow-up post-discharge Proportion discharged from clinic Proportion rereferred to clinic
Proportion on disability and social assistance Return to work
Physician document Pharmacy database system
Physician billings for pelvic pain
Ongoing improvement in symptoms and quality of life
Productivity
Cessation of narcotic medicine
Health care use
Individual’s ability to be pain free, to have an acceptable quality of life, and to engage in society and meaningful work
Ability to demonstrate overall cost savings from decreased health care use of program
Maintenance individual
Maintenance institutional
Discharge abstract database for hospital admissions Emergency department data for emergency visits
Rationale Measurement tool Indicator Program-specific definition
Continued
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework
childcare. Some interventions to help reduce these barriers include reimbursement of travel costs for patients from outside the Lower Mainland of British Columbia, the option of viewing online education modules and arranging for follow-up by telephone, and the ability to schedule physician visits, allied health professional visits, and group education sessions on the same day. Other barriers include wait times for consultation and surgery. Interventions to help alleviate this include physician triaging of referrals and the availability of telephone support for physicians in the community. The geographic reach of the program can be measured by mapping patients’ residential postal codes and tracking the use of transportation vouchers, while access to clinical services can be assessed by monitoring the length of the wait lists for consultations and surgical procedures. Efficacy
Efficacy refers to behavioural, quality of life, and patient satisfaction outcomes of a given intervention. With the use of patient questionnaires and the introduction of electronic medical records, a process is already in place for evaluation of clinic efficacy. Patients seen for CPP complete questionnaires on self-reported pain and the effect of pain on quality of life to determine the attributable level of pain and debilitation. Results are tracked through time to ascertain treatment effect. Physical examinations and treatments are also standardized and monitored to determine the efficacy of specific interventions for specific subpopulations. Measures of pelvic pain symptoms can be assessed with the Brief Pain Inventory,23,24 and quality of life can be measured using the Endometriosis Health Profile-30,25 which was specifically developed for endometriosis and also subsequently tailored for pelvic pain. In addition, all program participants can be surveyed to determine their satisfaction with treatment provided by the Centre. We anticipate that patients in the pre-contemplative stage of readiness for change may be resistant to the multidisciplinary model, and a Pain Stages of Change Questionnaire may be helpful in identifying these individuals to permit a separate analysis.26 Adoption
Adoption refers to the proportion of institutions that assume a policy or program. In this case, it refers to the uptake of the biopsychosocial model of care on two levels. The first level of adoption refers to the approach to patient care by individual physicians, and the ability of the physician to recognize the need to address the psychosocial aspect of pain and incorporate education, lifestyle, psychological, SEPTEMBER JOGC SEPTEMBRE 2015 l 807
Gynaecology
and physical therapies into a comprehensive treatment plan. At our Centre, this is enabled by providing physician education and support through dissemination of clinic resources via the Centre’s website, organization of CME courses, provision of physician telephone consultations, and incorporation of physician mentorship and training as part of the program. The Centre will monitor the involvement in such activities as measures of physician support, outreach, and monitoring. The second level of adoption refers to the adoption of the current multidisciplinary model of care at other locations within Canada. This is a longer term goal, and its realization will depend on the Centre’s ability to demonstrate efficacy through ongoing evaluation and to train physicians to replicate this model in other centres through the existing fellowship program. The Centre will identify other multidisciplinary care models in Canada as potential partners for collaboration. Implementation
At the individual level, implementation is a measure of a participant’s adherence to treatment, and is necessary for interpreting efficacy. At the program level, it refers to adherence to the principles and goals of the Centre; that is, the extent to which the program is delivered as it was intended. It refers to the clinic’s consistency in applying the biopsychosocial multidisciplinary framework for all patients requiring treatment for CPP, and its commitment to adopting and creating the best practices for treatment of this chronic condition. At present, the Centre conducts weekly multidisciplinary meetings to review complex CPP patients, develop patient care pathways, and review overall program functions. The level of these activities will be measured by holding and attending these multidisciplinary meetings, while commitment to best practices will be assessed by attendance at related CME events. The program also has a strong clinical and translational research mandate, and activity in this area is quantified by scientific presentations and publications and the development of patient education materials. Maintenance
At the individual level, this refers to the ability of the individual to maintain behavioural change. In the treatment of CPP, it refers to the individual’s ability to be pain free, to have an acceptable quality of life, and to engage in society and in meaningful work. This will be measured by the proportion of patients discharged from the Centre, the proportion requiring referral to the Centre, and efficacy indicators measured at one year follow-up after discharge. 808 l SEPTEMBER JOGC SEPTEMBRE 2015
At the institution level, it is the extent to which an intervention becomes part of the everyday culture and norms of the organization. The main determinant of sustainability is the ability to demonstrate overall cost savings from decreased use of health care services by program participants, as the program is directly tied to provincial support. Measures of economic burden of disease include those related to productivity, such as the proportion of patients on disability insurance and social assistance, as well as those able to return to work following treatment in the program. Use of narcotic medication tracked through physician documentation and pharmacy databases are an indirect measure of disability. Health care use is monitored through physician billings, emergency department visits, and hospital admissions. The execution of this evaluative model will involve the collection and management of data from a variety of sources. Clinical outcomes, patient-reported outcomes, and survey data can be entered by patients and clinicians into centralized, web-accessible databases housed on secure servers within the hospital. Provincial health care administrative databases can also be accessed to determine measures of health care use, including physician billings, emergency department visits, and hospital admissions. DISCUSSION
The use of the RE-AIM evaluation framework for a multidisciplinary chronic pelvic pain clinic represents a novel application of this framework to assess the reach, efficacy, adoption, implementation and maintenance aspects of this unique model of care in Canada. As a possible prototypic model for the treatment of chronic pelvic pain in Canada, the potential for institutional adoption, implementation and maintenance of this model in other Canadian centres will depend on the ongoing results of systematic evaluation. The strength of applying the RE-AIM framework for this purpose is that it takes into consideration other dimensions of impact rather than efficacy alone. Although the REAIM framework was initially developed for public health interventions, the utility of this approach for evaluation has been successfully demonstrated for other health care interventions.12 Glasgow et al. demonstrated how the RE-AIM framework can be used to evaluate components of care for chronic illness management, and showed the importance of considering multiple dimensions rather than efficacy alone in program planning, implementation, and funding allocation.11 In our search of the scientific literature, we were unable to identify any published reports
An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework
on an evaluative framework for multidisciplinary chronic pelvic pain models. As chronic disease becomes a greater health care priority, and the need for multidisciplinary models of care becomes increasingly recognized, the development of a framework for evaluation of these types of health care interventions becomes increasingly necessary. CONCLUSION
The RE-AIM framework can be successfully applied to the evaluation of a multidisciplinary chronic pelvic pain program. Such evaluation can inform key stakeholders and allow better assessment of the impact of innovative models of care for women with chronic pelvic pain. REFERENCES 1. Latthe P, Latthe M, Say L, Gülmezoglu M, Khan KS. WHO systematic review of prevalence of chronic pelvic pain: a neglected reproductive health morbidity. BMC Public Health 2006;6:177. 2. Ahangari A. Prevalence of chronic pelvic pain among women: an updated review. Pain Physician 2014;17(2):E141–7. 3. Mathias SD, Kuppermann M, Liberman RF, Lipschutz RC, Steege JF. Chronic pelvic pain: prevalence, health-related quality of life, and economic correlates. Obstet Gynecol 1996;87(3):321–7. 4. Levy AR, Osenenko KM, Lozano-Ortega G, Sambrook R, Jeddi M, Bélisle S, et al. Economic burden of surgically confirmed endometriosis in Canada. J Obstet Gynaecol Can 2011;33(8):830–7. 5. Jarrell JF, Vilos GA; SOGC Chronic Pelvic Pain Committee. Consensus guidelines for the management of chronic pelvic pain. SOGC Clinical Practice Guideline no. 164, part 1, August 2005. J Obstet Gynaecol Can 2005;27(8):781–826. 6. Jarrell JF, Vilos GA; SOGC Chronic Pelvic Pain Committee. Consensus guidelines for the management of chronic pelvic pain. SOGC Clinical Practice Guideline no. 164, part 2, September 2005. J Obstet Gynaecol Can 2005;27(9):869–910. 7. Leyland N, Casper R, Laberge P, Singh SS; SOGC Clinical Practice Gynaecology Committee. Endometriosis: diagnosis and management. SOGC Clinical Practice Guideline no. 244, July 2010. J Obstet Gynaecol Can 2010;32(7):S1–S32. 8. ACOG Committee on Practice Bulletins–Gynecology. ACOG practice bulletin no. 51. Chronic pelvic pain. Obstet Gynecol 2004;103(3):589–605. 9. Lamvu G, Williams R, Zolnoun D, Wechter ME, Shortliffe A, Fulton G, et al. Long-term outcomes after surgical and nonsurgical management of chronic pelvic pain: one year after evaluation in a pelvic pain specialty clinic. Am J Obstet Gynecol 2006;195(2):591–8. 10. Kames LD, Rapkin AJ, Naliboff BD, Afifi S, Ferrer-Brechner T. Effectiveness of an interdisciplinary pain management program for the treatment of chronic pelvic pain. Pain 1990;41(1):41–6.
11. Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns 2001;44(2):119–27. 12. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health 1999;89(9):1322–7. 13. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health 2013;103(6):e38–46. 14. Antikainen I, Ellis R. A RE-AIM evaluation of theory-based physical activity interventions. J Sport Exerc Psychol 2011;33(2):198–214. 15. Austin G, Bell T, Caperchione C, Mummery WK. Translating research to practice: using the RE-AIM framework to examine an evidence-based physical activity intervention in primary school settings. Health Promot Pract 2011;12(6):932–41. 16. De Meij JS, Chinapaw MJ, Kremers SP, Van der Wal MF, Jurg ME, Van Mechelen W. Promoting physical activity in children: the stepwise development of the primary school-based JUMP-in intervention applying the RE-AIM evaluation framework. Br J Sports Med 2010;44(12):879–87. 17. Matthews L, Kirk A, Macmillan F, Mutrie N. Can physical activity interventions for adults with type 2 diabetes be translated into practice settings? A systematic review using the RE-AIM framework. Transl Behav Med 2014;4(1):60–78. 18. Estabrook B, Zapka J, Lemon SC. Evaluating the implementation of a hospital work-site obesity prevention intervention: applying the RE-AIM framework. Health Promot Pract 2012;13(2):190–7. 19. Glasgow RE, Askew S, Purcell P, Levine E, Warner ET, Stange KC, et al. Use of RE-AIM to address health inequities: application in a low-income community health center based weight loss and hypertension selfmanagement program. Transl Behav Med 2013;3(2):200–10. 20. Almeida FA, Pardo KA, Seidel RW, Davy BM, You W, Wall SS, et al. Design and methods of “diaBEAT-it!”: a hybrid preference/randomized control trial design using the RE-AIM framework. Contemp Clin Trials 2014;38(2):383–96. 21. Glasgow RE, Nelson CC, Strycker LA, King DK. Using RE-AIM metrics to evaluate diabetes self-management support interventions. Am J Prev Med 2006;30(1):67–73. 22. Wozniak L, Rees S, Soprovich A, Al Sayah F, Johnson ST, Majumdar SR, et al. Applying the RE-AIM framework to the Alberta’s caring for diabetes project: a protocol for a comprehensive evaluation of primary care quality improvement interventions. BMJ Open 2012;(5). pii: e002099. doi: 10.1136/bmjopen-2012-002099. 23. Cleeland CS, Ryan KM. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singapore 1994;23(2):129–38. 24. Bourdel N, Alves J, Pickering G, Ramilo I, Roman H, Canis M. Systematic review of endometriosis pain assessment: how to choose a scale? Hum Reprod Update 2015;21(1):136–52. doi: 10.1093/humupd/dmu046. Epub 2014 Sep 1. 25. Jones G1, Jenkinson C, Kennedy S. Evaluating the responsiveness of the Endometriosis Health Profile Questionnaire: the EHP-30. Qual Life Res 2004;13(3):705–13. 26. Burns JW, Glenn B, Lofland K, Bruehl S, Harden RN. Stages of change in readiness to adopt a self-management approach to chronic pain: the moderating role of early-treatment stage progression in predicting outcome. Pain 2005;115(3):322–31. Epub 2005 Apr 19.
SEPTEMBER JOGC SEPTEMBRE 2015 l 809