- Email: [email protected]
The Use of the Analytic Hierarchy Process to Aid Decision Making in Acquired Equinovarus Deformity
457
ORIGINAL ARTICLE
The Use of the Analytic Hierarchy Process to Aid Decision Making in Acquired Equinovarus Deformity Janine A. van Til, MSc, Gerbert J. Renzenbrink, MD, James G. Dolan, MD, Maarten J. IJzerman, PhD ABSTRACT. van Til JA, Renzenbrink GJ, Dolan JG, IJzerman MJ. The use of the analytic hierarchy process to aid decision making in acquired equinovarus deformity. Arch Phys Med Rehabil 2008;89:457-62. Objective: To increase the transparency of decision making about treatment in patients with equinovarus deformity poststroke. Design: The analytic hierarchy process (AHP) was used as a structured methodology to study the subjective rationale behind choice of treatment. Setting: An 8-hour meeting at a centrally located rehabilitation center in The Netherlands, during which a patient video was shown to all participants (using a personal computer and a large screen) and the patient details were provided on paper. Participants: A panel of 10 health professionals from different backgrounds. Interventions: Not applicable. Main Outcome Measures: The performance of the applicable treatments on outcome, impact, comfort, cosmetics, daily effort, and risks and side effects of treatment, as well as the relative importance of criteria in the choice of treatment. Results: According to the model, soft-tissue surgery (.413) ranked first as the preferred treatment, followed by orthopedic footwear (.181), ankle-foot orthosis (.147), surface electrostimulation (.137), and finally implanted electrostimulation (.123). Outcome was the most influential consideration affecting treatment choice (.509), followed by risk and side effects (.194), comfort (.104), daily effort (.098), cosmetics (.065), and impact of treatment (.030). Conclusions: Soft-tissue surgery was judged best on outcome, daily effort, comfortable shoe wear, and cosmetically acceptable result and was thereby preferred as a treatment alternative by the panel in this study. In contrast, orthosis and orthopedic footwear are usually preferred in daily practice. The AHP method was found to be suitable methodology for eliciting subjective opinions and quantitatively comparing treatments in the absence of scientific evidence. Key Words: Decision making; Equinovarus; Rehabilitation; Stroke. © 2008 by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation
From the Roessingh Research and Development, Enschede, The Netherlands (van Til, IJzerman); University of Twente, Enschede, The Netherlands (van Til, IJzerman); Roessingh Rehabilitation Center, Enschede, The Netherlands (Renzenbrink); Unity Health System, Rochester, NY (Dolan); and University of Rochester, Rochester, NY (Dolan). Supported by ZONmw (grant no. 14350026). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are associated. Correspondence to Janine A. van Til, MSc, University of Twente, PO Box 217, 7500 AE Enschede, The Netherlands, e-mail: [email protected]. Reprints are not available from the author. 0003-9993/08/8903-00554$34.00/0 doi:10.1016/j.apmr.2007.09.030
STROKE, OR A cerebrovascular accident, is a common disorder that can result in paralysis of 1 side of the body. A About 20% of stroke patients are confronted with a deviant position of the ankle and foot that hinders standing and walking. The impairment is known as an equinovarus deformity. In 2003, the International Society for Prosthetics and Orthotics developed guidelines for the orthotic management of stroke patients.1,2 The guidelines provide insight in the available treatments for equinovarus deformity, but no support is given on the decision for treatment if multiple treatments are available. Although the specifics of the deformity influence the availability of treatments, in general, surgical,3,4 technologic,5 pharmaceutical,6 and orthotic7 treatments are available to correct the deformity.8 Treatment is primarily aimed at correcting the deviant foot position. Secondary beneficial effects include reduction or stimulation of muscle force, improvement of walking speed and distance, and reduction of energy consumption. Evidence-based choice of treatment is hindered because the available clinical evidence base consists of only a few articles with small sample size and poor methodologic quality.2,3,5,9 In clinical practice, however, the decision for treatment of equinovarus deformity has to be made on a daily basis in the absence of convincing evidence. Treatment decisions in acquired neurologic equinovarus deformity can thereby be described as a preference-sensitive, or equipoise, decision, where choice is based on personal preferences and convictions rather than evidence.10 According to Frith11 the perceived attractiveness of treatment involves many subjective elements in addition to potential effectiveness. The subjective preferences of clinicians in daily practice therefore might hold valuable information on the strengths and weaknesses of treatment alternatives. It is difficult to elicit and measure subjective preferences in health care decision making. The analytic hierarchy process (AHP) is an approach to multicriteria decision making problems of choice and prioritization. The AHP has been previously used to compare the performance of intensive care units,12 allocate livers among transplant patients,13 select tests for abdominal pain,14 and support the choice of reconstructive interventions in spinal cord injury.15 The AHP evaluates decision problems by determining the relative importance of criteria and the performance of available solutions through a series of trade-offs.16 The comparison can include medical, technical, economic, and social aspects. Subjective judgments on aspects of a problem for which no scale of measurement exists can be easily accommodated.17 The aim of this study was to test the AHP as a methodology for studying decision making regarding choice of treatment in rehabilitation medicine in equinovarus deformity. METHODS An AHP analysis can be divided into 4 stages, namely, (1) model development, (2) judgment stage, (3) synthesis of results, and (4) sensitivity analysis. For a more detailed discussion on the technique and underlying methodology, we refer the reader to the literature.16,18,19 Arch Phys Med Rehabil Vol 89, March 2008
458
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
which they were mentioned. These treatments were soft-tissue surgery, functional electric stimulation (FES), ankle-foot orthosis (AFO), pharmaceutical treatment, and orthopedic shoes. A decision tree was also developed based on the group responses.
Fig 1. The AHP decision model. Impact of treatment comprises the impact to the patient in the active treatment phase, and daily effort comprises the daily actions needed to maintain the effect of treatment afterward (mainly donning and doffing necessary equipment). *Criteria supported by literature.26
Model Development The AHP decision model in this study consists of 4 levels (fig 1). In the first level, the goal of the analysis is presented. The second level consists of the main decision criteria. These criteria are split into subcriteria, which are included in the third level. Finally, the available alternatives are presented in the fourth level of the model. The goal of the analysis was set at selecting the preferred treatment for equinovarus deformity after stroke. This goal supports the specific aim of the study, which was to obtain information on the preferred treatment of a selection of health care professionals for equinovarus deformity in stroke. To determine level 2 and 3 criteria, a Delphi-style paper-andpencil interview was conducted among a sample of Dutch physiatrists (n⫽28; response rate, 68%) united in a stroke interest group. The questionnaire was developed to identify a preliminary set of treatment requirements, criteria, and treatment alternatives for acquired equinovarus deformity in a postacute phase (⬎6wk) of stroke. An example of the first questionnaire as it was sent to the group is presented in appendix 1. Simultaneously, a literature search was conducted using the search terms stroke, equinovarus deformity, ankle-foot impairment, and treatment.3-7 A summary of responses to the initial questionnaire was sent back to participants, along with the treatments reported in literature and 2 additional rounds of feedback from the group were processed. After 3 rounds of questionnaires, 5 feasible treatment categories for the postacute management of equinovarus foot were identified from the treatments selected by the panel based on the frequency with Arch Phys Med Rehabil Vol 89, March 2008
Judgment Stage A panel of 10 professionals in rehabilitation medicine was asked to participate in the judgment phase. The panel consisted of 4 physiatrists, 1 orthopedic surgeon, 1 physical therapist, 2 senior researchers in the field of stroke, and 2 certified orthotists. To start the discussion, a patient case description (fig 2) and video were presented to the panel. The patient had moderate equinovarus deformity and was chosen so that it could be treated with the maximal number of treatments identified during the Delphi process. Based on their individual expertise, the panel members supported the use of the following treatments for equinovarus deformity: split tibial tendon transfer combined with a transfer of the hallucis longus muscle (soft-tissue surgery), an off-the-shelf AFO, off-the-shelf semi-orthopedic shoes, an implanted peroneal stimulator, and a surface peroneal stimulator. Pharmaceutical treatment was omitted as a treatment alternative because of the absence of marked spasticity. The decision model was presented to the panel on a projection screen, which stood in the front of the room. The judgment stage consists of 2 phases. First, the treatment performance was judged on the level 3 subcriteria. These judgments were made by comparing the treatments in a pairwise fashion using a reciprocal numeric scale ranging from 1 to 9 (an example can be found in appendix 2). In the AHP, the numbers are associated with verbal statements ranging from equally preferred (1) to extremely more preferred (9). In the next phase of the judgment stage, the panel members were first asked to judge the importance of the level 3 subcriteria with regard to the accomplishment of the level 2 criteria and then to judge the contribution (importance) of level 2 criteria in the achievement of the goal. These judgments were also made in a pairwise fashion using the reciprocal numeric scale described earlier using verbal statements ranging from equally important (1) to extremely more important (9).
Mrs. L. is a 61-year-old female who had a right-side stroke with hemiplegia on the left approximately 1 year ago. She is an independent outside walker (FAC 5) but complains about feelings of insecurity during walking on an uneven surface and walking during the night, for instance, toilet visits. Mrs. L. has normal cognitive functioning and no impairment in hand function due to stroke. Additionally, she suffers from high blood pressure. During physical assessment of walking pattern the following deviations were noticed: No deviations were seen in the right leg. During stance phase of the left leg, initial contact of the foot is seen on the lateral border and heel of the foot. Increased first rocker. The knee is in flexion at initial contact with delayed extension to terminal stance. During swing phase there is decreased flexion of the knee, not hindering foot clearance. The heel of the foot is in varus with deviation of calcaneus and supination of forefoot. Hyperextension of the first toe is present during swing and stance phase of walking. Strength of the hip in flexion and extension is MRC 4. Abduction is 5. Knee flexion is 5 and knee extension 4+. Ankle plantarflexion (in pattern) is 4, dorsiflexion is 4+, inversion is 5, and eversion is 4. Mobility of the hips and knees is normal. Ankle dorsiflexion left is limited with 5° (5/0/20) with a straight leg and normal with bend leg (10/0/20). Normal plantarflexion, decreased mobility of the calcaneus on the left (25/0/0). No marked spasticity, no sensibility disorder, and no peripheral circulation problems were present.
Fig 2. Patient case description. Observations were based on extensive manual testing in lying and sitting positions and visual observation of the patient during walking by an experienced physiotherapist. Abbreviations: FAC, Functional Ambulation Category; MRC, Medical Research Council scale.
459
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til Table 1: Importance Weights for All Criteria Level 2
Average
SD
Level 3
Average
SD
Outcome
.509
.180
Risks
.194
.115
Comfort Daily effort
.104 .098
.086 .110
Cosmetics Impact
.065 .030
.044 .023
Active ankle stability Passive ankle stability Foot position in stance Foot position in swing Short-term risks Long-term side effects Reliability of treatment Consequences on comfortable shoe wear Daily time investment Complexity of daily investment Cosmetic consequences Duration of treatment Complexity of treatment
.441 .059 .102 .398 .386 .449 .165 .104 .770 .230 .065 .760 .240
.023 .023 .057 .057 .099 .089 .035 .086 .031 .031 .044 .055 .055
Abbreviation: SD, standard deviation.
Initially, individual judgments were made by the panel members. When all judgments were completed, disagreement among panel members was made explicit by presenting the numeric judgments on a screen visible to each panel member and by verbal explanation of the discussion leader. In case of disagreement, panel members with the most diverging scores were asked to clarify their judgments. In many cases, this resulted in a discussion of pros and cons of the treatments or the relevance of criteria. Panel members were allowed to reconsider and to alter their judgments if they believed it was necessary based on the discussion. To reduce the time requirements of the session, the number of performance and importance judgments was reduced to the minimal number required for calculating weights on an individual and group level. A total of 52 performance and 13 importance judgments were made by the 10 members of the panel. There was a group discussion after each judgment. The duration of the session was about 7 hours (including breaks). Synthesis of Results In the AHP, the numeric judgments are put in a comparison matrix (A). The reciprocal scores are used to complete the matrix. The principle eigenvector method is used to calculate the individual importance and performance weights from the matrix. For a detailed explanation of the hows and whys of the eigenvector method the reader is referred to the literature,16,20 but for the scope of the current article it is enough to know that a close approximation of the priority vector of matrix A can be calculated by dividing each element in the matrix by the sum of its column and then dividing the sum of each row by the sum of the matrix. Group values are calculated using the geometric mean of individual judgments. A priority score (V) for each treatment (a) is calculated based on an additive value function:
study, we calculated consistency for the entire group using the group mean comparisons as described earlier. Sensitivity Analysis To determine the impact of importance weights on overall priority for treatment, the importance of each level 2 criterion was varied from 0 to 1, in increments of 0.1, and the sum of importance weights was kept at 1.0. The priority for treatment was calculated for all combinations. RESULTS Criteria Weights The relative priorities of the level 2 and 3 criteria are presented in table 1. In the opinion of the expert panel, on the second level the outcome of treatment is the most important criterion in making treatment decisions. With an importance weight of .509 it influences half of the decision. Active ankle stability (.44) and foot position in swing phase of walking (.4) were judged to be the most important level 3 parameters when outcome is considered, whereas passive ankle stability and improvement of the foot position in stance were judged less important. Performance Weights The performance values for all treatments are presented in figure 3.
i⫽1
Priority of Treatments Soft-tissue surgery was the first ranked treatment in all participants, with an average priority of .413 and a range between .313 and .614. Semi-orthopedic shoes (.181; range, .081–.230) ranked second, followed by the AFO (.147; range, .098 –.170), surface peroneal stimulator (.137; range, .079 –.230), and finally implanted peroneal stimulator (.123; range, .099 –.169). Individual priority scores along with participants’ professions are presented in table 2.
Criteria weights are multiplied with performance values and summed, resulting in an overall priority (performance rating) for each alternative. Measurement of the consistency of judgments is a routine part of the AHP and is measured by calculating a consistency index, which ranges from 0 to 1. Analyses with consistency indices less than or equal to 0.1 are generally considered acceptable. In certain cases, a cutoff value of 0.2 is used. In this
Sensitivity Analysis Sensitivity analysis was performed to study the influence of criteria weights on treatment ranking. The priority for treatment is pictured in figure 4. The first ranked treatment was soft-tissue surgery in 77%, orthopedic shoes in 3%, and AFO in 20% of cases. No combination of importance weights results in either surface or implanted electric stimulation having the highest priority.
V(a) ⫽
m
兺 w v (a) i i
Arch Phys Med Rehabil Vol 89, March 2008
460
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
Fig 3. The average performance values with standard deviation of the treatments on all level 3 criteria. IFES, implanted peroneal stimulator; SFES, surface peroneal stimulator; SOS, semiorthopedic shoes; TS, soft-tissue surgery.
DISCUSSION Using the AHP methodology, soft-tissue surgery was the preferred treatment in this study. The panel judged soft-tissue surgery performance best on outcome, cosmetics, comfort, and daily effort. Orthopedic shoes and AFOs perform modestly on most criteria. Strong features include the low impact of treatment and the low risk that is associated with treatment in both orthopedic shoes and AFOs. The performance of FES was judged low. As was shown in the sensitivity analysis, FES was dominated by the other treatment alternatives for equinovarus deformity, because no combination of criteria weights resulted in a priority for FES. According to the expert panel, problems with surface FES include the complexity of electrode and heel switch placement in a patient group that can be both cognitively and/or physically impaired. Although an effort to reduce daily effort was made by the development of an implantable device, the surgical procedure negatively influences treatment impact and risks and side effects. In terms of outcome, the timing of the electric stimulus, which is the signal for foot elevation during the swing phase of walking, is thought to be unreliable. Naturally, as a result of ongoing development in rehabilitation medicine the conclusions of this experiment regarding preference for treatment are temporary, and updating the model with new and if possible objective information as it becomes available would be essential.21
Table 2: Indvidual Priority for Treatment Treatment Priority
TS
IFES
SFES
AFO
SOS
Orthopedic surgeon Orthotist Orthotist Physiatrist Physiatrist Physiatrist Physiatrist Physical therapist Senior researcher Senior researcher
.614 .313 .343 .337 .541 .550 .430 .475 .453 .463
.114 .169 .138 .099 .116 .104 .131 .132 .129 .116
.092 .195 .139 .230 .096 .079 .117 .103 .094 .149
.098 .164 .150 .170 .104 .106 .159 .104 .128 .113
.081 .159 .230 .165 .142 .160 .163 .186 .196 .159
Abbreviations: IFES, implanted peroneal stimulator; SFES, surface peroneal stimulator; SOS, semi-orthopedic shoes; TS, soft-tissue surgery.
Arch Phys Med Rehabil Vol 89, March 2008
Fig 4. Sensitivity analysis. A 6-dimensional space is presented by showing each combination of weights for 2 criteria in 16 squares. If the preference for treatment is dependent on the weight of other criteria, multiple treatment priorities are piled for a 2-dimensional weight combination.
The preference for soft-tissue surgery expressed in this study is not in accordance with clinical practice. In the Netherlands, orthopedic shoes and AFOs are used most often in the treatment of equinovarus deformity. Panel discussion suggested that this might be explained by the low impact of these treatments in patients who are faced with many cognitive and physical disabilities and a great familiarity with and easy access to orthotic treatment in a rehabilitation setting. According to the panel the preference for soft-tissue surgery in this study is partly explained by the time since injury (⬎1y) and the good functional status of the patient in the case description. As a direct consequence, treatment impact is judged to be least and outcome is considered most important. In clinical practice the natural recovery of equinovarus deformity and walking ability is often maximal within weeks, and a decision regarding treatment has to be made at a time in which the treatment of equinovarus deformity as such might not have the highest priority. However, as time progresses, different trade-offs between the pros and cons of treatment are made. Another explanation for the limited use of soft-tissue surgery might be the limited experience of neurologists and physiatrists, who traditionally treat stroke patients, with the surgical possibilities in an adult population with upper motoneuron disorders. Consequently stroke patients are only rarely referred to orthopedic surgeons, although the results of this study indicate there is a high potential for the use of soft-tissue surgery in the later stages of recovery. This advocates a more active approach in bringing this often-neglected or unknown surgical approach in equinovarus deformity in stroke to the attention of treating physicians. In our opinion, the AHP methodology that was used in this study provided the condition for a structured discussion regarding the conditions for treatment decisions for equinovarus deformity in stroke.17,22 The conscious deliberation of relevant requirements for treatments gave insight into the thought processes of panel members. The judgment of performance on a standard set of well-defined criteria enabled a comparison of
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
treatments that was not possible based on the available literature. Individual knowledge and experience was made explicit and was shared among panel members. The reaction of the panel to the use of the AHP as an alternative tool for treatment comparison was diverse. During the analysis, some of the panel members raised concerns about the validity of the judgments because their experience and knowledge on some of the treatments was limited. In our opinion, the group discussion phase should have resolved these problems, because panel members were allowed to alter their judgments if they believed unknown information was presented. Moreover, the inclusion of and discussion between panel members from different backgrounds is essential in the comparison of conventional and innovative or specialized treatment options, because any individual’s knowledge and experience are always restricted. The time requirements to complete the model were considered a disadvantage, and the AHP was thought to be bothersome for use in day-to-day decision making. This reservation might be a result of the large size of the current model. Simpler models might be possible, although care must be taken to maintain important details. The AHP analysis gave valuable insights in the motivation of panel members and is thereby very well suited for both examination of treatment choice and guideline development in rehabilitation medicine when high-quality scientific evidence is unavailable. Retrospectively, panel members expressed concern regarding the generalizability of the results of the study to other patients because the judgment phase may have been influenced by the patient description used. We shall discuss these concerns on 2 levels—namely, the effect on performance values of the treatment alternatives and the effect on importance weights. First, the influence of the case description on the performance judgments of the expert panel was negligible. As became obvious from the videotape used to document the experiment, the patient description was rarely referred to by the panel members in the discussion, and judgments on performance were made on a more general level. Moreover, because the influence of minor variations in impairment on treatment performance on treatment outcome is only scarcely documented in literature, it is questionable whether it would become obvious from subjective judgments made by a panel of health professionals. For a number of other criteria, the effect of the case description would be comparable for all treatments (eg, more severe deformity would lead to higher treatment impact), which in turn does not influence the comparison. In our opinion, therefore, the judgments on treatment performance on the lowest level are generalizable to other patients as well. Second,
461
as was mentioned before, the specifics of the ankle-foot deformity and the time since injury probably did influence the panel’s judgments about the relative importance of the criteria on a higher level. In this patient, the relevance of treatment impact was judged to be negligible and outcome was deemed most important as a result of the specific status and demands of the patient. Because the impairment was especially bothersome in the swing phase of walking, the outcome of treatments on criteria related to the swing phase of walking were given a high importance weight. Criteria and subcriteria importance should therefore be determined for each patient to determine the preference for treatment in other cases. To support these statements, it would be worthwhile to repeat the analysis with another patient and/or another panel. The high importance that is given to health-related criteria such as outcome and risks of treatment may be a reflection of the preferences of the health professionals represented in the panel. As was shown by Hummel et al,15 patients value different aspects of treatment compared with health professionals. Treatment preference could thereby differ for patients. As patient involvement is seen as an important factor in ensuring quality improvement in health care,23,24 health professionals are increasingly encouraged to involve their patients in treatment decision making. The AHP model used in this study could be a way to include the personal aims, wishes, and demands of each patient into health care decision making. It has been shown previously15,25 that patients are capable of using the AHP to express their preferences. CONCLUSIONS When clinical practice is compared with the preference for treatment expressed using the AHP methodology, soft-tissue surgery is an undervalued treatment alternative in the treatment of moderate equinovarus deformity in the swing phase of walking, if a patient is willing and able to undergo a longer and more tedious treatment. However, the final choice of treatment remains dependent on individual trade-offs of patients and health care professionals, and this study showed that different trade-offs regarding criteria importance could lead to a different preference for treatment. Although some concerns were raised by the panel, these seem manageable, and the use of the AHP as a methodology to compare treatment alternatives in rehabilitation medicine is promising. Because large, randomized, controlled trials are not yet available and will remain difficult to execute because of the heterogeneity of the patient population and lack of funding in rehabilitation medicine, an alternative for decision aiding is highly applicable.
APPENDIX 1: QUESTIONNAIRE STROKE INTEREST GROUP An open-ended questionnaire with the following questions was sent to the participants. 1. How do you characterize equinovarus deformity poststroke? 2. Which interventions are you familiar with for equinovarus deformity poststroke? 3. Which of these interventions do you use in daily practice? 4. Which general patient characteristics do you take into account when prescribing treatment for equinovarus deformity poststroke? 5. Which specifics of the equinovarus deformity do you take into account when prescribing treatment? 6. Which general treatment characteristics do you take into account when prescribing treatment for equinovarus deformity poststroke? 7. Which specific characteristics or effects of the treatment do you take into account when prescribing treatment for equinovarus deformity poststroke? All responses were categorized and returned to participants. Participants were asked to check the boxes with the treatments, characteristics, or criteria that they found relevant in the decision for treatment. Based on frequency a selection of relevant criteria was made for use in the decision tree. Also, a selection of alternative treatments was made. Arch Phys Med Rehabil Vol 89, March 2008
462
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
APPENDIX 2: PERFORMANCE AND IMPORTANCE JUDGMENT Performance Which of the next treatments is preferred when the aim is to minimize risks? 9 □
8 □
7 □
6 □
5 □
4 □
3 □
2 □
Soft-tissue surgery
1 □
2 □
3 □
4 □
5 □
or
6 □
7 □
8 □
9 □
Ankle-foot orthosis
Importance Which of the next criteria is more important in determining the best treatments for ankle-foot impairments? 9 □
8 □
7 □
6 □
5 □
4 □
3 □
2 □
outcome
1 □
or
2 □
3 □
4 □
5 □
6 □
7 □
8 □
9 □
risks
Numeric Values
Verbal Description
Interpretation of Judgment
1 3 5 7 9 2,4,6,8
Equal Moderate Strong Very strong/demonstrated Extreme importance/dominance Compromise between judgments
Equal contribution to objective Experience or judgment slightly favors one criteria over the other Experience or judgment strongly favors one criteria over the other Dominance of criteria is demonstrated in practice Effect is significantly demonstrated in literature
References 1. Geertzen JH, Hijmans JM, van der Linde H. Prosthetic prescription in The Netherlands: an interview with clinical experts. Prosthet Orthot Int 2005;29:113-4. 2. Report of a consensus conference on the orthotic management of stroke patients. Copenhagen: International Society of Prosthetics and Orthotics; 2004. 3. Piccioni L, Keenan M. Surgical correction of varus and equinovarus deformity in the spastic patient. Oper Tech Orthop 1992;2:146-50. 4. Lawrence SJ, Botte MJ. Management of the adult, spastic, equinovarus foot deformity. Foot Ankle Int 1994;15:340-6. 5. Kottink AI, Oostendorp LJ, Buurke JH, Nene AV, Hermens HJ, Ijzerman MJ. The orthotic effect of functional electrical stimulation on the improvement of walking in stroke patients with a dropped foot: a systematic review. Artif Organs 2004;28:577-86. 6. Pittock SJ, Moore AP, Hardiman O, et al. A double-blind randomised placebo-controlled evaluation of three doses of botulinum toxin type A (Dysport) in the treatment of spastic equinovarus deformity after stroke. Cerebrovasc Dis 2003;15:289-300. 7. Lin SS, Sabharwal S, Bibbo C. Orthotic and bracing principles in neuromuscular foot and ankle problems. Foot Ankle Clin 2000;5: 235-64. 8. Jaivin JS, Bishop JO, Braly WG, Tullos HS. Management of acquired adult dropfoot. Foot Ankle 1992;13:98-104. 9. Beckerman H, Becher J, Lankhorst GJ, Verbeek AL. Walking ability of stroke patients: efficacy of tibial nerve blocking and a polypropylene ankle-foot orthosis. Arch Phys Med Rehabil 1996; 77:1144-51. 10. Trevena L, Barratt A. Integrated decision making: definitions for a new discipline. Patient Educ Couns 2003;50:265-8. 11. Frith L. Priority setting and evidence based purchasing. Health Care Anal 1999;7:139-51. 12. Hariharan S, Dey PK, Chen DR, Moseley HS, Kumar AY. Application of analytic hierarchy process for measuring and comparing the global performance of intensive care units. J Crit Care 2005;20:117-24; discussion 124-5. 13. Cook DR, Staschak S, Green WT. Equitable allocation of livers for orthotopic transplantation: an application of the Analytic Hierarchy Process. Eur J Oper Res 1990;48:49-56.
Arch Phys Med Rehabil Vol 89, March 2008
14. Castro F, Caccamo LP, Carter KJ, et al. Sequential test selection in the analysis of abdominal pain. Med Decis Making 1996;16:178-83. 15. Hummel JM, Snoek GJ, van Til JA, van Rossum W, IJzerman MJ. A multicriteria decision analysis of augmentative treatment of upper limbs in persons with tetraplegia. J Rehabil Res Dev 2005;42:635-44. 16. Saaty TL. How to make a decision: the analytic hierarchy process. Eur J Oper Res 1990;48:9-26. 17. Dyer RF, Forman EH. Group decision support with the Analytic Hierarchy Process. Decis Support Syst 1992;8:99-124. 18. Hatcher M. Voting and priorities in health care decision making, portrayed through a group decision support system, using analytic hierarchy process. J Med Syst 1994;18:267-88. 19. Dolan JG, Isselhardt BJ Jr, Cappuccio JD. The analytic hierarchy process in medical decision making: a tutorial. Med Decis Making 1989;9:40-50. 20. Saaty TL. Decision-making with the AHP: why is the principal eigenvector necessary? Eur J Oper Res 2003;145:85-91. 21. Dolan JG, Bordley DR. Using the analytic hierarchy process (AHP) to develop and disseminate guidelines. QRB Qual Rev Bull 1992;18:440-7. 22. Hummel JM, van Rossum W, Verkerke GJ, Rakhorst G. Medical technology assessment: the use of the analytic hierarchy process as a tool for multidisciplinary evaluation of medical devices. Int J Artif Organs 2000;23:782-7. 23. Kennedy A, Robinson A, Rogers A. Incorporating patients’ views and experiences of life with IBS in the development of an evidence based self-help guidebook. Patient Educ Couns 2003;50:303-10. 24. Robinson A, Thomson R. Variability in patient preferences for participating in medical decision making: implication for the use of decision support tools. Qual Health Care 2001;10(Suppl 1): i34-8. 25. Dolan JG, Bordley DR. Involving patients in complex decisions about their care: an approach using the analytic hierarchy process. J Gen Intern Med 1993;8:204-9. 26. Blankevoort L, Geertzen JH, Heerkens YF, Hijmans JM, Ursum J. Ontwikkeling van richtlijnen voor de indicatiestelling en het verstrekkingsproces van orthopedische orthesen (ORIVOO). Amsterdam: Eigen beheer; 2005.
ORIGINAL ARTICLE
The Use of the Analytic Hierarchy Process to Aid Decision Making in Acquired Equinovarus Deformity Janine A. van Til, MSc, Gerbert J. Renzenbrink, MD, James G. Dolan, MD, Maarten J. IJzerman, PhD ABSTRACT. van Til JA, Renzenbrink GJ, Dolan JG, IJzerman MJ. The use of the analytic hierarchy process to aid decision making in acquired equinovarus deformity. Arch Phys Med Rehabil 2008;89:457-62. Objective: To increase the transparency of decision making about treatment in patients with equinovarus deformity poststroke. Design: The analytic hierarchy process (AHP) was used as a structured methodology to study the subjective rationale behind choice of treatment. Setting: An 8-hour meeting at a centrally located rehabilitation center in The Netherlands, during which a patient video was shown to all participants (using a personal computer and a large screen) and the patient details were provided on paper. Participants: A panel of 10 health professionals from different backgrounds. Interventions: Not applicable. Main Outcome Measures: The performance of the applicable treatments on outcome, impact, comfort, cosmetics, daily effort, and risks and side effects of treatment, as well as the relative importance of criteria in the choice of treatment. Results: According to the model, soft-tissue surgery (.413) ranked first as the preferred treatment, followed by orthopedic footwear (.181), ankle-foot orthosis (.147), surface electrostimulation (.137), and finally implanted electrostimulation (.123). Outcome was the most influential consideration affecting treatment choice (.509), followed by risk and side effects (.194), comfort (.104), daily effort (.098), cosmetics (.065), and impact of treatment (.030). Conclusions: Soft-tissue surgery was judged best on outcome, daily effort, comfortable shoe wear, and cosmetically acceptable result and was thereby preferred as a treatment alternative by the panel in this study. In contrast, orthosis and orthopedic footwear are usually preferred in daily practice. The AHP method was found to be suitable methodology for eliciting subjective opinions and quantitatively comparing treatments in the absence of scientific evidence. Key Words: Decision making; Equinovarus; Rehabilitation; Stroke. © 2008 by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation
From the Roessingh Research and Development, Enschede, The Netherlands (van Til, IJzerman); University of Twente, Enschede, The Netherlands (van Til, IJzerman); Roessingh Rehabilitation Center, Enschede, The Netherlands (Renzenbrink); Unity Health System, Rochester, NY (Dolan); and University of Rochester, Rochester, NY (Dolan). Supported by ZONmw (grant no. 14350026). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are associated. Correspondence to Janine A. van Til, MSc, University of Twente, PO Box 217, 7500 AE Enschede, The Netherlands, e-mail: [email protected]. Reprints are not available from the author. 0003-9993/08/8903-00554$34.00/0 doi:10.1016/j.apmr.2007.09.030
STROKE, OR A cerebrovascular accident, is a common disorder that can result in paralysis of 1 side of the body. A About 20% of stroke patients are confronted with a deviant position of the ankle and foot that hinders standing and walking. The impairment is known as an equinovarus deformity. In 2003, the International Society for Prosthetics and Orthotics developed guidelines for the orthotic management of stroke patients.1,2 The guidelines provide insight in the available treatments for equinovarus deformity, but no support is given on the decision for treatment if multiple treatments are available. Although the specifics of the deformity influence the availability of treatments, in general, surgical,3,4 technologic,5 pharmaceutical,6 and orthotic7 treatments are available to correct the deformity.8 Treatment is primarily aimed at correcting the deviant foot position. Secondary beneficial effects include reduction or stimulation of muscle force, improvement of walking speed and distance, and reduction of energy consumption. Evidence-based choice of treatment is hindered because the available clinical evidence base consists of only a few articles with small sample size and poor methodologic quality.2,3,5,9 In clinical practice, however, the decision for treatment of equinovarus deformity has to be made on a daily basis in the absence of convincing evidence. Treatment decisions in acquired neurologic equinovarus deformity can thereby be described as a preference-sensitive, or equipoise, decision, where choice is based on personal preferences and convictions rather than evidence.10 According to Frith11 the perceived attractiveness of treatment involves many subjective elements in addition to potential effectiveness. The subjective preferences of clinicians in daily practice therefore might hold valuable information on the strengths and weaknesses of treatment alternatives. It is difficult to elicit and measure subjective preferences in health care decision making. The analytic hierarchy process (AHP) is an approach to multicriteria decision making problems of choice and prioritization. The AHP has been previously used to compare the performance of intensive care units,12 allocate livers among transplant patients,13 select tests for abdominal pain,14 and support the choice of reconstructive interventions in spinal cord injury.15 The AHP evaluates decision problems by determining the relative importance of criteria and the performance of available solutions through a series of trade-offs.16 The comparison can include medical, technical, economic, and social aspects. Subjective judgments on aspects of a problem for which no scale of measurement exists can be easily accommodated.17 The aim of this study was to test the AHP as a methodology for studying decision making regarding choice of treatment in rehabilitation medicine in equinovarus deformity. METHODS An AHP analysis can be divided into 4 stages, namely, (1) model development, (2) judgment stage, (3) synthesis of results, and (4) sensitivity analysis. For a more detailed discussion on the technique and underlying methodology, we refer the reader to the literature.16,18,19 Arch Phys Med Rehabil Vol 89, March 2008
458
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
which they were mentioned. These treatments were soft-tissue surgery, functional electric stimulation (FES), ankle-foot orthosis (AFO), pharmaceutical treatment, and orthopedic shoes. A decision tree was also developed based on the group responses.
Fig 1. The AHP decision model. Impact of treatment comprises the impact to the patient in the active treatment phase, and daily effort comprises the daily actions needed to maintain the effect of treatment afterward (mainly donning and doffing necessary equipment). *Criteria supported by literature.26
Model Development The AHP decision model in this study consists of 4 levels (fig 1). In the first level, the goal of the analysis is presented. The second level consists of the main decision criteria. These criteria are split into subcriteria, which are included in the third level. Finally, the available alternatives are presented in the fourth level of the model. The goal of the analysis was set at selecting the preferred treatment for equinovarus deformity after stroke. This goal supports the specific aim of the study, which was to obtain information on the preferred treatment of a selection of health care professionals for equinovarus deformity in stroke. To determine level 2 and 3 criteria, a Delphi-style paper-andpencil interview was conducted among a sample of Dutch physiatrists (n⫽28; response rate, 68%) united in a stroke interest group. The questionnaire was developed to identify a preliminary set of treatment requirements, criteria, and treatment alternatives for acquired equinovarus deformity in a postacute phase (⬎6wk) of stroke. An example of the first questionnaire as it was sent to the group is presented in appendix 1. Simultaneously, a literature search was conducted using the search terms stroke, equinovarus deformity, ankle-foot impairment, and treatment.3-7 A summary of responses to the initial questionnaire was sent back to participants, along with the treatments reported in literature and 2 additional rounds of feedback from the group were processed. After 3 rounds of questionnaires, 5 feasible treatment categories for the postacute management of equinovarus foot were identified from the treatments selected by the panel based on the frequency with Arch Phys Med Rehabil Vol 89, March 2008
Judgment Stage A panel of 10 professionals in rehabilitation medicine was asked to participate in the judgment phase. The panel consisted of 4 physiatrists, 1 orthopedic surgeon, 1 physical therapist, 2 senior researchers in the field of stroke, and 2 certified orthotists. To start the discussion, a patient case description (fig 2) and video were presented to the panel. The patient had moderate equinovarus deformity and was chosen so that it could be treated with the maximal number of treatments identified during the Delphi process. Based on their individual expertise, the panel members supported the use of the following treatments for equinovarus deformity: split tibial tendon transfer combined with a transfer of the hallucis longus muscle (soft-tissue surgery), an off-the-shelf AFO, off-the-shelf semi-orthopedic shoes, an implanted peroneal stimulator, and a surface peroneal stimulator. Pharmaceutical treatment was omitted as a treatment alternative because of the absence of marked spasticity. The decision model was presented to the panel on a projection screen, which stood in the front of the room. The judgment stage consists of 2 phases. First, the treatment performance was judged on the level 3 subcriteria. These judgments were made by comparing the treatments in a pairwise fashion using a reciprocal numeric scale ranging from 1 to 9 (an example can be found in appendix 2). In the AHP, the numbers are associated with verbal statements ranging from equally preferred (1) to extremely more preferred (9). In the next phase of the judgment stage, the panel members were first asked to judge the importance of the level 3 subcriteria with regard to the accomplishment of the level 2 criteria and then to judge the contribution (importance) of level 2 criteria in the achievement of the goal. These judgments were also made in a pairwise fashion using the reciprocal numeric scale described earlier using verbal statements ranging from equally important (1) to extremely more important (9).
Mrs. L. is a 61-year-old female who had a right-side stroke with hemiplegia on the left approximately 1 year ago. She is an independent outside walker (FAC 5) but complains about feelings of insecurity during walking on an uneven surface and walking during the night, for instance, toilet visits. Mrs. L. has normal cognitive functioning and no impairment in hand function due to stroke. Additionally, she suffers from high blood pressure. During physical assessment of walking pattern the following deviations were noticed: No deviations were seen in the right leg. During stance phase of the left leg, initial contact of the foot is seen on the lateral border and heel of the foot. Increased first rocker. The knee is in flexion at initial contact with delayed extension to terminal stance. During swing phase there is decreased flexion of the knee, not hindering foot clearance. The heel of the foot is in varus with deviation of calcaneus and supination of forefoot. Hyperextension of the first toe is present during swing and stance phase of walking. Strength of the hip in flexion and extension is MRC 4. Abduction is 5. Knee flexion is 5 and knee extension 4+. Ankle plantarflexion (in pattern) is 4, dorsiflexion is 4+, inversion is 5, and eversion is 4. Mobility of the hips and knees is normal. Ankle dorsiflexion left is limited with 5° (5/0/20) with a straight leg and normal with bend leg (10/0/20). Normal plantarflexion, decreased mobility of the calcaneus on the left (25/0/0). No marked spasticity, no sensibility disorder, and no peripheral circulation problems were present.
Fig 2. Patient case description. Observations were based on extensive manual testing in lying and sitting positions and visual observation of the patient during walking by an experienced physiotherapist. Abbreviations: FAC, Functional Ambulation Category; MRC, Medical Research Council scale.
459
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til Table 1: Importance Weights for All Criteria Level 2
Average
SD
Level 3
Average
SD
Outcome
.509
.180
Risks
.194
.115
Comfort Daily effort
.104 .098
.086 .110
Cosmetics Impact
.065 .030
.044 .023
Active ankle stability Passive ankle stability Foot position in stance Foot position in swing Short-term risks Long-term side effects Reliability of treatment Consequences on comfortable shoe wear Daily time investment Complexity of daily investment Cosmetic consequences Duration of treatment Complexity of treatment
.441 .059 .102 .398 .386 .449 .165 .104 .770 .230 .065 .760 .240
.023 .023 .057 .057 .099 .089 .035 .086 .031 .031 .044 .055 .055
Abbreviation: SD, standard deviation.
Initially, individual judgments were made by the panel members. When all judgments were completed, disagreement among panel members was made explicit by presenting the numeric judgments on a screen visible to each panel member and by verbal explanation of the discussion leader. In case of disagreement, panel members with the most diverging scores were asked to clarify their judgments. In many cases, this resulted in a discussion of pros and cons of the treatments or the relevance of criteria. Panel members were allowed to reconsider and to alter their judgments if they believed it was necessary based on the discussion. To reduce the time requirements of the session, the number of performance and importance judgments was reduced to the minimal number required for calculating weights on an individual and group level. A total of 52 performance and 13 importance judgments were made by the 10 members of the panel. There was a group discussion after each judgment. The duration of the session was about 7 hours (including breaks). Synthesis of Results In the AHP, the numeric judgments are put in a comparison matrix (A). The reciprocal scores are used to complete the matrix. The principle eigenvector method is used to calculate the individual importance and performance weights from the matrix. For a detailed explanation of the hows and whys of the eigenvector method the reader is referred to the literature,16,20 but for the scope of the current article it is enough to know that a close approximation of the priority vector of matrix A can be calculated by dividing each element in the matrix by the sum of its column and then dividing the sum of each row by the sum of the matrix. Group values are calculated using the geometric mean of individual judgments. A priority score (V) for each treatment (a) is calculated based on an additive value function:
study, we calculated consistency for the entire group using the group mean comparisons as described earlier. Sensitivity Analysis To determine the impact of importance weights on overall priority for treatment, the importance of each level 2 criterion was varied from 0 to 1, in increments of 0.1, and the sum of importance weights was kept at 1.0. The priority for treatment was calculated for all combinations. RESULTS Criteria Weights The relative priorities of the level 2 and 3 criteria are presented in table 1. In the opinion of the expert panel, on the second level the outcome of treatment is the most important criterion in making treatment decisions. With an importance weight of .509 it influences half of the decision. Active ankle stability (.44) and foot position in swing phase of walking (.4) were judged to be the most important level 3 parameters when outcome is considered, whereas passive ankle stability and improvement of the foot position in stance were judged less important. Performance Weights The performance values for all treatments are presented in figure 3.
i⫽1
Priority of Treatments Soft-tissue surgery was the first ranked treatment in all participants, with an average priority of .413 and a range between .313 and .614. Semi-orthopedic shoes (.181; range, .081–.230) ranked second, followed by the AFO (.147; range, .098 –.170), surface peroneal stimulator (.137; range, .079 –.230), and finally implanted peroneal stimulator (.123; range, .099 –.169). Individual priority scores along with participants’ professions are presented in table 2.
Criteria weights are multiplied with performance values and summed, resulting in an overall priority (performance rating) for each alternative. Measurement of the consistency of judgments is a routine part of the AHP and is measured by calculating a consistency index, which ranges from 0 to 1. Analyses with consistency indices less than or equal to 0.1 are generally considered acceptable. In certain cases, a cutoff value of 0.2 is used. In this
Sensitivity Analysis Sensitivity analysis was performed to study the influence of criteria weights on treatment ranking. The priority for treatment is pictured in figure 4. The first ranked treatment was soft-tissue surgery in 77%, orthopedic shoes in 3%, and AFO in 20% of cases. No combination of importance weights results in either surface or implanted electric stimulation having the highest priority.
V(a) ⫽
m
兺 w v (a) i i
Arch Phys Med Rehabil Vol 89, March 2008
460
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
Fig 3. The average performance values with standard deviation of the treatments on all level 3 criteria. IFES, implanted peroneal stimulator; SFES, surface peroneal stimulator; SOS, semiorthopedic shoes; TS, soft-tissue surgery.
DISCUSSION Using the AHP methodology, soft-tissue surgery was the preferred treatment in this study. The panel judged soft-tissue surgery performance best on outcome, cosmetics, comfort, and daily effort. Orthopedic shoes and AFOs perform modestly on most criteria. Strong features include the low impact of treatment and the low risk that is associated with treatment in both orthopedic shoes and AFOs. The performance of FES was judged low. As was shown in the sensitivity analysis, FES was dominated by the other treatment alternatives for equinovarus deformity, because no combination of criteria weights resulted in a priority for FES. According to the expert panel, problems with surface FES include the complexity of electrode and heel switch placement in a patient group that can be both cognitively and/or physically impaired. Although an effort to reduce daily effort was made by the development of an implantable device, the surgical procedure negatively influences treatment impact and risks and side effects. In terms of outcome, the timing of the electric stimulus, which is the signal for foot elevation during the swing phase of walking, is thought to be unreliable. Naturally, as a result of ongoing development in rehabilitation medicine the conclusions of this experiment regarding preference for treatment are temporary, and updating the model with new and if possible objective information as it becomes available would be essential.21
Table 2: Indvidual Priority for Treatment Treatment Priority
TS
IFES
SFES
AFO
SOS
Orthopedic surgeon Orthotist Orthotist Physiatrist Physiatrist Physiatrist Physiatrist Physical therapist Senior researcher Senior researcher
.614 .313 .343 .337 .541 .550 .430 .475 .453 .463
.114 .169 .138 .099 .116 .104 .131 .132 .129 .116
.092 .195 .139 .230 .096 .079 .117 .103 .094 .149
.098 .164 .150 .170 .104 .106 .159 .104 .128 .113
.081 .159 .230 .165 .142 .160 .163 .186 .196 .159
Abbreviations: IFES, implanted peroneal stimulator; SFES, surface peroneal stimulator; SOS, semi-orthopedic shoes; TS, soft-tissue surgery.
Arch Phys Med Rehabil Vol 89, March 2008
Fig 4. Sensitivity analysis. A 6-dimensional space is presented by showing each combination of weights for 2 criteria in 16 squares. If the preference for treatment is dependent on the weight of other criteria, multiple treatment priorities are piled for a 2-dimensional weight combination.
The preference for soft-tissue surgery expressed in this study is not in accordance with clinical practice. In the Netherlands, orthopedic shoes and AFOs are used most often in the treatment of equinovarus deformity. Panel discussion suggested that this might be explained by the low impact of these treatments in patients who are faced with many cognitive and physical disabilities and a great familiarity with and easy access to orthotic treatment in a rehabilitation setting. According to the panel the preference for soft-tissue surgery in this study is partly explained by the time since injury (⬎1y) and the good functional status of the patient in the case description. As a direct consequence, treatment impact is judged to be least and outcome is considered most important. In clinical practice the natural recovery of equinovarus deformity and walking ability is often maximal within weeks, and a decision regarding treatment has to be made at a time in which the treatment of equinovarus deformity as such might not have the highest priority. However, as time progresses, different trade-offs between the pros and cons of treatment are made. Another explanation for the limited use of soft-tissue surgery might be the limited experience of neurologists and physiatrists, who traditionally treat stroke patients, with the surgical possibilities in an adult population with upper motoneuron disorders. Consequently stroke patients are only rarely referred to orthopedic surgeons, although the results of this study indicate there is a high potential for the use of soft-tissue surgery in the later stages of recovery. This advocates a more active approach in bringing this often-neglected or unknown surgical approach in equinovarus deformity in stroke to the attention of treating physicians. In our opinion, the AHP methodology that was used in this study provided the condition for a structured discussion regarding the conditions for treatment decisions for equinovarus deformity in stroke.17,22 The conscious deliberation of relevant requirements for treatments gave insight into the thought processes of panel members. The judgment of performance on a standard set of well-defined criteria enabled a comparison of
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
treatments that was not possible based on the available literature. Individual knowledge and experience was made explicit and was shared among panel members. The reaction of the panel to the use of the AHP as an alternative tool for treatment comparison was diverse. During the analysis, some of the panel members raised concerns about the validity of the judgments because their experience and knowledge on some of the treatments was limited. In our opinion, the group discussion phase should have resolved these problems, because panel members were allowed to alter their judgments if they believed unknown information was presented. Moreover, the inclusion of and discussion between panel members from different backgrounds is essential in the comparison of conventional and innovative or specialized treatment options, because any individual’s knowledge and experience are always restricted. The time requirements to complete the model were considered a disadvantage, and the AHP was thought to be bothersome for use in day-to-day decision making. This reservation might be a result of the large size of the current model. Simpler models might be possible, although care must be taken to maintain important details. The AHP analysis gave valuable insights in the motivation of panel members and is thereby very well suited for both examination of treatment choice and guideline development in rehabilitation medicine when high-quality scientific evidence is unavailable. Retrospectively, panel members expressed concern regarding the generalizability of the results of the study to other patients because the judgment phase may have been influenced by the patient description used. We shall discuss these concerns on 2 levels—namely, the effect on performance values of the treatment alternatives and the effect on importance weights. First, the influence of the case description on the performance judgments of the expert panel was negligible. As became obvious from the videotape used to document the experiment, the patient description was rarely referred to by the panel members in the discussion, and judgments on performance were made on a more general level. Moreover, because the influence of minor variations in impairment on treatment performance on treatment outcome is only scarcely documented in literature, it is questionable whether it would become obvious from subjective judgments made by a panel of health professionals. For a number of other criteria, the effect of the case description would be comparable for all treatments (eg, more severe deformity would lead to higher treatment impact), which in turn does not influence the comparison. In our opinion, therefore, the judgments on treatment performance on the lowest level are generalizable to other patients as well. Second,
461
as was mentioned before, the specifics of the ankle-foot deformity and the time since injury probably did influence the panel’s judgments about the relative importance of the criteria on a higher level. In this patient, the relevance of treatment impact was judged to be negligible and outcome was deemed most important as a result of the specific status and demands of the patient. Because the impairment was especially bothersome in the swing phase of walking, the outcome of treatments on criteria related to the swing phase of walking were given a high importance weight. Criteria and subcriteria importance should therefore be determined for each patient to determine the preference for treatment in other cases. To support these statements, it would be worthwhile to repeat the analysis with another patient and/or another panel. The high importance that is given to health-related criteria such as outcome and risks of treatment may be a reflection of the preferences of the health professionals represented in the panel. As was shown by Hummel et al,15 patients value different aspects of treatment compared with health professionals. Treatment preference could thereby differ for patients. As patient involvement is seen as an important factor in ensuring quality improvement in health care,23,24 health professionals are increasingly encouraged to involve their patients in treatment decision making. The AHP model used in this study could be a way to include the personal aims, wishes, and demands of each patient into health care decision making. It has been shown previously15,25 that patients are capable of using the AHP to express their preferences. CONCLUSIONS When clinical practice is compared with the preference for treatment expressed using the AHP methodology, soft-tissue surgery is an undervalued treatment alternative in the treatment of moderate equinovarus deformity in the swing phase of walking, if a patient is willing and able to undergo a longer and more tedious treatment. However, the final choice of treatment remains dependent on individual trade-offs of patients and health care professionals, and this study showed that different trade-offs regarding criteria importance could lead to a different preference for treatment. Although some concerns were raised by the panel, these seem manageable, and the use of the AHP as a methodology to compare treatment alternatives in rehabilitation medicine is promising. Because large, randomized, controlled trials are not yet available and will remain difficult to execute because of the heterogeneity of the patient population and lack of funding in rehabilitation medicine, an alternative for decision aiding is highly applicable.
APPENDIX 1: QUESTIONNAIRE STROKE INTEREST GROUP An open-ended questionnaire with the following questions was sent to the participants. 1. How do you characterize equinovarus deformity poststroke? 2. Which interventions are you familiar with for equinovarus deformity poststroke? 3. Which of these interventions do you use in daily practice? 4. Which general patient characteristics do you take into account when prescribing treatment for equinovarus deformity poststroke? 5. Which specifics of the equinovarus deformity do you take into account when prescribing treatment? 6. Which general treatment characteristics do you take into account when prescribing treatment for equinovarus deformity poststroke? 7. Which specific characteristics or effects of the treatment do you take into account when prescribing treatment for equinovarus deformity poststroke? All responses were categorized and returned to participants. Participants were asked to check the boxes with the treatments, characteristics, or criteria that they found relevant in the decision for treatment. Based on frequency a selection of relevant criteria was made for use in the decision tree. Also, a selection of alternative treatments was made. Arch Phys Med Rehabil Vol 89, March 2008
462
ANALYTIC HIERARCHY PROCESS DECISION SUPPORT, van Til
APPENDIX 2: PERFORMANCE AND IMPORTANCE JUDGMENT Performance Which of the next treatments is preferred when the aim is to minimize risks? 9 □
8 □
7 □
6 □
5 □
4 □
3 □
2 □
Soft-tissue surgery
1 □
2 □
3 □
4 □
5 □
or
6 □
7 □
8 □
9 □
Ankle-foot orthosis
Importance Which of the next criteria is more important in determining the best treatments for ankle-foot impairments? 9 □
8 □
7 □
6 □
5 □
4 □
3 □
2 □
outcome
1 □
or
2 □
3 □
4 □
5 □
6 □
7 □
8 □
9 □
risks
Numeric Values
Verbal Description
Interpretation of Judgment
1 3 5 7 9 2,4,6,8
Equal Moderate Strong Very strong/demonstrated Extreme importance/dominance Compromise between judgments
Equal contribution to objective Experience or judgment slightly favors one criteria over the other Experience or judgment strongly favors one criteria over the other Dominance of criteria is demonstrated in practice Effect is significantly demonstrated in literature
References 1. Geertzen JH, Hijmans JM, van der Linde H. Prosthetic prescription in The Netherlands: an interview with clinical experts. Prosthet Orthot Int 2005;29:113-4. 2. Report of a consensus conference on the orthotic management of stroke patients. Copenhagen: International Society of Prosthetics and Orthotics; 2004. 3. Piccioni L, Keenan M. Surgical correction of varus and equinovarus deformity in the spastic patient. Oper Tech Orthop 1992;2:146-50. 4. Lawrence SJ, Botte MJ. Management of the adult, spastic, equinovarus foot deformity. Foot Ankle Int 1994;15:340-6. 5. Kottink AI, Oostendorp LJ, Buurke JH, Nene AV, Hermens HJ, Ijzerman MJ. The orthotic effect of functional electrical stimulation on the improvement of walking in stroke patients with a dropped foot: a systematic review. Artif Organs 2004;28:577-86. 6. Pittock SJ, Moore AP, Hardiman O, et al. A double-blind randomised placebo-controlled evaluation of three doses of botulinum toxin type A (Dysport) in the treatment of spastic equinovarus deformity after stroke. Cerebrovasc Dis 2003;15:289-300. 7. Lin SS, Sabharwal S, Bibbo C. Orthotic and bracing principles in neuromuscular foot and ankle problems. Foot Ankle Clin 2000;5: 235-64. 8. Jaivin JS, Bishop JO, Braly WG, Tullos HS. Management of acquired adult dropfoot. Foot Ankle 1992;13:98-104. 9. Beckerman H, Becher J, Lankhorst GJ, Verbeek AL. Walking ability of stroke patients: efficacy of tibial nerve blocking and a polypropylene ankle-foot orthosis. Arch Phys Med Rehabil 1996; 77:1144-51. 10. Trevena L, Barratt A. Integrated decision making: definitions for a new discipline. Patient Educ Couns 2003;50:265-8. 11. Frith L. Priority setting and evidence based purchasing. Health Care Anal 1999;7:139-51. 12. Hariharan S, Dey PK, Chen DR, Moseley HS, Kumar AY. Application of analytic hierarchy process for measuring and comparing the global performance of intensive care units. J Crit Care 2005;20:117-24; discussion 124-5. 13. Cook DR, Staschak S, Green WT. Equitable allocation of livers for orthotopic transplantation: an application of the Analytic Hierarchy Process. Eur J Oper Res 1990;48:49-56.
Arch Phys Med Rehabil Vol 89, March 2008
14. Castro F, Caccamo LP, Carter KJ, et al. Sequential test selection in the analysis of abdominal pain. Med Decis Making 1996;16:178-83. 15. Hummel JM, Snoek GJ, van Til JA, van Rossum W, IJzerman MJ. A multicriteria decision analysis of augmentative treatment of upper limbs in persons with tetraplegia. J Rehabil Res Dev 2005;42:635-44. 16. Saaty TL. How to make a decision: the analytic hierarchy process. Eur J Oper Res 1990;48:9-26. 17. Dyer RF, Forman EH. Group decision support with the Analytic Hierarchy Process. Decis Support Syst 1992;8:99-124. 18. Hatcher M. Voting and priorities in health care decision making, portrayed through a group decision support system, using analytic hierarchy process. J Med Syst 1994;18:267-88. 19. Dolan JG, Isselhardt BJ Jr, Cappuccio JD. The analytic hierarchy process in medical decision making: a tutorial. Med Decis Making 1989;9:40-50. 20. Saaty TL. Decision-making with the AHP: why is the principal eigenvector necessary? Eur J Oper Res 2003;145:85-91. 21. Dolan JG, Bordley DR. Using the analytic hierarchy process (AHP) to develop and disseminate guidelines. QRB Qual Rev Bull 1992;18:440-7. 22. Hummel JM, van Rossum W, Verkerke GJ, Rakhorst G. Medical technology assessment: the use of the analytic hierarchy process as a tool for multidisciplinary evaluation of medical devices. Int J Artif Organs 2000;23:782-7. 23. Kennedy A, Robinson A, Rogers A. Incorporating patients’ views and experiences of life with IBS in the development of an evidence based self-help guidebook. Patient Educ Couns 2003;50:303-10. 24. Robinson A, Thomson R. Variability in patient preferences for participating in medical decision making: implication for the use of decision support tools. Qual Health Care 2001;10(Suppl 1): i34-8. 25. Dolan JG, Bordley DR. Involving patients in complex decisions about their care: an approach using the analytic hierarchy process. J Gen Intern Med 1993;8:204-9. 26. Blankevoort L, Geertzen JH, Heerkens YF, Hijmans JM, Ursum J. Ontwikkeling van richtlijnen voor de indicatiestelling en het verstrekkingsproces van orthopedische orthesen (ORIVOO). Amsterdam: Eigen beheer; 2005.