- Email: [email protected]
Issues Affecting the Selection of Participation Measurement in Outcomes Research and Clinical Trials
S54
SPECIAL COMMUNICATION
Issues Affecting the Selection of Participation Measurement in Outcomes Research and Clinical Trials Gale G. Whiteneck, PhD ABSTRACT. Whiteneck GG. Issues affecting the selection of participation measurement in outcomes research and clinical trials. Arch Phys Med Rehabil 2010;91(9 Suppl 1):S54-S59. The ever-growing number of participation measures without consensus on which is best makes it difficult to determine which measure to use in rehabilitation research and clinical trials. In an effort to address issues affecting the selection of a participation measure for a specific research purpose, this article (1) outlines the types and characteristics of participation measures, (2) enumerates various uses of participation measures in disability and rehabilitation research, (3) discusses appropriate matching of the type of participation measure with the research task, and (4) offers recommendations for future participation research. Participation instruments vary in terms of their degree of participation specificity, the conceptual model that underlies their development, whether they include multiple domains or take a more global approach, the extent to which they are objective versus subjective, whether they use general population norms, who is the respondent, the method of item and scale development, and their psychometric properties. Participation measures are used in individual and population assessments, observational research, and interventional research. Selection of a participation measure for use in a specific study requires an understanding of the characteristics of available tools and the nature of the research design, but most importantly, it requires matching the instrument to the specific research question or hypothesis. Instruments assessing participation are currently appropriate as secondary outcomes in trials evaluating interventions targeting activity limitation, and they will become appropriate as primary outcomes when interventions are tested that target participation directly. It will be easier to apply participation measures appropriately to their many research uses once substantial progress is made in obtaining better participation measurements and consensus is reached about the best tools. Key Words: Clinical trials as topic; Role; Outcome and process assessment (health care); Rehabilitation. © 2010 by the American Congress of Rehabilitation Medicine
From Department of Research, Craig Hospital, Englewood, CO. Presented to the American Congress of Rehabilitation Medicine, Toronto, ON, Canada, October 14 –18, 2008. Supported by an honorarium by the Rehabilitation Institute of Chicago using funds received from the National Institute on Disability and Rehabilitation Research (grant no. H133B040032); the Rehabilitation Institute of Chicago (grant no. H133B040032) and Craig Hospital from the National Institute on Disability and Rehabilitation Research, Office of Special Education Services, Department of Education (grant nos. H133N06005 and H133A07022). The opinions contained in this publication are those of the grantees and do not necessarily reflect those of the U.S. Department of Education. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated. Correspondence to Gale Whiteneck, PhD, Director of Research, 3425 S Clarkson St, Englewood, CO 80113, e-mail: [email protected]. Reprints are not available from the author. 0003-9993/10/9109S-00524$36.00/0 doi:10.1016/j.apmr.2009.08.154
Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
HE CONCEPT OF participation as involvement in life T situations at the societal level has been highlighted as critical to disability and rehabilitation research by the ICF. 1
Accompanying the recognition of its importance, interest in the measurement of participation has increased exponentially over the last 3 decades, with over 30 instruments purporting to measure participation now appearing in the literature, but without any agreement on the most appropriate method of measurement, let alone consensus on a widely applicable, psychometrically sound specific assessment tool. The use of participation measures in disability and rehabilitation research has been only sporadic, in part because of this absence of a high-quality, fully developed participation assessment system. However, it is not unreasonable to assume that coming advances in participation measurement will usher in a burgeoning variety of research focusing on participation. The purpose of this article is to provide an overview of conceptual and methodologic issues surrounding participation measurement and offer guidance to researchers wishing to incorporate participation measures in their work. While comprehensive evidence-based reviews of the concept of participation, its measurement, and the inclusion of participation instruments in research are beyond the scope of this brief article, illustrative examples of the development and use of participation measures will be used to meet the following aims: (1) outline the types and characteristics of participation measures, (2) enumerate various uses of participation measures in disability and rehabilitation research, (3) discuss appropriate matching of the type of participation measure with the research task, and (4) offer recommendations for future participation research. TYPES AND CHARACTERISTICS OF PARTICIPATION MEASUREMENT There were precursors to participation measurement long before the advent of formal instruments and even before the concept of participation was clarified in models of disability. Simple indicators of what we would now call participation
List of Abbreviations CAT CHART CIM CIQ CPI ICF IRT PM-PAC POPS RCT SCI TBI
computer adaptive testing Craig Handicap Assessment and Reporting Technique Community Integration Measure Community Integration Questionnaire Community Participation Indicators International Classification of Functioning, Disability and Health item response theory Participation Measure for Post Acute Care Participation Objective, Participation Subjective randomized controlled trial spinal cord injury traumatic brain injury
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
include employment status, probably the earliest outcome used in rehabilitation. Another indicator that was promulgated by the independent living movement was independent living status or noninstitutional residence. It has only been in the last 3 decades that we have actual measures of participation that meet the criteria of instruments with multi-item scales that produce a score. Within the realm of formal instruments, there are a wide variety of tools that differ in their degree of specificity in participation measurement. Instruments can focus exclusively on the measurement of participation like the CHART2 and the PM-PAC,3 or instruments can include a few participation items in broader measures. For example, health status measures like Quality Metric’s SF-36v2 Health Survey4 and the Nottingham Health Profile5 include a few items that could arguably be considered participation but then include far more items addressing activity limitations, perceived well-being, and other domains. Because of the widespread use of these health status measures, researchers have often claimed they measured participation even when they were not designed specifically as participation measures. Within instruments purporting to assess participation exclusively, a variety of content is included.6 This disagreement about what item content defines participation is largely a result of the lack of conceptual and practical distinction between activity and participation in the ICF.7,8 Proposals to divide the single list of categories including both activity and participation found in the ICF into 2 distinct taxonomies may reduce the confusion about the boundaries on the participation domain and improve its measurement.9 The conceptual models of participation measures differ as well. While most recently developed instruments like the PMPAC cite their connection to the ICF, earlier tools have been based on other conceptual models. The Disability Creation Process or the Quebec Model10 led to the Assessment of Life Habits;11 the International Classification of Impairments, Disability, and Handicap12 was the basis for CHART and the London Handicap Scale.13 Other instruments like the CIQ14 and the CIM15 are either based on other models or do not claim to be related to a particular conceptual model that involves participation. Measures of participation can either be global or include multiple domains. The CIM is an example of a global participation measure that produces just 1 global score. On the other hand, there are measures that include domain specific subscales like the PM-PAC and the CHART that produce domain-specific subscores as well as a total score. Within the tools that claim to assess domains of participation, the most common dimensions are typically labeled with terms such as productivity, social integration, or community involvement. Theses domains are often supported more by their conceptual content than by empirical evidence. This issue of dimensionality is complicated by inconsistent factor analysis and correlations among various domain scores, resulting in a dearth of clear evidence about whether there are justifiable domains of participation or whether a global concept is more appropriate. Participation measures also differ in terms of whether they are general or impairment-specific. For example, the CIQ and the Mayo Portland Participation Index16 were developed for TBI. The Assessment of Life Habits has primarily been a SCI tool. Other measures like the CPI,17 the PM-PAC, and the CHART were designed with multiple impairment groups in mind or they have been validated on multiple impairment group populations. Participation measures also differ on the extent to which they are objective versus subjective. CHART is the classic example of an instrument specifically designed to be an objective per-
S55
formance measure that asks for the hours a week the person is engaged in various forms of participation, the frequency of events, the number of people encountered, and other fairly objective questions that could, at least theoretically, be observed and verified by others. An early and frequent critique of objective measures was that people with disabilities might not want to participate in the same ways as people without disability. For example, if a person with disability either could not work or chose not to work, the participation score was lowered. In reaction to that, the subjective perception measures of participation seek to assess a person’s satisfaction with participation rather than actual performance. The POPS18 was the first participation measure to add a subjective measure to the more traditional objective one. After asking typical objective questions, the POPS reviews each question and asks respondents how important each of the participation areas is, followed by how satisfied the person is in each area. This subjective section of the POPS yields an importance weighted satisfaction score. The CPI takes a different approach with a subjective section labeled enfranchisement that asks for respondents’ perceptions about such things as how accepted and valued they feel in their community, whether they feel they have control over what they choose to do, and whether they make a difference in their community. The CIM is an example of an instrument that asks only subjective questions. The PM-PAC assesses objective content but focuses on the perceived difficulty a person has engaging in various forms of participation. Related to the objective versus subjective distinction, there is a difference among participation measures in terms of whether they employ comparison group norms or not. Most objective measures use the performance of nondisabled members of the general population for comparison groups. The scoring of CHART explicitly defines the highest score of 100 on each subscale as indicating the absence of participation restrictions because the person is performing at the level expected of the general population. On the other hand, measures of subjective perceptions about participation do not seek a comparison with others. In asking levels of satisfaction, subjective measures ask respondents implicitly to compare their own current participation status with their desired status in responding how satisfied they are with various aspects of participation or how strongly they agree with statements about their participation. One must also consider who the respondent is when completing participation measures. Most objective measures and all of the subjective measures expect the person themselves to be the primary respondent. In fact, authors of the subjective measures would argue that if you are not able to assess perceptions from the individual, there is no proxy substitution. In contrast, some of the more objective measures like the CIQ and CHART allow a proxy to be the respondent when necessary. One objective measure—the Mayo Portland Participation Index— was originally designed for the clinician to rate the individual’s participation, but that tool now has other forms for the subject and a proxy or family member to complete. Participation measures have used different methods of item and scale development. Early measures of participation like the CHART and CIQ have been criticized for the fact that the items were developed by researchers, potentially reflecting their values and biases, and the scales were developed using classical test theory rather than more modern psychometric approaches. In response to this criticism, other methods have been employed in more recently developed tools. The CIM used qualitative methods to identify participation content. Extensive focus groups consisting primarily of people with disability and their caregivers, as well as a few rehabilitation professionals, insurance payors, and policy makers, were used to develop the Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
S56
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
enfranchisement items of the CPI. Rasch analysis or other IRT methods have been suggested as superior to classical test theory and used in the development of the PM-PAC and the CPI. In contrast, some researchers have suggested that participation items are not hierarchically ordered—an assumption of IRT—and that clinometric methods should be considered where potentially disparate items are summed to measure a construct like participation.9,19 Whatever development methods have been employed, all participation measures strive for good psychometric properties, with varying degrees of success. The standard measurement criteria of reliability (internal consistency, test-retest, and subject-proxy agreement for objective measures), validity (content, construct, concurrent, discriminant, predictive validity), responsiveness or sensitivity to change, and practicality of administration are all appropriately applied when assessing the quality of a participation measure. To summarize, there are a wide variety of types and characteristics of participation measures that exist among available, and soon to be available, instruments. These include the degree of participation specificity in the measure, the conceptual model that underlies the development of the tool, whether it is attempting to be a multiple domain instrument or takes a more global approach, the extent to which it is objective versus subjective, whether it uses general population norms, who is the respondent, the method of item and scale development, and the psychometric characteristics of the instrument. All of these properties of participation measures must be carefully considered in selecting appropriate instruments for different research purposes. USE OF PARTICIPATION MEASURES IN RESEARCH The most basic use of participation measures is in the assessment of individuals and populations. They can be used to describe a particular individual in terms of global participation score or in terms of the individual’s profile across the dimensions of participation. From a clinical perspective, a radar graph (sometimes called a spider or web plot) may be a good way to present progress over time toward participation goals in various domains. Those plots or simple participation domain scores can be used to diagnose areas that need improvement from a clinician perspective or from a consumer perspective. Participation data can also be used as a rehabilitation planning tool to prescribe the interventions that seem appropriate for that individual. Change in participation scores can then be used as feedback to the rehabilitation process to see whether there has been the impact on participation that the program sought to achieve. Moving from individual-level to population-level assessments, groups of people or populations can be described in terms of their participation level or their participation profile. Research frequently compares the participation characteristics of people with and without disabilities or compares various impairment groups to determine the degree to which their participation levels tend to differ overall or in various profile domains. Establishing rehabilitation outcome expectations and conducting program evaluation have been priorities for both the SCI and TBI Model Systems of care. Analyzing the large SCI and TBI Model System National Databases,20,21 which include measures of participation, has identified the typical participation levels of people rehabilitated in those comprehensive centers and determined how participation might be expected to differ based on the severity of impairment and other demographic characteristics.22 These Model System standards can then be used by rehabilitation facilities as benchmarks for Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
evaluating their program results. Also, the Model Systems have used their longitudinal databases to assess population trends in participation over time. Moving to observational research as opposed to simply describing persons or populations, there has been considerable work assessing the relationships among participation and other constructs. The relationships between participation and severity of impairment, activity limitations, personal factors, environmental factors, and subjective well-being have all been examined. Going beyond individual relationships between variables, models of disability have been tested that combine these concepts to see whether the hypothesized relationships underlying those conceptual models are supported in empirical data.22,23 Understanding relationships among rehabilitation concepts is an important use of participation measures in research. It allows empirical models to be built that help explain and predict participation among people with different characteristics. For example, program evaluation might be enhanced with regression models that predicted an expected level of participation based on impairment severity and various demographic characteristics of the individual and then compared the achieved participation level with the expectation based on a more complex model than a simple population norm. The potential for using participation measures in interventional research far exceeds its actual implementation. While participation instruments might be used as outcomes in preexperimental or quasi-experimental designs to gain some rough insight into the effects of different treatments or circumstances on participation, the real potential of participation assessment is in clinical trials, specifically RCTs. Participation measures might be used as secondary outcomes in RCTs targeting impairments or activity limitations with interventions more distal to the concept of participation. While participation instruments have been used as secondary outcomes in RCTs, this author has not found an example of a true participation measure being used as the primary outcome. However, it is clear that in the future, we will use participation as the primary outcome measure in a clinical trial that is testing an intervention designed to improve participation directly. The rehabilitation literature indicates that there is substantial room for improvement in assessing participation as part of clinical trials. In a review of 491 stroke clinical trials published between 1968 and 2005, Salter et al24 identified 1447 outcomes using 489 measurement scales. Less than 6% of the assessments in those stroke intervention trials were of participation, with the primary tools that were categorized as participation being the Nottingham Health Profile, the Medical Outcomes Study 36-Item Short-Form Health Survey, the Sickness Impact Profile,25 the London Handicap Scale, and the EuroQol.26 The London Handicap Scale is the only instrument in that series attempting to target participation exclusively. The other instruments are more global and contain only a few participation items; furthermore, there is no indication in the review by Salter24 that appropriate subscales were employed to assess participation more directly. With a series of participation measure used only 6% of the time, and with only 1 of the less frequently instruments an exclusive measure of participation, clearly participation is not a major outcome that is being investigated in stroke rehabilitation research. Similarly, in a review of peripheral neurologic disorders, van Nes et al27 identified 122 uses of outcome measures in RCTs, but only 5 of these were measures of participation, with the Rotterdam Handicap Scale28 used in 3 of those 5 instances. In this review as well, participation measurement was seldom used (less than 5% of the time), with the predominant measure seldom used in
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
other types of research. But van Nes et al27 did enter a plea for standardization of outcome measures in each of the ICF outcome domains, including participation. In summary, there is a clear potential use of participation measures in individual and population assessments, observational research, and interventional research. However, in clinical trials, participation measures are rarely used as secondary outcomes; only a minority of times that outcomes are classified as participation measures are they exclusively participation measures; and no examples of their use as primary outcomes have been found. Instruments assessing participation are currently appropriate as secondary outcomes when evaluating interventions targeting activity limitation in RCTs, although they will become appropriate as primary outcomes when interventions are tested that target participation directly. MATCHING PARTICIPATION MEASURES TO RESEARCH TASKS An understanding of the many types and characteristics of participation measures, coupled with insight into the many ways participation measures can be used in research, should result in a researcher selecting a measure which is a good match with their research task. One of the most basic guidelines is to measure participation if that is the research aim. Researchers wishing truly to measure participation should select an instrument that is dedicated exclusively to that task, rather than using global measures of outcome or health status measures like the Nottingham Health Profile or the SF-36v2, which include only a few participation items. Furthermore, it is important to be clear about whether the research aim is to measure participation or activity limitation or both. Because the ICF confounds activity and participation by providing only 1 list of categories for both concepts, one cannot look to the ICF for guidance about which content should be included in participation measures and which content should be included in activity measurement. Instruments claiming to assess participation and claiming to be based on the ICF may include different content depending on the views of the developer regarding the boundaries of the participation domain. Therefore, instruments based on the ICF may or may not be superior to earlier measures based on earlier models of disability. If a research design calls for a measure of participation, select one with content reflecting your view of what is exclusively included in the participation domain, without items reflecting the content of activity limitations or other conceptual rehabilitation outcome domains. Likewise, it is critical to determine whether the research goal is to evaluate objective participation performance or subjective perceptions of participation or both. There are good reasons to measure both concepts in certain situations, but in other situations, only 1 may be needed. It is the research question or hypothesis that determines the selection; match the concept of participation needed in the research with the instrument best able to assess that concept. The instrument development approach influences its appropriateness for various research tasks. As researchers move toward CAT that is based on IRT analysis, it is critical to select measurement tools that have been developed using IRT techniques. While excellent IRT-based measures of activity limitations exist that can be employed in CAT assessments, it is less clear that IRT-based measures of participation are as high in quality. In research already committed to CAT administration of an activity measure, it is only natural also to consider CAT administration of a participation measure. However, a researcher needs to be assured that the IRT-based participation measure meets the assumption of hierarchic ordering of items, which is particularly difficult in objective measures of partic-
S57
ipation. Early evidence from CPI development indicates that the subjective items appear hierarchic, but the objective items do not, probably because personal preference or choice (rather than a hierarchy) is involved in selecting one type of objective participation over another. If the assumption of hierarchy is not met, then an alternative participation measures should be considered. Desire for comparability across studies influences instrument selection. In order for research to be comparable with the widest range of other studies, a participation measure that was developed with all impairment groups in mind and/or one that has been validated in a variety of impairment groups is needed. The only reason for choosing a measure that focuses on a single impairment group is to compare results with other studies that have used that tool in that impairment group. Limiting the use of participation measures to those that are more broad-based is the first step in building a consensus around a much smaller number of tools and the eventual agreement among researchers about which is the most appropriate instrument. In clinical trials, it is critical to determine the target of the intervention and the proximity of that target to participation. Trials of interventions targeting impairment may have an impact so distal to participation that it is difficult to imagine how the intervention would influence participation. In such research situations in which there is no theory to support an intervention having impact on participation, there is no need to include a participation measure. The current trend of adding a quality of life or subjective well-being measure to all trials should not be used as an excuse for always adding a participation measure to all outcome test batteries. However, if reducing activity limitations is the primary target of an intervention, then adding a participation measure as a secondary outcome in the trial may be a reasonable decision, particularly if there is an expectation that improved activities of daily living functioning may lead to improved participation. In such situations, it is not as likely that there will be a specific hypothesis about which domain of participation will be effected. Therefore, it may be appropriate to choose a shorter global tool rather than one that captures multiple dimensions of participation. Finally, if the target of an intervention is actually increasing participation, then a participation measure will be the appropriate primary outcome of the study. In those situations, it is likely that a multiple domain participation measure will be needed because the intervention is likely to have more impact on one domain of participation than another. Examples of clinical trials with interventions targeting participation illustrate how the purpose of the intervention determines which participation domain will require measurement. If the intervention is removing financial disincentives to work, the productivity domain of participation would be the intervention target, while less impact would be expected on subjective perceptions of participation and social and community domains. In such a study, it would be reasonable that the productivity domain would be the primary outcome and the other domains might be secondary outcomes. If the intervention is providing free accessible transportation, then the primary outcome might be community integration, getting out and about in the community. If the intervention is adding support groups, then the primary intervention might be social interaction. If the intervention is self advocacy training, then it might be subjective perceptions. If the intervention is peer mentoring you might argue that the primary domain to be influenced might be the domain in which the peer mentor excels. If an intervention targets a very specific aspect of participation like increasing leisure pursuits or making new friends, the primary outcome may be a specific measure of that 1 participation aspect with a Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
S58
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
more general participation measure as the secondary outcome. In trials with repeated measures in longitudinal studies, it is important to pick participation measures that have high testretest reliability and responsiveness or sensitivity to change. Finally, all other things being equal, select the participation measure that has the better psychometric properties. RECOMMENDATIONS FOR FUTURE PARTICIPATION MEASUREMENT RESEARCH The first priority for advancing participation measurement research is to clarify the distinction between activity and participation, both conceptually and practically. Disability and rehabilitation research cannot move forward efficiently until these issues are resolved. Improving the conceptual distinctions between activity and participation has been called for by many, and some have made concrete suggestions.7-9,29 For the advancement of participation measurement research, a practical solution is needed that identifies the item content considered activity and the item content considered participation. The recommended approach is to divide the current single list of activity and participation categories (d codes) that appears in the ICF into 2 mutually exclusive lists for activity (a codes) and participation (p codes). One specific proposal has been put forward.9 It is recommended that consensus on the division could be facilitated by examining the content of items in current participation tools to determine which ICF codes are routinely present and which are routinely absent in participation measurement. This would dramatically narrow the controversial codes (appearing in some but not most instruments) to a manageable number. Application of better conceptual distinctions to the remaining list might be quite helpful in reaching a practical solution. A second priority for advancing participation measurement is to determine the dimensionality of participation. Because of the variety of conceptual and empirical methods used to examine the dimensionality of participation and the inconsistent results produced, further research needs to be conducted. Analysis of datasets including multiple participation instruments may be helpful in this regard. Because objective participation and subjective participation are conceptually distinct, it would seem prudent to keep items in objective measures of participation separate from items in subjective measures in future efforts to examine the dimensionality of participation. Determining whether objective participation is most productively thought of as a single global dimension or as multiple distinct domains (or potentially approaches are appropriate in different situations or for different purposes) would advance measurement and hasten consensus on a common instrument. Establishing whether subjective participation is best conceptualized as a single or multiple dimensions would similarly advance the field. A third research priority is to understand better the relationship between objective and subjective participation. Research needs to be conducted in which both objective and subjective participation measures can be correlated with each other and with other ICF domains and subjective quality of life, particularly with measures of life satisfaction. It will be important to determine the strength of the relationship between objective and subjective participation. It will be important to determine whether measures of subjective perceptions of participation are more closely related to measures of objective participation performance or are more strongly related to measures of subjective quality of life or life satisfaction. Understanding these relationships will contribute to deciding whether subjective participation is another domain within subjective quality of life or another domain within the concept of participation. Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
The fourth recommended research priority is to focus on scoring of participation instruments. A first step in this process would be to examine the hierarchy among participation items to determine whether Rasch or other IRT analyses are appropriate. The results of these analyses might strongly influence participation instrument development. In order to bridge the gap between objective and subjective participation measurement, it may be helpful to devise a scoring method for objective participation items that reflects individual preferences. This might be accomplished by asking how important each of the objective items is personally. An alternative scoring could then be developed as an importance weighted objective participation score. That would be a performance score weighted on respondents’ preferences about what is important in their lives. Another research task that is needed is to evaluate participation response pattern differences across people with and without disabilities and among impairment groups. That would contribute to a better understanding of whether some participation item questions are answered differently by different groups, including whether the participation measures have a sex bias. The 4 research priorities outlined may ultimately facilitate a fifth and final research priority for improving participation measurement—moving toward the goal of a common measure of participation. By examining which objective and subjective participation items and response categories are common across most instruments, a common pool of items or 1 instrument with the most common items might gain consensus for general adoption as the tool to be used in most research. The ideal participation measure would have good psychometric properties, include both objective and subjective subscales, and include a brief version that is global and a longer version that is multidimensional. It is critical that we move toward a common tool that is applicable across all impairment groups and covers the full range of participation. Then such a common tool could be applied to the full range of outcomes research and clinical trials. CONCLUSIONS It will be easier to apply participation measures appropriately to their many research uses once substantial progress has been made in obtaining better participation measurement. References 1. World Health Organization. International classification of functioning, disability and health: ICF. Geneva: World Health Organization; 2001. 2. Whiteneck GG, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil 1992;73:519-26. 3. Gandek B, Sinclair SJ, Jette AM, Ware JE Jr. Development and initial psychometric evaluation of the participation measure for postacute care (PM-PAC). Am J Phys Med Rehabil 2007;86:57-71. 4. Ware JE, Kosinski M, Dewey JE. How to score version two of the SF-36 health survey. Lincoln: QualityMetric Inc; 2000. 5. Hunt SM, McEwen J. The development of a subjective health indicator. Sociol Health Illn 1980;2:231-46. 6. Perenboom RJ, Chorus AM. Measuring participation according to the International Classification of Functioning, Disability and Health (ICF). Disabil Rehabil 2003;25:577-87. 7. Institute of Medicine. The future of disability in America. Washington (DC): National Academies Pr; 2007. 8. Whiteneck GG. Conceptual models of disability: past, present and future. In: Field MJ, Jette AM, Martin L, editors. Workshop on disability in America: a new look—summary and background papers. Washington (DC): National Academies Pr; 2006. p 50-66.
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
9. Whiteneck GG, Dijkers MP. Difficult to measure constructs: conceptual and methodological issues concerning participation and environmental factors. Arch Phys Med Rehabil 2009;90(11 Suppl 1):S22-35. 10. Fougeyrollas P, Cloutier R, Bergeron H, Coté J, Coté M, St Michel G, editors. Revision of the Quebec Classification: handicap creation process. Quebec: International Network on the Disability Creation Process; 1997. 11. Fougeyrollas P, Noreau L, Bergeron H, Cloutier R, Dion SA, St-Michel G. Social consequences of long term impairments and disabilities: conceptual approach and assessment of handicap. Int J Rehabil Res 1998;21:127-41. 12. World Health Organization. International Classification of Impairments, Disabilities, and Handicaps: a manual of classification relating to the consequences of disease. Geneva: World Health Organization; 1980. 13. Harwood RH, Rogers A, Dickinson E, Ebrahim S. Measuring handicap: the London Handicap Scale, a new outcome measure for chronic disease. Qual Health Care 1994;3:11-6. 14. Willer B, Ottenbacher KJ, Coad ML. The community integration questionnaire: a comparative examination. Am J Phys Med Rehabil 1994;73:103-11. 15. McColl MA, Davies D, Carlson P, Johnston J, Minnes P. The community integration measure: development and preliminary validation. Arch Phys Med Rehabil 2001;82:429-34. 16. Malec JF. The Mayo-Portland Participation Index: a brief and psychometrically sound measure of brain injury outcome. Arch Phys Med Rehabil 2004;85:1989-96. 17. Heinemann AW, Whiteneck G, Brooks CA, et al. Measurement of community participation from the perspective of multiple stakeholders. Arch Phys Med Rehabil 2007;88:E25. Poster presented at the 84th Annual Meeting of the American Congress of Rehabilitation Medicine, Washington, DC, October 3-7, 2007. 18. Brown M, Dijkers MP, Gordon WA, Ashman T, Charatz H, Cheng Z. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004;19:459-81. 19. Dijkers MP. Issues in the conceptualization and measurement of participation: an overview. Arch Phys Med Rehabil. In press.
S59
20. National Spinal Cord Injury Statistical Center. The Spinal Cord Injury Model Systems’ data collection syllabus for the National Spinal Cord Injury Database 2006-2011 project period. 2006. Available at: http://images.main.uab.edu/spinalcord/pdffiles/ Syllabus%202006-2011%20Revised%205-09AppdB-E.pdf. Accessed June 14, 2009. 21. Traumatic Brain Injury National Data and Statistical Center. Traumatic Brain Injury Model Systems national database syllabus. 2009. Available at: http://www.tbindsc.org/Syllabus.aspx. Accessed June 14, 2009. 22. Whiteneck G, Meade MA, Dijkers M, Tate DG, Bushnik T, Forchheimer MB. Environmental factors and their role in participation and life satisfaction after spinal cord injury. Arch Phys Med Rehabil 2004;85:1793-803. 23. Whiteneck GG, Gerhart KA, Cusick CP. Identifying environmental factors that influence the outcomes of people with traumatic brain injury. J Head Trauma Rehabil 2004;19:191-204. 24. Salter KL, Foley NC, Jutai JW, Teasell RW. Assessment of participation outcomes in randomized controlled trials of stroke rehabilitation interventions. Int J Rehabil Res 2007;30:339-42. 25. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health status measure. Med Care 1981;19:787-805. 26. The EuroQoL Group. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy 1990;16:199208. 27. van Nes SI, Faber CG, Merkies IS. Outcome measures in immunemediated neuropathies: the need to standardize their use and to understand the clinimetric essentials. J Peripher Nerv Syst 2008; 13:136-47. 28. Merkies IS, Schmitz PI, Van Der Meche FG, Samijn JP, Van Doorn PA. Psychometric evaluation of a new handicap scale in immune-mediated polyneuropathies. Muscle Nerve 2002;25: 370-7. 29. Badley EM. Enhancing the conceptual clarity of the activity and participation components of the International Classification of Functioning, Disability, and Health. Soc Sci Med 2008;66: 2335-45.
Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
SPECIAL COMMUNICATION
Issues Affecting the Selection of Participation Measurement in Outcomes Research and Clinical Trials Gale G. Whiteneck, PhD ABSTRACT. Whiteneck GG. Issues affecting the selection of participation measurement in outcomes research and clinical trials. Arch Phys Med Rehabil 2010;91(9 Suppl 1):S54-S59. The ever-growing number of participation measures without consensus on which is best makes it difficult to determine which measure to use in rehabilitation research and clinical trials. In an effort to address issues affecting the selection of a participation measure for a specific research purpose, this article (1) outlines the types and characteristics of participation measures, (2) enumerates various uses of participation measures in disability and rehabilitation research, (3) discusses appropriate matching of the type of participation measure with the research task, and (4) offers recommendations for future participation research. Participation instruments vary in terms of their degree of participation specificity, the conceptual model that underlies their development, whether they include multiple domains or take a more global approach, the extent to which they are objective versus subjective, whether they use general population norms, who is the respondent, the method of item and scale development, and their psychometric properties. Participation measures are used in individual and population assessments, observational research, and interventional research. Selection of a participation measure for use in a specific study requires an understanding of the characteristics of available tools and the nature of the research design, but most importantly, it requires matching the instrument to the specific research question or hypothesis. Instruments assessing participation are currently appropriate as secondary outcomes in trials evaluating interventions targeting activity limitation, and they will become appropriate as primary outcomes when interventions are tested that target participation directly. It will be easier to apply participation measures appropriately to their many research uses once substantial progress is made in obtaining better participation measurements and consensus is reached about the best tools. Key Words: Clinical trials as topic; Role; Outcome and process assessment (health care); Rehabilitation. © 2010 by the American Congress of Rehabilitation Medicine
From Department of Research, Craig Hospital, Englewood, CO. Presented to the American Congress of Rehabilitation Medicine, Toronto, ON, Canada, October 14 –18, 2008. Supported by an honorarium by the Rehabilitation Institute of Chicago using funds received from the National Institute on Disability and Rehabilitation Research (grant no. H133B040032); the Rehabilitation Institute of Chicago (grant no. H133B040032) and Craig Hospital from the National Institute on Disability and Rehabilitation Research, Office of Special Education Services, Department of Education (grant nos. H133N06005 and H133A07022). The opinions contained in this publication are those of the grantees and do not necessarily reflect those of the U.S. Department of Education. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated. Correspondence to Gale Whiteneck, PhD, Director of Research, 3425 S Clarkson St, Englewood, CO 80113, e-mail: [email protected]. Reprints are not available from the author. 0003-9993/10/9109S-00524$36.00/0 doi:10.1016/j.apmr.2009.08.154
Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
HE CONCEPT OF participation as involvement in life T situations at the societal level has been highlighted as critical to disability and rehabilitation research by the ICF. 1
Accompanying the recognition of its importance, interest in the measurement of participation has increased exponentially over the last 3 decades, with over 30 instruments purporting to measure participation now appearing in the literature, but without any agreement on the most appropriate method of measurement, let alone consensus on a widely applicable, psychometrically sound specific assessment tool. The use of participation measures in disability and rehabilitation research has been only sporadic, in part because of this absence of a high-quality, fully developed participation assessment system. However, it is not unreasonable to assume that coming advances in participation measurement will usher in a burgeoning variety of research focusing on participation. The purpose of this article is to provide an overview of conceptual and methodologic issues surrounding participation measurement and offer guidance to researchers wishing to incorporate participation measures in their work. While comprehensive evidence-based reviews of the concept of participation, its measurement, and the inclusion of participation instruments in research are beyond the scope of this brief article, illustrative examples of the development and use of participation measures will be used to meet the following aims: (1) outline the types and characteristics of participation measures, (2) enumerate various uses of participation measures in disability and rehabilitation research, (3) discuss appropriate matching of the type of participation measure with the research task, and (4) offer recommendations for future participation research. TYPES AND CHARACTERISTICS OF PARTICIPATION MEASUREMENT There were precursors to participation measurement long before the advent of formal instruments and even before the concept of participation was clarified in models of disability. Simple indicators of what we would now call participation
List of Abbreviations CAT CHART CIM CIQ CPI ICF IRT PM-PAC POPS RCT SCI TBI
computer adaptive testing Craig Handicap Assessment and Reporting Technique Community Integration Measure Community Integration Questionnaire Community Participation Indicators International Classification of Functioning, Disability and Health item response theory Participation Measure for Post Acute Care Participation Objective, Participation Subjective randomized controlled trial spinal cord injury traumatic brain injury
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
include employment status, probably the earliest outcome used in rehabilitation. Another indicator that was promulgated by the independent living movement was independent living status or noninstitutional residence. It has only been in the last 3 decades that we have actual measures of participation that meet the criteria of instruments with multi-item scales that produce a score. Within the realm of formal instruments, there are a wide variety of tools that differ in their degree of specificity in participation measurement. Instruments can focus exclusively on the measurement of participation like the CHART2 and the PM-PAC,3 or instruments can include a few participation items in broader measures. For example, health status measures like Quality Metric’s SF-36v2 Health Survey4 and the Nottingham Health Profile5 include a few items that could arguably be considered participation but then include far more items addressing activity limitations, perceived well-being, and other domains. Because of the widespread use of these health status measures, researchers have often claimed they measured participation even when they were not designed specifically as participation measures. Within instruments purporting to assess participation exclusively, a variety of content is included.6 This disagreement about what item content defines participation is largely a result of the lack of conceptual and practical distinction between activity and participation in the ICF.7,8 Proposals to divide the single list of categories including both activity and participation found in the ICF into 2 distinct taxonomies may reduce the confusion about the boundaries on the participation domain and improve its measurement.9 The conceptual models of participation measures differ as well. While most recently developed instruments like the PMPAC cite their connection to the ICF, earlier tools have been based on other conceptual models. The Disability Creation Process or the Quebec Model10 led to the Assessment of Life Habits;11 the International Classification of Impairments, Disability, and Handicap12 was the basis for CHART and the London Handicap Scale.13 Other instruments like the CIQ14 and the CIM15 are either based on other models or do not claim to be related to a particular conceptual model that involves participation. Measures of participation can either be global or include multiple domains. The CIM is an example of a global participation measure that produces just 1 global score. On the other hand, there are measures that include domain specific subscales like the PM-PAC and the CHART that produce domain-specific subscores as well as a total score. Within the tools that claim to assess domains of participation, the most common dimensions are typically labeled with terms such as productivity, social integration, or community involvement. Theses domains are often supported more by their conceptual content than by empirical evidence. This issue of dimensionality is complicated by inconsistent factor analysis and correlations among various domain scores, resulting in a dearth of clear evidence about whether there are justifiable domains of participation or whether a global concept is more appropriate. Participation measures also differ in terms of whether they are general or impairment-specific. For example, the CIQ and the Mayo Portland Participation Index16 were developed for TBI. The Assessment of Life Habits has primarily been a SCI tool. Other measures like the CPI,17 the PM-PAC, and the CHART were designed with multiple impairment groups in mind or they have been validated on multiple impairment group populations. Participation measures also differ on the extent to which they are objective versus subjective. CHART is the classic example of an instrument specifically designed to be an objective per-
S55
formance measure that asks for the hours a week the person is engaged in various forms of participation, the frequency of events, the number of people encountered, and other fairly objective questions that could, at least theoretically, be observed and verified by others. An early and frequent critique of objective measures was that people with disabilities might not want to participate in the same ways as people without disability. For example, if a person with disability either could not work or chose not to work, the participation score was lowered. In reaction to that, the subjective perception measures of participation seek to assess a person’s satisfaction with participation rather than actual performance. The POPS18 was the first participation measure to add a subjective measure to the more traditional objective one. After asking typical objective questions, the POPS reviews each question and asks respondents how important each of the participation areas is, followed by how satisfied the person is in each area. This subjective section of the POPS yields an importance weighted satisfaction score. The CPI takes a different approach with a subjective section labeled enfranchisement that asks for respondents’ perceptions about such things as how accepted and valued they feel in their community, whether they feel they have control over what they choose to do, and whether they make a difference in their community. The CIM is an example of an instrument that asks only subjective questions. The PM-PAC assesses objective content but focuses on the perceived difficulty a person has engaging in various forms of participation. Related to the objective versus subjective distinction, there is a difference among participation measures in terms of whether they employ comparison group norms or not. Most objective measures use the performance of nondisabled members of the general population for comparison groups. The scoring of CHART explicitly defines the highest score of 100 on each subscale as indicating the absence of participation restrictions because the person is performing at the level expected of the general population. On the other hand, measures of subjective perceptions about participation do not seek a comparison with others. In asking levels of satisfaction, subjective measures ask respondents implicitly to compare their own current participation status with their desired status in responding how satisfied they are with various aspects of participation or how strongly they agree with statements about their participation. One must also consider who the respondent is when completing participation measures. Most objective measures and all of the subjective measures expect the person themselves to be the primary respondent. In fact, authors of the subjective measures would argue that if you are not able to assess perceptions from the individual, there is no proxy substitution. In contrast, some of the more objective measures like the CIQ and CHART allow a proxy to be the respondent when necessary. One objective measure—the Mayo Portland Participation Index— was originally designed for the clinician to rate the individual’s participation, but that tool now has other forms for the subject and a proxy or family member to complete. Participation measures have used different methods of item and scale development. Early measures of participation like the CHART and CIQ have been criticized for the fact that the items were developed by researchers, potentially reflecting their values and biases, and the scales were developed using classical test theory rather than more modern psychometric approaches. In response to this criticism, other methods have been employed in more recently developed tools. The CIM used qualitative methods to identify participation content. Extensive focus groups consisting primarily of people with disability and their caregivers, as well as a few rehabilitation professionals, insurance payors, and policy makers, were used to develop the Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
S56
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
enfranchisement items of the CPI. Rasch analysis or other IRT methods have been suggested as superior to classical test theory and used in the development of the PM-PAC and the CPI. In contrast, some researchers have suggested that participation items are not hierarchically ordered—an assumption of IRT—and that clinometric methods should be considered where potentially disparate items are summed to measure a construct like participation.9,19 Whatever development methods have been employed, all participation measures strive for good psychometric properties, with varying degrees of success. The standard measurement criteria of reliability (internal consistency, test-retest, and subject-proxy agreement for objective measures), validity (content, construct, concurrent, discriminant, predictive validity), responsiveness or sensitivity to change, and practicality of administration are all appropriately applied when assessing the quality of a participation measure. To summarize, there are a wide variety of types and characteristics of participation measures that exist among available, and soon to be available, instruments. These include the degree of participation specificity in the measure, the conceptual model that underlies the development of the tool, whether it is attempting to be a multiple domain instrument or takes a more global approach, the extent to which it is objective versus subjective, whether it uses general population norms, who is the respondent, the method of item and scale development, and the psychometric characteristics of the instrument. All of these properties of participation measures must be carefully considered in selecting appropriate instruments for different research purposes. USE OF PARTICIPATION MEASURES IN RESEARCH The most basic use of participation measures is in the assessment of individuals and populations. They can be used to describe a particular individual in terms of global participation score or in terms of the individual’s profile across the dimensions of participation. From a clinical perspective, a radar graph (sometimes called a spider or web plot) may be a good way to present progress over time toward participation goals in various domains. Those plots or simple participation domain scores can be used to diagnose areas that need improvement from a clinician perspective or from a consumer perspective. Participation data can also be used as a rehabilitation planning tool to prescribe the interventions that seem appropriate for that individual. Change in participation scores can then be used as feedback to the rehabilitation process to see whether there has been the impact on participation that the program sought to achieve. Moving from individual-level to population-level assessments, groups of people or populations can be described in terms of their participation level or their participation profile. Research frequently compares the participation characteristics of people with and without disabilities or compares various impairment groups to determine the degree to which their participation levels tend to differ overall or in various profile domains. Establishing rehabilitation outcome expectations and conducting program evaluation have been priorities for both the SCI and TBI Model Systems of care. Analyzing the large SCI and TBI Model System National Databases,20,21 which include measures of participation, has identified the typical participation levels of people rehabilitated in those comprehensive centers and determined how participation might be expected to differ based on the severity of impairment and other demographic characteristics.22 These Model System standards can then be used by rehabilitation facilities as benchmarks for Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
evaluating their program results. Also, the Model Systems have used their longitudinal databases to assess population trends in participation over time. Moving to observational research as opposed to simply describing persons or populations, there has been considerable work assessing the relationships among participation and other constructs. The relationships between participation and severity of impairment, activity limitations, personal factors, environmental factors, and subjective well-being have all been examined. Going beyond individual relationships between variables, models of disability have been tested that combine these concepts to see whether the hypothesized relationships underlying those conceptual models are supported in empirical data.22,23 Understanding relationships among rehabilitation concepts is an important use of participation measures in research. It allows empirical models to be built that help explain and predict participation among people with different characteristics. For example, program evaluation might be enhanced with regression models that predicted an expected level of participation based on impairment severity and various demographic characteristics of the individual and then compared the achieved participation level with the expectation based on a more complex model than a simple population norm. The potential for using participation measures in interventional research far exceeds its actual implementation. While participation instruments might be used as outcomes in preexperimental or quasi-experimental designs to gain some rough insight into the effects of different treatments or circumstances on participation, the real potential of participation assessment is in clinical trials, specifically RCTs. Participation measures might be used as secondary outcomes in RCTs targeting impairments or activity limitations with interventions more distal to the concept of participation. While participation instruments have been used as secondary outcomes in RCTs, this author has not found an example of a true participation measure being used as the primary outcome. However, it is clear that in the future, we will use participation as the primary outcome measure in a clinical trial that is testing an intervention designed to improve participation directly. The rehabilitation literature indicates that there is substantial room for improvement in assessing participation as part of clinical trials. In a review of 491 stroke clinical trials published between 1968 and 2005, Salter et al24 identified 1447 outcomes using 489 measurement scales. Less than 6% of the assessments in those stroke intervention trials were of participation, with the primary tools that were categorized as participation being the Nottingham Health Profile, the Medical Outcomes Study 36-Item Short-Form Health Survey, the Sickness Impact Profile,25 the London Handicap Scale, and the EuroQol.26 The London Handicap Scale is the only instrument in that series attempting to target participation exclusively. The other instruments are more global and contain only a few participation items; furthermore, there is no indication in the review by Salter24 that appropriate subscales were employed to assess participation more directly. With a series of participation measure used only 6% of the time, and with only 1 of the less frequently instruments an exclusive measure of participation, clearly participation is not a major outcome that is being investigated in stroke rehabilitation research. Similarly, in a review of peripheral neurologic disorders, van Nes et al27 identified 122 uses of outcome measures in RCTs, but only 5 of these were measures of participation, with the Rotterdam Handicap Scale28 used in 3 of those 5 instances. In this review as well, participation measurement was seldom used (less than 5% of the time), with the predominant measure seldom used in
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
other types of research. But van Nes et al27 did enter a plea for standardization of outcome measures in each of the ICF outcome domains, including participation. In summary, there is a clear potential use of participation measures in individual and population assessments, observational research, and interventional research. However, in clinical trials, participation measures are rarely used as secondary outcomes; only a minority of times that outcomes are classified as participation measures are they exclusively participation measures; and no examples of their use as primary outcomes have been found. Instruments assessing participation are currently appropriate as secondary outcomes when evaluating interventions targeting activity limitation in RCTs, although they will become appropriate as primary outcomes when interventions are tested that target participation directly. MATCHING PARTICIPATION MEASURES TO RESEARCH TASKS An understanding of the many types and characteristics of participation measures, coupled with insight into the many ways participation measures can be used in research, should result in a researcher selecting a measure which is a good match with their research task. One of the most basic guidelines is to measure participation if that is the research aim. Researchers wishing truly to measure participation should select an instrument that is dedicated exclusively to that task, rather than using global measures of outcome or health status measures like the Nottingham Health Profile or the SF-36v2, which include only a few participation items. Furthermore, it is important to be clear about whether the research aim is to measure participation or activity limitation or both. Because the ICF confounds activity and participation by providing only 1 list of categories for both concepts, one cannot look to the ICF for guidance about which content should be included in participation measures and which content should be included in activity measurement. Instruments claiming to assess participation and claiming to be based on the ICF may include different content depending on the views of the developer regarding the boundaries of the participation domain. Therefore, instruments based on the ICF may or may not be superior to earlier measures based on earlier models of disability. If a research design calls for a measure of participation, select one with content reflecting your view of what is exclusively included in the participation domain, without items reflecting the content of activity limitations or other conceptual rehabilitation outcome domains. Likewise, it is critical to determine whether the research goal is to evaluate objective participation performance or subjective perceptions of participation or both. There are good reasons to measure both concepts in certain situations, but in other situations, only 1 may be needed. It is the research question or hypothesis that determines the selection; match the concept of participation needed in the research with the instrument best able to assess that concept. The instrument development approach influences its appropriateness for various research tasks. As researchers move toward CAT that is based on IRT analysis, it is critical to select measurement tools that have been developed using IRT techniques. While excellent IRT-based measures of activity limitations exist that can be employed in CAT assessments, it is less clear that IRT-based measures of participation are as high in quality. In research already committed to CAT administration of an activity measure, it is only natural also to consider CAT administration of a participation measure. However, a researcher needs to be assured that the IRT-based participation measure meets the assumption of hierarchic ordering of items, which is particularly difficult in objective measures of partic-
S57
ipation. Early evidence from CPI development indicates that the subjective items appear hierarchic, but the objective items do not, probably because personal preference or choice (rather than a hierarchy) is involved in selecting one type of objective participation over another. If the assumption of hierarchy is not met, then an alternative participation measures should be considered. Desire for comparability across studies influences instrument selection. In order for research to be comparable with the widest range of other studies, a participation measure that was developed with all impairment groups in mind and/or one that has been validated in a variety of impairment groups is needed. The only reason for choosing a measure that focuses on a single impairment group is to compare results with other studies that have used that tool in that impairment group. Limiting the use of participation measures to those that are more broad-based is the first step in building a consensus around a much smaller number of tools and the eventual agreement among researchers about which is the most appropriate instrument. In clinical trials, it is critical to determine the target of the intervention and the proximity of that target to participation. Trials of interventions targeting impairment may have an impact so distal to participation that it is difficult to imagine how the intervention would influence participation. In such research situations in which there is no theory to support an intervention having impact on participation, there is no need to include a participation measure. The current trend of adding a quality of life or subjective well-being measure to all trials should not be used as an excuse for always adding a participation measure to all outcome test batteries. However, if reducing activity limitations is the primary target of an intervention, then adding a participation measure as a secondary outcome in the trial may be a reasonable decision, particularly if there is an expectation that improved activities of daily living functioning may lead to improved participation. In such situations, it is not as likely that there will be a specific hypothesis about which domain of participation will be effected. Therefore, it may be appropriate to choose a shorter global tool rather than one that captures multiple dimensions of participation. Finally, if the target of an intervention is actually increasing participation, then a participation measure will be the appropriate primary outcome of the study. In those situations, it is likely that a multiple domain participation measure will be needed because the intervention is likely to have more impact on one domain of participation than another. Examples of clinical trials with interventions targeting participation illustrate how the purpose of the intervention determines which participation domain will require measurement. If the intervention is removing financial disincentives to work, the productivity domain of participation would be the intervention target, while less impact would be expected on subjective perceptions of participation and social and community domains. In such a study, it would be reasonable that the productivity domain would be the primary outcome and the other domains might be secondary outcomes. If the intervention is providing free accessible transportation, then the primary outcome might be community integration, getting out and about in the community. If the intervention is adding support groups, then the primary intervention might be social interaction. If the intervention is self advocacy training, then it might be subjective perceptions. If the intervention is peer mentoring you might argue that the primary domain to be influenced might be the domain in which the peer mentor excels. If an intervention targets a very specific aspect of participation like increasing leisure pursuits or making new friends, the primary outcome may be a specific measure of that 1 participation aspect with a Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
S58
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
more general participation measure as the secondary outcome. In trials with repeated measures in longitudinal studies, it is important to pick participation measures that have high testretest reliability and responsiveness or sensitivity to change. Finally, all other things being equal, select the participation measure that has the better psychometric properties. RECOMMENDATIONS FOR FUTURE PARTICIPATION MEASUREMENT RESEARCH The first priority for advancing participation measurement research is to clarify the distinction between activity and participation, both conceptually and practically. Disability and rehabilitation research cannot move forward efficiently until these issues are resolved. Improving the conceptual distinctions between activity and participation has been called for by many, and some have made concrete suggestions.7-9,29 For the advancement of participation measurement research, a practical solution is needed that identifies the item content considered activity and the item content considered participation. The recommended approach is to divide the current single list of activity and participation categories (d codes) that appears in the ICF into 2 mutually exclusive lists for activity (a codes) and participation (p codes). One specific proposal has been put forward.9 It is recommended that consensus on the division could be facilitated by examining the content of items in current participation tools to determine which ICF codes are routinely present and which are routinely absent in participation measurement. This would dramatically narrow the controversial codes (appearing in some but not most instruments) to a manageable number. Application of better conceptual distinctions to the remaining list might be quite helpful in reaching a practical solution. A second priority for advancing participation measurement is to determine the dimensionality of participation. Because of the variety of conceptual and empirical methods used to examine the dimensionality of participation and the inconsistent results produced, further research needs to be conducted. Analysis of datasets including multiple participation instruments may be helpful in this regard. Because objective participation and subjective participation are conceptually distinct, it would seem prudent to keep items in objective measures of participation separate from items in subjective measures in future efforts to examine the dimensionality of participation. Determining whether objective participation is most productively thought of as a single global dimension or as multiple distinct domains (or potentially approaches are appropriate in different situations or for different purposes) would advance measurement and hasten consensus on a common instrument. Establishing whether subjective participation is best conceptualized as a single or multiple dimensions would similarly advance the field. A third research priority is to understand better the relationship between objective and subjective participation. Research needs to be conducted in which both objective and subjective participation measures can be correlated with each other and with other ICF domains and subjective quality of life, particularly with measures of life satisfaction. It will be important to determine the strength of the relationship between objective and subjective participation. It will be important to determine whether measures of subjective perceptions of participation are more closely related to measures of objective participation performance or are more strongly related to measures of subjective quality of life or life satisfaction. Understanding these relationships will contribute to deciding whether subjective participation is another domain within subjective quality of life or another domain within the concept of participation. Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010
The fourth recommended research priority is to focus on scoring of participation instruments. A first step in this process would be to examine the hierarchy among participation items to determine whether Rasch or other IRT analyses are appropriate. The results of these analyses might strongly influence participation instrument development. In order to bridge the gap between objective and subjective participation measurement, it may be helpful to devise a scoring method for objective participation items that reflects individual preferences. This might be accomplished by asking how important each of the objective items is personally. An alternative scoring could then be developed as an importance weighted objective participation score. That would be a performance score weighted on respondents’ preferences about what is important in their lives. Another research task that is needed is to evaluate participation response pattern differences across people with and without disabilities and among impairment groups. That would contribute to a better understanding of whether some participation item questions are answered differently by different groups, including whether the participation measures have a sex bias. The 4 research priorities outlined may ultimately facilitate a fifth and final research priority for improving participation measurement—moving toward the goal of a common measure of participation. By examining which objective and subjective participation items and response categories are common across most instruments, a common pool of items or 1 instrument with the most common items might gain consensus for general adoption as the tool to be used in most research. The ideal participation measure would have good psychometric properties, include both objective and subjective subscales, and include a brief version that is global and a longer version that is multidimensional. It is critical that we move toward a common tool that is applicable across all impairment groups and covers the full range of participation. Then such a common tool could be applied to the full range of outcomes research and clinical trials. CONCLUSIONS It will be easier to apply participation measures appropriately to their many research uses once substantial progress has been made in obtaining better participation measurement. References 1. World Health Organization. International classification of functioning, disability and health: ICF. Geneva: World Health Organization; 2001. 2. Whiteneck GG, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil 1992;73:519-26. 3. Gandek B, Sinclair SJ, Jette AM, Ware JE Jr. Development and initial psychometric evaluation of the participation measure for postacute care (PM-PAC). Am J Phys Med Rehabil 2007;86:57-71. 4. Ware JE, Kosinski M, Dewey JE. How to score version two of the SF-36 health survey. Lincoln: QualityMetric Inc; 2000. 5. Hunt SM, McEwen J. The development of a subjective health indicator. Sociol Health Illn 1980;2:231-46. 6. Perenboom RJ, Chorus AM. Measuring participation according to the International Classification of Functioning, Disability and Health (ICF). Disabil Rehabil 2003;25:577-87. 7. Institute of Medicine. The future of disability in America. Washington (DC): National Academies Pr; 2007. 8. Whiteneck GG. Conceptual models of disability: past, present and future. In: Field MJ, Jette AM, Martin L, editors. Workshop on disability in America: a new look—summary and background papers. Washington (DC): National Academies Pr; 2006. p 50-66.
PARTICIPATION IN RESEARCH AND TRIALS, Whiteneck
9. Whiteneck GG, Dijkers MP. Difficult to measure constructs: conceptual and methodological issues concerning participation and environmental factors. Arch Phys Med Rehabil 2009;90(11 Suppl 1):S22-35. 10. Fougeyrollas P, Cloutier R, Bergeron H, Coté J, Coté M, St Michel G, editors. Revision of the Quebec Classification: handicap creation process. Quebec: International Network on the Disability Creation Process; 1997. 11. Fougeyrollas P, Noreau L, Bergeron H, Cloutier R, Dion SA, St-Michel G. Social consequences of long term impairments and disabilities: conceptual approach and assessment of handicap. Int J Rehabil Res 1998;21:127-41. 12. World Health Organization. International Classification of Impairments, Disabilities, and Handicaps: a manual of classification relating to the consequences of disease. Geneva: World Health Organization; 1980. 13. Harwood RH, Rogers A, Dickinson E, Ebrahim S. Measuring handicap: the London Handicap Scale, a new outcome measure for chronic disease. Qual Health Care 1994;3:11-6. 14. Willer B, Ottenbacher KJ, Coad ML. The community integration questionnaire: a comparative examination. Am J Phys Med Rehabil 1994;73:103-11. 15. McColl MA, Davies D, Carlson P, Johnston J, Minnes P. The community integration measure: development and preliminary validation. Arch Phys Med Rehabil 2001;82:429-34. 16. Malec JF. The Mayo-Portland Participation Index: a brief and psychometrically sound measure of brain injury outcome. Arch Phys Med Rehabil 2004;85:1989-96. 17. Heinemann AW, Whiteneck G, Brooks CA, et al. Measurement of community participation from the perspective of multiple stakeholders. Arch Phys Med Rehabil 2007;88:E25. Poster presented at the 84th Annual Meeting of the American Congress of Rehabilitation Medicine, Washington, DC, October 3-7, 2007. 18. Brown M, Dijkers MP, Gordon WA, Ashman T, Charatz H, Cheng Z. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004;19:459-81. 19. Dijkers MP. Issues in the conceptualization and measurement of participation: an overview. Arch Phys Med Rehabil. In press.
S59
20. National Spinal Cord Injury Statistical Center. The Spinal Cord Injury Model Systems’ data collection syllabus for the National Spinal Cord Injury Database 2006-2011 project period. 2006. Available at: http://images.main.uab.edu/spinalcord/pdffiles/ Syllabus%202006-2011%20Revised%205-09AppdB-E.pdf. Accessed June 14, 2009. 21. Traumatic Brain Injury National Data and Statistical Center. Traumatic Brain Injury Model Systems national database syllabus. 2009. Available at: http://www.tbindsc.org/Syllabus.aspx. Accessed June 14, 2009. 22. Whiteneck G, Meade MA, Dijkers M, Tate DG, Bushnik T, Forchheimer MB. Environmental factors and their role in participation and life satisfaction after spinal cord injury. Arch Phys Med Rehabil 2004;85:1793-803. 23. Whiteneck GG, Gerhart KA, Cusick CP. Identifying environmental factors that influence the outcomes of people with traumatic brain injury. J Head Trauma Rehabil 2004;19:191-204. 24. Salter KL, Foley NC, Jutai JW, Teasell RW. Assessment of participation outcomes in randomized controlled trials of stroke rehabilitation interventions. Int J Rehabil Res 2007;30:339-42. 25. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health status measure. Med Care 1981;19:787-805. 26. The EuroQoL Group. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy 1990;16:199208. 27. van Nes SI, Faber CG, Merkies IS. Outcome measures in immunemediated neuropathies: the need to standardize their use and to understand the clinimetric essentials. J Peripher Nerv Syst 2008; 13:136-47. 28. Merkies IS, Schmitz PI, Van Der Meche FG, Samijn JP, Van Doorn PA. Psychometric evaluation of a new handicap scale in immune-mediated polyneuropathies. Muscle Nerve 2002;25: 370-7. 29. Badley EM. Enhancing the conceptual clarity of the activity and participation components of the International Classification of Functioning, Disability, and Health. Soc Sci Med 2008;66: 2335-45.
Arch Phys Med Rehabil Vol 91, Suppl 1, September 2010