Evaluating quality-of-life and health status instruments: development of scientific review criteria

CLINICAL THERAPEUTICS’VVOL.

18, NO. 5, 1996

Evaluating Quality-of-Life and Health Status Instruments: Development of Scientific Review Criteria* Kathleen N. L.ohr, PhD,’ Neil K. Aaronson, PhD,2 Jordi Alonso, MD, PhD,3 M. Audrey Burnam, PhD,4 Donald L. Patrick, PhD, MSPH,s Edward B. Pen-in, PhD,5 and James S. Roberts, MD6 ‘Health Services and Policy Research Program, Research Triangle Institute, Research Triangle Park, North Carolina, 2Division of Psychosocial Research and Epidemiology, The Netherlands Cancer Institute, Amsterdam, The Netherlands, 3Department of Epidemiology and Public Health, Institut Municipal d’lnvestigacio Medica, Barcelona, Spain, 4The RAND Corporation, Santa Monica, California, ‘Department of Health Services, University of Washington, Seattle, Washington, and 6Voluntary Hospitals of America, Irving, Texas

ABSTRACT The Medical Outcomes Trust is a depository and distributor of high-quality, standardized, health outcomes measurement instruments to national and international health communities. Every instrument in the Trust library is reviewed by the Scientific Advisory Committee against a rigorous set of eight attributes. These attributes consist of the following: (1) conceptual and measurement model; (2) reliability; (3) validity; (4) responsiveness; (5) inter-

*Presented at the first annual international meeting of the Association for Pharmacoeconomics and Outcomes Research, May 12-15, 1996, Philadelphia, Pennsylvania.

0149-2918/96/$3.50

pretability; (6) respondent and administrative burden; (7) alternative forms; and (8) cultural and language adaptations. In addition to a full description of each attribute, we discuss uses of these criteria beyond evaluation of existing instruments and lessons learned in the first few rounds of instrument review against these criteria.

INTRODUCTION This paper reports on work from the Medical Outcomes Trust and its Scientific Advisory Committee (SAC).‘** The Trust is a nonprofit public service organization, established in 1994 to serve as a depository and distributor of highquality, standardized, health outcomes

979

CLINICAL THERAPEUTICS”

measurement instruments to national and international health communities. The goal of the Trust is to achieve universal adoption of health outcomes measurement to improve the value of health services. It believes that strategies for meamonitoring, and managing suring, functional outcomes, and for integrating those approaches into quality improvement programs, offer the best hope of achieving value from the nation’s investments in health care. Thus the Trust was established to ensure that advanced patient-based instruments and related support materials are easily available to all interested parties. The Trust does this through a variety of services intended to support the application of health outcomes assessment in everyday health care practice; those services include providing timely information and answers to questions, technical support, and ongoing educational activities.3 To ensure that the instruments accepted into the Trust meet rigorous scientific standards, it established the SAC to evaluate all instruments submitted and make recommendations to the Trust’s board of trustees. The SAC evaluates these instruments in the context of their intended applications as well as their general significance and contributions to the field, and also in relation to explicit, public criteria that ensure consistent and defensible recommendations and decisions. To that end, the SAC in 1995 made public its current set of eight attributes used to review and evaluate instruments. These are briefly presented in the remainder of this paper in eight “pairs”: first a brief definition of the attribute and then a short outline of the assessment criteria. The appendix gives the fully detailed set of attributes and criteria.

980

ATTRIBUTES CRITERIA

AND RELATED

Conceptual and Measurement Model The first attribute is the instrument’s conceptual and measurement model. The conceptual model comprises the underlying rationale for and description of the concepts that the measure is intended to assess and the relationships between or among those concepts. The measurement model is reflected in the instrument’s scale and subscale structure and the procedures that are used to create the scale scores. At least four major questions must be answered about the conceptual and measurement model. First, what is the basis for combining items into scales? Second, what descriptive statistics for scales can be provided, and do the scales demonstrate adequate variability? Third, what evidence describes or supports the intended level of measurement (eg, ordinal, interval, or ratio scales) or the scaling assumptions for preference-weighted measures? Fourth, are the procedures for deriving scale scores from raw scores adequately specified and justified?

Reliability Reliability is the degree to which the instrument is free from random error. This aspect of an instrument is examined in two ways-for internal consistency reliability, that is, high correlations among test items, and for reproducibility, that is, the stability over time in test-retest circumstances or the consistency of scores across raters at a point in time. The SAC uses several criteria to evaluate reliability. These include reliability estimates and standard errors of measure-

K.N. LOHR ET AL.

ment for all elements of an instrument; a clear, complete, and detailed description of how the reliability data are collected; reliability estimates for subpopulations (eg, persons with a specific chronic disease or groups with different languages or ethnic backgrounds); and the application of minimal reliability coefficients, depending on whether the comparisons are to be with groups or for individuals. In addition, specific information about test and retest scores is expected, including various coefficients appropriate for instruments yielding interval-level or nominal or ordinal scale values.

as an element of construct validation. Some experts think of it as the ratio of real change over time to variability in scores over time that is not associated with true change in status. More colloquially, responsiveness is the signal-to-noise ratio. The SAC assesses this attribute using longitudinal data, ideally from field tests that compare a group that is expected to change with one that is expected to stay the same, in which change scores are expressed as effect sizes. SAC reviewers look for clear descriptions of those populations and of the approaches used to calculate change and effect size.

Validity

Interpretability

In many arenas, validity is the most important attribute. The SAC uses a classic definition-namely, the degree to which an instrument measures what it purports to measure. We are concerned with three aspects of validity: content, construct, and criterion. The standards used by the SAC to evaluate validity include the methods used to collect content and construct validity as well as a defense of any criteria measures when the instrument developers make a claim for criterion validity. SAC reviewers also look for validity evidence for every proposed use of the instrument and for different populations if it is believed that validity may differ for such groups.

Interpretability is defined as the degree to which one can assign qualitative meaning-that is, clinical or commonly understood connotations-to quantitative scores. This can be done with various types of information intended to aid in interpreting scores on the instrument. Examples include comparative data on the distribution of scores from other groups, including the general public, and information on the relationship of scores to disease conditions or health care interventions, or to various events such as losing a job, graduating from college, or needing institutional care. Basically, in evaluating interpretability, the SAC seeks clear descriptions of the comparison populations and the means by which the relevant data (eg, on clinical conditions or life events) were amassed, recorded, interpreted, and displayed.

Responsiveness Responsiveness is an instrument’s ability to detect change in outcomes that matter to persons with a health condition, to their significant others, or to their providers. It sometimes is referred to as sensitivity to change, and it is also sometimes regarded

Respondent and Administrative Burden The sixth attribute is really a pair of properties-respondent burden and administrative burden. These involve the time,

981

CLINICAL THERAF’EUTICS”

energy, financial resources, personnel, or other resources required of respondents or those administering the instrument. With respect to respondent burden, the SAC is concerned that the instrument place no undue physical or emotional strain on the respondent. Among the important items of information are the following: time needed to complete the form or interview; levels of reading or comprehension assumed; any special requirements that might be placed on respondents, such as need to consult health or financial records; and acceptability of the instrument, such as the level of missing data or refusal rates. Regarding administrative burden, the SAC requires information about the time needed for a trained interviewer to administer the instrument, the assumptions about the level of training or expertise needed to administer the instrument, and any special resources needed, such as specific computer equipment to administer, score, or analyze the instrument.

Alternative Forms Alternative forms, the seventh attribute, refers to all the ways in which the instrument might be administered other than the original way; this particular characteristic can refer to many different modes of applying the instrument. In addition, alternative forms can include proxy versions of the original instrument. Two aspects of evaluating alternative forms of an instrument are important. The first is to evaluate these using most of the criteria already mentioned-that is, evidence of reliability, validity, responsiveness, interpretability, and burden. In addition, SAC reviewers want to see information indicating how compa-

982

rable each alternative inal document.

form is to the orig-

Cultural and Language Aaizptations Finally, the eighth attribute involves cultural and language adaptations. Here the focus is on conceptual and linguistic equivalence between the original instrument and its adaptations-that is, equivalence in relevance and meaning of the various concepts in an instrument as well as equivalence of the wording and meaning of items and response choices. In addition, the independent psychometric properties of the adaptations are significant aspects of the review, so again the SAC requires information to be provided about reliability, validity, responsiveness, interpretability, and burden. In judging this attribute, SAC reviewers seek material about how the developers sought to achieve conceptual equivalence, ideally in terms of content validity. Also, common rules concerning translations are applied: at least two forward translations and a pooled forward translation; at least one (more would be better) backward translation and another pooled translation; review of these versions by both lay and expert panels, with revisions as necessary; and field tests to gather evidence of comparability. Certainly any significant differences between the original and translated versions should be pointed out and explained.

LESSONS LEARNED IN EARLY ROUNDS OF INSTRUMENT REVIEW By early 1996, the SAC had reviewed perhaps two dozen instruments using these criteria and a wide array of information

K.N. LOHR ET AL.

groups (over time) may be found but may not be of any real importance to either the patients or the clinicians. Furthermore, data related to the clinical interpretation of scores are critical when health status or quality-of-life instruments are to be introduced into clinical practice settings, again because of the central importance of the patients’ and the clinicians’ perspectives. A second lesson is that, notwithstanding these gaps, developers tend to be focused on the conceptual and statistical properties of their measures, and issues of the practicality of the instrument get short shrift. Even when information on reliability and validity is available, for instance, facts and figures on administrative or respondent burden are not given. For users wanting good guidance from the Trust about applications of different instruments, this is an unfortunate omission. A third point is an inevitable tension between insisting on scientific rigor and passing instruments that may not achieve the level of scientific validity that we would like to see but that appear to have real promise or to fill an important gap in the

supplied by the developers or their representatives. The main purpose has been Eo make determinations about recommendations to the Trust’s board concerning instruments to be included in its library. (For instruments in the Trust’s library in September 1996, see the table.) This effort has provided several lessons. First, the psychometric analyses may well have been done for much of the work reviewed, but they often are not fully reported. The SAC often has to defer decisions and ask specifically for the missing information in a second round. In particular, two key pieces of information are often lacking altogether or are only partially available for review. One area involves data on the responsiveness of instruments to changes in health status over time, and the other concerns data that will facilitate (or even enable) the interpretation of scores in terms that are meaningful for clinicians or lay persons and policy makers. This gap appears particularly worrisome for largescale clinical trials, where statistically significant differences in scores between

Table.

Medical Outcomes Trust-approved

instruments

(as of September

1996).

London Handicap Scale Quality of Well-Being Scale Seattle Angina Questionnaire SF-12 Health Survey SF-36 Health Survey (standard and acute versions) Languages available: Australia/New Zealand (English) Canada (English) Germany (German) Spain (Spanish) Sweden (Swedish) United Kingdom (English) United States (English) Sickness Impact Profile

983

CLINICAL THERAPEUTICS’

measurement armamentarium. Generally, the SAC has had the primary objective of applying stringent standards to all instruments as a means of ensuring the quality of those contained in the Trust’s library. When the information needed may well be available or obtainable with relative ease, the SAC asks those submitting instruments to provide or collect and forward it, as a means of encouraging the full development of their work and this field.

CONCLUSIONS By consistently applying publicly known criteria, the SAC can contribute to the Trust’s objective of assuring widespread availability of a comprehensive library of high-quality health outcomes instruments. By making the criteria well known, we believe that others may use them for evaluating and revising their own instruments, for developing new measures, and for choosing functional outcomes instruments for both clinical trials and health services research. Finally, the criteria may prove valuable for others to use in assessing the adequacy of research, in evaluating claims

984

for new diagnostic or therapeutic modalities, and in reviewing publications in this field. The SAC and the Trust welcome both submissions of health status and quality-of-life instruments and comments and suggestions about the instrument review criteria and their applications.

Address correspondence to: Kathleen N. Lohr, PhD, Director, Health Services and Policy Research Program, Research Triangle Institute, PO Box 12194, 3040 Cornwallis Drive, Research Triangle Park, NC 27709-2194. REFERENCES Perrin EG. SAC instrument review process. Med Outcomes Trust Bull. 1995; 3:l.

Scientific Advisory Committee. Instrument review criteria. Med Outcomes Trust Bull. 199$3:1-N SourcePages. Vol. 1, No. 1. Boston: Medical Outcomes Trust; 1996.

K.N. LOHR ET AL.

Appendix.

Scientific Advisory Committee INSTRUMENT REVIEW CRITERIA The Scientific Advisory Committee of the Medical Outcomes Trust has established criteria by which to evaluate instruments that are submitted to the Trust for inclusion in the library. The Committee identified eight instrument attributes to serve as the principle foci of their review. The relative importance of criteria for instrument review may differ depending on the intended use(s) and application(s) of the instrument. Instruments may be intended to distinguish between two or more groups, assess change over time, or predict future status. Instruments will be reviewed in the context of documented

applications as stated in the Instrument Submission Form available by contacting the

Trust. The properties of an instmment are context specific. An instrument that works well for one purpose or in one setting or population may not do so when applied for another purpose or in another setting or population. Ideally, one would want to have evidence of the measurement properties of an instrument for each of its intended applications.

I. CONCEPTUAL AND MEASUREMENT MODEL DEFINITION A conceptual model is a rationale for and description

of the concept(s) that the measure is intended

to assess and the relationship between those concepts. A measurement

model is defined as an instru-

ment’s scale and subscale structure, and the procedures followed to create scale and subscale scores. The adequacy of the measurement

model can be evaluated by examining evidence: (1) that a scale

measures a single conceptual domain or construct; (2) that multiple scales measure distinct domains; (3) that variability in the domain is adequately represented by the scale; and (4) that the intended level of measurement

of the scale and its scoring procedures are well justified.

REVIEW CRITERIA (1) The conceptual and empirical basis for combining multiple items into a single scale score and/or multiple scale scores should be provided. This might include information

on factor structure, distinctiveness

consistency

of scales as generated by methods such as factor analysis and Rasch

of multiple scales, and internal

or structural equation modeling.

(2)

Descriptive

statistics for each scale should be provided, including information

on

central tendency and dispersion, skewness, and frequency of missing data. Any other evidence that the scale demonstrates adequate variability in a range that is relevant to its intended use should be provided.

985

CLINICAL THERAPEUTICS”

(3)

A description

of the intended level of measurement,

for example, ordinal, interval or

ratio scales, should be provided, along with available supportive evidence. For preference-weighted assumptions (4)

measures, a rationale and evidence supporting

scaling

should be provided.

Procedures for deriving scale scores from raw scores should be clearly specilied and a clear description standardization)

of and rationale for transformations

(such as weighting

and

should be provided.

II. RELIABILITY DEFINITION The principal definition of test reliability is the degree to which an instrument is free from random error. This succinct definition implies homogeneity sistency (ie, high correlations)

of content on multi-item tests and internal con-

among test items. The two approaches recommended

test reliability are coefftcient (Y(Cronbach’s alpha) and alternative form correlations. ter approach is seldom used in health status assessment,

for examining Because the lat-

the coefficient 0: can be considered the most

relevant approach to reliability estimation. A second definition

of reliability

is reproducibility

or stability of an instrument

retest) and interrater agreement at one point in time. The two definitions

over time (test-

are largely independent

of

one another.

A. Internal Consistency Coefficient

cx provides an estimate of reliability based on all possible correlations

between two sets of items within a test. For instruments

employing

response choices, an alternative formula, the Kuder-Richardson

dichotomous

formula 20 (KR-20).

is available.

B. Reproducibility Test-retest reproducibility: Test-retest reproducibility

is the degree to which an instrument yields stable scores

over time among respondents who are assumed not to have changed on the domains being assessed. The influence of test administration on the second administration may overestimate reliability. Conversely, variations in health, learning, reaction, or regression to the mean may yield test-retest data underestimating reliability. Despite these cautions, information on test-retest reproducibility data is important for the evaluation of the instrument. Inter-observer

(interviewer)

reproducibility:

For instruments administered by an interviewer, to both i&a-observer and inter-observer agreement.

986

test-retest

reproducibility

may refer

K.N. LOHR ET AL.

REVIEW CRITERIA

(1)

Reliability estimates and standard errors should be reported for all elements of an instrument, including both the total score and subscale scores, where appropriate.

(2)

A clear description should be provided of the methods employed to collect reliability data. This should include (a) methods of sample accrual and sample size; (b) characteristics of the sample (eg, sociodemographics, clinical characteristics if drawn from a patient population, etc.); (c) the testing conditions (ie, where and how the instrument of interest was administered); and (d) descriptive statistics for the instrument under study (ie, means, standard deviations, floor and ceiling effects, etc.).

(3)

Where there are reasons measurement will differ instrument is to be used, of interest (eg, different groups, etc.).

(4)

Test-retest reproducibility information should be provided as a complement to, not as a substitute for, internal consistency. Reproducibility is more important when repeated measures with the instrument are proposed.

(5)

A well-argued rationale should support the time elapsed between first and second administration, as well as the design of the study to ensure that changes in health status were minimal, ie, including transitional questions on general and specific health or functional status.

(6)

Information about test and retest scores should include the appropriate central tendency and dispersion measures of both test and retest administrations.

(7)

For instruments yielding interval-level data, information on test-retest reliability (reproducibility) and interrater reliability should include intraclass correlation coefficients (ICC). For nominal or ordinal scale values, kappa and weighted kappa, respectively, are recommended.

(8)

Commonly accepted minimal standards for reliability coefftcients comparisons and 0.90-0.95 for individual comparisons.

to believe that reliability estimates or standard errors of substantially for the various populations in which an these data should be presented for each major population chronic disease populations, different language or cultural

are 0.7 for group

987

CLINICAL THERAPEUTICS”

III. VALIDITY DEFINITION The validity of an instrument is defined as the degree to which the instrument measures what it purports to measure. There are three ways of accumulating A.

Content-related:

evidence that the content domain of an instrument

relative to its intended use. Methods commonly content-related

Construct-related:

is appropriate

used to obtain evidence about

validity include the use of lay and expert panel judgments

clarity, comprehensiveness, B.

evidence for the validity of an instrument:

and redundancy

of the

of items and scales of an instrument.

evidence that supports a proposed interpretation

of scores on the

instrument based on theoretical implications associated with the constructs. Common methods to obtain construct-related

validity include an examination

of the logical

relations that should exist with other measures and/or patterns of scores across groups of individuals. C.

Criterion-related:

evidence that shows the extent to which scores of the instrument

are related to a criterion measure. Criterion measures are measures of the target construct that are widely accepted valid measures of that construct. In the area of health status assessment,

criterion-related

validity is rarely tested because of

the absence of widely accepted criterion measures.

REVIEW CRITERIA

(1)

Evidence of content validity should be presented

for every major proposed use of

the instrument. Information about the methods for developing instrument should be provided.

(2)

the content of the

Evidence of construct validity should be presented for every major proposed use of the instrument.

When data related to criterion validity are presented,

a clear

rationale and support for the choice of criteria measures should be stated. A rationale should be provided to support the particular mix of evidence presented for the intended uses. (3)

A clear description should be provided of the methods employed to collect validity data. This should include (a) methods of sample accrual and sample size; (b) characteristics of the sample (eg, sociodemographics, clinical characteristics if drawn from a patient population, etc.); (c) the testing conditions (ie, where and how the instrument of interest was administered); and (d) descriptive statistics for the instrument under study (ie, means, standard deviations, floor and ceiling effects, etc.).

988

K.N. LOHR ET AL.

(4)

The composition of the validation sample should be described in sufficient detail to make clear the population(s) to which it applies. Available data on selective factors that might reasonably be expected to influence validity, such as gender, age, ethnicity, and language, should be described.

(5)

Where there are reasons to believe that validity will differ substantially for the various populations in which an instrument is to be used, these data should be presented for each major population of interest (eg, different chronic disease populations, different language or cultural groups, etc.).

IV. RESPONSIVENESS DEFINITION Sometimes

referred to as sensitivity to change, responsiveness

construct validation process. Responsiveness defined as the minimal change considered their significant

is viewed as an important part of the

refers to an instrument’s ability to detect change, often to be important by the persons with the health condition,

others, or their providers. The criterion of responsiveness

requires asking whether

the measure can detect differences in outcomes that are important, even if those differences are small. Responsiveness

can be conceptualized

also as the ratio of a signal (the real change over time that

has occurred) to the noise (the variability in scores seen over time that is not associated

with true

change in status). Common methods of evaluating responsiveness intervention

that is expected

include comparing

to affect the construct,

and comparing

scale scores before and after an changes in scale scores with

changes in other related measures that are assumed to move in the same direction as the target measure. Assessment

of responsiveness

often involves estimation of the effect size. Effect size is an estimate

of the magnitude of change in health status. Effect size translates the before-and-after a standard unit of measurement. tion of effects is discussed

changes into

Different methods may be used to calculate effect size. Interpreta-

in Section V.

REVIEW CRITERIA (1) For any claim that an instrument is responsive, evidence should be provided on the change scores found in field tests of the instrument. should be expressed

These change scores

ideally as effect sizes, with information

provided on the

methods used to calculate the effect size.

(2)

Claims for an instrument’s responsiveness

should be derived from longitudinal data,

preferably comparing a group that is expected to change with a group that is expected to remain stable. (3)

The population(s) on which responsiveness has been tested should be clearly identified including the time intervals of assessment, the interventions or measures involved in evaluating change, and the populations assumed to be stable.

989

CLINICAL THERAPEUTICS”

V. INTERPRETABILITY DEFINITION Interpretability

is defined as the degree to which one can assign qualitative meaning to an instru-

ment’s quantitative a quantitative

scores. Interpretability

of a measure is facilitated by information

that translates

score or change in scores to a qualitative category that has clinical or commonly

un-

derstood meaning.

There are several types of information

(1)

comparative

that can aid in the interpretation

data on the distribution

of scores:

of scores derived from a variety of defined

population groups, including, when possible, a representative

sample of the

general population,

(2)

information

on the relationship

of scores to clinically recognized

conditions

or need

for specific treatments, (3)

information on the relationship of scores or changes in scores to commonly recognized life events (such as the impact of losing a job), and

(4)

information

on how well scores predict known relevant events (such as death or need for

institutional

care).

REVIEW CRITERIA (1)

A clear description

should be provided of the rationale for selection of populations

for purposes of comparison

and interpretability

of data. This should include (a)

methods of sample accrual and sample size; (b) characteristics of the sample (eg, sociodemographics, clinical characteristics if drawn from a patient population, etc.); (c) the testing conditions administered);

(ie, where and how the instrument

and (d) descriptive

means, standard deviations, (2)

statistics for the instrument

of interest was under study (ie,

floor and ceiling effects, etc.).

Provide any information regarding various ways in which the data have been reported and displayed in order to facilitate interpretation.

VI. BURDEN DEFINITION Respondent burden is defined as the time, energy, and other demands placed on those to whom the instrument is administered. Administrative burden is defined as the demands placed on those who administer

990

the instrument.

K.N. LOHR ET AL.

REVIEW

(1)

CRITERIA:

Evidence

RESPONDENT

emotional strain on the respondent. circumstances

(2)

BURDEN

should be provided that the instrument places no undue physical or

Information

their instrument

Developers

should indicate when or under what

is not suitable for respondents.

should be provided on the average time and range of time needed to

complete the instrument

on a self-administered

administered

for all population

instrument,

basis or as an interviewer-

groups for which the instrument is

intended. (3)

Information

should be provided about the reading and comprehension

for all population (4)

Information

groups for which the instrument

should be provided about any special requirements

might be placed on respondents, or copy information (5)

Information

level assumed

is intended. or requests that

such as the need to consult health care records

about medications

used.

should be provided on the acceptability

of the instrument,

for example,

by indicating the level of missing data and refusal rates and the reasons for both. REVIEW (1)

CRITERIA:

ADMINISTRATIVE

For interviewer-administered

instruments,

BURDEN information

should be provided on the

average time and range of time required of a trained interviewer instrument.

If appropriate

(eg, if the times differ significantly),

should be given for face-to-face

interview, telephone,

to administer

the

the information

and computer-assisted

formats/applications. (2)

Information

should be provided on the amount of training and level of education or

professional

expertise

and experience

needed by administrative

staff to administer,

score, or otherwise use the instrument. (3)

Information

should be provided about any resources required for administration

the instrument,

software to administer, VII. ALTERNATIVE

of

such as the need for special or specific computer hardware or score, or analyze the instrument.

FORMS

DEFINITION Altemative forms of an instmment include all modes of administration other than the original source instrument. Depending on the nature of the original source instrument altemative forms can include self-administered self-report, interviewer-administemd self-mm trained observer rating, computer-assisted self-repo& computer-assisted interviewer-administered report_ and performancebased measures. hi addition altemative forms may include proxy versions of the original source instrument such as self-administemd proxy report and interviewer-administered proxy report.

991

CLINICAL THERAPEUTICS”

REVIEW CRITERIA Alternative forms of an instrument will be evaluated employing inal source instrument. pretability,

This will include evidence

the same criteria used for the orig-

of reliability,

and burden. An additional criterion for evaluation

validity, responsiveness,

of alternative

inter-

forms will be compa-

rability with the original instrument.

VIII. CULTURAL

AND LANGUAGE

ADAPTATIONS

DEFINITION The cross-cultural

adaptation of an instrument

ceptual and linguistic equivalence, alence refers to equivalence

involves two primary steps: (1) assessment

and (2) evaluation of measurement

in relevance and meaning of the same concepts being measured in dif-

ferent cultures and/or languages. Linguistic equivalence and meaning in the formulation its applications.

For evaluation

tion will be reviewed

of items, response of measurement

refers to equivalence

interpretability,

of question wording

choices, and all aspects of the instrument

properties,

and

each cultural and/or language adapta-

separately by the Scientific Advisory Committee

validity, responsiveness,

of con-

properties. Conceptual equiv-

for evidence of reliability,

and burden.

REVIEW CRITERIA (1)

Information

about methods to achieve conceptual

It is commonly

recommended

equivalence

should be provided.

that the content validity of the instrument

be

assessed in each cultural or language group to which the instrument is to be applied. (2)

Information

about methods to achieve linguistic equivalence

is commonly recommended

should be provided. It

that there should be: (a) at least two forward translations

from the source language, preferably

by persons experienced

in translations

in health status research. This should result in a pooled forward translation; least one, preferably

more, backward translations

and (b) at

to the source language that

results in another pooled translation; (c) a review of translated versions by lay and expert panels with revisions; and (d) field tests to provide evidence of comparability. (3)

Any significant identified

differences

between the original and translated versions should be

and explained.

Appendix. Scientific Advisory Committee instrument Medical Outcomes Trust, Boston, Massachusetts.

992

review criteria.

Reprinted,

with permission,

from the