- Email: [email protected]
Development and construct validity of a knee pain questionnaire
Pain, 31 (1987) 47-52 Elsevier
47
PAI 01099
development and construct validity of a knee pain questionnaire Michel E.H. Boeckstyns Department of Orthopaedic Surgery Municipality of Copenhagen Hospital, Hvidovre (Denmark] (Received
9 December
1986, accepted
19 March
1987)
A knee pain questionnaire, consisting of 15 dichotomous items, was submitted to an item Summary analysis, based on the probabilistic model of Rasch. Five items had to be discarded but the remaining 10 constituted a homogeneous set, fit to be used as a scale. In particular, this meant that the total number of positive responses to the questionnaire could be used as a simple measure of the patients’ knee pain. Further studies must be performed in order to analyze the reliability and empirical validity of the scale. Key words:
Arthroplasty;
Knee-joint;
Osteoarthritis;
Pain; Rheumatoid
arthritis
Introduction
The purpose of the present study was to develop a method of evaluating knee pain that could be used in clinical research on the treatment of degenerative and chronic inflammatory diseases of the knee, in particular in evaluating the results following total joint replacement.
Methods
A knee pain scale, consisting of a questionnaire in which each question was a dichotomous scale item and the total number of positive responses the total score, was constructed (Table I). The procedure corresponds to Guttman scaling as known in psychometrics [l]. The choice of the items was based on interviews of patients in whom prosthetic replacement of the knee had been performed; items that a priori were believed to be biased were excluded. In formulating the possible answers to the questionnaire items, any grading of the intensity of pain, such as ‘none,’ ‘slight, Correspondence ro: Michel E.H. Boeckstyns,
0304-3959/87/$03.50
A.N. Hansens
0 1987 Elsevier Science Publishers
Alle 27, DK 2900 Hellerup,
B.V. (Biomedical
Division)
Denmark.
4x
1 Do you suffer at all from any kind of pain in your knee’? 2 Do you wake up at night feeling pain in your knee’?
3 Does your knee hurt when you wake up in the morning?
4 Does your knee hurt when you get up in the morning’?
5 Does your knee hurt when you are lying down?
6 Does you knee hurt when you are sitting?
7 Does your knee hurt when you move it in certain
ways?
8 When you are lying down, do you have to keep you knee in a certain position to avoid pain? 9 When you are sitting, do you have to keep your knee in a certain position to avoid pain? IO To avoid pain, must you be careful not to bend your knee too much?
11 Does your knee hurt when you are getting out of a chair or trying to do so? 12 Does pain in your knee restrict you in your daily activities? 13 Does your knee hurt when the weather changes, when it is raining or when there is a fog? 14 Does your knee hurt when you are standing up, but not walking? 15 Does your knee hurt when you walk?
No
Ye3 Never Rarely Regularly Never Rarely Regularly Never Rarely Regularly Never Rarely Regularly Never Rarely Regularly N
‘moderate,’ ‘severe’ etc., was avoided. The only possibility was to state whether pain was present or not. The responses ‘yes’ and ‘regularly’ were considered positive, negative. Unanswered items were while ‘no,’ ‘ rarely’ and ‘never’ were considered counted as positively answered. Putients The questionnaire was tested in connection with a long-term follow-up of 60 patients in whom knee resurfacing had been performed at the Department of Orthopaedic Surgery at Hvidovre Hospital during the period 1976-1979. Excluding the dead and those who refused to participate, 49 patients entered the study. They
49
were requested to consider both knees in answering the questionnaire and thus 98 knees were evaluated for pain: 52 Marmor knees, 5 hinged knees, 5 total condylar knees, 1 arthrodetic knee, 1 knee that had been corrected with a valgus osteotomy and 34 knees that had not been operated upon. Seventeen patients suffered from rheumatoid arthritis, 28 from degenerative arthritis and 4 from miscellaneous diseases. Statistics An item analysis, based on the probabilistic model of Rasch [3,4], was made. This model is the only statistical model that allows the ranking of objects according to the sum of single scores. In short, it deals with the probability that a patient with a given degree of a latent trait, in casu knee pain, would answer a given item positively. The item analysis permitted the evaluation of the proposed scale’s construct validity, i.e., an evaluation of whether the total number of positive responses to the questionnaire could be used to rank the knees in terms of pain. In the model *, each item was characterised by a parameter that is a measure of the ease with which it elicits a positive response: e.g., the item ‘Does your knee hurt when you walk?’ yielded a positive answer more often that the item ‘Do you wake up at night feeling pain in your knee ?.’ The response to an item thus naturally depends on the pain in a given knee but the item parameter should be an inherent and constant characteristic of the item, regardless of which knee is considered and independent of any other external factors, such as diagnosis, age of the patient, etc. This independence has been called ‘objective specificity’ by Rasch and is the central concept in the item analysis [3]. Computer-conducted calculations based on the item parameters permitted an evaluation of test item bias [2], of test item interdependence and of scale dimensionality.
Results A crude estimate of the item parameter connected to each of the questionnaire’s items was given by the frequency of positive responses to the particular item in the material as a whole. Indeed, items that easily elicit a positive response would yield a high frequency. That the item parameter is independent of the patients’ pain is demonstrated by the fact that the parameters were constant throughout different subgroups of patients characterized by different degrees of pain, i.e., different scores on the scale. This was illustrated by the item characteristic curves (Fig. 1). Each item is characterized by a curve representing the frequency of positive responses in different pain score groups. The fact that these curves on the whole did not intersect demonstrated that the ranking of items according to their item parameter was independent of the pain scores.
* Mathematically, the model is P = (6 x [)/(l + S X 0, where P is the probability of a positive response to an item in the questionnaire, 6 is this item’s parameter and 5 the degree of pain in the knee under consideration.
so PROPORTION OF POSITIVE RESPONSE (%) 100 90 80 70 60 50 40 30 20 10
Fig. 1. Estimated item characteristic curves for items I-15 in the originaf 15-item questionnaire. The curves for items 9, 10 and 13 (enhanced) intersect a considerable number of other curves, indicating that the ranking of the items. according to the proportion of positive responses, differs from one score subgroup to another.
TABLE
II
OBSERVED AND EXPECTED FREQUENCIES OF POSITIVE RESPONSES TO ITEM 13 FOR EACH PAIN SCORE OBTAINED IN THE IS-ITEM QUESTIONNAIRE (scores 0 and 15 excluded) Total pain score
I 2 3 4 5 6 7 8 Y IO 11 12 13 14 * Significant
Observed frequency of positive responses
0.0 100.0 75.0 83.3 75.0 54.5 57.1 100.0 50.0 62.5 66.7 100.0 100.0 50.0 at the 0.05 level
(%)
Expected frequency of positive responses 13.8 27.2 39.8 51.2 61.3 70.0 77.3 83.3 X8.0 91.7 94.5 96.5 98.0 99.2
P
(%) 0.86229 0.02013 * 0.04853 * 0.12111 0.34273 0.21115 0.19800 0.33336 0.22478 0.02308 * 0.15640 0.89952 0.92389 0.00042 *
51 PROPORTlON OF POSITIVE RESPON$ES (‘1.) 100 90 80 70 60 50 40 30 20 to
Fig. 2. Estimated item characteristic curves for the revised questionnaire, i.e., after exclusion of items 3, 8, 9, 10 and 13. Compared to Fig. 1, the curves rarely intersect.
However, there were a number of irregularities, most pronounced in the case of 9,lO and 13, whose characteristic curves in fact did intersect with a number of other curves. Computer analysis permitted a much more accurate calculation of item parameters, in the material as a whole as well as in score subgroups. On the basis of these calculations, the hypothesis stating that the 15 items formed a homogeneous scale had to be rejected (likelihood ratio test: x2 = 62.83, df= 28, P = 0.00017). In particular, it was confirmed that item 13 was biased: a significantly higher proportion of positive answers was obtained in patients with a low total score (i.e., postulated low-grade pain) than expected from the calculated item parameters, and vice versa (Table II). The subsequent tests revealed that items 8, 9 and 10 on the one hand, and the remaining items on the other, did not measure the same dimension (Per Martin Lof test, likelihood ratio = 74.5, #= 35, P = 0.00011). Furthermore, items 2 and 3 proved to be mutually dependent to an extent that made them unfit as a pair in the scale model. No test item dependence on diagnosis (rheumatoid arthritis or other) or on treatment (operated or not) was shown. Thus, the analysis discarded items 3, 8, 9, 10 and 13. This having been achieved, re-analysis was performed, and now the hypothesis of homogeneity of items could no longer be rejected as far as the remaining 10 items were concerned (likelihood ratio test: x2 = 15.6, df= 18, P = 0.62058). Likewise, the simple graphical analysis showed considerable improvement of the relations between the item characteristic curves (Fig. 2). items
Discussion The purpose of the present study was to develop a tool that was specifically suitable to evaluate pain in chronically affected knee-joints. Pain was considered as a latent trait and accordingly, a method related to the latent trait theory as known in modern psychometrics was adopted to construct a suitable scale. Thus only the construct validity of the scale was considered and tested by means of the Rasch model, which is one of the most widely accepted models that may serve this purpose I21. The proposed questionnaire. consisting of 15 items, had to be discarded, but a revised questionnaire consisting of 10 items yielded a scale that highty fitted the statistical model, which means that the number of positive responses could be used to rank the knees according to a latent trait related to pain. An analysis of the scale’s empirical validity remains necessary in order to examine whether the particular aspect of pain it deals with is clinically relevant. Moreover, reliability tests must be performed before a final appraisal of the scale can be made.
References I Mclver. J.P. and Carmines, E.G.. Unidimensional Scaling, Sage llniversity Paper series OD Quantitative Applications in the Social Sciences, 07-024, Beverly Hills, 1981, pp. 40-61. 2 Osterlind, S.J., Test Item Bias, Sage University Paper series on Quantitative Applications in the Social Sciences. 07-030, Beverly Hills, 1983. 3 Rasch, G.. An item analysis which takes individual differences into account. Brit. J. math. Stat. Psychoi., 19 (1966) 49-57. 4 Rasch, G., Probabilistic Modeis for Some Intelligence and AttaiIlnlent Tests, University of Chicago Press. Chicago, IL. 1980.
47
PAI 01099
development and construct validity of a knee pain questionnaire Michel E.H. Boeckstyns Department of Orthopaedic Surgery Municipality of Copenhagen Hospital, Hvidovre (Denmark] (Received
9 December
1986, accepted
19 March
1987)
A knee pain questionnaire, consisting of 15 dichotomous items, was submitted to an item Summary analysis, based on the probabilistic model of Rasch. Five items had to be discarded but the remaining 10 constituted a homogeneous set, fit to be used as a scale. In particular, this meant that the total number of positive responses to the questionnaire could be used as a simple measure of the patients’ knee pain. Further studies must be performed in order to analyze the reliability and empirical validity of the scale. Key words:
Arthroplasty;
Knee-joint;
Osteoarthritis;
Pain; Rheumatoid
arthritis
Introduction
The purpose of the present study was to develop a method of evaluating knee pain that could be used in clinical research on the treatment of degenerative and chronic inflammatory diseases of the knee, in particular in evaluating the results following total joint replacement.
Methods
A knee pain scale, consisting of a questionnaire in which each question was a dichotomous scale item and the total number of positive responses the total score, was constructed (Table I). The procedure corresponds to Guttman scaling as known in psychometrics [l]. The choice of the items was based on interviews of patients in whom prosthetic replacement of the knee had been performed; items that a priori were believed to be biased were excluded. In formulating the possible answers to the questionnaire items, any grading of the intensity of pain, such as ‘none,’ ‘slight, Correspondence ro: Michel E.H. Boeckstyns,
0304-3959/87/$03.50
A.N. Hansens
0 1987 Elsevier Science Publishers
Alle 27, DK 2900 Hellerup,
B.V. (Biomedical
Division)
Denmark.
4x
1 Do you suffer at all from any kind of pain in your knee’? 2 Do you wake up at night feeling pain in your knee’?
3 Does your knee hurt when you wake up in the morning?
4 Does your knee hurt when you get up in the morning’?
5 Does your knee hurt when you are lying down?
6 Does you knee hurt when you are sitting?
7 Does your knee hurt when you move it in certain
ways?
8 When you are lying down, do you have to keep you knee in a certain position to avoid pain? 9 When you are sitting, do you have to keep your knee in a certain position to avoid pain? IO To avoid pain, must you be careful not to bend your knee too much?
11 Does your knee hurt when you are getting out of a chair or trying to do so? 12 Does pain in your knee restrict you in your daily activities? 13 Does your knee hurt when the weather changes, when it is raining or when there is a fog? 14 Does your knee hurt when you are standing up, but not walking? 15 Does your knee hurt when you walk?
No
Ye3 Never Rarely Regularly Never Rarely Regularly Never Rarely Regularly Never Rarely Regularly Never Rarely Regularly N
‘moderate,’ ‘severe’ etc., was avoided. The only possibility was to state whether pain was present or not. The responses ‘yes’ and ‘regularly’ were considered positive, negative. Unanswered items were while ‘no,’ ‘ rarely’ and ‘never’ were considered counted as positively answered. Putients The questionnaire was tested in connection with a long-term follow-up of 60 patients in whom knee resurfacing had been performed at the Department of Orthopaedic Surgery at Hvidovre Hospital during the period 1976-1979. Excluding the dead and those who refused to participate, 49 patients entered the study. They
49
were requested to consider both knees in answering the questionnaire and thus 98 knees were evaluated for pain: 52 Marmor knees, 5 hinged knees, 5 total condylar knees, 1 arthrodetic knee, 1 knee that had been corrected with a valgus osteotomy and 34 knees that had not been operated upon. Seventeen patients suffered from rheumatoid arthritis, 28 from degenerative arthritis and 4 from miscellaneous diseases. Statistics An item analysis, based on the probabilistic model of Rasch [3,4], was made. This model is the only statistical model that allows the ranking of objects according to the sum of single scores. In short, it deals with the probability that a patient with a given degree of a latent trait, in casu knee pain, would answer a given item positively. The item analysis permitted the evaluation of the proposed scale’s construct validity, i.e., an evaluation of whether the total number of positive responses to the questionnaire could be used to rank the knees in terms of pain. In the model *, each item was characterised by a parameter that is a measure of the ease with which it elicits a positive response: e.g., the item ‘Does your knee hurt when you walk?’ yielded a positive answer more often that the item ‘Do you wake up at night feeling pain in your knee ?.’ The response to an item thus naturally depends on the pain in a given knee but the item parameter should be an inherent and constant characteristic of the item, regardless of which knee is considered and independent of any other external factors, such as diagnosis, age of the patient, etc. This independence has been called ‘objective specificity’ by Rasch and is the central concept in the item analysis [3]. Computer-conducted calculations based on the item parameters permitted an evaluation of test item bias [2], of test item interdependence and of scale dimensionality.
Results A crude estimate of the item parameter connected to each of the questionnaire’s items was given by the frequency of positive responses to the particular item in the material as a whole. Indeed, items that easily elicit a positive response would yield a high frequency. That the item parameter is independent of the patients’ pain is demonstrated by the fact that the parameters were constant throughout different subgroups of patients characterized by different degrees of pain, i.e., different scores on the scale. This was illustrated by the item characteristic curves (Fig. 1). Each item is characterized by a curve representing the frequency of positive responses in different pain score groups. The fact that these curves on the whole did not intersect demonstrated that the ranking of items according to their item parameter was independent of the pain scores.
* Mathematically, the model is P = (6 x [)/(l + S X 0, where P is the probability of a positive response to an item in the questionnaire, 6 is this item’s parameter and 5 the degree of pain in the knee under consideration.
so PROPORTION OF POSITIVE RESPONSE (%) 100 90 80 70 60 50 40 30 20 10
Fig. 1. Estimated item characteristic curves for items I-15 in the originaf 15-item questionnaire. The curves for items 9, 10 and 13 (enhanced) intersect a considerable number of other curves, indicating that the ranking of the items. according to the proportion of positive responses, differs from one score subgroup to another.
TABLE
II
OBSERVED AND EXPECTED FREQUENCIES OF POSITIVE RESPONSES TO ITEM 13 FOR EACH PAIN SCORE OBTAINED IN THE IS-ITEM QUESTIONNAIRE (scores 0 and 15 excluded) Total pain score
I 2 3 4 5 6 7 8 Y IO 11 12 13 14 * Significant
Observed frequency of positive responses
0.0 100.0 75.0 83.3 75.0 54.5 57.1 100.0 50.0 62.5 66.7 100.0 100.0 50.0 at the 0.05 level
(%)
Expected frequency of positive responses 13.8 27.2 39.8 51.2 61.3 70.0 77.3 83.3 X8.0 91.7 94.5 96.5 98.0 99.2
P
(%) 0.86229 0.02013 * 0.04853 * 0.12111 0.34273 0.21115 0.19800 0.33336 0.22478 0.02308 * 0.15640 0.89952 0.92389 0.00042 *
51 PROPORTlON OF POSITIVE RESPON$ES (‘1.) 100 90 80 70 60 50 40 30 20 to
Fig. 2. Estimated item characteristic curves for the revised questionnaire, i.e., after exclusion of items 3, 8, 9, 10 and 13. Compared to Fig. 1, the curves rarely intersect.
However, there were a number of irregularities, most pronounced in the case of 9,lO and 13, whose characteristic curves in fact did intersect with a number of other curves. Computer analysis permitted a much more accurate calculation of item parameters, in the material as a whole as well as in score subgroups. On the basis of these calculations, the hypothesis stating that the 15 items formed a homogeneous scale had to be rejected (likelihood ratio test: x2 = 62.83, df= 28, P = 0.00017). In particular, it was confirmed that item 13 was biased: a significantly higher proportion of positive answers was obtained in patients with a low total score (i.e., postulated low-grade pain) than expected from the calculated item parameters, and vice versa (Table II). The subsequent tests revealed that items 8, 9 and 10 on the one hand, and the remaining items on the other, did not measure the same dimension (Per Martin Lof test, likelihood ratio = 74.5, #= 35, P = 0.00011). Furthermore, items 2 and 3 proved to be mutually dependent to an extent that made them unfit as a pair in the scale model. No test item dependence on diagnosis (rheumatoid arthritis or other) or on treatment (operated or not) was shown. Thus, the analysis discarded items 3, 8, 9, 10 and 13. This having been achieved, re-analysis was performed, and now the hypothesis of homogeneity of items could no longer be rejected as far as the remaining 10 items were concerned (likelihood ratio test: x2 = 15.6, df= 18, P = 0.62058). Likewise, the simple graphical analysis showed considerable improvement of the relations between the item characteristic curves (Fig. 2). items
Discussion The purpose of the present study was to develop a tool that was specifically suitable to evaluate pain in chronically affected knee-joints. Pain was considered as a latent trait and accordingly, a method related to the latent trait theory as known in modern psychometrics was adopted to construct a suitable scale. Thus only the construct validity of the scale was considered and tested by means of the Rasch model, which is one of the most widely accepted models that may serve this purpose I21. The proposed questionnaire. consisting of 15 items, had to be discarded, but a revised questionnaire consisting of 10 items yielded a scale that highty fitted the statistical model, which means that the number of positive responses could be used to rank the knees according to a latent trait related to pain. An analysis of the scale’s empirical validity remains necessary in order to examine whether the particular aspect of pain it deals with is clinically relevant. Moreover, reliability tests must be performed before a final appraisal of the scale can be made.
References I Mclver. J.P. and Carmines, E.G.. Unidimensional Scaling, Sage llniversity Paper series OD Quantitative Applications in the Social Sciences, 07-024, Beverly Hills, 1981, pp. 40-61. 2 Osterlind, S.J., Test Item Bias, Sage University Paper series on Quantitative Applications in the Social Sciences. 07-030, Beverly Hills, 1983. 3 Rasch, G.. An item analysis which takes individual differences into account. Brit. J. math. Stat. Psychoi., 19 (1966) 49-57. 4 Rasch, G., Probabilistic Modeis for Some Intelligence and AttaiIlnlent Tests, University of Chicago Press. Chicago, IL. 1980.