Interpretation of subjective measures in a clinical trial of hyperbaric oxygen therapy for multiple sclerosis

Journal of Psyhosomotrc Printed m Great Bntain.

Research,

Vol. 32. Nos. 4/S. pp. 365-372.

1988.

@X22-3999/8X $3.00 + .lXl 19X8 Pergamon Press plc

INTERPRETATION OF SUBJECTIVE MEASURES IN A CLINICAL TRIAL OF HYPERBARIC OXYGEN THERAPY FOR MULTIPLE SCLEROSIS JUDITH MONKS (Received 22 November

1987; accepted in revised form 8 March 1988)

Abstract-Inclusion of subjective measures in clinical trials has raised questions concerning the basis on which their results may be compared with medical findings. Central problems relate to the status of such measures as indicators of ‘real’ change and to appropriate methods of assessing their significance. These issues were explored in a controlled trial of hyperbaric oxygen therapy (HBO) for multiple sclerosis (MS). Subjective measures were incorporated on the basis of an explicit view of their status vis d vis medical assessments. The General Health Questionnaire (GHQ) and Nottingham Health Profile (NHP) were investigated with respect to their usefulness as indicators of the significance of reported benefit. While neither measure appeared sensitive to the kinds of significance involved, the NHP in particular might have a role in identifying orders of effectiveness which remain untapped by more commonly employed measures of perceived improvement.

INTRODUCTION REGARD to the social and personal contexts of medically prescribed therapy has brought increasing effort to incorporate patients’ self-assessments into the protocols of clinical trials. Problems associated with this practice centre on the definition of ‘objective’ and ‘subjective’ measures and the extent to which those designated as ‘subjective’ may be regarded as measuring ‘real’ change. A related problem concerns the relative weight given to objective and subjective change in balancing results in the overall conclusions of a trial, the (medical) significance of findings in the clinical domain being usually more fully understood than the social and personal significance of those relating to patients’ own assessments [l-3]. A clinical trial of hyperbaric oxygen therapy (HBO) in the management of multiple sclerosis (MS) provides a particularly suitable arena for investigating these issues. HBO therapy involves the breathing of oxygen under conditions of raised atmospheric pressure and has separately been claimed as beneficial in MS through immunological effects and through action on cerebral tissue made ischaemic by fat emboli [4]. A number of trials, both controlled and uncontrolled, have investigated these claims in an atmosphere of continuing controversy concerning the technical adequacy of the studies and interpretation of results. Reports have been inconsistent both in their designation of similar measures as ‘subjective’ or ‘objective’ and in related attributions of validity and significance [5-lo]. The medical debate has however proceeded alongside evident enthusiasm on the part of some of those with MS who have tried the therapy for themselves [ll-131. This paper reports how patients’ self-assessments were incorporated systematically into the protocol of a double-blind controlled trial of HBO for MS, and on

Brunei-ARMS Research Unit, Department of Human London, Uxbridge, Middlesex, UB8 3PH, U.K. 365

Sciences,

Brunei

The University

of West

366

JUDITH MONKS

the basis that they shared equal validity with medical findings.* After a description of the trial and principal results, a possible framework for assessing the significance of patients’ reported improvements is investigated, significance being defined in terms of psychosocial benefit as measured by the General Health Questionnaire and Nottingham Health Profile. METHOD The trial design The trial was of the double-blind controlled and crossover type in which all participants attended for treatment over a period of 16 weeks. All treatments were given in a multiplace chamber pressurized to 1.5 ATA, with oxygen (or air) being delivered via masks. Participants were allocated to either Regime 1 or Regime 2, the schedule for each regime being shown in Fig. 1. The decision that Regime 1 should continue with oxygen ‘top-ups’ during the third and fourth months was taken by the ARMS Research Committee and based on ethical considerations in relation to withholding treatment once commenced. Comparison between regimes after the first 2 months therefore had to take into account the lack of a wash-out period for Regime 1. It should be pointed out however that a wash-out period for HBO would be extremely difficult to determine since there is no effective way of measuring the presence of oxygen within the body and since the length of time over which the oxygen may have its effects is at present unknown. Following the 16 week trial period participants could, if they wished, arrange further HBO top-ups at one of the ARMS therapy centres. The conditions under which top-ups were given therefore varied quite considerably.

1 Treatment regime

Regime

Month

I

I

HBO 20 treatments

Month 2

Month

3

HE0 top-up weekly

Air 16 treatments

Month 4

HBO top-up weekly

+ HBO top-up 4 treatments Regime 2

Assessment point

Air 20 treatments

Air top-up weekly

I

HE0

20 treatments

HBO top-up weekly

2

FIG. l.-Plan

of double-blind

3

crossover

(4 at 6 month follow-up)

trial.

Participants Participants were selected from those who were currently attending an ARMS Research Unit for assessment and advice and from others who were actively seeking HBO treatment or who had volunteered to take part in the trial. All participants had a definite diagnosis of MS (according to the criteria of Rose et al. [14]) and Regimes 1 and 2 were matched both demographically and medically (Table I). Exclusion criteria included any history of relapse within 6 months of the start of the trial thus ensuring

*We see no a priori reason for polarizing measures as ‘subjective’ or ‘objective’ or for treating them as necessarily differing in validity. Our interpretation of results has proceeded on the basis that both clinical and self-assessments claim to have defined and measured their own particular objects of reference.

Interpretation TABLE

of subjective

measures

367

I.-DEMOGRAPHICANDSUBJECTIVEHEALTHDATAFOR REGIMES~AND 2 ON ENTRYTOTRIAL Regime 1 (n = 24)*

Regime 2 (n = 26)*

Male : female Mean age (range) Married : single : other Employed : not employed

7: 17 43.5 (26-69) 15:5:4 7: 17

8: 18 41.6 (24-65) 22:3:1 8: 18

Stability of MS ‘Improving steadily’ ‘Generally stable’ ‘Worsening steadily’

1 12 9

1 12 10

8 10 2

13 9 _

Health

data

Health today ‘Very good, good’ ‘Quite good’ ‘Quite poor’ ‘Poor, very poor’

1 @! = 21)

GHQ-12

(mean)

13.5 (n = 21)

NHP Part 1 (mean) Energy Pain Emotional reactions Sleep Social isolation Physical mobility

58.6(n 9.7(n 21.6(n 18.2(n 29.4 40.7

NHP Part 2 Health causing problems with work looking after the home social life home life sex life interests and hobbies

5 17 14 8 9 13

*Health data are based on populations owing to 5 participants failing to complete of treatment. “Tp < 0.04.

= = = =

12.9

21) 19) 19) 21)

54.1 14.8(n = 21) 18.1 (n = 22) 19.9 14.2t 41.3

2(n = 20) 15 12 Z(, 15

= 22)

of 22 (Regime 1) and 23 (Regime 2) health questionnaires before the start

that disease activity during the trial period would be in the direction of deterioration, spontaneous improvement without relapse not being a recognized feature of the disease. Of 57 participants selected, four did not complete the full number of sessions and two were judged to have been in relapse throughout the trial. Related data were excluded from our analyses. Data were also missing for one participant who did not wish to complete the questionnaires on which this report is based. Results presented here therefore relate to the 50 remaining participants who completed the 4 months of treatment. Data collection Medical assessments were carried out prior to the commencement of the trial and then at intervals for the 4 months of the trial period, and again 6 months later. For full details the referred to Worthington et al. [15]. Our own research aimed to describe and analyse self-assessments of both general health specific effects of HBO, together with the social and practical consequences of pursuing the

monthly reader is and the therapy.

368

JCJDITH MONKS

Three complementary techniques were employed - self-completed questionnaires, self-completed diaries and semi-structured interviews - although this report presents the analysis of questionnaire data only. Questionnaires were distributed at the time of the medical assessments relating to points 1, 2, 3, and 4 shown in Fig. 1. All contained a common core of questions, phrased appropriately according to the current stage of the study. These questions covered feelings and expectations in relation to HBO, practical and other consequences of undergoing treatment, and subjective health measures. The initial and final questionnaires also asked for demographic data. Subjective health measures 1. Self-assessed effects of HBO. Each questionnaire included our own instrument designed to assess participants’ estimations of the effects of HBO on their MS. A series of 5-point scales were employed on which effects were recorded in relation to MS symptoms as specified by each respondent, and also in relation to respondents’ ‘general MS’. This allowed investigation of subjective change on a wider basis than the more usually employed checklist of symptoms. In each case the effects of HBO in ‘curing’. ‘improving’, ‘stabilizing’, ‘being ineffective’, and ‘worsening’ were investigated (Fig. 2). For analysis, a series of 3-point scales was created by combining the 2 points at either extreme of each of the original 5-point scales. This eliminated any effect resulting from respondents’ avoidance of the extreme points, and also increased the number of responses in all but the middle category. The chi-squared one-sample test was used to assess the significance of distributions of responses at particular time points. Also included in each questionnaire were 2 complementary general health measures both requiring responses to a series of questions or statements. 2. The General Health Questionnaire (GHQ) [ 161. The GHQ is a self-administered screening instrument designed to detect minor psychiatric disorder in populations drawn from ‘community settings’. The version used in this study was the GHQ-12 which requires response to 12 questions covering aspects of mental functioning and emotional wellbeing. This is the shortest version of the GHQ and contains no questions relating to somatic symptoms (such as tremor) which may be experienced by people with multiple sclerosis. Its validation is described by J. De Jesus Mari and P. Williams [17]. The GHQ has been used previously with people with MS [18, 191 and one of these studies [19] reports a satisfactory level of test retest reliability. We would like HE0 treatment.

to

know

i Please what

circle on each you feel NOW).

Some of your

what

scale

effects

below

you have experienced

the

number

MS symptoms:

which

from your

comes

closest

Def mutely has

to

Definitely hasn’t

Cured some (which?

1 Improved

some IwhIch

Stablllzed

Stopped

some (which?

new symptoms

Been ineffective

Worsened

FIG. 2.-Scale

12,1,0,1,2,

1

2

I

0

I

1

2,

I ,o,

I

2

~

developing

,2,

I

,O,l

,2 ,2,

,2,1,0,1,2, L2,

I ,o,

used for recording participants’ impressions of treatment on their MS symptoms.

I

,2

of the effects

Interpretation

of subjective measures

369

3. The Nottingham Health Profile (NHP) [20, 211. The NHP aims to measure both perceived health problems and the extent to which these problems affect areas of daily life. It was designed as a self-administered questionnaire and is said to be suitable for both survey and clinical work. While it has not yet been employed widely in the clinical context and its use in a controlled trial has not been previously reported [22], the authors hope that its employment in this area will increase [23]. The Profile has two parts. Part 1 presents 38 statements selected to be representative of those commonly used to describe the effects of ill health. Each statement (which requires a ‘yes’ or ‘no’ response) is weighted to reflect its severity relative to others in the relevant dimension of the Profile. In Part 2 respondents are asked to indicate whether a number of specified activities are affected by their health, and again this requires a ‘yes’ or ‘no’ response. In Part 2 each response is treated as an independent variable. Following the advice of the authors, statistical comparisons between related samples were made using the Wilcoxon matched pairs signed rank test for Part 1, and the McNemar test for Part 2. For independent samples, the Mann-Whitney U-test was used for Part 1 and the chi-squared test for Part 2. RESULTS AND DISCUSSION

Principal results relating to the efficacy of HBO

Neither results of medical [15] nor our own assessments indicated any large benefit related specifically to HBO. This held both for the duration of the trial and the following 6 months during which some participants continued to receive treatment. Our own results relating to reported improvement in MS symptoms are shown in Tables II and III. However, in the total study population at point 3 significant majorities felt their MS symptoms to have improved (24/38, p c O.OO),although not to have been cured (19/33, p < O.OOl), and not to have become worse (19/34, p < 0.02). Six months later at point 4 the majority still reported improved symptoms (18/29, p < 0.004), improvements at point 3 apparently being more often maintained to the time of the next assessment than those reported at point 2. In respect of their ‘general MS’ at neither point 3 nor point 4 did a significant majority of either regime or the total population indicate HBO to have been effective. Deterioration was reported by 2 from each group at point 3 (for Regime

TABLE

WHOREPORTED

II.-PROPORTIONOFREGIME~ANDREGIME~MEMBERS IMPROVEMENTS

IN

MS

SYMPTOMS

AT

POINTS

2 AND 3

Point 2 (after 2 months) Symptoms

Point 3 (after 4 months)

Regime 1 (n = 20)

Regime 2 (n = 19)

Regime 1 (n = 17)

Regime 2 (n = 21)

Mobility-related Urinary system-related Other/unspecified

4 (20%) 1 (5%) 6 (30%)

2 1 7

(11%) (5%) (37%)

5 2 5

4 5 4

Indicated at least one improvement

8

(40%)

8

(42%)

4

(20%)

1

(5%)

Improvement (29%) (12%) (29%)

10 (59%)

(19%) (24%) (19%)

14 (67%)

Deterioration Indicated at least one deterioration

3

(18%)

3

(14%)

370

JUDITH

.

MONKS

TABLE III. -PROPORTION OF THOSE HAVING AND THOSE NOT HAVING FIB0 TOP-UPS BETWEEN POINTS 3 AND 4, WHO REPORTED IMPROVEMENTS IN MS SYMPTOMSAT POINT 4 Point 4 (after 10 months) No

Symptoms

Top-ups (n = 19)

Top-ups (n = 10)

4 3 8

1 (10%) 4 (40%) 4 (40%)

Improvement Mobility-related Urinary system-related Other/unspecified Indicated at least one improvement

(21%) (16%) (42%)

12 (63%)

6

(60%)

Deterioration Indicated at least one deterioration

2

(11%)

-

1, n = 15 and for Regime 2, n = 18), and from one member of Regime 1 and 3 members of Regime 2 at point 4 (n = 11 in both cases). It is possible that HBO had a less dramatic effect than could be judged within a period of 2 months, or that there was a high degree of variability in response or a large placebo effect, either of which could have masked HBO’s particular action. Whether or not however these explanations account for the apparent ineffectiveness of HBO, a significant majority of trial participants did report improvement in symptoms which they themselves attributed to the therapy. The remainder of this paper addresses the question of how the significance of these improvements (to those experiencing them) may be assessed, and particularly to the usefulness of the GHQ and NHP in this respect. The GHQ and NHP as indicators of the significance of reported improvement GHQ and NHP scores and reported symptom improvement. GHQ and NHP

scores varied little over the course of the study and there was no evidence of any effect related specifically to HBO. For the total study population significant beneficial changes occurred over points l-4 in GHQ scores @ < 0.035), and in NHP Part 1 scores on ‘energy’ 0, < 0.0085) and ‘social isolation’ (p < 0.05). Over points l-3 significantly fewer participants felt their health to be causing problems with ‘social life’ (NHP Part 2) and over points 3-4 a negative change was found in the NHP Part 1 ‘sleep’ scores. Score changes did not however reflect reported improvements in symptoms (chi-squared or Fisher’s exact test). We might conclude that perceived symptom improvements were too trivial to affect self-assessments of overall health status [24]. However the significance of reported improvement may instead have lain outside the parameters of both the NHP and GHQ, or alternatively NHP and GHQ score changes may reflect benefit lying outside the sphere of perceived symptom improvement, while still being derived from participation in the trial. Investigation of these possibilities required

Interpretation

of subjective measures

371

an independent indicator of the personal significance of perceived symptom improvement, using which the strength of the association of each measure might be tested. For reasons described below, the decision whether to continue with HBO top-ups following the trial was selected as such an indicator. Reported symptom improvement continue with HBO top-ups

and GHQ and NHP scores, and the decision to

Other data from our questionnaires suggested that perceived benefit from HBO played an important role in the decision to continue with therapy after the end of the trial [25]. Since therapy is costly to those involved in terms of both time and money, it seemed reasonable to select the top-up decision as an indicator of perceived significance in symptom improvement. Unsurprisingly, a significant relationship was found between reported symptom improvement and opting for top-ups (n = 34, p < 0.03). In contrast, GHQ and most NHP scores showed no relationship, but an improvement in ‘social isolation’ scores did prove significant (n = 34, p < 0.008). As however symptom improvement and ‘social isolation’ scores were not themselves associated (this holding both for those having and not having top-ups) it seems that whatever personal significance symptom improvement carried for participants it differed from that reflected by the NHP. This proposition was tested by examining the separate relationships of symptom improvement and ‘social isolation’ scores with the top-up decision, controlling for the relevant third variable in each case. For those who reported improvement in symptoms, there remained a significant relationship between improvement in ‘social isolation’ scores and opting for top-ups (n = 19, p < 0.04). Only six participants returning relevant valid data had improved their ‘social isolation’ scores by the end of the trial and since all chose to have top-ups it was not possible to test the relationship of this decision to symptom improvement. However 4 of the 6 reported such improvement (and opted to have top-ups). For those whose ‘social isolation’ scores were the same at the start and finish of the trial, the relationship between symptom improvement and choosing top-ups just missed significance at p < 0.08 (n = 16). Across the remaining categories numbers were too small to draw conclusions. In general then the data suggested that symptom improvement and ‘social isolation’ scores were independent of each other in their relationship to taking top-ups, supporting the proposition that the measures related to different phenomena with different significances for the participants. CONCLUSION

This paper began by drawing attention to problems associated with self-assessment in clinical trials and particularly to the related questions of how self-reports and medical findings may be balanced when results do not coincide, and how the significance of self-reports may be systematically measured. A trial of HBO for MS was described in which a range of subjective measures indicated perceived benefit (although not related to HBOperse) without concurrent medical corroboration. That this perceived benefit was an important factor in the decision to continue with therapy seemed to indicate that it carried personal signifi-

372

JUDITH MONKS

cance. Two validated measures, of psychological health (GHQ) and general wellbeing (NHP), were investigated with respect to their potential usefulness in indicating the degree of this significance but neither appeared sensitive to the particular kinds of significance involved. There was evidence however that particularly the NHP might be helpful in uncovering orders of effectiveness of therapy which remained untapped by the more commonly employed measures of perceived symptom change. Acknowledgements-The study on which this paper is based was a product of collaboration between colleagues at every stage. Thanks is due to those who have helped with data preparation, computing, administration and secretarial tasks, and also of course to the trial participants. In addition I am grateful to colleagues for their valuable comments on earlier drafts of this paper. The study was supported by ‘Action for Research into Multiple Sclerosis’. REFERENCES 1. 2. 3. 4. 5. 6. 7. 8.

9. 10. 11. 12. 13. 14. 15.

16. 17.

18. 19. 20. 21. 22. 23. 24. 25.

ROBINSON I. Analysing the structure of 23 clinical trials in multiple sclerosis. Neuroepidemiol 1987; 6: 46-76. CAY EL, PHILIP AE, SMALL WP, NEILSON J, HENDERSON MA. Patients’ assessment of the result of surgery for peptic ulcer. The Lancer 1975; l(7897): 29-31. GREER S. The psychological dimension in cancer treatment. Sot Sci Med 1984; 18(4): 345-349. BATES D. Hyperbaric oxygen in multiple sclerosis: discussion paper. JI Sot Med 1986; 79: 535-537. WILES CM, CLARKE CRA, IRWIN HP, EDGAR EF, SWAN AV. Hyperbaric oxygen in multiple sclerosis: a double-blind trial. Br Med J 1986: 292: 367-371. FISCHER BH, MARKS M, REICH T. Hyperbaric oxygen treatment of multiple sclerosis: a randomized, placebo-controlled, double-blind study. New Eng[, / Med 1983: 308(4): 181-186. DAVIDSON DLW. Hyperbaric oxygen treatment for multiple sclerosis. The Practifioner 1984; 288: 903-905, BARNES MP, BATES D, CARTLIDGE NEF, FRENCH JM, SHAW DA. Hyperbaric oxygen and multiple sclerosis: short-term results of a placebo-controlled, double-blind trial. The Lancer 1985; ix: 297-300. WOOD J, STELL R, UNSWORTH I, LANCE JW, SKUSE N. A double-blind trial of hyperbaric oxygen in the treatment of multiple sclerosis. Med J Austral 1985: 143: 238-239. ROSEN JA. Hyperbaric oxygenation does not improve chronic progressive multiple sclerosis. Ann Neural 1985; 17: 615. HAVRON D. Fighting MS to standstill: the evidence. Med Trib 1983; 24(4): 1. LEWIS EM. Treating an MS sufferer in a recompression chamber. Occupational Health 1984; 36(2): 5C60. MONTAGUE K. John’s story. Nursing Mirror 1985; 161(15): 40. ROSE AS, ELLISON GW, MYERS LW. TOURTELLO~E WW. Criteria for the clinical diagnosis of multiple sclerosis. Neurology 1976; 26: 20-22. WORTHINGTON JA, DE SOUZA LH, FORTI A, JONES R. MODARRES-SADEGHI H, BLANEY A. A double-blind controlled cross-over trial investigating the efficacy of hyperbaric oxygen in patients with multiple sclerosis. In Multiple Sclerosis Immunological, Diagnostic and Therapeutic Aspecrs (Edited by ROSE FC and JONES R). London: John Libbey, 1987. GOLDBERG D. Manual of the General Health Questionnaire. NFER-Nelson. 1978. DE JESUS MARI J, WILLIAMS P. A comparison of the validity of two psychiatric screening questionnaires (GHQ-12 and SRQ-20) in Brazil, using relative operating characteristic (ROT) analysis. Psycho1 Med 1985; 15: 651459. RABINS PV, BROOKS BR. Emotional disturbances in multiple sclerosis patients: validity of the General Health Questionnaire (GHQ). Psycho1 Med 1981; 11: 4255427. DE PAULO JR. F~LSTEIN MF. Psychiatric disturbances in neurological patients: detection, recognition and hospital course. Ann Neural 1978; 4(3): 225-228. HUNT SM, MCKENNA SP, MCEWEN J. The Nottingham HealthProfile: Manual, 1982 (mimeo). MCEWEN J. The Nottingham Health Profile: A Measure of Perceived Health. In Measuring the Social Benefits of Medicirte (Edited by TEELING SMITH G). Office of Health Economics, 1983. HUNT SM. Personal communication, 1987. HUNT SM, MCEWEN J,MCKENNA SP.Measuring Health Status. London: Croom Helm. 1986. HUNT SM, MCEWEN J, MCKENNA SP, BACKETT EM, POPE C. Subjective health assessments and the perceived outcome of minor surgery. J Psychosom Res 1984; 28(2): 105-114. WYNNE A. (Forthcoming).