- Email: [email protected]
On the analysis of repeated measurements of the same subjects
Letters
1217
to the Editor
THE LETIER of RAGLAND and COGTILOEexpresses views which are quite close to my own in some respects, and their disagreement with me arises from a misunderstanding. They say of rxy only when the ‘null hypothesis is true’ (when rxy =O). that rxiy p-y is independent Their addition in brackets is not a necessary interpretation of the phrase in inverted commas. Whatever value rxy has, rX+y,y_-x is zero when the null hypothesis is true because s2y will then equal s: . GARSIDE differs from me and from RAGLAND and COSTILOE in thinking that in addition I’~). must be unity (not zero) on the null hypothesis, but I do not think he would disagree that on the null hypothesis the standard deviations must remain unaltered by treatment. criticisms of my treatment of the data of This point really covers your correspondents’ TURPEINEN et 01. There can be no question that these authors did claim that their rrn of -0.355 was caused by treatment. though it was a secondary conclusion which should not be taken as seriously damaging their paper. They wrote: “The work reported here shows that the replacement of whole milk by soyhean-oil emulsion in skim milk, and that of butter by a special type of margarine, results in a statistically highly significant, though not very large, decrease in the mean serum-cholesterol value. The practical significance of this change is somewhat increased by the fact that the subjects with higher initial serum-cholesterol levels RAGLAND and COSTKOE suggest that the meaning of also tend to show greater decreases.” this finding must remain obscure because there was no control group. In fact my proposed analysis led to a conclusion which is nicely confirmed by the control data they quote, in which the correlation of initial value and change deviates from -0.355 to a quite insignificant degree (t=0.53,
P i7.0.6).
The method of approach suggested by R.4GLAND and COSTILOE is based on their assumption that nothing can safely be learnt from a follow-up study unless a control group is available. This seems a counsel of despair. There are still difficulties of definition that need to be cleared up, in particular of what constitutes a ‘differential effect of treatment’, and I would like to make another attempt to make my position clear. 1. The relationship of change to initial value reflects differential effects of treatment, but it also depends on the correlation of initial and final values in the absence of treatment. Thus as a single index it is virtually useless, and potentially misleading. It certainly describes something real, but it is almost impossible to interpret. ?i. A differential effect of treatment may be distinguished from a consistent effect of treatment by the fact that the standard deviation of initial values will then differ from that of the final values. The three diagrams illustrate this. All three show final values plotted against initial values. forming correlation ellipses. The broken line is the regression of x? on ~1. In Fig. 1 there arc
FIG. 1. First and second observations on the same subjects plotted other; case one : no changes but random ones. ~I=TZ.
against each
1218
Letters
to the Editor
no changes but random ones; the major axis of the ellipse lies along the line through the origin at 4S’, and a square can be drawn to contain the ellipse-the standard deviations of XI and xz are equal. In Fig. 2 a consistent effect of treatment is present; the ellipse is displaced
XI
FIG. 2. First and second observations on the same subjects plotted against each other; case two: second observation less than tirst, on average equally for all subjects. (T~=Q. bodily downwards, but its major axis (dotted line) is still at 45’. The ellipse can still be contained within a square-the standard deviations of _rr and A-Zare still equal. In Fig. 3 a differential effect of treatment has occurred, so that the ellipse is both displaced and rotated. standard Its major axis no longer lies at 45’, and a rectangle is needed to contain it-the deviation of the final values is smaller than that of the initial values.
xi FIG. 3. First and second observations other; case three: second observation large than when it is small. ‘TI> ~2.
on the same subjects plotted against each reduced more when first observation is
3. In view of the central importance of the slope of the major axis of the correlation ellipse, a change of variables to the sum (or mean) and difference of XI and ~2 (the new axes are shown in the diagrams) is a convenient way of exploring the situation. The new variables are easy to interpret, and are independent of each other unless a differential effect of treatment is present. If it is present, knowledge of the errors and correlation of xr and x2 may be needed
Letters
to the Editor
1219
This will seldom be the case; usually the fact if some estimate of its magnitude is required. of its existence will be all that is needed before investigation of its cause is begun. P. D. OLDHAM
Medical Research Council, Pneumoconiosis Llandough Hospital, Penarth, Glamorgan, Wales.
Research Unit,
IN THEIR letter RAGLAND and COSTILOE say that I do not show any awareness that naturally occurring fluctuation reduces the correlation between initial and final biological measurements (rxy) and thus that between initial measurements and changes (rXD). In the first sentence Of my second letter, however, I mentioned the “effect of treatment or of the passage of time” ln relation to rXD. It is thus clear that I was aware of this issue. I did not pursue this matter in my previous letters because I was more concerned to show and that, when corrected for this bias, it does that rXn was biased by errors of measurement provide a valid index of the relationship of initial values with effect of treatment or with the passage of time or both. If an investigator wishes to know whether changes after treatment is a are related to initial values, then rxD (corrected if necessary for errors of measurement) perfectly valid index. It should, in this situation, not be corrected for the effects of natural If it is desired fluctuation, because treated people will fluctuate naturally as well as untreated. to predict true changes in relation to initial value, then the appropriate regression coefficient (brjx) must be corrected for bias due to errors of measurement [ll. of initial values to the effect of treatment or to the Whether rxn is due to the relationship effect of natural variation is another matter. To decide this question it will usually be necessary to use a control group, as RAGLAND and C~TILOE suggest. But if rxy is negative then it may sometimes be concluded that the effect of treatment is related to initial measurements If, for example, after treatment (such as E.C.T.) more severely without using a control group. depressed patients get better quickly whereas more mildly depressed patients do not, then it seems reasonable to conclude that treatment has a differential effect because such a situation is unlikely to occur naturally. It was this kind of situation which I used in the example in data. not mv second letter. OLDHAM objected that a negative rxy indicates heterogeneous differential treatment effects. I agree that the data may be heterogeneous, but neverthelessperhaps because the data are heterogeneous-differential treatment effects are indicated. between initial scores and treatment The fact that a negative rXy can indicate a relationship is. I think, relevant to OLDHAM’S obiection in his second letter that r,, has “no direct bearing 0’1 the assessment of the treatment effects.” A positive rXy, by itielf, cannot indicate the differential effect of treatment. But if, as RAGLAND and COSTKOE infer, it can be shown that rv,, in a control and treated group differ, then it must be concluded that treatment has a differential effect. The two values of rTn may differ because of a difference between the two values of ryy, or because of a difference in the ratios of my to cy in the two groups. or both.
Thus the magnitude of r9,., even when positive, is not irrelevant to the assessment of treatment effects. as OLDHAM claims in his second letter. It is worth noting that ryy may equal zero as a result of spontaneous remission. Consider. for example. the temperatures of a group of influenza patients during and after their illness. There is no reason why their temperatures when ill should be related to their temperatures when better. This point seems to have been overlooked by RAGLAND and COSTILOE who state that “one expects rather high positive correlations between repeated measures.” They also overlook the fact that such natural spontaneous remission will also reduce the standard deviation of the temperatures. Successful treatment might also produce a zero correlation, whereas before treatment the correlation between temperatures on two different occasions would tend to be positive. This again suggests that the magnitude of rxY may have a “direct bearing
on the assessment
of treatment
effects.”
which is denied by OLDHAM.
While drawing attention to the fact that ryY may be naturally less than unity, RAGLAND and COSTILOE state that this “and not errors of measurement, produce the negative relationshins between initial scores and change scores. This may be seen by setting the reliabilities equal to one in G.~RsIDE’s first equation . . . ” But if the reliabilities are equal to one. then the
measurements are exact and there are no errors of measurement. So RAGLAND and‘cosr~~o~ are saying that if there are no errors, then these errors do not reduce rYD. This seems to be a peculiar way of reasoning. It is quite obvious from my (or rather THOMSON’S) formula that
1217
to the Editor
THE LETIER of RAGLAND and COGTILOEexpresses views which are quite close to my own in some respects, and their disagreement with me arises from a misunderstanding. They say of rxy only when the ‘null hypothesis is true’ (when rxy =O). that rxiy p-y is independent Their addition in brackets is not a necessary interpretation of the phrase in inverted commas. Whatever value rxy has, rX+y,y_-x is zero when the null hypothesis is true because s2y will then equal s: . GARSIDE differs from me and from RAGLAND and COSTILOE in thinking that in addition I’~). must be unity (not zero) on the null hypothesis, but I do not think he would disagree that on the null hypothesis the standard deviations must remain unaltered by treatment. criticisms of my treatment of the data of This point really covers your correspondents’ TURPEINEN et 01. There can be no question that these authors did claim that their rrn of -0.355 was caused by treatment. though it was a secondary conclusion which should not be taken as seriously damaging their paper. They wrote: “The work reported here shows that the replacement of whole milk by soyhean-oil emulsion in skim milk, and that of butter by a special type of margarine, results in a statistically highly significant, though not very large, decrease in the mean serum-cholesterol value. The practical significance of this change is somewhat increased by the fact that the subjects with higher initial serum-cholesterol levels RAGLAND and COSTKOE suggest that the meaning of also tend to show greater decreases.” this finding must remain obscure because there was no control group. In fact my proposed analysis led to a conclusion which is nicely confirmed by the control data they quote, in which the correlation of initial value and change deviates from -0.355 to a quite insignificant degree (t=0.53,
P i7.0.6).
The method of approach suggested by R.4GLAND and COSTILOE is based on their assumption that nothing can safely be learnt from a follow-up study unless a control group is available. This seems a counsel of despair. There are still difficulties of definition that need to be cleared up, in particular of what constitutes a ‘differential effect of treatment’, and I would like to make another attempt to make my position clear. 1. The relationship of change to initial value reflects differential effects of treatment, but it also depends on the correlation of initial and final values in the absence of treatment. Thus as a single index it is virtually useless, and potentially misleading. It certainly describes something real, but it is almost impossible to interpret. ?i. A differential effect of treatment may be distinguished from a consistent effect of treatment by the fact that the standard deviation of initial values will then differ from that of the final values. The three diagrams illustrate this. All three show final values plotted against initial values. forming correlation ellipses. The broken line is the regression of x? on ~1. In Fig. 1 there arc
FIG. 1. First and second observations on the same subjects plotted other; case one : no changes but random ones. ~I=TZ.
against each
1218
Letters
to the Editor
no changes but random ones; the major axis of the ellipse lies along the line through the origin at 4S’, and a square can be drawn to contain the ellipse-the standard deviations of XI and xz are equal. In Fig. 2 a consistent effect of treatment is present; the ellipse is displaced
XI
FIG. 2. First and second observations on the same subjects plotted against each other; case two: second observation less than tirst, on average equally for all subjects. (T~=Q. bodily downwards, but its major axis (dotted line) is still at 45’. The ellipse can still be contained within a square-the standard deviations of _rr and A-Zare still equal. In Fig. 3 a differential effect of treatment has occurred, so that the ellipse is both displaced and rotated. standard Its major axis no longer lies at 45’, and a rectangle is needed to contain it-the deviation of the final values is smaller than that of the initial values.
xi FIG. 3. First and second observations other; case three: second observation large than when it is small. ‘TI> ~2.
on the same subjects plotted against each reduced more when first observation is
3. In view of the central importance of the slope of the major axis of the correlation ellipse, a change of variables to the sum (or mean) and difference of XI and ~2 (the new axes are shown in the diagrams) is a convenient way of exploring the situation. The new variables are easy to interpret, and are independent of each other unless a differential effect of treatment is present. If it is present, knowledge of the errors and correlation of xr and x2 may be needed
Letters
to the Editor
1219
This will seldom be the case; usually the fact if some estimate of its magnitude is required. of its existence will be all that is needed before investigation of its cause is begun. P. D. OLDHAM
Medical Research Council, Pneumoconiosis Llandough Hospital, Penarth, Glamorgan, Wales.
Research Unit,
IN THEIR letter RAGLAND and COSTILOE say that I do not show any awareness that naturally occurring fluctuation reduces the correlation between initial and final biological measurements (rxy) and thus that between initial measurements and changes (rXD). In the first sentence Of my second letter, however, I mentioned the “effect of treatment or of the passage of time” ln relation to rXD. It is thus clear that I was aware of this issue. I did not pursue this matter in my previous letters because I was more concerned to show and that, when corrected for this bias, it does that rXn was biased by errors of measurement provide a valid index of the relationship of initial values with effect of treatment or with the passage of time or both. If an investigator wishes to know whether changes after treatment is a are related to initial values, then rxD (corrected if necessary for errors of measurement) perfectly valid index. It should, in this situation, not be corrected for the effects of natural If it is desired fluctuation, because treated people will fluctuate naturally as well as untreated. to predict true changes in relation to initial value, then the appropriate regression coefficient (brjx) must be corrected for bias due to errors of measurement [ll. of initial values to the effect of treatment or to the Whether rxn is due to the relationship effect of natural variation is another matter. To decide this question it will usually be necessary to use a control group, as RAGLAND and C~TILOE suggest. But if rxy is negative then it may sometimes be concluded that the effect of treatment is related to initial measurements If, for example, after treatment (such as E.C.T.) more severely without using a control group. depressed patients get better quickly whereas more mildly depressed patients do not, then it seems reasonable to conclude that treatment has a differential effect because such a situation is unlikely to occur naturally. It was this kind of situation which I used in the example in data. not mv second letter. OLDHAM objected that a negative rxy indicates heterogeneous differential treatment effects. I agree that the data may be heterogeneous, but neverthelessperhaps because the data are heterogeneous-differential treatment effects are indicated. between initial scores and treatment The fact that a negative rXy can indicate a relationship is. I think, relevant to OLDHAM’S obiection in his second letter that r,, has “no direct bearing 0’1 the assessment of the treatment effects.” A positive rXy, by itielf, cannot indicate the differential effect of treatment. But if, as RAGLAND and COSTKOE infer, it can be shown that rv,, in a control and treated group differ, then it must be concluded that treatment has a differential effect. The two values of rTn may differ because of a difference between the two values of ryy, or because of a difference in the ratios of my to cy in the two groups. or both.
Thus the magnitude of r9,., even when positive, is not irrelevant to the assessment of treatment effects. as OLDHAM claims in his second letter. It is worth noting that ryy may equal zero as a result of spontaneous remission. Consider. for example. the temperatures of a group of influenza patients during and after their illness. There is no reason why their temperatures when ill should be related to their temperatures when better. This point seems to have been overlooked by RAGLAND and COSTILOE who state that “one expects rather high positive correlations between repeated measures.” They also overlook the fact that such natural spontaneous remission will also reduce the standard deviation of the temperatures. Successful treatment might also produce a zero correlation, whereas before treatment the correlation between temperatures on two different occasions would tend to be positive. This again suggests that the magnitude of rxY may have a “direct bearing
on the assessment
of treatment
effects.”
which is denied by OLDHAM.
While drawing attention to the fact that ryY may be naturally less than unity, RAGLAND and COSTILOE state that this “and not errors of measurement, produce the negative relationshins between initial scores and change scores. This may be seen by setting the reliabilities equal to one in G.~RsIDE’s first equation . . . ” But if the reliabilities are equal to one. then the
measurements are exact and there are no errors of measurement. So RAGLAND and‘cosr~~o~ are saying that if there are no errors, then these errors do not reduce rYD. This seems to be a peculiar way of reasoning. It is quite obvious from my (or rather THOMSON’S) formula that