An alternative to interference indexes in neuropsychological time-sharing research

Neuropsycholoy~a,Vol. 25, No. 4. pp. 719-724. 1987. Prmted in Great Britain.

0028%3932/87 $3.00+ 0.00 8~~ 1987 Pergamon Journ& Ltd.

NOTE AN ALTERNATIVE TO INTERFERENCE INDEXES IN NEUROPSYCHOLOGICAL TIME-SHARING RESEARCH* W. GRANT Wn,Llst University

of Colorado

and

LAURA

at Denver,

D.

GOODWIN

Colorado.

U.S.A.

(Received 25 July 1986; accepted 24 February 1987) Abstract-Interference indexes are used as criterion measures in a significant proportion of timesharing paradigms in order to control for the initial discrepancy between left- and right-handed tapping rates. Problems associated with these indexes include low reliability, low statistical power, and especially correlations with baseline tapping rates. An analysis of covariance procedure using raw-score concurrent tapping rates as criterion measures, and baseline tapping rates as covariates, can effectively address these problems as demonstrated through a reanalysis of data.

INTRODUCTION IN TIME-Sharing (or dual-task) paradigms, individuals are required to perform two tasks concurrently [22]. During the past decade, time-sharing paradigms have been used to test hypotheses about functional cerebral organization (particularly with reference to relative lateralization of functions), and to investigate possible regional cerebral specializations for perceptual, cognitive, and motor behaviors 18, 171. When used for these purposes, a major assumption is that the degree of interference of one task on another unrelated, but concurrently performed, task is inversely correlated with the functional distance between the cerebral foci associated with the processing of the two tasks [16]. In a typical investigation of this nature, for example, a task, x, is performed concurrently with left- then right-handed finger tapping. Left- and right-handed finger tapping are selected because their cerebral foci have been relatively localized to the contralateral hemisphere [3,9]; however, cf. [27]. Consequently, if performance of x interferes relatively more with right- than left-handed tapping, it is likely that performance of x is associated with relatively greater left- than right-hemispheric neuronal activation. Similarly, greater left- than right-handed interference is likely to be associated with greater right- than left-hemispheric activation. Demonstrations of these lateralized interference effects are consequently highly dependent on interhand comparisons during concurrent-task performance. A potential problem with analyzing raw scores generated through these paradigms is that they may be insensitive to lateralized interference effects. This is because research participants are usually right-handed and interhand comparisons of initial (i.e., baseline) tapping rates typically favor the right. Given this initial discrepancy between the hands, differential interference effects associated with the concurrent performance of an unrelated task may be due to initial differences in tapping speed rather than lateralization effects. In this respect, interference might be greater for right- than left-handed tapping because, due to the higher range of initial values for the right hand, there is a higher possible range for reduction.

THE INITIAL-VALUES

PROBLEM

This interhand discrepancy in initial tapping rate has been labeled the “initial-values problem” [l l] and has been addressed in at least three different ways. First, statistically reliable hand-by-task interactions, where right-handed

*We gratefully acknowledge Jani Little and the Institute of Behavioral Science, University Boulder, for valuable computer assistance. tCorrespondence to be addressed to: Dr W. Grant Willis, who is now at the Department University of Rhode Island, Kingston, RI 02881, U.S.A. 719

of Colorado of Psychology,

at

NATE

720

tapping is faster during baseline conditions but left-handed tapping is faster during concurrent-task conditions 1231, cannot be attributed to the initial-values problem. This kind of interaction clearly suggests lateralized interference effects. Moreover, the investigator is unable to control for the latter possibility. Second, some investigators have required each participant to perform in concurrent-task conditions that theoretically are associated with differential relative hemispheric neuronal activation. For example, nonverbal tasks such as humming 1231 or spatial reasoning 1281 theoretically rely to a lesser extent on left-hemispheric activation than more verbal tasks such as reciting a nursery rhyme or a list of animal names 112, 131. Dealing with the initialvalues problem in this fashion is somewhat analogous to the experimental method of double dissociation [25] because it permits investigators to determine if a particular effect (e.g., a lateralized manual interference) results from a specific concurrent task or, instead, results from a more genera1 factor shared by a variety of tasks. The argument is also somewhat tautological, however, because lateralization effects are presumed at the outset yet these lateralization effects are essentially the subject of investigation. Finally, “interference indexes” (originally introduced by HISCOCK and KINSBOURNE1121) have been calculated as criteria that are putatively orthogonal to initial tapping rates. These indexes have rapidly become popular criterion measures in neuropsychological time-sharing research paradigms. For example, of 42 research reports describing this kind of paradigm (identified by a manual search ofjournals published within the past decade), 17 (i.e., 40.5%) used interference indexes as criterion measures.*

INTERFERENCE The interference

index is a proportionate

decrease

INDEXES

or reduction

in tapping

rate. The basic equation

is:

R, -R,

(1)

Yi =R,

where: X, = an individual’s interference index; R, = tapping rate for either right or left hand in the control (baseline) condition; R, = tapping rate for the same hand in a concurrent-task condition. Arguments favoring the use of Equation I, rather than the use of R, alone, center on the concern that R, alone does not account for hand differences in initial tapping rates. This is particularly important because left-right hand differences are one of the major foci of neuropsychological time-sharing investigations. The numerator of the interference index, R, - R,, is a change (or “gain”) score. As other statisticians have pointed out [ 1, 2. 19, 203, change scores have three statistical shortcomings that limit their utility: (a) low reliability, (b) low power when used in inferential statistical tests, and (c)correlations with the initial scores. Although the denominator ofthe interference index renders it mathematically different from the ordinary difference score, these three problems are not “corrected” or removed by dividing the numerator by R,. It is not known exactly what statistical effects the division of (R, -R,) by R, has on reliability, power, and correlations with initial scores. However, the effects due just to the numerator are serious enough to warrant a brief description here.? Low reliahilify With R, and R, representing two times of measurement, the reliability of their difference, D, is a function of the reliability of R,, the reliability of R, and the correlation between R, and R, [see 1, 19, 201. The equation for obtaining the reliability of D (which assumes that the measurement error in R, is independent of the measurement error in R,) is:

where: p,, = reliability of R,; p,,=reliability of R2;p12= correlation between R, and R,; 6, =standard R,; a2 =standard deviation of R,. If p1 1=p22=p. and u, = cr2, Equation 2 is reduced to:

deviation

of

(3) As can be seen readily in Equation 3, a high correlation between R, and R, is associated with a low reliability estimate for D. For example, if the reliability estimates for both R, and R, are 0.90, and their correlation is 0.60, the

*The following journals were searched from January 1975 to July 1985: Acta Psycholoyia, American Journal of Occupational Therapy, Brain and Cognition, Brain and Language, Child Development, Cortex, Developmental Psychology, Journal of Comparative and Physiological Psychology, Journal of Motor Behavior, Neuropsychologia. tWe are grateful to an anonymous reviewer for encouraging gain score and the interference index, and that the denominator effects) of the denominator deserve methodological study.

us to point out this difference between an ordinary cannot be viewed as irrelevant. Clearly, the role (and

NOTE

721

reliability estimate for D is 0.75; if their correlation is 0.80, the reliability estimate is only 0.50. In either case, the reliability estimate for D is less than the reliability estimate for either R, or R, alone, because the measurement errors that are present in R, and in R, are amplified when the two variables are combined. The fact that the reliability of D is weakened by higher correlations between R, and R, presents a dilemma, because low values for that correlation are atypical for time-sharing paradigms. Low reliability associated with interference indexes is illustrated by data reported by HISCIXK and KINSBOLJRNE [13]. Test-retest (i.e., stability) reliability coefficients, based on a one-year interval, were calculated for both rawscore tapping rates and for interference indexes. The data consisted of six different measures: left- and right-handed baseline tapping and left- and right-handed tapping during each of two concurrent vocalization conditions. The six reliability coefficients for raw scores ranged from 0.57 to 0.70. The four reliability coefficients for interference indexes ranged from 0.25 to 0.34. The median difference between these two kinds of reliability estimates (i.e., raw scores and interference indexes) for the four concurrent-task conditions was 0.35. Clearly, reduced reliability resulting when differences are involved in calculating criterion measures attenuates the utility of the data. Moreover, relatively low reliability adversely affects the precision and power of subsequent statistical analyses. Low statistical

power

When interference indexes are used in inferential statistical tests, the power of these tests may be adversely affected. This is partly due to the lowered reliability of the interference indexes, but is also related to the size of the linear regression coefficient @I)of R, on R,.COX'S[S] suggestion that an ANOVA on difference scores is preferable to an ANOVA on raw post test scores only when 0 < (l//I) < 2 is important here because interference indexes involve calculating difference scores as numerators and concurrent-tapping rates are analogous to posttest scores. Nonzero correlations Difference scores, (D),usually correlate with initial scores, R, [l, 261. In the case of interference indexes, the correlation, RID, usually will be positive, indicating that high initial values for tapping rates (e.g., right-handed baseline tapping rates) are associated with high levels of proportionate reduction in tapping rates during concurrent-task conditions. In contrast, low initial values for tapping rates (e.g., left-handed baseline tapping rates) are associated with low levels of concurrent-task proportionate reductions. The magnitude of RID is largely an artifact of the sharing of the same measurement errors in initial values and interference indexes. Given that one of the most prevalent arguments for using interference indexes instead of raw-score tapping rates is that interference indexes are putatively independent ofinitial (baseline) tapping rates, the nonzero correlations that usually occur between these two measures are troublesome. For example, HISC~CK and KINSBOURNE [13] conducted regression analyses to express interference indexes as a function of baseline tapping rates after three other predictors had been partialled out of the equations. Even so, baseline tapping rates accounted for 2.2 to 5.6% of the variance in interference indexes for the left and right hands. In an earlier study, HISCOCK and KINSBOURNE [12] concluded that, for the left hand under one of two concurrent-task conditions, baseline tapping rate accounted for 2% of the interference-index variance; for the left hand under the other condition, and for the right hand under both conditions, the variance accounted for by baseline tapping rate was nonsignificant (no actual values were given). In a study conducted by WILLIS and HYND [29], simple correlations between baseline tapping rates and four criterion measures (all expressed as interference indexes) ranged from 0.32 to 0.51 for the left hand and 0.23 to 0.41 for the right hand. Regression analyses, similar to those conducted by HISCOCK and KINSBOURNE[13] were used to determine the influence of baseline tapping rates on the interference indexes after two other predictors were partialled out ofequations; baseline tapping rates accounted for 8 to 18% of the variance in left-handed interference indexes and 5 to 17% of the variance in right-handed indexes. Further, the increments in R,,when the baseline variables were included in the regression analyses (after the other predictors had already been entered into the equations) were statistically significant at the P-co.05 level for seven of the eight analyses and significant at the PiO.10 level for the eighth. The results of these correlational analyses clearly illustrate that, contrary to their intended purpose, interference indexes are not always empirically orthogonal to baseline tapping rates.

ANCOVA AS AN ALTERNATIVE

PROCEDURE

An alternative procedure that has greater potential to address the initial-values problem more effectively is analysis of covariance (ANCOVA). Essentially, this analysis is conducted using raw-score tapping rates during concurrent-task conditions as criterion measures, and baseline tapping rates as covariates. Levels of the criterion measure, that is, left- versus right-handed concurrent tapping rates, are regressed on their own (i.e., left- vs righthanded, respectively) baseline tapping rates. Thus, left-handed baseline tapping rate is the covariate for all lefthanded concurrent tapping measures, and right-handed baseline tapping rate is the covariate for all right-handed concurrent tapping measures. This kind of analysis addresses the initial-values problems by statistically controlling

722

NOTE

for the effects of initial hand differences in tapping rate on subsequent, concurrent tapping rate. There are several important assumptions that must be met to use ANCOVA appropriately, such as linearity and homogeneity of regression coefficients [IO, 15, 171. Further, its effectiveness depends very heavily on the size of the correlation between the covariate and the criterion measure (i.e., R, and R,); the higher the correlation, the better the adjustment [IO].

Data generated through a prior neuropsychological time-sharing investigation 1291, originally analyzed with ANOVA on interference indexes, were reanalyzed with the alternative ANCOVA procedure. (Preliminary tests indicated that all ANCOVA assumptions were met.) Approximately equal numbers of F ratios increased and decreased from the original analysis to the reanalysis. Comparing the two analyses, all statistically significant effects (P <0.05) in the original analysis remained significant in the reanalysis with the exception of hand; similarly, all nonsignificant effects (P>O.O5) in the original analysis remained nonsignificant in the reanalysis. The difference in the main effect of hand (Fs = 14.81 and 0.09 for the ANOVA and ANCOVA, respectively) is particularly striking. Although this effect was statistically significant at the P
An important additional advantage of the suggested ANCOVA procedure is that it atlords Increased power, especially in true-experimental designs [ 10, 15. IS]. The increase in power is achieved via an elimination of a portion oferror variance,due to the correlation between the covariate and thecriterion measure. That is, the denominator of the F ratio in ANCOVA (MS, ) is smaller than the MS, in ANOVA by a factor that depends on the correlation between the covariate and criterion measure: MS,. = MS,( I -rz). where r, is the within-groups correlation between the covariate. X, and the criterion measure, Y. To illustrate the approximate amount of reduction in the error mean square that resulted by using ANCOVA on concurrent tapping rates (instead of ANOVA on interference indexes), the median of the eight correlation coefficients between R, and R,, 0.71. was used to calculate the approximate reduction in the error mean square: 1-0.71’=0.50. Thus, the expected MS,. from the ANCOVA was only half of the MS, from the ANOVA. In the reanalysis presented here. the increased power generally did not alter major conclusions drawn about the significant interactions present. This may be due to the large number of factors in the design. which accounted for much of the variance in the criterion measure and left relatively little error variance to be reduced. The reduction in MS, afforded by ANCOVA, however, could have important substantive implications for conclusions in neuropsychological time-sharing studies that have fewer sources of variation (and are thus less powerful initially).

SUMMARY A major problem with ustng raw-score concurrent tapping rates as criterion measures in neuropsychological time-sharing research is that there are initial interhand differences in these measures. This initial-values problem has been addressed in a significant proportion ofstudies by analyzing interference indexes. in which difference scores arc calculated as numerators of criterion measures. Limitations associated with the use of interference indexes arc similar to those encountered with the use of difference scores: (a) low reliability and (b) low power when used in inferential statistical tests. A more serious problem with interference indexes ts that, contrary to their Intended purpose, they are not always empirically orthogonal to initial tapping rates. A reanalysis of interference indexes and a critical examination of prior reports demonstrated that stattstically reliable correlations may be present between these two variables cvcn after other factors considered in the design of the studies have been partialled out of multiple regression equations.

NOTE

123

In contrast, ANCOVA on concurrent tapping rates, with initial tapping rates used as covariates, represents a viable alternative approach to addressing the initial-values problem. Such a statistical control for initial interhand differences is associated with equal interference potentials for the two hands. Moreover, additional advantages of the alternative analytical procedure are increased reliability of criterion measures and increased power of inferential analyses.

REFERENCES 1. BEREITER,C. Some persisting dilemmas in the measurement of change. In Problems in Measuring Changes, C. W. HARRIS (Editor), pp. 3-20. University of Wisconsin Press, Madison, 1963. 2. BOCK, R: D. Mult&ariate analysis of iariance of repeated measurements. In Problems in Measuring Change, C. W. HARRIS (Editor). DD. 85-103. University of Wisconsin Press, Madison, 1963. 3. BRINKMAN, j. and %PERS, H. G. J. M. Splitbrain monkeys: cerebral control of ipsilateral and contralateral arm, hand, and finger movements. Science 176, 536539, 1972. 4. COOK, T. D. and CAMPBELL, D. T. Quasi-Experimentation: Design and Analysis Issuesfor Field Settings. Rand McNally, Chicago, 1979. 5. Cox, D. R. The use of a concomitant variable in selecting an experimental design. Biometrika 44,15&l 58.1957. 6. CRONBACH, L. J. and others. Analysis of Covariance in Nonrandomized Experiments: Parameters Aflecting Bias (Occasional Paper). Stanford Evaluation Consortium, Stanford, CA, 1977. 7. FELDT, L. S. A comparison of the precision of three experimental designs employing a concomitant variable. Psychometrika 23, 335-353, 1958. 8. FRIEDMAN, A. and POLSON, M. C. Hemispheres as independent resource systems: limited-capacity processing and cerebral specialization. J. exp. Psychol. Hum. Percept. Perform. 7, 1031-1058, 1981. 9. GAZZANIGA, M. S. The Bisected Brain. Appleton-Century-Crofts, New York, 1970. 10. GLASS, G. V. and HOPKINS, K. D. Statistical Methods in Education and Psychology (2nd edn). Prentice-Hall,

Englewood Cliffs, NJ, 1984. 11. HISCOCK, M. Verbal-manual time sharing in children as a function of task priority. Brain Cognit. 1, 119~131, 1982. 12. HISCOCK, M. and KINSBOURNE,M. Ontogeny of cerebral dominance: evidence from time-sharing asymmetry in children. Dev[ Psychol. 14, 321-329, 1978. 13. HISCOCK, M. and KINSBOURNE, M. Asymmetry of verbal-manual time sharing in children: a follow-up study. Neuropsychologia 18, 151-162, 1980. 14. HOPKINS, K. D. Preventing the number one misinterpretation ofbehavioral research, or how to increase power. J. Special Educ. 7, 103-107, 1973. 15. KEPPEL, G. Design and Analysis: A Researcher’s Handbook (2nd edn). Prentice-Hall, Englewood Cliffs, NJ, 1982. 16. KINSBOURNE, M. and HICKS, R. E. Functional cerebral space: a model for overflow, transfer and interference effects in human performance: a tutorial review. In Attention and Performance VII, J. REQUIN (Editor), pp. 345-362. Lawrence Erlbaum, Hillsdale, NJ, 1978. 17. KINSBOURNE, M. and HICKS, M. Asymmetries of dual-task performance. In Cerebral Hemisphere Asymmetry, J. B. HELLIGE (Editor), pp. 255-334. Praeger, New York, 1983. 18. KIRK, R. E. Experimental Design for the Behavioral Sciences (2nd edn). Brooks-Cole, Belmont, CA, 1982. 19. LINN, R. L. and SLINDE, J. A. Determination of the significance of change between pre- and post testing periods. Rev. Educ. Re. 47, 121-150, 1977. 20. LORD, F. M. Elementary models for measuring change. In Problems in Measuring Change, C. W. HARRIS (Editor), pp. 21-38. University of Wisconsin Press, Madison, 1963. 21. MCFARLAND, K. and ASHTON, R. A developmental study of the influence of cognitive activity on an ongoing manual task. Acta psychol. 39,447456, 1975. 22. NAVON, D. and GOPHER, D. On theeconomy ofthe human-processing system. Psycho/. Rev. 86,214225,1979. 23. PIAZZA, D. M. Cerebral lateralization in young children as measured by dichotic listening and finger tapping tasks. Neuropsychologia 15,417425, 1977. 24. REICHHARDT, C. S. The statistical analysis of data from nonequivalent group designs. In QuasiExperimentation: Design and Analysis Issues,for Field Settings, T. D. COOK and D. T. CAMPBELL (Editors), pp. 147-205. Rand McNally, Chicago, 1979. 25. TEUBER, H. L. Some alterations in behavior after cerebral lesions in man. In Evolurion of Nervous Control fromPrimitive Organisms to Man, A. D. BASS (Editor), p. 187. American Association for the Advancement of Science, Washington, D.C., 1959. 26. THORNDIKE, E. c The influence of change imperfections of measures upon the relation of initial score to gain or loss. J. exp. Psychol. 7, 225-232, 1924. 27. TROPE, I., FISHMAN, B., SUSSMAN,N. M. and CUR, R. E. Contralateral and ipsilateral control of fingers following callostomy. In Proceedings of the 13th Annual Meeting of the International Neuropsychological Society, p. 42, 1985.

724

NOTE

28. WHITE, N. and KINSBOURNE,M. Does speech output control lateralize over time? Evidence from verbal-manual time-sharing tasks. Brain Lang. 10, 2155223, 1980. 29. WILLIS, W. G. and HYND, G. W. Lateralized interference effects: evidence for a processing style by modality interaction. Brain Cognit. 6, 112-126, 1987.