- Email: [email protected]
Microcomputer version of a digit span test in clinical use
interacting
with Computers
uol 4 no 2 (1992) 163-178
Microcomputer version of a digit span test in clinical use Christopher
C. French and J. Graham Beaumont*
At four clinical sites, 188 psychiatric patients were assessed with both the standard auditory verbal digit span test and a computerised visual digit span test requiring manual response. Correlations between the two test forms were comparable with other similar studies. Subjects obtained higher scores on the standard version than the computerized version, probably reflecting differences in the mode of presentation and the mode of response, but this difference was much less pronounced in those subjects receiving the standard version first rather than the computerized version first. It is clear that the computerized test cannot be considered to be psychometrically parallel to the standard WAIS subtest version, but it may still be of some clinical value. Keywords:
Computerized
testing, digit span, clinical applications.
The field of computerized
(or automated) assessment has developed rapidly in recent years (see e.g. Bartram and Bayliss, 1984; Beaumont, 1981a; Burke and Normand, 1987; Butcher et al., 1985; French, 1985, 1986, 1990; Hedlund and Vieweg, 1988; Sampson, 1983; Thompson and Wilson, 1982). The relative advantages and disadvantages of computerized compared to standard assessment, although beyond the scope of the current article, have been widely discussed and the interested reader is referred to the above articles for more detail. This paper will focus specifically upon the comparability of the standard version of the digit span task and a computerized version as used in the Leicester/DHSS project. The Leicester/DHSS project on microcomputer-aided assessment took place in two phases. In Phase 1, the hardware and software system of an inexpensive microcomputer (the Apple II+) was evaluated with respect to its suitability for administering a range of psychometric tests, software was prepared to administer a number of such tests, and limited pilot investigation of the psychometric performance of this software was undertaken Department of Psychology, Goldsmiths’ College, University 6QE, England. Tel: 081-692 7171 Ext. 2075. *University College Swansea, Swansea, Wales. 0953-5438/92/020163-16
0
1992 Butteworth-Heinemann
of London, New Cross, London SE14
Ltd
163
(Beaumont, 1981b). Results were sufficiently encouraging for Phase 2 to be undertaken. A general survey of the findings of Phase 2 of the Leicester/DHSS project is presented by Beaumont and French (1987). This was a large scale study of microcomputer-aided assessment involving the administration of eight psychological tests to 367 patients at five clinical sites. As stated, it is the purpose of this paper to focus in more detail upon the results relating to the automated version of a digit span test used in that study. Digit span tasks are probably the single more commonly used means of assessing short-term memory and it is likely that as clinicians make ever greater use of microcomputers in their day-to-day work, computerized versions of the procedure will become commonplace. The question of comparability between standard and computerized forms is therefore of considerable importance. Beaumont (1981~) presented a description of the digit span test used in the project. Various types of digit span test are in existence, but this version is largely based on the subtest of the Wechsler Adult Intelligence Scale and 1945, 1955; Saville, 1971). One major Wechsler Memory Scale (Wechsler, difference is that in the computerized version digits are presented visually rather than auditorily as in the traditional version, but at an equivalent rate. The program measures forward span, backward span and total span. Sequence length increases by one for every correct response and two error responses conclude that section of the test. The maximum span which can be measured is 30. A second major difference between the computerized digit span test and the standard version is in terms of subjects’ response mode. In the standard version, subjects simply repeat back the digit strings orally. In the computerized version, responses can be input either by keying in the sequence from the keyboard or touching the appropriate areas of a touch-sensitive screen. The sequence entered is shown on the screen, and the response can either be started over again, or the last digit deleted, by pressing a single key or touching the appropriate screen area. Finally, subjects press a key or touch the appropriate screen area to indicate completion of a response. A full record of performance is available, but a password must be entered before results are presented. In line with other automated versions of digit span tests (e.g. Wilson et al., 1982; Wilson, 1987), the current system presents different randomly generated strings of digits to the subject on each occasion of testing. This is a major advantage of automated digit span assessment over the standard WAIS version, allowing for frequent re-use of the test on the same individual, if required. In certain situations, e.g. psychopharmacological studies of the effects of different drugs on memory, it is necessary to make such assessments repeatedly over a relatively short period potentially confounding drug effects with practice effects which are known to occur (e.g. Hebb, 1961). It should be noted, however, that practice effects would be expected in the early stages on tasks such as this even if the digit strings are varied, but such effects will be minimized if different digit strings are employed. However, no system can fully control for the effects of strings which may have personal significance for the subject, for example corresponding in whole or in 164
Interacting
with Computers
uol 4 no 2 (1992)
part with the subject’s telephone number, address, date of birth and so on. Once again, however, the automated versions are superior to the standard WAIS version in that any such distortions will not be constant across repeated assessments because different strings will be generated on each occasion. The fact remains that the standard WAIS version of the digit span test is the most commonly used and it is therefore of some interest to examine the relationship between performance on the two versions of the test in a clinical setting. As patients in this study were only tested once on the standard version of the test and once on the computerized version, the disadvantages of the former with respect to repeated testing will have no bearing on the results obtained. Any differences found between standard and computerized presentations will reflect differences in the method of presentation and response rather than differences due to the composition of the strings themselves. It is worth noting here that Beaumont (1985; Experiment 2) has presented results showing that microcomputer presentation of the digit span test produced significantly poorer digit spans than the standard auditory-verbal administration when administered to groups of undergraduate students. Beaumont compared the performance of subjects in the following conditions: 0
0 l
vocal presentation of stimuli with vocal response (i.e., standard digit span as described above); visual presentation of stimuli with responses input to computer (i.e., computerized form described above); visual presentation of stimuli with vocal response (i.e., computerized presentation, but standard response mode).
In this way he showed that the higher scores on the standard compared to the computerized version were due to both the use of computer-based response devices rather than vocal responding and the presentation of stimulus material in the visual rather than the auditory modality. Penney and Blackwood (1989) report two experiments in which serial recall of lists of digits was investigated with subjects responding either by writing their responses or entering them into a computer by means of a keyboard. Keyboard response mode reduced recall from the final serial positions of the lists. This reduction in the recency effect was much more pronounced for visual presentation of stimuli than auditory presentation. These investigators raise the possibility that allocation of mental resources to the unpracticed keyboard response diverts resources away from the maintenance of list items in some kind of rehearsal loop or response buffer. The general superiority of the auditory modality over the visual modality in terms of memory for verbal material is a well-established effect in psychological research. Penney (1989) has suggested that verbal information presented to the auditory and visual modalities is processed in separate streams that have different properties and capabilities. In particular, she claims that recall of auditory stimuli will be better with successive presentation of items, whereas recall of visual items will favour simultaneous presentation of items. Her theory has obvious relevance to the issue of computerization of digit span tasks. French and Beaumont
165
This paper presents data relating to the forward span, backward span and total span, as all three measures are commonly cited in the clinical literature. It should be noted, however, that there is considerable disagreement regarding the value of combining the forward and backward spans in a single measure rather than considering each separately. For example, Rude1 and Denckla (1974) collected data from 297 children with learning disabilities. They claimed that subjects presumed to have left cerebral hemisphere damage were impaired in terms of forward span, whereas subjects presumed to have right hemisphere damage were impaired in terms of backward span. Based upon the known functional properties of the cerebral hemispheres, they suggested that the forward span was based largely upon an auditory verbal component whereas the backward span task requires subjects to form a visual image of the auditorily presented digits and to use visual scanning mechanisms to produce the reversed sequence. Gardner (1981) has argued that although the total span may be useful in a general test of intelligence, consideration of forward and backward spans separately can provide important additional insights. To this end, he has provided normative data from 1567 school children for forward and backward spans separately. Other studies which are consistent with the notion that forward span is largely dependent upon verbal processing whereas backward span involves visuospatial processing include Banken (1985)‘ Black (19831, Costa (1975), and Weinberg et al. (1972). However, other investigators have produced data which are not consistent with this explanation. For example, Richardson (1977) found no correlation between backward span and a ‘spatial span’ score although forward and backward digit span were highly correlated. He concluded that both digit span measures employ a common verbal representation. Newcombe (1969) found that left hemisphere damage led to impaired performance on both forward and backward spans, whereas right hemisphere damage did not impair either. Other studies which have failed to support the claim that visuospatial processing is an important component of the backward span task include Black (1986), Black and Strub (1978) and Gupta et al. (1986). It is not even clear whether forward span or backward span is more sensitive to brain damage. Costa (1975) found no difference between brain-damaged patients and normal controls for forward span, but brain-damage was associated with impaired backward span. Black (1986) produced data generally in line with the claim that backward span was a more sensitive indicator of brain damage. Griffin and Heffernan (1983) found that total span and backward span differed across five ability levels in a group of intellectually subnormal subjects, but that forward span did not. In contrast, Black and Strub (1978) claimed that forward span was a more sensitive measure of brain dysfunction. Kear-Colwell (1977) concluded that there is no point in providing separate scores for forward and backward spans, as they provide little unique information. Finally, Sterne (1969) was unable to demonstrate any differences at all between brain-damaged patients and non-brain-damaged controls. Thus, although many studies indicate that backward span is more impaired by brain damage than forward span, a substantial number of studies do not support this. 166
Interacting
wifh Computers
~014 no 2 (1992)
It is worth noting here that Schofield and Ashman (1986) present data supporting the serial-processing character of forward digit span, in line with the position adopted by other investigators (e.g. Kaufman and Kaufman, 1983). Backward span, on the other hand, involved not just sequential processing, but was also closely related to planning abilities.
Method Design This paper focuses on data from patients who were tested on one version of the test and retested on the other. All sites were instructed to allocate patients randomly to: 0
0
either the standard or computerized version of the test on first testing (with the alternate test form employed for retest); either the keyboard or touch-screen as the response mode for computerized testing.
The decision as to whether a particular patient was to take part in the project or not was left solely in the hands of the clinicians involved, but was based upon the normal clinical appropriateness of the test procedures for that patient. Site teams were issued with specific and detailed instructions with regard to the allocation of subjects to conditions, but examination of the number of subjects in each condition reveals that these procedures were not always followedi. There appears to have been some reluctance to run patients on the touch-screen version of the test, despite the fact that the patients themselves seemed to find the test as acceptable as the keyboard version (see French and Beaumont, 1987). The general conclusions with regard to the comparability of the standard versus the computerized test would not be affected by this departure from protocol, but previously published conclusions with regard to the comparability of the keyboard and touch-screen versions (Beaumont and French, 1987) should be treated with some caution.
Assessment system The sites involved in the project were provided with a minimum configuration of Apple II+ microcomputer with single disc drive, Language Card, communications interface card, and a Mountain Hardware clock. The display device was a Portatel colour monitor to which was fitted a capacitative touch-sensitive screen (Interaction Systems Inc.), thus allowing either the standard keyboard or the touch-sensitive screen to be used to input responses. Discussion of the ‘We are grateful to an anonymous
reviewer for pointing out this departure from the intended experimental protocol. It should be noted that the random allocation procedures appear to have been followed properly when the Eysenck Personality Questionnaire (French and Beaumont, 1989), the Money Road Maps Test (Beaumont and French, 1987), the Mill Hill Vocabulary Test and Standard Progressive Matrices Test (French and Beaumont, 1990) were administered as part of this project, but departures from true random allocation are also apparent in the data for the DAT subtests (French and Beaumont, 1991). French and Beaumont
167
principles 1982).
adopted in designing
this system can be found in Beaumont
(1981d;
Patients and clinical sites The mean age of the subjects (age data available for 176 of 188 subjects) was 37.31 years (standard deviation = 13.45 years), ranging from 18 to 76 years. The group was composed of 115 males and 73 females. With respect to marital status, 108 were single, 46 married, 20 divorced or separated, 11 widowed and three were not classified. With respect to neurological status, 158 were neurologically sound and 30 were definitely or possibly neurologically impaired. The diagnostic composition of the group was: l l
0 l
0 l l
affective psychosis (23.4%) schizophrenia (25.0%) neurosis (14.9%) personality disorder (16.5%) mental disorders not specified as psychotic tions (4.3%) special symptoms not elsewhere classified mild mental retardation (5.9%)
associated
with physical
condi-
(4.3%)
These diagnoses account for 94.3% of the group. The sites at which the system was located for clinical evaluation were two large general psychiatric hospitals (Psychiatric Hospitals A and B, respectively), one of which had a relatively high proportion of long-stay chronic patients; a district general hospital; and a high security Special Hospital for mentally disordered offenders. The number of patients tested at each site was as follows: l l l l
Psychiatric Hospital A Psychiatric Hospital B District General Hospital Special Hospital
36 57 45 50
Procedure The standard version of the test was administered according to the instructions in the WAIS manual. The computerized version was designed to be as similar to the standard version as possible, except for the mode of stimulus presentation and response. Tests were usually, though not necessarily, administered by the same investigator on each occasion. Following the completion of the test, the subject was required to complete a Patient Response Questionnaire. The results of this questionnaire have been reported elsewhere (French and Beaumont, 1987). As already noted, the data collected for this study were part of a much larger project (Beaumont and French, 1987). For most of the other tests investigated it was desirable that, whenever possible, site teams allowed at least three days to elapse between testing and retesting. The average test-retest interval was much longer for all tests, including the digit span (M = 36.62 days, SD = 58.70, 168
Interacting
with Computers
ml
4 no 2 (1992)
Table 1. Mean scores, standard deviations and reliabilities for all subjects for forward span Test condition All retest data:
Keyboard
only:
Touch-screen
only
Computer first (n=97) Standard first (n=91) Overall (n=lSS) Computer first (n=60) Standard first (n=57) Overall (n=117) Computer first (rl=37) Standard first (n=34) Overall (n=71)
maximum = 374 days). With test-retest intervals were not of collecting data as part of ensure a stringent test of the span test.
M SD M SD M SD M SD M SD M SD M SD M SD M SD
Standard
Computer
6.15 1.41 6.15 1.40 6.15 1.40 6.15 1.38 5.98 1.47 6.07 1.42 6.16 1.48 6.44 1.23 6.30 1.37
4.93 1.51 5.32 1.38 5.12 1.46 5.00 1.63 5.33 1.39 5.16 1.52 4.81 1.31 5.29 1.38 5.04 1.36
Y 0.505 p
0.630 p
0.424 p=O.OOl 0.744 p
0.436 p=O.OlO 0.571 p
respect to the digit span assessments, such large actually necessary, and occurred simply as a result a larger investigation. However, they obviously relationship between the two versions of the digit
Results Tables 1,2 and 3 present mean scores, standard deviations and reliabilities for forward, backward and total span respectively for all subjects. The reliability of the Digit Span test in the WAIS manual was estimated by correlating Digits Forward with Digits Backward and was found to be moderate (between 0.60 and 0.69; Wechsler, 1955; Saville, 1971). As described in the introduction, there is good reason to believe that the forward and backward spans might involve different cognitive functions, and the wisdom of combining them into a single total score is open to question. By the same token, it would seem to be unwise to attempt to assess the ‘reliability’ of the digit span task by correlating the two spans, as reported in the original WAIS manual. Correlation coefficients in this study were calculated by correlating the standard score with the computer score for each of the three measures. They too were moderate, yielding values of 0.557 for the forward span, 0.708 for the backward span and 0.732 for the total span (all p
169
Table 2. Mean scores, standard deviations and reliabilities for all subjects for backward span Test condition
All retest data:
Computer first (n=97) Standard first (n=91) Overall (n=188)
Keyboard
only
Touch-screen
only:
Computer first (n=60) Standard first (n=57) Overail (n=117) Computer first (n=37) Standard first (n=34) Overall (n=71)
M SD M SD M SD M SD M SD M SD M SD M SD M SD
Standard
Computer
r
4.53 1.47 4.45 1.40 4.49 1.44 4.60 1.46 4.35 1.47 4.48 1.47 4.41 1.50 4.62 1.28 4.51 1.39
4.07 1.61 4.54 1.57 4.30 1.60 4.17 1.62 4.53 1.73 4.34 1.68 3.92 1.61 4.59 1.28 4.23 1.49
0.766 p
0.690 p
0.697 p
0.614
p
0.731 p
Table 3. Mean scores, standard deviations and reliabiiities for all subjects for total span Test condition All retest data:
Keyboard
only:
Touch-screen
170
only:
Computer first (n=97) Standard first (n=91) Overall (n=188) Computer first (n=60) Standard first (n=57) Overall (n=117) Computer first (n=37) Standard first (n=34) Overall (n=71)
M SD M SD M SD M SD M SD M SD M SD M SD M SD
Standard
Computer
Y
10.68 2.61 10.60 2.51 10.64 2.56 10.75 2.55 10.33 2.68 10.55 2.61 10.57 2.73 11.06 2.17 10.80 2.48
9.00 2.77 9.86 2.61 9.41 2.72 9.17 2.84 9.86 2.82 9.50 2.84 8.73 2.65 9.85 2.27 9.27 2.52
0.719 p
Interacting
with Computers
p-eo.001
0.732 p
0.718 p
0.807 p
0.720 p
0.772 p
no 2 (1992)
Table 4. Mean spans on standard and computerized tests for possibly or definitely neurologically impaired group (NI?) versus neurologically sound (NS) group
Standard spans: Forward Backward Total Computerized Forward Backward Total
NI? mean (n = 30)
NS mean
t
P
5.43 3.97 9.40
6.29 4.59 10.88
3.15 2.20 2.97
4.63 3.87 8.50
5.21 4.38 9.59
2.00 1.61 2.03
co.05 not significant co.05
(n = 158)
spans:
Before carrying out any analyses involving sex as a between-subjects factor, it was felt that it was appropriate to exclude data from two partially overlapping subgroups: subjects who were definitely or possibly neurologically impaired, and subjects from the Special Hospital. The reason for this is that both of these groups would be expected to perform poorly on the digit span tasks, and this expectation was confirmed by exploratory data analysis using t-tests (see Tables 4 and 5). The single exception was that there was no significant difference between the neurologically suspect patients and the neurologically sound patients with respect to the computerized backward span. With respect to sex, 22 out of 115 male subjects (19%) were possibly or definitely neurologically impaired, whereas only eight out of 73 female subjects (11%) fell into this category. Furthermore, all of the Special Hospital subjects were male. These subgroups were therefore excluded in order to avoid confounding neurological status and clinical site with sex. Data for the remaining 113 subjects for whom age and sex data were available were submitted to analysis of variance, with age level (above versus below
Table 5. Mean spans on standard and computerized tests for Special Hospital Group (SHG) versus others
Standard spans: Forward Backward Total Computerized spans: Forward Backward Total French and Beaumont
SHG (n = 50)
Others (n = 138)
5.28 3.86 9.14
6.47 4.72 11.19
5.55 3.74 5.18
4.68 3.76 8.44
5.28 4.49 9.77
2.51 2.82 3.02
co.05 co.01 co.01
P
171
Table 6. Means scores on standard and computerized digit spans (excluding neurolo~cally impaired and Special Hospital patients) for patients receiving standard version first (n = 56) and computer version first (n = 57) Computer
Standard Forward span Standard 1st Computer 1st Backward span Standard 1st Computer 1st Total span Standard 1st Computer 1st
6.61 6.46
5.77 4.96
4.84 4.74
4.93 4.16
11.45 11.19
10.70 9.12
versus touch-screen, and standard-first versus median), sex, keyboard computer-first as between-subject variables, and standard score versus computer score as a within-subjects factor. Separate analyses of variance were carried out for forward span, backward span and total span. It was found that subjects had significantly shorter forward spans on the computer (M = 5.36) than on the standard version (M = 6.53; F(1, 97) = 66.03, p<.OOl). This effect was also found for the backward span (standard M = 4.79; computer M = 4.54; F(l, 97) = 4.60, p<.O5) and for the total span (standard M = 11.32; computer M = 9.90; F(l, 97) = 50.01, p<.OOl). A significant interaction of the factors standard-first versus computer-first x standard score versus computer score was found for forward span (F(l, 97) = 4.95, p<.O5), backward span (F(l,97) = 5.13,p<.O5),and total span (F(l,97) = 8.86, p<.Ol). In each case, the group receiving the computer version first showed a greater difference between the computer and standard scores than did the group receiving the standard version first (see Table 6). One other significant effect was found, but only in the analysis of forward span data, where the keyboard versus touch-screen and standard score versus computer score factors appeared to interact significantly at a just-significant level (F(1, 97) = 3.97, ~7=.049). This interaction reflects the fact that while the keyboard and touch-screen groups obtained very similar scores on the computerized test (keyboard M = 5.37; touch-screen M = 5.35)‘ they differed with regard to ther standard forward spans (keyboard group M = 6.33; touch-screen group M = 6.90). It is likely that this interaction is due to the selection bias of the clinical site teams noted earlier. It appears that the teams were reluctant to allocate psychiatric patients to the touch-screen condition presumably because they felt that it was too complicated for general use (only 40 subjects out of 113 were allocated to the touch-screen condition). They appear to have been willing to allocate only relatively bright patients to this condition, thus explaining the difference in standard forward spans. While this underlines the problems of reconciling appropriate experimental design with clinical concern for patients, it should be noted that the general conclusions regarding the comparison between the standard and computerized versions still stand. 172
~nfe~ucfing
with
Conzputers
vol 4 no 2 11992)
Discussion Reliabilities on this test were moderate for all three span measures, although they were of the same order of magnitude as correlations between forward and backward span on the standard version of the test and test-retest correlations over similarly long periods for the standard version (e.g., Moore et al., 1990). It is instructive to compare these results with those of Wilson et al. (1982) and Wilson (1987) who compared scores on standard and computerized versions of a digit span test using severely physically disabled adults as subjects. Wilson et al.‘s (1982) test differed from both the original test and the current implementation in that only the digits l-6 were used and only the forward digit span was measured. Subjects were retested after a minimum interval of two weeks. Wilson’s (1987) version, known as PADS (Putney Automated Digit Span), uses the digits 1-9 and continues until the subject makes five errors at a particular string length. It also incorporates the partial scoring scheme described by Longstreth and Madigan (1982), in which the subject’s score for each part of the test is the length of the longest string correctly recalled plus a partial score from the performance of the last five incorrectly recalled trials. In this study, each subject took part in two sessions (separated by an average of three weeks), but received both the computer and the standard version for each session. In general, this study and those carried out by Wilson et al. (1982) and Wilson (1987) show good overall agreement with respect to the pattern of correlations (with correlations ranging from 0.48 to 0.86 across conditions in the studies by Wilson and colleagues). This suggests that the long test-retest intervals used in the present study have not unduly distorted the correlations between the standard and computerized digit span tests and that a similar pattern of results might have been obtained even if a much shorter interval had been used. We can conclude that the current system performs at least as well as other similar systems in terms of standard computer correlations. Minor differences between the studies are likely to reflect the fact that Wilson’s studies used physically disabled patients (on a specially designed assessment system) and also that Wilson’s (1987) correlations are based upon same-day retesting. This study found that patients who were definitely or possibly neurologically impaired performed significantly less well on all measures, with the single exception of the computerized backward span, and offers no support for the claim that backward span is a more sensitive measure of neurological impairment. It should be borne in mind, however, that patients were classified simply on the basis of the clinicians’ response to the single question, “Is the patient definitely or possibly neurologically impaired?“, which was asked mainly in order to allow the screening out of such patients from the main analysis, if required. Patients at the Special Hospital also showed impaired performance compared to the rest of the group. In view of the findings of Beaumont (1985), and the general findings of modality effects in short-term memory (Penney, 1989), it came as no surprise that the computerized version of the digit span produced lower scores than the standard version. Beaumont had shown that this was due to both the use of computer-based response devices and the presentation of stimuli in the visual French
and Beaumont
173
rather than the auditory modality. Similar findings are reported by Wilson et al. (1982) and Wilson (1987). A recent study of a HyperCard-based digit-span task by Martin and Allan (1991) should be mentioned here. The computerized version of the digit-span task used by these investigators presented the digit sequences by highlighting the keys on a numeric display on the computer screen while simultaneously presenting the stimuli in a digitized female voice. The first experiment reported by these investigators indicated that there was no difference in performance on this task as a result of employing a touch-screen compared to a mouse as the response input device. In their second experiment they found that touch-screen input was superior to vocal responding, in apparent contradiction to Beaumont’s (1985) result. In fact, however, this result is almost certainly due to the fact that the investigators did not allow subjects to view the screen during stimulus presentation in the vocal response condition, as Martin and Allan themselves acknowledge. In other words, the result probably reflects encoding differences rather than response mode differences. The use of computerized auditory presentation of stimuli in conjunction with speech recognition technology for responding is certainly to be encouraged, however, if the aim is to produce a version of the digit span task which is psychometrically parallel to the original WAIS version. In the current study, for all measures, subjects receiving the standard version first performed much better on the computerized version than subjects receiving the computerized version first. This suggests that subjects do not find the cognitive load of computer-based response systems quite so disruptive when they are familiar with the basic requirements of the digit span test per se. It is interesting to note that in the Wilson et al. (1982) study no significant difference was found between the standard and computerized versions of the test for those subjects receiving the standard version first, although a difference was found both for those subjects receiving the computerized version first and for the two groups combined. Although the data was not analysed using analysis of variance, this suggests a similar interaction to those reported here. These interactions support the claim that task novelty is an extremely important factor in performance. At least three kinds of novelty are relevant to performance in this study. First, the general nature of the digit span task itself, which is common to both versions of the test. Most subjects had probably never performed a digit span task before, and the backward span task in particular requires a certain degree of planning, particularly in lower ability subjects (Schofield and Ashman, 1986). Second, these subjects were unlikely to be very familiar with computer technology generally. This aspect is likely to become less important as the technology plays an ever increasing role in everyday life. Finally, particular implementations of computerized digit span tasks will vary with respect to how natural they are in terms of stimulus presentation and responding. The particular version studied here led to inferior performance compared to the standard version, but technological advances may serve to reduce this problem in future implementations. These latter two aspects of task novelty were only applicable to the computerized version and would probably both demand a certain degree of planning and the formulation of a general 174
interacting
with Computers
vol4
no 2 (1992)
cognitive strategy. The pattern of results reported here fits in well with Stemberg’s (1985) triarchic concept of intelligence, one component of which deals specifically with the role of novelty. Sternberg points out that the ability to cope with novelty is a very important aspect of intelligence, as is the related skill of ability to automatize various aspects of task performance (that is, relegating aspects of performance to a non-conscious, automatic level). The more able one is to cope with novel aspects of a task, the more cognitive resources are available to automatize aspects of task performance. Conversely, the more aspects of a task which are already automatized, the more cognitive resources are available to cope with novel aspects. In the current study, presenting the standard version first allowed subjects to cope with the limited novel aspects of the task and to begin to automatize aspects of general digit span performance. The subsequent performance of the computerized version, therefore, was already partly automatized, leaving more resources free to deal with the novel aspects. Presenting the computerized version first led to cognitive overload in terms of task novelty, and thus fewer aspects of the task would become automatized leading to inferior performance. The computer-based digit span test presented here cannot be said to by psychometrically parallel to the usual WAIS subtest version insofar as spans are clearly lower on the computerized test in comparison to the standard test. However, that does not necessarily mean that the computerized version, if standardized on a large normative sample, would be of no clinical value. Indeed, Webster et al. (1983) suggest that dissociation in performance on visual and auditory digit span could be of considerable use in clinical assessment. The standard-computer correlations in this study seem to be comparable with the test-retest reliability of the standard test (Moore et al., 1990) over similarly lengthy period. Future research in this area should be directed at describing in detail the cognitive processes involved in digit span tasks across a range of stimulus presentation and response input modalities in order to facilitate the development of new tests which take full advantage of the capabilities of modern microcomputer technology. There are grounds for believing that such tests could be of considerable value to clinicians and others involved in assessment.
Acknowledgements This research, which was carried out when the authors were at the Department of Psychology, University of Leicester, was supported by the UK Department of Health and Social Security. We thank the copyright holders of the test for permission to use it in this research. We would also like to thank the members of the clinical site teams: Mr G. Ives (Barnsley District General Hospital; Mr M. Binns, Mr C. Broom, Mr C. Burdett, MS K. Burley and MS L. Fountain (Carlton Hayes Hospital, Leicester); Mr S. Duffy, Mr W. Hall and Mr M. Lee-Evans (Rampton Hospital); Mr M. Corp, MS L. Lawrence, MS C. Spreadbury and MS C. Watson (Towers Hospital, Leicester). French and Beaumont
175
Thanks
are
Hemmings,
also
due
technical
to to Mr W.W.
Williamson,
Mr I’. Benson
of Psychology,
staff of the Department
Mr R. Universi-
and
Leicester
ty, for their support.
References Banken, J.A. (1985) ‘Clinical utility of considering digits forward and digits backward as separate components of the Wechsler Adult Intelligence Scale - Revised' I. Clinical Psychol. 41,68&691 Bartram, D. and Bayliss, R. (1984) ‘Automated Occupational Psychol. 57,221-237
testing: past, present and future’!.
Beaumont, J.G. (1981a) ‘Microcomputer aided assessment and its implications clinical practice’ Orchard Lodge Studies of Deviancy 1,21-42
for
Beaumont, J.G. (1981b) Report: Microcomputer aided assessment in psychiatric and neurological states. Phase I (May 1980-March 1982) Department of Health and Social Security, London, UK Beaumont, J.G. (1981~) ‘A PASCAL program to administer Psychol. Rev. 1,115117
a digit span test’, Current
Beaumont, J.G. (1981d), ‘Microcomputer-aided assessment using standard psychometric procedures’ Behaviour Research Methods b Instrumentation 13,430433 Beaumont, J.G. (1982) ‘System requirements Studies 17,311-320.
for interactive
testing’ Int. J. Man-Mach.
Beaumont, J.G. (1985) ‘The effect of microcomputer presentation on digit span performance’ Int. J. Man-Mach. Studies 22‘11-18
and response medium
Beaumont, J.G., and French, C.C. (1987) ‘A clinical field study of eight automated psychometric procedures: the Leicester/DHSS project’ Int. J. Man-Mach. Studies 26, 661-682 Black, F.W. (1983) ‘Digit repetition in learning disabled children/I. Clinical Psychol. 39, 281-284 Black, F.W. (1986) ‘Neuroanatomic and neuropsychologic correlates of digit span performance by brain-damaged adults’ Perceptual and Motor Skills 63,815822 Black, F.W., and St-rub, R.L. (1978) ‘Digit repetition brain damage’ Cortex 14,12-21
performance
in patients with focal
Burke, M.J., and Normand, J. (1987) ‘Computerized psychological and critique’ Professional Psychol. Research and Prac. 18,42-51
testing: Overview
Butcher, J.N., Keller, L.S. and Bacon, S.F. (1985) ‘Current developments and future directions in computerized personality assessment’ I. Consulting and Clinical Psychol. 53, SO>815 Costa, L.D. (1975) ‘The relation of visuospatial patients with cerebral lesions’ Cortex 11,31-36 French, C.C. (1985) ‘Automated Review 2‘56
dysfunction
to digit span performance
in
testing: The need for caution’ Guidance and Assessment
French, C.C. (1986) ‘Microcomputers Counselling 14,3345.
and psychometric
assessment’
Brit. J. Guidance and
French, C.C. (1990) ‘Computer-assisted assessment’ in Beech, J.R. and Harding, L. (Eds) Testing people: a practical guide to psychometrics NFER-Nelson, Windsor, 13.5142 French, C.C. and Beaumont, J.G. (1987) ‘The reaction of psychiatric 176
Interacting
patients to
with Computers
vo14 no 2 (1992)
computerized
assessment’
Brit. 1, Clinical Psychol. 26,267-278
French, CC. and Beaumont, J.G. (1989) ‘A computerized form of the Eysenck Personality Questionnaire: A clinical study’ Personality and individual Differences 1027-1032
10,
French, C.C. and Beaumont, J.G. (1990) ‘A clinical study of the automated assessment of intelligence by the Mill Hill Vocabulary Test and the Standard Progressive Matrices Test’ I. Clinical Psychol. 46,129-140 French, C.C. and Beaumont, J.G. (1991) ‘The Differential Aptitude Test (Language Usage and Spelling): a clinical study of a computerized form’ Current Psychol. Research and Rev. 10,314s Gardner, R.A. (1981) ‘Digits forward and digits backward as two separate tests: Normative data on 1567 school children’Iourna1 of Clinical Child Psychology 10,131-135 Griffin, P.T. and Heffernan, A. (1983) ‘Digit span, forward and backward: separate and unequal components of the WAIS digit span’ Perceptual and Motor Skills 56,335-338 Gupta, S., Mahto, J., Tandon, P.N., and Singh, B. (1986) ‘Digit span test in epileptic patients with left and right hemispheric focal EEG abnormalities’ I. Personality and Clinical Studies 2,21-26 Hebb, D.0, (1961) ‘Distinctive features of learning in the higher animal’ in Delafesnaye, J.F. (Ed.) Brain mechanisms and learning Oxford University Press, 37-46 Hedlund, J.L., and Vieweg, B.W. (1988) ‘Automation Psychiatric Annals 18,217-227
in psychological
Kaufman, A., and Kaufman, N. (1983) Kaufman Assessment American Guidance Service, Circle Pines, MN, USA
testing’
Battery for Children (K-ABC)
Kear-Colwell, J.J. (1977) ‘The structure of the Wechsler Memory Scale: a replication’ Clinical Psychol. 33,48%485
I.
Longstreth, L.E., and Madigan, S. (1982) ‘Sex differences in the correlation of memory span with scan and other episodic memory tasks’ intelligence 6,37-56 Martin, T.A., and Allan, W.E. (1991) ‘An evaluation of touch-screen input for a HyperCard-based digit-span task’ Behaviour Research Methods, Instruments, & Computers 23,25%255 Moore, A.D., Stambrook, M., Hawryluk, G.A., Peters, L.C. eta1 (1990) ‘Test-retest stability of the Wechsler Adult Intelligence Scale - Revised in the assessment of head-injured patients’ Psychological Assessment 2,98-100 Newcombe, F. (1969) Missile wounds of the brain: A study of psychological University Press
deficits Oxford
Penney, C.G. (1989) ‘Modality effects and the structure of short-term memory’ Memory 6 Cognition 17,398-422 Penney, C.G., and Blackwood, P.A. (1989) ‘Recall mode and recency in immediate recall: Computer users beware !’ Bull. Psychonomic Sot. 27,54!%547
serial
Richardson, J. (1977) ‘Functional relationship between forward and backward digit repetition and a nonverbal analogue’ Cortex 13,317-320 Rudel, R.G. and Denckla, M.B. (1974) ‘Relation of forward and backward digit repetition to neurological impairment in children with learning disabilities’ Neuropsychologia 12,109-118 Sampson, J.P. (1983) ‘Computer-assisted implications for the future’ Measurement Saville, P. (1971) British Supplement French and Beaumont
testing and measurement: current status and and Evaluation in Guidance 15,293-299
to the Manual of the Wechsler Adult Intelligence
Scale 177
NFER, Windsor, Berks, UK Schofield, N.J. and Ashman, A.F. (1986) ‘The relationship between digit span and cognitive processing across ability groups’ Intelligence 10,59-73 Stemberg, University
R.J. (1985) Beyond IQ: A Triarc~z~c Theory of ~~~a~ l~fei~igence Cambridge Press
Steme, D.M. (1969) ‘The Benton, Porteus and WAIS digit span tests with normals and brain injured subjects’ J. Clinical Psychol. 25,173-175 Thompson,
J.A. and Wilson,
S.L. (1982) ‘Automated
psychological
testing’ Inf. J.
Man-Mach. Sfudies 17‘279-289 Webster, J.S., Gouvier, W.D. and Gross, R.T. (1983) ‘A portable computerized technique for assessing visual digit span’ ~ehavioura~ Eng. 8,98-101 Wechsler,
D. (1945) ‘A standardized
memory scale for clinical use’ 1. Psychol. 19,87-95
Wechsler, D. (1955) Manual for the Wechsler AduLf Intelligence Corporation Weinberg, J., Diller, L., Gerstman, L. and Schulman, left hemiplegics’ f, Cli~jcal Psychol. 28,361
Scale The Psychological
P. (1972) ‘Digit span in right and
Wilson, S.L. (1987) ‘The development of an automated test of immediate memory and its evaluation on severely physically disabled adults, Applied Psychol. Inf. Rev. 36,311-327 Wilson, S.L., Thompson, J.A. and Wylie, G. (1982) ‘Automated psychological for the severely physically handicapped‘ lnf. J. Man-Mach. Studies 17,291-296
178
testing
Interacting with Computers vol4 no 2 (1992)
with Computers
uol 4 no 2 (1992) 163-178
Microcomputer version of a digit span test in clinical use Christopher
C. French and J. Graham Beaumont*
At four clinical sites, 188 psychiatric patients were assessed with both the standard auditory verbal digit span test and a computerised visual digit span test requiring manual response. Correlations between the two test forms were comparable with other similar studies. Subjects obtained higher scores on the standard version than the computerized version, probably reflecting differences in the mode of presentation and the mode of response, but this difference was much less pronounced in those subjects receiving the standard version first rather than the computerized version first. It is clear that the computerized test cannot be considered to be psychometrically parallel to the standard WAIS subtest version, but it may still be of some clinical value. Keywords:
Computerized
testing, digit span, clinical applications.
The field of computerized
(or automated) assessment has developed rapidly in recent years (see e.g. Bartram and Bayliss, 1984; Beaumont, 1981a; Burke and Normand, 1987; Butcher et al., 1985; French, 1985, 1986, 1990; Hedlund and Vieweg, 1988; Sampson, 1983; Thompson and Wilson, 1982). The relative advantages and disadvantages of computerized compared to standard assessment, although beyond the scope of the current article, have been widely discussed and the interested reader is referred to the above articles for more detail. This paper will focus specifically upon the comparability of the standard version of the digit span task and a computerized version as used in the Leicester/DHSS project. The Leicester/DHSS project on microcomputer-aided assessment took place in two phases. In Phase 1, the hardware and software system of an inexpensive microcomputer (the Apple II+) was evaluated with respect to its suitability for administering a range of psychometric tests, software was prepared to administer a number of such tests, and limited pilot investigation of the psychometric performance of this software was undertaken Department of Psychology, Goldsmiths’ College, University 6QE, England. Tel: 081-692 7171 Ext. 2075. *University College Swansea, Swansea, Wales. 0953-5438/92/020163-16
0
1992 Butteworth-Heinemann
of London, New Cross, London SE14
Ltd
163
(Beaumont, 1981b). Results were sufficiently encouraging for Phase 2 to be undertaken. A general survey of the findings of Phase 2 of the Leicester/DHSS project is presented by Beaumont and French (1987). This was a large scale study of microcomputer-aided assessment involving the administration of eight psychological tests to 367 patients at five clinical sites. As stated, it is the purpose of this paper to focus in more detail upon the results relating to the automated version of a digit span test used in that study. Digit span tasks are probably the single more commonly used means of assessing short-term memory and it is likely that as clinicians make ever greater use of microcomputers in their day-to-day work, computerized versions of the procedure will become commonplace. The question of comparability between standard and computerized forms is therefore of considerable importance. Beaumont (1981~) presented a description of the digit span test used in the project. Various types of digit span test are in existence, but this version is largely based on the subtest of the Wechsler Adult Intelligence Scale and 1945, 1955; Saville, 1971). One major Wechsler Memory Scale (Wechsler, difference is that in the computerized version digits are presented visually rather than auditorily as in the traditional version, but at an equivalent rate. The program measures forward span, backward span and total span. Sequence length increases by one for every correct response and two error responses conclude that section of the test. The maximum span which can be measured is 30. A second major difference between the computerized digit span test and the standard version is in terms of subjects’ response mode. In the standard version, subjects simply repeat back the digit strings orally. In the computerized version, responses can be input either by keying in the sequence from the keyboard or touching the appropriate areas of a touch-sensitive screen. The sequence entered is shown on the screen, and the response can either be started over again, or the last digit deleted, by pressing a single key or touching the appropriate screen area. Finally, subjects press a key or touch the appropriate screen area to indicate completion of a response. A full record of performance is available, but a password must be entered before results are presented. In line with other automated versions of digit span tests (e.g. Wilson et al., 1982; Wilson, 1987), the current system presents different randomly generated strings of digits to the subject on each occasion of testing. This is a major advantage of automated digit span assessment over the standard WAIS version, allowing for frequent re-use of the test on the same individual, if required. In certain situations, e.g. psychopharmacological studies of the effects of different drugs on memory, it is necessary to make such assessments repeatedly over a relatively short period potentially confounding drug effects with practice effects which are known to occur (e.g. Hebb, 1961). It should be noted, however, that practice effects would be expected in the early stages on tasks such as this even if the digit strings are varied, but such effects will be minimized if different digit strings are employed. However, no system can fully control for the effects of strings which may have personal significance for the subject, for example corresponding in whole or in 164
Interacting
with Computers
uol 4 no 2 (1992)
part with the subject’s telephone number, address, date of birth and so on. Once again, however, the automated versions are superior to the standard WAIS version in that any such distortions will not be constant across repeated assessments because different strings will be generated on each occasion. The fact remains that the standard WAIS version of the digit span test is the most commonly used and it is therefore of some interest to examine the relationship between performance on the two versions of the test in a clinical setting. As patients in this study were only tested once on the standard version of the test and once on the computerized version, the disadvantages of the former with respect to repeated testing will have no bearing on the results obtained. Any differences found between standard and computerized presentations will reflect differences in the method of presentation and response rather than differences due to the composition of the strings themselves. It is worth noting here that Beaumont (1985; Experiment 2) has presented results showing that microcomputer presentation of the digit span test produced significantly poorer digit spans than the standard auditory-verbal administration when administered to groups of undergraduate students. Beaumont compared the performance of subjects in the following conditions: 0
0 l
vocal presentation of stimuli with vocal response (i.e., standard digit span as described above); visual presentation of stimuli with responses input to computer (i.e., computerized form described above); visual presentation of stimuli with vocal response (i.e., computerized presentation, but standard response mode).
In this way he showed that the higher scores on the standard compared to the computerized version were due to both the use of computer-based response devices rather than vocal responding and the presentation of stimulus material in the visual rather than the auditory modality. Penney and Blackwood (1989) report two experiments in which serial recall of lists of digits was investigated with subjects responding either by writing their responses or entering them into a computer by means of a keyboard. Keyboard response mode reduced recall from the final serial positions of the lists. This reduction in the recency effect was much more pronounced for visual presentation of stimuli than auditory presentation. These investigators raise the possibility that allocation of mental resources to the unpracticed keyboard response diverts resources away from the maintenance of list items in some kind of rehearsal loop or response buffer. The general superiority of the auditory modality over the visual modality in terms of memory for verbal material is a well-established effect in psychological research. Penney (1989) has suggested that verbal information presented to the auditory and visual modalities is processed in separate streams that have different properties and capabilities. In particular, she claims that recall of auditory stimuli will be better with successive presentation of items, whereas recall of visual items will favour simultaneous presentation of items. Her theory has obvious relevance to the issue of computerization of digit span tasks. French and Beaumont
165
This paper presents data relating to the forward span, backward span and total span, as all three measures are commonly cited in the clinical literature. It should be noted, however, that there is considerable disagreement regarding the value of combining the forward and backward spans in a single measure rather than considering each separately. For example, Rude1 and Denckla (1974) collected data from 297 children with learning disabilities. They claimed that subjects presumed to have left cerebral hemisphere damage were impaired in terms of forward span, whereas subjects presumed to have right hemisphere damage were impaired in terms of backward span. Based upon the known functional properties of the cerebral hemispheres, they suggested that the forward span was based largely upon an auditory verbal component whereas the backward span task requires subjects to form a visual image of the auditorily presented digits and to use visual scanning mechanisms to produce the reversed sequence. Gardner (1981) has argued that although the total span may be useful in a general test of intelligence, consideration of forward and backward spans separately can provide important additional insights. To this end, he has provided normative data from 1567 school children for forward and backward spans separately. Other studies which are consistent with the notion that forward span is largely dependent upon verbal processing whereas backward span involves visuospatial processing include Banken (1985)‘ Black (19831, Costa (1975), and Weinberg et al. (1972). However, other investigators have produced data which are not consistent with this explanation. For example, Richardson (1977) found no correlation between backward span and a ‘spatial span’ score although forward and backward digit span were highly correlated. He concluded that both digit span measures employ a common verbal representation. Newcombe (1969) found that left hemisphere damage led to impaired performance on both forward and backward spans, whereas right hemisphere damage did not impair either. Other studies which have failed to support the claim that visuospatial processing is an important component of the backward span task include Black (1986), Black and Strub (1978) and Gupta et al. (1986). It is not even clear whether forward span or backward span is more sensitive to brain damage. Costa (1975) found no difference between brain-damaged patients and normal controls for forward span, but brain-damage was associated with impaired backward span. Black (1986) produced data generally in line with the claim that backward span was a more sensitive indicator of brain damage. Griffin and Heffernan (1983) found that total span and backward span differed across five ability levels in a group of intellectually subnormal subjects, but that forward span did not. In contrast, Black and Strub (1978) claimed that forward span was a more sensitive measure of brain dysfunction. Kear-Colwell (1977) concluded that there is no point in providing separate scores for forward and backward spans, as they provide little unique information. Finally, Sterne (1969) was unable to demonstrate any differences at all between brain-damaged patients and non-brain-damaged controls. Thus, although many studies indicate that backward span is more impaired by brain damage than forward span, a substantial number of studies do not support this. 166
Interacting
wifh Computers
~014 no 2 (1992)
It is worth noting here that Schofield and Ashman (1986) present data supporting the serial-processing character of forward digit span, in line with the position adopted by other investigators (e.g. Kaufman and Kaufman, 1983). Backward span, on the other hand, involved not just sequential processing, but was also closely related to planning abilities.
Method Design This paper focuses on data from patients who were tested on one version of the test and retested on the other. All sites were instructed to allocate patients randomly to: 0
0
either the standard or computerized version of the test on first testing (with the alternate test form employed for retest); either the keyboard or touch-screen as the response mode for computerized testing.
The decision as to whether a particular patient was to take part in the project or not was left solely in the hands of the clinicians involved, but was based upon the normal clinical appropriateness of the test procedures for that patient. Site teams were issued with specific and detailed instructions with regard to the allocation of subjects to conditions, but examination of the number of subjects in each condition reveals that these procedures were not always followedi. There appears to have been some reluctance to run patients on the touch-screen version of the test, despite the fact that the patients themselves seemed to find the test as acceptable as the keyboard version (see French and Beaumont, 1987). The general conclusions with regard to the comparability of the standard versus the computerized test would not be affected by this departure from protocol, but previously published conclusions with regard to the comparability of the keyboard and touch-screen versions (Beaumont and French, 1987) should be treated with some caution.
Assessment system The sites involved in the project were provided with a minimum configuration of Apple II+ microcomputer with single disc drive, Language Card, communications interface card, and a Mountain Hardware clock. The display device was a Portatel colour monitor to which was fitted a capacitative touch-sensitive screen (Interaction Systems Inc.), thus allowing either the standard keyboard or the touch-sensitive screen to be used to input responses. Discussion of the ‘We are grateful to an anonymous
reviewer for pointing out this departure from the intended experimental protocol. It should be noted that the random allocation procedures appear to have been followed properly when the Eysenck Personality Questionnaire (French and Beaumont, 1989), the Money Road Maps Test (Beaumont and French, 1987), the Mill Hill Vocabulary Test and Standard Progressive Matrices Test (French and Beaumont, 1990) were administered as part of this project, but departures from true random allocation are also apparent in the data for the DAT subtests (French and Beaumont, 1991). French and Beaumont
167
principles 1982).
adopted in designing
this system can be found in Beaumont
(1981d;
Patients and clinical sites The mean age of the subjects (age data available for 176 of 188 subjects) was 37.31 years (standard deviation = 13.45 years), ranging from 18 to 76 years. The group was composed of 115 males and 73 females. With respect to marital status, 108 were single, 46 married, 20 divorced or separated, 11 widowed and three were not classified. With respect to neurological status, 158 were neurologically sound and 30 were definitely or possibly neurologically impaired. The diagnostic composition of the group was: l l
0 l
0 l l
affective psychosis (23.4%) schizophrenia (25.0%) neurosis (14.9%) personality disorder (16.5%) mental disorders not specified as psychotic tions (4.3%) special symptoms not elsewhere classified mild mental retardation (5.9%)
associated
with physical
condi-
(4.3%)
These diagnoses account for 94.3% of the group. The sites at which the system was located for clinical evaluation were two large general psychiatric hospitals (Psychiatric Hospitals A and B, respectively), one of which had a relatively high proportion of long-stay chronic patients; a district general hospital; and a high security Special Hospital for mentally disordered offenders. The number of patients tested at each site was as follows: l l l l
Psychiatric Hospital A Psychiatric Hospital B District General Hospital Special Hospital
36 57 45 50
Procedure The standard version of the test was administered according to the instructions in the WAIS manual. The computerized version was designed to be as similar to the standard version as possible, except for the mode of stimulus presentation and response. Tests were usually, though not necessarily, administered by the same investigator on each occasion. Following the completion of the test, the subject was required to complete a Patient Response Questionnaire. The results of this questionnaire have been reported elsewhere (French and Beaumont, 1987). As already noted, the data collected for this study were part of a much larger project (Beaumont and French, 1987). For most of the other tests investigated it was desirable that, whenever possible, site teams allowed at least three days to elapse between testing and retesting. The average test-retest interval was much longer for all tests, including the digit span (M = 36.62 days, SD = 58.70, 168
Interacting
with Computers
ml
4 no 2 (1992)
Table 1. Mean scores, standard deviations and reliabilities for all subjects for forward span Test condition All retest data:
Keyboard
only:
Touch-screen
only
Computer first (n=97) Standard first (n=91) Overall (n=lSS) Computer first (n=60) Standard first (n=57) Overall (n=117) Computer first (rl=37) Standard first (n=34) Overall (n=71)
maximum = 374 days). With test-retest intervals were not of collecting data as part of ensure a stringent test of the span test.
M SD M SD M SD M SD M SD M SD M SD M SD M SD
Standard
Computer
6.15 1.41 6.15 1.40 6.15 1.40 6.15 1.38 5.98 1.47 6.07 1.42 6.16 1.48 6.44 1.23 6.30 1.37
4.93 1.51 5.32 1.38 5.12 1.46 5.00 1.63 5.33 1.39 5.16 1.52 4.81 1.31 5.29 1.38 5.04 1.36
Y 0.505 p
0.630 p
0.424 p=O.OOl 0.744 p
0.436 p=O.OlO 0.571 p
respect to the digit span assessments, such large actually necessary, and occurred simply as a result a larger investigation. However, they obviously relationship between the two versions of the digit
Results Tables 1,2 and 3 present mean scores, standard deviations and reliabilities for forward, backward and total span respectively for all subjects. The reliability of the Digit Span test in the WAIS manual was estimated by correlating Digits Forward with Digits Backward and was found to be moderate (between 0.60 and 0.69; Wechsler, 1955; Saville, 1971). As described in the introduction, there is good reason to believe that the forward and backward spans might involve different cognitive functions, and the wisdom of combining them into a single total score is open to question. By the same token, it would seem to be unwise to attempt to assess the ‘reliability’ of the digit span task by correlating the two spans, as reported in the original WAIS manual. Correlation coefficients in this study were calculated by correlating the standard score with the computer score for each of the three measures. They too were moderate, yielding values of 0.557 for the forward span, 0.708 for the backward span and 0.732 for the total span (all p
169
Table 2. Mean scores, standard deviations and reliabilities for all subjects for backward span Test condition
All retest data:
Computer first (n=97) Standard first (n=91) Overall (n=188)
Keyboard
only
Touch-screen
only:
Computer first (n=60) Standard first (n=57) Overail (n=117) Computer first (n=37) Standard first (n=34) Overall (n=71)
M SD M SD M SD M SD M SD M SD M SD M SD M SD
Standard
Computer
r
4.53 1.47 4.45 1.40 4.49 1.44 4.60 1.46 4.35 1.47 4.48 1.47 4.41 1.50 4.62 1.28 4.51 1.39
4.07 1.61 4.54 1.57 4.30 1.60 4.17 1.62 4.53 1.73 4.34 1.68 3.92 1.61 4.59 1.28 4.23 1.49
0.766 p
0.690 p
0.697 p
0.614
p
0.731 p
Table 3. Mean scores, standard deviations and reliabiiities for all subjects for total span Test condition All retest data:
Keyboard
only:
Touch-screen
170
only:
Computer first (n=97) Standard first (n=91) Overall (n=188) Computer first (n=60) Standard first (n=57) Overall (n=117) Computer first (n=37) Standard first (n=34) Overall (n=71)
M SD M SD M SD M SD M SD M SD M SD M SD M SD
Standard
Computer
Y
10.68 2.61 10.60 2.51 10.64 2.56 10.75 2.55 10.33 2.68 10.55 2.61 10.57 2.73 11.06 2.17 10.80 2.48
9.00 2.77 9.86 2.61 9.41 2.72 9.17 2.84 9.86 2.82 9.50 2.84 8.73 2.65 9.85 2.27 9.27 2.52
0.719 p
Interacting
with Computers
p-eo.001
0.732 p
0.718 p
0.807 p
0.720 p
0.772 p
no 2 (1992)
Table 4. Mean spans on standard and computerized tests for possibly or definitely neurologically impaired group (NI?) versus neurologically sound (NS) group
Standard spans: Forward Backward Total Computerized Forward Backward Total
NI? mean (n = 30)
NS mean
t
P
5.43 3.97 9.40
6.29 4.59 10.88
3.15 2.20 2.97
4.63 3.87 8.50
5.21 4.38 9.59
2.00 1.61 2.03
co.05 not significant co.05
(n = 158)
spans:
Before carrying out any analyses involving sex as a between-subjects factor, it was felt that it was appropriate to exclude data from two partially overlapping subgroups: subjects who were definitely or possibly neurologically impaired, and subjects from the Special Hospital. The reason for this is that both of these groups would be expected to perform poorly on the digit span tasks, and this expectation was confirmed by exploratory data analysis using t-tests (see Tables 4 and 5). The single exception was that there was no significant difference between the neurologically suspect patients and the neurologically sound patients with respect to the computerized backward span. With respect to sex, 22 out of 115 male subjects (19%) were possibly or definitely neurologically impaired, whereas only eight out of 73 female subjects (11%) fell into this category. Furthermore, all of the Special Hospital subjects were male. These subgroups were therefore excluded in order to avoid confounding neurological status and clinical site with sex. Data for the remaining 113 subjects for whom age and sex data were available were submitted to analysis of variance, with age level (above versus below
Table 5. Mean spans on standard and computerized tests for Special Hospital Group (SHG) versus others
Standard spans: Forward Backward Total Computerized spans: Forward Backward Total French and Beaumont
SHG (n = 50)
Others (n = 138)
5.28 3.86 9.14
6.47 4.72 11.19
5.55 3.74 5.18
4.68 3.76 8.44
5.28 4.49 9.77
2.51 2.82 3.02
co.05 co.01 co.01
P
171
Table 6. Means scores on standard and computerized digit spans (excluding neurolo~cally impaired and Special Hospital patients) for patients receiving standard version first (n = 56) and computer version first (n = 57) Computer
Standard Forward span Standard 1st Computer 1st Backward span Standard 1st Computer 1st Total span Standard 1st Computer 1st
6.61 6.46
5.77 4.96
4.84 4.74
4.93 4.16
11.45 11.19
10.70 9.12
versus touch-screen, and standard-first versus median), sex, keyboard computer-first as between-subject variables, and standard score versus computer score as a within-subjects factor. Separate analyses of variance were carried out for forward span, backward span and total span. It was found that subjects had significantly shorter forward spans on the computer (M = 5.36) than on the standard version (M = 6.53; F(1, 97) = 66.03, p<.OOl). This effect was also found for the backward span (standard M = 4.79; computer M = 4.54; F(l, 97) = 4.60, p<.O5) and for the total span (standard M = 11.32; computer M = 9.90; F(l, 97) = 50.01, p<.OOl). A significant interaction of the factors standard-first versus computer-first x standard score versus computer score was found for forward span (F(l, 97) = 4.95, p<.O5), backward span (F(l,97) = 5.13,p<.O5),and total span (F(l,97) = 8.86, p<.Ol). In each case, the group receiving the computer version first showed a greater difference between the computer and standard scores than did the group receiving the standard version first (see Table 6). One other significant effect was found, but only in the analysis of forward span data, where the keyboard versus touch-screen and standard score versus computer score factors appeared to interact significantly at a just-significant level (F(1, 97) = 3.97, ~7=.049). This interaction reflects the fact that while the keyboard and touch-screen groups obtained very similar scores on the computerized test (keyboard M = 5.37; touch-screen M = 5.35)‘ they differed with regard to ther standard forward spans (keyboard group M = 6.33; touch-screen group M = 6.90). It is likely that this interaction is due to the selection bias of the clinical site teams noted earlier. It appears that the teams were reluctant to allocate psychiatric patients to the touch-screen condition presumably because they felt that it was too complicated for general use (only 40 subjects out of 113 were allocated to the touch-screen condition). They appear to have been willing to allocate only relatively bright patients to this condition, thus explaining the difference in standard forward spans. While this underlines the problems of reconciling appropriate experimental design with clinical concern for patients, it should be noted that the general conclusions regarding the comparison between the standard and computerized versions still stand. 172
~nfe~ucfing
with
Conzputers
vol 4 no 2 11992)
Discussion Reliabilities on this test were moderate for all three span measures, although they were of the same order of magnitude as correlations between forward and backward span on the standard version of the test and test-retest correlations over similarly long periods for the standard version (e.g., Moore et al., 1990). It is instructive to compare these results with those of Wilson et al. (1982) and Wilson (1987) who compared scores on standard and computerized versions of a digit span test using severely physically disabled adults as subjects. Wilson et al.‘s (1982) test differed from both the original test and the current implementation in that only the digits l-6 were used and only the forward digit span was measured. Subjects were retested after a minimum interval of two weeks. Wilson’s (1987) version, known as PADS (Putney Automated Digit Span), uses the digits 1-9 and continues until the subject makes five errors at a particular string length. It also incorporates the partial scoring scheme described by Longstreth and Madigan (1982), in which the subject’s score for each part of the test is the length of the longest string correctly recalled plus a partial score from the performance of the last five incorrectly recalled trials. In this study, each subject took part in two sessions (separated by an average of three weeks), but received both the computer and the standard version for each session. In general, this study and those carried out by Wilson et al. (1982) and Wilson (1987) show good overall agreement with respect to the pattern of correlations (with correlations ranging from 0.48 to 0.86 across conditions in the studies by Wilson and colleagues). This suggests that the long test-retest intervals used in the present study have not unduly distorted the correlations between the standard and computerized digit span tests and that a similar pattern of results might have been obtained even if a much shorter interval had been used. We can conclude that the current system performs at least as well as other similar systems in terms of standard computer correlations. Minor differences between the studies are likely to reflect the fact that Wilson’s studies used physically disabled patients (on a specially designed assessment system) and also that Wilson’s (1987) correlations are based upon same-day retesting. This study found that patients who were definitely or possibly neurologically impaired performed significantly less well on all measures, with the single exception of the computerized backward span, and offers no support for the claim that backward span is a more sensitive measure of neurological impairment. It should be borne in mind, however, that patients were classified simply on the basis of the clinicians’ response to the single question, “Is the patient definitely or possibly neurologically impaired?“, which was asked mainly in order to allow the screening out of such patients from the main analysis, if required. Patients at the Special Hospital also showed impaired performance compared to the rest of the group. In view of the findings of Beaumont (1985), and the general findings of modality effects in short-term memory (Penney, 1989), it came as no surprise that the computerized version of the digit span produced lower scores than the standard version. Beaumont had shown that this was due to both the use of computer-based response devices and the presentation of stimuli in the visual French
and Beaumont
173
rather than the auditory modality. Similar findings are reported by Wilson et al. (1982) and Wilson (1987). A recent study of a HyperCard-based digit-span task by Martin and Allan (1991) should be mentioned here. The computerized version of the digit-span task used by these investigators presented the digit sequences by highlighting the keys on a numeric display on the computer screen while simultaneously presenting the stimuli in a digitized female voice. The first experiment reported by these investigators indicated that there was no difference in performance on this task as a result of employing a touch-screen compared to a mouse as the response input device. In their second experiment they found that touch-screen input was superior to vocal responding, in apparent contradiction to Beaumont’s (1985) result. In fact, however, this result is almost certainly due to the fact that the investigators did not allow subjects to view the screen during stimulus presentation in the vocal response condition, as Martin and Allan themselves acknowledge. In other words, the result probably reflects encoding differences rather than response mode differences. The use of computerized auditory presentation of stimuli in conjunction with speech recognition technology for responding is certainly to be encouraged, however, if the aim is to produce a version of the digit span task which is psychometrically parallel to the original WAIS version. In the current study, for all measures, subjects receiving the standard version first performed much better on the computerized version than subjects receiving the computerized version first. This suggests that subjects do not find the cognitive load of computer-based response systems quite so disruptive when they are familiar with the basic requirements of the digit span test per se. It is interesting to note that in the Wilson et al. (1982) study no significant difference was found between the standard and computerized versions of the test for those subjects receiving the standard version first, although a difference was found both for those subjects receiving the computerized version first and for the two groups combined. Although the data was not analysed using analysis of variance, this suggests a similar interaction to those reported here. These interactions support the claim that task novelty is an extremely important factor in performance. At least three kinds of novelty are relevant to performance in this study. First, the general nature of the digit span task itself, which is common to both versions of the test. Most subjects had probably never performed a digit span task before, and the backward span task in particular requires a certain degree of planning, particularly in lower ability subjects (Schofield and Ashman, 1986). Second, these subjects were unlikely to be very familiar with computer technology generally. This aspect is likely to become less important as the technology plays an ever increasing role in everyday life. Finally, particular implementations of computerized digit span tasks will vary with respect to how natural they are in terms of stimulus presentation and responding. The particular version studied here led to inferior performance compared to the standard version, but technological advances may serve to reduce this problem in future implementations. These latter two aspects of task novelty were only applicable to the computerized version and would probably both demand a certain degree of planning and the formulation of a general 174
interacting
with Computers
vol4
no 2 (1992)
cognitive strategy. The pattern of results reported here fits in well with Stemberg’s (1985) triarchic concept of intelligence, one component of which deals specifically with the role of novelty. Sternberg points out that the ability to cope with novelty is a very important aspect of intelligence, as is the related skill of ability to automatize various aspects of task performance (that is, relegating aspects of performance to a non-conscious, automatic level). The more able one is to cope with novel aspects of a task, the more cognitive resources are available to automatize aspects of task performance. Conversely, the more aspects of a task which are already automatized, the more cognitive resources are available to cope with novel aspects. In the current study, presenting the standard version first allowed subjects to cope with the limited novel aspects of the task and to begin to automatize aspects of general digit span performance. The subsequent performance of the computerized version, therefore, was already partly automatized, leaving more resources free to deal with the novel aspects. Presenting the computerized version first led to cognitive overload in terms of task novelty, and thus fewer aspects of the task would become automatized leading to inferior performance. The computer-based digit span test presented here cannot be said to by psychometrically parallel to the usual WAIS subtest version insofar as spans are clearly lower on the computerized test in comparison to the standard test. However, that does not necessarily mean that the computerized version, if standardized on a large normative sample, would be of no clinical value. Indeed, Webster et al. (1983) suggest that dissociation in performance on visual and auditory digit span could be of considerable use in clinical assessment. The standard-computer correlations in this study seem to be comparable with the test-retest reliability of the standard test (Moore et al., 1990) over similarly lengthy period. Future research in this area should be directed at describing in detail the cognitive processes involved in digit span tasks across a range of stimulus presentation and response input modalities in order to facilitate the development of new tests which take full advantage of the capabilities of modern microcomputer technology. There are grounds for believing that such tests could be of considerable value to clinicians and others involved in assessment.
Acknowledgements This research, which was carried out when the authors were at the Department of Psychology, University of Leicester, was supported by the UK Department of Health and Social Security. We thank the copyright holders of the test for permission to use it in this research. We would also like to thank the members of the clinical site teams: Mr G. Ives (Barnsley District General Hospital; Mr M. Binns, Mr C. Broom, Mr C. Burdett, MS K. Burley and MS L. Fountain (Carlton Hayes Hospital, Leicester); Mr S. Duffy, Mr W. Hall and Mr M. Lee-Evans (Rampton Hospital); Mr M. Corp, MS L. Lawrence, MS C. Spreadbury and MS C. Watson (Towers Hospital, Leicester). French and Beaumont
175
Thanks
are
Hemmings,
also
due
technical
to to Mr W.W.
Williamson,
Mr I’. Benson
of Psychology,
staff of the Department
Mr R. Universi-
and
Leicester
ty, for their support.
References Banken, J.A. (1985) ‘Clinical utility of considering digits forward and digits backward as separate components of the Wechsler Adult Intelligence Scale - Revised' I. Clinical Psychol. 41,68&691 Bartram, D. and Bayliss, R. (1984) ‘Automated Occupational Psychol. 57,221-237
testing: past, present and future’!.
Beaumont, J.G. (1981a) ‘Microcomputer aided assessment and its implications clinical practice’ Orchard Lodge Studies of Deviancy 1,21-42
for
Beaumont, J.G. (1981b) Report: Microcomputer aided assessment in psychiatric and neurological states. Phase I (May 1980-March 1982) Department of Health and Social Security, London, UK Beaumont, J.G. (1981~) ‘A PASCAL program to administer Psychol. Rev. 1,115117
a digit span test’, Current
Beaumont, J.G. (1981d), ‘Microcomputer-aided assessment using standard psychometric procedures’ Behaviour Research Methods b Instrumentation 13,430433 Beaumont, J.G. (1982) ‘System requirements Studies 17,311-320.
for interactive
testing’ Int. J. Man-Mach.
Beaumont, J.G. (1985) ‘The effect of microcomputer presentation on digit span performance’ Int. J. Man-Mach. Studies 22‘11-18
and response medium
Beaumont, J.G., and French, C.C. (1987) ‘A clinical field study of eight automated psychometric procedures: the Leicester/DHSS project’ Int. J. Man-Mach. Studies 26, 661-682 Black, F.W. (1983) ‘Digit repetition in learning disabled children/I. Clinical Psychol. 39, 281-284 Black, F.W. (1986) ‘Neuroanatomic and neuropsychologic correlates of digit span performance by brain-damaged adults’ Perceptual and Motor Skills 63,815822 Black, F.W., and St-rub, R.L. (1978) ‘Digit repetition brain damage’ Cortex 14,12-21
performance
in patients with focal
Burke, M.J., and Normand, J. (1987) ‘Computerized psychological and critique’ Professional Psychol. Research and Prac. 18,42-51
testing: Overview
Butcher, J.N., Keller, L.S. and Bacon, S.F. (1985) ‘Current developments and future directions in computerized personality assessment’ I. Consulting and Clinical Psychol. 53, SO>815 Costa, L.D. (1975) ‘The relation of visuospatial patients with cerebral lesions’ Cortex 11,31-36 French, C.C. (1985) ‘Automated Review 2‘56
dysfunction
to digit span performance
in
testing: The need for caution’ Guidance and Assessment
French, C.C. (1986) ‘Microcomputers Counselling 14,3345.
and psychometric
assessment’
Brit. J. Guidance and
French, C.C. (1990) ‘Computer-assisted assessment’ in Beech, J.R. and Harding, L. (Eds) Testing people: a practical guide to psychometrics NFER-Nelson, Windsor, 13.5142 French, C.C. and Beaumont, J.G. (1987) ‘The reaction of psychiatric 176
Interacting
patients to
with Computers
vo14 no 2 (1992)
computerized
assessment’
Brit. 1, Clinical Psychol. 26,267-278
French, CC. and Beaumont, J.G. (1989) ‘A computerized form of the Eysenck Personality Questionnaire: A clinical study’ Personality and individual Differences 1027-1032
10,
French, C.C. and Beaumont, J.G. (1990) ‘A clinical study of the automated assessment of intelligence by the Mill Hill Vocabulary Test and the Standard Progressive Matrices Test’ I. Clinical Psychol. 46,129-140 French, C.C. and Beaumont, J.G. (1991) ‘The Differential Aptitude Test (Language Usage and Spelling): a clinical study of a computerized form’ Current Psychol. Research and Rev. 10,314s Gardner, R.A. (1981) ‘Digits forward and digits backward as two separate tests: Normative data on 1567 school children’Iourna1 of Clinical Child Psychology 10,131-135 Griffin, P.T. and Heffernan, A. (1983) ‘Digit span, forward and backward: separate and unequal components of the WAIS digit span’ Perceptual and Motor Skills 56,335-338 Gupta, S., Mahto, J., Tandon, P.N., and Singh, B. (1986) ‘Digit span test in epileptic patients with left and right hemispheric focal EEG abnormalities’ I. Personality and Clinical Studies 2,21-26 Hebb, D.0, (1961) ‘Distinctive features of learning in the higher animal’ in Delafesnaye, J.F. (Ed.) Brain mechanisms and learning Oxford University Press, 37-46 Hedlund, J.L., and Vieweg, B.W. (1988) ‘Automation Psychiatric Annals 18,217-227
in psychological
Kaufman, A., and Kaufman, N. (1983) Kaufman Assessment American Guidance Service, Circle Pines, MN, USA
testing’
Battery for Children (K-ABC)
Kear-Colwell, J.J. (1977) ‘The structure of the Wechsler Memory Scale: a replication’ Clinical Psychol. 33,48%485
I.
Longstreth, L.E., and Madigan, S. (1982) ‘Sex differences in the correlation of memory span with scan and other episodic memory tasks’ intelligence 6,37-56 Martin, T.A., and Allan, W.E. (1991) ‘An evaluation of touch-screen input for a HyperCard-based digit-span task’ Behaviour Research Methods, Instruments, & Computers 23,25%255 Moore, A.D., Stambrook, M., Hawryluk, G.A., Peters, L.C. eta1 (1990) ‘Test-retest stability of the Wechsler Adult Intelligence Scale - Revised in the assessment of head-injured patients’ Psychological Assessment 2,98-100 Newcombe, F. (1969) Missile wounds of the brain: A study of psychological University Press
deficits Oxford
Penney, C.G. (1989) ‘Modality effects and the structure of short-term memory’ Memory 6 Cognition 17,398-422 Penney, C.G., and Blackwood, P.A. (1989) ‘Recall mode and recency in immediate recall: Computer users beware !’ Bull. Psychonomic Sot. 27,54!%547
serial
Richardson, J. (1977) ‘Functional relationship between forward and backward digit repetition and a nonverbal analogue’ Cortex 13,317-320 Rudel, R.G. and Denckla, M.B. (1974) ‘Relation of forward and backward digit repetition to neurological impairment in children with learning disabilities’ Neuropsychologia 12,109-118 Sampson, J.P. (1983) ‘Computer-assisted implications for the future’ Measurement Saville, P. (1971) British Supplement French and Beaumont
testing and measurement: current status and and Evaluation in Guidance 15,293-299
to the Manual of the Wechsler Adult Intelligence
Scale 177
NFER, Windsor, Berks, UK Schofield, N.J. and Ashman, A.F. (1986) ‘The relationship between digit span and cognitive processing across ability groups’ Intelligence 10,59-73 Stemberg, University
R.J. (1985) Beyond IQ: A Triarc~z~c Theory of ~~~a~ l~fei~igence Cambridge Press
Steme, D.M. (1969) ‘The Benton, Porteus and WAIS digit span tests with normals and brain injured subjects’ J. Clinical Psychol. 25,173-175 Thompson,
J.A. and Wilson,
S.L. (1982) ‘Automated
psychological
testing’ Inf. J.
Man-Mach. Sfudies 17‘279-289 Webster, J.S., Gouvier, W.D. and Gross, R.T. (1983) ‘A portable computerized technique for assessing visual digit span’ ~ehavioura~ Eng. 8,98-101 Wechsler,
D. (1945) ‘A standardized
memory scale for clinical use’ 1. Psychol. 19,87-95
Wechsler, D. (1955) Manual for the Wechsler AduLf Intelligence Corporation Weinberg, J., Diller, L., Gerstman, L. and Schulman, left hemiplegics’ f, Cli~jcal Psychol. 28,361
Scale The Psychological
P. (1972) ‘Digit span in right and
Wilson, S.L. (1987) ‘The development of an automated test of immediate memory and its evaluation on severely physically disabled adults, Applied Psychol. Inf. Rev. 36,311-327 Wilson, S.L., Thompson, J.A. and Wylie, G. (1982) ‘Automated psychological for the severely physically handicapped‘ lnf. J. Man-Mach. Studies 17,291-296
178
testing
Interacting with Computers vol4 no 2 (1992)