- Email: [email protected]
Hypermedia for the learning disabled: Expensive luxury or useful tool
Compurers Educ. Vol. 22, No. 3, pp. 215-223, 1994 Copyright 0 1994 ElsevierScienceLtd
Pergamon
Printed in Great Britain. All rights reserved 0360-1315/94$6.00+ 0.00
HYPERMEDIA EXPENSIVE
FOR THE LUXURY
LEARNING DISABLED: OR USEFUL TOOL
ANDREW ROSTRON, RICHARD PLANT and CLAIRE HERMANN Department of Psychology, University of Hull, Hull HU6 7RX, England (Received 6 July 1993; accepted
14 September
[Fax:
+44 482 4655991
1993)
Abstract-This project was designed to assess and evaluate the possible applications of a particular piece of information technology for a young person with severe learning difficulties and severe communication problems. This single subject study was set up within the broader context of information technology for the disabled. The software used was EasySpeaker-a hypermedia speech prosthesis for the vocally and linguistically impaired, the key feature of which is flexibility to adapt to an individual’s needs. The results were encouraging, illustrating the value of relating software to an individual’s own experience. The potential was clearly demonstrated for software to provide a stimulating sensory environment in which an individual of low cognitive ability can access and learn the structure of a small purpose-built set of material. The findings of this research seriously challenge the common view that IT cannot be utilized by the severely learning disabled.
INTRODUCTION
The care of those with learning difficulties requires a great deal of resources. With the advent of care in the community policies and privately run, smaller residential facilities, the structure of this labour-intensive area is undergoing radical changes. In recent years there has at last been a welcome concentration on improving the quality of life for people with learning difficulties, particularly those with severe difficulties, who are often isolated from society. This paper is directed towards answering the question of whether information technology can play any significant role in these changes. There is little doubt that IT can be an invaluable tool in terms of administration-for example providing a nationally accessible network of information, regarding policy formation and regulation[l], but of more interest to the psychologists and educationalists is whether the learning opportunities offered by IT can be harnessed directly to meet the needs of the learning disabled. Some would argue that, although microcomputer technology is playing an increasing role in enhancing communication abilities[2], research should centre on the intellectually able, since the severely learning disabled do not possess the cognitive skills required to benefit from IT. In other words valuable resources allocated for the care of the learning disabled, should not be expended on an expensive luxury, from which it is far from certain any benefit may be derived. Unfortunately this apathetic view is encountered all too often in the field of severe learning difficulties. Change has been slow in coming and the establishment has very durable perceptions as to what can be achieved. For a review of the background see[3]. It was, for example, believed that the learning disabled would not appreciate travel, therefore rendering any investment in transport for them useless. Fortunately such views were overcome and considerable rewards, in terms of quality of life have been reaped. A further example is given by Hegarty and Dean [4], who state their findings illustrate that providers of education tend to make very limited assumptions as to what those with severe learning difficulties can achieve. Heddell and Lee[S] cite the results of an undated study by Spencer on the effect of tuition given to the learning disabled in conjunction with the BBC television series Let’s Go. Spencer’s results showed the use of media augmentation of the tuition had no significant motivational effect. Heddell and Lee claim this was due to a very small sample size and they themselves found exactly the opposite. Although attitudes are changing slowly, Wilkinson and Murray[6] found that in schools “The focus has shifted to children’s strengths . . . [which] can overcome learning difficulties.” Cain and Taber[7] believe the computer provides the ideal teaching accessory as it is truly interactive and 215
ANDREWROSTRON et al.
216
it “allows the matching of the students preferential learning modality with the method of receiving and expressing information.” The present study reports the data obtained from a severely learning disabled individual for whom a small stack of customized hypermedia was constructed and implemented on the speech synthesis system, EasySpeaker, developed by Rostron and Plant [8]. The purpose of this material was to (i) provide a means of assessing the user’s capabilities (ii) devise an environment where communication skills could (iii) offer a novel entertainment and recreation opportunity. The particular aim of the study was to gather be given to the following questions:
be developed
some data which could enable
some answers
to
(i) (ii) (iii) (iv)
can the user recognize and identify pictures and symbols? what aspects of language can be understood and responded to? are there any signs of productive language use? to what extent is the users attention held by the particular environment and how long does it remain of interest? (v) is the situation sufficiently motivating to warrant developing material for other specific learning tasks?
The following paragraphs give some idea of the technology that has been used in the present study. At its most simple level EasySpeaker can be regarded as consisting of a set of screens, stacked like a deck of cards, which the user can traverse by selecting pictures which signify jumps to different cards. When each screen, or card, is displayed the user can select either, ‘speech pictures’, and/or pictures which signify jumps to other related screens. Thus, by navigation of jumps and selection of speech pictures, users can build up sentences or whole dialogues which can then be output as real digitized speech. The source of the pictures themselves can either be drawings or photographs, which are then scanned into the machine. When displayed they are about passport size. To construct language learning environments, the carer has to do no more than select the name of a picture, the screen pigeonhole into which it will placed, and the name of the piece of digitized speech it represents, or the screen to which it is linked. The recording, or digitizing, of speech remains as simple, with carers simply selecting a microphone icon, speaking into the microphone and selecting a relevant name for the segment of speech. Since EasySpeaker records digitally, the stored sound can be music, or indeed any other noise, as well as speech. To track user’s actions, the system stores a record of where the user has been and what they have said. This record can then be viewed by carers once the user has finished their EasySpeaker session. For technology such as this to be acceptable to the caring community in general, it is vital that it is easy to understand and operate. The aim has therefore been to make it as simple for carers as for users, but this aspect of the technology needs to be the subject of further design and evaluation work. Figure 1 shows a screen taken from a real EasySpeaker session.
METHOD Two separate sessions were undertaken with the same subject second session took place 14 days after the first.
in two different
situations.
The
Subject Male aged 22 with severe learning difficulties, who will be referred to as R. R has not been formally assessed for several years, the last occasion was on refusal of entrance to an Adult Training Centre to receive employment related training. It was indicated at the time that R’s lack of communication skills was the major reason for refusal: it was also stated his potential to learn was minimal. R does have some basic monosyllabic speech, however this is limited to a range of basic
Learning
217
disabled
command forms and names of family members e.g. go home; come back; tea; cry; Daddy; Jean; Granny Ivy. R has been known to the third author for some 2: years in which time R has demonstrated an ability to learn to say a small number of new words, but has shown no evidence of being able to produce even short phrases as at the moment he can only use nouns and verbs in a limited manner and has no use of prepositions and conjunctions. R is in the early stage of being taught to use Makaton, the basic sign and symbol system [9] often used in an attempt to facilitate communication for people with severe learning difficulties. As yet there is no evidence that R is using Makaton in any way; however he does often use simple gestures, again a limited number, when trying to communicate. R is very excitable and requires (and demands) constant attention and stimulation, enjoys listening to music, watching music videos, looking at and showing people photographs. R has in the past had medical problems with the cornea of his eyes and has to be given eye drops regularly. It was assumed he had some sight defect but this had not been formally assessed. Session
I
This session took place in R’s home. The materials available for this session were general purpose screens, Pets I Like, and Food. The size of each icon was 2 x 3 cm. R was seated in front of the screen and asked to look at the pictures. The researcher then demonstrated how the mouse was used to move the cursor around the screen and to click on the icons. R was asked to attempt this for himself. A sentence was then produced on screen by the experimenter using the mouse. This sentence was then spoken by the machine. This process was repeated several times. R was again asked to use the mouse himself. R found it difficult to attend to the task and was allowed to move from the computer more or less on demand. The aim was to let him collaborate with the researchers as often as possible. The session was ended after about an hour.
Richard R Plmt TheauthaofEm#+mkw
n ,
I
lhings I like to eat
Fig.
I. A sample
screen
grab
from
r L
6mwd
EasySpeaker. The set of smaller frequently used words or phrases.
wards
icons on the left is available
for
218
ANmEW RosrRoFi et al. Session 2
Software
A hypertext stack was prepared based on responses made in session I and from personal knowledge of R gained over the previous 24 years, relating to his interests, likes and dislikes. The first file was based on a trip to the seaside, the second a small selection of musical extracts represented by icons. The music file contained one icon representing Kylie Minogue, of whom it is known that R is a great fan, and five other icons representing five musical extracts with which R was not familiar. The third was based on Christmas. The size of the pictures was increased to about 4 x 4 cm. The digitized speech to be used in conjunction with the files was spoken by the third author, with whom R is familiar. Some sample screens of the material are presented in Fig. 2. The means of interaction for R was via a simulated touch screen. The way this worked was that R was requested to touch a particular icon and a researcher operated the relevant control once the instruction had been carried out. Procedure
This session took place in a quiet bare room. The only contents of the room apart from the computer system were some chairs and tables. To begin with-for R to feel secure in a strange setting-his parents, and the researcher, whom he knows well, were present along with the computer operator/researcher. R was initially seated in front of the computer screen, the new files were demonstrated as was each icon, the text and digitized speech associated with it. For the remainder of the session R was allowed to freely explore the range of material available, although some initial guidance was given by the computer operator. For example R was told that he only had to touch the screen at his chosen icon to activate the text and digitized speech e.g.” If you want to hear Kyhe point to the picture of Kylie on the screen.” As time progressed the number of prompts was reduced. At a later stage in the session R was asked questions about the material and some simple conversation was attempted. Once R appeared to be comfortable with the situation, his parents and the researcher left the room-their presence was felt to be distracting as R repeatedly turned towards them to indicate his excitement at the speech or music produced. It was also thought that the digitized speech of the researcher would have greater reinforcement value when the researcher was not present. The whole session lasted about If h and a record of all interactions with the system was obtained from the tracking system incorporated in the software.
RESULTS Session
I
Initially R was very excited at the researcher (who had been a friend of his for some time) and two new visitors arriving at his home, but became quieter as the equipment was set up. The software was demonstrated to R but his attempts at using the mouse were repeatedly unsuccessful. In most instances this was due to his clicking both keys of the mouse simultaneously, since he has only moderate fine motor control. R could only be persuaded to sit in front of the screen for a maximum of 5 min at a time. He persistently got up and moved around the room to show his visitors various of his belongings and on three occasions went into his bedroom to gather more of his possessions. When asked if he could identify any of the icons for example “R where is the hamburger? Can you point to the hamburger?” he was unable to do so. He did not show any interest in any of the icons or give any response to the digitized speech and seemed unable to focus his attention on the screen for more than a very short time. A final attempt to elicit some interactive response was made by presenting R with the opening screen of EasySpeaker which contains a digitized picture of the second author. R correctly identified who, out of the six people in the room at the time, was shown in the picture. At this time R was not really in a state where he could be asked to be alone with just one experimenter.
Learning disabled
219
Session 2 After an initial expression of excitement, R remained seated in his chair in front of the screen for the entire session, and for the most part his attention was focused directly upon the screen. Indeed once his parents and the researcher with whom R was familiar left the room, R was not distracted from the screen at all. It took two to three trials in each case for R to learn the association between the pictorial representation (icon) and the speech or piece of music they represented. For example, when asked “Where is Kylie?” this could be correctly answered by selecting the relevant picture on the screen. When R was prompted with questions such as “What did you drink in the cafe when you were on holiday?’ R pointed to the icon which represented the speech “When I was on holiday for a week I went in a cafe and drank lots of cups of tea.” R also recognized the voice of the researcher used to produce the digitized speech, as, when the speech was produced he appeared excited and called out the researcher’s name. R quickly learnt that once he had identified an icon by touching the screen, it was followed by the computer operator pressing a combination of keys and speech was then produced from the speaker. After pointing to the screen, he pointed to the keyboard and then to the speaker-this pattern was repeated for the majority of trials. R could clearly identify all the pictorial representations. He could point to each picture when asked to do so, e.g. “R can you just point to the deck chair ?“, although he had to bring his face fairly close to the screen, possibly indicating some visual impairment. When the main menu was on the screen showing the links to the three screens at the next level down in the hierarchy, he learnt which icon provided the link to each screen. So when music was requested he would point to the Walkman schematic, the deck chair icon when holiday information was required and the present schematic when asked about Christmas. In most cases R had to be prompted to point to a particular icon by asking him a question and did not show any inclination to form a brief storyline by linking two icons together. However this was achieved if a second prompting question was given after one icon had been activated and R appeared to hear two icons in succession. However it is not possible to draw conclusions about the content of pairs of responses since these are more likely to be due to the computer operator’s biases rather than reflecting R’s desire to produce more sentence-like structures. In contrast, discrimination between the icons was evident on the music screen. R frequently made verbal requests for Kylie (though he did also make some verbal requests for the researcher’s voice). When the music screen was displayed the prompt question was non-directive-variations on “Who do you want to hear?” During the course of the session icons from the music screen were activated 35 times in total. Of these, 25 were activations of the piece of music by Kylie Minogue. Only seven times were any of the other five music items activated. To ensure that R had not learnt the position of the Kylie icon only, its place was altered and R continued to identify it correctly.
DISCUSSION
Session I Session one was essentially a baseline study which served a number of useful purposes. Firstly it provided R with a relaxed environment in which he could be introduced to the computer and the EasySpeaker software. His parents also received a full demonstration of what was involved, and introductions could be made. The flexibility of EasySpeaker is one of its most valuable features; enabling users to tailor it to the needs of particular individuals. For this reason it was important to assess R’s interest in and ability to interact with the program in its standard untailored form to provide a new base from which to assess the benefits of creating new files specifically for R. Although superficially this session appeared to be of no benefit to R although he did enjoy the visit, it provided information on which to base a second session. The most obvious fact was the unsuitability of the mouse. As an initial tool this appeared too complex. It also appeared that the screen icons were not recognisable to R even when they depicted items such as a cup, which
220
ANDREW ROSTRONrt al.
I
I
Fl
n piece of ausic
S-Y-
from
Shak~lpcrs
most r-t
Sister
The Rollins
Happy md
clap
dbu
-
Stacws
you knou it yul
hulds
-
Husic that I like listminq to
tly summer Holiday
in Scarbarm.k#l
things
i like
at
dwistmas
Fig. 2 (cuptim
on opposite page)
1+1-
Holidwr
221
Learning disabled
I_
in
the
I had lots and lots 04 lolli~ on hollidy
summnr
I ma& a big sand castle en the bead9
Fig. 2. Samples
I!Jmtoscubor~
in . CMd?
tbsic thmt I like listming to
IJhWlIlJUOnhOlidUJI drank lots at tea
of some of the EasySpeaker
screens constructed
for the present
project
the researcher’s past experience suggested R should recognize. This, taken together with positive recognition of digitized pictures of a larger size is suggestive of some visual deficits. The digitized, good-quality natural sounding speech which is an important aspect of EasySpeaker, was not met with the interest and enthusiasm hoped for. It had been expected that this would be stimulating and so attract R’s attention. Overall the whole package did not succeed in any significant way in attracting R’s attention or give any indication that R had any potential for using EasySpeaker. In short the prospects for making any progress with R seemed extremely bleak.
Session 2 The creation of customized material clearly made a vital contribution to R’s ability to interact with the computer system. In order to create new screens, in conjunction with accompanying speech and sound, specifically for R’s use, the third author used knowledge from the 2; y relationship with R. In particular it was noted that R uses only a limited number of words and tends only to learn words which represent a very positive meaning to him. Evidence of this is that these words and sometimes gestures are repeated consistently, often long after the event took place. This strategy was clearly effective and is consistent with current theories of HCI. For example Johnson [ lo] writes: “Understanding the structure of information both in the world and in the memory of a person is one of the most crucial aspects of ensuring a usable design.” A further example of the importance of the link between the structure of information in the external world and in the memory of the person is shown in the case of the response made by R to the third author’s digitized voice. This familiar sound stimulated R’s interest and he became excited over it. The voice was obviously recognized and R pointed to the third author and said her name. This together with the digitized recording of a piece of Kylie Minogue music, appeared to be the factors which initially aroused R’s attention. R’s initial requests were to hear this voice
222
ANDREW ROSTKON et (11
and the piece of music. It is worth noting that even at this stage R was distracted by the presence of the researcher and his parents. Once the room was empty of any familiar people or objects, attention was complete. This mode of response suggests that some basic elements of humancomputer interaction need to be established before the system is seen as a useful and potentially effective means of interaction and communication. “Foot in the door” and “key in the lock” are both apt metaphors to describe the type of experience that is being sought. The observations showing the distracting nature of others in the room point to the fragility of this initial link between man and machine. This also illustrates why great care should be taken should it be decided to record any first interactions. Any video cameras would need to be completely concealed. There is a similarity between university students and those with SLD when it comes to dealing with new learning environments. The provision of an immediate sensible response makes a great deal of difference when it comes to introducing psychology undergraduates to statistical packages. Many are easily put off if they cannot make sense of how the input commands have to be structured. Once an easy introduction is made available, many soon become enthusiastic about dealing with numbers. In one sense access to language and communication for those with SLD is rather like access to a difficult statistical package for the average psychology undergraduate, but once some simple means of access are available so that the results of any interaction are clearly seen, this can often release a surge of enthusiasm for learning with the subsequent acquisition of a number of useful skills. In general it was clear that R soon learnt the relationship between the icons on the screen and what they represented. Indeed there was evidence that an understanding of the structure of the information stored had been attained. In this second trial R was clearly highly motivated by the hypermedia concept which had a multiple appeal in that the text and icons were relevant to his experience, the digitized speech was that of a friend and he was also able to activate his favourite music. In this stimulating environment R attended well for the whole session of over an hour. There was no indication even at the end of the session that interest was fading. There is also some indication that R has some idea of how to access the different sections of the hypermedia structure. For example, the text on one of the icons from the holiday screen mentions Kylie Minogue: R recognized this. He was then able to identify the Walkman schematic which represented the link to the music screen. This ability could possibly provide a foundation for using the system for effective productive communication. The simulation of the touch screen was good evidence that, were a touch screen available, R would quite possibly have the ability to operate the system unaided for reasonable lengths of time. This is a highly valuable potential to provide respite for carers since it may provide a means of occupying users for some period of time. Another obvious application is as a tool to aid the learning of Makaton symbols. Another clue which indicates that use of EasySpeaker might be used to facilitate productive language was that R often attempted to elaborate on what was said as the speech output linked with an icon. For example when speech was related to a holiday, he elaborated that he had tea with Terry (a friend) on holiday. Future work could build up related screens in a hierarchy form related to R’s experience. Slightly less familiar concepts could then be gradually introduced. It may be that this physical manifestation of memory, which does not suffer from losses or distortions, could act as support to the user’s own memory. The hope is that the technology can provide a motivating and accessible mental equivalent of notebooks and drawings for those that have not got the intellectual skills to produce their own. The added benefit of technology in this case is that additional sound cues can be incorporated as necessary. For such a mental crutch to function it is necessary that the user has sufficient cognitive skills to be able to access and have some idea of the structures that have been built. In this instance it has been demonstrated a number of these skills are indeed evident. This may eventually lead to the ability to combine icons to form phrases. The overall picture that has emcrgcd from this short study is that this particular technology, in combination with carefully structured materials. has enabled R to demonstrate that he can function at a level at which it was previously thought he was unable to attain. Clearly much remains to be done to explore how the package can be used to develop linguistic and communication skills. but the clear-cut results with particularly unpromising clients, indicate that further work is likely to have benefits for a much wider population.
Learning
disabled
223
REFERENCES I. Dean A. and Hegarty 2. 3. 4. 5. 6. 7. 8. 9. 10.
S., Learning for Independence: Post-16 Educational Provision for People with Severe Learning DeJcits. Further Education Unit, London (1984). Low D. and Beukelman D. R., The use of microcomputer technology with persons unable to speak: an overview. Computers Human Behav. 4, 355-366 (1988). Clarke A. M. and Clarke A. D. 9. (Eds), Readings in Mental DeJiciency: The Changing Outlook. Methuen, London (1974). Hegarty S. and Dean A., Learning for Independence: Post-16 Educational Provision for People with Severe Learning Di&%ulties. NFER, Slough (1985). Heddell F. and Lee R., Educational television for handicapped people: the impact of Let’s Go. In Ref. [4], pp. 1044121 (1984). Wilkinson J. E. and Murray K., Wurnock Seven Years On-A Scottish Perspective. National Children’s Bureau, Scottish Group, Glasgow (1986). Cain E. J. and Taber F. M., Educating Disabled Peoplefor the 21st Century. Little, Brown & Co. Boston, Mass. (1987). Rostron A. 9. and Plant R., Finding a voice on disk. Br. J. Special Educ. 19, 136 (1992). Kiernan C. C., Reid 9. and Jones L., Signs and Symbols: Use of Non-vocal Communication Systems. Heineman Education, London (1982). Johnson P., Human Computer Interaction: Psychology, Task Analysis and Software Engineering. McGraw Hill, Berkshire (1992).
Pergamon
Printed in Great Britain. All rights reserved 0360-1315/94$6.00+ 0.00
HYPERMEDIA EXPENSIVE
FOR THE LUXURY
LEARNING DISABLED: OR USEFUL TOOL
ANDREW ROSTRON, RICHARD PLANT and CLAIRE HERMANN Department of Psychology, University of Hull, Hull HU6 7RX, England (Received 6 July 1993; accepted
14 September
[Fax:
+44 482 4655991
1993)
Abstract-This project was designed to assess and evaluate the possible applications of a particular piece of information technology for a young person with severe learning difficulties and severe communication problems. This single subject study was set up within the broader context of information technology for the disabled. The software used was EasySpeaker-a hypermedia speech prosthesis for the vocally and linguistically impaired, the key feature of which is flexibility to adapt to an individual’s needs. The results were encouraging, illustrating the value of relating software to an individual’s own experience. The potential was clearly demonstrated for software to provide a stimulating sensory environment in which an individual of low cognitive ability can access and learn the structure of a small purpose-built set of material. The findings of this research seriously challenge the common view that IT cannot be utilized by the severely learning disabled.
INTRODUCTION
The care of those with learning difficulties requires a great deal of resources. With the advent of care in the community policies and privately run, smaller residential facilities, the structure of this labour-intensive area is undergoing radical changes. In recent years there has at last been a welcome concentration on improving the quality of life for people with learning difficulties, particularly those with severe difficulties, who are often isolated from society. This paper is directed towards answering the question of whether information technology can play any significant role in these changes. There is little doubt that IT can be an invaluable tool in terms of administration-for example providing a nationally accessible network of information, regarding policy formation and regulation[l], but of more interest to the psychologists and educationalists is whether the learning opportunities offered by IT can be harnessed directly to meet the needs of the learning disabled. Some would argue that, although microcomputer technology is playing an increasing role in enhancing communication abilities[2], research should centre on the intellectually able, since the severely learning disabled do not possess the cognitive skills required to benefit from IT. In other words valuable resources allocated for the care of the learning disabled, should not be expended on an expensive luxury, from which it is far from certain any benefit may be derived. Unfortunately this apathetic view is encountered all too often in the field of severe learning difficulties. Change has been slow in coming and the establishment has very durable perceptions as to what can be achieved. For a review of the background see[3]. It was, for example, believed that the learning disabled would not appreciate travel, therefore rendering any investment in transport for them useless. Fortunately such views were overcome and considerable rewards, in terms of quality of life have been reaped. A further example is given by Hegarty and Dean [4], who state their findings illustrate that providers of education tend to make very limited assumptions as to what those with severe learning difficulties can achieve. Heddell and Lee[S] cite the results of an undated study by Spencer on the effect of tuition given to the learning disabled in conjunction with the BBC television series Let’s Go. Spencer’s results showed the use of media augmentation of the tuition had no significant motivational effect. Heddell and Lee claim this was due to a very small sample size and they themselves found exactly the opposite. Although attitudes are changing slowly, Wilkinson and Murray[6] found that in schools “The focus has shifted to children’s strengths . . . [which] can overcome learning difficulties.” Cain and Taber[7] believe the computer provides the ideal teaching accessory as it is truly interactive and 215
ANDREWROSTRON et al.
216
it “allows the matching of the students preferential learning modality with the method of receiving and expressing information.” The present study reports the data obtained from a severely learning disabled individual for whom a small stack of customized hypermedia was constructed and implemented on the speech synthesis system, EasySpeaker, developed by Rostron and Plant [8]. The purpose of this material was to (i) provide a means of assessing the user’s capabilities (ii) devise an environment where communication skills could (iii) offer a novel entertainment and recreation opportunity. The particular aim of the study was to gather be given to the following questions:
be developed
some data which could enable
some answers
to
(i) (ii) (iii) (iv)
can the user recognize and identify pictures and symbols? what aspects of language can be understood and responded to? are there any signs of productive language use? to what extent is the users attention held by the particular environment and how long does it remain of interest? (v) is the situation sufficiently motivating to warrant developing material for other specific learning tasks?
The following paragraphs give some idea of the technology that has been used in the present study. At its most simple level EasySpeaker can be regarded as consisting of a set of screens, stacked like a deck of cards, which the user can traverse by selecting pictures which signify jumps to different cards. When each screen, or card, is displayed the user can select either, ‘speech pictures’, and/or pictures which signify jumps to other related screens. Thus, by navigation of jumps and selection of speech pictures, users can build up sentences or whole dialogues which can then be output as real digitized speech. The source of the pictures themselves can either be drawings or photographs, which are then scanned into the machine. When displayed they are about passport size. To construct language learning environments, the carer has to do no more than select the name of a picture, the screen pigeonhole into which it will placed, and the name of the piece of digitized speech it represents, or the screen to which it is linked. The recording, or digitizing, of speech remains as simple, with carers simply selecting a microphone icon, speaking into the microphone and selecting a relevant name for the segment of speech. Since EasySpeaker records digitally, the stored sound can be music, or indeed any other noise, as well as speech. To track user’s actions, the system stores a record of where the user has been and what they have said. This record can then be viewed by carers once the user has finished their EasySpeaker session. For technology such as this to be acceptable to the caring community in general, it is vital that it is easy to understand and operate. The aim has therefore been to make it as simple for carers as for users, but this aspect of the technology needs to be the subject of further design and evaluation work. Figure 1 shows a screen taken from a real EasySpeaker session.
METHOD Two separate sessions were undertaken with the same subject second session took place 14 days after the first.
in two different
situations.
The
Subject Male aged 22 with severe learning difficulties, who will be referred to as R. R has not been formally assessed for several years, the last occasion was on refusal of entrance to an Adult Training Centre to receive employment related training. It was indicated at the time that R’s lack of communication skills was the major reason for refusal: it was also stated his potential to learn was minimal. R does have some basic monosyllabic speech, however this is limited to a range of basic
Learning
217
disabled
command forms and names of family members e.g. go home; come back; tea; cry; Daddy; Jean; Granny Ivy. R has been known to the third author for some 2: years in which time R has demonstrated an ability to learn to say a small number of new words, but has shown no evidence of being able to produce even short phrases as at the moment he can only use nouns and verbs in a limited manner and has no use of prepositions and conjunctions. R is in the early stage of being taught to use Makaton, the basic sign and symbol system [9] often used in an attempt to facilitate communication for people with severe learning difficulties. As yet there is no evidence that R is using Makaton in any way; however he does often use simple gestures, again a limited number, when trying to communicate. R is very excitable and requires (and demands) constant attention and stimulation, enjoys listening to music, watching music videos, looking at and showing people photographs. R has in the past had medical problems with the cornea of his eyes and has to be given eye drops regularly. It was assumed he had some sight defect but this had not been formally assessed. Session
I
This session took place in R’s home. The materials available for this session were general purpose screens, Pets I Like, and Food. The size of each icon was 2 x 3 cm. R was seated in front of the screen and asked to look at the pictures. The researcher then demonstrated how the mouse was used to move the cursor around the screen and to click on the icons. R was asked to attempt this for himself. A sentence was then produced on screen by the experimenter using the mouse. This sentence was then spoken by the machine. This process was repeated several times. R was again asked to use the mouse himself. R found it difficult to attend to the task and was allowed to move from the computer more or less on demand. The aim was to let him collaborate with the researchers as often as possible. The session was ended after about an hour.
Richard R Plmt TheauthaofEm#+mkw
n ,
I
lhings I like to eat
Fig.
I. A sample
screen
grab
from
r L
6mwd
EasySpeaker. The set of smaller frequently used words or phrases.
wards
icons on the left is available
for
218
ANmEW RosrRoFi et al. Session 2
Software
A hypertext stack was prepared based on responses made in session I and from personal knowledge of R gained over the previous 24 years, relating to his interests, likes and dislikes. The first file was based on a trip to the seaside, the second a small selection of musical extracts represented by icons. The music file contained one icon representing Kylie Minogue, of whom it is known that R is a great fan, and five other icons representing five musical extracts with which R was not familiar. The third was based on Christmas. The size of the pictures was increased to about 4 x 4 cm. The digitized speech to be used in conjunction with the files was spoken by the third author, with whom R is familiar. Some sample screens of the material are presented in Fig. 2. The means of interaction for R was via a simulated touch screen. The way this worked was that R was requested to touch a particular icon and a researcher operated the relevant control once the instruction had been carried out. Procedure
This session took place in a quiet bare room. The only contents of the room apart from the computer system were some chairs and tables. To begin with-for R to feel secure in a strange setting-his parents, and the researcher, whom he knows well, were present along with the computer operator/researcher. R was initially seated in front of the computer screen, the new files were demonstrated as was each icon, the text and digitized speech associated with it. For the remainder of the session R was allowed to freely explore the range of material available, although some initial guidance was given by the computer operator. For example R was told that he only had to touch the screen at his chosen icon to activate the text and digitized speech e.g.” If you want to hear Kyhe point to the picture of Kylie on the screen.” As time progressed the number of prompts was reduced. At a later stage in the session R was asked questions about the material and some simple conversation was attempted. Once R appeared to be comfortable with the situation, his parents and the researcher left the room-their presence was felt to be distracting as R repeatedly turned towards them to indicate his excitement at the speech or music produced. It was also thought that the digitized speech of the researcher would have greater reinforcement value when the researcher was not present. The whole session lasted about If h and a record of all interactions with the system was obtained from the tracking system incorporated in the software.
RESULTS Session
I
Initially R was very excited at the researcher (who had been a friend of his for some time) and two new visitors arriving at his home, but became quieter as the equipment was set up. The software was demonstrated to R but his attempts at using the mouse were repeatedly unsuccessful. In most instances this was due to his clicking both keys of the mouse simultaneously, since he has only moderate fine motor control. R could only be persuaded to sit in front of the screen for a maximum of 5 min at a time. He persistently got up and moved around the room to show his visitors various of his belongings and on three occasions went into his bedroom to gather more of his possessions. When asked if he could identify any of the icons for example “R where is the hamburger? Can you point to the hamburger?” he was unable to do so. He did not show any interest in any of the icons or give any response to the digitized speech and seemed unable to focus his attention on the screen for more than a very short time. A final attempt to elicit some interactive response was made by presenting R with the opening screen of EasySpeaker which contains a digitized picture of the second author. R correctly identified who, out of the six people in the room at the time, was shown in the picture. At this time R was not really in a state where he could be asked to be alone with just one experimenter.
Learning disabled
219
Session 2 After an initial expression of excitement, R remained seated in his chair in front of the screen for the entire session, and for the most part his attention was focused directly upon the screen. Indeed once his parents and the researcher with whom R was familiar left the room, R was not distracted from the screen at all. It took two to three trials in each case for R to learn the association between the pictorial representation (icon) and the speech or piece of music they represented. For example, when asked “Where is Kylie?” this could be correctly answered by selecting the relevant picture on the screen. When R was prompted with questions such as “What did you drink in the cafe when you were on holiday?’ R pointed to the icon which represented the speech “When I was on holiday for a week I went in a cafe and drank lots of cups of tea.” R also recognized the voice of the researcher used to produce the digitized speech, as, when the speech was produced he appeared excited and called out the researcher’s name. R quickly learnt that once he had identified an icon by touching the screen, it was followed by the computer operator pressing a combination of keys and speech was then produced from the speaker. After pointing to the screen, he pointed to the keyboard and then to the speaker-this pattern was repeated for the majority of trials. R could clearly identify all the pictorial representations. He could point to each picture when asked to do so, e.g. “R can you just point to the deck chair ?“, although he had to bring his face fairly close to the screen, possibly indicating some visual impairment. When the main menu was on the screen showing the links to the three screens at the next level down in the hierarchy, he learnt which icon provided the link to each screen. So when music was requested he would point to the Walkman schematic, the deck chair icon when holiday information was required and the present schematic when asked about Christmas. In most cases R had to be prompted to point to a particular icon by asking him a question and did not show any inclination to form a brief storyline by linking two icons together. However this was achieved if a second prompting question was given after one icon had been activated and R appeared to hear two icons in succession. However it is not possible to draw conclusions about the content of pairs of responses since these are more likely to be due to the computer operator’s biases rather than reflecting R’s desire to produce more sentence-like structures. In contrast, discrimination between the icons was evident on the music screen. R frequently made verbal requests for Kylie (though he did also make some verbal requests for the researcher’s voice). When the music screen was displayed the prompt question was non-directive-variations on “Who do you want to hear?” During the course of the session icons from the music screen were activated 35 times in total. Of these, 25 were activations of the piece of music by Kylie Minogue. Only seven times were any of the other five music items activated. To ensure that R had not learnt the position of the Kylie icon only, its place was altered and R continued to identify it correctly.
DISCUSSION
Session I Session one was essentially a baseline study which served a number of useful purposes. Firstly it provided R with a relaxed environment in which he could be introduced to the computer and the EasySpeaker software. His parents also received a full demonstration of what was involved, and introductions could be made. The flexibility of EasySpeaker is one of its most valuable features; enabling users to tailor it to the needs of particular individuals. For this reason it was important to assess R’s interest in and ability to interact with the program in its standard untailored form to provide a new base from which to assess the benefits of creating new files specifically for R. Although superficially this session appeared to be of no benefit to R although he did enjoy the visit, it provided information on which to base a second session. The most obvious fact was the unsuitability of the mouse. As an initial tool this appeared too complex. It also appeared that the screen icons were not recognisable to R even when they depicted items such as a cup, which
220
ANDREW ROSTRONrt al.
I
I
Fl
n piece of ausic
S-Y-
from
Shak~lpcrs
most r-t
Sister
The Rollins
Happy md
clap
dbu
-
Stacws
you knou it yul
hulds
-
Husic that I like listminq to
tly summer Holiday
in Scarbarm.k#l
things
i like
at
dwistmas
Fig. 2 (cuptim
on opposite page)
1+1-
Holidwr
221
Learning disabled
I_
in
the
I had lots and lots 04 lolli~ on hollidy
summnr
I ma& a big sand castle en the bead9
Fig. 2. Samples
I!Jmtoscubor~
in . CMd?
tbsic thmt I like listming to
IJhWlIlJUOnhOlidUJI drank lots at tea
of some of the EasySpeaker
screens constructed
for the present
project
the researcher’s past experience suggested R should recognize. This, taken together with positive recognition of digitized pictures of a larger size is suggestive of some visual deficits. The digitized, good-quality natural sounding speech which is an important aspect of EasySpeaker, was not met with the interest and enthusiasm hoped for. It had been expected that this would be stimulating and so attract R’s attention. Overall the whole package did not succeed in any significant way in attracting R’s attention or give any indication that R had any potential for using EasySpeaker. In short the prospects for making any progress with R seemed extremely bleak.
Session 2 The creation of customized material clearly made a vital contribution to R’s ability to interact with the computer system. In order to create new screens, in conjunction with accompanying speech and sound, specifically for R’s use, the third author used knowledge from the 2; y relationship with R. In particular it was noted that R uses only a limited number of words and tends only to learn words which represent a very positive meaning to him. Evidence of this is that these words and sometimes gestures are repeated consistently, often long after the event took place. This strategy was clearly effective and is consistent with current theories of HCI. For example Johnson [ lo] writes: “Understanding the structure of information both in the world and in the memory of a person is one of the most crucial aspects of ensuring a usable design.” A further example of the importance of the link between the structure of information in the external world and in the memory of the person is shown in the case of the response made by R to the third author’s digitized voice. This familiar sound stimulated R’s interest and he became excited over it. The voice was obviously recognized and R pointed to the third author and said her name. This together with the digitized recording of a piece of Kylie Minogue music, appeared to be the factors which initially aroused R’s attention. R’s initial requests were to hear this voice
222
ANDREW ROSTKON et (11
and the piece of music. It is worth noting that even at this stage R was distracted by the presence of the researcher and his parents. Once the room was empty of any familiar people or objects, attention was complete. This mode of response suggests that some basic elements of humancomputer interaction need to be established before the system is seen as a useful and potentially effective means of interaction and communication. “Foot in the door” and “key in the lock” are both apt metaphors to describe the type of experience that is being sought. The observations showing the distracting nature of others in the room point to the fragility of this initial link between man and machine. This also illustrates why great care should be taken should it be decided to record any first interactions. Any video cameras would need to be completely concealed. There is a similarity between university students and those with SLD when it comes to dealing with new learning environments. The provision of an immediate sensible response makes a great deal of difference when it comes to introducing psychology undergraduates to statistical packages. Many are easily put off if they cannot make sense of how the input commands have to be structured. Once an easy introduction is made available, many soon become enthusiastic about dealing with numbers. In one sense access to language and communication for those with SLD is rather like access to a difficult statistical package for the average psychology undergraduate, but once some simple means of access are available so that the results of any interaction are clearly seen, this can often release a surge of enthusiasm for learning with the subsequent acquisition of a number of useful skills. In general it was clear that R soon learnt the relationship between the icons on the screen and what they represented. Indeed there was evidence that an understanding of the structure of the information stored had been attained. In this second trial R was clearly highly motivated by the hypermedia concept which had a multiple appeal in that the text and icons were relevant to his experience, the digitized speech was that of a friend and he was also able to activate his favourite music. In this stimulating environment R attended well for the whole session of over an hour. There was no indication even at the end of the session that interest was fading. There is also some indication that R has some idea of how to access the different sections of the hypermedia structure. For example, the text on one of the icons from the holiday screen mentions Kylie Minogue: R recognized this. He was then able to identify the Walkman schematic which represented the link to the music screen. This ability could possibly provide a foundation for using the system for effective productive communication. The simulation of the touch screen was good evidence that, were a touch screen available, R would quite possibly have the ability to operate the system unaided for reasonable lengths of time. This is a highly valuable potential to provide respite for carers since it may provide a means of occupying users for some period of time. Another obvious application is as a tool to aid the learning of Makaton symbols. Another clue which indicates that use of EasySpeaker might be used to facilitate productive language was that R often attempted to elaborate on what was said as the speech output linked with an icon. For example when speech was related to a holiday, he elaborated that he had tea with Terry (a friend) on holiday. Future work could build up related screens in a hierarchy form related to R’s experience. Slightly less familiar concepts could then be gradually introduced. It may be that this physical manifestation of memory, which does not suffer from losses or distortions, could act as support to the user’s own memory. The hope is that the technology can provide a motivating and accessible mental equivalent of notebooks and drawings for those that have not got the intellectual skills to produce their own. The added benefit of technology in this case is that additional sound cues can be incorporated as necessary. For such a mental crutch to function it is necessary that the user has sufficient cognitive skills to be able to access and have some idea of the structures that have been built. In this instance it has been demonstrated a number of these skills are indeed evident. This may eventually lead to the ability to combine icons to form phrases. The overall picture that has emcrgcd from this short study is that this particular technology, in combination with carefully structured materials. has enabled R to demonstrate that he can function at a level at which it was previously thought he was unable to attain. Clearly much remains to be done to explore how the package can be used to develop linguistic and communication skills. but the clear-cut results with particularly unpromising clients, indicate that further work is likely to have benefits for a much wider population.
Learning
disabled
223
REFERENCES I. Dean A. and Hegarty 2. 3. 4. 5. 6. 7. 8. 9. 10.
S., Learning for Independence: Post-16 Educational Provision for People with Severe Learning DeJcits. Further Education Unit, London (1984). Low D. and Beukelman D. R., The use of microcomputer technology with persons unable to speak: an overview. Computers Human Behav. 4, 355-366 (1988). Clarke A. M. and Clarke A. D. 9. (Eds), Readings in Mental DeJiciency: The Changing Outlook. Methuen, London (1974). Hegarty S. and Dean A., Learning for Independence: Post-16 Educational Provision for People with Severe Learning Di&%ulties. NFER, Slough (1985). Heddell F. and Lee R., Educational television for handicapped people: the impact of Let’s Go. In Ref. [4], pp. 1044121 (1984). Wilkinson J. E. and Murray K., Wurnock Seven Years On-A Scottish Perspective. National Children’s Bureau, Scottish Group, Glasgow (1986). Cain E. J. and Taber F. M., Educating Disabled Peoplefor the 21st Century. Little, Brown & Co. Boston, Mass. (1987). Rostron A. 9. and Plant R., Finding a voice on disk. Br. J. Special Educ. 19, 136 (1992). Kiernan C. C., Reid 9. and Jones L., Signs and Symbols: Use of Non-vocal Communication Systems. Heineman Education, London (1982). Johnson P., Human Computer Interaction: Psychology, Task Analysis and Software Engineering. McGraw Hill, Berkshire (1992).