Computer-assisted teaching and learning in medicine

Int JBiomed Comput, 26 (1990) l-27 Elsevier Scientific Publishers Ireland Ltd.

Review Article

COMPUTER-ASSISTED TEACHING AND LEARNING IN MEDICINE

R. KLAR and U. BAYER Abteilung Medizinische Informatik, Albert Ludwigs Universittit. D-7800 Freiburg, Stephan-MaierStrasse 26 (F.R. G.) (Received January 8th, 1990) (Accepted February 2Oth, 1990)

Induced mainly by the increased spreading of personal computers in the last few years computerassisted instruction (CAI) systems for medicine have been developed on a large scale. Proven structure principles are above all the simulation of patient management in a problem-orientated approach, the mathematical simulation of (patho-) physiological functions independent of particular patients and the separation of educational mode and scoring mode. There exists already a large choice in programs dealing with topics of internal medicine - especially cardiology - while operative disciplines are less represented so far. Programs accredited in the US for continuing medical education (CME) are usually of high quality as to medical contents. Other important quality criteria to be mentioned concerning simulation programs are algorithms of medical decision making, completeness and refinement of the medical knowledge base, software design and user interface. CA1 is a unique tool to enhance clinical problem solving skills although - of course - it can by no means replace bedside teaching. Keywords: Computer-assisted systems; Quality criteria.

instruction; Simulation; Learning software; Medical education; Expert

1. Introduction For the last 30 years, people have been trying to use computer technology to pass on knowledge or skills. In the beginning the results were not very attractive, as the programs were low in flexibility. What computer-assisted instruction (CAI) mainly did was direct straight forward presentation of the topics being taught. However in the early seventies, above all in pedagogics and university didactics, great expectations arose concerning computer based instruction. There was hope for considerable ameliorations and facilitations both for teaching as well as for learning. Only few of these dreams came true at that time. It had been underestimated how time consuming it was to create valuable programs and how difficult to integrate expert knowledge into them. Sufficient technical facilities for a broad use of CA1 just did not exist. The time between 1977 and 1983 revealed little that was new. The year 1984, however, saw an explosion of activity that continues to this day. Since then, things have developed at a terrific speed and decisive changes have taken place. Undoubtedly the decisive reason for heightened activity was the event of the personal computer. Now PCs have become comparatively cheap and are 0020-7101/90/$03.50 0 1990 Elsevier Scientific Publishers Ireland Ltd. Published and Printed in Ireland

8

therefore widespread. Today hardware performance renders fascinating multimedia scenarios feasible. However, in each individual case it will have to be decided whether it is really sensible to make use of the whole range of technical possibilities. Often, it will just be too costly in relation to the learning improvement achieved. Programming languages have become more flexible and new programming tools enable even non-experts in computer science to create software of good quality in their fields of knowledge. Finally, methods of artificial intelligence help the programs to gain knowledge about the users’ strategies, his speed and style of learning (student model) so they can adapt to him as far as possible. Virtually all medical schools in the US now employ the computer at least to some extent in their educational enterprise [ 11. Some of the traditional obstacles for the development of high quality learning software still exist. Software industry e.g. realistically does not regard medical education as a very lucrative area and faculty members at medical schools trying to create new programs cannot really count on internal funding, promotions and proportionate salary increases as teaching endeavors are still not rewarded the way research or clinical activities are. Therefore, there are still not enough high quality programs but too many that must be characterised as electronic page turners or relashes of paper pencil simulations [2]. CBT (computer-based teaching) programs are usually available on diskettes for the personal computer. Often they have been produced in university institutions. The development in informatics as well as in medicine is proceeding at a breathtaking speed so that it may occur that one particular product is already outof-date before it is ready to appear on the market. Many programs therefore constantly improve their quality and keep up with the progress in science which is fortunately much easier than updating textbooks. (This is one of the crucial points of these means of transmitting information: updating is easy and comparatively inexpensive.) Personal computers and word processors inevitably outdated the conventional typing machines, for well known reasons. Good CA1 programs might as well prove superior to some parts of conventional methods of passing on information in the near future. 2. Problems in Giving an Overview It is intended here to give a critical overview about programs which have been designed to improve learning in medical professional training or continuing medical education. Such an analysis poses problems, some of which are not easily solved. Nevertheless, this must not let us forget, that many software products of remarkable quality already do exist and are ready for use. The first problem is the incompleteness. There is no obligatory registration of software for learning purposes (like there is in F.R.G., e.g. for schoolbooks) and even concerning some of the good, well-tried products, that have been in use for years, there has been nothing published. Many flourish in some hidden place, while others have already faded and are not available anymore. So on this short-lived busy market it can not be avoided, that even important software is overlooked. The second problem is the inaccuracy of the information collected about the software programs. Unfortunately, an internationally accepted scientific institution,

9

validating the whole collection of programs, does not exist. This problem is partly solved by the distribution of so called CME-credits (continuing medical education) by the Accreditation Council of the American Medical Association. Software offering CME-credits can be considered to be thoroughly validated. Often the information about one particular product is merely derived from the advertising literature of their distributors or from what somebody said about it at an international symposium. The main publications used for this overview are a buyers guide [3], a catalog about IBM-university collaboration in the field of learning software [4] and a recent symposium about medical informatics and education [5]. The third problem is the subjectivity of any judgement on the programs. Whether it is factually correct is not the only thing that matters. What is also important is the question from what point of view it is being judged. While students e.g. might be keen on programs helping them pass their exams, clinicians might insist on the program’s ability to directly improve the user’s performance in hospital everyday life. 3. Principles in the Structure of CA1 Programs Media didacticians have established theories and general construction plans applicable to CA1 programs. Although they are able to provide stimulating ideas at times, they differ from one another considerably and it does not seem likely that there will ever be generally accepted construction principles. However, some of the programs are part of a whole series using the same constructional frame. Examples are the RX-DX-series of exercises in clinical problem solving produced in the laboratory of computer science of the Massachusetts General Hospital by Hoffer and Barnett [6] (distributed by Williams and Wilkins, Table I). Another example is the CYBERLOG-series by Cardinal Health Systems inc. (Table I) or the MAC-series by IRL Press (Table I) (parts of the MAC-series are unfortunately quite awkward to use, in spite of the fact that it has been available for many years and has undergone numerous updatings). Schuhneister [7] made the following list of possible didactical functions (Fig. 1). Applied to medicine, besides the lively presentation of the topic, there is no way around stressing drill and practice. Compared to the tutorial mode or the simulation of clinical encounters this is certainly the simplest mode of instruction in the hierarchy of CA1 but unfortunately, in medicine some considerable amount of learning by heart of facts, terms, proper names etc. cannot be avoided no matter how hard one tries to decrease that kind of learning by trying to understand the underlying patterns. Many effects or side effects of drugs e.g. need to be memorised although they are not really understandable. Medicine is a practical field of knowledge and so drill and practice will keep playing an important part in the programs. On the other hand it is clear that rote memorization of proliferating facts is condemned to be of declining importance in the future. Between 6000 and 7000 scientific articles are published each day in the world and the scientific database is doubling every 5.5 years [l]. It is simply not possible to infuse a bulkpart of the biomedical knowledge into a student. One sensible way to face this ocean of information is to teach more and more problem solving and decision making skills by special CA1 as well as information retrieval.

10 TABLE I INFORMAL LIST OF ADDRESSES OF DISTRIBUTERS OF SOME LEARNING SOFTWARE AND SYSTEMS MENTIONED IN THIS ARTICLE Course Builder, Mac MED, Hypertext, -card, etc.: Apple Computer GmbH Marketing Ingoldstadter Str. 20 8000 Milnchen 45

Cyberlog Lernsoftware Serie: CMS Biomedical Verlag Belforstrasse 8 8000 Milnchen 80

Pro. M.D., INTERNIST, PDQ DIAGNOSIS: Thieme Verlag/Medisoft Rtidigerstrasse 14 7ooo Stuttgart 30

Hoffer and Barnett Clinical Problem Solving Courseware: Williams and Wilkins P.O. Box 1496, USA Baltimore MD 21203-9990

DXTER, LTS 90 etc. IBM, Fachbereich Lehre und Forschung Postfach 800980 8000 Mtlnchen 80

MAC Serie: IRL Press Limited P.O. Box 1 Eynsham Oxford OX8 lJJ, U.K.

AIMS etc. produced by Micro Med Distributed by The Norweigan Datasecretariat Ministry of Education and Research P.O. Box 8183 Dep. 0034 Oslo 1 Norway

- information - orientation - drill and practice - illustration - visualisation - construction - modelling - simulation

- measuring - aoimatioo

Fig. 1. Didactical functions of learning software according to Schulmeister [2] with their focal points applied to medicine.

11

Besides, animated graphics and other means of visualisation are especially valuable items for the learning tasks of a physician. In contrast to non-medical fields of knowledge, problem-orientated learning is of outstanding importance in medicine. This approach to learning by solving problems or case studies was first realised in CA1 by the Canadian McMaster University (Mac series by IRL). Today it appears in a bulkpart of the programs in some form. It has been considerably improved by Hasman [8,9] who integrated methods of medical informatics and statistics into it. This problem-orientated approach can be either focused on, or independent of the patient, in the latter case covering basic medical knowledge. Often both components are part of one and the same program as is the case in some of the series mentioned above. Parts of the programs focused on patients mostly refer to specific diseases, complaints, therapies and complexes of symptoms, that are revealed step by step in the dialogue with the learner. Usually, the programs initially present quite typical clinical problem solving situations giving access on the screen to history, findings, current medication, the results of physical or laboratory examinations and the like. With growing experience, the systems often offer more complicated or unusual clinical settings. This kind of learning from typical patient examples occupies a broad space in medical CA1 programs. Some systems even consist entirely of a collection of case studies such as Advanced Clinical Problems (distributed by Williams and Wilkins) or the cancer treatment records and therapy protocols PDQ (distributed in the F.R.G. by Medisoft - Table I); they are well usable for learning although originally not designed for this purpose. The interactive learning in these case studies does not only lead to the proper diagnosis but also shows reasonable ways of choosing diagnostic or therapeutic items, often allows a glance on the future devolopment of the disease in the patient or his recovery or gives prognostical and epicritical information. Most notable are the series of sinulations called DISCOTEST (Scientific American Inc, New York), the RX-DX-series of simulations developed at the Massachusetts General Hospital by Hoffer and Barnett (available for $95 each set of about 30-50 cases via Williams and Wilkins, Table I) and above all the simulation model known as Computer-Based Examination CBX [lo], a result of nearly 20 years research by the National Board of Medical Examiners. It is almost certainly the most complex simulation model that exists. CBX has more than 2000 types of treatment orders in its memory which is quite different from most simulation systems which have a somewhat limited 50-lOO-item list. Orders can be made in free text, the patient responds to treatment in a realistic manner, realtime is simulated and the computer is linked to a videodisk so that candidates can interpret blood smears, radiographic images etc. to manage the patient properly. More than 200 CBX cases have been developed so far [I]. The better ones of patient-focused programs provide the user with the option to model his own case by putting in the data of his particular patient such as age, sex, weight, medication, test results and so on (Fig. 2). The software then reaches expert system character which will be dealt with later. Often other useful components are diagnostic and epidemiological morbidity indices (they usually refer to US-American population). Diagnostic indices describe the efficacy of tests, signs and symptoms according to sensitivity and specifity and in a

12

patient short description of the situation, complaints problems, first findings I

computer anamnesis, treatment

as preexisting

.

simulation

0f

examination,

case

medical explanations, for the the programs

case freely generated by the user

reasons behaviour

-

via fundamental medical knowledge, independent of a particular patient: anatomy, physiology, pathophysioiogy ... diagnostic techniques, therapy methods, pharmacodynamics ...

via morbidity indices: incidence, prevalence, sensitivity, specificity, risk-factors, utilites

Fig. 2. Structure scheme of patient-focused learning software.

weighted form as utilities or costs. From patient data bases probabilities can be estimated out of the frequency of the disease in the examined population (prevalence), about the rate of the newly acquired diseases (incidence) or about the prognosis (survival time, progress of recovery etc.). Methodically, of special interest are the supplementations towards the theory of medical decision making as offered by Hasman [B] (predictive values, decision trees, ROC-curves, etc.). Programs, or parts of the programs independent of a particular patient deliver theoretical background of medical basic knowledge in an interactive format, e.g. physiological or pathophysiological relations or functions, anatomical conditions and so on up to very specific and complex knowledge from all relevant medical and natural science fields. This kind of information independent of particular patients is often presented in the form of tutorials, that use besides the normal texts of ques-

13

tion-answer dialogue graphics, animated graphics, pictures, sound, sequences of pictures etc. [11,12]. No matter whether the programs are patient-dependent or not, there are often two kinds of dialogue realized. As the learner has to first aquire knowledge, the dialogue will be in a learning mode in the beginning. That means, that there will be a maximum of explanation and background information, there will be comments to the learners mistakes and depending on his choices and answers the program will branch out individually.

I. Learning mode - individual learning - free choice of the way through the program - help, comments about mistakes - explanations Usually there is also a scoring mode available in order to measure the results of the learning effort. Of course, there is no help and no explanation given in this test mode; often the time needed to answer a question is paid attention to by the scoring module and sometimes the learner can compare his performance with the average of other candidates.

2. Scoring mode - examination, questions - control of the grasp of the learning objectives - attention to the time needed for answering - scores As already mentioned, North American programs often provide a special kind of scoring mode by offering the above mentioned CME-credits to those answering correctly on a post-test-questionnaire. 4. Distribution of Software Throughout the Various Fields of Medical Specializations What is available for whom? That is not an easy question to answer due to the increasing amount of software and the already mentioned problem of making judgements about them. We have considered about 150 programs and Fig. 3 shows how they are distributed. About 20% of the software is concerned with General Medicine. According to the predominant role of case-studies, and just as in real patient management, more than one field of specialisation is touched. They deal with clinical settings that general practitioners might come across. Besides, about 30% of the software is concerned with Internal Medicine, more than half of which is dedicated to cardiology. Pediatry, emergency medicine, and pharmacology/toxicology are also already well represented. However, there are comparatively few surgical programs. They often have high demands on the quality of graphics and make generous use of the new media such as

14

general medicine 20 96

other fields of specialimtion 20 % biochemistry, Iaboratry medicine 4%

cardiology 16 %

orthopedics 5 % toxicology/pharmacology

6 96

emergency medicine 6 96 pediatry 8 96 Fig. 3. Distributionof the availablelearningsoftwarefor medicaleducationover the various fields of internal medic‘ine 1

specialization (n > 140).

interactive video. Although small in number, there are some outstanding surgical programs, e.g. MAC APPITM and MAC SWITCHTM developed by J. Weinstein in Cleveland for Macintosh computers [ 131. He put excellent demonstrations of appendectomy (Mac APPI) and the correction of transposition of the great arteries onto an interactive videodisk using HYPERCARD and SUPERCARD as authoring languages. The various steps of the operation can be chosen and looked at separately and at the same time much oral and textual explanations are available. So without loosing valuable time by being forced to look at already well understood sequences, the user can watch the points of the OP that are crucial to him individually just before or after seeing the OP live, for example. This is a really sensible use of the new media. The production of more videodisks in this format is being planned by Weinstein. As a typical example of software for medical learning, Fig. 4 shows a schedule of the underlying decision tree of one issue of the CYBERLOG series dealing with diabetes type II. Keeping in mind that it is covering just one single disease, it demonstrates an already remarkable degree of complexity. Although merely text and animated graphics are used, this is a good, partly interactive program, consisting of tutorials, tools and case studies. As far as the learning objectives are concerned, in the existing software there are two focal points. 1. The passing on of knowledge, that can be derived form studying typical patient cases in a simulation, providing drill and exercice concerning: (a) (b) (c)

diagnostic decision making choice of appropriate items to narrow the differential diagnosis indication of various therapies

2. The passing on of knowledge about complex physiological or pathophysiological functions and intercorrelations using the computers unique ability to deal with and to demonstate cybernetic models.

I

J-

I

Pour qualion

at eachstage: I. Immediateinsulin therapy rueded? 2. Adequate glucose colufol achieved? 3. Pqpwab&gmadd 4. Puther improvemmr possible?

Self blood gkn-ose monitoring Monitoriw glcontrd Gluco8yhted lwlTlv&obin test #lb Alc) Associated risk facto0

I

r I Hypertension-
>-Hyperlipidemia

I

I

I

SmoLiry

I

I

I

Ii ~Newopathy’

I $ggy itancm

I

I Kidney

-Protein& -Edema alai failure

Heart

Brain

Extremit*r

-c~nuv atcry disease

-Stroke

.awdiatiotl

Peripheral -Pain -Numbness

Autonomic -&troiitestid dydunction -8ladde dydwtion -1xhywdia -.Sud&n death

Fig. 4. Basic structure scheme of a learning software dealing with diabetes type II (part of the CYBERLOG-learning software-series) to demonstrate the considerable complexity of decision trees hidden in many programs.

16

Apart from these aims for learning CA1 has a few useful aspects for the teaching staff as well. 1. Rationalization of teaching, from never ending repetition 2. Amelioration of the quality carefully selected examples graphics while lecturing

facilitation of the preparations for the teaching, ease of the same subjects with changing student groups. of the teaching, e.g. by making use of the often most appearing in CA1 programs or by using animated

However, it remains doubtful whether computer patient simulations can prevent medical teachers in hospital from always being aware of appropriate patients for their students. Personal contact between medical students and the patients can by no means be simulated because the thing that matters in these contacts is the improvement in history taking and performing the skills of physical examination with all the psychological aspects involved in it! The performance of parts of the physical examination can possibly be improved by CAL One example is a program called ‘HeartLab’ 114,151, designed for the Macintosh computer teaching medical students the essentials of the auscultatory cardiac exam by using the mouse as stethoscope (distributed by Williams and Wilkins, Table I, about $150). Even such a program, however, cannot replace the auscultation of real patients. What it can do is to render these efforts much more effective. 5. Methods of Medical Informatics and New Media Applied to CA1 Which of the methods developed by medical informatics are used to achieve the above mentioned aims and what kind of technical devices are available to help? 5.1. Software design and methods of informatics The transformation of decision trees for diagnosis and therapy into program plans and computer programs is the first problem to conquer. Part of that work is the direct programming of therapy patterns, decision tables, as well as the input of individual reaction to the learners performance in multiple choice questions. They are usually presented on the screen in the form of menus to choose from. The individual learning objective can be chosen from a hierarchy of menus and they also play a decisive part in interactive learning and testing dialogue. Another important form of software design is the already mentioned simulation of a patient case. This often includes the use of decision trees. In the huge collections of cases such as the PDQ cancer therapy protocols (Table I) but also in the CYBERLOG-series pointers or index tables are used to put into relation on the one hand findings (signs, symptoms, results of history taking, physical or laboratory examinations etc.) and diagnoses or therapeutic categories on the other hand, just as it is done in databases. Most of the case simulations are retrospectively constructed, e.g. a review of a completed case study is presented. Some of them involve large scale retrieval in huge databases. To do so, authors have at their disposal a highly developed retrieval language and an extensive thesaurus of medical terms, e.g. of the

17

National Library of Medicine [16]. Other case simulations are designed in a prospective format. They try to simulate typical physician-patient-situations. In the beginning the computer might offer only a little information like e.g. “male, age 60 years, presents with severe distress at your office with a pain in the abdomen, BP 140/90, P %, regular”. The learner, playing the part of the examining physician, is now supposed to take history, ask for the results of physical or laboratory examinations, order therapeutic items and so on. Good and costly manufactured systems involve the new media mentioned below and provide a realtime physician-patient simulation with the option of recording and judging the learners time efficiency in his workup. Other kinds of simulation programs for learning deal with physiological models, the simulation of functions, cascades, pathophysiology, pharmacodynamics etc.. Usually, what is applied here, is the simulation of steady processes by continuous mathematical functions or the simulation of discrete procedures with the help of special programming languages or program systems designed to develop simulations. The above mentioned Mac series (Table I) of the Mac Masters university and of the London Bartholomews Hospital provides typical examples of simulations, but some of these models are obsolete or not well accepted any more. New clinical skillbuilding simulations like HeartLab and EkgLab [14,15] use improved physiological models and special simulation languages. Cybernetic models of automatic control systems and closed control loops in the human body, compartment models of organ function and so on were developed in medical informatics very early and they are of good use for teaching purposes [17]. Induced by the problems arising from AIDS-pandemy, there has been a world-wide increase in attention to epidemiological models about the spreading of infectious diseases and they have already found their way into the programs for learning; one example is AIMS by Micro Med. (Table I) dealing with AlDS and other infectious diseases. Another method of medical informatics used is the integration of the workup of measurement data into the learning programs. The methods of biosignal and laboratory data processing are utilised, e.g. in CA1 about ECG or EEG analyses or about the interpretation of laboratory data. 5.2. Authoring systems Authoring systems or teachware are computers or programs especially made for the design of software for learning. According to Renscbler [18] there is an abundance of more than 200 of such products and some of them have only been used once for the production of just one software program so that they are often scarcely transferable. That is why some possibly good authoring systems will not be mentioned here but others must be referred to, as they play an important part in the understanding of software for learning. Using Coursebuilder for Macintosh computers e.g. one can build up programs while profiting from the well proven user friendliness of Apple soft- and hardware with its ikon and window technique. In this teachware special schedules for the construction of decision trees are included. Programming skills, in the conventional meaning of the word, are no longer necessary. One can easily produce graphics and

18

animate, store and integrate them into the learning dialogue mainly by choosing from menus or constructing link-arrows between various frames on the screen. Scanned text or pictures can be worked on, stored and retrieved. This seems to be a system that requires remarkably little time to get acquainted with and allows to build up courses within a few weeks. HYPERCARD, and its extended new (colour) version SUPERCARD, can be used as authoring systems for Apple computers as well, supporting an independent style of learning. They build up a good surrounding for associative learning, giving the user the pleasure of discovering the knowledge in the program. On the other hand, some users feel lost in it and miss stricter guidance. With HYPERCARD and SUPERCARD hypermedia as e.g. interactive videodisk players can easily be controlled. Similarly convenient teachware is available by IBM although it is not so well standardized yet. Software products established with the authoring systems HYPERCARD (programming language HYPERTEXT), SUPERCARD and COURSE BUILDER from Apple company ([19-211, Table I) are especially well developed like e.g. the Mac MED system. An example of an IBM system in this category is DXTER [22]. It could be very useful, if the resulting learning program ran on both Macintosh and IBM computers. Authorware Professional (Table I) seems to be the only authoring system at the moment, that allows the creation of a CA1 program on a Macintosh computer as well as its conversion into MS-DOS format [23-25,361. Special authoring systems for medicine are described by Wiemer [ll] and Hasman [8]. Wiemer focuses on the integration of complex technical media such as video, spoken comments, workup of measurement data for physiological simulations and analyses. Hasman is dedicated to the problem-orientated approach towards decision making, e.g. in the context of optimized construction of decision trees. PRO M.D., distributed by MEDISOPT (Table I) is one example of an expert system shell usable as authoring system. It is built upon the programming language of artificial intelligence PROLOG and uses a knowledge base as well as estimations, probabilities etc. 5.3. Software for learning including interactive complex new media It is intended to give an outline of the existing software but nevertheless we cannot completely neglect the computers’ hardware and other supplemental media, as they are used in close connection with software and contribute strongly to the quality and acceptance oflearning systems. Videotapes have more or less run out of fashion for integrating film material into the learning dialogue. They only allow serial access to an entire sequence as a whole, and this way no real dialogue is possible. But in more costly produced CA1 programs, there is increasing application of interactive videodisk, CD ROM and other means of picture storage with almost immediate access to pictures, sound and videofilm sequences. So this means that besides the usual floppy disks a device for optical storage is applied, that requires its own drive mechanism connected to the computer. These media include sound storage that can be given out by loudspeakers or headphones [22]. Some software specializing in the workup of laboratory or biosignal data include

19

L

Knowledge BeH

-

RUkS

F8CtS b

I

t

Knowledge

AC@Sitii

ICI

Reeaonlng

uau -

Interhe

t User

Fig. 5. Structure of a knowledge based expert system. the reasoning subsystem and knowledge base of which can function as important components of learning software.

the interactive use of an analog/digital converter for the input and output of measured data. Hypermedia form a special category of new high performance interactive components of CA1 [ZO].Besides the usual computer techniques they invoIve video, sound, touch screen, windowing, text analysis etc. with several input-output options completely integrated into the learning dialogue. 5.4. Expert systems’ contribution to software for medical learning The contribution of expert systems to learning software is of special importance; it might even be the most benificial utilization of expert systems in medicine.

Medical expert systems are computer applications that dispose of an integrated expert knowledge and they are trying to solve medical problems just the way medical experts would do. One component they consist of is a knowledge base containing the rules, data, facts and rules of thumb (Fig. 5 and [26]). Another component is the inference machine, a program making logical deductions and controlling the entire expert system. Other parts are the user interface allowing to get in touch with the expert system, a subsystem for the acquisition of knowledge and last but not least another subsystem responsible for the reasoning of the system’s decisions. As giving reasons and explanations plays an essential part in learning programs, expert systems with a good explanation subsystem can function as basis of high performance software for learning. Similarly knowledge bases can be very useful for these purposes especially if they dispose of automatic updating options. For example, some issues of the CYBERLOG-software series (available on floppy disks) skillfully integrate software tools with expert system character into the learning program; they do not claim to be real expert systems. But there are other systems that are much more extensive as to their knowledge bases and their algorithms such as QMR [27] based on the expert system INTERNIST-1 and the Iliad software [28] that is based upon the hospital information and expert system HELP. Iliad and QMR are two of the largest software systems for learning in medicine that exist. Attempts have been made to develop a program that is able to automatically generate new patient simulations from pre-existing knowledge bases or expert systems. Chin and Cooper [29] called their system KBSimulator (Knowledge-Bases patient Simulator); it uses INTERNIST-l to generate patient simulations from. They limited the range of diseases to cardiovascular disorders. This program also disposes of a tutoring module and its built-in student model is directing the generation on the subsequent patient cases. In the end, above all those diseases that the student had difficulty diagnosing are generated. 6. Discussion The systems and concepts mentioned so far will be discussed now in a general way concerning their quality, without paying attention in detail to particular systems. What are the factors that the quality of learning systems can depend on? In the first place, it must not be overlooked that normally the production of CA1 is much more expensive compared to conventional means of teaching and learning such as books, lectures and so on. Therefore an important question to ask is, does a particular program perform something, that can hardly, if at all, be achieved by conventional means of teaching? It cannot be the destination of CA1 to be used for teaching work that can more effectively be done by human tutors. Therefore, it is essential to once again consider what the specific domains of CA1 are, in which it cannot be excelled by other methods, because that is where it should find its application. 1st domain of CAI. It offers the unique chance to exercise medical thinking and

21

acting without doing harm to anyone, without being the cause of unnessesary costly or high risk examinations. The user can still learn from the consequences of his actions. He has at his disposal a tutor with virtually unlimited patience and he can exercise at his individual speed as often and as long as it is required for his achieving a certain confidence in handling a particular kind of medical problem. In every day hospital life it is often difficult for a young physician to really become familiar with the management of one particular problem or disease. Perhaps he only occasionally comes across one disease or the patient being transfered to another ward is often lost out of sight. Using CA1 simulation programs, he himself can manage a whole series of similar cases from the history taking up to the ordering of therapy items and this way he can gain confidence in a particular field in a relatively short time. 2nd domain of CAI. Another strong point of computers is their ability to simulate cybernetic automatic control systems in the human body. This provides a chance for the user to experience how a system reacts to manipulations from outside. Even if the learner does not always understand in detail and in every single step why the system reacts this way and not in another (the human mind finds it hard to handle cybernetic systems) still he can experience the system. This way, step by step, he learns to forecast the system’s probable behaviour. These experiences can help the physician act correctly in critical situations. Besides these general thoughts about CAI’s application the following items need to be examined. 6.1. Quality of medical contents Unfortunately there is a considerable amount of software on the market containing wrong or imperfect information, proposing outdated diagnostic or therapy items or leaving important findings unconsidered. Therefore it is to be claimed that the quality and relevance of the medical contents should be validated. Another special problem is the fact that differences in opinion are often not shown as such, as is usually done in textbooks. Concerning the medical content updating is an important item to consider. New scientific and medical literature, improved EDP procedures, error corrections etc. are only offered by few CAI-series (e.g. the CYBERLOG-series). Finally, there are programs that do not fail to be correct, but they are kept on such a low level that they are not suited for the professional training of physicians and of course even less suitable for their continuing medical education. On the other hand there are good programs especially designed for non-physician staff such as nurses, ambulance staff etc. Medical committees of experts internationally accepted as competent should validate the medical contents of the software and determine whether it is relevant and suitable for professional training or continuing medical education [30]. In the US, such a validation is already obligatory for programs that make CME credits available as well as for computer-based examinations [lo].

22

6.2. Quality of algorithms and software design As far as the structure of the program is concerned (algorithms, programming, system design), there exists the danger of the following two extremes; some systems tend to simplify medical reality, others are so difficult to survey, that learners may get lost in them. Especially the problem-orientated approach risks producing badly arranged programs, as it naturally contains many individual cases, reflecting the whole biological variability, which is not easy to put into a logical system. Good systems should be able to give correct or at least plausible answers to more or less freely formulated questions concerning a particular subject matter. Especially for questions concerning differential diagnosis and indication of therapy items, there is a need for quickly running decision procedures that are close to reality. They will either look for the most suitable cases in big case collections and then present these retrieval findings, or produce artificial answers by decision algorithmns (see Fig. 6). These two procedures can also be combined as e.g. Chin and Cooper [29] have done in their Knowledge Based Patient Simulator for cardiovascular diseases. Literature investigation systems have existed for many years. For this reason much experience has already been collected with retrieval systems and highly efficient methods of searching exist using a combination of key words and weighted rank sorting of the possible best findings [31]. The algorithms for decision making in the available software are of varying quality [32] (see Fig. 7). Often plain symptom-diagnosis-matrices are applied for differential diagnosis and the disease covering the biggest number of symptoms is offered. Accordingly, therapeutic decisions are controlled and branched. Occasionally, empirical weights are attributed to the symptoms, as of course not all the symptoms and pathological findings are equally specific in the diagnosis of one particular disease. The next step is the integration of Bayes-procedures (used e.g. in the already mentioned Iliad-system for learning) attributing conditional probabilities to symptoms and findings (specificity and sensitivity) as well as attributing unconditional a-priori probabilities (prevalences) to the diseases. Then the a-posteriori Bayes probabilities can be computed for all relevant diagnoses and the one diagnosis with the highest Bayes probability is selected as the best or correct one. As Bayes-procedure does not take into account that there are often relationships among the symptoms (i.e. it assumes conditional independence), they can be up graded by log. lin.

system answers to user questions bY

Fig. 6.The two methods to analyse and answer questions.

23

. 1)

simple unweighted matrices of symptoms (manifestations) versus diagnoses

2)

roughly weighted symptoms, ad hoc reasoning

3)

use of sensitivities,

4)

simple BAYES procedures, use of prevalences, mapping frequencies into probabilities

5)

BAYES procedures with corrections for independence. e.g. log.regreaion

6)

full spectrum of discriminant and clustering

specificities

analysis

Fig. 7. Quality degrees of alorithms in medical decision making.

regressions, e.g as it is done in the learning system by Hasman [8] or by subjective estimates [29]. Even more sophisticated procedures of medical decision theory are at our disposal as e.g. the various forms of discriminant analysis and cluster-analysis [37], fuzzy tests or predicate logic and so on. However, it must be stressed that especially in medicine, inspite of all these highly developed mostly statistical algorithms, there is a need for heuristics and rules of thumb and sometimes rather vague descriptions of medical experience. This must also find its way into the software for learning. 6.3. Quality of the user interface The primary goal of software for learning is to render the learning process more efficient and easier for the user as compared with conventional teaching methods. Therefore, the means of communication between the software and the learner must be reliable in working and easy to handle. Software, hardware as well as supplemental technical media can contribute to this so called user interface. Its quality varies considerably. The quality of the screens is often not sufficient either technically (resolving power, size, flickering, grey-tones, colour etc.) or in their system-software (presentation of letters, graphics, images, the option of mixing colours, animation etc.). The money spent on modern, high quality screens is well invested, as this improves the acceptance and the pleasure of learning via computer software. However, especially concerning the reproduction of graphics and pictures, attention must be paid to the compatibility between what the software requires in terms of graphic adapters (EGA, CGA,VGA etc.) and the hardware the user owns. Often, many hours are needed to go through a program and poor quality of the screen leads to a too early withdraw1 from the learning session. Long texts and complex graphics are more easily read in printed form, so they should either be part of an accompanying text or the program should offer printing options where they are required. An intelligent and skillful mutual integration of

24

software and printed text utilising the strong points of both will play an important role for the acceptance of CA1 in the future. The quality of the dialogue is another important item to consider. In most cases the control of the interaction between the computer and the learner is based on multiple-choice-style questions. The user’s answers to these questions can lead him through the system which might contain as many branches as one likes containing as much explanation, help, comment, hints to tutorials etc. as the author likes. However, especially in medicine, the problem of multiple-choice examination is well known and here, in the software, this problem often already emerges in the learning phase. Although these systems are easy to score and prove consistent in their ability to test a physician’s or student’s recall of medical information, multiple choice prompting teaches students to study to pass examinations rather than enhance their problem-solving skills [2]. To integrate free answers is rather difficult and costly with respect to the software design due to the fact that there are always many ways of formulating the same information and, in medical terminology, there always exist many different terms for the same disease [33]. Until now it is a severe problem for many CA1 systems (even for the better ones mentioned in Table I) to ‘understand’ in a semantically correct way the free text input by the user without again applying multiple choice techniques. What is needed are comprehensive and complex text analyses with large synonym- and abbreviation-tables, context-dependent homonym resolution, robust methods for typing-error detection etc. However, natural language input by the learner is already technically possible up to a certain degree and it remains essential, as the success of clinical simulations and computer-aided learning is vitally dependent on maintaining an illusion of reality. Abdulla, Henke and Watkins [2] e.g. spent five years developing an authoring system, that is able to recognize 10 000 or more different types of input or as many as 600 different questions, requests or orders entered by the learner. No matter whether he asks: ‘Where is your pain located?’ ‘What is the location of your discomfort?’ ‘Where is the tightness located?’ or ‘Tell me where your pain is situated ?‘, the system will understand and answer the question [2]. Another important question to ask is, does the system try to build up a student model? Does it try to use the learner’s answers for collecting information about his state of knowledge as well as about his style and speed in the learning process and finally, does it try to react flexibly according to the information collected about the student? This can be a fascinating way of improving and keeping up a learner’s motivation. A system, the structure of which can be easily and quickly looked through by the student, might not be able to build up as much authority as a program making suddenly unexpected comments about the student’s performance that individually hit the point in the particular situation. Thus, the learner gets a feeling of being closely watched and permanently analysed. This may help to keep him aware and it does not hinder him, as the ‘superviser’ is only a machine. On the other hand, a student model is not always essential. There might be a danger of exaggerating and thus rendering the production of CA1 programs far too costly. Computer programs are inferior to human tutors in this field anyway, but an individually available CA1 program with a fairly well working student model is better than a perfect human tutor who just cannot afford the time and patience needed

25

for each individual student. However, CA1 programs must on no account be misused in a way that bedside teaching suffers. An often sufficient alternative to a student model is to render the program flexible enough to let the learner choose the level of knowledge on which he wishes to use the program. He should permanently have the option to determine how deeply he wants to go into the matter. The structure of the program should be easy to survey so that the user can always easily omit learning steps that are irrelevant to him for one reason or another. These points might be important in the decisions to whether an institution should buy an expensive program or not. Can it only be used by a small group of the staff or is it flexible enough to be useful for physicians on different levels of knowledge and experience? In most cases software for medical learning lacks appropriate users’ hand-books. Especially if the user does not own a highly equipped expensive hardware, many points can be more easily made clear in printed form on paper than on a poorly resolving, small screen that might perhaps not be able to display graphics. Procedure-plans, block-diagrams, medical concepts, decision trees and the like, as well as the entire plan of the software, should be available printed on paper and not only be accessible by various choices from hierarchic menus on the screen. Generally, in many programs without a handbook the user is at risk of losing orientation. He might not be able to tell in which part of the program he has run into, how to find a particular part of a tutorial again or he might find it difficult to survey what he has at his disposal in the program. High reliability and safety in function as well as tolerance of users’ mistakes are other important quality items to claim. Today it must be possible to use software for learning without any knowledge about data-processing. Mistakes like choosing programs that do not exist, typing wrong letters, pushing two keys at the same time and the like must not lead to any breakdown of the program or even the system, they must be answered by appropriate comments. Fortunately, in the last few years there have been considerable improvements in this field. Nearly all the good programs with respect to their contents, that have been worked over several times, no longer possess these problems. Delayed reactions by the computer may be due to technical problems in software or hardware. Having to wait for the next screen for too long can be extraordinarily annoying. The immense amount of data should be well organised and structured by data-base systems and the software has to comprise efficient algorithms to search and find something in these huge data-bases. Didactical problems seem to be of minor importance in the software for medical learning. Users seldom complain about insufficient concepts or poor pedagogics. Software design, user interface and structure principles of the learning programs may be responsible for some of the difficulties that used to be attributed to didactics in the past. 7. Conclusions It was intended in this overview to show that computer-based teaching and learning is taking place with a great variety of products coming mostly from the North

26

American market, designed rather for continuing medical education than for students. Many programs do not yet have a high standard of quality, but there are today already some outstanding learning systems, the future development of data processing and medical informatics giving hope for further considerable improvements. New media and authoring languages have made it easier to integrate medical expert knowledge into the programs but this still remains a central problem. The learner must take into account that indeed there are programs that are excellently suited for a problem-orientated approach, patient simulations and the presentation of pathophysiological functions but nevertheless CA1 cannot cover all aspects of learning in medicine and there are fundamental limits to software for learning. CA1 should be applied in areas in which it is superior to conventional teaching methods offering experiences that otherwise can hardly be made. CA1 must on no account diminish personal contact between medical teachers and learners or between medical students and patients. Intelligently applied, there is even a chance of achieving the contrary. CA1 application could free the teaching staff at universities from parts of their lectures - the amount of personal interaction in lectures is insignificant anyway - and thus give them time to hold courses in small groups, where real interaction between the teachers and the learners could take place. References 1 2

6 7 8 9 10 11 12 13

14

15 16

Piemme TE: Computer-assisted learning and evaluation in medicine, JAMA, 260(3) (1988) 367372. Abdulla AM, Henke JS and Watkins LO: Computer-aided learning: experiences, perspectives and promises, J Am Co11Cardiol, 9(3) (1987) 678-683. The Fifth Annual Medical Software Buyers’ Guide. M.D. Computing, 5(6) (1988) 16-151. IBM Academic Information Systems, Tools for learning, Courseware Cutulog, March 7 (1988) 239 -254. Salamon R, Protti D and Moehr J: Medical Informatics and Education. Proceedings International Symposium, May 1989, Victoria B.C. Canada. Hoffer EP and Barnett GO: Rx-Dx series, distributed by Williams&Wilkins (Table I). Schulmeister R (editor): Computereinsatz im Hochschulunterricht. Verlag an der Lottbek, Ammersbek 1989. Hasman A: Use of authoring systems for constructing medical teachware. In [34]. Hasman A: The role of medical informatics in a problem-oriented educational environment. In [5], 97-102. Clayrnan SG: Computer based medical examinations. In [34]. Wiemer W: Computerausbildung in der Physiologie. In [34]. Quadde. In [35]. Weinstein J: The Surgical Information Management Project: A Hypermedia Approach to Surgical Education - ‘MacAppY and ‘MacSwichm’. Proceedings of the International Symposium: Computeranwendungen in der Medizin mit Apple Macintosh, FRG, Munich, 10th and 1lth of November 1989; camp. [ 141. Proceedings of the International Symposium: Computeranwendungen in der Medizin mit Apple Macintosh, FRG, Munich, 10th and 1lth of November 1989, organised by Apple Computer GmbH, Ingoldsmdter Str. 20,800O Mtlnchen, FRG and the Ludwigs-Maximilian+Universitat, Mtlnchen. Bergeron BP and Greens RA: Clinical skill-building simulations in cardiology: HeartLab and EkgLab. Computer MethodsPrograms Med, 30 (1989) 11I-126. Lindberg DB: Information systems to support medical practice and scientific discovery. In [5], 1922.

21 17

Wiiems J, van Bemmel J and Michel J (eds.): Progress in Computer Assisted Function Analyst. North Holland Publ. Co., Amsterdam, 1988. 18 Renschler H: Die Unterstiitzung der Medizinerausbildung durch EDV. In [34]. 19 Engelbrecht R: Der Macintosh in der Medixin. Biomed J, 21(1989) 18-20. 20 Maurer H (ed.): Computer Assisted Learning. 2nd Internat. Conf., Dallas, May 1989. Springer Verlag. Berlin, 1989. 21 Smith JB and Weiss St F: Hypertext. Comm. ACM, 31(7) (1988) 816-886. 22 Seibt. In [35]. In Persot& Computer, 9 (1988) 64-7 1. 23 Tober K, Issing LJ: Autorensysteme-Lernprogrammacher. 24 Publication by The Training and Consultation Branch of the Lister Hill National Center for Biomedical Communications: Systems for Authoring Computer-Based Instruction. January 1985. 25 Lo&is C and Carr V: Authoring systems and assumptions about them. J Biocommun [HIG], 13(2) (1986) 4-9. 26 Puppe F: DiagnostischesProblemliken mit Expertensystemen. Springer Verlag, Berlin 1987. 27 Miller RA: Knowledge-based systems in medical education. In [34]. 28 Turner C et al: Clustered disease fhtdings, Aspects ofExpert Systems, 25,259-263. 29 Chin HL and Cooper GF: Case-based tutoring from a medical knowledge base, Computer Methods ProgBiomed, 30(1989) 185-198. 30 Selbmann HK: Validierung von Software und Teachware. In [34]. 31 Salton G: Automatic Information, Orgunisution und Retrieval. McGraw-Hill, New York, 1968. 32 Weinstein MC and Fineberg HV: ClinicalDecision Analysis, W.B. Sanders Company. 33 Wingert F: Automatic Indexing of SNOMED statements into ICD. Methods IfMed, 26 (1987) 93 -98. 34 Bauer MP and Michaelis J (eds): Computer in der Arzteausbildung, R. Oldenbourg Verlag Milnchen Wien 1990 (Tagungsband mm gleichnamigen internationalen Symposium im Mai 1989 in Bonn, F.R.G.). 35 Proceedings of the International Symposium: Computer in der Arzteausbildung, Mai 1989, Bonn, F.R.G. 36 Desch LW: Use of Commercial ‘Authoring Systems’ for medical education. Med. Educ. [MZ 31, 20(5) (1986) 417-423. 37 Discriminant Analysis, Classification and Clustering. In Panel on Discriminant Analysis, Classification and Clustering, StatSci, 4 (1989) 34-69.