A computer-assisted examination resource

Computer Programsin Biomedicine 11 (1980) 238-248 © Elsevier/North-Holland Biomedical Press

A COMPUTER-ASSISTED EXAMINATION RESOURCE A. Arthur STEELE, Michael J. McCUMBER, Paul J. DAVIS, Thomas BURFORD and Frank RENS Division o f Endocrinology, Department of Medicine, Department of Social and Preventive Medicine and Division o f Educational Communications. State University of New York at Buffalo School o f Medicine, Buffalo, N Y 14215, USA

A computer program (MICHELE) has been written to facilitate preparation by medical faculty of reference examinatioa questions based on a 'core curriculum'. Questions may be used by faculty to test student command of eoursework and by students at their initiative to assess their progress. Students may also use the program to direct their future studies according to computer-generated suggestions. MICHELE is a medical-instructional, computer-handled evaluation and learning enhancement system which consists of access, executive, author-instruction, student and statistics subsystems. These subsystems guarantee security of the data base, generate examinations according to category, difficulty and format (true-false, multiple choice, case history), generate practice sessions for students and student self-tests, and analyze the individual and cumulative results of examination and self-test sessions. The program is directed primarily toward evaluation of performance and instruction in the clinical disciplines; in this setting it is desirable to develop a sequence of unique, equally weighted examinations throughout the academic year and, at the same time, encourage mastery of core material encompassed in the data base. MICHELEis written in CDC i:ORTRAN IV and utilizes an indexed sequential f'de organization for the data. Computer-assisted instruction

Corecurriculum

FORTRAN

I. Introduction

Self-test

Our departmental policy currently requires formal evaluation of students at the conclusion of their medical rotation. A core curriculum of study materials is provided, mastery of which is a minimum requirement for successful completion of the rotation. Against this background we have developed a computer-assisted program to generate and administer, at faculty initiative, a sequence of examinations throughout the academic year. In addition, the program directs a selfinstructional and self-test system based on the core material. A subsystem to provide statistical analysis of testing results for faculty and immediate feedback to students in the self :est mode has been included. The program was written to be easily transferable from institution to institution. Computer-assisted independent study programs have been written in a variety of institutions [ 1 - 1 0 ] , as have programs to guarantee security of objective examination questions [ 11 ]. In contrast to the foregoing, our program develops on faculty demand a series of unique examinations and, on student demand, a self-instruction and self-test format, all of which are based on a core curriculum.

Practical instruction in the final two years of medical school is traditionally based upon exposure of small groups of students, in rotation, to the major clinical disciplines. These disciplines face twelve or more such groups in each academic year and are confronted with the task of providing qualitatively and quantitatively similar educational experiences to each student group. Student performance must also be fairly evaluated in the clinical setting. The enormity of these tasks is widely appreciated. Development of a 'core curriculum" in the clinical disciplines has been fostered at a number of medical schools to compensate for potentially wide variations in the quality of education in clinical rotations within a single medical institution. Formal evaluation of students at the conclusion of e!inical rotations may require twelve or more unique, equivalently difficult examinations or a single, class-wide (finzd) examination which, because of its timing and academic importance, has inherent biases. Address reprint requests to: Dr Paul J. Davis, Erie County Medical Center, Buffalo, NY 14215, USA 238

A,A. Steele et al., Computer assisted examination

2. Description of system

category of questions is divided into levels, where level zero represents a question pool from which the program will select only when preparing an examination ordered by a faculty member. Levels one through nine contain questions of progressively increasing degrees of difficu!ty that are routinely accessible to the student user. Examinations can be generated from levels one through nine, as well as level zero, at the option of faculty, and students have access to levels one through nine for practice sessions. The data base is submitted by medical subspecialty faculty on an annual or other regular basis. It consists of a group of questions in various formats (true-false, multipie choice, case history) and of varying degrees of difficulty. A case history multiple question format contains a patient summary followed by combinations of

An overview of MICHELE (medical-instructional, computer-handled evaluative and learning-enhancement system) is provided in fig. 1. Access validates users' credentials (fig. 2) and permits the user to comment at anytime on any portion of his interaction. The executive subsystem is shown in fig. 3; its functions include the listing and editing (addition, deletion) of users. This subsystem guarantees the security of the data base. The author-instructor subsystem (fig. 4) is the data management component of the program and is accessible only to t~culty users. According to subject matter, the data base entry is partitioned into major categories and thence into subcategories. Each sub-

Selection, of Dam Sine

239

II ~~ IZ IZ IZ

--

Variou+ Data Seres

i

i-

C'?-Los

_

Failed

,.

,

+

Security._I Check

J

"S--rob -+

J

+ Lomoff

X_

J-

i

Sulmymm

I Student Acorn i

I Executive Subsystem

Fig. 1.

I I I I I I I I--

I I I ! I I I

I

Au~orInstru~or Subwl~m

I I I

Statistics Subsystem

Student Subsystem

I I i

m

I

I

4

I- . . . . .

Author-lmtmctor Access

-J

I J

General schematic of medical-instructional, computer-handled evaluative and learning.enhancement program

(MICHELE).

A.A. Steele et aL, Computer assisted examination

240

NELCONE TO THE HED-INSTRUCTIONAL COHPUTER HANDLED EVACUATION AND LEARNING ENHANCENENT ( HZCHELE ) SYSTEM, HI!

I ' N HZCHELE.

PLEASE TYPE IN YOUR LAST NANE. ? NCCUHBER ARE YOU HICHAEL JACK HCCUHDER T YES

?

NOW PLEASE TYPE IN YOUR PASSWORD. I F THIS I S YOLIRFIRBT TINEp YOUR PASSWORD 15 S BLANKS. ? HIKE | F YOU DON'T UISH TO CHANGE YOUR PASSNGRD~ PLEASE TYPE • START • t ELSE TYPE * PASBWOR * .

YOU ARE NOW IN THE AUTHOR BUBSYBTEH. TO BEE A L I 8 T OF THE CONH&NDBp YOU CAN TYPE * COHHANDS * , WHEN YOU ARE FINISHED WITH THE AUTHOR-XNBTRUCTOR 6YBTEHp TYPE * F|N|IMNk'D TO LEAVE THE GUBBYBTEH.

PLEASE ENTER A COWHAND. XF YOU HEED HELP YOU CANTYPE

HELP

•

DO YOU WANT TO SEE A L I S T OF THE CATEaORXEB ON FILE? ? mHELP XF YOU NEED WORE HELP TYPE YES ELSE TYPE THE N I J ~ R OF THE INFORHnTION NEEDED, ? ~ mYES mmm YOU HAVE REQUESTED HELP. YOU ARE NOU IN THE PROCESS OF ADDING A QUESTION TO THE DATA BASE, Z CAN HELP YOU ON ANY OF THE FOLLOUINQI 1) WHAT KINDS OF QUESTION ARE ACCEPTABLE LIST OF THE CATEGORIES FOR MHICH THERE ARE ALREADY GUEETXONB 3) CURRENT NAJOR CATEGORY FOR QUESTION INPUT 4 ) CURRENT NAJOR AND SUB- CATEGORIES FOR QUESTION INPUT

2)

Fig. 2. The access program. A user with instructor status logs into the MICHELE system. He tion to the data base.

the other types of question formats. Questions relate to core curriculum materials written by subspecialty faculty. A command function, ADD, enables the instructor to add questions to the data base, under new or existing classifications of subject matter (fig. 5). The ADD program will also accept a list of references for the question, which the system prints out when the student user answers that question incorrectly. Additional question formats may be added by making minor programming changes. The EDIT programs allow modification of the existing data base. These changes include modifica-

requires HELP while adding a ques-

tion or deletion of individual questions, the transferral of questions to different categories, the transferral of subcategories and changing the names of categories. SET enables an instructor to set the parameters for student examinations and work sessions. The instructor is able to set the number of questions to be drawn from a particular level in each of the subcategories that he wants represented on the examination and weigh each level and subcategory in an examination. SET also allows an instructor to define work sessions, rather than examinations, thereby introducing a course of study for students. DUMP generates a printout of all or part of the

241

A.A. Steele et al., Computer assisted examination

M.I.C.H.E.L.E. MAIN Routing

i

:x=::::

....1

Routing

Addition of Users

Deletion of Users

Changes in Users" Status

Generation of Messages to User Community

F;g. 3. The executive subsystem of MICHELE.The subsystem manages the roster of users.

questions in the data base at the discretion of the instructor. Thus, instructors can analyze the data base for errors and periodically review the data base to confirm relevance of questions to subject matter. The student subsystem (fig. 6) is the focal pcint of the computer-directed (or computer-managed) instruction phase of the program. It has two command functions, EXAM and SESSION. The EXAM program administers the examinations which the student wishes to take. Examinations can have time and date restrictions (locks) on their use or may be requested at any time. Some examinations can aiiow the student the option of selecting specific subject areas for coverage. For example, the student might have the option of choosing three subject areas from a maximum ten areas offered by MICHELE for a course in addition to a set of mandatory areas. MICHELE randomly chooses questions within limits selected by the instructo~ or, in some cases, determined by the student - and weighs them according to guidelines provided by the instructor. When taking an examination the ~tudent has the option of deferring a question and not answering it until the end of the examination. The system will agk

the questions and record the results, informing the student if he answered correctly or incorrectly; it records any deferred questions. The student's raw score is reported to the student at the end of the examination. The system records the scores and all data needed to analyze item difficulty and item discrimination. It also keeps records of each student's performance by leveJ in each subcategory and major category for use in generating future work sessions for that individual student. Although structured for the development and administration of examinations, MICHELE also is utilized as a study and self.evaluation resource. The SESSION programs have three different options for generating work sessions. The student may choose a work session assigned by an instructor, or may pick the subject of the work session, or can allow MICHELE to design a work session (fig. 7) based on a user's achievement level. A work session defined by the instructor is more rigid than the other two. The number of questions from each concept area is fixed and the student must reach a specified level of achievement. When the student selects the subject areas for the session, MICHELE

242

A.A, Steele et aL, Computer anisted examination

IM.I.C.tt.|.L|. MAIN Roullinl

Ir.s~

~.owlinl

Additions Quen~n~ to Data h w

Definition of Ilmmlllrs f~r SllMiml Examj and i.mom

Modification of Daze Base

i i

1-

1

2=2J

I~inteut of Dill Bill

Review of Exininl Gum md I.ewon Paranmm

Name Chanlm

t

Gemrazion d Non-lnlWaczJu Exams

inPut of Data from Non I ntlwa~'qliveExams

Fig. 4. The author-instructor subsysiem of MICHELE. This component is the data management subsystem; access to the component is limited to faculty users. Types of questions stored in this subsystem axe described in the text.

randomly chooses questions from those areas, the same number of questions from each subject. If the student all~,ws MICHELE to design the work session, the system analyzes the data from examinations and previous work sessions of the student and identifies the areas of poor performance. Questions relating to such areas are asked first and more questions are asked in :'.~ese subjects than in areas ~vhere higher

competence has been demonstrated. MICHELE will draw questions from a higher level of difficulty if the student has answered a specified percentage of the questions correctly in any one level. For example, if a student has answered ninety percent of the questions in level one correctly ia a subcategory, MICHELE will draw further questions in that subcategory from level two,

A.4. Steele et aL, Computer assisted examination PLEASE ENTER A COHHAND. ? , n mADD mmml DO YOU ilANT TO SEE A LZST OF THE CATEGORZES ON FXLE?

?_-o PLEASE TYPE ZN THE NAHE OF THE HAJOR CATEGORY THAT YOU WXSH THE OUEGT][.ONS TO DE: UNDER. ? THYROlrD ][$ THE FOLLOU][N6 THE CORRECT HAJOR CATEGORY NAHE THAT YOU Ul'SH? THYROXD ? YES

NOW sPLEASE TYPE lrN THE NAHE OF THE SUB-CATEGORY YOU N][GH THE OUESTZONS TO BE UNDER. ZFYOU UANT THE GUEST]IONS UNDER GENERAL ][NSTE~tD OF A SUB-CATEGORY JUST PRESS ' RETURN " .

? sHEd...

xs T . S FOLLOUZMO T.E CORRECT SUS-CATESORY NAHE THAT YOU iiXSH? GENERAL ? YES LEVEL ][8 THE OUEOTZON?

?j. WHAT KlrND OF OUESTXON Z8 XT? ][F YOU DO NOT KNOU UHAT K][NI)8 OF OUI[ST][ONB X ACCEPT PLEASE JUST PRESS * RETURN ** ? ImJLTXPLE CHOXCE I I II U I ~ T ZS THE VALUE OF THE OUESTZON? NOW PLEASE TYPE ZN YOUR OUEST][ON EXACTLY AS YOU MOULD LXKE ST ¥0 APPEAR ON THE SCREEN. PRESS * RETURN * AT THE END OF EACH LZNE, UHEN YOU ARE FlrNlrSHEDp PRESS * RETURN * XNHED][ATELY AFTER THE OUEBTZON HARK APPEARS. ? THX8 SO A TEST OLIEST][ON.FOfl A DEHO ? A laUEBTZON CAN BE ANY I ~ I ~ I ~ R OF LXNES PLEAE4EENTER THE CHOZCE8 FOR THE ANBNER. PRESS * RETURN * AFTER THE OUEBTZON HARK ( ? )

APPEARS AFTER THE LAST CHOZCE.

2)? 3)? 4)? 5)? DO YOU UZSH THE CORRECT ANBNER PRZNTED OUT AFTER THE STUDENT ANSWERS? ? SURE

PLEASE NON TYPE ZN THE RESPONSES THAT MZLL BE CONBZDERED CORRECT, ONE RESPONSE PER LZNE, PRESS " RETURN " UHEN THE OUESTZON HARK ( T ) APPEARS ZF THERE MILL BE NO NORE RESPONSES,

DO YOU UXBH TO ASK THE STUDENT A RELATED OUESTZON ZF HE OXQEB AN ZNCORRECT ANSWER TO T H I S OUESTZON? ? NO

THZ8 XB A TEST QUESTION FOR A DEHO A OUERTXON CAN BE ANY NUHBER OF LZNE8 I)CHOXCE 1 2)CHOZCE 2 3)CHOXCE 3 4)CHOXCE 4

243

244

A.A. Steele et el., Computer assisted examination

THE 8TUDENT RESPONGES THAT MOULD BE CONGZDERED CORRECT ARE A8 FOLLOWS S 3 HA8 THE QUESTZON AS YOU'HANTED XT? DO YOU UXSN TO ENTER A REFEREHCE FOR THZS QUESTZON ? NmO DO YOU MZBH TO ADD ANOTHER OUEBTZOH ZN THE BIMIE I'IAJOR MiD SUB-CATEGORY? ?J~O ~mR YOU ARE LEAVING THE ADD PROGRAN, PLERSE ENTER A COHHAND. ? aEND mmmmm YOU ARE NOW LEAVZNQ THE AUTHOR-ZNBTRUCTOR BUB-BYSTEN OOOD-BYEpNXCHAEL BEE YOU AOA|H,

lrig. 5. The author-instructor subsystem, ADD ~nction. The instructor enters a new multiple choice qu~tion to the da~ base. Notice that the program will number the multiple choices ~ r ~ e author. The question ~ printed out after entry ~ r p u ~ o ~ s of review, if the entry is incorrect the author can edit it be~re it is plac~ m the data ba~, At the end o f each work session, MICHELE provides the student with an assignment for further study, including literature references, available audiovisual aids and the name o f a faculty member who

will tutor the student in the specific area. The statistics subsystem (fig. 8) provides information to both instructor and student. The ipstructor is provided with a scattergram o f the examination raw

M.I.C.H.E.L.E. MAiN Rout;ng

Student Subsystem Routing

Generation of Exams r

Retrieval

References Submitted by I nstructors

Fig. 6. Student subsystem of MICHELE. Facuhy-mandated examinations are administered by this component. Work sessions (practice exams) are also administered by this subsystem and may cover topics selected by students or may be defined by faculty.

A.A. Steele et al., Computer assisted examination

245

PLEASE ENTER A CONFIAND. ? iSESSION mmmlmnml) YOU CAN 1 ) TAKE AN INSTRUCTOR DEFINED MORK SESSION 2 ) PICK THE AREAS YOU W~NT THE SESSION TO BE ON~ OR 3 ) LET FIE HAKE UP THE SESSION BASED ON YOUR STRENGTHS" AND klEAKNESSE5. PLEi4SE TYPE TN l e 2~ OR 3 ? 2a t o m DO YOU MZSH TO SEE A L I S T OF THE H~JOR C~TEGORZES ? NO MHAT I S THE NANE OF A HAJOR CATEGORY THAT YOU M1~SH TO STUDY? ? 3RD YD HED THE FOLLOMING ][S A L I S T OF THE SUB-CATEGORIES UNDER 3RD YD PIED GENERAL 3RD YO HED su~ ONE klHAT I S THE NANE OF A SUB-CATEGORY THAT YOU klZSH TO STUDY? ? GENERAL DO YOU NISH TO PICK ANOTHER SUB-CATEGORY UNDER THIS MAJOR CATEGORY? DO YOU NISH TO PICK ANOTHER HAJOR CATEGORY? ? mNO

1 VALUE 5 THE HOST ][HPORTANT AND HOST RELTABLE HEANS OF DIAGNOSING ANGINA PECTORTS TS THE 1 ) ELECTROCARDIOGRAN 2)I'IiASTERS TMO-STEP TEST 3 ) C L I N I C A L HISTORY 4)PHYSICAL EXAHINATION 5)ANOXEHIA TEST (HYPOXEH]A TEST) ? CLZllIC/tL. HISTORY |fill I NEED THE NUNI~ER OF YOUR CHOICE, ? 3n . m THE CORRECT ANSiIER(S) I S : 3 YOU'RE hIGHT,

DO YOU WISH ANOTHER QUESTION? ? NO YObK SC~)RE I 8

1,0000

PLEASE ~[NTER A COHHAND. ? END YOU ARE LEAVIH~ THE STUbENT'SUB-SYSTEH¢ HICHAEL GOOD-DY~[JHICHAEL BEE YOU ABAIN,

Fig.?.A work ~ssion. Thesmdent m ~ b SESSION s ~ t s t h e s u ~ e c t h e wbh~.MICHELE ~e~ntsa multiOecho~equestion ~omlevdofdffficultylandinsists ~atthesmdentanswerbytheselecting ~om mult~lechoiceoptions.

A.4. Steele et al., Computer assisted examination

246

M.I.C.H.E.L.E. MAIN Routing

Ii

Exam Raw korea

StetistirJI Analysisof ExamScores

J ]

L

Item Difficulty Analysis

Individud ~alms

_ _

Fig. 8. Statistic~ subsystem ,,f MICHELE. Results of data analysis by this component are shown in fig. 9.

PLEASE ENTER A COHHANa~.

* llagUig1 YOU ARE NOM LEAVING THE ,~UTHOR-1NSTRUCTOR SUS-SYSTEH YOU ARE IN THE STATZSTZCS SUBSYSTEN. I F YOU NEED HELPe YOU CAN TYPE HELP. PLEASE ENTER A COHHAND. ? TEST SCORES MH&T lrS THE NAflE OF THE EXAN? ? I)EHO 21 4 . 3 2 * *Z 1. i** *Z 0

~EHO 2

8

$

1,000

09.47.19.

77/05/12.

MEAN= .4315 STANDARD OEQIATION: .0602 VARIANCE= .0036 HEAN DEQZATZQN= .2031 ROOT HEAN SQUARE= .4978 COEFFICIENT OF VARIATZON= 1 3 . 9 5 6 7 6RAB&U HEIH

N&HE JOHN LYNNE

C H

147151 123456

SCORE .1200 0,0000

Z-SCORE TINE -5.1723 13.27.37. -7.1650 13.55.56.

DATE 77/04/07 77/04/07

Fig. 9. Statistics subsy~,tem outt*ut. In the scattergram the numbers on the y-axis indicate the number of students whose ~rades were in the range indicated on the x-axis. It shows that four students scored .60 to .65 (60% to 65%) on the examination Demo 2. Only users with instru(tor status would receive additional information after the coefficient of variation. They would receive the data by student number as shown by the two examples here.

A.A. Steele et al., Computer assisted examination

scores, the mean, standard deviation, variance, mean deviation, root mean square, and coefficient of variation (fig. 9). These statistics can be provided to the instructor on a selected basis, i.e., only for certain students, or only for tests taken on certain days, or both. For each student the instructor can receive the student's full name, identification number, percentage correct on the examination, z-score, and time and date that the student took the examination. The instructor is also able to monitor the progress and percentage correctly answered by level in each subcategory for all examinations and work sessions taken by the student. MICHELE also summarizes for all examinations and work sessions the same statistics as for an cxamination, but analyzed for instructors by level and subcategol3,, item difficulty and item discrimination analysis are also available to the instructor. The student is able to retrieve only his scores and to relate them anonymously to the mean scores of previous students. Finally, MICHELE responds to three override commands. In ,~ch subsystem there are commands for listing the available commands, for sending messages, for seeking help and for entering the other subsystems available to the user. In addition, the user can at any time issue one of the three override commands: RESTART, KILL, or HELP. RESTART restarts the user at the beginning of the program or command he is using. KILL stops the program in use and returns control to the subsystem routing program. HELP acti~/ates a function which gives pertinent information about the program or command currently being used at any point in any of the subsystems.

247

specific subspecialty subjects [ 18-20]. A chronic need exists, however, for systematic evaluation of the performances of large numbers of students and for providing the student with recommendations which will direct further study. A programming language has been described which simplifies data-base assembly and entry [21] for such evaluation and direction. Our aim was to develop a program which is easy for the faculty to use, which allows the student several options of self-testing in various subject areas, which evaluates the student and directs further study, and which affords the faculty an evaluative tool. The MICHELE program currently meets these requirements. It has the capability of conditional branching during work sessions, allowing it to provide more help to the student when he answers incorrectly. Multiple phase, instructor-defined work sessions are possible in which the system will branch to different areas depending on the time required for the student to complete the session's requirements. MICHELE individualizes the pace of instruction and identifies areas in which instructors should intensify teaching efforts. Within our medical school, the Department of Physiology currently uses MICHELE for their skipbranched questions and source-referenced study program. MICHELE has been designated to assist in the analysis of student performance in the third year clerkship program in the Department of Medicine and will ultimately be used for continuous monitoring of student progress in medical clerkships throughout the clinical years.

Acknowledgements. 3. Digcussion The problem of valid evaluation of student performance is re-emphasized in the context of modern instructional technology [ 12]. Computers are in use extensively both as instructional aids and as evaluative tools. A number of groups have described computer programs which simulate clinical experiences. Sequential diagnostic models have ~een developed which simulate physicians' decision making [ 13-15]. Summaries of actual case histories for teaching clinical diagnosis have been developed to allow the user free text inquiry [ 16,17] and involve a number of

The authors acknowledge the support of Dean John Naughton, SUNY at Buffalo School of Medicine, in the completion of this work. The effort was supported in part by the Endocrine Research Fund, Department of Medicine.

References [1] C.R. Brigham and C. Kamv, J. Med. F_Auc.49 (1974) 278. ~ [2] E.P. Hoffer, G.O. Barnett, B.B. Farquhar and P.A. Prather, Annu. Rev. Biophys. Bioeng. 4 (1975) 103.

248.

A.A. Steele et al., Computer assisted examination

i31 A.D. Weinberg, Comput, Biol. Med. 3 (1973) 305. [41 R.E. Pengov, Proc. EDUCOM Fall Conf., ch. 14, pp. 141-154 (1974). W.D. Hagamen, D. Linden, W. Leppo, W. Bell and J.C. 15] Weber, Comput. Biol. Med. 3 (1973) 205. [6] G.M. Nashimoto and R.F. Waiters, J. Med. Educ. 47 (1972) 487. 171 G.J. Griesen, R. Beran, R. Folk and J. Prior, Proc. Roc[~ester Conf. Self-lnstructional Me¢ical Educ. pp. 3-27 (1971). 18] H. Wooster and J.F. Lewis, Comput. Biol. Med. 3 (1973) 319. [9] W.E. c h.~!ieand L.A. Jones, J. Med. Educ. 50 (1975) 81. llOl F.T..qtritter, H.J. Burford, S.R. Johnson and L.M. Talbert, J. Med. Educ. 48 (1973) 11~9. [111 N,A. Frigerio and R.S. Stone, J. Med. Educ. 49 (1974) 286. [12i R. Kane, F.R. Woolley and R. Kane, J. Med. Educ. 48 (1973) 615.

[131 G.A. Gorry and G.O. Barnett, Comput. Biomed. Res. 1 (1968) 490.

1141 H.R. Warner, B.D. Rutherford and B. Houtchens, Cornput. Biomed. Res. 5 (1972) 256.

!151 H.R. Warner, F.R. Woolley and R.L. Kane, Comput. Biomed. Res. 7 (1974)564.

1161 W.G. Hatless, G.G. Drennon, J.J. Marxer, J.A. Root, L.L. Wilson and G.E. Miller, Comput. Biol. Med. 3 (1973) 227. [171 W.G. Hatless, G.G. Drennon, LJ. Marxer, J.A. Root, L.L. Wilson and G.E. Miller, Comput. Biol. Med. 3 (1973) 247. [181 E.P. Hoffer, G.O. Barnett and B.B. Farquhar, J. Med. Educ. 47 (1972) 343. !191 E.P. Hoffer, Comput. Biol. Meal. 3 (1973) 269. 1201 A.A. Steele, P.J. Davis, E.P. Hoffer and K.T. Famiglietti, Comput. Biomed. Res. 11 (1978) 133. 1211 S.G. Smith and B.A. Sherwood, Science 192 (1976) 344.