Operationalizing a bedside pen entry notebook clinical database system in consultation-liaison psychiatry

Operationalizing a bedside pen entry notebook clinical database system in consultation-liaison psychiatry

Operationalizing a Bedside Pen Entry N Clinical Database System in C~n~~ltatiun-Liaison Psychia~ k Jeffre S. Hammer, M.D., James J. Strain, M.D., Ah...

990KB Sizes 0 Downloads 4 Views

Operationalizing a Bedside Pen Entry N Clinical Database System in C~n~~ltatiun-Liaison Psychia~

k

Jeffre S. Hammer, M.D., James J. Strain, M.D., Ahron Friedberg, M.D., and E eorge Fulop, M.D. Abstract: No current system of com~~teyize~ data en@ of clench i~~~r~ti~n in consultation-l~ison (C-L) ~~~~~~ has been well received 0~ has d~onsfrated that it saves the consultant’s time. The inabilify to achieve accurate, complete, systematic collection ofdiscrete variables and data entry in the harried C-L setting is a major impediment to the advancement of the subtilty andhealthservicesresearch.Thehand-held Notebookcomputerwith Windows PEN ENTRY MICROCARES~~~it~ haspermittedone-timedirectentry of data at the timeo~col~ecfion at the~fi~~fs ~ds~. Variablechoice and erection dances fk comp~ef~ess and accuracyof data collection.For example,ICD-9, Axis III diagnosesmay be selected from a “look-up“ whichat thesametimeautomatically assignsthe appropriatecodeand diagnostic-relatedgroups (DRG) number.A patientnarrativecanbetypedat thenurse*s station, a chart note printed for the medicalrecord,and the MICRO-CARES literaturedatabase perusedwith theprinting of seventyci~f~ns, ~t~fs, and in somecasesexperts’com~fa~es for the cons~~iee. The co~uliant‘s d~~rn~taf~on fime is halvedusing theN~TEBUO~ ~~N~WS PEN ENTRY MICRO-CARES software, with the advantageof moye accurateand completedata descriptionthan with the traditional handwritten consultationrecords.Consultees preferred typewritten in contrastto handwritten notes.The costof the hardware(about$2000)is lessthan that of an opticalscanner, and it pffmits report genffat~n and archival searches at the ntf~secfsfa~n without refusing to the C-L officefog scanning. Radio~e~~~~ OYefhernefdolled from the Note~k permitsdirecf data transferto th C-L officearchivecomputer. Mount Sinai School of Medicine, New York, New York (J.J.S., A.F., G.F.); and VA Medical Center, West Los Angeles, California (J.S.H.) Address reprint requests to: James J. Strain, Division of Behavioral Medicine and Co~uitation Psychiatry, Mount Sinai Hospital School of Medicine, 1 Gustave L. Levy Place, New York, NY 10029

General Hospital Psychiatry 17, 165-172, 1995 63 199.5 Ebevier Science Inc. 655 Avenue of the Americas, New York, NY 10010

Introduction Health care professionalsand policy makers seeking to ensure greater value in health care services face many boundary conditions that are fixed and a host of problems that are not. Today, the opportunity to affect one of these boundary conditiuns-the information management capabihties in health care- is within our grasp, This report advoc&s the prompt development and irnp~m~n~~on of ~p~~r~s~ patient records (Cl’%). Put simply, t&s Institute of Medicine committee believes that CFRs and the CPR systemshave a unique potentiat to improve the care of both individual p&ients and popuI&ons and, concurrently, to reduce waste through contir~ous quality improvement. . . . CPRsare a key infrastructural requirement to support the information rn~~ent needsof physicians, other health pr~~~~, and a variety of other legitimate usersof pkl-tit information. . . . ~~CPR~bea~~~[~development of guidelines]. . . . [the Cl33 wiM address] the need for improved patient data collection to support quality assurance,utilization management, and effectivenessresearch. What we be&eve will emerge will be a more caring, more scientific, and no less important, cost-effective heahh care system (Xnstitute of Medicine 1991[page v}) fll.

No current system of computerized data entry of clinical information in consultation-liaison (C-L) psychiatry has been well received or has demonstrated that it saves the consultant’s time [2-q, The inability to achieve accurate, complete, systematic collection of discrete variables and data entry in the harried C-L setting has been a major impediment to the advancement of this interface

J. S. Hammer et al.

subspecialty, in particular, health services research. This is the case in other sectors of medicine as well [1] (Table 1). It has been demonstrated that information can be found four times faster in a structured flow sheet than in the paper medical record [$I. Despite the availability of optical scan techniques, designed forms that require 5-10 minutes for their completion, manuals and prescribed definitions, and the provision of training sessions, attending physician and resident resistance to completing forms for direct computer entry remains the overarching impediment. All current systems consume more time than the traditional chart note documentation. Nurses and physicians use up to 50% and 38%, respectively, of their professional effort preparing patient records [9,10]. Critical feedback is received beyond the time when it can materially affect the training experience of a resident or the care of a specific patient. The time saved and enhanced accuracy from new computer technologies have not been demonstrated to the resident/attending. This paper describes a new WINDOWS PEN ENTRY hand-held NOTEBOOK computer system, with data entry at point of care, and its advantages compared with current systems. Three methods of documentation for the medical chart note will be described and compared: 1) handwritten consultation form; 2) MICRO-CARES OPTISCAN computer-generated note; and, 3) WINDOWS PEN ENTRY bedside data entry. Up to 70% of physician information needs are unmet during the patient visit [ll]. The WINDOWS PEN ENTRY NOTEBOOK permits “point of care data capture of all ~consultation] care processes” as suggested by the Institute of Medicine in the Cu~p~~~~ze~ Me~~c~~ Record [ 11.

Historical

Review

Our research group has been developing pragmatic database protocols and computer software systems since 1979 [12]. A 1988 IOM report recommended that a uniform national dataset (database) be established to permit valid comparisons 1131. Inlay, our database was several pages long, included 384 items for initial and final observations, and took 15-20 minutes of manual data entry for the consultant to complete and for office personnel to enter into the computer [4,12]. In 1988, this evolved into a shorter 102 variable OPTISCAN format with form completion requiring 5-10 minutes,

166

and computer optical scanning data entry potential of 200 cases an hour [2]. However, even the OPTISCAN approach engendered significant consultant resistance: completing the form required 5-10 minutes; lack of immediate bedside scanning meant that a computergenerated chart note was not available to place in the medical record at the conclusion of the consultation. Returning to the consultation office, waiting for the scanning to be done, and walking back to the patient’s floor with the computerized chart note could require 20-30 minutes. Current systems were unable to add a dated daily type-written progress note to the medical chart that could also be stored in the computer. The consultant had no access at the bedside to the patient archive to follow chronologically, or even for the last 24 hours, a patient’s care.

Description

of Hardware

and Software

The COMPAQ PEN ENTRY computer NOTEBOOK using WINDOWS is the new and versatile hardware for MICRO-CARES 13-51. The DOSdriven computer has 250 megabytes, 484/33 MHz speed, monochrome screen ($1500), infra-red or radio frequency download capacity ($500), and portable printer ($200) (total weight 5 pounds). It has either keyboard, or PEN (touch or handwritten) ENTRY for all data variables. (There is an adjustment period to enhance computer recognition of handwriting. Misunderstood letters can be corrected.) The keyboard is detachable and the NOTEBOOK is of the same size and configuration as the standard physician’s clipboard. With the MICRO-CARES PEN ENTRY WINDOWS system, data are entered by selecting from lists or occasionally writing in choices. Demographic information, reasons for consultation, psychiatric and medical diagnostic categories, as well as medication, administrative action, treatment and discharge recommendations, and hospital process variables can be entered by selection with a stylus on the screen. This process does not require writing or typing for the majority of variables, but rather identification and “touching” of preferred choices in “Lookup” and the use of simple WINDOWS commands. Visible preferred choices on the touch screen ensure accuracy of diagnoses and their correct numeric codes; drugs and their correct numeric codes; and so forth. The majority of the data can be entered while with the patient (or at the nurse’s

bedside Pen Entry Notebook

Reprinted with permission from the COMl?UTER-BASED National Academy of Sciences. Courtesy of the National Table 1. Selected parameters Author(s)/Parameters Tufo and Speidel (1971) Purpose

Sample Findings

Dawes (1972) Purpose Sample Findings

Zuckerman

and findings

PATIENT Academy

of studies of patient

in C-t Psychiatry

RECORD. Copyright 1991 by t&e Press, Washington, D.C.

record content

Study data

---

.____--_-

Evaluate record availability, missing data, recording of laboratory results incomplete physician narrative, and data collected for general health evaluations 1,149 patient visits in five outpatient U.S. Army facilities 11% of patients had no past medical data available 520% of charts had information missing: 75% of missing data were laboratory test results or reports of radiologic examinations 25% of missing data were lost, incomplete, or illegible data from previous visits 13-79% of laboratory results were not placed in the record 1049% of visits did not have a well-defined problem in the record 649% of visits did not have a well-defined treatment in the record 40-73% of records did not have evidence of general medical information useful for preventive medicine Determine presence of 18 data elements 1,628 medical records (the last episode of disease) in general practices 10% of patient ages were not recorded 30% of episodes had no therapeutic agent recorded: of those recorded, 75% were missing the amount prescribed, and 80% were missing dosages 40% of episodes had no diagnosis recorded 60% of males and 77% of females had no occupation recorded 99% of males and 21% of females had no marital status recorded

et al. (1975)

PUrpOSe

Sample Findings

Bentsen (1976) Purpose Sample Findings IOM (19~) Purpose Sample Findings

Appraise the extent to which records document adequately the content of verbal communication between physicians and patients 51 tape-recorded physi~an-patient encounters in pediatric clinic Percent present on tape and absent on record 0% of chief complaints 6% of reason for visit 10% of degree of disability 12% of allergies 22% of compliance data 31% of indications for follow-up 51% of cause of illness Assess the validity of data in the information system of the department 59 patient encounters in family medicine clinics 41% of problems identified by observers were not recorded Assess the reliability of data collected as part of the National Hospital Discharge Survey 3.313 medical records from 66 hospitals that participated in the National Hospital Discharge Survey 75% of face sheets had no discharge disposition 48% of face sheets were inadequate for determining principal diagnosis 15% of face sheets and discharge summaries were inadequate for determining principal diagnosis

Romm and Putnam (1981)

167

J. S. Hammer et al.

Table 1. (Continued) Author(s)/Parameters Purpose Sample Findings

Hsia et al. (1988) Purpose Sample Findings

Study data Document extent of agreement between the record and the verbal content of the physician-patient encounter 55 patient encounters in general medicine clinics Percent agreement between record and observation of encounter: 29% for other medical history 66% for therapy 71% for information related to current illness 72% for tests 73% for impression/diagnosis 92% for chief complaint Verify coding of diagnosis-related groups (DRGs) 7,050 medical records of Medicare patients from 239 hospitals 20.8% of the discharges were coded incorrectly (in the direction of higher weighted DRGs)

station) by simply touching the appropriate sex, age, referral problem, income source, past stressors, DSM-III-R diagnoses in five axes, drugs recommended and their reactions, discharge planning possibilities, and so forth. A major affective disorder, single episode, moderate, for example, has to be 296.22 and is entered by touching the written diagnostic description that appears. Previous consultations with the patient and the results from previous admissions can be queried while the psychiatrist is with the patient by a key stroke or a pen touch. A narrative, dynamic formulation and medical diagnoses can be entered at the nurses’ station, by keyboard or handwritten on the screen with the electronic pen. The MICRO-CARES literature database may be searched for the pertinent citation applicable to the case, again with a key stroke or a pen touch [14]. A computer-generated chart note including essential data elements, narrative, dynamic formulation, and key literature citation, abstract, and commentary regarding the citation, is printed on hospital medical note paper with a portable printer at the nurse’s station for inclusion in the patient’s chart at the completion of the consultation. Progress notes and updates may be entered directly into the NOTEBOOK and printed for inclusion on a daily basis into the medical record. The WINDOWS MICRO-CARES NOTEBOOK application encourages changes in choices without erasing, damaging, or the need to redo altered OPTEXAN forms during the evolution of a consulta-

168

tion Software for conversion than English is also available.

to a language

other

Comparison of Methods of Documentation Eighty-two consecutive consultation patients at the Mount Sinai Hospital were examined to ascertain the time requirements for documentation with 1) traditional handwritten consultation notes; 2) typed consultation notes; 3) MICRO-CARES OPTISCAN form plus handwritten notes; 4) PEN ENTRY WINDOWS MICRO-CARES NOTEBOOK alone; and 5) PEN ENTRY WINDOWS MICROCARES with typed narrative and/or formulation (Table 2). It was observed that items l-5 above required on the average 22, 25, 34, 2, and 10 minutes for documentation. There was a net savings of 20 and 10 minutes (90%, 55%) with methods 4 and 5, and the printed note was on the patient’s medical chart at the nurse’s station at the conclusion of the consultation. Twenty handwritten consultation notes selected at random, and compared with 10 prescribed variables on the OPTISCAN sheet revealed that many core variables were not recorded on the traditional handwritten consultation note (range 3-20) (Table 3). For a given variable, approximately 55% were entered on written notes and 95% on the MICROCARES OPTISCAN-generated medical chart notes. Variables, such as “Time of first visit,” had

Bedside Pen Entry Notebook in C-t Psychiatry

Table 2. Time to obtain consultation Time* spent

report

documenting

obtain printout

type

(min)

(tin)

Handwritten Typed

22

0

25

0

MICRO-CARES

12

(form on@)

22

-

**

2

MICRO-CARES NOTEBOOK ~CRO~A~~ NOTEBOOK with typed summary

10

22 25 34

2

2

12

even greater disparity: 15% (handwoven) as opposed to 100% (~~SC~-fo~). This difference is not insignificant: New York State mandates that chart notes have the time and date when the patient was seen. The same 20 handwritten notes were reviewed

for diagnoses (Table 4). Seventy percent had DSM-

Variable Date of admission Admitted from Time of first visit Age of patient Education

~p~oyment status Primary income source Major life event in patient Major life event in significant other

Reasonfor consultation Stated by consultee Stated by consultant n A response of unknown

of variables

in consultation

Written

MICRO-CARES”

17120 7120 3i20 20120 7120 8120 11120

20120 18120 2OJ20 20120 17120 19120 17120

15120

20120

3120

20120

lli20

20/20

19120

20120

was scored as 0.

--.MKRO-

Total (mutest

* Times listed are based on average times of 77 cases. ** Included in time spent with patient.

Table 3. Availability

hsed

Time* to

Consuhation

with handwritten note

Table 4. Diagnostic specificity on DSM-III-R

Axis I (clinical syndrome) Axis II (personality/ development disorder) Asix III (physical D/O or condition) Axis IV (severity of psychosocialstress) Axis V global assessmentof functioning

Handwritten

CARES/

note

~Q~~OK

14120

20120

6i20

18120

19120

20120

10120

'l7QO

18120

“-.-

20520

III-R diagnostic categories as opposed to 100% on MICRO-CARES OPI’ISCAN forms on which an actual diagnostic subtype (e.g., major affective disorder, depression, single episode, m~erate)~ was usually present.

Patient Response to Documentation Methods A sernistructured interview was administered to the 82 patients in the study for the purpose of obtaining feedback on the use of the different consultation data documentation methods to ascertain their reactions. Of the ‘77 patients interorie7nre$, 54 were considered able to respond meaningfuliyly, i.e., without dementia or severe delirium: had Mini-Mental Status Examinations of greater than 20 out of 30; and were not d&rious at the time of the interview. Of the 15 cases in which the traditional pen and paper were used when necessary to take notes, two patients commented aptly on feeling less comfortable with the physician taking notes. One patient commented that “the physician was not giving his full and undivided attention.” With the second group (N = 21), a standard laptop computer was utilized while the interview was in progress, and seven patients complained that the process was intrusive: the physician was “not looking at them enough,“ “the ‘computer thing’ is coming between us,” “that th&g is dehumanizing.“ A modified procedure was employed in which the narrative and more elaborate portions

169

J. S. Hammer et al. of the consultation were entered at the nursing station so as not to further disrupt the consultation process. In the third group of patients (N = 18) who were interviewed with the MICRO-CARES WINDOWS PEN ENTRY NOTEBOOK system, three expressed negative feelings about the “machine.” One patient who felt he was not receiving appropriate medical care stated that he was being reduced to tests, numbers, and meaningless bits of data. Seven others were positive: “I feel like I’m on the cutting edge of medicine, and at least I know the information will not be lost.” The remainder had neither negative nor positive reactions.

Consultee Response A survey was administered to 50 of the physicians who requested the 77 consultations. Thirty-one (62%) returned the survey, with a greater response among house officers than attendings. Ninety percent of the respondents (N = 28) said they preferred a typewritten note. Fifty-eight percent (N = 16) felt they were more likely to follow recommendations that were typewritten or clearly legible. Fifty-one percent (N = 16) felt that the inclusion of a reference and abstract would be helpful. Fiftynine percent (N = 18) preferred both diagnoses and a brief, psychodynamically oriented formulation to just an assessment (37%) or a formulation

(5%).

Discussion To date, neither the McKegney (KISS) nor the MICRO-CARES (OPTISCAN) systems have reduced documentation time in the inpatient C-L setting [2-5,7]. The addition of OPTISCAN forms has increased the demands of documentation without a time offset value, thereby increasing the resistance of consultants. Other benefits do accrue: the availability of the consultant’s patient profile, the CONDAT [15], ability to peruse an archive, generation of reports and publications for consultants, and so forth. Resistance is enhanced by the lack of timely feedback and turnaround of OPTICAL SCAN data for its use in the ongoing care of patients and generation of medical chart notes, in contrast to microcomputer systems which have proved effective in aiding clinical decision-making [16,171.

170

The MICRO-CARES WINDOWS NOTEBOOK demonstrates a 55% reduction in documentation time, and produces an immediate printed medical chart note including a psychodynamic formulation, augmented with appropriate literature citations, abstracts, and expert commentaries [14]. There are several additional issues for the consultant using the PEN ENTRY WINDOWS NOTEBOOK: typing skills are not necessary to write a formulation or narrative; handwriting is possible but slower with the stylus on the screen. Choices are made from a universe to be considered: all the depressions in DSM-IV [21], all the ICD-10 categories, all the anxiolytics, and so forth, so that consultants are not limited by their own, at times incomplete, repertoire. It requires less time to touch a selection than to write, circle, or to darken a series of bubbles. This “guided choice” methodology enhances accuracy, especially in recording of code numbers. Finally, the MICRO-CARES WINDOWS NOTEBOOK format becomes a pedagogic tool in ensuring that a consultant is aware of the expanded diagnostic entities in DSM-IV which appear in “Lookup” [18]. The archive at the patient’s bedside presents a new power in clinical decision-making. For example, in a case of mental status change in systemic lupus erythematosis (SLE) (ICD-9 Code 710.0), the consultant can search the database in less than a minute and observe all the experience of previous consultants with SLE. Shortliffe et al. [19] state that the CPR systems allow easy access to clinical decision supports that provide “customtailored assessment based on sets of patientspecific data.” The MICRO-CARES WINDOWS LITERATURE SEARCH in the NOTEBOOK accesses pertinent literature annotated by experts. Data from the institutional archive, expert-selected literature, drugpsychotropic drug-interactions, etc., augment clinical decision-making at the patient’s bedside or at the nurses’ station. Ongoing dated daily notes can track outcome and monitor the quality of patient care and resident supervision. Variables may be added or deleted without reprinting the OPTICAL SCAN form. A question or scale can be added for a short survey, e.g., the Hamilton Depression Scale [20], for 1 month, to one type of patient, or a selected cohort of patients. PEN ENTRY avoids multiple erasures on a scan sheet, and by its fluidity promotes unlimited decision changes, thereby facilitating the evolving process of diag-

RedsidePen Entry Notebook in t’-L I~s~~hia~

nostic reevaluation ture 1211.

and the unfolding

clinical pic-

In the Compufd3ased Patient Record: An Essential T~~~o~~ BOYdearth Care from the &X4, five objectives are proposed for the CPR (1): 1. support patient

care and improve its quality; of health care professionals and reduce the administrative costs associated with health care delivery and ~nancing; 3. support clinical and health services research; 4. accommodate future developments in health care technology, policy, management, and finance; 5. have mechanisms in place to ensure patient data confidentia~~ at all times. 2. enhance p~du~~t~

The TOM report underscores that “merelyt automating the form, content, and procedures of current patient records will perpetuate their deficiencies and will be ~suf~~e~t to meet emerging user needs. The CPR is an electronic patient record that resides in a system specifically designed to support users through availability of complete and acturate data, practitioner reminders and alerts, clinical decision support systems, links to bodies of medical knowledge, and other aids. . . . Most systems on the computer are in the main keeping track of patient care events.” If all data entries are timed and dated, the “CPR provides a continuous ~onologic~ history of the patient’s medical care“ [ill which serves as the d~rne~~~on for quality assurance, as substantial legal do~menta~~n, as an avenue to unravel untoward events, and as a powerful research tool for costs, effectiveness, and appropriateness of medical care. The availability of random control trial results enhances these capacities f23]. The PEN ENTRY is a LEAP forward in that it permits a Life (narrative) Entry At Patient’s side.

1. Committee on Improving the Pat&t Record: The Computer-BasedPatient Record. Dick RS, Steen EB

(eds), Institute of Medicine, vision of He&h Care Services. Washi~~o~~ DC, rational Academy Press, 1991 2. Hammer JS, Strain JJ, Lyerly M: An czptieal scan/ statistical package for clinical data advent in C-L psychiatry. Gen Hasp Psychiatry 15:95-101, 1993 3. Hammer JS, Lyons JS, Strain JJ: EvoI~~o~ of standalone integrated m~~~rnputer software system for psychiatric services, Part I. Compuz ~sychiatryj Psycho17(1):7, I985 4. Hammer JS, Lyons JS, Strain JJ:Strucpureand design of a stand-aloneintegrated ~~~~rn~uter software systemfor psychiatric services, Part II. Camput Psyc~at~~sychol’7(2):8, 1985 5. Hammer JS, Lyons JS, Strain Jf: ~v~Io~rn~nt of a stand-alonemicrocomputer system for consultation/ liaison psychiatry services. Comput Psychiatry/ Psycho17(4):15, 1986 6. Hammer JS, Strait JJ: Co~s~~urn-based consultation/liaison research.Int J Psychiatry Med 17(3):237371, 1987 7. McKegney FP, Schwarts CE, O’Dowd MA, et al: Development of an optically scanned ~o~s~ltatio~liaison database. Gen Hasp Psychiatry 12:7X-76, 1990 8, FriesJF:Alternatives in medicai recoxd formats. Med Care 12:87&-881,1974 9. Korpman RA, Lincoln TL: The computer-stored medicalrecord: For whom? JAMA 24:.X-63, 3988 10. Mamlin JJ, Baker DH: Combined t~~-rno~~n and work sampling study in a general medicine clinic. Med Care 11:449456, 1973 Xl. Covell DG, Uman GC, Mamting PR: ~~fo~atio~ needs in office Practice: Are they being met? Ann

Intern Med 1~3:~~5~~

1985

12, Taintor Z, Gise I-H, Spikes J, Strain JJ: Recording psychiatric cons~tations: a pre~rnt~a~ report, Gen Hosp Psychiatry 20:139-149,1979 13. Institute of Medicine: The Future of Efublic I-Iealth. Was~ngton, DC, National Academy Press, 1988 14. Hammer JS, Strain JJt Lewin C, et al: The continuing evolution and update of literature databasefor consultation-liaison psychiatry: MXXCXARES Literature Search System 1993. Gen Hasp Psychiatry

(SuppXf 15{6):1!%-12% 1993 R5. Strain Jay, Fulop G, Strain JJ, I%nmer JS: Useof the computer for teapot in the psy~a~ ~~sid~n~. J Psychiatric Educ ~~(3):~7~~~, 1986 16. McDonald CJ, Tiemey WM: The medical gopfrer-a microcomputer system to heip find, organize and decide about patient data. Medical Informatics fspeda1issue). West J Med 145~823-829,19% 17. Zak J, Kamett-Sheehan K, Roth R, Shaw DOK: Palmtop computer residency log. Acad Psychiatry 17(3):143-148,1993 18. Diagnostic and Statistical Manual of Menial Disorders, 4th ed (DSM-IV). Washington, DC, American Psychiatric Association, 1994

J. S. Hammer et al. 19. ShortIiffe EH, Perreault LE, Fagan LM, Wiederhold G (eds): Medical Inforrnatics Computer Applications in Health Care. Reading, MA, Addison-Wesley Publishing Company, 1990 20. Ham&on M: A rating scale for depression. J Neurol Neurosurg Psychiatry 2356-62, 1960 21. Snyder S, Strain JJ, Waif D: Differentiation of major depression and adjustment disorder with depressed

172

mood in the medical setting. Gen Hosp Psychiatry 12:159-165, 1989 22. Greenes RA, Shortliffe EH: Medical informatics: an emerging discipline and institutional priority. JAMA 263:111P1120, 1990 23. Chalmers I, Dickersin K, Chalmers TC: Getting to grips with Archie Cochrane’s agenda. Arch Gen Psychiatry 4:561571, 1961