Interpretative reporting and alarming based on laboratory data

ELSEVIER

Clinica Chimica Acta 222 (1993) 37-38

OpenLabs Project

Interpretative

reporting and alarming based on laboratory data

Pirkko Nyk5nen*a, Gerard Boranb, Hilde Pin&“, Kevin Clarke”, Michael Yearworth’, Jos L. Willems”, Rory O’Moored “Technical Research Centre of Finland, Medical Engineering Laboratory, 33101 Tampere, Finland ‘Deparatment of Chemical Pathology and Endocrinology, United Medical and Dental School (Guy’s Campus) London, UK ‘Katholieke Universiteit Leuven. University Hospital Gasthuisberg. Division of Medical Informatics, 3000 Leuven, Belgium dFederated Dublin Voluntary Hospitals, St James> Hospital, Dublin 8. Ireland eTrinity College, Department of Computer Science, Dublin 2, Ireland /Bristol Transputer Centre, University of the West of England, Frenchay Campus, Bristol, BSM lQY, UK

Abstract

The utilisation of laboratory services for patient diagnosis and management involves many steps with both clinical and laboratory components. The clinical components include the decision to order a test, intepretation of the test results and actions taken on the basis of the results. The laboratory components on the other hand include receipt of the request, specimen collection, preparation and analysis, result entry, test result validation and verification and reporting of the results. In this paper, which is part of the OpenLabs project, we concentrate on the post-analytical applications which include interpretation and reporting of the laboratory results to the users in primary care and in high dependency care units. The final objective of the work described is to develop generic modules which can be integrated both with an Open laboratory information system architecture and existing laboratory information processing environment. Key words: Post-analytical functionalities; Clinical laboratory services; Primary care; Intensive care; Interpretative reporting; Alarms generation

1. Introduction

Data processing technology has now reached a stage where devices for the generation, storage and distribution of data and information are being developed and l

Corresponding author.

0009~8981/93/.$06.00 0 1993 Elsevier Science Publishers B.V. All rights reserved. SSDI 0009-8981(93)05720-Y

38

P. Nykiinen

et al. / Clin. Chim. Acta 222 (1993) 37-48

manufactured faster than society is capable of assimilating them [l]. This can be readily seen in clinical decision making where, during the last two decades, the volume and variety of clinical laboratory tests have expanded substantially. In a study of the use of clinical laboratory services, it has been noted that a change in style in the use of the laboratory on the part of primary care physicians was one of the most important factors explaining laboratory utilisation [2]. This suggests that the clinical diagnostic process has changed from a problem oriented to a data oriented one. Rapid technological progress and increased utilisation of tests have brought their own problems. There is growing evidence that information overload or data intoxication is occurring. It may now be easier to obtain test results than it is to understand them, giving rise to the situation where physicians might fail to respond properly to, or to notice, important laboratory findings, much less to realise their full implications [3]. Another cause for concern is that of escalating health care costs. Among the leading growth factors are the cost of hospital services, notably laboratory services. With respect to the clinical laboratory, rising costs appear to be almost entirely attributable to expanding utilisation and introduction of new services. For example, in the UK, laboratory costs consume some 9% of acute hospital costs and workloads have increased 10% per year for the last 25 years [4]; in the USA some 10% of health care expenditure is on laboratory investigations, 20-60% of which are deemed to be unnecessary [5]. In the intensive care unit, laboratory information, which is monitored discontinuously but regularly, plays an important part in making therapeutic decisions. It has been reported that decisions in patient management in intensive care are based on laboratory data 40% of the time [6,7]. It is evident that we need initiatives towards optimal use of clinical laboratory services. Information technology and decision support systems provide a challenge to improve the efficiency and quality of the production and delivery of laboratory services. The main task of the OpenLabs project will be the integration of the heterogeneous data from multiple sources for subsequent processing. This will be achieved by defining uniform means for modelling of data and information, and by providing translation and integration techniques to map the various data sources to a common model. 2. Interpretative reporting for primary care Over the past 20 years the usage of clinical laboratory services by primary care physicians has increased significantly. The modern general practitioner (GP) uses the laboratory as an essential back-up service particularly to monitor patients with chronic illness (e.g. diabetes, renal disease, thyroid and lipid disorders, anaemia, cardiac failure and certain malignancies). The workload from general practice in most clinical laboratories is now over loo/o. From the laboratory standpoint, provision of services to the GPs is difficult. In particular getting the results back to the practice is usually a time-consuming combination of mail and telephone. This makes the reporting of laboratory results a potentially fertile application area for telecommunications technology.

I? ~~k~~

et al. / Clin. Chim. Acta 222 (1993) 37-48

39

A further problem is the relatively small number of GPs with computerised practices. This is rapidly growing in most European countries where the average is now over 10%. In the UK the number is almost 800/o[8]. The conclusion is that improvement of laboratory services in primary care must take into account the reality that, although growing numbers of GPs are computerised, the majority, at present, in Europe are not. It is probable that only a clear demonstration of the benefits of teleco~unication links to hospitals will entice the remaining GPs into the world of telemedicine. The aim of interpretative reporting is to produce information rather than a mass of data. Although vast amounts of laboratory data are requested every day, not all physicians are equally capable of interpreting them. Decision support systems in well-defined specialised domains can provide an easily accessible ‘back-up opinion’ for less experienced physicians. As an example of interpretative reporting we now describe a medical decision support system for thyroid function diagnosis. 2. I. Thyroid decision support As for any laboratory test, the full spectrum of modifying factors should be taken into account when interpreting thyroid function tests 191. For example, the effects of analytical and biological variation and the use and validity of reference values have important implications for correct interpretation [lo]. Adequate clinical information is essential to make interpretations as clinically relevant as possible. Computer-assists decision making systems are gaining acceptance in laboratory medicine and have the potential to improve both the consultative and teaching roles of the laboratory 1111. A variety of approaches have been adopted to interpret thyroid function tests. For example, robust systems based on algorithms [ 12-151 or mathematical approaches [ 16-181 have been described. At least one of the algorithms has been used routinely for several years [ 151. Expert systems, computer programs based on symbolic reasoning, appeared more recently. Several of these programs have secured a role in thyroid function test inte~retation [ 19-221. An algorithm for inductive learning in data-rich domains has also been applied to thyroid function test interpretation 1231.Whereas all of these approaches provide effective decision support, some have not been flexible enough to accommodate different test strategies or local user modifications. The program described by Brosnan et al. [20], an expert system initially developed using Common Lisp and subsequently translated into C, specifically addresses these problems of transferability. It relies primarily on fixed production rules which trigger associated comments or chains of co~ents in a branching tree structure, but accommodates a range of tests with revisable decision levels so that the program can be transferred to other laboratories and used successfully. A method for interpretative reporting of thyroid function tests based on automatic knowledge acquisition has recently been described [24]. In common with certain other programs based on expert system technology [20-221, this method aims to avoid the transferability problems which were a limitation of earlier systems. Each application has its own characteristic set of clinical and laboratory variables. These are entered into the system by the observer, or are transferred from laboratory databases. The knowledge acquisition mechanism, which is common to all applica-

40

P. Nykiinen

et al. / Clin. Chim. Acta 222 (1993) 37-48

Table 1 Variables used in the thyroid decision support Qualitative

Patient’s sex Clinical details Drug therapy

Quantitative

Patient’s age Serum total thyroxine concentration Serum thyrotropin concentration Serum free thyroxine concentration Serum free triiodothyronine concentration

tions, generates a rule from these variables. Table 1 lists the variables in use for the thyroid application. Quantitative data are classified into mutually exclusive decision ranges. So as to be locally modifiable, these ranges are user-defined, fully-revisable and age-specific. Qualitative data may also be entered in any combination by selecting the appropriate finding from user-defined expandable lists of options. Table 2 gives representative examples of the options available for the qualitative variables used to represent clinical details and drug therapy relevant to thyroid function. After data entry, the program obtains an interpretation from the observer and links it to the data-derived rule so that an appropriate interpretation can be provided automatically whenever similar data are encountered. Table 3 gives an example of an interpretation for a set of thyroid function tests performed on a 75-year-old woman with a developing autonomously functioning thyroid nodule who presented with tiredness and anxiety. In contrast to existing methods for interpretation of thyroid function tests, interpretative skills in this system are exclusively derived from the data and interpretations entered by the observer. Although the observer is required to enter interpretations for most or all of the data in the initial stages of development, the program quickly accumulates knowledge which reflects his opinions, practice and literary style. Furthermore, the clinician receives computer-generated reports bear-

Table 2 Examples of qualitative variables for drug therapy Clinical details

Drug therapy

Suspected hyperthyroidism Suspected hypothyroidism Post radioiodine therapy Post thyroid surgery On thyroxine treatment On triiodothyronine treatment On carbimazole treatment On propylthiouracil treatment Non-specific

Lithium Oestrogens Amiodarone Beta-blockers Glucocorticoids Phenytoin Carbamazepine Androgens No relevant drugs

41

P. Nykinen et al. / Clin. Chim. Acta 222 (1993) 37-48 Table 3 Example of a thyroid interpretative report Thyroid function tests

Units

Reference ranges

Serum

nmolfl munits/l pmolil pmohl

60-160 0.25-5 3.3-8.2 9-25

Total T4 TSH Free T3 Free T4

256 0.25 6.6 20.1

The total T4 is markedly raised and the TSH is suppressed. However, the free T3 and free T4 levels are not raised and frank thyrotoxicosis therefore appears unlikely. Consider a developing autonomously functioning thyroid nodule, possibly with elevated binding proteins. Assay of thyroxine-binding globulin may be helpful to confirm the latter.

ing his hallmark, just as his handwritten interpretations did. Because the proportion of automatically assigned interpretations might initially be low, the system was primed with a set of selected data and associated interpretations which were expected to occur commonly. The knowledge acquisition capability was then used to extend or modify the knowledge base as the need arose. Using this technique, the system automatically assigned interpretations to 66% of thyroid function test requests during its first exposure to a batch of 92 requests. Over 3,000 requests were subsequently entered during the first 4 months of operation and interpretations are now available for at least 90% and up to 97% of all requests in any given batch (mean batch size: 78 requests). The remaining 3% of unusual cases in each batch do not receive automatically assigned interpretations. These must be interpreted by the observer but will be added to the knowledge base when an interpretation has been entered. 3. Interpretation and alarming in intensive care In the intensive care unit (ICU) the patient’s state is continuously monitored and therapy is given to maintain the patient in optimal condition. The medical staff frequently base therapeutic decisions on continuously measured signals (such as heart rate, blood pressure, cardiac output and temperature). Discontinuous measurements (laboratory and clinical), patient history and visual examination of the patient are also important. The equipment of many modern intensive care units includes patient monitors and a system for managing patient data automatically. A patient monitor operates in real time and handles several signals measured continuously from the patient. The long-term trends of the signals may be collected from the patient monitors to a patient data management system (PDMS). Discontinuous measurements, whether they are made in a laboratory or at a bedside, are also stored into the PDMS which manages the data of every patient in the unit. The laboratory report is a vital link between the laboratory and the clinician. The manner in which the laboratory data are presented can have a great impact on the action taken by the clinician. It is common that laboratory results are overlooked or that their significance is underestimated. Laboratories usually mark abnormal values and provide clinicians with reference values. However the report layout often

4.2

P. Nykiinen er al. / Clin. Chim. Acta 222 (1993) 37-48

could be improved. A graphical display of laboratory data can be interpreted faster and more accurately by users than columns of numbers. It has been shown that the occurrence of critically abnormal laboratory results is associated with prolonged stays and high mortality rates [7]. Immediate detection of laboratory data alarms and automated notification of the ICU staff may allow for earlier treatment of these high risk patients. Thus automated laboratory data alarms represent a valuable decision tool for the management of high risk ICU patients IS]. It has been reported also that implementation of hospital wide alerts based on laboratory data led to some significant improvements: increase in the frequency of appropriate treatment, less time spent in life threatening situations and shorter hospital stays for patients [6]. 3.1. rnter~retation of acid-base disorders Acid-base disturbances occur frequently in high dependency environments. To make an assessment of the acid-base status of a patient, physicians use clinical data but also need several sets of laboratory measurements. Arterial blood gases and serum electrolytes are analysed daily and in acute situations several times a day. The correct interpretation of these laboratory data may be rather difficult for two main reasons. Firstly, acid-base disturbances can exhibit a broad spectrum of complexity, from a simple single disturbance up to a complex triple disorder. It is indeed possible to interpret a single compensated disorder at first glance; however, recognising a double and particularly a triple disorder, demands a systematic and thorough analysis of the different laboratory data. An expert system, which includes the knowledge of an expert in the field of acid-base disorders, could guarantee a systematic and consistent approach to this problem. Secondly, the acid-base equilibrium typically is an area where (patho)physiologic processes can be described by mathematical formulas. Unfortunately, these formulas are not easy to work with. Inclusion of these mathematical relationships in a computer program allows the user to concentrate on the clinical importance of the different pathologic processes rather than on the complex calculations needed for an accurate acid-base interpretation. A system for the automatic interpretation of acid-base disorders was developed and was put into routine use, as a pilot system, in the Intensive Care Unit and the Emergency Department of the University Hospital Gasthuisberg in Leuven, Belgium. The program performs a systematic acid-base analysis in three major steps: the interpretation of the primary acid-base data, the calculation and interpretation of the anion gap and the comparison of the increase in the anion gap with the decrease in serum bicarbonate. The input laboratory data are retrieved from the laboratory information system. An example of an output report produced by the system is illustrated in Fig. 1. Users of the laboratory information system can request this kind of acid-base report for each patient admitted to the Intensive Care Unit or the Emergency Department. The accuracy of the system was evaluated using the traditional error rate approach, The interpretation of the medical expert involved in the development of the system was the so-called gold standard. For a training set of 202 cases an accuracy of 100% was reached. The evaluation of a test set of 194 cases resulted in an accuracy of 92.8%. Two independent experts were also invited to participate in the evaluation

43

P. Nyk&en et ai. /C&L Chim. Aeta 222 (1993) 37-48 PATIENT: PATlENTC B&d

gas data and swumelec~oly!s

PH -la’

P&O2 44.9

HW 21.0*

of 31iW92 at 7.00 PM

Na 136.2

Pa02 84.8

K 5.01’

a 101.1

PI01 13.2’

CO2 23.0

Ca 8.1’

Phos 2.5

Expcted anion gap 7.79 -+ 13.79

Anion gap 19.11**

ACID _ BASE IN~RPR~ATIGN

: MODERATE

ACtDEMlA DUE TO

ACUTE. OR CHRONIC RESPIRATGRY ACIDOSIS WITH HIGH ANION GAP Ml?.TABOLIC ACIDOSIS AND METABOLIC ALKALOS IS which in fact can be the renal compensation of the chronic respiratory acidosis OR METABOLIC ALKALOSIS WITH ACUTE HIGH ANION GAP METABOLIC ACIDOSIS BEFORE FULL RESPIRATORY COMPENSATlON

NOTE: Ihe arterial pa&l

pressure of oxygen (PaOzf (8.48 mm Hg) is lower than expected for all values oFFi

z 0.24

Fig. I. Example of an acid-base interpretative report.

study. The results are presented in Fig. 2. As expected the highest agreement was found between the system and the expert involved in the development of the system. However, the levels of agreement between the inde~ndent experts and the system and between the different experts themselves, are similar. Therefore the accuracy of the system can be interpreted as acceptable. The utility of the system is being assessed by user questionnaires. After using the system for a few months, physicians of the Intensive Care Unit and the Emergency Department are invited to answer some questions concerning the practical use of the system, the output of the system, the possible impact for their patients and their interests in future developments. To improve the service provided by the current system, several enhanced facilities will be elaborated in the future. The most important topics are (1) to provide a graphic presentation of the primary acid-base data pH, Pacoz and [HCOs-] on a PaCo,-[HCOs-] diagram and (2) elaborate a temporal reasoning module which will allow the system to interpret consecutive acid-base analyses of the same patient. Figure 3 is a preliminary specification of an output screen of an enhanced system able

PRIM.PAR ANION GAP TOTAL

PRO-MD1 92.8% 96.9% 89.7%

PRO-MD2 80.4% 91.2% 73.2%

PRO-MD3 88.7% 89.7% 81.0%

MDI-MD2 82.5% 92.8% 76.3%

MDI-MD3 82.5% 91.2% 76.3%

MDZMD3 73.2% 96.4% 70.6%

Fig. 2. Results of the evaluation of the acid-base disorders interpretation system. Overview of programexpert and inter-expert agreements for the interpretation of the primary acid-base data, the interpretation of the anion gap and the overall agreement for the 194 patients of the test set. PRO, knowledge-based system for the inte~~tation of acid-base disorders; MDl, medical expert involved in the development of the system; MD2 and MD3, independent medical experts.

ACID-BASE INTERPRETATION

: the most probable interpretations for this patient are

:

-

the patient moved from a hyper~hiofemic metaboiic actdo% to a metabolic acidosis, mlxed with an acute metabolic alkalosis due to bicarbonate therapy OR - the patient moved from a hyperchloremic metalJolic acidosis to a mixed metabolic acidosis + respiratory alkalosis; the respiratory alkalosis IS in fact the sustained respiratory compensation of the metabolic acidosis

Fig. 3. A preliminary specification of the output screen for the acid-base interpretation system.

to present the laboratory data and the evolution of the data, in a graphic way, as well as to interpret serial acid-base data. This system for the interpretation of acid-base disorders provides information rather than raw laboratory data. The aim is to deliver a rapidly and easily accessible back-up opinion, available for physicians working under constant stress and forced to make decisions based on rapid judgements. 3.2. Afarms and alerts generation The need for alarms and alerts in intensive care must be balanced against the risks of excessive and false alarms. Many of the current monitoring systems are plagued with several sources of error, producing a large number of false alarms [25]. False alarms naturally undermine the credibility of alarms in general, causing staff to ignore alarms. Our system includes a large amount of laboratory data and clinical information. Individual variable ranges, drug interferences and combinations of variables which are physiologically interdependent are considered. This is very important since an individual result, e.g. potassium of 6.0 mmoVl, might indicate the need for an alarm, but perception is altered when this result is considered in conjunction with the clinical condition, e.g. chronic renal failure. Temporal reasoning on this data is a major component and will reduce the number of unnecessary alarms, e.g. a potassium of 2.9 mmolll indicates the need for an alarm but previous results of 2.0 and 2.5 mmolll over time imply that the low potassium was recognised and is being treated. Whilst using reference ranges for laboratory data is important, ‘normal’ results in a given patient may be abnormal, e.g. a patient with a high protein level may have a

45

P. Nykiinen et al. /Clin. Chim. Acta 222 (1993) 37-48

pseudohyponatraemia if the sodium is measured by an indirect ion select electrode. This is recognised by the system and text is provided to explain. Delta checks are used to signal an alarm for sudden changes in a variable - that may or may not bring it outside the normal range. The rate of change is measured between two readings or averaged over a few readings to detect a more general trend. The allowed rate of change varies according to the deviation from the mean; close to the mean higher rates of change are tolerated, whereas closer to the critical limits the alarms will be more sensitive to rates of change. Trend analysis is very useful in detecting subtle but dangerous trends which are missed if only critical value limits are utilised by the system [7]. Laboratory data are not sufficient for a comprehensive assessment of the state of the patient and usually a considerable amount of other clinical information must be

Check CVP/PCWP Check fluid halme

I

CVP/PCWPHigh +ve fluidhalance I

CVPIFCWPNorm +I- fluid l&ace

CvPlpcwp Low

I -ve fluidbalam

I

I t

t VP/PcwPNoml.

I

Fig. 4. A decision tree for fluid balance alarms.

cw/PcwP

High

46

P. Nykiinen

et ul. / Clin. Chim. Arm 222 (1993) 37-48

available before alarms and alerts can be generated based on laboratory values [5]. For this reason the alarms and alerts system takes into account the patient’s current physiological status. This prevents an unwarranted number of alarms when the abnormal results are expected, e.g. high urea and creatinine in known renal failure should only carry an alarm if marked deterioration has occurred; raised AST and CK post-operatively are expected since muscle has been traumatised. To date ‘intelligent alarms’ have been implemented in three areas: acid-base balance, fluid balance and ventilation. Figures 4 and 5 are sections from the fluid balance and ventilation decision algorithms, respectively. 4. Discussion Decision support systems described in this article and further developed in the OpenLabs project enable clinical laboratories to provide their primary care and intensive care users with locally approved expert interpretations for laboratory tests. Since the systems handle borderline findings and complex mixtures of clinical and laboratory data with ease, users receive high quality decision support targeted to the

VENTILATION

Fig. 5. A decision

tree for ventilation

I

alarms.

I

needs of their patients, This advice is additional to the numerical results and is generated without a reduction in laboratory productivity. Learning progresses from commonplace cases to the more unusual cases in these systems, with the safeguard that all novel cases must initially be interpreted by the observer. The developed systems are suitable for integration into a telecommunications network so that results and interpretations could be provided te~~rnati~~~yto remote users. fn keeping with the gene& aims of the OpenLabs project, these are examples of systems which promise to improve eEciency in the use of health care resources and tu enhance the quality of iaboratory medicine services to the users in primary care and in intensive care. The development of an open laboratory system architecture which allows the encapsulation or irxcorporation of different existing systems is a challenge for the OpenLabs project, Networked decision support systems with distributed database processing are possibiiities for the future. Decision support systems can become co~nen~ of ~~ysic~~~~ works~tions that are connected to multiple systems and to databases with standardised data exchange and inter&ace protocols. 5. Acknowledgement This work is partially funded by the Commission of the European Communities in the Research Programme AIM (Telematic Systems in Health Care) under the OpenLabs Contract ~A~~~g~.

McNair P, O’Maore R, Brender J. Data reduction and informatian enhancement in clinical decision making. In: Barber B, Cao D, Qin D and Wagner G, eds. Proceedings of the 6th World Congress on Medical Informatics; 16-20 October 19139,Beijing and Singapore. Amsterdam: North-Holland, 19&9:201-203. change and laboratory test volume. In: Benson ES, Rubin M, a%. 2 Finkeistein SN. ~~hn~o~~ ~~~~ oftbe conference: The &&c and Economics ofCknical Laboratory Test Se&&on and Use; 4-7 March 1978, Cancun. Mexico. Amste~am: Bi~vjer-North HoBand, f9?8225-237. 3 McDonald CJ, Smith DM, Tiemey WM. Cohen SJ, Weinberger M, M&&e GP. Reminders to physicians from an introspective computer medical record. Ann Intern Med 1984;10th130- 138. 4 Fowkes FGR. Containing the use of diagnostic tests. Br Med J 1985;290:488-489. 5 Speicher CE, Do duplicate chemistry profiles correlate with multiple physicians? Arch Pathol Lab Med 1988;112:235-236. 6 Bradshaw KE, Gardner RM, Pryor TA. Development of a computer&d laboratory alerting system. Comput Biomed Res f989$2575-587. and 7 Shabot MM, Lo&e M, keyede BJ, Dubin SB. Decision support alerts for c&&al narrator blood gas data, fnt J Cii~ Mot& Comput 1990;7:27-3L 8 Goat R. The integration and ~ommuni~tio~ iofpatientrecords in primary and secondary heafthcare computing. In: Richards B, ed. Current perspectives in health care computing; Proceedings of the conference Healthcare Computing; 25-27 March 1991, Harrogate, UK.. Weybridge, Surrey: Br J Healthcare Comput 1991. 9 Fraser CO. Interpretation of clinical chemistry laboratory data. Oxford; Blackwell Scientific Publications, 1986, 10 Browning MCK, Ford RF, CaBaghan SJ. In&a- and in~rin~vidua~ biological variation of five an&ytes usei[ in assessing thyroid function: ~p~~~ati~ns for szeazwfy standards ofperformanee and the int~~a~o~ of resuhs. Chn Chem 1985;32:%2-966. 1

Wink& P. The application of expert systems in the cfinical laboratory. Clin Chem 198935: 1595-1600. Dito WR. An octal algorithm for pattern coding and computer-assisted interpretative reporting. Am J Clin Path01 19773683575-583. McConnell TH, Ashwortb CT, Ashworth RD. Algorithm-derived, computer+%mrated interpretive comments in the reporting of laboratory tests. Am J Clin Pathol 1979;72:32-41, Hyde TA, The interp~~atio~ of in vitro tests of thyroid function with a ~~n~~#rnput~r. Am J Clin Pa&o1 f981;75:70-74. MeNair P, Brender J$ Ladefoged S, fmpaet on resource ~ons~p~o~ from ap~~~~a~~on of a sequentiat test selection strategy, In: @Moore R, Bengtsson S, Bryant JR, 3ryden JS, eds. Lecture notes in medical informatics. Proceedings of 9th internatianal Congress of the European Federation for Medical lnformatics (MIE), 20-23 August 1990, Glasgow. Berlin: Springer-Verlag, 1990;40: 381-387. Vogt W, Sandel P, Schwarztischer P. Cluster-oriented discriminant analysis: taxonomic classifica. tion of the thyroid function. Clin Chim Acta 1981;112:213-223. Coomans D, Broeekaert 1, Jonckheer M. Comparison of muftivariate discr~~~nat~o~ techniques for ciinicai data-appl~~tion ta the thyroid ~un~~jonal state. Methods Inf Med 1983;22:93- 101. Coomans D> Broeckaert 1, D&e MP. Use of a rn~ero~~puter for the defmition of muhivariate confidence regions in medical diagnosis based on clinicat laboratory profiles. Comput Lamed Res 1984;17:1-14. Kulikowski CA. Expert systems for thyroid function testing. Diagn Ned X981;2:99-102. Saarinen K, Irjala K, Nuutila P, NykHnen P. A knowledge based system in a hospital - a discussion of improvements in clinical practice, In: O’Moore R, Bengtsson S, Bryant JR, Bryden JS, eds. Lecture notes in medical informatics. Proceedings of 9th International Congress of the European Federation for Medical Xnformatics (MIE), 20-23 August 1990, Glasgow. B&n: Springer-Verlag, ~~~~:343-~7, Brosnan P, Boran G, Crimson .FB3O’Moore. RR. A decision support system for the assessment of thyroid hnction. Automedica t9887;8:169-178 Nolan J, Brosnan P, Murnane L et ai. A PC-based de&ion support/patient management system For thyroid disease. In: Hunter J, Cookson J, Wyatt J, eds. Lecture notes in medical informatics. Prow ceedings of the 2nd Europwn Congress on Artificial intelligence in Medicine; 29-31 August 1989, London. Berlin: Springer-Verlag, 1989;38:189- X98. Talmon JL, Braspenning P, Brender J, McNair P. Machine teaming in data rich domains: some experiences from the KAVAS-project. In: Stefanelti M, Hasman A, Fieschi M, Talmon J, eds. Lecturn notes in medical ~n~~~ati~. Proceedings of the 3rd European Conference on Artificial B&&igence in Medicine, 24-27 June $991, Maastricht. Be&n: Sp~ng~r-Ver~a~~ ~~91;~28~-293~ Boran G, EIdridge P, N&n 3, Brosnan P, O’Moore R. A decision support tool for laboratory medicine based on automatic knowledge acquisition. In: O’Moore R, Bengtsson 5. Bryant JR, Bryden JS, eds. Lecture notes in medical informatics. Proceedings of 9th International Congress of the European Federation for Medical lnfonnatics (MHZ), 20-23 August 1990, Glasgow, Berlin: SpringerVerlag, 1990;40:329-333, Kestin IG, Miller BR, Lockhart CH. Auditory alarms during anaesthesia monitoring, Anaesthesiology 1988;69 IO6- f09. Astion ML, WiMing P, Appbcation of neural networks to the ~nte~~~atj~~ of taboratory data in cancer diagnosis. Ctin Cbem f99%38:34-38. Rumelhari DE, Hinton GE. Wifhams Rd. Leawniag &~ernat representations by error propagation. Nature

28

1986;323:533-536.

Cicchetti DV. Neural networks

and diagnosis

ia the clinical

laboratory.

Clin Chem

1992;38:9-IO.