Retrospective validation of a physiological model of glucose-insulin interaction in type 1 diabetes mellitus

Retrospective validation of a physiological model of glucose-insulin interaction in type 1 diabetes mellitus E.D. Lehmann*,

I. Hermanyi’ and T. Deutsch*

*Diabetes Research Laboratory, Medical Unit (4NW), Department of Endocrinology and Chemical Pathology, United Medical and Dental Schools of Guy’s and St Thomas’ Hospitals (University of London), St Thomas’ Campus, London, UK; ‘First Department of Internal Medicine, Erzsebet Hospital, Budapest, Hungary; %omputer Centre, Semmelweis University of Medicine, Budapest, Hungary and Centre for Measurement and Information in Medicine, Department of Systems Science, City University, London, UK ABSTRACT We have previously described a physiological model of glucose-insulin interaction in insulin-dependent (type 1) diabetes mellitus which has been developed for patient and medical staff education about diabetes mellitus, as well as possibly for clinical use. The model attempts to re$?ect the underlying (patho)physiology of insulin action and carbohydrate absorption in quantitative terms such as insulin sensitivity, volume ofglucose and insulin distribution and maximal rate ofgastric emptying. The model'spredictions also allow a 24 h simulation ofpatient bloodglucose profiles to be generated. Advice is provided by a qualitative knowle&e based system which suggests what the next step in improvingglycaemic control might be for a given patient, e.g. ‘increase before breakfast long-acting insulin by 2 units’. Validation work pe7formed on a previous version of the knowledge based system has demonstrated that it can provide qualitative advice comparable to that of a clinician. Furthermore, bench testing of the predictive accuracy of the model has yielded encouraging results. We therefore set out to perfOrm a preliminary retrospective medical validation of the physiological model using data collected by 30 insulin-dependent diabeticpatients attending diabetes out-patient clinics at various centres throughout Europe. We found that thephysiological model could only be parameterizedfor datafrom 24 (80%) of the 30patients in the study. Comparison of observed and predicted blood glucose data from these 24 patients over a period of 5-6 days following parameter estimation revealed a mean (? SD) root mean square deviation between measured and simulated blood glucose values of 1.93 + 0.86mmoll~‘. T?te implications of these results are discussed. Keywords:

Validation, insulin dosage adjustment, diabetes mellitus, computer simulation

Med. Eng. Phys., 1994, Vol. 16, 193-202,

May

INTRODUCTION Diabetes mellitus is one of the major noncommunicable chronic diseases in Western society. It results from under production or reduced action of the hormone insulin and is characterized by high blood glucose levels. It is a lifelong disorder with a number of debilitating and life-threatening complications. These include blindness, renal failure, amputations and heart attacks as well as the more acute problems of hypoglycaemic and ketoacidotic coma. Once diagnosed insulin-dependent diabetic patients require insulin therapy for the rest of their lives. If the insulin regimen is chosen appropriately then the blood glucose profile can be well controlled, reducing the incidence of h poglycaemic and the hyperglycaemic episodes an dy decreasing incidence and severity of later life complications’32. A number of computer based approaches to assist in the management of insulintreatment or long-term Correspondence and reprint requests to: Dr E.D. Lehmann, Division of Medicine, 4th Floor, North Wing, St Thomas’ Hospital, Lambeth Palace Road, London SE1 7EH, UK 0 1994 Butterworth-Heinemann 1350-4533/‘J4/03 11x- IO

for BES

dependent diabetic patients have been previously reported in the literature ‘. These include knowledge based systems (KBS) to advise on patient management in out-patient clinics4, computer algorithms for insulin dosage adjustment”-‘, ap roaches drawing on and the use of causal probabilistic reasoning 8 mathematical models as a means of predicting or simulating patient blood glucose levels’@‘2. Our approach for making the required expertise for the management of insulin-dependent diabetic patients more widely available is different in that it is based on an integrated methodology which links ‘period orientated’ knowledge based reasoning with qualitative insulin algebra and uantitative modelling in order to provide advice to If, e referring clinician. Evaluation is an essential and integral part of a prototype’s development’“. It helps system developers identify deficiencies in their system thereby permitting modifications and improvements to be made, as well as also assisting in assuring end users of the quality and safety of the finished work14. Any strategy for the evaluation of decision-support systems needs to comprise at least four different levels

Validation of a model of type I diabetes: E.D. Lehnana et al.

of analysis: verification, validation, human factors assessment and clinical assessment15. Verification refers to the internal static checks in the knowledge base which can be performed without test cases. It has two main components, the testing of the rule set or other knowledge representation for internal consistency and the presentation of the knowledge base to external experts in a readable form for judgements concerning its validity. Validation refers to testing performed in order to check the accuracy of the results given by the system. This must normally be performed both at the level of the individual module - does this component produce the results intended when used in isolation; and at the level of the integrated system - do all the components’ working together produce the intended results. The results of validation studies are normally judged against clinical criteria. In the absence of an absolute or ‘gold’ standard this normally involves some measure of agreement with the consensus of a panel of experts. Human factors assessment addresses such questions as whether the system is useful and usable and whether it will meet user requirements, while clinical assessment refers to testing via simulations or field trials as to whether the use of the knowledge based system in its intended environment is effective in improving the process and outcome of clinical care. Normally this latter assessment is a test of a system plus a user compared with either an unaided user or a user aided by an alternative system15. While four distinct components of the evaluation process can be identified, it is necessary for these to form an integral part of any system’s development. Figure 7 shows how the evaluation process should ideally be tightly coupled with the development cycle of a decision support system. In this paper an overview is provided of the rototype computer system which is intended for e B ucational use as well as possibly as a therapeutic aid for the clinical management of insulin-dependent diabetic patients. A case study is used to demonstrate some of the principles of the model based component of the system in operation, and the results of preliminary verification and validation work performed on the physiological model are described.

VERIFICATION

SYSTEM

OVERVIEW

We are interested in established insulin-dependent diabetic patients who require restabilization of their blood glucose profile. In such patients blood glucose levels are monitored at home several times per day and these measurements, along with the registered of any hypoglycaemic episodes, occurrences represent the observations upon which the insulin regimen and/or diet plan can be ad’usted in order to achieve an improved degree of g i ycaemic control. e developed to achieve this is an The proto integrated P? based computer system which consists of two main components: a ph siological model and a knowledge based system (KB J ). A description of the structure of the integrated proto e has recently appeared elsewhere in this journal %p. The model, which assumes a patient completely lacking endogenous insulin secretion, contains a single extracellular glucose compartment into which glucose enters via both intestinal absorption and hepatic glucose production. Glucose enters the portal circulation via first-order absorption from the gut; the rate of gastric emptying which provides the glucose flux into the small intestine being controlled by a complex process which has been defined as a function of the carbohydrate content of the meal ingested17. Glucose is removed from the extracellular s ace by insulin-independent glucose utilization in tKe central nervous system and red blood cells as well as by insulin-dependent glucose utilization in the liver and periphery. Hepatic and peripheral handling of glucose in the model are dealt with separately; the net hepatic glucose balance being computed as the sum of gluconeogenesis, glycogen breakdown and glycogen synthesis data derived for different blood glucose and insulin levels from nomograms given in Gu on et al.18. Glucose excretion from the extracel i” ular space takes place above the renal threshold of glucose as a function of the creatinine clearance rate. The only insulin input into the model comes from the absorption site following subcutaneous injection; the pharmacokinetics of insulin absorption being derived from a recent description of that process by Berger and Rodbard lg. The model contains separate compartments for plasma and ‘active’ insulin, the latter being responsible for glycaemic control while insulin is removed from the former by hepatic degradation. Insulin affects the net hepatic glucose balance characterized by a liver sensitivity parameter,

BENCH TESTING

USER EXPERIENCE ad PROTOTWlNG ad CONSENSUS

cAREloHnJRmE INTAKE

UTILlSATlON

WNTENANCE

Figure 1 Development-evaluation spiral systems (adapted from Engelbrecbt et al. “)

194

Med. Eng. Phys. 1994, Vol. 16, May

for

decision

support

Figure 2 Compartmental Lehmann and Deutsch’“)

structure of the model (modified from

Validation of a model of type I diabetes: E.D. Lehmann et al.

SJ,, as well as enhancing peripheral glucose utilization described by a peripheral sensitivity parameter, Sp; peripheral glucose uptake being saturable with respect to blood glucose. Figure 2 summarizes the compartmental structure of the model which has been described in detail elsewhere in this journa117~20. The algorithm used for parameter estimation of values for Sh and S, determines estimates which give the best ‘fit’ between the observed and predicted data. Fit is assessed using data-trend sensitive least squares criteria to calculate the root mean square (RMS) deviation between the observed and predicted blood glucose data sets at the observed blood glucose time points. RMS values are calculated using the equation, RMS = v\/cd’/(n-np)

(1)

between each pair of where d is the difference observed and predicted blood glucose readings, n is the number of pairs of blood glucose values and np is the number of arameters in the arameter estimation procedure P2; S,, and S,). In B etermining the fit hy oglycaemic episodes are assigned a blood glucose va Pue of 1.0 mmolll’. Parameter values for which there is a conflict of trends between the two data sets in any time period are assigned a very poor fit b using an empirical, ‘penalty’ score for such cases2 Y. Following parameter estimation if the best fit obtainable is greater than 3 mmoll-’ then the user is informed that it is not possible to fit the model to the data sufficiently accurately to permit individual patient parameterization and simulation to be performed. A best fit value <3mmoll-’ was found, by inspection, to be the upper limit of acceptable parameter estimation. In principle the glycaemic response of an insulindependent diabetic patient goes through transitory phases leading to a steady state glycaemic profile Blood

ao

CASE STUDY

Figure 3 shows the front end of the model with clinical and nutritional data collected by a 60 kg, insulindependent diabetic patient on intensive insulin therapy (injections four times per day). The distribution of the log carbohydrate bread equivalents that the patient consumed can be seen in the lower panel Liver:

Level

0.70

PmriaherY: 3

z \

Olucosa

following a change in either the diet or insulin regimen. The purpose of our model is to simulate these theoretical steady state responses which would be produced assuming no intra-individual variation and external disturbances. Since the length of action of some insulin preparations may exceed 1 day simulations are carried out over a 2 day (48 h) period and the second day’s blood glucose and lasma insulin profiles are assumed to represent stea B y state profiles as responses to the current insulin therapy and diet plan. These profiles are displayed on the computer screen as the results of the simulation and are also used for parameter estimation based on home monitoring blood glucose data. The aim of the KBS which has been linked to the model is to provide alternative suggestions whereby the quality of the patient’s glycaemic control can be improved1”,2’. The current version of the system runs under DOS on an IBM PC or compatible computer. A multitasking version of the program is also available for 80486 based machines running WINDOWS ~3.1. This allows the display of multiple windows showing different parts of the system in operation. For example the data entry screen can be displayed in one window with the results of a simulation in a second and advice from the KBS in a third; the number of windows displayed at any one time being wholly dependent on the memory capabilities of the machine.

15

0. so

Fit:

4.6

nnol/l

DFN:

4.5

nmol/l

u 0

3

6 Insulin

9 and

1 im

xourr,

Carbohtirrte

15

18

a1

a4

Intake

Figure 3 Front end of the model with a simulation superimposed after parameter estimation has been performed. Upper panel: observed (0) and predicted blood glucose levels. Lower panel: insulin and carbohydrate intake with predicted plasma insulin curve

Med. Eng. Phys. 1994, Vol. 16, May

19.5

Validation of a model of type I diabetes: E.D. Lehmann et al.

intake. The overall RMS deviation (‘Fit’) between observed and predicted blood glucose values obtained following fitting was 2.7 mmol 1-r for hepatic and peripheral insulin sensitivities of Sh = 0.7 and s, = 0.5, respectively. Having fitted the model to the patient’s observed blood glucose data it is now possible to use the model to simulate the glycaemic effect of alterations to the patient’s therapeutic regimen. Figure 4 demonstrates some examples of this. Figure 4a shows the predicted effect on the patient’s simulated blood glucose profile of increasing the loam mid-morning snack from

along with the four times daily regular (Actrapid) and intermediate acting (Lente) injection regimen that the atient was on. The upper panel shows the observed 1 lood glucose readings recorded by the patient using a portable electronic lop. Superimposed on these graphs are the results of a 24 h simulation, as predicted by the physiological model, after parameter estimation has been performed. The lower panel lasma insulin curve for the shows the simulated patient’s current insu Pin regimen while the upper rohle predicted panel shows the blood glucose for the patient’s current carbohy B rate and insulin

Blood

0

a

3

6

Dlucora

9 Tim

Blood

20

12

Olucosc

0.70 Liurr: Perimhw-Y: 0.50 Fit: 2.3 nnol/l DFN: 4.6 rrnol/l

Laucl

15

19

21

Liver: 0.70 PlriDhrrY: 0.50 Fit: 5.1 nnol/l DFN: 7.6 mrrol/l

Lavcl

"

b

’

3

6

9

Blood

0

15 Tim

3

6

Tim Insulin

and

19

21

24

::hour,>

Qlucosc

9

24

Liver: 0.70 Plriaherr: 0.50 Fit: 2.7 mnol/l DFN: 3.2 mnol/l

Level

12

CarbohYdratc

15

18

21

24

Intake

? ?Actrwid

w

196

Med. Eng. Phys. 1994, Vol. 16, May

Carbohrdrato intrkr

Validation of a model of type I diabetes: E.D. Lehmann et al.

Blood

i

o

PRriDhmM:

Level

0

3

3

6

6

9 Tlrn

12

Tim

:Lrr,

9

15

12

21

15

18

21

0. so

Fit:

2.4

rrmol/l

DFN :

3.0

nnol/1

1, , , , , , , , , , , , , , , , , , , , , , , 0

d

Olucorc

( 24

a4

OlDA

~2.3

Figure 4 Educational examples of the use of the system. a, Increasing the carbohydrate content of the 10 am mid-morning snack from 20 g to 50 g in log increments. b, Reducing the 10 pm (bedtime) Lente injection from 6 units to zero in I unit increments. c, Moving the 7:30 am before breakfast Actrapid injection from 7 am to 8:30 am in 15 min steps. d, Demonstrates the effect on the simulation of combining the previous split evening dose, following the doctor’s correction of the hypoglycaemic episode, into a single Actrapid and Lente injection at 7 pm

its original 20 g of carbohydrate to 50 g (in 10 g increments). The accompanying hyperglycaemia which would result is clear to see. Figure 4b shows the predicted effect on the patient’s blood glucose profile of reducing the bedtime (10 pm) Lente injection from 6 units to zero in 1 unit increments; the resulting rise in glycaemia being evident. Figure 4c demonstrates the effect on the patient’s blood glucose profile of changing the time of the early morning insulin injection with res ect to the meal. In this example the 7:30 am Actrapi B injection has been moved in 15 min steps from 7 am to 8:30 am and the effect on the patient’s blood glucose profile simulated. As can be seen having the injection too early leads to relatively low blood glucose levels between 9 am and 11 am whereas having it too long after the meal results in a marked hyperglycaemia. These simulations demonstrate how the system could be used in an educational setting, allowing a patient, student or member of the medical team to ‘dial in’ changes to a patient’s diet or insulin regimen and observe the predicted effect on the patient’s blood glucose profile. Figure 4d provides another example of how the system could be used in this way. The figure shows a simulation from the same patient as in Figure 3, following the advice of a doctor, which has reduced both the before breakfast Actrapid dose and the before bedtime Lente dose by 2 units each. As can be seen this adjustment is predicted to correct the previous ‘hypo’ which occurred at lo:30 am. In Figure 4d, however, two simulations are shown. In the first simulation the patient had a split evening dose, injecting 3 units of regular insulin (Actrapid) at 7 m and 4 units of intermediate acting insulin (Lente P at 10pm. In the second simulation these two evening injections have been combined into a single Actrapid

and Lente injection administered at 7 m and the effect on the patient’s blood glucose pro File has been simulated. As can be seen, there are now on1 three peaks in the patient’s plasma insulin profile rcorresponding to the three injections). Furthermore the change in the therapeutic regimen ap ears to have had a beneficial effect on the patient’s g lood glucose profile, increasing the glycaemic level in the morning when it was previously low and decreasing it overnight when it was previously raised. CLINICAL

ASSESSMENT

Clinical evaluation is an imprtant and integral part of a system’s development 3. It is a wide-ranging process, considering features such as efficacy (does it help or alter patient management?), sensitivity (how would conclusions vary with small changes in quantitative input?) and consistency (are there contradictions in the knowledge base?). Evaluation should also be able to provide an insight into the ease of use, clinical acceptability and costing of the system although these are usually more realistically studied in prospective hospital and primary care trials24. Medical validation is that part of the evaluation process which concentrates on the fundamental issues of medical accuracy, acceptability and scope2” whereas verification is a more static process which aims to ensure that the representation of knowledge coded in the system is correct; the goal being that the results of such work should be able to determine what modifications to a system (if any) are required prior to proceeding to a more substantial and time consuming clinical evaluation. As a first step towards a full scale clinical evaluation of the whole integrated prototype work has been

Med. Eng. Phys. 1994, Vol. 16, May

197

Validation of a model of type I diabetes: E.D. Lehmann et al.

carried out to separately validate the individual components of the system. The first part of the prototype to be formally evaluated has been a former version of the KBS; the resu&tssof this work being . In the rest of this described in detail elsewhere section we shall focus on preliminary retrospective validation work which has been carried out on the model-based component of the system. Verification Verification of the system is possible in a number of different ways. The educational examples presented in Figure 4 allow clinicians to try out the model with insulin-dosage and dietary adjustments and a praise the appropriateness or otherwise of the simu Pations. When two consultant physicians did this they found the simulations to be realistic. Furthermore they were able to change the patient’s regimen and design new regimens for hypothetical patients with great ease. While this is an important part of the validation process, ex erimenting with the system in this way does not al Pow one to investigate or verify the actual workings of the prototype. This capability is, however, provided by another function of the system which is demonstrated in Figure 5. The figure shows how it is possible to look in more detail at the simulation shown in Figure 4d, following the doctor’s correction of the hypoglycaemic episode. As before the blood glucose and plasma insulin curves are shown but this time with the different glucose fluxes within the model. ‘UPTAKE is the grouped peripheral, central nervous system and red blood cell utilization of glucose; ‘NHGB’ is the net hepatic

Blood

ao-

15

--

Olucosc

glucose balance; ’GLUCOSE ABSORPTION’ refers to the systemic appearance of glucose from the gut and ‘RENAL EXCRETION’ describes the loss of glucose via the kidneys into the urine. It should be noted that the time scales on the x-axes are removed when using this function in order to allow the graphs to be viewed more easily. Also the graphs are not scaled on the y-axis; as such values can only be read off by use of the clock function. As can be seen the rate of glucose uptake is at a maximuum (72.2 mmol h-‘) at 9: 15 am when the rate from the gut peaks at of glucose absorption 85.0 mmol h-’ following breakfast at 7:30 am. As the plasma insulin level is relatively high (4 1.7 mU 1-l) at this time, following the early morning (7:30 am) injection of 6 units of Actrapid and as the blood glucose level is elevated at 9:15 am the liver is predominantly taking up glucose from the plasma. Thus the net hepatic glucose balance is negative (- 26.1 mmol h-l). Furthermore, as the blood glucose level (10.3mmoll-‘) is above the renal threshold of glucose (default value = 9.0 mmol 1-l) glucose is being excreted via the kidneys into the urine (10.6mmolh-’ at 9:15am). After 9am, the fall in plasma insulin combined with the fall in glucose absorption from the gut between 9:15 am and 10: 15 am contrive to cause a decrease in peripheral glucose uptake and a compensatory increase in the net hepatic glucose balance. While the system has an infinite number of inputs which cannot all be tested, such verification work with a wide range of clinical examples does enable the general integrity of the model to be checked; any inappropriate or surprise findings being investigated

Level

LiWW-:

0.70

PerirrhrrY:

0. so

Fit:

a.4

UPTAKE

-0 I

8

8

I

’

I

’

’

I

’

‘V’ Umtmkr

’

I

I

’

I

.

t

9 lucorr rwOl/l

1 72.2

WIOB

IWlOlNh

-26.1

Plasma 41.7

80

. --

lood

10.3

\

9:15

Tim:

slo- 0

m~l/l

nmol/h insulin nicroWn1

70 m

60

ktrraid

SO 40 30

0 lucosr

20 ““0

10

l/h

0

EXCRETION Figure 5 ‘UPTAKE

Simulation shown in Figure 4d in ‘WORA7AG.S’mode allowing the different fluxes within the glucose model to be viewed interactively. is the grouped peripheral, central nervous system and red blood cell utilization of glucose; ‘NHGB’ is the net hepatic glucose balance; ‘GLUCOSE ABSORPTION’ refers to the systemic appearance of glucose from the gut and ‘RENAL-EXCRETION’ describes the loss of glucose via the kidneys into the urine

198

Med. Eng. Phys. 1994, Vol. 16, May

Validation of a moo21of type I diabetes: E.D. Lehmann et al.

closely to determine whether the equations are being handled correctly and whether the model needs to be altered in any way to improve the simulation. Having performed such static testing, the next stage in the evaluation process is to ascertain the predictive capability of the system using clinical test cases. Validation We have previously described an evaluation protocol which could be used for the prospective medical validation of a decision support system such as that described here2’. Leicester et ~1.~’ have also suggested statistical analyses which could be used to assist in the interpretation of such data. As a pilot study, prior to carrying out such a formal prospective validation, we have performed a preliminary retrospective study of the predictive accuracy of the physiological model. Figure 6 demonstrates the use of the system in a multi-tasking environment and shows how this can be

a

b Figure 6 a, Demonstrates the use of the system in a multitasking environment; clinical and nutritonal data being given in the ‘DATA’ window while a simulation following parameter estimation is shown in the ‘DISPLAY’ window. b, A simulation for the following day is shown after a change in the patient’s therapeutic regimen as advised by the patient’s own doctor; new clinical and nutritional data being given in the ‘DATA’ window. The simulation, using the same parameter values as for Figure 6a, is shown in the ‘DISPLAY’ window while the ‘FIT’ window demonstrates how the multitasking environment can be used to help in validation work; the observed and predicted blood glucose readings being shown along with the value of the root mean square tit calculated by the system

used to assist in the validation process. The ‘DATA’ window in F&we 6a shows clinical and nutritional data from a 70 kg insulin-dependent diabetic patient who was on a mixed insulin regimen of Actrapid and NPH (isophane) with a split evening dose (receiving three insulin injections in total per day). A simulation following parameter estimation is shown in the main (‘DISPLAY’) window; the values for the parameters S,, and S, selected by the system being 0.1 and 1.0 respective1 . Figure 6P shows a simulation for the same patient a day later after a change in the patient’s therapeutic regimen, as advised by the patient’s own doctor. The same ammeter values (,I$ = 0.1 and S, = 1.0) determine B for the first day have been used to simulate the effect of the doctor’s advice; decreasing the Actrapid injection at 5pm by 1 unit and adding 1 unit of Actrapid and NPH at 10:30pm (the patient also having a much earlier mid-morning snack and a slightly later afternoon snack). The ‘FIT’ window in Figure 6b gives values for the observed blood glucose readings recorded by the patient using a portable electronic 10$~ along with the blood glucose levels predicted b the model. In this window the system has also ca lyculated its own predictive capability in terms of the RMS deviation or ‘fit’ between the two data sets; fit being computed using equation (1). In this particular case the fit between the two data sets was 1.92 mmol 1-l. This approach has been adopted in order to allow a quantitative assessment to be made of the predictive accuracy of the model. Blood glucose data, insulin dosage information and carbohydrate intake mealrelated data were collected over a 5-6 day period from 30 insulin-dependent diabetic patients attending diabetes out-patient clinics in the Erzsebet Hospital [Budapest] as well as in the Diabetes Centre Bogenhausen [Munich], Istituto Patologia e Metodolo gica Clinica [Perugia], St Thomas’ Hosyftal [London] and Hospital Ramon y Cajal [Madrid] . The data for a particular patient were entered into the system for the day before a change in the insulin injection or dietary regimen (defined as day 1). Parameter estimation was performed on these data and the values of S,, and S, determined in this way were used to simulate the effect of changes in the thera eutic regimen for day 2. Parameter estimation coul B only be performed on data from 24 (80%) of the 30 patients in the study; crossover trends17 between observed and predicted data preventing the model from being used with data from the other six patients. Upon simulation of the new insulin dosage or dietary regimen the RMS deviation between observed and predicted blood glucose profiles was automatically calculated. These calculations were performed at each observed blood glucose time point and a mean value for the RMS fit determined for that particular simulation (Figure 6b). This process was repeated for all 24 patients over a period of 4-5 consecutive days yielding a total of 578 pairs of blood glucose measurements for simulation over 94 days; data from the first day of the study was not used in order to allow theoretical steady state conditions to apply. The RMS values for the fit obtained ran ed from 0.8 to 4.6 mmol 1-l with a mean error (+ SD$ of 1.93 f 0.86 mm01 1-r.

Med. Eng. Phys. 1994, Vol. 16, May

199

Validation of a model of type I diabetes: E.D. Lehmann et al.

0’ 0

1

2

3 DAY

4

5

6

(number)

Figure 7 Root mean square (RMS) deviation between observed and predicted blood glucose data values as a function of time from parameter estimation for one of the patients in the study

Figure 7 shows how the RMS deviation between observed and predicted blood glucose values varied over a period of 6 days for one of the patients in the study; parameter estimation having been performed on day 0. Re-estimation of the insulin sensitivity parameter values for day 5 of the study resulted in an improvement in the RMS deviation between observed and redicted blood glucose data from 3.4 to 2.2mmoll- P. DISCUSSION The model presented here focuses on the adjustment of insulin and/or diet in the insulin-dependent diabetic patient. In developing the model we have attempted to find a concise mathematical formulation to represent the body’s glucoregulatory system ossible in physiological terms with the fewest ammeters. As such the model has intentiona Ply been E ept simple because with increasing complexity the number of parameters increases as do the difficulties of determining their values for individual patients. When fitting the model to a particular patient the parameter estimation routine not only minimizes the least s uares difference between observed and predicted 1 ata sets but also assesses the direction of change in the data. In this way it is possible for the computer to reject ammeter values for which there is a good ‘traditiona P fit’ as assessed by least squares criteria, but clearly contradictory trends in the observed and simulated data. If no parameter values satisfy both criteria then the computer informs the clinician that the model cannot be used with that patient’s data. We have shown in previous work that the insulin sensitivity parameters Sh and S, estimated for one set of patient data on one daJ may not necessarily be accurate several days later . In the current study we have found that the RMS deviation between the observed and predicted blood glucose values became s stematically worse as time progressed from the d ate of the original parameter estimation (Figure 7). Hosker et aLs2 have reported that patients’ insulin sensitivity can vary quite appreciably day-by-day. Furthermore, we have found with our model that

200

Med. Eng. Phys. 1994, Vol. 16, May

re-estimation of patient specific parameters can lead to a significant improvement in the fit obtained upon simulation. Given this, it is clear that the estimation of insulin sensitivity parameters for an individual patient needs to be part of an on-going process. One possible way of achieving this would be to create a dynamic physiological model of glucose-insulin interaction in type 1 diabetes mellitus. In such a model, insulin sensitivity parameter values could be continuously updated as new observed blood glucose measurements became available. If the simulated blood glucose profile no longer closely approximated to the measured blood glucose data then re-estimation of insulin sensitivity parameter values could automatically be performed. While the preliminary validation work described here has demonstrated that the model can make reasonably accurate glycaemic predictions for some patients the exact proportion of patients for which the approach can be used still needs to be ascertained in a large scale randomized study; although clearly the 80% success rate observed in this preliminary study is One fundamental question, very encouraging. however, which still needs to be addressed is what sort of predictive accuracy will be re uired by such systems before they become clinical 9 y useful. For example is a predictive capability with a mean error (*SD) of I.93 + 0.86 mmoll-’ likely to be helpful to physicians when making clinical decisions? Even if the system is found to be unsuitable for individual patient parameterization and simulation, we believe that it will still have application as a general teaching tool; its educational use being enhanced by its ability to identify problems in a patient’s blood glucose profile and suggest remedies33”5.

Future work As with all prototypes there are a number of areas in which the current system could be improved. As outlined above, parameter estimation is performed at present using an exhaustive search enumeration algorithm, by trying to simulate the glycaemic response for all combinations of two discrete insulin sensitivity parameters, & and S,. Furthermore, once the best fit is achieved no estimates are provided related to the errors associated with such estimates. Also many important processes such as exercise and stress are not included in the model. These problems are either related to (i) the extent and quality of the knowledge that is included in the system (e.g. missing processes or simple parameterization that has not yet been rigorously evaluated), or (ii) the functions the system is intended to perform (e.g. no error estimates associated with parameter estimation and no simulation of transient conditions), or (iii) the computer implementation of the system (e.g. its lack of a polished front-end and a flexible interface to data collection and other pre-processing systems). These restrictions are intended to be addressed by the work outlined below, which is planned to be undertaken in the Department of Endocrinology at St Thomas’ Hospital, London. Future work will

Figure 8 Demonstrates how the simulation results from Figure 4d could be presented with bounds providing the confidence intervals of the predicted blood glucose values over time

rewrite the parameter estimation routine using a more efficient algorithm (such as a Gauss-Newton method) which would allow the values and errors associated with more than two model parameters to be estimated. Furthermore, descriptions of processes that play important roles in interpreting blood glucose data including exercise and stress will be added to the model. It is also planned to present simulation results with bounds (as shown in Figure 8 for the simulated data from F&we 4d), by providing the confidence intervals of such predicted values over time. This uncertainty in the predictions would reflect how the uncertainty in the parameter estimates propagate through the model and appear as uncertainty in the final simulated blood glucose levels. These confidence intervals would allow a clinician to assess whether a new blood glucose measurement was in agreement with the ‘profile’ which has been constructed about the patient based on previous data. In the case of a deviation between the observed and predicted data the doctor may wish to analyse the possible reasons that may have given rise to such discrepancies. Alternatively, if a dynamic modelling approach is applied, in such a situation re-estimation could automatically be of model parameters performed. Given the encouraging results which have been reliminary retrospective obtained in the current, validation study describe B here it is planned to embark on a more comprehensive follow-up study shortly. In this blood glucose, insulin injection and nutritional data will be collected from a larger number of insulin-dependent diabetic patients for testing of the system once the new Gauss-Newton parameter estimation module is in operation. This study will address the following questions: 1.

2.

3.

4.

For what proportion of compliant, regular blood glucose monitoring insulin-dependent diabetic patients can parameter estimation be performed using the new Gauss-Newton approach? Can the physiological model be used to make reasonably accurate glycaemic predictions of the insulin-dosage adjustment advice given by a doctor or nurse? If so, what is the predictive accuracy of the compared with the blood glucose model, measurements actually made by the patient? If the model provides reasonable simulations for some patients but not for others, is it possible to

predict on the basis of the model’s past performance which patients the model will work best for? A formal multi-centre clinical validation study to address these issues is currently being planned jointly in the Department of Endocrinology at St Thomas’ Hospital, London and the First Department of Internal Medicine at Erzsebet Hospital, Budapest, Hungary. SYSTEM

AVAILABILITY

A copy of the software used to perform the simulations shown here and a user guide are available for health-care professionals without charge by writing to Dr E.D. Lehmann at the Division of Medicine, St Thomas’ Hospital, London SE1 7EH, UK. The system runs on IBM PC or compatible 803861486 based machines and requires approximately 1*/2 Mb of hard disk storage space. ACKNOWLEDGEMENTS This work was supported by grants from the EEC EASTERN Mobility Scheme (CIPA 3510 CT 927064) and the EEC AIM (Advanced Informatics in Medicine) Exploratory Action [EURODIABETA Project No. A10191 as well as by the loan of computer equipment from IBM (UK) Ltd and IBM Europe. REFERENCES PH, LauJ, Chalmers TC. Meta-analysisof effects of intensive blood-glucose control on late complications of type I diabetes. Lancet 1993; 341: 1306-g. 2. The Diabetes Control and Complications Trial Research 1. Wang

3. 4.

5. 6.

Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl JMed 1993; 329: 977-86. Third Workshop of the EASD Study Group DO IT. Situation analysis: Needs in diabetes care and existing IT solutions. Diab Nuts Metab 1993; 6: 173-82. Carson ER, Carey S, Harvey FE, Sijnksen PH, Till S, Williams CD. Information technology and computerbased decision support in diabetic management. Camp Meth Frog Biomed 1990; 32: 179-88. SkylerJ, Skyler D, Seigler D, Sullivan MO. Algorithms for adjustment of insulin dosage by patients who monitor blood glucose. Diabetes Care 198 1; 4: 3 1 1- 18. Albisser AM, Schiffrin A, Schulz M, Tiran J, Leibel BS. Insulin dosage adjustment using manual methods and

Med. Eng. Phys. 1994, Vol. 16, May

201

Validation of a model of tyju I diabetes: E.D. Lehmann et al.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16. 17.

18.

19.

20. 21.

22.

202

computer algorithms: a comparative study. Med Biol Eng Comput 1986; 24: 577-84. Chanoch LI-I, Jovanovic L, Peterson CM. The evaluation of a pocket computer as an aid to insulin dose determination by patients. Diabetes Care 1985; 8: 172-6. Andreassen S, Hovorka R, Benn J, Olesen KG, Carson ER. A model-based approach to insulin adjustment. In: Stefanelli M, Hasman A, Fieschi M, Talmon J, eds Proceedings of 3rd European Conference on Arttficial Intelligence in Medicine. Heidelberg: Springer Verlag, 1991; 239-48. Hovorka R, Andreassen S, Benn JJ, Olesen KG, Carson ER. Causal probabilistic network modeling - An illustration of its role in the management of chronic diseases. ZBMSystems J 1992; 31: 635-48. Albisser AM, Schulz M. Simulating human intermediary metabolism: OMNI et al. In: Marks V, Alberti KGMM, eds Clinical Biochemistry Nearer the Patient II. London: Bailliere Tindall, 1986; 59-67. Carson ER, Cobelli C, Finkelstein L. Mathematical Modelling of Metabolic and Endocrine Systems. New York: Wiley, 1983. Cobelli C, Ruggeri A. Evaluation of portal/peripheral route and of algorithms for insulin delivery in the closedloop control of glucose in diabetes. A modelling study. IEEE Trans Biomed Eng 1989; 30: 93-103. Wyatt J, Speigelhalter D. Evaluating medical expert systems: what to test and how? Med Inform 1990; 15: 205-17. Gaschnig J, Klahr P, Pople H, Shortliffe T, Terry A. Evaluation of expert systems: issues and case studies. In: Hayes-Roth F, Waterman DA, Lenat DB, eds Building Expert Systems, Reading: Addison Wesley, 1983; 241-80. Engelbrecht R, Moser W, Schneider J, Rector AL. Evaluation of knowledge-based systems: The EURODIABETA framework. Diab Nutr Metab 1991; 4 (Suppl.): 209-12. Lehmann ED, Deutsch T. AIDA’: A Mk 11 Automated Insulin Dosage Advisor. J Biomed Eng 1993; 15: 201-l 1. Lehmann ED, Deutsch T. A physiological model of glucose-insulin interaction in type I diabetes mellitus. JBiomed Eng 1992; 14: 235-42. Guyton JR, Foster RO, Soeldner JS, Tan MH, Kahn CB, Koncz L, Gleason RE. A model of glucose-insulin homeostasis in man that incorporates the heterogenous fast pool theory of pancreatic insulin release. Diabetes 1978; 27: 1027-42. Berger M, Rodbard D. Computer simulation of plasma insulin and glucose dynamics after subcutaneous insulin injection. Diabetes Care 1989; 12: 725-36. Lehmann ED, Deutsch T. Insulin dosage adjustment in diabetes. JBiomed Eng 1992; 14: 243-9. Lehmann ED, Deutsch T. An integrated decision support system to assist in the management of insulin-dependent diabetic patients: A case study. Diab Nutr Metab 1992; 5: 283-94. Braun D, Heinrichs HR. Monitoring of metabolic control using the Camit system: practical experience. Diabetes News 1989; 10: 6-10.

Med. Eng. Phys. 1994, Vol. 16, May

23. Wyatt J. Computer-based knowledge systems. Lancet 1991; 338: 1431-6. 24. EURODIABETA. Information technology for diabetes care in Europe: the EURODIABETA initiative. Diabetic Med 1990; 7: 639-50. 25. Roudsari AV, Leicester HJ, Lehmann ED, Hovorka R, Andreassen S, Deutsch T, Carson ER, Sonksen PH. A validation methodology for testing decision support systems for insulin-dosage adjustment. In: Adlassnig K-P, Grabner G, Bengtsson S, Hansen R, eds Lecture Notes in Medical Informatics. Berlin: Springer-Verlag, 1991; 45: 382-8. 26. Lehmann ED, Roudsari AV, Deutsch T, Carson ER, BennB, Sonksen PH. Clinical assessment of a computer system for insulin dosage adjustment. In: Adlassnig K-P, Grabner G, Bengtsson S, Hansen R, eds Lecture Notes in 1991; 45: Medical Informatics. Berlin: Springer-Verlag, 376-81. 27. Lehmann ED, Deutsch T, Roudsari AV, Carson ER, Benn JJ, Sonksen PH. A metabolic prototype to aid in the management of insulin-treated diabetic patients. Diab Nutr Metab 1991; 4 (Suppl. 1): 163-7. 28. Lehmann ED, Deutsch T, Roudsari AV, Carson ER, Benn JJ, Sonksen PH. An integrated approach for the computer assisted treatment of diabetic patients on insulin. Med Inform 1992; 17: 105-23. 29. Lehmann ED, Deutsch T, Roudsari AV, Carson ER, Sonksen PH. Validation of a metabolic prototype to assist in the treatment of insulin-dependent diabetes mellitus. Med Inform 1993; 18: 83-101. 30. Leicester HJ, Roudsari AV, Lehmann ED, Carson ER. Methodological issues in validating decision-support systems for insulin dosage adjustment. Artificial Intelligence in Medicine 1994; (in press). 31. Andreassen S, Bauersachs R, Benn J, Carson E, Gomez E, Hovorka R, Lehmann E, Nahgang P, de1 Pozo F, Roudsari A, Schneider J. Report on developed prototypes integrating KBS and other methodologies for insulin therapy advisory systems, EURODIABETA Technical Report to the EEC Advanced Informatics in Medicine Exploratory Action, 15, EEC-AIM, Brussels, 1990. 32. Hosker JP, Matthews DR, Rudenski AS, Burnett MA, Darling P, Brown EG, Turner RC. Continuous infusion of glucose with model assessment: Measurement of insulin resistance and p-cell function in man. Diabetologia 1985; 28: 401-11. 33. Lehmann ED, Deutsch T. An interactive, educational model for insulin dosage and dietary adjustment in type I diabetes mellitus. In: Frisse ME, ed SCAMC Proceedings, 76th Annual Symposium on Computer Applications in Medical Care, New York: IEEE Computer Society Press, 1992; 16: 205-9. 34. Lehmann ED, Deutsch T, Carson ER, Sonksen PH. Combining rule-based reasoning and mathematical modelling in diabetes care. Artificial Intelligence in Medicine 1994; (in press). 35. Lehmann ED, Deutsch T, Carson ER, Sonksen PH. AIDA: An Interactive Diabetes Advisor. Camp Meth Prog Biomed 1994; 41: 183-203.