- Email: [email protected]
5. Charter of the european Working Groups (Adopted by council on July 23, 1980)
36
Profiles of the European WorkingGroups
tists led to the next major event which was set up by Professor Ian Masser, then with the Instituut voor Planologie at the Rijksuniversiteit Utrecht in the Netherlands. Professor Masser initiated and organized what in the light of the two following meetings turned out to be the 1st International Workshop on Strategic Planning. It was conceived and implemented as a joint venture of the British section of the Regional Science Association workshop on regional science methods in strategic planning, of the Instituut voor Planologie, Rijksuniversiteit Utrecht, and the EURO Working Group "OR in Regional and Urban Planning", taking place at Utrecth on April 24-26, 1978. The workshop found very favorable resonance among the participants and Dr. Henk Voogd from the Department of Urban and Regional Planning of the University of Technology, Delft, Netherlands, spontaneously offered to set up a 2nd International Workshop on Strategic Planning at Delft in 1980. In the meantime proceedings of this workshop are available, entitled "Strategic Planning in a Dynamic Society", Henk Voogd (Ed.), Delftsche Uitgevermij, Delft, 1981. The Delft meeting, too, was a joint venture of the above mentioned British and EURO working groups and, last but not least, the Delft University Department of Urban and Regional Planning. One year later, on May 8-9, 1981, the 3rd International Workshop on Strategic Planning took
place at the Institut for Raumplanung of the University of Dortmund, Germany, most efficiently and generously organized by Dr. Michael Wegener. During the 3rd International Workshop the intent was expressed to set up a 4th Workshop in 1983 at some locality, not yet fixed, in Great Britain. In the meantime, this workshop has been postponed until 1984. There is a comment in order as to the professional affiliation of the attendants of the workshops. The vast majority of the participants of the three International Workshops would, on request, presumably declare themselves regional scientists rather than working in OR. At the first meeting at Erlangen, in spring 1976, there were about 50% of the attendants who were genuine OR people. As emphasis of the working group activities noticeably shifted towards strategic planning they by and large withdrew. In the meantime, preponderance of regional scientists in the workshops has grown so marked that it would appear natural to conduct future workshops no longer under the cosponsorship of the EURO Working Group. With these premises in mind and with the assent of the Vice President of The Association of European Operational Research Societies within IFORS, the chairman of the Working Group has decided to cease its activities (notwithstanding, naturally, the possibility of its reactivation by another convenor).
5. Charter of the European Working Groups (adopted by Council on July 23, 1980) Although the "Appendix to the Agreement" creating EURO on 20th January 1975 was not very explicit about how the European Working Groups (EWGs) should be organized, the existence of such groups was always considered as one of the main objectives of EURO. After five years of existence of EURO, it appears that a reasonable number of EWGs have spontaneously reached a certain degree of activity and development, and that it is time to establish more precise (however flexible) rules for them.
(a) Creation of groups The creation of new EWGs should be left to individual initiative, as was always done. Nonetheless, it might be useful to distinguish two cases: (1) creations while a EURO-K Conference is held, (2) creations between EURO-K and EURO-(K +1). In case (1), a convener can for instance begin by sticking up a bill proposing to interested persons to meet at specified time and place in a quite
Profiles of the European WorkingGroups
informal way. If he gets enough people together and if they agree upon a tentative schedule of activity, they produce a short report which is forwarded to the EURO Executive Committee; the latter can then either deliver the EURO label at once or postpone it after a further meeting of the group. In case (2), the convener either applies first to the Executive Committee, who will decide whether and under what conditions they will help him achieving his ideas, or he tries first on his own responsibility to get together enough people interested to work on his subject, and then applies to the Executive Committee. The foundation of the group should be announced in the Bulletin. In both cases (1) and (2), the EURO label means that E U R O considers itself informed well enough about the group and includes it into the list of its official activities.
(b) Activities EURO does not try to impose unified regulations for the way each working group displays its activities. Nonetheless a few general principles are worthwhile being recalled: (1) The EWGs should as far as possible bring together people with different professional backgrounds (industry, university, etc.). (2) They should be actually international; this requirement is met satisfactorily if at least 4, but possible 5 or 6 nations are represented at each meeting, and if the meetings do not take place too often in'the same country. (3) Any group should be open to any individual member of any EURO member society. Persons who are not members of a national OR Society are not excluded from the meetings; however, a friendly pressure should be exerted upon them to become members. (4) If one of the EWGs plans a joint meeting with some other (e.g. professional) body, it should provide EURO with general information about the latter. (5) As a general rule, EURO-K conferences will provide time and space for meetings of the EWGs which will be planned in connection with them. At
37
least for these particular meetings, no entrance-fee should be requested from participants. (6) Routine internal informatio~ of the groups, such as announcements of the next meeting and short reports about the last, should be conveyed to EJOR for publication in the " E U R O Bulletin". It will be up to each group chairman to maintain the connection with the Editor of the Bulletin (presently Dr. Soren Kruse Jacobsen, in Lyngby), independently from the necessary contacts with the Executive Committee.
(c) Dissolution of groups There are two possible cases of dissolution of a EWG: (1) the chairman of the group feels that, for some reasons, the group can not go on with its activities and he informs EURO about these reasons, (2) the Executive Committee ascertains that the group has in fact stopped its activities or has given them a direction that is inconsistent with the goals of EURO. In both cases, the Executive Committee can either try to restore a better situation or delete the group from the list of EWGs.
(d) Financial arrangements Experience shows that most of the groups take advantage of the EURO-K conferences to organize their own meetings, either as side meetings or within the framework of the Conference itself. This is indeed a good habit, but does not preclude organizing meetings between the Conferences. Although EWGs may in some cases request a small financial contribution from the members who participate in one of their meetings, they are usually not supposed to handle significant amount of money. In case they wish to organize some specific colloquium, seminar, etc., which involves more finance, they can apply to the treasurer of E U R O with a budget; in each case of that type, efforts will be made on both sides to reach an agreement about the imputation of possible profits or losses.
38
Quantifying the benefits of information systems J.P.C. K L E I J N E N Department of Business and Economics, Tilburg University (Katholieke Hogeschool Tilburg), 5000 LE Tilburg, Netherlands Received June 1981 Revised July 1982
This paper discusses the quantification of the financial benefits of computerized information systems. It is relatively easy to analyse the clerical applications of computers. In management information systems,however,revenues arise only if computers yield better data and if these data are used to improve decision-making.A new frameworkis presented plus a few theories and techniques. The framework comprises the sequence transaction-data creation-decision-reaction. Relevant theories are Bayesian Information Economics, Control Theory, and System Dynamics. Relevant techniques are simulation and management gaming.
background; this survey will show the proper role of these theories and techniques. However, obtaining insight into the problem of benefit quantification is only a first step. It remains to apply relevant theories and techniques to practice. Unfortunately, at this time we can not report any applications with a comprehensive scientific analysis of automation benefits. In practice the benefits problem is 'solved' by treating the benefits as 'intangibles'. We hope that our paper will help to progress towards the quantification of benefits. Note that this article is based on a recent book we published [17]. That book contains about 800 references; the present article contains a selection of the most relevant publications and adds some very recent publications.
2. Economic evaluation of clerical applications I. Introduction Because automation projects (more precisely 'computerization' projects) are often expensive, a cost-benefit analysis is required so that costs can be justified. This paper ignores the cost aspect and concentrates on the benefit aspect within such an analysis. The benefit aspect is further limited to the financml benefits of computerization. Additionally, the focus is on business organizations rather than on non-profit organizations. Finally, after a brief discussion of clerical applications of computers, attention is concentrated on the use of computers in the management of the company, i.e., on the Management Information System (MIS). Given the above restrictions, it is still extremely difficult to quantify the benefits of automation. Our purpose is to provide some insight into this problem. For that purpose a number of concepts, theories, and techniques are brought together. The individual theories and techniques may be wellknown to the reader with an operational research
North-Holland European Journal of Operational Research 15 (1984) 38-45
In principle it is easy to quantify the financial benefits of computers in clerical applications such as salary administration. In the economics jargon one production technique is replaced by a different, more capital-intensive technique. It is relatively easy to determine the increase in capital expenses and the net decrease in personnel expenses when switching to a (more) computerized technique. Next the Net Present Value (NPV) criterion can be used to quantify the benefits of the project: if C, denotes the cash flow change in the t th period, p the cost of capital, and n the life time of the project, then NPV = ~
Ct/(1 +p)t.
(1)
t~l
Applications of formulas like eq. (1) to automation projects do indeed exist; see [2,8,9,11,14,!5]. This relatively large number of references does not mean that in practice such a quantification of benefits in clerical applications is always done. Nevertheless, the NPV formula is a generally accepted measure. Note that the application of eq. (1) outside the specific automation area has been examined in at least 25 surveys; [1]. As with any investment, an additional problem
0377-2217/84/$3.00 © 1984, Elsevier Science Publishers B.V. (North-Holland)
J.P.C. Kleijnen / Quantifying the benefits of information systems
is that of the uncertainty of the cash flows in eq. (1). For instance, how high will the future expenses be for system development and system maintenance? The consequences of these uncertainties can be quantified through risk analysis, a different term for Monte Carlo simulation in an investment analysis context. In risk analysis the uncertainty of the cash flows is quantified by specifying probability functions for the components, say X,, of the cash flows. Hence, a probability function f ( - ) is derived for the X , in
Ct -~- g( Xl, . . . . . X, . . . . . ).
(2)
The probability function can represent either objective or subjective probabilities. Objective probabilities can be specified if actual data on similar projects have been collected in the past. Application of, e.g., regression analysis to this data base generates objective probabilities; for case studies see [6,23]. If actual data are missing, then the analyst has to resort to expert opinions, resulting in subjective probabilities. Techniques for solliciting and validating expert opinions are wellknown from Bayesian analysis, utility analysis, etc.; many references can be found in [17]. Once the probability function f ( . ) has been determined, Monte Carlo simulation yields the probability distribution of the resulting NPV. It is prudent to check how changes in the model g(.) and the assumed distribution f ( . ) affect the NPV distribution. Efficient and systematic statistical techniques for such a sensitivity analysis are explained in [17, pp. 75-79]. Note that computerization can also lead to increased gross revenues, instead of decreased expenses. For example, computerization may mean faster invoicing leading to interest gains. These financial benefits can still be captured through the cash flows in eq. (1). Automation may also be justified, not on financial grounds, but on technical grounds. For instance, the supply of the necessary personnel may be so inadequate that computerization remains the only realistic technical alternative (banks could not have handled the increased business volume without computers). Also computers may result in a better product or better service (e.g, computerized cash dispensers) leading to more sales.
39
3. Clerical systems versus management information systems In clerical applications computerization means that a different production technique is used to produce the same product, i.e., the more capitalintensive office yields the same salary slips, invoices, etc. More generally, the product of a clerical application is a document, a material carrier of data. These documents may be required by employers, vendors, government agencies, etc. If data is used to support decision-making (i.e. to 'manage') then we speak of (management) information: also see [22]. For example, the data contained in the invoices may be used in inventory management; in that case the computer plays a role not only in the clerical processing of the various documents accompanying the flow of materials (invoices, workorders), but then the computer also provides information for the decision whether to order or not, and how much to order. Computerization results in information that is more accurate, timely, and detailed. Intuitively we would say that hence this information is of better quality; in the following sections we shall return to this issue. Financial revenues arise if information results in better decisions, e.g. ordering in time so that service improves, or combining orders for different articles in such a way that discounts on orders and transportation can be obtained. Quantifying the revenues of computerization is much more difficult for an MIS than for a clerical system. As a company gets more familiar with computers it will become more aware of the great potential of computers, not so much in saving labour expenses in the clerical processes as in the creation of higher revenues through better information: 'doing the thing right' (efficiency) versus 'doing the right thing' (effectiveness). A recent Booz, Allen and Hamilton study amongst fifteen major U.S. companies revealed that forty billion dollars were spent on clerical applications against only seventeen billion on management support [7].
4. Information and decision making In the preceding section (management) information was defined as data utilized in decisionmaking. Decision-making occurs at low and high levels in the company. Different sorts of decisions
40
J.P.C. Kleijnen / Quantifyin8 the benefits of information systems
require different sorts of information. It is useful to distinguish two classical viewpoints: - The level in the management hierarchy: operational, tactical and strategic decisions, e.g., inventory control, price setting, investment policy, respectively. - Programmable versus non-programmable decisions: structured versus unstructured problems. Structured problems are essentially solved through fixed rules. In unstructured problems, however, the manager does not know how exactly to formulate his problem because he is confronted with multiple criteria, a number of fuzzy side-conditions, and qualitative variables. Computers can play a more important role on the operational level than on the tactical and strategic levels. Several reasons can be given: - Operational information must be recent because operational decisions concern the short term, e.g., weekly orders require up-to-date information. Modern point-of-sale terminals provide continuous monitoring of inventories. (Whether such a continuous inventory registration is desirable, depends on the costs besides the benefits.) Large quantities of detailed information are collected routinely by several departments, and in principle this detailed information is important as a basis for decision-making (bottom-up approach). For instance, in inventory control the manager is interested in the sales figures per article, per store, per day. In production management he is interested in the bill of materials, working inventories, availability of personnel. A modern Data Base Management System (DBMS) makes it possible to couple data that used to be stored in separate files (personnel file, inventory file). These coupled data can be further processed to compute averages, trends, and the like. Next the presentation of this information can be adjusted to the user: graphs, tables, possibly in colour. - The decision can be based on simple rules of thumb or on computerized, complicated models. As an example consider an assortment of ten thousand articles. For stable articles the computer can calculate the economic order quantity as soon as the inventory hits a specific level. For articles for which a sales promotion has been started, the inventory manager personnally makes decisions. In general, if many decisions have to be made, routine affairs can be delegated to the computer. Summarizing, at the operational level com-
puters can provide fast, accurate, and detailed information. Moreover the information can be processed via simple or complicated rules. In tactical and strategic decisions computers play a smaller role, for the following reasons: - These decisions concern the medium and long term so that information does not need to be up-to-date. Nevertheless the computer's speed can be important in so far as information collected somewhere in the company, can be rapidly retrieved, coupled, and presented. So we should distinguish the time period between the occurrence of an event and its data creation, and the period between the request for data and its presentation. Human intuition plays a bigger role on this level of decision-making. Nevertheless, in larger companies there is an increased usage of strategic corporate models. Special software enables staff people to build simple models for their problems at the terminal; [21]. In strategic decision-making the manager often needs external data, i.e., data not generated within the company. External data concern competitors, the market, etc. (Data on the decreased market share could have warned U.S. automobil manufacturers!). Recently public databanks, accessible through international telephone lines, provide data on the stock-market, exchange rates, etc. Note that computerized models--in practice often very simple models--serve as an aid in decision-making, i.e., models do not replace the manager but they free him of routine work (dataretrieval, tedious calculations) so that he has more time to think about alternatives and to evaluate alternatives, i.e., he has more time for his actual job.
5. The
product
'information'
in economic
theory
Whereas the preceding sections emphasized the management science view of information, the present section considers information from the viewpoint of economic theory. In general the value of, say, additional machinery is determined by the revenues realized when the resulting additional output is sold on the market place. However, the relationships among sales revenues, production costs, and information is more subtle. The utility of information is indirect, since information has value only in so far as 'better' decisions result, i.e.,
J.P. c Kleijnen / Quantifying the benefits of information systems
decisions increasing revenues or decreasing total costs. (Psychologically, information can also have value if the manager does not necessarily make a better decision but if he has more confidence in the correctness of his decision.) Moreover, a closer look reveals that the product 'information' has the following peculiarities: - The manager can not buy, say, ten units of information. What he can do is buy a certain quantity of data, e.g., he can send a telegram with twenty words. He does not known how much information is contained in that telegram; that depends on what the addressee knew already; also see the concept of 'surprise' in Section 9 on "Information Economics". Note that the ShannonWeaver 'information theory' tries to quantify the amount of information. However, this theory neglects what can be done with information, i.e., this theory ignores the resulting actions and their consequences; see [17, p. 147]. - When the manager reads a document, he does not destroy the information within that document. Consequently other people can also read that document, and retrieve their information from it. Information is comparable to 'public' goods like natural scenery and national defence. - Information can have 'external' effects, i.e. uni n t e n d e d favourable or unfavourable consequences. For instance, ecological damage is the unfavourable consequence of an electricity plant. In information systems the unintended unfavourable consequences consist of the risks of industrial espionnage and privacy infringement. In classical economics the production factor information is neglected (because this economic theory assumes perfect knowledge). Since the sixties a few economists have addressed the information issue. Unfortunately, until now their results seem not directly applicable to MIS; also see [17, pp. 62-64].
6. The quality attributes of information
Section 3 contained some quality characteristics of information, namely accuracy, timeliness, and level of detail. For practical purposes it is useful to have a checklist enumerating the various possible effects of computerization. Common practice may not distinguish between imperfections resulting from errors or from delays. Because there is no
41
standard terminology, terms like accuracy and precision are often used interchangeably. In [17] we h~ve discussed in detail the following checklist: - T i m e l i n e s s (recency, delay) with components like retrieval delay (response time) and update interval (say, weekly versus monthly batch updating). - A c c u r a c y (errors) affected by several sources: sampling, measurement, transcription (from one data carrier to another carrier), transformation (truncation, programming bugs) and logical errors. A g g r e g a t i o n (level of detail). - R e p o r t m o d e : exception reporting, query facilities with a DBMS, report format (requiring good human engineering resulting in, say, colour graphics and a certain precision). R e t e n t i o n time (how long to keep data available on on-line or off-line devices). - S e c u r i t y a n d p r i v a c y (including auditing problems). -
-
-
R e l i a b i l i t y a n d recovery.
S c o p e of data base (including the centralization/decentralization issue). - U s e r - m a c h i n e modes (batch/interactive modes). - F l e x i b i l i t y (adaping the MIS to new requirements). - M u l t i p l i c i t y o f users (different types of users of the same MIS). Note that r e l e v a n c e of information is not mentioned in the checklist, because we distinguish between the MIS and the Decision System: the Decision System determines which data are relevant, whereas the MIS then provides the data (possibly with some delay or inaccuracy); see [17, p. 109]. -
7. A new framework
Several MIS frameworks can be found in the literature, especially the literature on systems analysis and design techniques. By definition a framework provides one way of looking at a problem; in practice it does not yield immediate answers but it helps to ask the right questions, i.e., it helps structuring the ' mess'. To the best of our knowledge the following framework is new. (i) There occur a number of e v e n t s or t r a n s a c tions. For instance, a customer decides to make a purchase; a meeting of managers is organised. In practice many types of events occur, both simultaneously and successively, either at fixed or at
42
J.P.c. Kleijnen / Quantifying the benefits of information systems
unexpected points of time. (ii) Some of these events are reflected in one or more documents such as invoices or written orders. With on-line data capture devices (like point-ofsale equipment) individual events are recorded without delay. (iii) Decisions are made, taking into account data on passed events (and on expected events which, however, are extrapolated from the past). A decision can be triggered by: -A specific event, e.g., the arrival of a customer at the counter; - T h e accumulation of a number of transactions, e.g., so many sales have occurred that the inventory level hits the reorder point; - The arrival of a certain point of time, e.g., every Monday morning orders are placed, or every Friday travel-agents meet. The time needed to make a decision, can be reduced drastically through computerization. For instance, information can be retrieved, coupled, and presented much faster, especially when a DBMS is available. Alternatives can be computed much faster with the help of models that do not always need to be really complicated (computing mortgage expenses, discounts). More alternatives can be computed within the same time span. This is the area of the m o d e m decision support system (DSS). In one real-world case the decision delay was reduced from six days to half a day; [20]. (iv) Because of the inertia of the technology and the organization it always takes a certain amount of time before the decision affects the controlled system. Computerization might reduce this reaction delay. For example, in inventory control the leadtime consists of a number of components, some of which cannot be reduced through automation (e.g., the travelling time of a truck) but other components can be reduced (the clerical processing of orders--within one's own company or within the supplying c o m p a n y - - c a n be computerized, and postal delays can be eliminated through teleprocessing). A practical example is reported in [12]: at Eastern Airlines forecasts and the resulting scheduling decisions are communicated much faster after the installation of a realtime computer system.
8. The role of computers in forecasting
Management requires forecasting: 'gouverner c'est pr6voir'. Historical data are necessary to know the current status of the company (e.g., the inventory level) and to forecast new events. The predictability of future events depends on the following factors: - The accuracy and the recency of information concerning the past: the input of the forecasting model can be improved through computerization. - The adequacy of the forecasting model: computers can yield accurate and fast computations of complicated models (such as exponential smoothing, corporate simulation and econometric regression modelling). - The planning horizon: as the manager considers a more distant future, the reliability of his forecast decreases. Especially at the strategic level a longterm vision is required. Computerization does not affect the planning horizon. - The stability of the environment: it is easier to forecast a placid environment. It is difficult for the company to choose and influence its environment, and hence the stability of the environment is (more or less) given.
9. Bayesian information economics
Bayesian Information Economics (abbreviated to IE in this paper) is the only theory explicitly aimed at evaluating the value of information in decision-making. This theory is based on highly simplified mathematical models of managers and systems to be managed. Notwithstanding these simplifications IE requires rather complicated mathematics, which we shall not discuss. IE concentrates on the role of the accuracy of the information. In a later variant known as team theory the focus is on the problems of central decisionmaking with global, aggregated information versus decentralised decision-making with local, detailed information; [19]. Qualitative characteristics such as presentation m o d e - - a behavioural aspect--are difficult to handle within the IE framework, because this theory assumes purely rational decisionmakers. Bayesian theory has been used in practice for one-shot, strategic decisions such as where to search for oil, or when to introduce a new product; see
J.P.C. Kleijnen / Quantifying the benefits of information systems
[16,18]. It is more difficult to apply IE to the evaluation of an MIS supporting a dynamic system; [13]. For such a dynamic situation IE uses the framework of Fig. 1. In this figure the databank is updated through data about the environment and about the actions and results of the company, so that a learning process and adaptive behaviour become possible. Actions (together with the environment) influence not only the immediate result of the company, but also the future environment. Forecasts about the environment are based on old and new information. Unfortunately the mathematical solution of models corresponding with Fig. 1 is extremely difficult. Until now 'applications' have been limited to academic studies, mainly on inventory systems; see the summaries of six studies in [17, pp. 125-127]. The four elements of the framework in Section 7 are also found in the framework of Fig. 1. However, IE emphasizes the following aspects: - The surprise content of information: a company will not pay much money for, say, a marketing report, stating only that demand will decrease if price is increased. The surprise content is reflected by the difference between the prior and the posterior distributions; see [17]. - Sensitivity analysis: information is useless if the decision is fixed, or if the result is insensitive to the exact decision. For example, it is well known that the exact inventory costs per unit need not be known because the optimal order quantity (square root formula) and the resulting inventory costs are not very sensitive to this cost parameter. - Upperlimit for information value: the analyst should know that a realistic MIS can never yield more gross benefits than a perfect MIS (that is, an
; STATE OF ' IHF Wr'RI DI
I
_,
I
L...
J
L "~"°'~ I
•
4-£
.
.
.
.
j C-4--J L ~'~Fig. 1. Dynamic information economics model.
-
-
43
MIS yielding information 100% accurate, detailed, etc.). If the costs of the realistic MIS are higher than the gross revenues of the ideal MIS, then an MIS can never be cost-effective. - The manager should be aware of the inaccuracy of information. The analyst should try to express this inaccuracy explicitly through probability distributions: in the conditional probability P ( y l e ) the symbol e denotes the reported data and y denotes the actual (unknown) value (for perfect information P ( y I e) = 1 if y = e). For instance, in inventory control it is traditional to account explicitly for the unreliability of the forecasted demand; however, the inventory position itself is treated as 100% accurate. - If the quality of the information changes, it may be necessary to adapt the decision. Unreliable information may require a conservative strategy. Although we do not know any application of IE to a practical MIS, we think that the aspects highlighted by the IE framework are of practical relevance indeed.
10. Control theory and system dynamics
There are a number of theories and techniques that concentrate on the management of systems, be these systems technical or socio-technicai; and information is needed for such management. Unlike IE these theories and techniques do not focus on the value of information but nevertheless they are relevant to the topic of this paper. Originally Control Theory (CT) concerned the optimal management of technical systems, an issue in chemical industry, electrical-engineering, etc. This ' theory' consists of a collection of mathematical optimization techniques, on which we shall not dwell. Instead we shall present some interesting concepts highlighted by CT: - The concept of feedback: information on a variable is compared to a norm, and in case of an undesirable deviation corrections are made. A well-known example is provided by the thermostat. A business example is the budget system: actual expenses are compared to the budgetted expenses. - Steering frequency: organisations implicitly assume a traditional frequency of meetings (say, weekly), whereas CT investigates the optimal frequency of decision-making.
44
J.P.C Kleijnen / Quantifying the benefits of m/orrnation systems
- Delays and oscillations: collecting information and making decisions requires time; also see the framework in Section 7. These delays lead to oscill a t i o n s - d a m p e d or exploding--of the system output. Internal business cycles of a company can be explained by these lags: the purchase and production departments react with a delay to disturbances in the sales. Notice that these delays can be reduced through the integration of the information systems of the various departments. Such integration has become technically feasible through the use of a central databank (with a DBMS), accessible via terminals. Control Theory inspired Industrial Dynamics, currently known as System Dynamics (SD). SD models are more realistic (and hence more complicated) but consequently the optimisation of decisions has become impossible and SD must restrict itself to describing the consequences of various decision strategies: ' w h a t - i f approach. CT models are optimised through the techniques of mathematical analysis, whereas SD models are evaluated through the simulation technique. Moreover, SD emphasizes that dynamic systems often react counterintuitively and that intuitive rules of thumb for managing systems can, therefore, cause very undesirable behaviour. CT and SD have inspired well-known MIS texts like Blumenthal [3]. Practical studies at the Dutch Philips organisation have been strongly influenced by the C T / S D philosophy and models; [24].
vestigating the effects of the presentation mode (visual display or paper output) on - the financial benefits; - the time required by players to reach decisions; - the confidence they have in their decisions. Management games can also be used as a demonstration tool to show users (in a simulated environment) how their effectiveness is influenced by, say, the accuracy of information. Although simulation and gaming have been quite extensively used in academic studies on MIS evaluation, we do not know any real-world applications. Our explanation is that simulation and gaming are techniques for solving models; however, the practitioner does not known how to build a (simple) model for MIS evaluation. The lack of practical MIS models may be explained by the fact that the science of MIS is still in its infancy (which is not surprising if we realize that the use of computers in the management of organizations has just started). Note that there are a number of techniques which are indeed used in practice. Examples are prototypes (an MIS on a small scale), sample surveys among users (which data have already been used by which departments; which information is considered worthwhile by users), case studies, and so on. The main disadvantage of these 'practical' techniques is that they provide no general conclusions because of the lack of experimental control.
11. Simulation, gaming, and other techniques
12. Conclusions
If exact figures on the benefits of an MIS are desired then a formal, mathematical model is recommended over an informal, implicit model. Such a formal modal can be structured through the concepts, frameworks and theories of IE, CT and SD and the framework of Section 7. The solution of the model can be obtained through the wellknown technique of simulation. Indeed a number of academic studies have used simulation to quantify the influence of accuracy and recency of information; [4,5,17]. These simulation results are valid--strictly speaking--for the specific simulation model only. Psychological or behavioural aspects can be investigated through business games. A number of academic studies--see [lO]--have been made in-
The financial benefits of computers in clerical applications can be measured quite simply (although this does not mean that in practice such an economic evaluation is always made). Computerization of an MIS leads to indirect revenues in so far as better information attributes like accuracy and speed lead to better decisions. One possible framework for MIS evaluation is formed by the sequence: transaction-data creation-decision-reaction. Other frameworks and concepts are provided by Information Economics, Control Theory, and System Dynamics. The application of IE, CT or SD models to the evaluation of an MIS, however, remains very difficult. Nevertheless these models may assist the informal, intuitive reasoning about MIS. If exact figures on the benefits of a
J.P.C Kleijnen / Quantifying the benefits of information systems
particular MIS are required, then a mathematical model is necessary and the techniques of simulation and gaming becoming useful. The higher the decision-making level, the more difficult it will be to quantify the value of information because at higher management levels the relationships among information, decision, and result are fuzzy. The MIS area certainly provides a challenge to the management scientist. Note that the relationship between the practice and the theory of MIS can be compared to the relationship between the engineering practice and theoretical physics: theory forms the basis on which genuine breakthroughs in practice are based; nevertheless theory can not directly answer the many detailed questions raised in practice.
References [1] R. Aggarwal, Corporate use of sophisticated capital budgeting techniques: a strategic perspective and a critique of survey results, Interfaces 10 (2) (1980) 31-34. [2] C.W. Blaxter and W.E.M. Morris (Editors), Economic Evaluation of Computer Based Systems, Book 2 (The National Computing Centre Limited (NCC), Manchester, 1971). [3] S.C. Blumenthal, Management Information Systems; A Framework for Planning and Development (Prentice-Hall, Englewood Cliffs, N J, 1969). [4] C.P. Bonini, Simulation of Information and Decision Systems in the Firm (Prentice-Hall, Englewood Cliffs, N J, 1963). (Reprinted by Markham Publishers, Chicago, 1967). [5] D.F. Boyd and H.S. Kxasnow, Economic evaluation of management information systems, IBM Systems J. 2 (1963) 2-23. [6] E. Chrysler, Some basic determinants of computer programrning productivity, Comrr~ ACM 21 (6) (1978) 472-483. [7] Computable 14 (7) (1981) 2.
45
[8] J.E. Cooke and D.H. Drury, Managing and Accounting for Information Systems (The Society of Industrial Accountants of Canada, Hamilton, Ontario, 1972). [9] N.J. Dean, The computer comes of age, Harvard Business Rev. 46 (1) (1968) 83-91. [10] G.W. Dickson, J.A. Senn and N.L. Chervany, Research in management information systems: The Minnesota experiments, Management ScL 23 (9) (1977) 913-923. [11] P.J. Dixon, Planning MIS investment and expenses levels, Performance Evaluatwn Rev. 9 (!) (1980) 35-37. [12] R.L. Duley, Communicating and executing the short range forecast, Production and Inventory Management 22 (3) (1981) 1-20. [13] G.A. Feltham, The value of information, Accounting Rev. 43 (4) (1968) 684-696. [14] P.G. Hall and T.J. Lincoln, A cost effectiveness study of production and engineering systems within a tightly integrated .manufacturing environment, Management Datamatics 5 (6) (1976) 261-273. [15] J.G. Head, Estimating the costs and benefits and monitoring the effectiveness of large government installations, in: A.B. Frielink, Ed., Economics of Informatics (North-Holland, Amsterdam, 1975). [16] W.C. House (Editor), Interactive Decision Oriented Data Base Systems (Petrocelli/Charter, New York, 1977). [17] J.P.C. Kleijnen, Computers and Profits: Quantifying Financial Benefits of Information (Addison-Wesley, Reading, MA, 1980). [18] A.S. Manne, R.G. Richels and J.'P. Weyant, Energy policy modeling: a survey, Operations Res. 27 (1) (1979) 1-36. [I 9] J. Marschak and R. Radner, Economic Theory of Teams (Yale University Press, New Haven, 1972). [20] M.S.S. Morton, Management Decision Systems (Harvard University Press, Boston, 1971). [21] T.H. Naylor, Corporate Planning Models (Addison-Wesley, Reading, MA, 1979). [22] K.J. Radford, Information Systems for Strategic Decisions (Reston, Reston, VA, 1978). [23] V. Schneider, Prediction of software effort and project duration--four new formulas, A CM Sigplan Notices 13 (6) (1978) 49-59. [24] J.E. Van Aken, On the Control of Complex Industrial Organizations (Martinus Nijhoff, Leiden, The Netherlands, 1978).
Profiles of the European WorkingGroups
tists led to the next major event which was set up by Professor Ian Masser, then with the Instituut voor Planologie at the Rijksuniversiteit Utrecht in the Netherlands. Professor Masser initiated and organized what in the light of the two following meetings turned out to be the 1st International Workshop on Strategic Planning. It was conceived and implemented as a joint venture of the British section of the Regional Science Association workshop on regional science methods in strategic planning, of the Instituut voor Planologie, Rijksuniversiteit Utrecht, and the EURO Working Group "OR in Regional and Urban Planning", taking place at Utrecth on April 24-26, 1978. The workshop found very favorable resonance among the participants and Dr. Henk Voogd from the Department of Urban and Regional Planning of the University of Technology, Delft, Netherlands, spontaneously offered to set up a 2nd International Workshop on Strategic Planning at Delft in 1980. In the meantime proceedings of this workshop are available, entitled "Strategic Planning in a Dynamic Society", Henk Voogd (Ed.), Delftsche Uitgevermij, Delft, 1981. The Delft meeting, too, was a joint venture of the above mentioned British and EURO working groups and, last but not least, the Delft University Department of Urban and Regional Planning. One year later, on May 8-9, 1981, the 3rd International Workshop on Strategic Planning took
place at the Institut for Raumplanung of the University of Dortmund, Germany, most efficiently and generously organized by Dr. Michael Wegener. During the 3rd International Workshop the intent was expressed to set up a 4th Workshop in 1983 at some locality, not yet fixed, in Great Britain. In the meantime, this workshop has been postponed until 1984. There is a comment in order as to the professional affiliation of the attendants of the workshops. The vast majority of the participants of the three International Workshops would, on request, presumably declare themselves regional scientists rather than working in OR. At the first meeting at Erlangen, in spring 1976, there were about 50% of the attendants who were genuine OR people. As emphasis of the working group activities noticeably shifted towards strategic planning they by and large withdrew. In the meantime, preponderance of regional scientists in the workshops has grown so marked that it would appear natural to conduct future workshops no longer under the cosponsorship of the EURO Working Group. With these premises in mind and with the assent of the Vice President of The Association of European Operational Research Societies within IFORS, the chairman of the Working Group has decided to cease its activities (notwithstanding, naturally, the possibility of its reactivation by another convenor).
5. Charter of the European Working Groups (adopted by Council on July 23, 1980) Although the "Appendix to the Agreement" creating EURO on 20th January 1975 was not very explicit about how the European Working Groups (EWGs) should be organized, the existence of such groups was always considered as one of the main objectives of EURO. After five years of existence of EURO, it appears that a reasonable number of EWGs have spontaneously reached a certain degree of activity and development, and that it is time to establish more precise (however flexible) rules for them.
(a) Creation of groups The creation of new EWGs should be left to individual initiative, as was always done. Nonetheless, it might be useful to distinguish two cases: (1) creations while a EURO-K Conference is held, (2) creations between EURO-K and EURO-(K +1). In case (1), a convener can for instance begin by sticking up a bill proposing to interested persons to meet at specified time and place in a quite
Profiles of the European WorkingGroups
informal way. If he gets enough people together and if they agree upon a tentative schedule of activity, they produce a short report which is forwarded to the EURO Executive Committee; the latter can then either deliver the EURO label at once or postpone it after a further meeting of the group. In case (2), the convener either applies first to the Executive Committee, who will decide whether and under what conditions they will help him achieving his ideas, or he tries first on his own responsibility to get together enough people interested to work on his subject, and then applies to the Executive Committee. The foundation of the group should be announced in the Bulletin. In both cases (1) and (2), the EURO label means that E U R O considers itself informed well enough about the group and includes it into the list of its official activities.
(b) Activities EURO does not try to impose unified regulations for the way each working group displays its activities. Nonetheless a few general principles are worthwhile being recalled: (1) The EWGs should as far as possible bring together people with different professional backgrounds (industry, university, etc.). (2) They should be actually international; this requirement is met satisfactorily if at least 4, but possible 5 or 6 nations are represented at each meeting, and if the meetings do not take place too often in'the same country. (3) Any group should be open to any individual member of any EURO member society. Persons who are not members of a national OR Society are not excluded from the meetings; however, a friendly pressure should be exerted upon them to become members. (4) If one of the EWGs plans a joint meeting with some other (e.g. professional) body, it should provide EURO with general information about the latter. (5) As a general rule, EURO-K conferences will provide time and space for meetings of the EWGs which will be planned in connection with them. At
37
least for these particular meetings, no entrance-fee should be requested from participants. (6) Routine internal informatio~ of the groups, such as announcements of the next meeting and short reports about the last, should be conveyed to EJOR for publication in the " E U R O Bulletin". It will be up to each group chairman to maintain the connection with the Editor of the Bulletin (presently Dr. Soren Kruse Jacobsen, in Lyngby), independently from the necessary contacts with the Executive Committee.
(c) Dissolution of groups There are two possible cases of dissolution of a EWG: (1) the chairman of the group feels that, for some reasons, the group can not go on with its activities and he informs EURO about these reasons, (2) the Executive Committee ascertains that the group has in fact stopped its activities or has given them a direction that is inconsistent with the goals of EURO. In both cases, the Executive Committee can either try to restore a better situation or delete the group from the list of EWGs.
(d) Financial arrangements Experience shows that most of the groups take advantage of the EURO-K conferences to organize their own meetings, either as side meetings or within the framework of the Conference itself. This is indeed a good habit, but does not preclude organizing meetings between the Conferences. Although EWGs may in some cases request a small financial contribution from the members who participate in one of their meetings, they are usually not supposed to handle significant amount of money. In case they wish to organize some specific colloquium, seminar, etc., which involves more finance, they can apply to the treasurer of E U R O with a budget; in each case of that type, efforts will be made on both sides to reach an agreement about the imputation of possible profits or losses.
38
Quantifying the benefits of information systems J.P.C. K L E I J N E N Department of Business and Economics, Tilburg University (Katholieke Hogeschool Tilburg), 5000 LE Tilburg, Netherlands Received June 1981 Revised July 1982
This paper discusses the quantification of the financial benefits of computerized information systems. It is relatively easy to analyse the clerical applications of computers. In management information systems,however,revenues arise only if computers yield better data and if these data are used to improve decision-making.A new frameworkis presented plus a few theories and techniques. The framework comprises the sequence transaction-data creation-decision-reaction. Relevant theories are Bayesian Information Economics, Control Theory, and System Dynamics. Relevant techniques are simulation and management gaming.
background; this survey will show the proper role of these theories and techniques. However, obtaining insight into the problem of benefit quantification is only a first step. It remains to apply relevant theories and techniques to practice. Unfortunately, at this time we can not report any applications with a comprehensive scientific analysis of automation benefits. In practice the benefits problem is 'solved' by treating the benefits as 'intangibles'. We hope that our paper will help to progress towards the quantification of benefits. Note that this article is based on a recent book we published [17]. That book contains about 800 references; the present article contains a selection of the most relevant publications and adds some very recent publications.
2. Economic evaluation of clerical applications I. Introduction Because automation projects (more precisely 'computerization' projects) are often expensive, a cost-benefit analysis is required so that costs can be justified. This paper ignores the cost aspect and concentrates on the benefit aspect within such an analysis. The benefit aspect is further limited to the financml benefits of computerization. Additionally, the focus is on business organizations rather than on non-profit organizations. Finally, after a brief discussion of clerical applications of computers, attention is concentrated on the use of computers in the management of the company, i.e., on the Management Information System (MIS). Given the above restrictions, it is still extremely difficult to quantify the benefits of automation. Our purpose is to provide some insight into this problem. For that purpose a number of concepts, theories, and techniques are brought together. The individual theories and techniques may be wellknown to the reader with an operational research
North-Holland European Journal of Operational Research 15 (1984) 38-45
In principle it is easy to quantify the financial benefits of computers in clerical applications such as salary administration. In the economics jargon one production technique is replaced by a different, more capital-intensive technique. It is relatively easy to determine the increase in capital expenses and the net decrease in personnel expenses when switching to a (more) computerized technique. Next the Net Present Value (NPV) criterion can be used to quantify the benefits of the project: if C, denotes the cash flow change in the t th period, p the cost of capital, and n the life time of the project, then NPV = ~
Ct/(1 +p)t.
(1)
t~l
Applications of formulas like eq. (1) to automation projects do indeed exist; see [2,8,9,11,14,!5]. This relatively large number of references does not mean that in practice such a quantification of benefits in clerical applications is always done. Nevertheless, the NPV formula is a generally accepted measure. Note that the application of eq. (1) outside the specific automation area has been examined in at least 25 surveys; [1]. As with any investment, an additional problem
0377-2217/84/$3.00 © 1984, Elsevier Science Publishers B.V. (North-Holland)
J.P.C. Kleijnen / Quantifying the benefits of information systems
is that of the uncertainty of the cash flows in eq. (1). For instance, how high will the future expenses be for system development and system maintenance? The consequences of these uncertainties can be quantified through risk analysis, a different term for Monte Carlo simulation in an investment analysis context. In risk analysis the uncertainty of the cash flows is quantified by specifying probability functions for the components, say X,, of the cash flows. Hence, a probability function f ( - ) is derived for the X , in
Ct -~- g( Xl, . . . . . X, . . . . . ).
(2)
The probability function can represent either objective or subjective probabilities. Objective probabilities can be specified if actual data on similar projects have been collected in the past. Application of, e.g., regression analysis to this data base generates objective probabilities; for case studies see [6,23]. If actual data are missing, then the analyst has to resort to expert opinions, resulting in subjective probabilities. Techniques for solliciting and validating expert opinions are wellknown from Bayesian analysis, utility analysis, etc.; many references can be found in [17]. Once the probability function f ( . ) has been determined, Monte Carlo simulation yields the probability distribution of the resulting NPV. It is prudent to check how changes in the model g(.) and the assumed distribution f ( . ) affect the NPV distribution. Efficient and systematic statistical techniques for such a sensitivity analysis are explained in [17, pp. 75-79]. Note that computerization can also lead to increased gross revenues, instead of decreased expenses. For example, computerization may mean faster invoicing leading to interest gains. These financial benefits can still be captured through the cash flows in eq. (1). Automation may also be justified, not on financial grounds, but on technical grounds. For instance, the supply of the necessary personnel may be so inadequate that computerization remains the only realistic technical alternative (banks could not have handled the increased business volume without computers). Also computers may result in a better product or better service (e.g, computerized cash dispensers) leading to more sales.
39
3. Clerical systems versus management information systems In clerical applications computerization means that a different production technique is used to produce the same product, i.e., the more capitalintensive office yields the same salary slips, invoices, etc. More generally, the product of a clerical application is a document, a material carrier of data. These documents may be required by employers, vendors, government agencies, etc. If data is used to support decision-making (i.e. to 'manage') then we speak of (management) information: also see [22]. For example, the data contained in the invoices may be used in inventory management; in that case the computer plays a role not only in the clerical processing of the various documents accompanying the flow of materials (invoices, workorders), but then the computer also provides information for the decision whether to order or not, and how much to order. Computerization results in information that is more accurate, timely, and detailed. Intuitively we would say that hence this information is of better quality; in the following sections we shall return to this issue. Financial revenues arise if information results in better decisions, e.g. ordering in time so that service improves, or combining orders for different articles in such a way that discounts on orders and transportation can be obtained. Quantifying the revenues of computerization is much more difficult for an MIS than for a clerical system. As a company gets more familiar with computers it will become more aware of the great potential of computers, not so much in saving labour expenses in the clerical processes as in the creation of higher revenues through better information: 'doing the thing right' (efficiency) versus 'doing the right thing' (effectiveness). A recent Booz, Allen and Hamilton study amongst fifteen major U.S. companies revealed that forty billion dollars were spent on clerical applications against only seventeen billion on management support [7].
4. Information and decision making In the preceding section (management) information was defined as data utilized in decisionmaking. Decision-making occurs at low and high levels in the company. Different sorts of decisions
40
J.P.C. Kleijnen / Quantifyin8 the benefits of information systems
require different sorts of information. It is useful to distinguish two classical viewpoints: - The level in the management hierarchy: operational, tactical and strategic decisions, e.g., inventory control, price setting, investment policy, respectively. - Programmable versus non-programmable decisions: structured versus unstructured problems. Structured problems are essentially solved through fixed rules. In unstructured problems, however, the manager does not know how exactly to formulate his problem because he is confronted with multiple criteria, a number of fuzzy side-conditions, and qualitative variables. Computers can play a more important role on the operational level than on the tactical and strategic levels. Several reasons can be given: - Operational information must be recent because operational decisions concern the short term, e.g., weekly orders require up-to-date information. Modern point-of-sale terminals provide continuous monitoring of inventories. (Whether such a continuous inventory registration is desirable, depends on the costs besides the benefits.) Large quantities of detailed information are collected routinely by several departments, and in principle this detailed information is important as a basis for decision-making (bottom-up approach). For instance, in inventory control the manager is interested in the sales figures per article, per store, per day. In production management he is interested in the bill of materials, working inventories, availability of personnel. A modern Data Base Management System (DBMS) makes it possible to couple data that used to be stored in separate files (personnel file, inventory file). These coupled data can be further processed to compute averages, trends, and the like. Next the presentation of this information can be adjusted to the user: graphs, tables, possibly in colour. - The decision can be based on simple rules of thumb or on computerized, complicated models. As an example consider an assortment of ten thousand articles. For stable articles the computer can calculate the economic order quantity as soon as the inventory hits a specific level. For articles for which a sales promotion has been started, the inventory manager personnally makes decisions. In general, if many decisions have to be made, routine affairs can be delegated to the computer. Summarizing, at the operational level com-
puters can provide fast, accurate, and detailed information. Moreover the information can be processed via simple or complicated rules. In tactical and strategic decisions computers play a smaller role, for the following reasons: - These decisions concern the medium and long term so that information does not need to be up-to-date. Nevertheless the computer's speed can be important in so far as information collected somewhere in the company, can be rapidly retrieved, coupled, and presented. So we should distinguish the time period between the occurrence of an event and its data creation, and the period between the request for data and its presentation. Human intuition plays a bigger role on this level of decision-making. Nevertheless, in larger companies there is an increased usage of strategic corporate models. Special software enables staff people to build simple models for their problems at the terminal; [21]. In strategic decision-making the manager often needs external data, i.e., data not generated within the company. External data concern competitors, the market, etc. (Data on the decreased market share could have warned U.S. automobil manufacturers!). Recently public databanks, accessible through international telephone lines, provide data on the stock-market, exchange rates, etc. Note that computerized models--in practice often very simple models--serve as an aid in decision-making, i.e., models do not replace the manager but they free him of routine work (dataretrieval, tedious calculations) so that he has more time to think about alternatives and to evaluate alternatives, i.e., he has more time for his actual job.
5. The
product
'information'
in economic
theory
Whereas the preceding sections emphasized the management science view of information, the present section considers information from the viewpoint of economic theory. In general the value of, say, additional machinery is determined by the revenues realized when the resulting additional output is sold on the market place. However, the relationships among sales revenues, production costs, and information is more subtle. The utility of information is indirect, since information has value only in so far as 'better' decisions result, i.e.,
J.P. c Kleijnen / Quantifying the benefits of information systems
decisions increasing revenues or decreasing total costs. (Psychologically, information can also have value if the manager does not necessarily make a better decision but if he has more confidence in the correctness of his decision.) Moreover, a closer look reveals that the product 'information' has the following peculiarities: - The manager can not buy, say, ten units of information. What he can do is buy a certain quantity of data, e.g., he can send a telegram with twenty words. He does not known how much information is contained in that telegram; that depends on what the addressee knew already; also see the concept of 'surprise' in Section 9 on "Information Economics". Note that the ShannonWeaver 'information theory' tries to quantify the amount of information. However, this theory neglects what can be done with information, i.e., this theory ignores the resulting actions and their consequences; see [17, p. 147]. - When the manager reads a document, he does not destroy the information within that document. Consequently other people can also read that document, and retrieve their information from it. Information is comparable to 'public' goods like natural scenery and national defence. - Information can have 'external' effects, i.e. uni n t e n d e d favourable or unfavourable consequences. For instance, ecological damage is the unfavourable consequence of an electricity plant. In information systems the unintended unfavourable consequences consist of the risks of industrial espionnage and privacy infringement. In classical economics the production factor information is neglected (because this economic theory assumes perfect knowledge). Since the sixties a few economists have addressed the information issue. Unfortunately, until now their results seem not directly applicable to MIS; also see [17, pp. 62-64].
6. The quality attributes of information
Section 3 contained some quality characteristics of information, namely accuracy, timeliness, and level of detail. For practical purposes it is useful to have a checklist enumerating the various possible effects of computerization. Common practice may not distinguish between imperfections resulting from errors or from delays. Because there is no
41
standard terminology, terms like accuracy and precision are often used interchangeably. In [17] we h~ve discussed in detail the following checklist: - T i m e l i n e s s (recency, delay) with components like retrieval delay (response time) and update interval (say, weekly versus monthly batch updating). - A c c u r a c y (errors) affected by several sources: sampling, measurement, transcription (from one data carrier to another carrier), transformation (truncation, programming bugs) and logical errors. A g g r e g a t i o n (level of detail). - R e p o r t m o d e : exception reporting, query facilities with a DBMS, report format (requiring good human engineering resulting in, say, colour graphics and a certain precision). R e t e n t i o n time (how long to keep data available on on-line or off-line devices). - S e c u r i t y a n d p r i v a c y (including auditing problems). -
-
-
R e l i a b i l i t y a n d recovery.
S c o p e of data base (including the centralization/decentralization issue). - U s e r - m a c h i n e modes (batch/interactive modes). - F l e x i b i l i t y (adaping the MIS to new requirements). - M u l t i p l i c i t y o f users (different types of users of the same MIS). Note that r e l e v a n c e of information is not mentioned in the checklist, because we distinguish between the MIS and the Decision System: the Decision System determines which data are relevant, whereas the MIS then provides the data (possibly with some delay or inaccuracy); see [17, p. 109]. -
7. A new framework
Several MIS frameworks can be found in the literature, especially the literature on systems analysis and design techniques. By definition a framework provides one way of looking at a problem; in practice it does not yield immediate answers but it helps to ask the right questions, i.e., it helps structuring the ' mess'. To the best of our knowledge the following framework is new. (i) There occur a number of e v e n t s or t r a n s a c tions. For instance, a customer decides to make a purchase; a meeting of managers is organised. In practice many types of events occur, both simultaneously and successively, either at fixed or at
42
J.P.c. Kleijnen / Quantifying the benefits of information systems
unexpected points of time. (ii) Some of these events are reflected in one or more documents such as invoices or written orders. With on-line data capture devices (like point-ofsale equipment) individual events are recorded without delay. (iii) Decisions are made, taking into account data on passed events (and on expected events which, however, are extrapolated from the past). A decision can be triggered by: -A specific event, e.g., the arrival of a customer at the counter; - T h e accumulation of a number of transactions, e.g., so many sales have occurred that the inventory level hits the reorder point; - The arrival of a certain point of time, e.g., every Monday morning orders are placed, or every Friday travel-agents meet. The time needed to make a decision, can be reduced drastically through computerization. For instance, information can be retrieved, coupled, and presented much faster, especially when a DBMS is available. Alternatives can be computed much faster with the help of models that do not always need to be really complicated (computing mortgage expenses, discounts). More alternatives can be computed within the same time span. This is the area of the m o d e m decision support system (DSS). In one real-world case the decision delay was reduced from six days to half a day; [20]. (iv) Because of the inertia of the technology and the organization it always takes a certain amount of time before the decision affects the controlled system. Computerization might reduce this reaction delay. For example, in inventory control the leadtime consists of a number of components, some of which cannot be reduced through automation (e.g., the travelling time of a truck) but other components can be reduced (the clerical processing of orders--within one's own company or within the supplying c o m p a n y - - c a n be computerized, and postal delays can be eliminated through teleprocessing). A practical example is reported in [12]: at Eastern Airlines forecasts and the resulting scheduling decisions are communicated much faster after the installation of a realtime computer system.
8. The role of computers in forecasting
Management requires forecasting: 'gouverner c'est pr6voir'. Historical data are necessary to know the current status of the company (e.g., the inventory level) and to forecast new events. The predictability of future events depends on the following factors: - The accuracy and the recency of information concerning the past: the input of the forecasting model can be improved through computerization. - The adequacy of the forecasting model: computers can yield accurate and fast computations of complicated models (such as exponential smoothing, corporate simulation and econometric regression modelling). - The planning horizon: as the manager considers a more distant future, the reliability of his forecast decreases. Especially at the strategic level a longterm vision is required. Computerization does not affect the planning horizon. - The stability of the environment: it is easier to forecast a placid environment. It is difficult for the company to choose and influence its environment, and hence the stability of the environment is (more or less) given.
9. Bayesian information economics
Bayesian Information Economics (abbreviated to IE in this paper) is the only theory explicitly aimed at evaluating the value of information in decision-making. This theory is based on highly simplified mathematical models of managers and systems to be managed. Notwithstanding these simplifications IE requires rather complicated mathematics, which we shall not discuss. IE concentrates on the role of the accuracy of the information. In a later variant known as team theory the focus is on the problems of central decisionmaking with global, aggregated information versus decentralised decision-making with local, detailed information; [19]. Qualitative characteristics such as presentation m o d e - - a behavioural aspect--are difficult to handle within the IE framework, because this theory assumes purely rational decisionmakers. Bayesian theory has been used in practice for one-shot, strategic decisions such as where to search for oil, or when to introduce a new product; see
J.P.C. Kleijnen / Quantifying the benefits of information systems
[16,18]. It is more difficult to apply IE to the evaluation of an MIS supporting a dynamic system; [13]. For such a dynamic situation IE uses the framework of Fig. 1. In this figure the databank is updated through data about the environment and about the actions and results of the company, so that a learning process and adaptive behaviour become possible. Actions (together with the environment) influence not only the immediate result of the company, but also the future environment. Forecasts about the environment are based on old and new information. Unfortunately the mathematical solution of models corresponding with Fig. 1 is extremely difficult. Until now 'applications' have been limited to academic studies, mainly on inventory systems; see the summaries of six studies in [17, pp. 125-127]. The four elements of the framework in Section 7 are also found in the framework of Fig. 1. However, IE emphasizes the following aspects: - The surprise content of information: a company will not pay much money for, say, a marketing report, stating only that demand will decrease if price is increased. The surprise content is reflected by the difference between the prior and the posterior distributions; see [17]. - Sensitivity analysis: information is useless if the decision is fixed, or if the result is insensitive to the exact decision. For example, it is well known that the exact inventory costs per unit need not be known because the optimal order quantity (square root formula) and the resulting inventory costs are not very sensitive to this cost parameter. - Upperlimit for information value: the analyst should know that a realistic MIS can never yield more gross benefits than a perfect MIS (that is, an
; STATE OF ' IHF Wr'RI DI
I
_,
I
L...
J
L "~"°'~ I
•
4-£
.
.
.
.
j C-4--J L ~'~Fig. 1. Dynamic information economics model.
-
-
43
MIS yielding information 100% accurate, detailed, etc.). If the costs of the realistic MIS are higher than the gross revenues of the ideal MIS, then an MIS can never be cost-effective. - The manager should be aware of the inaccuracy of information. The analyst should try to express this inaccuracy explicitly through probability distributions: in the conditional probability P ( y l e ) the symbol e denotes the reported data and y denotes the actual (unknown) value (for perfect information P ( y I e) = 1 if y = e). For instance, in inventory control it is traditional to account explicitly for the unreliability of the forecasted demand; however, the inventory position itself is treated as 100% accurate. - If the quality of the information changes, it may be necessary to adapt the decision. Unreliable information may require a conservative strategy. Although we do not know any application of IE to a practical MIS, we think that the aspects highlighted by the IE framework are of practical relevance indeed.
10. Control theory and system dynamics
There are a number of theories and techniques that concentrate on the management of systems, be these systems technical or socio-technicai; and information is needed for such management. Unlike IE these theories and techniques do not focus on the value of information but nevertheless they are relevant to the topic of this paper. Originally Control Theory (CT) concerned the optimal management of technical systems, an issue in chemical industry, electrical-engineering, etc. This ' theory' consists of a collection of mathematical optimization techniques, on which we shall not dwell. Instead we shall present some interesting concepts highlighted by CT: - The concept of feedback: information on a variable is compared to a norm, and in case of an undesirable deviation corrections are made. A well-known example is provided by the thermostat. A business example is the budget system: actual expenses are compared to the budgetted expenses. - Steering frequency: organisations implicitly assume a traditional frequency of meetings (say, weekly), whereas CT investigates the optimal frequency of decision-making.
44
J.P.C Kleijnen / Quantifying the benefits of m/orrnation systems
- Delays and oscillations: collecting information and making decisions requires time; also see the framework in Section 7. These delays lead to oscill a t i o n s - d a m p e d or exploding--of the system output. Internal business cycles of a company can be explained by these lags: the purchase and production departments react with a delay to disturbances in the sales. Notice that these delays can be reduced through the integration of the information systems of the various departments. Such integration has become technically feasible through the use of a central databank (with a DBMS), accessible via terminals. Control Theory inspired Industrial Dynamics, currently known as System Dynamics (SD). SD models are more realistic (and hence more complicated) but consequently the optimisation of decisions has become impossible and SD must restrict itself to describing the consequences of various decision strategies: ' w h a t - i f approach. CT models are optimised through the techniques of mathematical analysis, whereas SD models are evaluated through the simulation technique. Moreover, SD emphasizes that dynamic systems often react counterintuitively and that intuitive rules of thumb for managing systems can, therefore, cause very undesirable behaviour. CT and SD have inspired well-known MIS texts like Blumenthal [3]. Practical studies at the Dutch Philips organisation have been strongly influenced by the C T / S D philosophy and models; [24].
vestigating the effects of the presentation mode (visual display or paper output) on - the financial benefits; - the time required by players to reach decisions; - the confidence they have in their decisions. Management games can also be used as a demonstration tool to show users (in a simulated environment) how their effectiveness is influenced by, say, the accuracy of information. Although simulation and gaming have been quite extensively used in academic studies on MIS evaluation, we do not know any real-world applications. Our explanation is that simulation and gaming are techniques for solving models; however, the practitioner does not known how to build a (simple) model for MIS evaluation. The lack of practical MIS models may be explained by the fact that the science of MIS is still in its infancy (which is not surprising if we realize that the use of computers in the management of organizations has just started). Note that there are a number of techniques which are indeed used in practice. Examples are prototypes (an MIS on a small scale), sample surveys among users (which data have already been used by which departments; which information is considered worthwhile by users), case studies, and so on. The main disadvantage of these 'practical' techniques is that they provide no general conclusions because of the lack of experimental control.
11. Simulation, gaming, and other techniques
12. Conclusions
If exact figures on the benefits of an MIS are desired then a formal, mathematical model is recommended over an informal, implicit model. Such a formal modal can be structured through the concepts, frameworks and theories of IE, CT and SD and the framework of Section 7. The solution of the model can be obtained through the wellknown technique of simulation. Indeed a number of academic studies have used simulation to quantify the influence of accuracy and recency of information; [4,5,17]. These simulation results are valid--strictly speaking--for the specific simulation model only. Psychological or behavioural aspects can be investigated through business games. A number of academic studies--see [lO]--have been made in-
The financial benefits of computers in clerical applications can be measured quite simply (although this does not mean that in practice such an economic evaluation is always made). Computerization of an MIS leads to indirect revenues in so far as better information attributes like accuracy and speed lead to better decisions. One possible framework for MIS evaluation is formed by the sequence: transaction-data creation-decision-reaction. Other frameworks and concepts are provided by Information Economics, Control Theory, and System Dynamics. The application of IE, CT or SD models to the evaluation of an MIS, however, remains very difficult. Nevertheless these models may assist the informal, intuitive reasoning about MIS. If exact figures on the benefits of a
J.P.C Kleijnen / Quantifying the benefits of information systems
particular MIS are required, then a mathematical model is necessary and the techniques of simulation and gaming becoming useful. The higher the decision-making level, the more difficult it will be to quantify the value of information because at higher management levels the relationships among information, decision, and result are fuzzy. The MIS area certainly provides a challenge to the management scientist. Note that the relationship between the practice and the theory of MIS can be compared to the relationship between the engineering practice and theoretical physics: theory forms the basis on which genuine breakthroughs in practice are based; nevertheless theory can not directly answer the many detailed questions raised in practice.
References [1] R. Aggarwal, Corporate use of sophisticated capital budgeting techniques: a strategic perspective and a critique of survey results, Interfaces 10 (2) (1980) 31-34. [2] C.W. Blaxter and W.E.M. Morris (Editors), Economic Evaluation of Computer Based Systems, Book 2 (The National Computing Centre Limited (NCC), Manchester, 1971). [3] S.C. Blumenthal, Management Information Systems; A Framework for Planning and Development (Prentice-Hall, Englewood Cliffs, N J, 1969). [4] C.P. Bonini, Simulation of Information and Decision Systems in the Firm (Prentice-Hall, Englewood Cliffs, N J, 1963). (Reprinted by Markham Publishers, Chicago, 1967). [5] D.F. Boyd and H.S. Kxasnow, Economic evaluation of management information systems, IBM Systems J. 2 (1963) 2-23. [6] E. Chrysler, Some basic determinants of computer programrning productivity, Comrr~ ACM 21 (6) (1978) 472-483. [7] Computable 14 (7) (1981) 2.
45
[8] J.E. Cooke and D.H. Drury, Managing and Accounting for Information Systems (The Society of Industrial Accountants of Canada, Hamilton, Ontario, 1972). [9] N.J. Dean, The computer comes of age, Harvard Business Rev. 46 (1) (1968) 83-91. [10] G.W. Dickson, J.A. Senn and N.L. Chervany, Research in management information systems: The Minnesota experiments, Management ScL 23 (9) (1977) 913-923. [11] P.J. Dixon, Planning MIS investment and expenses levels, Performance Evaluatwn Rev. 9 (!) (1980) 35-37. [12] R.L. Duley, Communicating and executing the short range forecast, Production and Inventory Management 22 (3) (1981) 1-20. [13] G.A. Feltham, The value of information, Accounting Rev. 43 (4) (1968) 684-696. [14] P.G. Hall and T.J. Lincoln, A cost effectiveness study of production and engineering systems within a tightly integrated .manufacturing environment, Management Datamatics 5 (6) (1976) 261-273. [15] J.G. Head, Estimating the costs and benefits and monitoring the effectiveness of large government installations, in: A.B. Frielink, Ed., Economics of Informatics (North-Holland, Amsterdam, 1975). [16] W.C. House (Editor), Interactive Decision Oriented Data Base Systems (Petrocelli/Charter, New York, 1977). [17] J.P.C. Kleijnen, Computers and Profits: Quantifying Financial Benefits of Information (Addison-Wesley, Reading, MA, 1980). [18] A.S. Manne, R.G. Richels and J.'P. Weyant, Energy policy modeling: a survey, Operations Res. 27 (1) (1979) 1-36. [I 9] J. Marschak and R. Radner, Economic Theory of Teams (Yale University Press, New Haven, 1972). [20] M.S.S. Morton, Management Decision Systems (Harvard University Press, Boston, 1971). [21] T.H. Naylor, Corporate Planning Models (Addison-Wesley, Reading, MA, 1979). [22] K.J. Radford, Information Systems for Strategic Decisions (Reston, Reston, VA, 1978). [23] V. Schneider, Prediction of software effort and project duration--four new formulas, A CM Sigplan Notices 13 (6) (1978) 49-59. [24] J.E. Van Aken, On the Control of Complex Industrial Organizations (Martinus Nijhoff, Leiden, The Netherlands, 1978).