Improving MIS project selection

O~4EG 4 The Int. JI of Mgmt Sc~.. Vol. 7. No. 6. pp 527 to 537 Pergamon Press Lid 1979 Printed in Great Bnlain

!)31)5-0..t83 "'~ I II)I-P,4~IS02.00 0

Improving MIS Project Selection I MICHAEL

J GINZBERG

Graduate School of Business, Columbia University (Receired January 1979: i,i revised Jerm April 1979)

develop and install computer-based information systems because they expect their benefits to exceed their costs. Benefits come in all varieties, but our measurement skills are best developed in the area of efficiency improvement---reducing the cost of data processing. Since there are normally more potential projects than available resources, this imbalance in our capacity to measure the full range of benefits can lead to accepting projects of lower overall value than that of some which are rejected. This paper analyzes the impact of present benefits measurement practices, and suggests some possible improvements. Organizations

systems:

BENEFITS M E A S U R E M E N T : T H E BASIC P R O B L E M IN PROJECT SELECTION

(1) equipment displacement,

GAUGING the benefits of management information system (MIS) projects is not a new problem. It has been discussed for many years; yet it remains, for the most part, an unsolved problem. The usual focus in discussions of MIS benefits is on the evaluation process, measuring benefits after the system has been implemented. This paper focuses on another phase of the MIS development process--project selection. The criteria used to evaluate a system after implementation should be the same as those used to select that system for development in the first place. Thus, selection and evaluation are two sides of the same coin. And in many respects, selection is a more difficult problem. Early efforts to use computers to support the management of organizations focused on automating clerical tasks. The primary benefit of these systems was a reduction in the cost of data processing. The use of computers has now moved well beyond clerical automation, and this has greatly expanded the range of benefits. Knutsen & Nolan [7] suggest the following six classes of benefits arising from computer-based t T h i s r e s e a r c h w a s s u p p o r t e d in p a r t b y a g r a n t f r o m the Faculty Research Fund of the Graduate School of Business, Columbia University. t ~ . 7 ~, L,

527

(2) reduction of personnel in data processing tasks, (3) increased operational efficiency in functional areas, (4) increased sales, (5) better managerial planning and control, and t6) other organizational bility.

impacts---e.g,

flexi-

Every system is likely to offer a unique mix of these six types of benefits, and this is the source of the measurement problem. We must find techniques which allow the comparison of systems offering essentially different types of benefits. Some measurement of system benefits is required at both the selection and evaluation stages, but the nature of the necessary measurement differs due to differences in the decisions being made. Four distinct decisions can be identified: (1) go/no go (investment) decisions,

528

G inzberg--l mprovin 9 M I S Project Selection

TABLE 1. DECISIONSREQUIRINGSYSTEMEVALUATION Decision Selection decisions I. Go/No Go decision 2. Resource allocation decision Evaluation decisions 3. Post project audit 4. System tuning

Implicit question

Measurement demands

does this project meet the minimum requirements for consideration'? does this project represent the "'best" use of resources among the available (and potential) uses?

identification of benefits sufficient to meet minimum required level

is this project providing the promised benefits? bow could this system be modified to provide greater benefit to the organization?

tracking of all expected benefits; measurements need not be comparable identification of all benefits: measurements need not be comparable

(2) resource allocation decisions,

identification of all benefits: measurement on a scale which allows comparison across projects

A N O R M A T I V E F R A M E W O R K FOR PROJECT SELECTION

(3) post project audits, and (4) system tuning (performance diagnosis and improvement). The first two of these decisions relate to project selection, and the last two to after the fact evaluation. Table 1 outlines these decisions, the implicit questions upon which they are based, and the demands they place on the measurement of benefits. Measuring system benefits becomes more difficult either (1) when it is necessary to measure all benefits or (2) when benefits must be measured on comparable scales. For the go/no'go decision on a single project, neither of these requirements apply. For after the fact evaluation decisions it is often necessary to consider the full range of benefits, but comparable scales are seldom required. It is only in the case of project selection in a limited resource e n v i r o n m e n t - - t h e Resource Allocation Decision--that it becomes critical to measure all types of benefits a n d to use comparable measures for all types. If only a few organizations needed to face this question of resource allocation, the need for improved project selection techniques would not be great. However, this is not the case. Most organizations have more opportunities than available resources [12]. Thus, the value of improved techniques for project selection could be substantial.

Ideally, the solution to the project selection problem should be the application of costbenefit analysis. Carlson [3], in reviewing six selection and evaluation techniques, concludes that cost-benefit analyses should provide more meaningful results than any of the other available techniques. He notes, however, that there are often problems in obtaining measures of project benefits. The existence of these measurement problems led Knutsen & Nolan [7] to reject costbenefit analysis as a valid tool for MIS project selection. They contend that the technique's focus on monetary measures of benefit will lead organizations to choose projects which are clerical in nature, because the resulting cost savings are easily measured. Thus, they argue, projects in areas of great importance to the organization will not be undertaken because their benefits are more difficult to measure. Knutsen & Nolan's point should not be ignored. If the use of cost-benefit analysis results in ignoring the harder to measure benefits, it will often lead to unwise decisions. We must ask, however, whether this is a necessary result of using the technique. King & Schrems [6], in a tutorial on costbenefit analysis for MIS, suggest that the technique can be used to consider a wide range of benefit types. They point to certain measurement problems, but they do not believe these problems are insurmountable. Indeed, they

Omega. Vol. 7, No. 6

describe the ways in which the dollar value of certain hard to quantify benefits has been estimated in past projects. To be sure, there are difficulties in estimating the value of some project benefits. King & Schrems [6] list a number of these problems, including)-' (l) the "natural' unit of measurement may not be comparable across all benefits; (2) some benefits will be of different value to different people (e.g. different users of a multi-user system); (3) the quantification of some benefits is, of necessity, highly subjective and subject to great uncertainty; (4) the benefits actually obtained may depend on the operating environment for the system (e.g. response time of the system may affect the value of the information it produces); and (5) benefits are estimated at the start of the project but may change over the life of the system. Their implicit response i s that while these problems make the proper use of cost-benefit analysis difficult, they do not make it impossible. Indeed, if we consider the types of benefits claimed for MIS projects (Knutsen & Nolan's list, for example), it should be possible to quantify all of them. Each of these benefits is aimed at increasing the efficiency or effectiveness of the organization. Ultimately, such changes are expected to have a financial impact on the organization. In some cases---e.g., equipment displacement, personnel reduction--the impact is immediate and easy to measure. In other cases--e.g, better planning and control, increased flexibility--the financial impact is less direct and less certain in magnitude. Nonetheless, the expectation of an impact is there. The discussion so far has considered only the favorable impacts of information systems. It has often been noted (e.g. [1, 2, 9]), that not all

529

impacts of a change to an organization's information system are beneficial. These unfavorable impacts have costs associated with them. Further, they are likely to lead to resistance to the new system, thus reducing or even eliminating the benefits to be derived from it. A comPlete discussion of these unfavourable impacts is beyond the scope of this paper. Nonetheless, the possibility of this type of impact must be considered in evaluating any new information system, expecially when a choice must be made among alternative systems. We can summarize this discussion by sketching a normative model of project selection. In order to properly allocate the organization's resources among competing alternative uses, we must be able to identify these alternatives and to compare the benefits offered across alternatives. This comparison must include the total package of benefits offered by each project, and should recognize the possibility of unfavorable impacts. Since different projects offer different mixes of benefit types, performing this comparison requires that benefits be quantified. The expectation of a financial impact is the key to this quantification. In the next section, certain aspects of the MIS project selection process in a major multinational corporation will be examined. The process followed by this corporation will be compared to the normative process outlined above. Where discrepancies are noted, an attempt to explain them will be made. In the final section of the paper, some ways to bring practice more closely into line with the normative model will be suggested. P R O J E C T S E L E C T I O N AT ALPHA P R O D U C T S

Alpha Products is a major corporation, with headquarters in the United States and operations throughout the world. Alpha has a reputation for being progressive in its use of management science techniques (including information systems) and has a long record of experience in this area. Alpha uses a formalized systems life cycle approach similar to those now used by many large organizations. Thus, while not a statistical sample, Alpha should provide data representative of the current state 2A similar list of difficulties, raised in the context of of practice in MIS project selection methodcomputer selection, can be found in [I I]. ology.

530

Ginzber~--lmproving M I S Project Selection

Data were collected by analyzing written proposals for 71 MIS projects at Alpha. The bulk of these (61) came from three Corporate ONce units--Finance, Personnel, and Operations--the remainder coming from overseas units which report to the Corporate ONce. The projects ranged widely in size--direct development costs varied from $I000 t o nearly $500,0(XI--and in type--including new system development, system extension, program modification, support of production runs, consulting services, special one time analyses, etc. For purposes of analysis, projects were placed into three groups: (1) Development--the development of a new system or the extension of the functional capabilities of an existing system; (2) Modification--changing report formats, file structures or the specifics of processing in an existing system; and (3) Support--provision of general MIS consulting services, one-shot reports, etc. Alpha's written project proposals do not reveal the entire project selection process. They do enable us to examine, however, the estimation of project benefits--the extent to which benefits are quantified, whether quantification is based on a common scale, and the breadth of benefits considered. As suggested earlier, the benefits claimed for MIS projects are increases in the organization's efficiency--reducing the Cost of accomplishing clerical tasks--or effectiveness--enabling better performance of some substantive task. 3 Most of the benefits cited in Alpha's project proposals could be classified as one of these types. However, in a number of instances, another type of 'benefit' was cited--addressing a mandated reporting or recordkeeping requirement. With the inclusion of this third type, all bene-

J The definition of efficiency being used here is a relatively narrow one. Improvements in the O u t p u t - l n p u t ratio in non-clerical areas will be considered increases in effectiveness for the remainder of the paper. "~Some projects cited no benefits at all. While these data could be used for some parts of the analysis, no attempt was made to determine which type(s) of benefit the project provided.

fits cited could be classified as one of the three types--Efficiency, Effectiveness or Mandated. ~

The pattern of benefits quantification Of the 71 project proposals in our sample, 25 (35.2~) make some attempt to quantify (on some scale) the expected benefits (or a portion thereof); the remaining 46 projects either make no attempt to quantify the suggested benefits (38 cases) or list no benefit at all. All proposals in this sample were for projects which had been approved. Thus, a sizable percentage of the projects selected by Alpha for implementation were chosen without documentation of any quantitative estimates of their likely benefits. To some extent, the lack of quantification is due to the variety of project types and sizes included in the sample (see Table 2). It is apparent that benefits were less likely to be quantified for maintenance projects and small projects than for the development projects and larger projects. Overall, though benefits were quantified for many of the largest projects, for a large number of significant projects this was not the case. Further, we have considered only whether any benefits were quantified, not whether the full range of benefits had been examined and their values estimated. The data in Table 3 address this latter question. The majority of projects claiming any benefit at all list only a single specific benefit or impact on the organization. While this does not prove a failure to consider the full range of system benefits, it is suggestive of one, as most projects have multiple impacts on an organization (consider Knutsen & Nolan's six categories). These data also shed more light on the extent to which benefits are quantified. The tendency to quantify benefits increases--both in the percentage of projects for which at least some benefits are quantified and in the average percentage of the indicated benefits quantified per project--with the number of specific benefits identified. However, even in the categories with the highest degrees of quantification, only about half of the indicated benefits are quantified. In summary, the pattern of MIS project benefits quantification at Alpha shows that for the majority of projects, no quantification of benefits has been made. This is most apparent on small projects and on projects to support or modify existing systems. The evidence also sug-

Omega. Vol. 7, No. 6

531

TABLE 2. BENEFIT QUANTIFICATIONa BY PROJECT TYPE AND SIZE Size s Less than $ l 0,000 Type Production support System modification System development

$10,00050,000

Greater than $ 50,000

8*

4

0

12

(0%),

(0%)

(--)

i0%)+

23 (34.8~;) 12 (8.3~)

4 (25.0%) t2 (66.75;)

0 (--) 8 (87.5~o)

27 (33.390) 32 (50.0°(,)

43 (20.9~)t

20 (45.0~)

8 (87.5~)

71

* N u m b e r of projects. ~ Percentage of projects having quantified benefits. ' O n e entry per project, classified as quantified if any benefits for the project are quantified. b Development cost.

We will consider four possible explanations for the existing practices:

gests that consideration of the full range of benefits is not occurring in a large number of Alpha's project proposals. Further, there is no evidence that possible unfavorable system impacts were considered in any of the 71 projects examined. These findings indicate substantial discrepancies between Alpha's practices and the normative model. We can infer that MIS project selection decisions at Alpha are being based on inadequate data, data which cannot assure the best allocation of resources among competing project proposals. And, it is unlikely that this situation is unique to Alpha.

(1) the economics of analysis, (2) resource allocation procedures at Alpha, (3) the political nature of the organizational environment, and (4) the linkage between observable impacts and financial outcomes.

Economics of analysis. Performing a costbenefit analysis is time consuming and costly. For some projects, especially small ones, the possible benefits to be derived from a rigorous examination may be smaller than the cost of examination, For such projects, it would not make economic sense to conduct an analysis. Thus, the substantial number of small projects included in the sample may provide a partial explanation for the existing pattern.

Possible explanations for the pattern observed Before attempting to prescribe ways to bring practice more in line with the normative model, we should consider some possible explanations for the existing situation. Without some understanding of why things are as they are, it is difficult to suggest meaningful changes.

TABLE 3. NUMBER OF BENEFITS LISTED BY PROJECTS

N u m b e r of specific benefits claimed 0 1

2 3 4 5 or more

N u m b e r of projects

Percent of projects

Percent of projects with some benefits quantified

8 32 15 9 3 4

11,3 45.1 21.1 12.7 4.2 5.6

0 28.1 46.7 44.4 66,7 100

71

100

Percent of specific benefits quantified 0 28.1 36.7 22.2 50.0 51.9

Gin:berg--Improving

532

MIS Project Selection

TABLE 4. BENEFIT QUANTIFICATION BY BENEFIT TYPE WITHIN FUNCTIONAL AREA

Corporate finance Mandated Efficiency Effectiveness

Corporate personnel

Corporate operations

2*

13

0%*

0%

--

0

15

5 20°/~ 7 0]°/o

15 46.7~o 11 0~/o

7 71.47o 8 62.590

27 (48.19/o) 26 (I9.2~)

14 O/ (7.1/o)'P

39 (17.9°)

15 {66.7~o)

68,

o.o%~

* Number of benefit classes listed by projects. "p Percentage of benefit classes listed having quantified benefits; a class was considered quantified if any benefit of that class was quantified. ' Total number of benefit classes listed in the 61 projects from corporate office units.

Resource allocation procedures. Alpha allocates a major portion of available system development resources to organizational units when annual budgets are drawn up. Each unit has the responsibility to manage the resources allocated to it, and the allocation of resources to specific projects is accomplished in the local environment. The manager of an organizational unit may feel that he (or she) has enough understanding of that environment to make allocation decisions without rigorous analyses of project impacts. Again, this may explain in part the observed pattern. The political nature of the environment. Organizations are acknowledged to be political entities [13]. As a result, actions taken are not always based solely on economic criteria. Other criteria may supplement, or even supplant, economics in the project selection decision. If non-economic criteria play a dominant role in project selection, there is little reason to conduct formal economic analyses. No formal data were collected at Alpha to test the extent to which po!itical (non-economic) factors influence project selection decisions. However, the large number of projects selected at Alpha without formal economic justifications is consistent with political behavior. Linkage to financial outcomes. Some of the impacts of MIS projects can be linked quite readily to financial outcomes; for others the linkage is not so straightforward. The data from Alpha provide two examples of this phenomenon--the contrast between efficiency and effectiveness benefits, and differences in the degree of linkage as between different functional areas. In the former case, better models generally exist of the relationship between effi-

ciency improvements (i.e. changes in data handling operations) and financial outcomes than of the relationship between effectiveness improvements and financial outcomes. The data (Table 4) bear this out. There is a markedly higher incidence of quantification of efficiency benefits than of either of the other types. Across functional areas, generally the models linking activity to financial outcomes are better for functions close to the organization's outputs (e.g. production or distribution) than for functions which are distant from those outputs (e.g. finance or personnel). The more explicit the model for a functional area is, the more likely it is that the value of changes in the information system for that area can be estimated. Table 4 indicates that Finance and Personnel show markedly lower tendencies to quantify project benefits than does Operations, and this is consistent with the linkage hypothesis. This difference across departments also appears when we look at the three classes of benefits separately. Operations is less likely than the staff departments to claim a system as mandated. And, in both the efficiency and effectiveness classes, Operations shows substantially higher percentages of benefits quantified than do the Personnel and Finance departments. While the linkage hypothesis might explain the difference for effectiveness benefits, improvements in data processing efficiency should be no more difficult to quantify in Personnel than in Operations. Perhaps some sort of 'halo effect' is in operation; the practice of not quantifying benefits in areas where models are fuzzy, spills over to the efficiency area where good models do exist.

Omega. Vol. 7. No. 6

These four practices may well account for much of the incomplete analysis of project benefits observed at Alpha. Before suggesting ways to change the project selection process, we must consider the purposes served by the four practices. Not performing a rigorous analysis where the cost of analysis is high and that of the project low is sensible. Submitting all projects to thorough analyses could increase costs far more than it would likely increase benefits. There is a danger, however, that this procedure will result in the bulk of available resources being allocated to low cost, and likely low benefit, projects, leaving inadequate resources for larger projects with greater potential. By placing decision and control responsibility in the hands of the local unit manager, lump-sum allocations garner all of the benefits normally suggested for organizational decentralization [I0, Chapter 24]. However, by giving individual organizational units uncontested claim to a certain quantity of resources, this procedure introduces a disincentive to rigorous analysis of project benefits. Such analyses could show that one unit's projects were not as beneficial as those in other units, and this could lead to a loss of resources. If the analyses are never made, inter-unit comparisons cannot be made, and each unit is assured of keeping its pre-allocated share of the pie. Political behavior in organizations is a fact of life. It would be difficult, and probably undesirable, to eliminate political criteria from the project selection decision. While the use of non-economic criteria is not by itself bad, the impact of that use may be to drive out hard analysis. Without analysis, it is difficult to assess how well the allocation of resources to projects matches the organization's objectives. Economic analyses should be one input to the admittedly political decision making process. In the case of the linkage hypothesis, the question is not so much whether it is desirable, but rather whether it is inevitable. Are staff functions so remote from the business of the organization that the impacts of changes in their methods are essentially, non-quantifiable? The ten project proposals from Alpha's overseas divisions suggest this is not so. All of the projects proposed by these divisions listed quantified benefits. All efficiency benefits are quantified (7 projects), as are half of the effec-

533

tiveness benefits (6 projects). Of these ten projects, nine are in the finance area and one is in operations. Two of the financial systems are justified solely on the basis of quantified effectiveness type benefits. Thus. it is possible to estimate the impact of effectiveness improving information systems in areas remote from the organization's product. In summary, of the four practices suggested as possible explanations for the pattern of incomplete project analysis observed at Alpha:

(1) two--economics of analysis and lump-sum allocations---can serve useful purposes but can also be dysfunctional;

(2) o n e - - t h e linkage hypothesis--is probably quite common, but serves no useful purpose and can be avoided; and

(3) o n e - - t h e political environment--is a fact of life sometimes used as an excuse for failing to perform an analysis. The next section of this paper considers an approach to modifying the project selection process to bring it closer to the normative model. These modifications directly address the problem of linking information system changes to financial outcomes, and may impact the economics of analysis by making good analysis less costly. They do not address the problems posed by lump-sum resource allocations nor by a political environment which chooses to avoid explicit analysis. SOME D I R E C T I O N S FOR C H A N G E One key requisite for improving benefits esti' mation is the development of a meaningful taxonomy of benefit types. The purpose of this taxonomy is to help identify the full range of project benefits and to enable their measurement. T o achieve this end, the taxonomy should (1) be comprehensive, covering all types.of system benefits, (2) be concise, lest it be too cumbersome to be of use, and (3) provide guidance for the benefit measurement and comparison processes.

534

Ginzberg-.-Improc'ing M IS Project Selection TABLE 5. CLASSIFICATIONOF CITED INFORMATION SY$TEM BENEFITS Benefit class

No, of cases

°o of cases

e~, quantified

Mandates

1. Record or report required information ('Mandated')

14

11.3

0.0

Organizational processes

2. Reduce information processing or handling costs ('Efficiency') 3. Improve asset utilization and resource control 4. Improve planning process 5. Increase organizational flexibility 6. Promote organizational learning

47

37.9

70.2

12

9.7

83.3

1 4

0,8 3.2

100.0 0.0

0

--

--

7. Provide greater accuracy in clerical operations.

20

16.1

0.0

6

4.8

0.0

16

12.9

6.3

3.2

50.0

Information characteristics

fewer errors

8. Provide information on a more timely basis 9. Provide new/more/better information 10. Other

4

Total

124

The taxonomies introduced so far in this paper meet the first two criteria, but fail on the third. The following taxonomy is proposed as a starting point: (1) Record or report (Mandated);

required information

(2) Reduce information processing or handling costs (Efficiency); (3) Improve asset utilization and resource control; (4) Improve planning process; (5) Increase organizational flexibility; (6) Promote organizational learning; (7) Provide greater accuracy in clerical operations, fewer errors; (8) Provide information on a more timely basis; and (9) Provide new/more/better information. This taxonomy appears to meet the three criteria suggested above. Clearly, it is concise;

100

it includes only nine categories of benefits. We can test the comprehensiveness of these categories using data from Alpha. The benefits listed in the 71 project proposals were classified according to the proposed taxonomy. Table 5 shows the results of this classification, and samples of the benefits falling into each category are presented in the appendix. All but four of the 124 specific benefits listed in these proposals could be placed in one of the nine categories. The remaining four could not be categorized because they differed in level of analysis from the categories in the taxonomy (to be discussed below). While the proposed taxonomy was adequate for classifying almost all statements of benefit included in Alpha's MIS project proposals, it is doubtful that the taxonomy is exhaustive as it stands. First, only beneficial impacts have been included. Categories representing unfavorable impacts might be added; but the analysis of these impacts and their classification is beyond the scope of this paper. Second, additional categories of beneficial impacts might prove necessary, and these could be added as the need for them is shown. Finally, we can consider the guidance this taxonomy provides to the measurement and comparison processes. Information system impacts can be analyzed at a number of levels.

Omega. Vol. 7. No. 6 The highest level is the system's impact on organizational outcome variables--sales revenue, customer satisfaction, profit contribution, etc. These are the ultimate impacts of the system. They do not follow from it directly and immediately, but rather result from changes to organizational processes----e.g, planning, control, learning. These processes represent a second level of analysis, and are included in the taxonomy as benefit classes 2-6. There is a third, still lower, level at which information system impacts can be analyzed: the immediate changes to the information produced by the system and available to the organization. These changes, represented by benefit classes 7-9, affect the organization only through the processes listed above them. Eight of the nine benefit classes included in the taxonomy fall into these latter two groups---changes to organizational processes and changes to information produced. Ultimate impacts were not included; first, because the number of unique ultimate impacts is too large, and second, because they can be determined once the process changes are identified. The ninth benefit class included in the taxo n o m y - - m a n d a t e d changes to information syst e m s - f o r m s a group by itself. The final column in Table 5 shows the degree to which benefits were quantified within each of the classes. The bulk of the quantification occurred in those classes representing changes to organizational processes. This is not a coincidence, and it provides useful insight. Changes to processes are the link between changes to information and organizational outcomes. It is only once we understand how the new information will be used that its value can be estimated. Thus, efforts to quantify benefits should focus on the changes in organizational processes which will result from changes to information systems. It should be possible to develop estimates for all benefits in the process change category. The degree of certainty will vary from class to class, and the ordering of these classes is meant to approximate decreasing certainty of estimates. At one extreme, the impact of increased data processing efficiency can be estimated with relative certainty. Toward the other extreme, the impact of increased organizational flexibility is far less certain. But, this does not mean that it cannot be estimated. There is evidence

535

to suggest that managers have fairly good implicit models of the processes that affect their organizations [4]. We should be able to tap into these models to develop at least a range (e.g. best case-worst case scenarios) of estimates for the value of these less certain process changes. Changes to information characteristics are, by themselves, of no value, and it does not make sense to attempt to estimate values for informational changes as such. Rather, consideration should be given to how these changes will affect organizational processes. The impact of those process changes can then be estimated. In the case of mandated changes, there really is no reason to estimate the value of the change. Where a particular change truly is mandatory, emphasis should be placed on accomplishing the change at the lowest cost. Care must be taken, however, to assure that classifying a change as mandatory is not simply a device used to avoid rigorous analysis of a project. In summary, the taxonomy helps to guide the measurement process by focusing it on those types of benefits which can be estimated, process changes. Other benefit types should first be converted to a process change, and then an estimate can be made. One problem remains. Since uncertainty about the value of a benefit will vary from class to class, meaningful project to project comparisons cannot be made simply by summing all benefits for a given project and comparing totals across projects. That would imply that the sole concern was the expected value of the benefits without regard to risk. Risk and return must both be considered, and the organization's risk preference must enter into this consfderation. At least two general approaches for comparing projects having differing mixes of benefits (and hence, uncertainty) seem possible. One is to apply 'discount factors' to expected benefits according to the degree of uncertainty they present, and then to sum across benefits. The more uncertain the benefit, the larger the discount would be. Scoring models (such as in E8]) are representative of this general approach. The problems with this technique are (1) it likely understates the relative value of a project with a preponderance of benefits in the more uncertain categories, and (2) it may

536

Ginzbery--Improving MIS Project Selection

be difficult to determine discount factors which reflect the organization's risk preferences over a range of situations (e.g. type of benefit, size of the minimum or maximum benefit). A second approach is to present each project as a vector, showing for each benefit category minimum, maximum, and most likely values. This addresses the two problems raised above, but presents a new one. The individual decision maker must now make the risk-return tradeoff, and there is evidence which suggests that individuals who see only a subset of the organization's activities are more risk averse than are top managers of the organization [5]. No solution is offered at this point, but both approaches seem to be worth trying. SUMMARY AND D I R E C T I O N S FOR F U R T H E R RESEARCH This paper has suggested that project selection is an important, often overlooked step in the MIS development process. Effective project selection decisions require consideration of all benefits to be derived from a project and measurement of these benefits on comparable scales. Current practice often involves estimating those classes of benefits which are easiest to measure and about which there is little uncertainty, and describing other very real benefits as intagnible. In fact, these latter benefits a r e tangible; however, they cannot be measured as easily or with as great certainty as can the former group. A taxonomy to help identify the full range of benefits was presented, and the outline of an approach to benefits measurement was sketched. The next step must be a test of this taxonomy and refinement of the measurement approach. Among the questions requiring answers are : (1) will application of the taxonomy lead to more complete identification of project benefits? (2) is the taxonomy rich enough to capture the full range of benefits? (3) can changes to information characteristics be linked to organizational process changes? 5 Number of specific statements in this class in the sample of 71 projects at Alpha Products.

(4) how can estimates differing in uncertainty best be combined and compared?

(5) can this approach work in an inherently political environment? and (6) what impact, if any, does following this approach have on project selection decis i o n s - t h a t is, will they improve?

ACKNOWLEDGEMENT The author wishes to thank Martin Starr. Jon Turner and Miklos Vasarhelyi for their helpful comments on earlier versions of this paper.

REFERENCES I. ARGYRIS C (1971) Management information systems: the challenge to rationality and emotionality. Mgmt Sci. 17(6), B275-B292. 2. ARGVRIS C (1977) Organizational learning and management information systems. Acct 9, Orgns & Soc. 2(2), 113-123. 3. CARLSONED (1974) Evaluating the impact of information systems. Mgmt Informatics 3(2), 57-67. 4. GORR¥ GA (1971) The development of managerial models. SIoan Mgmt Rev. 12(2), 1-16. 5. HAMMONDJS III (1967) Better decisions with preference theory. Harvard Busin. Rev. 45(6), 123-141. 6. KtNG JL & SCHREMSEL (1978) Cost-benefit analysis in information system development and operation. Cornput. Surv. 10(1), 19-34. 7. KNUTSEN KE & NOLAN RL (1974) On cost-benefit of computer-based systems. In Manaoin 9 the Data Resource Function (Ed NOLAN RL), West Publishing Co, St. Paul, USA. 8. LUCASHC JR & MOOREJR (1976) A multiple-criterion scoring approach to information system project selection. INFOR 14(1), 1-12. 9. MUMFORDE & BANKSO (1967) The Computer and the Clerk. Routledge & Kegan Paul, London, UK. 10. SHILLINGLAWG (1977) Managerial Cost Accounting, 4th ed. Richard D Irwin, Homewood, Illinois, USA. 11. TIMMRECK EM (1973) Computer selection methodology. Comput. Surv. 5(4), 199-222. 12. TUCKER CC (1978) From the chairman. Data Base 9(3), 2-3. 13. TUSHMANML (1977) A political approach to organizations: a review and rationale. Acad. Mgmt Rev. 2, 206-216. ADDRESS FOR CORRESPONDENCE: Michael J Ginzber#, Esq, Associate Professor of Business, Graduate School of Business, Columbia University, New York, N Y 10027, USA. APPENDIX Samples of Benefits Specified in Alpha Products' MIS Project proposals I. Record/Report Required Information ('Mandated') - - n = 145 1. comply with court order 2. top management committee mandate

Omega, Vol. 7, No. 6 3. to comply with ERISA 4. audit department recommendations 5. company policy 6. tax department request II, Reduce Information Processing,,Handling Costs ('El~ciency')-,--n = 47 1. relieve payroll department manual effort 2. reduce reruns 3. reduce time spent researching and correcting errors 4. man-hour time savings over prior method 5. reduce number of Master Files created 6. savings in cost of report mailing 7. eliminate double data entry and m a n u a l updating 8. manpower savings through automation of analysis 9. eliminate maintenance on existing system 10. reduce training needs I I. improved et~ciency in assembling and distributing bills of material 12. simplify user procedures for processing requests 13. save time and effort on future system modification III. Improve Asset Utilization & Resource C o n t r o l - n=12 1. reduced transport delays 2. reduction in equipment maintenance costs and costs due to equipment failure 3. better allocation leading to increased product value 4. better control of inventory 5. better use of reworked goods through better information about product characteristics 6. permit more effective control of projects by managers and analysts IV. Improve Planning Process--n = I 1. better composition of product lines V. Increase Organizational Flexibility--n = 4 I. flexibility to handle special mailings 2. system easily modifiable as operating environment changes 3. system will allow quick and accurate response to crises

VI. VII.

VIII.

IX.

X.

537

4. flexibility to adapt system to changing management needs for information Promote Organizational Learning--n = 0 None Provide Greater Accuracy in Clerical Operations; Fewer E r r o r s - - n = 20 1. assure proper calculation of premiums 2. better control over price changes--fewer failures to update billing price to current level 3. better recovery of customs duty paid 4. improve audit, control, and coordination of benefit plan payments 5. provide m a n a g e m e n t with current reports and insure data integrity 6. verify aggreement with insurance c o m p a n y records 7. elimination of inter-system inconsistencies 8. identify errors before check printing Provide Information on a More Timely Basis-n=6 1. enable running of tests on regular basis without need for analyst involvement 2. provide accurate information more quickly so that decisions can be made sooner {production and inventory decisions) 3. complex analyses can be made available in more detail, more quickly, and more accurately Provide New/More/Better Information--n = 14 1. file needed to produce retirement projections 2. system easier to use and more understandable 3. more effective provision of information required for purchasing and vendor delivery scheduling 4. able to do more analysis of operations data and determine financial effects 5. improve communication with beneficiaries 6. analyze impacts of regulations on existing systems 7. make needed format changes Other--n = 4 1. prevent possible recurrence of a past problem 2. better technical assistance to customers leading to higher profit contribution 3. increased profit contribution 4. provide needed support