- Email: [email protected]
Measuring the strategic dimensions of the quality of information
Journal of Strategic Information
Systems 1994 3 (3) 179- 190
Measuring the strategic dimensions of the quality of information G J van der Pijl Tilburg
University,
5000 LE
Tilburg,
Department
of Information
Systems,
P 0 Box 90153,
The Netherlands
Although much has been written on quality of information there is no uniform framework in which different approaches to quality can be brought together. In this article such a framework is presented. From the framework an approach for assessing the strategy and goal related elements of quality of information in an organization is derived. Some examples of the first applications of this approach in practical situations are included. Keywords:
quality
of information,
information
systems
The more dependent on information organizations become, the more important it is to ensure that the information is of good quality. Hence, organizations should, from time to time, assess the quality of the information they use. This means that there is a need for a measuring instrument. When the research on which this article is based was started, the intention was to construct such a measuring instrument using the quality concepts that could be found in literature. This instrument should enable a company’s management to get a well-founded judgement of the quality of the information they are provided with, and thus of the quality of the information systems installed. The instrument should, in other words, support an ex post quality assessment. A first survey however proved that, although a lot has been written on concepts of quality of information, the literature offers no uniform or consistent framework to approach the subject. In the late 1970s well-known authors with a software engineering background, like Boehm (1978) and McCall (1979), tried to describe the quality of software in terms of series of quality attributes or quality trees. Different sets of quality attributes were proposed and attributes were described in terms of more and more detailed attributes until the level of measurable properties of the software was This paper was one of a small number of papers presented at the SISnet Conference held at Tilburg University, The Netherlands in June 1993 recommended for publication in JSIS by members of the SISnet Conference Programme Committee, namely Professors Pieter Ribbers (chair), Joachim Griese and Helmut Krcmar. Revised paper accepted for publication by Professor R D Galliers, January 1994 The quality model presented in this paper was developed in cooperation with Mrs C A van Egten of the Free University, Amsterdam (1992). The approach to assessing the teleological elements of the quality of information is described more elaborately in Pijl (1993).
0963-8687/94/030179-
12 0
1994 Butterworth-Heinemann
Ltd
179
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
reached. The purpose of this effort was twofold. On the one hand the aim was to formulate a set of quality attributes as the basis for quality specifications in software contracts. On the other hand specification of a set of quality properties should simplify the quality assurance process of software development. More recent European contributions in this field have been presented by Wilmer (1985) and Delen and Rijsenbrij (1990). Another approach to quality can be found in the literature on financial auditing and on electronic data processing (edp) auditing. Here reliability, continuity and exclusiveness are seen as the most important quality attributes. Only in recent years has attention also been paid to effectiveness and efficiency. A classic Dutch publication in this field is by Starreveld et al. (1986). In the 1980s the quality literature concentrates on measuring user satisfaction. Bailey and Pearson’s (1983) article is a well-known representative of this period. Their method uses insights into quality attributes developed earlier. Their questionnaire combines these attributes with questions on the functioning of the systems development and data-processing departments. At about the same time we see publications on success and failure factors for information systems, which also can be seen as an approach to quality of information systems. In recent years quality issues have again been widely discussed after the presentation of the IS0 9000 (1987) norms for quality assurance. The orientation here is mainly on the quality attributes of the software development process, the idea being that improving the process will also improve the quality of the software product. The supposed relations between quality aspects of the process and quality aspects of the products, however, are not supported by published empirical facts. Last but not least the literature on information planning and on capital-budgetting aspects of investment in information systems (e.g. McFarlan, 1984; Bedell, 1985; Parker and Benson, 1988; Hammer, 1990) has to be mentioned here. Although these publications are seldom referred to in discussions on quality of information, they are very relevant: the quality of information should be judged by its contribution to the goals of the organization. Assessing the relationships between the organization and its information systems is exactly what these publications are about. From the foregoing analysis two conclusions can be drawn. First, the literature offers no uniform approach to quality of information. Second, a connection to the literature on infornformation planning is lacking. As stated before, the goal of this study was to develop an instrument to measure the quality of information. For this purpose a model of quality of information had to be developed. This model should capture relevant aspects of quality, including those related to the organizational goals. The quality model In order to derive a conceptual model for the quality of information in an organization we start with a discussion of the concepts of ‘quality’ and ‘information in organizations’. From there we try to derive the elements of the conceptual model. Quality
The first definitions of the term ‘quality’ come from the ancient Greek philosophers. In more recent literature we find studies on the general notion of 180
Journal of Strategic Information Systems 1994 Volume3
Number3
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
quality in the work of Pirsig (1974) and Hofstadter (1979). Traditionally the term quality is used in relation to products and people, in more recent definitions, like for example the one from IS0 8402 it becomes clear that it can also be related to services and processes. The IS0 8402 definition for quality is the totality of features and characteristics of a product, process or service that bear on its ability to satisfy stated or implicit needs. The concept of quality can be related to processes in two different ways: l
l
First, products and services to which quality is related are produced in a process. The features and characteristics that determine the quality of products and services, are created in that process. Hence, control of the quality of those products and services can be exercised by controlling the production process. Authors like Deming (1982) and Juran (1974) argue that all steps of the production process have to be thoroughly controlled in order to create products with the desired quality features. Second, quality features and characteristics, can be determined for processes and thus also for production processes. Of course the quality features of production processes should be chosen in such a way that they guarantee the quality of the final products.
Any conceptual quality model should take account of the importance of the production process for the quality of goods and services and for the possibility of determining quality features for the production process itself. An important point on which almost all definitions of quality agree is the fact that quality of a product or service always has to be considered in the light of the use that is made of it. This is clear from the IS0 8402 definition but, for example, also from Juran who states that quality is ‘fitness for use’ or in other words: ‘the extent to which the product successfully serves the purpose of the user during usage’. From this last definition it is also clear that in Juran’s vision the user is the one who decides what are the features and characteristics that are important for the use of the product. The IS0 definition of quality takes into the account that there is not always a situation in which the needs of a future user are described in a contract. Therefore it states that quality can also be related to ‘implied needs’. This supposes that in certain cases or for certain features a more objective description of quality, which is not associated with a particular user, is possible. The definition, however, does not say how these ‘implied needs’ can be determined. From the above we conclude that a conceptual model should relate quality features and characteristics of information to the situation in which it is used. As will be described later on, the goals and targets of organizations will be used as descriptors of this situation of use. Another important feature of most quality definitions is that quality is described in terms of a series of specific features and characteristics of a product, a service or a process. The user presents a set of features and characteristics that describe his demands. The producer will try to translate the demand-oriented set into a set of features and characteristics he can make. In literature the features and characteristics specified by users are associated with the ‘fitness for use’ or product oriented approach to quality while the set specified by the producers is associated with the ‘conformity to specifications’ or production process oriented aspects of quality. In both approaches quality can be decomposed in sets of features and characteristics. This is of course necessary when one wants to control for quality in Journal of Strategic Information Systems 1994 Volume 3 Number 3
181
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
production processes. Such processes cannot be controlled if quality is not specified. Specifying can be done by translating quality in demand specifications and next in product specifications. The production process can then be controlled in such a way that the product conforms to the product specifications and thus to the quality demanded. This way of quality control is also followed in the design and construction of information systems. Often, however, information systems that conform to all carefully selected quality features and characteristics don’t deliver, in the view of the user, the quality demanded. This raises the question whether the ‘reductionist’ approach to quality might be wrong. An alternative would be to look for a holistic approach like in Zen and the Art of Motorcycle Maintenance. Pirsig (1974) supposes that the term quality relates to objects as indivisible entities. The quality-experience is directly coupled to the objects observed and cannot be isolated. In other words: the quality of a product is more than the sum of its parts. The problem with such an approach is that the idea of quality cannot help controlling the production of goods and services. As Smits (1990) showed, Hofstadter’s approach indicates a possible way out of this dilemma. Hofstadter (1979) says that the difference between an object and the sum of its parts is caused by the complexity of the object observed. This complexity makes it very difficult to see the relations between all parts of the object observed. This does not mean that in reality there are no logical and systematic relations. People can only communicate about such complex systems if they have a common conceptual framework. In the context of such a framework opinions and experiences can be exchanged using a limited number of concepts that are clear to everybody. The lack of such a conceptual framework may well be one of the reasons why systems development, in spite of very conscious analysis of information needs, often does not deliver the system the user is looking for. The conclusion should not be that the reductionist concept of quality is wrong. It should be that we obviously do not yet have a set of common concepts that enable us clearly to describe the desired quality of an information system. Developing a conceptual framework, in which statements on the quality of information systems can be made, can be seen as a contribution to the construction of such a common set of concepts. Such a framework should, in view of the discussion above, allow a very detailed approach to quality as well as an approach in more global qualitative terms. Information
in organizations
Looking at the use of information systems in organizations we see two main trends. First, the amount of information produced in organizations has been growing tremendously. Several general studies (like Inose, 1990) show that the amount of information used grows less than the amount of information offered. It is not known whether this is also true for the use of information in organizations. There are however several reasons why the use of information in organizations should probably grow. One reason is that the growing size and complexity of organizations makes it impossible to control them only on the basis of one’s own observations. Another important reason is the growing complexity and dynamism of the environment in which most organizations operate. Second, the way in which organizations use information systems is changing. Initially they were mainly used to automate specific, labour intensive types of information processing such as financial administration. Later on attention shifted to the information needed to control business processes and organizations as a whole. The next step was not only to use information systems for internal purposes 182
Journal of Strategic Information Systems 1994 Volume 3 Number3
Measuring
the strategic dimensions of the quality of information:
G J van der Pijl
but also for reaching a good competitive position (Porter, 1980, 1985; Wiseman, 1985; Parker and Benson, 1988). The most recent development is to use information systems for new ways of organizing (see e.g. Nolan et al., 1989). All of these changes in the use of information systems also cause changes in the way in which decisions on information systems are (should be) made. In the time of specific labour intensive systems decisions could often be made for each system separately. When the attention shifted to management information systems focused on control, decisions to be made often involved an integrated set of systems. Because of the later shift to strategic use of information systems decisions were taken against the background of the organization’s strategy. Since market strategies are mostly decided on at the level of the lines of business, the decisions on information systems are also taken at this level. Decisions on information infrastructure enabling the redesign of organizations are also taken at a fairly high level in the organization. The same goes for decisions on specific systems that are considered vital for the organization as a whole (Keen, 1988). The changing ways in which information systems are used within organizations and in which decisions on information systems are taken have to be incorporated into our concept of quality of information. This means that the set of quality features and characteristics that together make up the quality of an information system should not only reflect the very specific and detailed demands that are important when decisions are made on specific systems at a low level of the organization. They should also reflect the more general demands (support decision making, strengthen market position, contribute to the organizations strategy) that play a role at higher levels of decision making. From the introduction of this paper it will be clear that none of the existing approaches to quality pays attention to this kind of consideration. It is however possible to make use of the literature on information planning and on strategic use of information systems and incorporate this in a conceptual model for the quality of information in an organization. Constructing the quality model
From the discussion on the concepts of ‘quality’ and ‘information in an organization’ we now have the elements to construct a conceptual framework for the quality of information in an organization. Our starting point is that from the definition of quality we can look at the quality of information from a causal and from a teleological point of view. From the causal point of view the quality of information is seen as the result of the quality of the process in which it is produced. The first step in this process is information analysis. In this step the link between the organization’s needs and the information systems is established. First the information policy is formulated and then the more detailed information needs are derived. The next step is that of designing and building information systems. The IS0 9000-3 quality standard is related to this step of the process. This standard describes which measures have to be taken in order to deliver information systems of good quality. Finally the systems produced in the designing and building steps are used in the data processing step to produce the information the organization needs. Several authors have elaborated on the quality features and characteristics of the production process that influence the quality of the final product. Delen and Rijsenbrij (1990) describe quality features of both the design and construction steps, the data processing process and the information originating from these processes. The essence of the causal point of view in ex post quality assessments is that not Journal of Strategic Information Systems 1994 Volume 3 Number 3
183
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
all aspects of the quality of information can be measured from that information itself. For some features it is necessary to look at one or more of the steps of the production process. It is, for example, very hard to measure directly the reliability of information. It can however be assessed by looking at the level of reliability measures taken in systems development and in data processing. In order to get a complete picture of the quality of information in an organization it is therefore necessary to look at the steps of information analysis, designing and building the systems, and of data processing. From the teleological point of view the quality of information is seen as the degree to which it satisfies ‘stated or implicit needs’, derived from the situation in which it is used. This expression, used in the IS0 definition of the quality concept, is however still very general. We can make it more specific by stating that the quality of information is the degree to which it supports the goals and targets of the organization in which it is used. These goals and targets can be divided into a number of categories: l
l
l
l
The organizational goals. Almost every organization is characterized by the fact that its members come together to realize some kind of common goal. This common or organizational goal reflects the expectations, ambitions and/or aspirations of those who depend on the organization. On the level of the organization as a whole, organizational goals have to be translated into strategies that describe how these goals can be reached. Strategies arise explicitly or implicitly in an interaction between structure, culture and goals of the organization. Traditionally we suppose that information has to support the organization’s strategies. Recently we see, however, that information systems can also be used to shape as well as support organizational strategies, and that they make it possible to aim for new goals. The business process targets. The existing division of labour in the organization is the basis for translating organizational goals and strategies into targets for each business process, department and individual within the organization. The degree of detail in which these targets have to be described when studying the quality of information depends on the organizational level that is chosen as a starting point for the analysis. Some organizations have explicit mechanisms for tuning organizational goals and business process targets for different processes and hierarchical levels, others do not. In some organizations there is no strictly hierarchical relationship between goals and targets at all levels. The personal interests. Each individual in the organization has, besides the business process targets, their own individual interests. Status, power, responsibility, prestige and money are well-known examples of personal targets. Background, experience and knowledge can influence the targets of the individual. Personal interests do not have to compete with organizational goals and/or business process targets. In the ideal situation they even complement each other. In other cases higher order goals and targets leave room for striving after personal goals. The information needs of a person in a certain function in an organization are influenced by both business process targets and personal interests. A judgement of the quality of information available for such an individual has to take both elements into account. The users’ targets and the providers’ targets. Goals and targets cannot only be subdivided according to levels in the organization but also into targets of those who are using information and targets of those who are providing others with information. A difference in position may lead to differences of opinion on the
184
Journal of Strategic Information Systems 1994 Volume 3 Number3
Measuring the strategic dimensions of the quality of information:
quality features and characteristics
of the information
G J van der Pijl
received or provided.
Judging the teleological aspects of the quality of information in an organization means that one has to assess how much the information systems in the organization contribute to each of the goals and targets described above. Of course it is also possible to take only a subset of goals and targets into consideration. If we look at individual systems at the level of user or provider targets we will have to study in detail which quality features and characteristics are determining the contribution of systems to reaching the targets, and how well the systems does for each of these. If we look at the configuration of systems available for the organization as a whole we take a much more global view. In that case we ask ourselves which functional contribution the systems make to the goals and targets of the organization without specifying detailed quality characteristics. Thus the detailed view of quality is replaced by a more global view in which quality of information in the organization is understood as the degree of fit between the goals and targets of the organization and the information systems supporting those. Since the organizational goals are mostly derived from the environment in which an organization operates, the influence of the environment on the need for information systems should also be taken into account in a teleological assessment of the quality of information. Likewise the strong connection between organizational goals and targets and the structure and culture of the organization make it worthwhile to include cultural and structural aspects in the teleological evaluation. Bringing together the elements of the causal point of view and the elements of the teleological point of view, as in Figure I, we arrive at a conceptual framework for studying the quality of information in an organization. At the bottom of the figure we see the steps of the process that has to be studied in the causal approach. On the upper right of the figure the set of goals and targets that has to be considered in the teleological approach has been presented. The vertical lines indicate the correspondence between the different levels of goals and targets and the hierarchical levels of the organization depicted on the left.
/organisationl_____-----____-------_
\
I
systems design informadesign build Ft$‘- -_ lion needs Figure 1
The quality
dataprocessing colht record process _-_
/
information
model
Journal of Strategic Information Systems I994 Volume 3 Number3
185
Measuring the strategic dimensions of the quality of information:
G .I van der Pijl
An approach to measuring the strategic aspects of quality Recently much has been written on the causal approach to quality of information, often connected to the IS0 9000 set of quality norms. Much has also been written on approaches for measuring the teleological aspects of quality with respect to the lower levels of targets depicted in Figure I. Hardly anything however has been written on making ex post judgements of the overall configuration of information systems in an organization from the point of view of the organization goals and the higher level business process targets. Therefore, starting from the above model, an approach was designed for judging the teleological, i.e. strategy- and goal-related, aspects of quality. In this approach the organization’s environment, its goals and targets, its structure and its culture are described first. From those the need for information and the types of information systems required are deduced. Finally the quality assessment can be made by comparing the required and the available information and the required and the actually installed information systems. We briefly comment upon each of the elements of the approach. The approach was tested by applying it to a division of the Postal Services of the Dutch PTT (PTT-Post) and to the Association of Netherlands Municipalities (ANM). Environment An analysis of the opportunities offered and the threats posed by the environment, makes it possible to draw conclusions with respect to the information needs of the organization. Porter (1980) offers a framework for making such an analysis for industrial organizations. Parts of the framework are also useful for analysing other types of organizations, such as service organizations. Comparing the information needs derived from the analysis of the environment with the actual situation gives the first part of the evaluation of the quality of information. Postal Services in the Netherlands have been changing from a government organization to a private, market-oriented company. This means that information on market opportunities and trends in the environment become of critical importance. Yet very few systems providing market and/or environmental information were in place when we visited the organization. Thus information quality was poor in this respect. The
Another element of the analysis of the environment is to establish its diversity. The reason for this is that different parts of an organization will probably deal with different parts of the environment. Lawrence and Lorsch (1973) demonstrated that departments of an organization tune their behaviour to those parts of the environment with which they are communicating most intensively. Therefore, the bigger the differences between partial environments are, the bigger the differences between the departments dealing with them and the less likely it will be to find a strictly hierarchical and unequivocal structure of goals and targets in the organization. Thus if we find that the environment of an organization is very diverse, we won’t be surprised to find a diversity of goals and targets for different levels and groups in the organization in the next steps of the analyses. Some departments of the ANM are involved in lobbying the central government on behalf of the municipalities, others provide commercial services (central purchase, organization advice) to municipalities. Thus there is a very wide variation in environments in which departments operate. This is reflected in the variation in needs for information 186
systems.
They vary from the need for information
on forthcoming
laws
Journal of Strategic Information Systems 1994 Volume 3 Number 3
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
to cost/price information for the services to be offered to the municipalities. Different parties in the organization have clearly different views on the importance of financial information for their activities.
The goals of the organization
the contribution of information systems to the goals and targets of the organization is at the heart of the teleological approach to quality of information. To do so we have to start describing the organizational goals. From the literature on organizational goals it is clear that it is impossible to formulate a normative set of goals that each organization has to pursue. Therefore one has to find out the specific set of goals of a specific organization by analysing its written statements on this topic and by interviewing the relevant stakeholders. The literature gives, however, some idea of what type of goals to look for. Ansoff (1965) shows that both long- and short-term goals are relevant. Rhenman (1973) points out the relevance of both internal and external goals: for some organizations the goals will mainly be set by external parties. Bahlman and Meesters (1988) demonstrate that variety in goals and targets (that may arise when parts of the organization have diverse tasks, products/services and customers) can improve the chances of survival for organizations. Simon (1960) argues that maximization of goals is hardly ever realistic. This means that we have to look for a set of goals from different parties which can be satisfied instead of optimized at the same time. Organizational goals will be translated into strategies or strategic actions. From these we try to derive the demands on information systems. For drawing conclusions with respect to the information needs from strategies directed at economic goals, we find a number of instruments in the literature. Useful approaches are those of Porter (1985) and Wiseman (1985). For other strategies no instruments have been found. In those cases it is recommended to make the strategies as concrete and explicit as possible. Common sense and knowledge of the field can be used to ascertain where strategies can be supported by information systems. Once the demands on information systems that arise directly from the organizational goals and strategies are specified we can compare them with the actual situation. The result of this comparison is the next part of the quality assessment. Within organizations there always will be several levels of goals, ranging from very broad mission statements to concrete targets for business processes or parts thereof. In exceptional situations it is possible that goals formulated on different levels and by different persons fit together nicely, but generally there will be discrepancies. In the context of judging the quality of information in an organization these differences have to be made explicit and translated into consequences for the quality judgement. Assessing
Organizational goals could be traced easily in both organizations in which we tested the instrument. F’TT-post clearly changed from a government-like service orientation to more commercial goals. The ANM’s goals were formulated in terms of the products the organization offers the municipality: lobbying, advice and commercial services.
The process targets
In order to reach its goals an organization will arrange its activities as business processes. Davenport (1993) defines a business process as a structured, measured set of activities designed to produce a specified output for a particular customer or Journal of Strategic Information Systems I994 Volume 3 Number 3
187
Measuring the strategic dimensions of Ihe quality of information:
G J van der Pijl
market. The organizational goals are, more or less accurately, translated into targets for the business processes. Information systems can be used to reach these process targets. In order to assess the quality of information we have to consider whether the organization has made use of the opportunities offered by information systems when structuring these processes. The method used in the analysis is an adapted form of the method proposed by Bedell (1985). It starts out describing the business processes using Porter’s Value Chain or some other descriptive tool. Next the information systems are described. For each system we describe which (parts of the) business process it supports, who provides and who uses the information, a global indication of the users’ satisfaction with the system and the way in which the system contributes to the goals and targets. For the latter classification we use the categories: l l l l
automating parts of the business process; using information for managing the business; incorporating IT in the product; using the communications possibilities of the IT infrastructure.
Based on these descriptions the method gives an impression of the degree to which the important (parts of) business processes are supported by information systems. These partial conclusions can be summarized to a final global judgement if the assessment is done using detailed quantitative scales as a basis for the interviews. Of course the quality judgement has to be completed by looking at the performance of the existing systems. Systems supporting important parts of the business processes that perform poorly diminish the quality of information. On the other hand systems that support unimportant parts of the business processes but perform excellently may be a waste of money. The shift to private enterprise generated a decentralization in the hierarchical structure of PTT-post. Decision making was delegated to lower levels. The structure of the systems for management information did not yet reflect this shift. Most information went to the top of the organization without supporting the decisions of lower level management. Thus it was concluded that the quality of information was insufficient in this respect. In the ANM the lobbying activities were traditionally seen as the most important activities of the organization. The structure of the information systems was mainly designed for supporting these activities. This made support for other activities much less efficient. In recent years a growing number of decision makers in the organization have come to believe these other activities to be as important as the lobbying activities. Hence the advice was to change the structure of the information systems so as to reflect this new view on the relative importance of the organization’s activities. This meant for example that a good financial system had to be made available to the commercial activities.
Organizational structure
The structure of the organization is an important determinant of the amount of communication that is needed to co-ordinate its activities. Thus, when one looks at factors influencing information needs, the structure of the organization has to be included in the analysis. It would appear that the only author hypothesizing a strict relationship between organizational structure and the information needs of the organization is Leifer (1988). He uses a typology of organizational structure based on Mintzberg (1983). Leifer’s ideas, however, have not been tested in practice. 188
Journal of Srrategic Information Systems 1994 Volume 3 Number 3
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
In this study the structure is described in terms of its hierarchy and in terms of Mintzberg’s typology. From these, conclusions about the need for information are drawn using common sense and the ideas of Leifer and Mintzberg. Comparing these needs with the actual situation gives the structure related part of the quality judgement . The largest part of the ANM could be described as a professional organization. In Leifer’s terms this should mean that the information infrastructure leaves ample room for the individual needs of the professionals and supports their structured and unstructured communication. In both respects ANM’s infrastructure was not very effective. Since decisions on information systems were highly centralized, they tended to support big central systems at the cost of small local systems for the professionals. Also the electronic mail facility offered was rather poor. Therefore the quality of information was considered to be poor in these respects. Organizational culture seems reasonable to hypothesize that there is a relationship between organizational culture and the role of information in the organization. The literature, however, does not describe any explicit relationship. Therefore an attempt was made, following Schein (1985)) to describe different levels of culture in the PIT-post organization. As such an approach takes too much time in the context of our research, this was not successful. Therefore, in the case of the ANM, the approach was to describe culture by determining the position of the organization and its parts on a set of scales representing different aspects of culture. The scales are borrowed from Hofstede et al. (1988). It turned out, however, that respondents hardly recognized the items on the scales. Thus no satisfactory approach to culture has been found thus far. We intend to return to this matter in future research projects. It
Conclusions The study described in this paper offers a framework that encompasses relevant aspects of quality of information in an organization. Starting from the framework, an approach has been developed to support management in judging the quality of information. This ‘teleological’ or ‘strategic’ approach links organizational goals and targets with information (systems) requirements. The empirical results presented above indicate the applicability of the approach in different practical situations. Descriptions of the organization’s environment, goals, business-process targets and structure led, in a very natural way, to requirements that the information systems of the organizations have to meet. The actual situation could be offset against these requirements. Discrepancies between the two give rise to negative quality judgements. Correspondence between requirements and reality leads to positive quality judgements. Only the analyses of culture based on Schein’s levels of culture or Hofstede’s dimensions proved to be impracticable. Respondents could not handle the description of the culture dimensions very well and gave them their own, varying interpretations. Besides, they had their own vision on and terminology for cultural differences between parts of the organization. Although there was no strictly hierarchical structure of goals, strategies and targets, it proved possible to pinpoint the important information systems and the important gaps in the information structure. This led to clear cut ex post conclusions on the quality of information and information systems in these particular organizations. Journal of Strategic Information Systems 1994 Volume 3 Number 3
189
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
References Ansoff, H (1965) Corporate Strategy, an Analytic Approach to Business Policy for Growth and Expansion McGraw-Hill, New York Bahlman, J P and Meesters, B A C (1988) Denken & doen, een studie naar de ontwikkeling en strategische herorientatie van zes Nederlandse ondernemingen KUB, Tilburg, Netherlands Bailey, J E and Pearson, S W (1983) ‘Development of a tool for measuring and analyzing computer user satisfaction’ Manage. Sci. 29, 4, pp 395-398 Bedell, E F (1985) The Computer Solution; Strategies for Success in the Information Age Dow Jones-Irwin, Homewood, IL Boehm, B W, Brown, J R, Karpar, H, Lipow, M, McLeod, G J and Merritt, M J (1978) Characteristics of Software Quality North Holland, Amsterdam Delen, G P A J and Rijsenbrij, D B B (1990) ‘Kwaliteitsattributen van automatiseringsprojecten en informatiesystemen’ Informatie 32, 1, pp 41-55 Deming, W E (1982) Quality, Productivity and Competitive Position, MIT Centre for Advanced Engineering Study, Cambridge, MA Davenport, T H (1993) Process Innovation; Reengineering Work Through Information Technology Harvard Business School Press, Boston, MA Egten, C A van and Pijl, G J van der (1992) Kwaliteit van Informatie Limperg Instituut, Amsterdam Hammer, M (1990) ‘Reengineering work: don’t automate, obliterate’ Harvard Business Rev. July-August, pp 104-112 Hofstadter, D R (1979) Godel, Escher, Bach: An Eternal Golden Braid Basic Books, New York Hofstede, G et al. (1988) Organizational Cultures - Beyond the Fad; A Qualitative/Quantitative Study Across Twenty Cases (in manuscript) RUL Inose, H (1990) ‘Informatietechnologie is technologie van de vrijheid’ N R C Handelsblad 4 December ISOto 9004 International Standards, International Organization for Standardization. Geneva Juran, J M (1974) Quality Control Handbook McGraw-Hill, New York Keen, P G W (1988) Competing in Time; Using Telecommunications for Competitive Advantage Ballinger Publishing Company, Cambridge, MA Lawrence, P R and Lorsch, J W (1973) Organization and Environment; Managing Differentiation and Integration Irwin, Homewood, IL Leifer, R L (1988) ‘Matching computer-based information systems with organizational structures’ MIS Quarterly March, pp 63-73 McCall, J A (1979) ‘An introduction to software quality metrics’ in Cooper, J B and Fisher, M J (eds) Software Quality Management PBI, New York McFarlan (1984) ‘Information technology changes the way you compete’ Harvard Business Rev. May- June, pp 98- 103 Mintzberg, H (1983) Structure in Fives; Designing Effective Organisations Prentice-Hall, Englewood Cliffs, NJ Nolan, R L, Pollock, A J and Ware, J P (1989) ‘Toward the design of network organizations’ Stage by Stage 9, 1, pp 1-12 Parker, M M and Benson, R J (1988) Information Economics; Linking Business Performance to Information Technology Prentice-Hall, Englewood Cliffs, NJ Pijl, G J van der (1993) Kwaliteit van informatie in theorie en praktijk, PhD thesis, Limperg Instituut, Amsterdam Pirsig, R M (1974) Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values William Morrow, New York Porter, M E (1980) Competitive Strategy: Techniques for Analyzing Industries and Competitors The Free Press, New York Porter, M E (1985) Competitive Advantage: Creating and Sustaining Superior Performance The Free Press, New York Rhenman, E (1973) Organization Theory for Long Range Planning Wiley, New York Schein, E H (1985) Organizational Culture and Leadership Jossey-Bass, San Francisco, CA Simon, H A (1960) The New Science of Management Decision Harper and Row, New York Smits, M T (1990) The Quality of Information Systems, an Inquiry into Complexity, Working paper series Washington University, St Louis, MO Starreveld, R W, Mare, H B de and Jo&B, E J (1986) Bestuurlijke Informatieverzorging. Algemene Grondslagen Samsom, Alphen a.d. Rijn H (1985) Systematische Softwarequalitiitssicherung an Hand von Qualitats und Willmer, Produktmodellen Springer Verlag, Berlin Wiseman, C (1985) Strategy and Computers: Information Systems as Competitive Weapons Dow Jones-Irwin, Homewood, IL
190
Journal of Strategic Information Systems 1994 Volume 3 Number 3
Systems 1994 3 (3) 179- 190
Measuring the strategic dimensions of the quality of information G J van der Pijl Tilburg
University,
5000 LE
Tilburg,
Department
of Information
Systems,
P 0 Box 90153,
The Netherlands
Although much has been written on quality of information there is no uniform framework in which different approaches to quality can be brought together. In this article such a framework is presented. From the framework an approach for assessing the strategy and goal related elements of quality of information in an organization is derived. Some examples of the first applications of this approach in practical situations are included. Keywords:
quality
of information,
information
systems
The more dependent on information organizations become, the more important it is to ensure that the information is of good quality. Hence, organizations should, from time to time, assess the quality of the information they use. This means that there is a need for a measuring instrument. When the research on which this article is based was started, the intention was to construct such a measuring instrument using the quality concepts that could be found in literature. This instrument should enable a company’s management to get a well-founded judgement of the quality of the information they are provided with, and thus of the quality of the information systems installed. The instrument should, in other words, support an ex post quality assessment. A first survey however proved that, although a lot has been written on concepts of quality of information, the literature offers no uniform or consistent framework to approach the subject. In the late 1970s well-known authors with a software engineering background, like Boehm (1978) and McCall (1979), tried to describe the quality of software in terms of series of quality attributes or quality trees. Different sets of quality attributes were proposed and attributes were described in terms of more and more detailed attributes until the level of measurable properties of the software was This paper was one of a small number of papers presented at the SISnet Conference held at Tilburg University, The Netherlands in June 1993 recommended for publication in JSIS by members of the SISnet Conference Programme Committee, namely Professors Pieter Ribbers (chair), Joachim Griese and Helmut Krcmar. Revised paper accepted for publication by Professor R D Galliers, January 1994 The quality model presented in this paper was developed in cooperation with Mrs C A van Egten of the Free University, Amsterdam (1992). The approach to assessing the teleological elements of the quality of information is described more elaborately in Pijl (1993).
0963-8687/94/030179-
12 0
1994 Butterworth-Heinemann
Ltd
179
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
reached. The purpose of this effort was twofold. On the one hand the aim was to formulate a set of quality attributes as the basis for quality specifications in software contracts. On the other hand specification of a set of quality properties should simplify the quality assurance process of software development. More recent European contributions in this field have been presented by Wilmer (1985) and Delen and Rijsenbrij (1990). Another approach to quality can be found in the literature on financial auditing and on electronic data processing (edp) auditing. Here reliability, continuity and exclusiveness are seen as the most important quality attributes. Only in recent years has attention also been paid to effectiveness and efficiency. A classic Dutch publication in this field is by Starreveld et al. (1986). In the 1980s the quality literature concentrates on measuring user satisfaction. Bailey and Pearson’s (1983) article is a well-known representative of this period. Their method uses insights into quality attributes developed earlier. Their questionnaire combines these attributes with questions on the functioning of the systems development and data-processing departments. At about the same time we see publications on success and failure factors for information systems, which also can be seen as an approach to quality of information systems. In recent years quality issues have again been widely discussed after the presentation of the IS0 9000 (1987) norms for quality assurance. The orientation here is mainly on the quality attributes of the software development process, the idea being that improving the process will also improve the quality of the software product. The supposed relations between quality aspects of the process and quality aspects of the products, however, are not supported by published empirical facts. Last but not least the literature on information planning and on capital-budgetting aspects of investment in information systems (e.g. McFarlan, 1984; Bedell, 1985; Parker and Benson, 1988; Hammer, 1990) has to be mentioned here. Although these publications are seldom referred to in discussions on quality of information, they are very relevant: the quality of information should be judged by its contribution to the goals of the organization. Assessing the relationships between the organization and its information systems is exactly what these publications are about. From the foregoing analysis two conclusions can be drawn. First, the literature offers no uniform approach to quality of information. Second, a connection to the literature on infornformation planning is lacking. As stated before, the goal of this study was to develop an instrument to measure the quality of information. For this purpose a model of quality of information had to be developed. This model should capture relevant aspects of quality, including those related to the organizational goals. The quality model In order to derive a conceptual model for the quality of information in an organization we start with a discussion of the concepts of ‘quality’ and ‘information in organizations’. From there we try to derive the elements of the conceptual model. Quality
The first definitions of the term ‘quality’ come from the ancient Greek philosophers. In more recent literature we find studies on the general notion of 180
Journal of Strategic Information Systems 1994 Volume3
Number3
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
quality in the work of Pirsig (1974) and Hofstadter (1979). Traditionally the term quality is used in relation to products and people, in more recent definitions, like for example the one from IS0 8402 it becomes clear that it can also be related to services and processes. The IS0 8402 definition for quality is the totality of features and characteristics of a product, process or service that bear on its ability to satisfy stated or implicit needs. The concept of quality can be related to processes in two different ways: l
l
First, products and services to which quality is related are produced in a process. The features and characteristics that determine the quality of products and services, are created in that process. Hence, control of the quality of those products and services can be exercised by controlling the production process. Authors like Deming (1982) and Juran (1974) argue that all steps of the production process have to be thoroughly controlled in order to create products with the desired quality features. Second, quality features and characteristics, can be determined for processes and thus also for production processes. Of course the quality features of production processes should be chosen in such a way that they guarantee the quality of the final products.
Any conceptual quality model should take account of the importance of the production process for the quality of goods and services and for the possibility of determining quality features for the production process itself. An important point on which almost all definitions of quality agree is the fact that quality of a product or service always has to be considered in the light of the use that is made of it. This is clear from the IS0 8402 definition but, for example, also from Juran who states that quality is ‘fitness for use’ or in other words: ‘the extent to which the product successfully serves the purpose of the user during usage’. From this last definition it is also clear that in Juran’s vision the user is the one who decides what are the features and characteristics that are important for the use of the product. The IS0 definition of quality takes into the account that there is not always a situation in which the needs of a future user are described in a contract. Therefore it states that quality can also be related to ‘implied needs’. This supposes that in certain cases or for certain features a more objective description of quality, which is not associated with a particular user, is possible. The definition, however, does not say how these ‘implied needs’ can be determined. From the above we conclude that a conceptual model should relate quality features and characteristics of information to the situation in which it is used. As will be described later on, the goals and targets of organizations will be used as descriptors of this situation of use. Another important feature of most quality definitions is that quality is described in terms of a series of specific features and characteristics of a product, a service or a process. The user presents a set of features and characteristics that describe his demands. The producer will try to translate the demand-oriented set into a set of features and characteristics he can make. In literature the features and characteristics specified by users are associated with the ‘fitness for use’ or product oriented approach to quality while the set specified by the producers is associated with the ‘conformity to specifications’ or production process oriented aspects of quality. In both approaches quality can be decomposed in sets of features and characteristics. This is of course necessary when one wants to control for quality in Journal of Strategic Information Systems 1994 Volume 3 Number 3
181
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
production processes. Such processes cannot be controlled if quality is not specified. Specifying can be done by translating quality in demand specifications and next in product specifications. The production process can then be controlled in such a way that the product conforms to the product specifications and thus to the quality demanded. This way of quality control is also followed in the design and construction of information systems. Often, however, information systems that conform to all carefully selected quality features and characteristics don’t deliver, in the view of the user, the quality demanded. This raises the question whether the ‘reductionist’ approach to quality might be wrong. An alternative would be to look for a holistic approach like in Zen and the Art of Motorcycle Maintenance. Pirsig (1974) supposes that the term quality relates to objects as indivisible entities. The quality-experience is directly coupled to the objects observed and cannot be isolated. In other words: the quality of a product is more than the sum of its parts. The problem with such an approach is that the idea of quality cannot help controlling the production of goods and services. As Smits (1990) showed, Hofstadter’s approach indicates a possible way out of this dilemma. Hofstadter (1979) says that the difference between an object and the sum of its parts is caused by the complexity of the object observed. This complexity makes it very difficult to see the relations between all parts of the object observed. This does not mean that in reality there are no logical and systematic relations. People can only communicate about such complex systems if they have a common conceptual framework. In the context of such a framework opinions and experiences can be exchanged using a limited number of concepts that are clear to everybody. The lack of such a conceptual framework may well be one of the reasons why systems development, in spite of very conscious analysis of information needs, often does not deliver the system the user is looking for. The conclusion should not be that the reductionist concept of quality is wrong. It should be that we obviously do not yet have a set of common concepts that enable us clearly to describe the desired quality of an information system. Developing a conceptual framework, in which statements on the quality of information systems can be made, can be seen as a contribution to the construction of such a common set of concepts. Such a framework should, in view of the discussion above, allow a very detailed approach to quality as well as an approach in more global qualitative terms. Information
in organizations
Looking at the use of information systems in organizations we see two main trends. First, the amount of information produced in organizations has been growing tremendously. Several general studies (like Inose, 1990) show that the amount of information used grows less than the amount of information offered. It is not known whether this is also true for the use of information in organizations. There are however several reasons why the use of information in organizations should probably grow. One reason is that the growing size and complexity of organizations makes it impossible to control them only on the basis of one’s own observations. Another important reason is the growing complexity and dynamism of the environment in which most organizations operate. Second, the way in which organizations use information systems is changing. Initially they were mainly used to automate specific, labour intensive types of information processing such as financial administration. Later on attention shifted to the information needed to control business processes and organizations as a whole. The next step was not only to use information systems for internal purposes 182
Journal of Strategic Information Systems 1994 Volume 3 Number3
Measuring
the strategic dimensions of the quality of information:
G J van der Pijl
but also for reaching a good competitive position (Porter, 1980, 1985; Wiseman, 1985; Parker and Benson, 1988). The most recent development is to use information systems for new ways of organizing (see e.g. Nolan et al., 1989). All of these changes in the use of information systems also cause changes in the way in which decisions on information systems are (should be) made. In the time of specific labour intensive systems decisions could often be made for each system separately. When the attention shifted to management information systems focused on control, decisions to be made often involved an integrated set of systems. Because of the later shift to strategic use of information systems decisions were taken against the background of the organization’s strategy. Since market strategies are mostly decided on at the level of the lines of business, the decisions on information systems are also taken at this level. Decisions on information infrastructure enabling the redesign of organizations are also taken at a fairly high level in the organization. The same goes for decisions on specific systems that are considered vital for the organization as a whole (Keen, 1988). The changing ways in which information systems are used within organizations and in which decisions on information systems are taken have to be incorporated into our concept of quality of information. This means that the set of quality features and characteristics that together make up the quality of an information system should not only reflect the very specific and detailed demands that are important when decisions are made on specific systems at a low level of the organization. They should also reflect the more general demands (support decision making, strengthen market position, contribute to the organizations strategy) that play a role at higher levels of decision making. From the introduction of this paper it will be clear that none of the existing approaches to quality pays attention to this kind of consideration. It is however possible to make use of the literature on information planning and on strategic use of information systems and incorporate this in a conceptual model for the quality of information in an organization. Constructing the quality model
From the discussion on the concepts of ‘quality’ and ‘information in an organization’ we now have the elements to construct a conceptual framework for the quality of information in an organization. Our starting point is that from the definition of quality we can look at the quality of information from a causal and from a teleological point of view. From the causal point of view the quality of information is seen as the result of the quality of the process in which it is produced. The first step in this process is information analysis. In this step the link between the organization’s needs and the information systems is established. First the information policy is formulated and then the more detailed information needs are derived. The next step is that of designing and building information systems. The IS0 9000-3 quality standard is related to this step of the process. This standard describes which measures have to be taken in order to deliver information systems of good quality. Finally the systems produced in the designing and building steps are used in the data processing step to produce the information the organization needs. Several authors have elaborated on the quality features and characteristics of the production process that influence the quality of the final product. Delen and Rijsenbrij (1990) describe quality features of both the design and construction steps, the data processing process and the information originating from these processes. The essence of the causal point of view in ex post quality assessments is that not Journal of Strategic Information Systems 1994 Volume 3 Number 3
183
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
all aspects of the quality of information can be measured from that information itself. For some features it is necessary to look at one or more of the steps of the production process. It is, for example, very hard to measure directly the reliability of information. It can however be assessed by looking at the level of reliability measures taken in systems development and in data processing. In order to get a complete picture of the quality of information in an organization it is therefore necessary to look at the steps of information analysis, designing and building the systems, and of data processing. From the teleological point of view the quality of information is seen as the degree to which it satisfies ‘stated or implicit needs’, derived from the situation in which it is used. This expression, used in the IS0 definition of the quality concept, is however still very general. We can make it more specific by stating that the quality of information is the degree to which it supports the goals and targets of the organization in which it is used. These goals and targets can be divided into a number of categories: l
l
l
l
The organizational goals. Almost every organization is characterized by the fact that its members come together to realize some kind of common goal. This common or organizational goal reflects the expectations, ambitions and/or aspirations of those who depend on the organization. On the level of the organization as a whole, organizational goals have to be translated into strategies that describe how these goals can be reached. Strategies arise explicitly or implicitly in an interaction between structure, culture and goals of the organization. Traditionally we suppose that information has to support the organization’s strategies. Recently we see, however, that information systems can also be used to shape as well as support organizational strategies, and that they make it possible to aim for new goals. The business process targets. The existing division of labour in the organization is the basis for translating organizational goals and strategies into targets for each business process, department and individual within the organization. The degree of detail in which these targets have to be described when studying the quality of information depends on the organizational level that is chosen as a starting point for the analysis. Some organizations have explicit mechanisms for tuning organizational goals and business process targets for different processes and hierarchical levels, others do not. In some organizations there is no strictly hierarchical relationship between goals and targets at all levels. The personal interests. Each individual in the organization has, besides the business process targets, their own individual interests. Status, power, responsibility, prestige and money are well-known examples of personal targets. Background, experience and knowledge can influence the targets of the individual. Personal interests do not have to compete with organizational goals and/or business process targets. In the ideal situation they even complement each other. In other cases higher order goals and targets leave room for striving after personal goals. The information needs of a person in a certain function in an organization are influenced by both business process targets and personal interests. A judgement of the quality of information available for such an individual has to take both elements into account. The users’ targets and the providers’ targets. Goals and targets cannot only be subdivided according to levels in the organization but also into targets of those who are using information and targets of those who are providing others with information. A difference in position may lead to differences of opinion on the
184
Journal of Strategic Information Systems 1994 Volume 3 Number3
Measuring the strategic dimensions of the quality of information:
quality features and characteristics
of the information
G J van der Pijl
received or provided.
Judging the teleological aspects of the quality of information in an organization means that one has to assess how much the information systems in the organization contribute to each of the goals and targets described above. Of course it is also possible to take only a subset of goals and targets into consideration. If we look at individual systems at the level of user or provider targets we will have to study in detail which quality features and characteristics are determining the contribution of systems to reaching the targets, and how well the systems does for each of these. If we look at the configuration of systems available for the organization as a whole we take a much more global view. In that case we ask ourselves which functional contribution the systems make to the goals and targets of the organization without specifying detailed quality characteristics. Thus the detailed view of quality is replaced by a more global view in which quality of information in the organization is understood as the degree of fit between the goals and targets of the organization and the information systems supporting those. Since the organizational goals are mostly derived from the environment in which an organization operates, the influence of the environment on the need for information systems should also be taken into account in a teleological assessment of the quality of information. Likewise the strong connection between organizational goals and targets and the structure and culture of the organization make it worthwhile to include cultural and structural aspects in the teleological evaluation. Bringing together the elements of the causal point of view and the elements of the teleological point of view, as in Figure I, we arrive at a conceptual framework for studying the quality of information in an organization. At the bottom of the figure we see the steps of the process that has to be studied in the causal approach. On the upper right of the figure the set of goals and targets that has to be considered in the teleological approach has been presented. The vertical lines indicate the correspondence between the different levels of goals and targets and the hierarchical levels of the organization depicted on the left.
/organisationl_____-----____-------_
\
I
systems design informadesign build Ft$‘- -_ lion needs Figure 1
The quality
dataprocessing colht record process _-_
/
information
model
Journal of Strategic Information Systems I994 Volume 3 Number3
185
Measuring the strategic dimensions of the quality of information:
G .I van der Pijl
An approach to measuring the strategic aspects of quality Recently much has been written on the causal approach to quality of information, often connected to the IS0 9000 set of quality norms. Much has also been written on approaches for measuring the teleological aspects of quality with respect to the lower levels of targets depicted in Figure I. Hardly anything however has been written on making ex post judgements of the overall configuration of information systems in an organization from the point of view of the organization goals and the higher level business process targets. Therefore, starting from the above model, an approach was designed for judging the teleological, i.e. strategy- and goal-related, aspects of quality. In this approach the organization’s environment, its goals and targets, its structure and its culture are described first. From those the need for information and the types of information systems required are deduced. Finally the quality assessment can be made by comparing the required and the available information and the required and the actually installed information systems. We briefly comment upon each of the elements of the approach. The approach was tested by applying it to a division of the Postal Services of the Dutch PTT (PTT-Post) and to the Association of Netherlands Municipalities (ANM). Environment An analysis of the opportunities offered and the threats posed by the environment, makes it possible to draw conclusions with respect to the information needs of the organization. Porter (1980) offers a framework for making such an analysis for industrial organizations. Parts of the framework are also useful for analysing other types of organizations, such as service organizations. Comparing the information needs derived from the analysis of the environment with the actual situation gives the first part of the evaluation of the quality of information. Postal Services in the Netherlands have been changing from a government organization to a private, market-oriented company. This means that information on market opportunities and trends in the environment become of critical importance. Yet very few systems providing market and/or environmental information were in place when we visited the organization. Thus information quality was poor in this respect. The
Another element of the analysis of the environment is to establish its diversity. The reason for this is that different parts of an organization will probably deal with different parts of the environment. Lawrence and Lorsch (1973) demonstrated that departments of an organization tune their behaviour to those parts of the environment with which they are communicating most intensively. Therefore, the bigger the differences between partial environments are, the bigger the differences between the departments dealing with them and the less likely it will be to find a strictly hierarchical and unequivocal structure of goals and targets in the organization. Thus if we find that the environment of an organization is very diverse, we won’t be surprised to find a diversity of goals and targets for different levels and groups in the organization in the next steps of the analyses. Some departments of the ANM are involved in lobbying the central government on behalf of the municipalities, others provide commercial services (central purchase, organization advice) to municipalities. Thus there is a very wide variation in environments in which departments operate. This is reflected in the variation in needs for information 186
systems.
They vary from the need for information
on forthcoming
laws
Journal of Strategic Information Systems 1994 Volume 3 Number 3
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
to cost/price information for the services to be offered to the municipalities. Different parties in the organization have clearly different views on the importance of financial information for their activities.
The goals of the organization
the contribution of information systems to the goals and targets of the organization is at the heart of the teleological approach to quality of information. To do so we have to start describing the organizational goals. From the literature on organizational goals it is clear that it is impossible to formulate a normative set of goals that each organization has to pursue. Therefore one has to find out the specific set of goals of a specific organization by analysing its written statements on this topic and by interviewing the relevant stakeholders. The literature gives, however, some idea of what type of goals to look for. Ansoff (1965) shows that both long- and short-term goals are relevant. Rhenman (1973) points out the relevance of both internal and external goals: for some organizations the goals will mainly be set by external parties. Bahlman and Meesters (1988) demonstrate that variety in goals and targets (that may arise when parts of the organization have diverse tasks, products/services and customers) can improve the chances of survival for organizations. Simon (1960) argues that maximization of goals is hardly ever realistic. This means that we have to look for a set of goals from different parties which can be satisfied instead of optimized at the same time. Organizational goals will be translated into strategies or strategic actions. From these we try to derive the demands on information systems. For drawing conclusions with respect to the information needs from strategies directed at economic goals, we find a number of instruments in the literature. Useful approaches are those of Porter (1985) and Wiseman (1985). For other strategies no instruments have been found. In those cases it is recommended to make the strategies as concrete and explicit as possible. Common sense and knowledge of the field can be used to ascertain where strategies can be supported by information systems. Once the demands on information systems that arise directly from the organizational goals and strategies are specified we can compare them with the actual situation. The result of this comparison is the next part of the quality assessment. Within organizations there always will be several levels of goals, ranging from very broad mission statements to concrete targets for business processes or parts thereof. In exceptional situations it is possible that goals formulated on different levels and by different persons fit together nicely, but generally there will be discrepancies. In the context of judging the quality of information in an organization these differences have to be made explicit and translated into consequences for the quality judgement. Assessing
Organizational goals could be traced easily in both organizations in which we tested the instrument. F’TT-post clearly changed from a government-like service orientation to more commercial goals. The ANM’s goals were formulated in terms of the products the organization offers the municipality: lobbying, advice and commercial services.
The process targets
In order to reach its goals an organization will arrange its activities as business processes. Davenport (1993) defines a business process as a structured, measured set of activities designed to produce a specified output for a particular customer or Journal of Strategic Information Systems I994 Volume 3 Number 3
187
Measuring the strategic dimensions of Ihe quality of information:
G J van der Pijl
market. The organizational goals are, more or less accurately, translated into targets for the business processes. Information systems can be used to reach these process targets. In order to assess the quality of information we have to consider whether the organization has made use of the opportunities offered by information systems when structuring these processes. The method used in the analysis is an adapted form of the method proposed by Bedell (1985). It starts out describing the business processes using Porter’s Value Chain or some other descriptive tool. Next the information systems are described. For each system we describe which (parts of the) business process it supports, who provides and who uses the information, a global indication of the users’ satisfaction with the system and the way in which the system contributes to the goals and targets. For the latter classification we use the categories: l l l l
automating parts of the business process; using information for managing the business; incorporating IT in the product; using the communications possibilities of the IT infrastructure.
Based on these descriptions the method gives an impression of the degree to which the important (parts of) business processes are supported by information systems. These partial conclusions can be summarized to a final global judgement if the assessment is done using detailed quantitative scales as a basis for the interviews. Of course the quality judgement has to be completed by looking at the performance of the existing systems. Systems supporting important parts of the business processes that perform poorly diminish the quality of information. On the other hand systems that support unimportant parts of the business processes but perform excellently may be a waste of money. The shift to private enterprise generated a decentralization in the hierarchical structure of PTT-post. Decision making was delegated to lower levels. The structure of the systems for management information did not yet reflect this shift. Most information went to the top of the organization without supporting the decisions of lower level management. Thus it was concluded that the quality of information was insufficient in this respect. In the ANM the lobbying activities were traditionally seen as the most important activities of the organization. The structure of the information systems was mainly designed for supporting these activities. This made support for other activities much less efficient. In recent years a growing number of decision makers in the organization have come to believe these other activities to be as important as the lobbying activities. Hence the advice was to change the structure of the information systems so as to reflect this new view on the relative importance of the organization’s activities. This meant for example that a good financial system had to be made available to the commercial activities.
Organizational structure
The structure of the organization is an important determinant of the amount of communication that is needed to co-ordinate its activities. Thus, when one looks at factors influencing information needs, the structure of the organization has to be included in the analysis. It would appear that the only author hypothesizing a strict relationship between organizational structure and the information needs of the organization is Leifer (1988). He uses a typology of organizational structure based on Mintzberg (1983). Leifer’s ideas, however, have not been tested in practice. 188
Journal of Srrategic Information Systems 1994 Volume 3 Number 3
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
In this study the structure is described in terms of its hierarchy and in terms of Mintzberg’s typology. From these, conclusions about the need for information are drawn using common sense and the ideas of Leifer and Mintzberg. Comparing these needs with the actual situation gives the structure related part of the quality judgement . The largest part of the ANM could be described as a professional organization. In Leifer’s terms this should mean that the information infrastructure leaves ample room for the individual needs of the professionals and supports their structured and unstructured communication. In both respects ANM’s infrastructure was not very effective. Since decisions on information systems were highly centralized, they tended to support big central systems at the cost of small local systems for the professionals. Also the electronic mail facility offered was rather poor. Therefore the quality of information was considered to be poor in these respects. Organizational culture seems reasonable to hypothesize that there is a relationship between organizational culture and the role of information in the organization. The literature, however, does not describe any explicit relationship. Therefore an attempt was made, following Schein (1985)) to describe different levels of culture in the PIT-post organization. As such an approach takes too much time in the context of our research, this was not successful. Therefore, in the case of the ANM, the approach was to describe culture by determining the position of the organization and its parts on a set of scales representing different aspects of culture. The scales are borrowed from Hofstede et al. (1988). It turned out, however, that respondents hardly recognized the items on the scales. Thus no satisfactory approach to culture has been found thus far. We intend to return to this matter in future research projects. It
Conclusions The study described in this paper offers a framework that encompasses relevant aspects of quality of information in an organization. Starting from the framework, an approach has been developed to support management in judging the quality of information. This ‘teleological’ or ‘strategic’ approach links organizational goals and targets with information (systems) requirements. The empirical results presented above indicate the applicability of the approach in different practical situations. Descriptions of the organization’s environment, goals, business-process targets and structure led, in a very natural way, to requirements that the information systems of the organizations have to meet. The actual situation could be offset against these requirements. Discrepancies between the two give rise to negative quality judgements. Correspondence between requirements and reality leads to positive quality judgements. Only the analyses of culture based on Schein’s levels of culture or Hofstede’s dimensions proved to be impracticable. Respondents could not handle the description of the culture dimensions very well and gave them their own, varying interpretations. Besides, they had their own vision on and terminology for cultural differences between parts of the organization. Although there was no strictly hierarchical structure of goals, strategies and targets, it proved possible to pinpoint the important information systems and the important gaps in the information structure. This led to clear cut ex post conclusions on the quality of information and information systems in these particular organizations. Journal of Strategic Information Systems 1994 Volume 3 Number 3
189
Measuring the strategic dimensions of the quality of information:
G J van der Pijl
References Ansoff, H (1965) Corporate Strategy, an Analytic Approach to Business Policy for Growth and Expansion McGraw-Hill, New York Bahlman, J P and Meesters, B A C (1988) Denken & doen, een studie naar de ontwikkeling en strategische herorientatie van zes Nederlandse ondernemingen KUB, Tilburg, Netherlands Bailey, J E and Pearson, S W (1983) ‘Development of a tool for measuring and analyzing computer user satisfaction’ Manage. Sci. 29, 4, pp 395-398 Bedell, E F (1985) The Computer Solution; Strategies for Success in the Information Age Dow Jones-Irwin, Homewood, IL Boehm, B W, Brown, J R, Karpar, H, Lipow, M, McLeod, G J and Merritt, M J (1978) Characteristics of Software Quality North Holland, Amsterdam Delen, G P A J and Rijsenbrij, D B B (1990) ‘Kwaliteitsattributen van automatiseringsprojecten en informatiesystemen’ Informatie 32, 1, pp 41-55 Deming, W E (1982) Quality, Productivity and Competitive Position, MIT Centre for Advanced Engineering Study, Cambridge, MA Davenport, T H (1993) Process Innovation; Reengineering Work Through Information Technology Harvard Business School Press, Boston, MA Egten, C A van and Pijl, G J van der (1992) Kwaliteit van Informatie Limperg Instituut, Amsterdam Hammer, M (1990) ‘Reengineering work: don’t automate, obliterate’ Harvard Business Rev. July-August, pp 104-112 Hofstadter, D R (1979) Godel, Escher, Bach: An Eternal Golden Braid Basic Books, New York Hofstede, G et al. (1988) Organizational Cultures - Beyond the Fad; A Qualitative/Quantitative Study Across Twenty Cases (in manuscript) RUL Inose, H (1990) ‘Informatietechnologie is technologie van de vrijheid’ N R C Handelsblad 4 December ISOto 9004 International Standards, International Organization for Standardization. Geneva Juran, J M (1974) Quality Control Handbook McGraw-Hill, New York Keen, P G W (1988) Competing in Time; Using Telecommunications for Competitive Advantage Ballinger Publishing Company, Cambridge, MA Lawrence, P R and Lorsch, J W (1973) Organization and Environment; Managing Differentiation and Integration Irwin, Homewood, IL Leifer, R L (1988) ‘Matching computer-based information systems with organizational structures’ MIS Quarterly March, pp 63-73 McCall, J A (1979) ‘An introduction to software quality metrics’ in Cooper, J B and Fisher, M J (eds) Software Quality Management PBI, New York McFarlan (1984) ‘Information technology changes the way you compete’ Harvard Business Rev. May- June, pp 98- 103 Mintzberg, H (1983) Structure in Fives; Designing Effective Organisations Prentice-Hall, Englewood Cliffs, NJ Nolan, R L, Pollock, A J and Ware, J P (1989) ‘Toward the design of network organizations’ Stage by Stage 9, 1, pp 1-12 Parker, M M and Benson, R J (1988) Information Economics; Linking Business Performance to Information Technology Prentice-Hall, Englewood Cliffs, NJ Pijl, G J van der (1993) Kwaliteit van informatie in theorie en praktijk, PhD thesis, Limperg Instituut, Amsterdam Pirsig, R M (1974) Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values William Morrow, New York Porter, M E (1980) Competitive Strategy: Techniques for Analyzing Industries and Competitors The Free Press, New York Porter, M E (1985) Competitive Advantage: Creating and Sustaining Superior Performance The Free Press, New York Rhenman, E (1973) Organization Theory for Long Range Planning Wiley, New York Schein, E H (1985) Organizational Culture and Leadership Jossey-Bass, San Francisco, CA Simon, H A (1960) The New Science of Management Decision Harper and Row, New York Smits, M T (1990) The Quality of Information Systems, an Inquiry into Complexity, Working paper series Washington University, St Louis, MO Starreveld, R W, Mare, H B de and Jo&B, E J (1986) Bestuurlijke Informatieverzorging. Algemene Grondslagen Samsom, Alphen a.d. Rijn H (1985) Systematische Softwarequalitiitssicherung an Hand von Qualitats und Willmer, Produktmodellen Springer Verlag, Berlin Wiseman, C (1985) Strategy and Computers: Information Systems as Competitive Weapons Dow Jones-Irwin, Homewood, IL
190
Journal of Strategic Information Systems 1994 Volume 3 Number 3