Expert systems in manufacturing. Part 1: A users' perspective on expert-systems innovation

Expert systems in manufacturing. Part 1: A users' perspective on expert-systems innovation P Holden

The development of apphcations with proven benefits, the availabthty of better development methodologies and tools, realism about what expert systems are capable of, and the integration of expert systems mto mainstream mformation technology are all factors that, m the 1990s, will promote increased exploitation of this technology m the manufacturing sector. Despite these developments, however, there still remains some uncertamty as to precisely how expert systems are being used in manufacturing other than m the 'show-case' and large-scale demonstratzon systems, and m the more general attempts to disseminate selected companws ' experwnces in developing apphcattons in an attempt to define 'best practice'. Further, little attention has been pard to the managerial context and human and orgamzattonal processes revolved m expert-systems renovation. The paper ts one of two that review the management of expertsystems technology m manufacturmg, It begins by evaluating current experiences wtth expert systems through a study of 145 manufacturmg compames zn the UK, from which an agenda of business, technical and managerial tssues Is raised. The analysts hzghlights the tmportance of human and organizational perspectives m the understandmg of why expert systems succeed, or indeed fad, and tt points to the need for greater attentwn to be paid to the process issues of innovation that help compantes to define and manage appropnate development pathways in thezr organtzation - - appropriate in the sense that they are meanmgful within a specific organizattonal context, and adaptable given a company's specific needs and constramts. The second paper extends these arguments to a consideratton of the wider processes of technology (and knowledge) transfer, and it develops a management framework to facihtate successful innovation m expert systems, and, more importantly, to enable desirable organizational change to take place. Innovation & TechnologyAssessment Umt, IERC, Bmldmg37, Cranfield Institute of Technology,Cranfield,Beds MK43 0AL, UK Paper recewed 22 May 1991 Revised paper received 12 November 1991 Accepted2 December 1991

Vol 5 No 2 June 1992

Keywords. manufacturing, mnovatton, human perspectives, organizational perspectives

There has been a scarcity of feedback on precisely how expert systems are introduced into manufacturing organizations ~, how they are justified and developed 26, and how they are actually used in an operational context 2. The analysis has been subsequently limited in scope to discussions on the market and uses for the technology (see Reference 3, for example), or to the merits of specific development tools and methodologies, rather than to understanding and definition o f company needs and the actual organizational and management processes that take place. Moreover, where research m the past has looked at the types of user organtzatlon that make use of expert systems and their choice of development approach, tt has done so from the perspective o f the suppliers, and not the users, of the technology (see Reference 4). Recent studtes have begun to evaluate expert systems from a user-organization perspective, and to provide useful insights into the structures and mechanisms of expert-systems innovation, and also to discuss the organizational barriers and impacts of such changes. However, tt ts difficult to make generahzations for the manufacturing sector from these studies, because the sample size is either too small (for example m Reference 1), or because they address all sectors of mdustry and education, such as finance, commerce and government, rather than concentrate on manufacturing alone (see, for example, Reference 5), Further, the motivahon for these and other similar studies carried out in Europe and USA ts the ehcitat~on o f guidehnes and models of 'best practice' in expert-systems development, rather than simply the understanding of more about the managerial and organizational processes revolved. For these reasons, the author collaborated with a U K consultancy in the carrylng out of a national survey of expert-system users. The purpose of the survey was to increase understanding of

0950-7051/92/020149-09 © 1992 Butterworth-Hememann Ltd

149

companies" experiences in using expert systems. A number of key issues were addressed • What is the overall level of use of expert systems in, and impact of expert systems on, manufacturing organizations 9 • How is the organizational and business value of expert systems defined and assessed9 • How is assessment and development actually undertaken, and what methods and tools are used 9 • What are the problems and barriers m the technology-transfer process9 • How do orgamzations manage the process of change? • How are expert systems evaluated9 These questions raise issues not only about how expertsystems products and expertise should be transferred, and the appropriate tools, processes and methods that are necessary in this process, but also about how companies should go about defining the strategic value of, and the organizational need for, the technology SURVEY DESIGN AND METHODOLOGY The questionnaire consisted of seven mare sections the first provided an aggregate measure of the overall status of expert systems in user organizations (UO), and a more detailed breakdown of the number and statuses of specific expert-system projects being undertaken. The possible statuses of projects varied from nothing more than an idea to a fully operational system, and they were intended to provide a measure of commitment by UOs towards expert systems The second section of the questionnaire provided a description of the industry in which the UO was based; these were later coded according to standard SIC company classifications It also requested details about the job function of the end user of the system, and how it was intended to be used. After Section 2 of the questionnaire, the focus of questions shifted from an organizational level, and concentrated on specific application projects Section 3 began by determining the current status of the application, the hardware and software being used to develop the system, and the job function of the individual held responsible for its development. As the application might be used for a number of reasons, Sectmn 4 asked the respondents to rank those task functions that were most important, e g design, configuration, planning etc. This also indicated functions that might be supported by the application From this classlficatmn, Section 5 looked more closely at how the system was used. Questions distinguished between the original systems design (e.g to replace an expert or provide advice only) and the actual design in practice They identified the level of integration necessary for the application to function - - whether it was sufficient as a stand-alone system, or whether it required interfacing with a database, for instance. The questionnaire also sought to identify how, if at all, the maintenance of the knowledge base was carried out, and by whom. From the analysis of the applications, the survey went on to look at how UOs go about developing their systems: whether, for example, they use formal methodologies or informal guidelines, and whether they use software-engineering and project-planning tools or software 150

as an aid to development It also determined the sources of Information on which the knowledge base was founded On the assumption that a system of some form had been developed, the final section attempted to quahfy the system's organizational impact and implementation benefits, together with constraints that might have prevented or impinged on its successful implementation In terms of benefits and constraints, the questionnmre made use of ranking and rating methods of data input The ranking was intended to provide a measure of importance for each constraint or benefit in relation to all the others In contrast, the ratings indicated the significance of individual benefits or constraints as perceived by the questionnaire respondents In some cases, follow-up Interviews were held to gain an understanding of more about the process and contextual issues raised in the survey Over 460 completed questionnaires were returned From this number, the author selected 145 for detailed analysis on the basis of three limiting criteria. • The application had to be in the UK, initially. A second phase of analysis is under way that wdl make comparisons between activities in the UK and other European countries. • The apphcatlon had to be in the manufacturing sector. • All the respondents selected had to be 'end users' rather than producers or suppliers of expert-systems software or speclahst IT consultants. ORGANIZATIONAL INNOVATION

CHARACTERISTICS

OF

This section of the paper is interested in the organizational attributes of the companies that were using expert systems. For the purposes of analysis, the manufacturing sectors were divided into seven main categories: transportation manufacture (which included aeronautical and automotive manufacture and ship building), computer and related electronic-component manufacture, consultancy and consultant engineering, plant and process control (this included chemical companies, paper and adhesives, and the fabrication of materials such as rubber and plastics), supply and light engineering (such as cable manufacturing, electrical equipment, telecommunications and precision Instruments), and heavy engineering and manufacture (such as steel manufacture, metal products and machinery fabrication and power engineering) Although there was an even spread of companies across each of the manufacturing sectors, a first observation from the survey was that expert-systems (ES) activity was most concentrated in high-technology sectors, as the diffusion and dissemination of ES ideas and concepts was most vigorous there. In addition to the level of ES activity, a second measure of organizational commitment was the stage of innovation that companies had reached. The survey showed that, although most of the companies were receptive to expert-systems ideas and concepts, and more receptive to their potential benefits, 60% of the sample were no further forward In development than the provlstun of demonstration systems in their respective organizations. Indeed, for a quarter of the sample, there was no commitment towards development per se, but rather an interest in the technology and an obligation to 'maintain

Knowledge-Based Systems

a watching brief' and find out more about the technology and its strengths and weaknesses. This may be a reflection of the relatively immature and uncertain state of the consumer base, and an indication that the market penetration is not as deep as vendors may suggest. More convincing is the argument put forward by D'Agapeyeff and Hawkins 6 (p 187), who state that one reason why a large proportion of manufacturing is stall at this level is that: the contradictory explanations they receive as to what these systems are, or should be In these circumstances, it is understandable for management to pause untd the matter is resolvedby the emergenceof a cdnsensus view The authors add that, moreover, many companies will fall to move beyond this information-gathering level because of the lack of management commitment, business secrecy, which prevents proven expert systems in companies from being demonstrated (thus inhibiting the transfer process between organizations), and a fear of the nature and the costs of this technology. The latter pomt is demonstrated by the lack of demonstrable systems and benefits. It is, perhaps, for this reason that 28% of respondents have budt a demonstration expert system in their organization prior to further development Many respondents spoke of the 'immaturity of the technology', and the purpose of the demonstrator was to be a fad safe that allowed some experimentation without risk or significant costs Moreover, respondents clearly separated this from a further stage of development, the production of the full-scale prototype, in which 29% of the sample were involved. Other surveys have shown the difficulty that organizations have in making the transition from working or fullscale prototypes to operational systems. O'Neill and Morris 4 found that, of a sample of 38 software houses representing 600 applications, only 2 5 0 had reached operational status A more recent survey by Harris et al. 5 showed that, when user organizations were interviewed directly, this figure was only 18%. Similarly, this survey showed that only 14% of respondents had operational systems. This need not suggest that the apphcat~on is commercially viable m traditional terms necessarily, but that it has achteved 'hve' status by being Implemented and used in the company. For a number of compantes in the sample, however, the organizational learning process was considered to be more important than an economic return on the final system For this reason, especially at earlier stages of knowledge transfer, justification is a difficult and clouded issue In the case of the sample companies, an important factor in understanding the level of progress through the innovation hfecycle was organizational size. It was found that larger companies tended to be closer to operatlonalizing expert systems than smaller ones. Large companies often had improved access to the latest ES tools and methods, and the resources to experiment more. Further, many of the large companies had formal 'gatekeeper' functions, wtth the task of scanning the latest outstde technologies and developments to see how they might be used in their organizations. Smaller companies, in contrast, undertook scanning on a much more informal basts, this competed for time with routine operational tasks Where expert systems were developed in small companies, however, follow-up interviews revealed that Vol 5 No 2 June 1992

the price/performance and business benefits were significant. It would be incautious to conclude from the survey results that some manufacturing sectors are more successful in producing operational systems than others, although this is a valid research hypothesis that deserves pursuing Companies from all sectors were involved in earlier development stages. However, the computing and electronic-component sector, as well as having the greatest number of operational systems, also had the most problems in terms of abandoned projects and lack of interest. Of those companies that expressed no interest in expert systems, a pnncipal reason was a lack of awareness; as one respondent put it, 'nobody knows what they are or how they are used' The other main reasons suggested that a prehmlnary evaluation of the technology had taken place, and that it had been concluded that there was no use for the technology (30%), that it was too costly (20%), or that the intended problem domain was judged to be too complex for the technology (10%).

WHERE EXPERT SYSTEMS ARE USED This section of the paper looks at individual apphcations Most respondents provided mformatlon on their latest project, and, for this reason, most were at the ideas and early stages of development; only 6% were at an operational stage. For this reason, the questionnaire tended more to mirror respondents' expectations rather than actual systems performance. Despite the potential diversity of applications, the survey showed a convergence not just on generic applications (over 60% of the applications were concentrated m only five areas' diagnosis, information interpretation (l e. procedural animation), alert/warning, design and selection), but also on specific applications such as equipment diagnostics By far the most significant role for apphcations was in the diagnosis of faults, with nearly one-fifth of all the systems being designed for this purpose. This suggests a preference for applying expert systems to a core of apphcation domains with a proven record of success in other organizations, rather than for risk developing 'innovative' systems It may also indicate, as D'Agapeyeff and Hawkins suggest, a preference for 'simpler' expert systems. This is endorsed by the survey, which showed that, of those applications that were of operational status, most were shell-based, and limited to diagnosis, selection and procedural-animation domains. Very few applications made use of uncertainty reasoning, and more than 60% used rule-based representations exclusively Their sgnphcity has benefits in banishing the widespread misconceptions about expert systems being 'lntelhgent learning systems'. Indeed, part of the problem in previous years in defining the business and organizational role for expert systems has been in understanding what they actually are. Lawrence 27 argues that terms such as 'AI' and 'expert systems' are 'anthropomorphic' in that they ascribe human characteristics to nonhuman things, 'thereby sumulating a response that is often more emotional then objective'. Fortunately, the integration of expert systems into mainstream Information technology has meant that much of the hype and questionable terminology has been replaced by more meaningful labels. 151

knowledge-based data-management systems, heuristicbased front ends, decision-support systems, and so on.

H U M A N ROLE FOR EXPERT SYSTEMS This section of the paper looks at the actual and intended end users for applications Seven broad categories of end user were identified from the survey These were operations management, business management, design and project engineers, planning and industrial engineers, IT/ computing specialist, shop-floor and operational workers, and commercial and administrative staff There was a fairly even distribution among these job functions, with the most common end users being design and project engineers and manufacturing (shop-floor) operators (both at 22%), and the least common bemg the IT/ computer specialist (at 7%), in the user organization This suggests that expert systems are being used across a spectrum of skills and job descripttons It also dispels the notion that expert systems are being used exclusively at particular levels in organizations An analysts of end uses for expert systems is sensltwe to claims that there is an implicit replacement focus in the use of this technology 7, or that it downgrades human knowledge and skills 8 Certainly, there ts a series of h u m a n - c o m p u t e r interaction issues that should be consldered and is covered better elsewhere (see, for instance, Reference 9)_ It is also important to complement such studies with a wider understanding of the organizational and human contexts of the technology as a social system. However, expert systems are tools, and, as with all tools, their value and impact depends on the effectiveness of their use The statement that expert systems are inherently deskilhng is therefore unwarranted; however, a poorly designed system can have this effect. Results from the survey showed that over half of the respondents had planned or had systems whose most important role was the support of a knowledgeable end user in a support capacity (this might be a design expert or a skilled machine operator, for instance). This is significant in the light of the above discussion, in that the expert systems were being used to improve or enhance the capabilities of the expert by improving decisionmaking productivity, speed of recall, presentation of results and so on_ A second important use of applications was the provision of adwce to qay' people so that, for example, they could undertake their own trouble shooting or perform selection and configuration procedures without continually referring to specialists. Where the end user was an expert substitute, the person was still required in most cases to have a good understanding of the domain, and, moreover, the dependency on the expert or specialist was seldom removed completely. Indeed, only 9% of respondents intended the most important role for the application to be the replacement of the expert or the removal of human participation entirely from a work task A third category of use applied to monitoring, realtime and process-control systems whose function was to interpret feedback information from machinery and other online information, and to take subsequent control, planning, diagnostic or scheduling action A crosstabulatlon of the class of the end user against the manufacturing sector highlighted certain matches between job function and organizational role m the use 152

of expert systems For instance, in computer- and electronics-based companies, the end user tended to be an IT/computer speclahst, and, m process-control and chemical companies, the applications were primarily shop-based, and used by shop-floor operators Similarly, in consultancy and consultant-engineering organizations, there were many cases in which the end users were engineers using expert systems as decision-support aides There were clearly anomalies to this rule, the most consplcuous of which was a high use of expert systems by business managers in light-engineering and supply companies

CONFIGURATION OF E X P E R T - S Y S T E M S TECHNOLOGY It was indicated above that there was a preference by companies in the survey to 'keep it simple' and focus on proven applications and domains. In terms of software, this was evidenced by the fact that over one-third of all the respondents made use of shell-based ES software, one-quarter made use of AI tools, and only a small percentage made use of AI languages and conventional programming techniques No application-specific shells were used, and nor were there any ES environments. In terms of hardware, the most c o m m o n development and delivery medium was the personal computer (at about 60% in both cases), while the second most common piece of hardware was the organization's existing mainframe system (at 21% and 23%, respectively). The results reflect a national trend away from dedicated AI machinery towards equipment that is available in the user organization They might also reflect a reluctance to commit more substantial resources to expert-systems development until the technology is considered to be proven in less complex applications A further measure of commitment and complexity ts the extent to which the application is mtegrated in the organization Results from the survey show that, although the single most c o m m o n level was the standalone level, most systems were either embedded, linked or integrated in some way to other company systems or software The most frequent form of integration was as an interface to databases, with dBase III, Clipper and Oracle proving the most popular 12% of the sample also interfaced thetr ES applications to other computer systems, such as computer-aided-design (CAD), toolmanagement, finite-element-analysis and materialsrequirements-planning (MRP) systems. A further 12% also designed ESs that were embedded and that formed part of a wider information system, for example intelligent master production-scheduling systems and computer-aided process-planning systems. The remainIng 15% were linked systems that were downloaded to, or loaded from, other systems, or that formed a front end to other systems. A possible hypothesis that deserved testing was that the level of integration of the expert system altered the nature of the end users' role. A crosstabulation of integration level against system use, however, revealed no indication that stand-alone systems were likely to encourage a 'replacement focus', or that interfacing would necessarily enhance the quality of the end-user function This is more dependent on the organizational

Knowledge-Based Systems

integration of the system in the workplace than on its technical interfacing features ~°. M E T H O D O L O G I C A L A P P R O A C H E S AND T E C H N I Q U E S U S E D IN D E V E L O P M E N T Other work has shown the absence of a development approach to be a key feature in the development of expert systems in user orgamzaUons5, although httle research is available to explain why this is so. Moreover, Ackroff et al. H note the organizational difficulties of integrating ES development approaches within the prevailing management, control and administrative systems of compames. The user survey identified a number of possible reasons for the above problems, but, foremost, it highhghted a great deal of confusion over how best to develop ES technology, and, therefore, which methodology, tool or techmque to adopt. For instance, only 65% of respondents endeavoured to use a methodology or published guidelines. With these, there appeared to be no coherent pattern of development, with companies placing different emphases on d~fferent phases of development. Further, w~th those respondents that used guidelines, httle or no attention was paid to preproject specification and analysis, maintenance and upgrading, and evaluauon and justification. Indeed, Jamleson and Szeto 2, in a s~milar study, found that many evaluations and justifications were carried out post hoc, or were accounted for within an 'experimentation' or R & D budgets. With regard to the principal actwlties involved in development, 78 % of those that followed a methodology or gmdeline adopted prototyping-based techniques. For many respondents, this was seen as a substitute for requ~rements-specxficat~on and project-planning actiwties. With the respondents that did attempt to incorporate pre~mplementation design and analysis activities into the development methodology, the most frequent approach taken was the making use of currently used software-engineering techniques and planning systems in an attempt to structure and control the development process. Methodologies such as the Mascot, DeMarco and Yourdon methodologaes and company-specific planning and control techniques were used for this purpose. Of those companies m the sample that considered design and analysis activities, only two organizations made use of ES-specific development lifecycles, moreover, both of these identified problems in their application due to their 'complexity'. Very few respondents from the sample had attempted to perform business and functional analyses to identify and define systematically possible ES applications. Despite this, those that d~d were enthusiasUc, and stated that their particular approach had been successful in the choice of a wable and useful application. One organization, for instance, had made use of Checkland's Soft Systems Methodology in combination with IBM's Business Planning Approach for this purpose. Indeed, more recent approaches have begun to take account of the organizational and business context of expert-systems renovation by providing 'soft' front ends (see, for example, Reference 12) to their development methods Whachever method or combination of techniques is used, there is some eqmvocation at a research level over Vol 5 No 2 June 1992

the most appropriate development paradigm to adopt. Debate is polarized between two schools of thought. The first is based on the belief that, to be commercially viable, expert systems should adopt traditional software-engineering methods of development based on a structured model of 'specify-prove-implement-verify'j3. In contrast, the second school of thought argues that, by virtue of the uniqueness of expert systems, the traditional model of IT development is inappropriate, and that it should be superseded by a second model based on a rununderstand-debug-edit cycle An important indicator m the above debate, therefore, is the extent to which the design and intended use of expert systems changes or is modified through development In the survey findings, one-quarter of the respondents stated that their designs had changed through development. W~th these, three categories of change emerged" changes to the size m the domain (45%) (many respondents described how implementation constraints, such as shortage of time and the difficulties of knowledge verification, had forced a reduction in the scope of the project or design features), changes to the user interface (33%) (companies were generally over-ambitious in terms of the interfaces that they hoped to achieve given the limitations of the software tools that they used), and, finally, respondents described how their systems were used in different ways from those intended (m all cases, this was an improvement on the original design that only became ewdent during the development process itself). It seems, therefore, that a hybrid approach that provides a structured speofication framework, but is flexible enough for some iteration, is a compromise in terms of the above debate, and this ~s precisely the d~rect~on in which development methodologies are evolving (see Reference 14) KNOWLEDGE ACQUISITION The literature has, since a relatwely early period, indicated that knowledge acquisition is a major bottleneck m expert-systems development15. This view is supported from the survey results, which showed this to be a consistent problem that, while not preventing implementation, frequently hampered development progress. Despite this, the takeup of knowledge-acquisition tools by respondents was mlmmal. O'Neill and Morris 4 explore the usefulness of the increasing number of methodologies produced by the research community for elicitmg and orgamzing expert knowledge. They argue on the basis of survey results that, in practice, only the most simple interviewing techniques are being used. This, however, might reflect user organizations' lack of appreciation of the importance and value of knowledge-acquisition methodologies as much as a mlsahgnment of research needs. Results from the user survey showed that the mare source of information for systems was through structured and unstructured interviews Although this technique was used by nearly 37% of the respondents, equal use was made of secondary sources of reformation such as techmcal literature and manuals. User observation techmques and the more analyUcal deep-reasoningbased methods, such as repertory grid analysis, were used infrequently. Both observations reflect again a preference for simple shallow reasoning systems whose 153

purpose is more to present information and high-level knowledge m an effectwe manner than to perform complex decision-making actmns

RESPONSIBILITY FOR DEVELOPMENT The choice of development methodology may be as much a functmn of who is responsible for development and personal preferences than any rationale based on what is required The responses broadly fell into five main categories of development responsibility: systems or computer departments, end users, specmhst knowledge engineers, consultants, and other functmns Flndmgs from the survey showed that 28% of the projects were the responsibility of a specmlist knowledge engineer This person was e~ther tramed or employed specially to manage the project Other users preferred to retam responsibd~ty within the computer department (23%) or with the person considered to be techmcally the most competent (if this was not the knowledge engineer, then it might be an external or internal consultant (20%)) In the remainder of the cases, responsibility was attributed to semor figures in the organization who had champmned the project (most frequently the technical director or eqmvalent) With the crosstabulation of the intended design use against development responslblhty, ~t was hoped to estabhsh whether certain project viewpomts invoked part~cular design uses. The results showed significantly that, m all the cases in which the intended design use would replace an individual, the person responsible for development was the end user. In contrast, where the expert system was designed to give adwce to a lay person, m most cases, the person held responsible for development was the knowledge engineer This dichotomy leads to a number of conclusions, foremost, ~t suggests that the personal and pohtlcal ends of the end user may diverge from those of the organization, and, second, that, although end-user partlcipatmn is essential for the success of the project, actual control and responsibility by the end user may have negatwe organizatmnal effects In all the cases in which an expert system was designed for realtime feedback control, the person held responsible for the project was an outside consultant Th~s h~ghhghts the complexity of the task and the need for user organizations to liaise with speciahst expert-systembased consultancy orgamzatlons MANAGING DEVELOPMENT IMPLEMENTATION

Markus argues that end-user computing as a development strategy often suffers from an 'reward wew' of technology, with needs and benefits that are defined m techmcal rather than organizatmnal terms Further, in the longer term, the technology becomes outdated or underutihzed, because of the failure to integrate the system m the organization. Many of the orgamzatlonal problems described in the next section of the paper, such as a lack of commitment, uncoordinated and informal developments, and a failure to operatmnahze, are themselves characteristic shortcomings of end-user computmg It ~s for these reasons that a centralized but participatory approach to development ~s recommended. An 'end-user' philosophy might also explain a dlstmct 'hobbyist' approach towards the orgamzation of projects, although larger organizations tended to be more coordinated, as they were more able to establish centres of excellence within the company and to manage to cover the overheads Changes m the marketing of ES products and services m future may have a beneficial effect m definmg this technology as 'just another computing tool'. This will facditate the integration of ES developments w~th current computer-department activities, and thereby remove the tendency to view expert systems as a 'layman's' programming tool and alternative to the IT functmn. The survey showed that the lnitmtwe for ES Innovation is generally taken by technologists rather than management. This may generate tension between the technologist's understanding of the technology and ~ts capabdmes and functional and senior management's understanding of the organizatmn and its needs and priorities w~thm an overall business and IT strategy. The author's experiences have shown that the only way to reconcile these two perspectives is to combine them both withm a formal top-down programme of development; however, again, not many of the companies were committed to or able to support this infrastructure

AND

A double hurdle is faced by user organizations when they first learn about expert-systems technology and understand its capabilities, and then attempt to assess ~ts potentml when th~s ~s matched against company problems and needs. H o w a company undertakes such tasks varies accordmg to its structure and internal operations There are, however, a number of organizational strategies that may be adopted, and these are positioned between two extremes. • end-user computing, or what Markus ~6 calls a ' D I Y strategy', m which individuals or groups within a company avoid contact w~th the internal computing department, and rely instead on external informatmn

154

services and reformation and personal knowledge to develop systems. • m-house development assumes that the orgamzatmn already has the infrastructure to develop and support expert-systems projects in a centrahzed and controlled way. The role of the vendor is therefore minimal. The sensttwlty to specific organizational requirements that characterize m-house developments is offset by the high costs of maintaining a development infrastructure

BENEFITS OF USE OF EXPERT SYSTEMS This sectmn of the paper looks at the percewed and actual expressed benefits of the use of expert systems. These were analysed with both ranking and rating methods Ranking allowed those factors to be identified that were considered to be essential to the success of the apphcatlon. The most important expressed key benefit (ranked first) was that the expert system would mcrease the accuracy of decisions; this was followed by an increase in problem-solving abihty and an increase in the quality of work A slightly different approach was also taken that consisted in ordering benefits according to those ranked first, second or third in importance In th~s case, an increase in the quality of work was considered to be the most important, followed by increases in problem-

Knowledge-Based Systems

solving ability, accuracy of work and accuracy of decision making. Both results showed a tendency to express benefits in 'added-value' terms rather than costreduction terms such as a reduction in staff numbers or the use of cheaper staff with lower levels of skills. Moreover, for those expert systems in the sample whose role was to 'support a knowledgeable user', these qualitative benefits were consistently rated highest. In contrast, where their intended use was to replace people, key benefits were the reduction of skill levels, increases in work load, and staff reductions. Where the system was designed to support a 'lay person', respondents tended to provide their own benefits, which were not supplied by the questionnaire. These"included such factors as improving training capabilities, allowing operators to work independently, allowing faults to be recorded effectively, and so on. However, 'increased work load' and 'increased output' were more important benefits in this segment. A second means of measuring benefit is by rating the strength of feehng that a factor is important. This was calibrated on a 0-4 scale in the questionnaire, and the results were expressed relative to the sample midpoint or mean for all factors. A positive factor suggested that the respective benefit was more important than the average, and a negative factor less important The results were broadly s~mllar to those of the rankings, except for one aspect" in expressing the perceivedtmportance of benefits, respondents considered the cost effectiveness of the development project Itself to be the most important factor, while ranking it as being only moderately important relative to the other factors. A possible explanation is that most respondents were in the process of development rather than at the stage of operations, and they were therefore more preoccupied about cost effectiveness and cost control than the end benefits. It is again significant that the benefits considered to be the least important relate to the downgrading of individual roles through a reduction in the skilled personnel required, or a general reduction in skill levels. This suggests that many of the adverse social consequences forecast by a number of social researchers (see Reference 17, for instance) are somewhat extreme. BARRIERS TO EXPERT-SYSTEMS

INNOVATION Measured against the benefits of using expert systems are the problems and costs associated with development and Implementatmn. Expressed constraints were also analysed by the ranking and rating of data to assess the strength of feeling that a particular factor contributed towards implementation problems, and the factors that actually contributed to the prevention of Implementation. The most important constraint, that there was a lack of budget provision, was due in the mare to the fact that management could not be convinced of the technology's benefits. Indeed, a lack of management support, awareness and understanding of expert-systems benefits were all rated highly. This does not suggest that management is an unjust encumbrance on the process of expertsystems transfer, but that it is more of a 'regulator', ensuring that ES projects are developed because they have a business and organizational value, rather than because they are purely of technical interest Vol 5 No 2 June 1992

It was apparent from the survey that many of the systems were developed by 'enthusiasts', whether users, experts or appointed knowledge engineers. As a result, their activities were often separated from their company's mainstream information technology and computIng actwities, with the effect that the systems were assessed and developed outside existing evaluation and specification procedures and IT strategy. Moreover, the fact that awareness and support was a rarity, and there was a lack of confdence in these systems, suggests a predominantly bottom-up and technology-driven approach towards development by such enthusiasts rather than a business-driven, 'top-down' approach. This is reaffirmed by earher results, which show a clear focus on the use of tools at a development level, and the omission of predevelopment analysis such as business planning and functional specification. The least important constraints were those that related to personnel resource problems, such as a lack of suitable experts and a lack of identified users, although a lack of knowledge engineers was cited as the third most important implementation constraint. The label 'knowledge engineer' was not helpful in the study, as many of the projects were small-scale, and the expert was often responsible for development as well as being the knowledge engineer and eventual end user. Some of the most strongly felt constraints were specific to organizations These included too-high expectations of what the project team and expert systems could achieve, a lack of time, no access to experts, no access to end users, changes in the organization, and a lack of interest by end users. Moreover, organizations that expressed these and s~mllar constraints tended not to use a top-down approach, but rather adopted guidehnes that focused on technical aspects of design and development. DEFINING MAINTENANCE

NEEDS -- A

CASE FOR PLANNING A final and often neglected issue in expert-systems development is postimplementatlon maintenance TM. This section of the paper looks at respondents' attitudes towards maintenance and the allocation of resources to the performance of this function. The first interesting observation is that 40% of the respondents did not have a view on maintenance (and this did not Include those who did not know what to do). From the responses of those who did comment, six clear categories of maintenance were defined From these, the largest group responsible for maintenance was that of the systems developers (33.5%) The respondents in this group viewed maintenance as anything from changes in screen layout and bug fixes to major enhancemen)s to the knowledge base, and they believed that formal documentation and change-control procedures should be managed and undertaken by the most qualified person available, although most were unsure precisely how this should be carried out. In contrast, 25% of maintenance work was carried out by the end user, reflecting the high proportion of experienced users who were developing their own systems. The respondents in this category argued that the end user (and, in some cases, the end user was the expert) had the greatest understanding of the domain, and would remain committed to using the system if the individual had an active role in future maintenance and development work 155

Nearly 10% of the respondents intended to use automatic methods of maintenance by prowdmg automatic input through feedback-control realume data updates, and automatic-induction techmques However, few of these companies had actually implemented a system and were in the process of development, suggesting, possibly, that they were not fully aware of the techmcal complexity of achieving this in practice Perhaps the most sxgmficant finding, however, was that more than one-fifth of those that commented on maintenance indicated that they were unsure about how to go about performing maintenance work, or that they would not bother to do any, and would therefore leave the knowledge base stauc Very few companies from the total sample discussed maintenance as a design Issue, and they instead considered it as being something to be consldered during implementation. On this issue, the attitudes towards maintenance changed over phases of the development cycle At the problem-selection stage, well over half of the respondents were undecided as to how to maintain the system, or assumed that it might be left static. As the development progressed to the demonstration and prototyping phases, possibly because of the necessity to involve the end user and expert in these acUwtles, almost half of the respondents believed that either the expert or end user should undertake maintenance work Further development showed an increased preference towards the systems developer, until, at the operational phase, over 50% of the respondents believed that the maintenance should be undertaken by this person. Crosstabulatmns were made to check whether attitudes towards maintenance differed according to who was responsible for development There were two significant observations first, when the user (who might be the expert or end user or both) was responsible for development, the responsibility for maintenance was also passed on to the user, second, when development was the respons~bihty of the specmllst knowledge engineer, maintenance was clearly viewed as a systems-development function. It was reasonable to assume that maintenance responsibilities might change if the software tools adopted were easier to use. However, there was no evidence that shells, which were considered to he the least difficult expertsystems software to use, promoted maintenance by end users or experts. In fact, over 30% of shell-based systems were either left static, or it could not be decided how maintenance should be carried out It was also interesting to see how the level of mtegrattan affected attitudes towards maintenance. A comparison was made between applications that were classed as being 'stand-alone' and those that were embedded In the latter, the expert system formed only part of a w~der integrated system In that most embedded systems share information with other systems, it is essential that the knowledge base is continually updated For this reason, no embedded systems were left stauc, and relatively few, compared with stand-alone systems, had a maintenance plan. One of the greatest benefits of embedded systems is that maintenance is made easier by direct mformaUon entry into database files or relaUonal databases held m other computer systems and then converted into a knowledge format for inclusion into the knowledge base of the expert system ~9. In contrast, the maintenance of shells 156

requires the encoding of knowledge into producuon rules using the specific syntax of the programming tool This accounts for there being twice as many user- and expertmaintained embedded systems as stand-alone systems, and, conversely, twice as many stand-alone systems as embedded systems mamtained by systems developers CONCLUSIONS Expert systems are now seen by many manufacturing organizations as offering a useful set of techniques in their systems-development toolbox There is certainly less hype and more practical bias in the literature coupled with an increasing portfoho of successful applications that are being used in industry. However, mysticism still remains over the business, managerial, organizational and human processes Involved in expert-systems mnovatlon Nevertheless, the survey shows that it is precisely these issues that determine the relative effectiveness with which companies introduce and exploit expert systems m their orgamzaUons. More research and experience sharing is clearly required for more to be learned about the processes and context of expert-systems m n o v a u o n Fortunately, this is being driven by two forces first, the fact that operational successes remain elusive for many companies, and second, that there is increased recognmon that business success does not always follow from technological success (see Reference 20, p 3). Thus, although a technology focus is clearly necessary in the development of expert systems, it ~s not sufficient for the understanding and management of the total change process This sentiment is embodied in the view of Bobrow and Stefik ~'l' ' remember that you are not designing a computer system, but are putting a process In place in a user orgamsaUon' Recent hterature is much more responsive to the business and managerial needs of expert-systems development For instance, W a m w n g h t 22 offers a framework for the management of AI-based projects, and Van de Brugg and Orcluch 2~ and Dutta 24 provide much-needed help to assist companies to integrate and position ES projects w~thln their corporate strategy and to learn more about their business impact However, there is insufficient feedback and research on the process issues of ES lnnovaUon, which seek to understand more of the human and organizaUonal processes and effects of ES technology-induced change There is also a need to consider the contextual factors involved in the successful deployment of ESs These address the specific moUves, organizational chmate, culture, needs and condmons in which innovauon takes place. For both process and context issues to be addressed, a change of reference is necessary, from one that centres on methods, tools and technology (i.e. content) to one that begins by understanding individual orgamzaUonal and business needs and priorities and the circumstances of change This requires new eclectic models of enquiry that are able to define problems and reqmrements in soft (human, organizational and cultural) as well as hard (technology, methodologies and tools) terms ~ The successful management of expert-systems reduced change requires, first, that the practitioner be aware of the potential pitfalls and difficulties with which he/she may be faced through the innovation lifecycle, in this sense, the survey has identified a number of technical,

Knowledge-Based Systems

managerial and organizational barriers that should be avoided. It also requires that the practitioner take stock of the current situation and chmate (social, political and financial) in which the change process is expected to take place, as these will affect the eventual outcome25. For this to be done, it is necessary that the individual responsible for change ~s able to view the innovation task from different viewpoints and levels of analyses to take account of the different processes and effects involved The second paper of the pair of which this paper is the first develops a conceptual framework for the management of expert-systems innovation It adopts a multidimensional and multilevel frame of reference so that applications meet content-based needs by satisfying the task requirements of development- and context-based needs through adapting to specific organizational settings. It does so by adopting a knowledge-transfer perspective of innovation through which appropriate pathways of technology development and organizational change may be defined ACKNOWLEDGEMENTS The author would like to thank his colleagues at the Innovation and Technology Assessment Unit at Cranfield Institute of Technology, UK, for their support, Bob Muller and David Greggory for their stimulating and valuable contributions to this paper, and to the UK Science and Engineering Research Council for funding.

9 10

11 12

13

14

15

16 17

18

REFERENCES 1 Bramer, M (Ed.) Practical Experiences m Building Expert Systems John Wiley, UK (1990) (Introduction) 2 Jamieson, R and Szeto, R 'Impact of knowledgebased information systems on organisatlons' J Inf Tech. Vol 4 No 3 (1989) pp 145-158 3 'West European market for expert systems' Report E1266ID Frost & Sullivan, USA (1990) 4 O'Neill, M and Morris, A 'Expert systems in the United Kingdom" an evaluation of development methodologies' Expert Syst J. Voi 6 No 2 (1989) pp 90-99 5 Harris, D G, Dawson, J A and Davies, B K 'Report on a Computer Board initiative on knowledge based systems' Institute for Retail Studies, University of Sttrhng, UK (1990) 6 D'Agapeyeff, A and Hawkins, C J B 'Expert systems in UK business, a critical assessment' Knowledge Eng_ Rev Vol 2 No 1 (1988) pp 185-201 7 Guller, P 'Automation-skill-practice' m Goranzon, B and Josefson, I (Eds.) Knowledge, Skdls and Artificial lntelhgence Springer-Verlag, Germany (1988) 8 Goranzon, B 'The practice of the use of computers. A paradoxical encounter between different traditions of knowledge' m Goranzon, B and Josefson, I (Eds.)

Vol 5 No 2 June 1992

19 20

21 22

23

24

25

26 27

Knowledge, Skills and Artificial Intelhgence SpringerVerlag, Germany (1988) pp 9-18 Berry, D and Hart, A Expert Systems Human Issues Chapman & Hall, UK (1990) Dawson, P 'Intelligent knowledge based systems (IKBS) organlsational implications' New Tech., Work & Employ. Vol 3 No I (1988) pp 56-65 Ackroff, J, Surko, P, Vesonder, G and Wright, J 'SARTS AUTOTEST-2' in Reference 1 pp 209-226 Sharma et aL 'A SOClO-technlcal model for deploying expert systems. Part 1 The general theory' IEEE Trans. Eng. Manage. Vol 38 No 1 (1991) Partridge, D "AI and software engineering a survey of possibilities' Inf & Soft Tech Vol 30 No 3 (1988) pp 146-152 Taylor, R M, Bright, C, Martil, R and de Hoog, R 'The management of KBS development and maintenance under KADS-II' m Graham, I M and Milne, R W (Eds.) Research and Development m Expert Systems VIH Cambridge University Press, UK (1991) pp 52 66 Welbank, M 'A review of knowledge acqmsltion techtuques for expert systems' British Telecom Research Laboratories, Martlesham Heath, UK (1983) Markus, L Systems in Organlsattons Bugs and Features Pitman, USA (1984) Smith, D 'AI and the human EPROM' AI & Soc J Vol 1 No 2 (1987) pp 146-150 Compton, P, Horn, K, Quinlan, I, Lazarns, L and Ho, K 'Maintaining an expert system' /n Quinlan, I and Ross, J (Eft.) Apphcatlons of Expert Systems - - Vol H Addison-Wesley (1989) pp 366-384 Herrmann, F 'OCEX - - order clearing expert system' in Reference 1 pp 209-226 Milne, R 'Where is the business benefit?' m Business Benefits of Expert S.vstems British Computer Society (1990) p p l 7 Bobrow, D G and Stefik, M 'Knowledge based programmmg' Alvey News No 13 (1985) pp 13, 14 Wainwright, C 'A framework for artificial intelligence management' m Busmess Benefits of Expert Systems British Computer Society (1990) pp 62-79 Van de Brugg, A and Orciuch, E "Successful corporate strategy' m Graham, I M and Milne, R W (Eds.) Research and Development m Expert Systems VIII Cambridge University Press, U K (1991 ) Dutta, S 'A framework and methodology for enhancing the business impact of AI apphcatlons' Workmg Paper 90/68/TM Insead, France (1991 ) Tarr, S C 'Multiple perspectives analys~s for integrating technology into a business: a knowledge systems case study' Tech Forecasting & Soc Change Vol 40 (1991) pp 165 182 Harbridge, M "Expert systems how to identify suitable applications" es(connect), London, UK (1989) Lawrence, A 'Survey' In)Cormatws (Jan 1989) p 74

157