Hypermedia maintenance support applications: Benefits and development costs

Computers in Industry 56 (2005) 681–698 www.elsevier.com/locate/compind

Hypermedia maintenance support applications: Benefits and development costs Richard Crowder *, Gary Wills, Wendy Hall Intelligence Agent Multimedia Group, School of Electronics and Computer Science, Department of Electronics and Computer Science, University of Southampton, UK Received 15 April 2004; accepted 21 March 2005 Available online 20 July 2005

Abstract The most common questions asked by industrialists regarding any electronic based systems to support maintenance activities are, what are the benefits and how much will it cost. In this paper we will consider these questions by examining two hypermedia based maintenance support systems. In order to access the tangible and intangible benefits of hypermedia, the identification of benefits and cost are required at an early stage of the development process. This paper discusses some of the benefits to be gained by industry from using hypermedia applications to support their information provision. The cost is directly related to the effort required to produce a hypermedia application, with the greatest effort in authoring. Two methods of costing are presented: a detailed engineering approach and an approach using heuristics. In addition we will comment on how the presented approaches can be applied to Web and Semantic Web applications. # 2005 Elsevier B.V. All rights reserved. Keywords: Open Hypermedia; Manufacturing; Effort Analysis

1. Introduction Due to the complexity and sophistication of current industrial systems, there is normally a considerable amount of documentation that provides the instructions and technical information for their operation and maintenance. The scope of technical details needed to solve a particular problem often covers many different * Corresponding author. Tel.: +44 23 8059 3441; fax: +44 23 8059 2865. E-mail addresses: [email protected] (R. Crowder), [email protected] (G. Wills), [email protected] (W. Hall).

documents and requires constant cross-referencing, and involves time consuming and inconvenient searching and browsing. Use of these documents in the event of system failure is time-critical, [1]. The concept of using open hypermedia for information and document management within manufacturing organisations was proposed by Malcolm [2], who argued that hypermedia systems had to evolve beyond the stand alone application and allow the integration of resources across an enterprise. A number of hypermedia applications have been reported including engineering applications [1,3–8], medical record keeping [9], historical archives [10], and education [11].

0166-3615/$ – see front matter # 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.compind.2005.03.004

682

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

In our previous paper [12], we discussed the methodologies used to evaluate an industrial hypermedia application from the user’s viewpoint. This paper discusses how to estimate effort in producing such an application, and how benefits can be assessed. When discussing the introduction of an open hypermedia or hypertext system to an organisation’s management team, almost without fail the first question is what are the benefits to my organisation, followed closely by how much will it cost. The cost is directly related to the effort required to produce a hypermedia application, with the effort required being application specific and cost being application and organisation specific. The importance of realistic estimates of effort has generally been recognised by software developers as an essential requirement for the successful management of large software projects, and hypermedia applications are no exception. Organisations generally need to be able to describe projects in financial terms. The process of costing can in itself be time consuming. Therefore, in addition to the identification of detailed costing, developers need a robust method which gives reliable estimates of the effort required to produce a hypermedia application. Using realistic estimates at an early stage in a project’s life allows project managers and development organisations to manage resources effectively. Prediction—a necessary part of an effective process, be it authoring, design, testing, or development— requires the capture of data from similar past projects, leading to the development of prediction models for future applications. To date there appears to be little work on the development effort prediction for hypermedia applications; that which is published relates to Web applications. It can be argued that the relative novelty of these types of applications and the related data collection problems are the primary cause for the lack of robust approaches [13]. Accurate cost and effort estimations are an essential element in the production of a bid document at the beginning of a software project, [14]. It is also apparent that while effort estimation is important in web application, the take up of formal methods is relatively poor. Rollo [15] explores the issues of measuring effort relating to the size of Web applications using several approaches, all of which consider size from the perspective of functionality. Rollo discusses difficulties and contra-

dictions between these techniques and concludes, using empirical data, that the sizing method common software measurement international consortium-full function points (COSMIC-FFP) [16], is the most flexible approach for counting the functional size of Web sites and can be applied to any Web site. This work does not present a specific prediction model, but instead explores the measurement of functional size. Although the empirical data was obtained from measuring a Web site, this approach can also be used with hypermedia applications developed using other environments, e.g. Microcosm [17], Storyspace [18], and VIKI [19]. The work by Baresi [13] on estimating the design effort is based on projects developed by students using W2000, an object-orientated design notation similar to UML for Web applications. Mendes et al. [20,21] provide an overview of the approaches that can be used, classified as Expert Judgement, Algorithmic Models and Machine Learning. The approach considered in this paper aligns with their Algorithmic Models, where the effort is determined by the major characteristics of the application, in particular size and complexity. In the analysis of the benefits, we focus on the design and authoring processes. We adopt the classification proposed by Lowe and Hall ([22] p. 97) where authoring encompasses the management of activities for the actual content and structure of the application and its presentation. Design covers the methods used for generating the structure and functionality of the application, and typically does not include aspects, such as application requirements elicitation, feasibility consideration and applications maintenance. In addition, our design phase also incorporates the conceptual design of the application, reflected in map diagrams showing documents and links. The structure of the paper is as follows. We first describe the maintenance application used in Section 2, and then summarise in Section 3 the benefits to be gained from hypermedia applications within the maintenance and technical support area. Prior to discussing the approaches to estimation, Section 4 details an model of the authoring process. Section 5 presents a detailed method of estimating the effort of producing a hypermedia application, while Section 6 details an approach by which heuristics can be used to produce rough estimates. Section 7 considers the conversion of effort into cost. A comparison of the

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

approaches to effort estimation is given in Section 8. The paper concludes with a discussion on whether hypertext used to support maintenance is beneficial to an organisation.

2. Exemplar maintenance applications The industrial hypermedia maintenance applications considered in this paper were developed as part of the factory information resource management (FIRM) [5,23] and HELPMATE [24] projects. Both applications used Microcosm. Microcosm [17] is an open hypermedia system, where the links are stored separately from the documentation and through filtering processes are brought together when the user views the document. Microcosm has the concept of a generic link or glossary link, which is an association made from an object at any position in any document to a particular object in a destination document. In addition, compute links, which are dynamically computed when requested, and a guided tour that takes the user through a set of documents, are also features of Microcosm. 2.1. Factory information resource management The FIRM project moved away from a monolithic application to a modular approach for hypermedia development, by using modular hypermedia applica-

683

tions (MHA) [25]. The MHA approach is based on the principle that a large problem can be more easily managed by decomposing it into smaller modules. This approach can be applied to the information domain resulting in the large information space being broken down into smaller hypermedia applications, for ease of handling and authoring. An authoring environment then compiles the individual MHA into a single application for delivery. As previously reported [26], the application was developed for a supertension cable sheathing line at Pirelli, Eastleigh, UK. The line extrudes the crossedlinked polyethylene outer-sheath onto a supertension cable. In addition to the extruder, the line includes a bitumen tank, drive systems, cooling tank, spark tester and pay-off and take-up stands, Fig. 1. Due to the scope of the application, the development of the hypermedia application not only involved the operating company, but also the line builder and a number of line builders suppliers both in the UK and Europe. At the conclusion of the project, the information resource base comprised 850 nodes in 26 MHA, equating to over 640 Mbytes of information. Over 4500 links had been produced and were held in over seventy linkbases. Fig. 2 shows the wide range of material available to the user during a maintenance procedure. This includes not only the information specific to the line either in text or engineering drawings, but also access to company wide information on processes and

Fig. 1. Supertension sheathing line, showing the hypermedia system and its interconnection to external resources and the line.

684

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

Fig. 2. Typical maintenance information, including detailed operating instruction for a valve (right), spares list (centre), access to the store’s stock database (lower left) and line schematic(top left).

safety. One of the advantages of this approach is the ability to directly link from the list of spares to the company’s stores database, hence reducing the time and error associated with manually ordering a component. 2.2. HELPMATE Odense steel shipyards (OSS) build container ships and use a high level of automation in their design and production processes, including robotic welding systems. Their largest robotic welding system is designed to weld hull sections up to 32 m
22 m
6 m, and 400 tonnes in weight. The cell consists of 12 individual robot gantries, each with 8 d.o.f, suspended 17 m above the shop floor. The cell is capable of welding up to 2 km of steel per day. As OSS has only

one such cell, any failure will impact on the material flow through the yard, hence the requirement to minimise the time to rectify any fault. Any form of maintenance requires the application of knowledge held by people familiar with the system. The system was designed and built by OSS, and includes both custom designed and commercial robots. The cell is controlled by an OSS developed control package, which handles the planning and scheduling of the robot’s welding jobs and downloads the postprocessed robot program to the local robot controllers for execution. In addition, the control package also includes sub-systems that schedule and plan preventative maintenance activities. The controller has a fault detection capability, with the detected fault being communicated to the operator as a numeric code; however, it does not include any diagnostic

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

685

Fig. 3. Basic structure of the HELPMATE application, showing the relationship between the open hypermedia and CBR systems. The system can provide information both for browsing (via the open hypermedia system) and diagnostics via the CBR and open hypermedia systems.

capabilities. The resolution of the fault is performed by the operators and specialists. While some faults (e.g. replacement of welding wire) can be resolved with ease, more difficult faults (e.g. a drive or servo failure) will require information from system documentation, or in extreme cases discussion with the robot cell’s system specialist. Fig. 3 shows the basis structure of the application that has undergone evaluation at OSS. Users of HELPMATE [24] can obtain information from the system as result of browsing the information resource or during a diagnostic session. Information browsing does not require the use of the CBR system. CBR is a technology that helps to solve current problems by retrieving and adapting past situations or cases. A case can be defined as a contextual piece of knowledge representing an experience that teaches a lesson fundamental to achieving the goal of the reasoner [27]. A case can be considered to include three components:  A description of the problem. In the case of HELPMATE, the description can obtained from a number of sources, a Rob–Ex fault code, information relating to the current state of the welding cell, or its maintenance history.  The solution, or procedure to be followed to resolve the problem.

 The outcome, indicating how successful the procedure is in solving the problem. In a diagnostic process, the CBR system will retrieve the cases most similar to the current problem using a number of AI techniques. At the completion of the maintenance task, the user supplying the outcome, the case originally retrieved will be revised and retained for future use. It is this ability to adapt the cases to a maintenance history that makes the use of CBR so attractive to the operators and maintainers of this, and other large industrial systems. In HELPMATE, the initial set of cases were defined by capturing the OSS organisational memory. This was achieved by using existing procedures, past maintenance history, and the tacit knowledge of the robot specialist. Whilst the CBR technology provides the diagnosis, CBR does not provide any of the information required to undertake a repair. In HELPMATE, hypermedia linking is used to link the CBR results to the detailed descriptions of the required procedures. A small-scale system has been installed in order to obtain initial user feedback, and to let the operators gain confidence in the technology. It should be recognised that the users of these systems must have a degree of ownership. This can be achieved by actively encouraging the users to voice their concerns and

686

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

criticism, and feeding this back into the design process. The pilot application was based on two particular error codes; these gave a total of 109 cases. It has been estimated that the pilot contained less than 10% of the cases and documentation of the final version of HELPMATE currently under development. However, even with the small size, it proved to be of value, both to prove the concepts, and to introduce the users at OSS to the concepts of using CBR and hypermedia within their normal activities. In a typical session, the operator is presented with two options: to use the system as an information resource or to follow a diagnostic procedure. Using it as an information resource, the operator is guided to an index page, and by following links can navigate the resource to locate the information. When undertaking a diagnostic procedure, the operator follows a question and answer dialogue session, using a simple user interface, Fig. 4. The interface has been designed using the ‘select and click’ principle. Once the error code has been selected, selection of ‘OK’ takes the user to the next stage in the case. If the question can not be easily answered, selection of the ‘?’ presents the operator with a range of options, with corresponding probabilities. The most probable solution, based on the operator’s knowledge of the problem, can be selected. At the end of the diagnosis process, a point is reached where the solution has been determined, Fig. 4. A link then takes the user to the open hypermedia resource base, allowing the maintenance information to be retrieved as in FIRM.

3. Benefits from using hypermedia in the maintenance process The introduction of hypermedia systems, such as FIRM and HELPMATE should be seen in the context of an overall competitive strategy. A number of benefits resulting from introducing a hypermedia application within an industrial environment have been defined [4,5]. Evaluation of industrial hypermedia systems on the factory floor have shown that users can find information much more easily and quickly using a hypermedia system [28]. While the benefits can be easily identified, assigning a monetary value to the benefits is not so straight forward. Laudon et al. [29] classified the benefits of an information system as either,

 Tangible Benefits — those that can be quantified and assigned a monetary value.  Intangible Benefits — those where assigning a monetary value can be difficult. The benefits that are associated with hypermedia information systems used on the factory floor are shown in Table 1. These will of course only become benefits if they exceed the capital and ongoing cost of the application. The costs involved in producing an industrial hypermedia application are typically application authoring, including document collection and conversion, provision of hardware and software, and operator training. The work by Ginige et al. [30] has suggested the largest individual cost in the provision of a hypermedia application is authoring. We have identified that the benefits of a hypermedia system, when compared to a paper-based system, come directly from electronic information provision, which is then enhanced by the use of hyperlinks. In itself, hypermedia will not absolve the organisation from requiring change, focusing on customer needs, improving the quality of its products or services and continuing to develop new ones. Hogbin and Thomas [31] have also put forward a number of business principles and technology trends that suggest that investment in information technology may be beneficial. One of the more important benefits from investment in hypermedia systems, and yet one of the most difficult to quantify, relates to the organisation’s market position and the income generated from that position. Investments which improve the quality of service to the customer, internal and external, or support new products or service may lead to an increase in market share or at least maintain it against strong competition. All change will involve cost but there is also a cost by not changing. One of the key factors in understanding the cost is the ability to accurately determine the authoring effort.

4. Modelling the authoring activity We believe that most the suitable approach to authoring hypermedia for the engineering environment is an information based one [22]. In this approach the content, and to a lesser extent the structure, of the application is obtained from existing information.

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

687

Fig. 4. Screenshots of the HELPMATE application in use. (a) The user starts the process by entering the diagnostic code provided by the controller. (b) The diagnostic screen from which the user can select the required information, and update the CBR case base.

688

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

Table 1 Benefits of using hypermedia to support factory floor maintenance Tangible benefits Saving time locating information. Not having to publish and distribute paper copies. Customers or individual departments not having to control paper documents. Reduced staff effort in documentation control. Intangible benefits Locating information more easily. Ensuring that users have the correct issue of the information appropriate to the task at all times. Reducting errors due to easier cross-referencing between different information systems. Providing and linking to additional useful information, where appropriate. Hyperlinks allow quick and effective cross-referencing. Opportunity cost (or time), in being able to do something else. Greater customer satisfaction and improved corporate image. Help close the quality loop for many of the tasks performed by the operators. Automatic data manipulation and dissemination. Easier access to information from any point in the factory environment. Improved quality of the document control system. Ensuring documents do not go missing. The use of templates allows consistency of documents. The legibility of the documents is improved. The ability to capture and share information gained by experience.

The approach taken in FIRM and HELPMATE was to use open hypermedia. In open hypermedia the links are held separatly from the content and only at viewing time are they presented together. To determine which hyperlinks are to be inserted into the document, there is a spectrum of approaches from simple lexical matching of terms to a more sophisticated approach of user modelling and ontological matching. Which approach is used for determining the hyperlinks to use is not relevant to the cost model discussed in this paper, as the model only records the effort required by authors to determine which hyperlinks are necessary. Therefore, the model is relevant when estimating the effort of any application that involves hyperlinks. For example, it could be a corporate Health and Safety Website, which is developed using the traditional button links of the World Wide Web (often referred to as static hyperlinks). Alternatively, it could be a multimedia manual for a drive unit which uses adaptive hyperlinking to present

the different service options, depending on which variant of the drive is installed [32]. 4.1. Process model As authoring is the largest single cost when developing a hypermedia application, an ability to quantify the cost of authoring an application is required. In order to estimate the effort for authoring an industrial hypermedia while using a conventional authoring methodology [25,33], it is essential to first understand the process involved. An effective way of achieving this is to model the process. The complete process model for authoring an industrial hypermedia application is shown in Fig. 5. As part of the process model, the application structure needs to be considered. Typically one of two approaches can be taken: either a single monolithic application, or one based on smaller modules that are complied to give the complete application. In Microcosm [17], used for the applications discussed in this paper, an application is defined to be a collection of links, documents and processes that, when operated together, can be considered an information system. This model can be extended to also include a set of child modular hypermedia applications (MHA) that provide additional information, [25,33]. Each MHA is considered to be selfcontained (i.e. it does not contain explicit references to documents that are not a part of the MHA). The benefits of using MHAs are:  Once an MHA has been developed, it can then be used in many different contexts.  The modules can be managed centrally, which helps increase the maintainability and quality of the final application.  The authoring effort required for building applications becomes less, as they can be built largely from pre-existing information modules. 4.2. Modelling the learning curve In the calculation to estimate effort it is important to model the effects on performance of undertaking repetitive tasks. When people are first introduced to a task, they frequently take longer to perform it than when they have repeated the task a number of times,

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

689

Fig. 5. Authoring Process Model.

known as the learning-curve effect [34]. The learning curve effect can only be applied to direct labour or variable overheads that are directly effected by labour effort, and is expressed by T ¼ aX b

(1)

where T is the cumulative average of the time required to carry out the task X times and a is the time required to carry out the task the first time. The exponential, b, is defined as b¼

ln ðlearning rateÞ ln 2

(2)

and will range between 1 and 0. The learning curve is based on real world observations and hence the relationships described are empirical [35]. The learning rate can vary between 65 and 90% in the early stages of production, and levels out to reach a steady state. The learning curve can be applied to each of the tasks identified in the process model. Using a different number of people will not effect Eq. (1). Therefore the more people assigned to the task, the fewer items they individually process and hence the time they require to carry out the task will be less affected by the learning curve. In practice common sense is required here, as it is not possible to allocate more people to a task than there is

690

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

equipment available. The simplest approach is to divide the total number of times that an individual activity has to undertaken during a process by the number of people directly involved in the process. If Pis the number of people involved, the time will become TP ¼ aX b Pb

(3)

This will have the effect of slightly increasing the total effort time allowing for individual learning. Similarly, there is a limit to the learning rate, due to the physical limitations of the equipment and operators.

5. Engineering approach to effort estimation Using information provided by the process model, the effort required can be estimated. The method discussed in this section is termed the engineering method [35], as this is a detailed estimate based on the sum of the individual efforts from each activity in the process. The total authoring effort will be the sum of the efforts to produce the individual MHA in an application. It should be recognised that the default number of MHAs is one, when a single monolithic application is being considered. The individual activities that need to be considered are:      

Conversion of paper legacy documents (EP ). Conversion of electronic legacy documents (Ee ). Authoring electronic documents (En ). Creating the structural links (Es ). Creating cognitive links (Ec ). Maintaining records, reports and statistics (Em ).

The individual efforts discussed below allow the effort required to author a monolithic application or individual hypermedia module (MHA) to be estimated. In all cases the total time to complete an activity is modified by the learning curve discussed in Section 4.2. 5.1. Paper legacy documents The main effort with existing paper documentation is converting the information into a suitable

Table 2 Activities associated with converting paper documents into a suitable format for hypermedia authoring Activity (i)

Description

1 2

Collecting the information to be converted Converting the information from paper to a standard raster format Cleaning-up the information after conversion Processing the information into a vector format Saving the information with appropriate file names and file hierarchy

3 4 5

electronic format. This task can be sub-divided into five activities, Table 2. In practice while the processes are the same, there is a difference between processing text or images (including engineering drawings). As some documents only need to be in the raster format supplied by the conversion process, while others will need to be converted into a vector format, for example ASCII text using OCR techniques, the number of documents going through individual activities will differ for a specific application. Paper documents that require retyping, drawings that need to be electronically redrafted, or forms converted to electronic data entry, can be classified as new electronic documents and the required effort is calculated using the algorithm presented in Section 5.2. For some large applications the conversion process can be sub-contracted to any number of companies that offer a bureau service. These companies will also do some of the post-conversion processing. This results in a reduction of in-house effort; however the cost must be taken into account when computing the benefits. The effort required to process the paper legacy documents (EP ) is the sum of the effort for gathering, converting, correcting, and processing the paper information, and depend on:  The equivalent number of A4 sheets of paper per document set that need to be processed (S). For sheets larger than A4, the equivalent is used, (e.g. A3 ¼ 2
A4).  The average time taken to perform the individual activity per A4 sheet (Ti ), dependent on the document type.

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

The effort is expressed as
X 
X  ½STi  þ ½STi  Ep ¼ i¼15

images

i¼15

(4)

text

The number of documents in each activity may not be the same, for instance not all diagrams may be converted to a vector format, as sometimes the raster image may be sufficient for their use. 5.2. Electronic legacy documents The legacy electronic information requires a common look and feel. In addition, the majority of existing electronic information has been written for the paper paradigm, therefore the reader only obtains the full benefits of the document layout when the document is printed. Hence essential documents need to conform to the templates created by the designers for the appropriate electronic format. This will not only ensure that they are readable in the electronic format but also aid link generation and screen management. In addition, the effort required to ensure that the converted paper documents conform to the required standard must be included within this calculation. Another part of the effort required would be to dissect long documents into information nodes and assign meaningful labels to each node. The activities that need to be undertaken to process existing electronic information are summarised in Table 3. The effort required to process the existing electronic (Ee ) information as,
X 
X  Ee ¼ ½DT j  þ ½DT j  j¼12

documents

j¼12

images

(5) where D is the number of existing electronic documents, and T j is the average time taken to perform the Table 3 List of activities associated with processing existing electronic documents into a suitable format Activity (j)

Description

1

Conversion of electronic documents to standard templates Division of long documents into appropriate length

2

691

activity. It is assumed that the length of documents are statistically normally distributed, and hence the average size can be used to calculate the effort required. If this is not the case then a separate time for the average length of each different type of document must be calculated. 5.3. Authoring electronic information A significant amount of information within the working environment is held by the organisation and needs to be incorporated into the application. To process this information, it first has to be collected, then entered into the system. The effort will vary with the information type (i.e. straight text or engineering drawings) and the number of pieces of information of that type. Let Tn be the average time taken to locate the required information and T the authoring time. Hence if N is the number of new nodes that are required to be created then the authoring effort required, to produce the new information (En ) is X En ¼ N½Tn þ T (6) for all media

This relationship can also be used to include any paper documents that are retyped, drawings that are electronically redrafted, and forms that are converted to electronic data entry. 5.4. Structural links Once the information is in the correct format, linking may begin. The first and easiest is the structural linking, as this can be considered to be an administrative task. This should be an automated process especially where the documents conform to the standard templates. The structural linking is carried out usually on a document or set of nodes, hence the effort required is, on average, the same irrespective of the size of the document, and is only dependent on the number of documents, Dn . Ta is the average time taken to generate the links automatically. In addition, effort is required to supervise and organise the process, and to create the structural links. The manual linking is required to link menus and other small groups of documents that do not conform to any standard templates and depends on the number of links (Ls ) required. Tm is the average time taken to produce

692

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

the links manually. In addition, effort is required to make a number of link databases (Bs ) for both the manual and automatic linking processes, and TL is the average time taken to make the link databases. All documents require registering with the hypermedia document management system and grouped into a number of MHAs (Tmha ). The effort can therefore be expressed as Es ¼ ½Dn Ta  þ ½Ls Tm  þ ½Bs TL  þ Tmha

(7)

An estimate of the number of structural links, Ls , within hypermedia module or application, can be determined from the number of:    

Nodes, figures, and tables. Items in indexes and menus. References to external information. Items on a drawing and the reference to specifications.  Items on the material lists multiplied by the number of related documents. For example, items on a material list will be linked to at least one drawing and a specification. 5.5. Concept linking

Concept linking is required to link various concepts and ideas in the document set that can not be determined by a mechanistic approach. The effort requires the involvement of experts from the appropriate disciplines and departments identifying and making the associations, or in the case of HELPMATE, the initial population of the case base. The effort will be dependent on the number of links (Lc ) required to be made and the number of link databases (Bc ) required. In addition, included in this effort is the time taken to produce the link clusters (Llc ) as discussed in [5]. TL , Tc and Tlc are the average times taken to perform these respective activities. The cognitive linking effort can be expressed as Ec ¼ ½Lc TL  þ ½Bc Tc  þ ½Lc Tlc 

(8)

It should be recognised that the reduction in concept linking is possible by the introduction of well defined ontologies and natural language processing of the documents being incorporated into the application.

5.6. Administration effort In many applications it is essential that the documentation audit trail is maintained for many reasons. Hence, all documents that have been changed or created must be recorded in the company’s document control system Nc is defined as the number of nodes changed. If the effort model is to be managed effectively it will need to be monitored and actual times compared with forecast times, in order to keep it within budget. The requirement thus is to record statistical information, and is represented by the number of statistical records kept, Ns . In addition, it is essential, for auditing purposes, to report on the progress of the process with R being the number of reports required. Tr , Ts and Trp are the cumulative average times taken to perform these respective activities. The effort is expressed as Em ¼ ½Nc Tr  þ ½Ns Ts  þ ½RTrp 

(9)

5.7. Total effort The total effort for authoring any form of hypermedia entity, ranging from a single MHA to a monolithic application is given by E ¼ Ep þ E e þ E n þ E s þ E c þ E m

(10)

If this is a hypermedia module, the total effort for the application will be the sum of the individual modules, plus effort required for complication and integration, which will be additional to the linking effort.

6. Using heuristics to estimate effort The method of calculating the effort, as discussed above, is very detailed. However, the process of carrying out the detailed estimating itself requires effort and therefore is an overhead. In addition, once the effort of a number of hypermedia applications has been costed and verified, it is then possible to use the data to estimate the effort required to produce similar sized applications. This type of effort and costing is referred to as historical cost-based estimation [35]. A hypermedia application is a complex entity to define accurately. It is not just a matter of the number

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

of documents and links, but also the way in which the application is linked together and the user guided through the application. Botafogo et al. [36] defines the amount of cross-referencing as compactness. Their method of calculation is applicable only to completed applications. In addition, Botafogo assumed bidirectional links and did not allowed for the concept of generic links or link clusters. Mendes [37] has demonstrated that the larger the hyperdocument size, or the greater the connectivity (number of links), or the higher the compactness, the more time consuming the development of an application. However, while she showed a correlation, Mendes was unable to qualify this by giving a relationship for size/complexity to effort. 6.1. Estimating the variables The size of a hypermedia module (MHA) is usually much smaller than the final hypermedia application. To ensure accuracy, comparisons should be made between MHAs rather than complete applications. It is not advisable to compare a similar sized MHA consisting of background information, with one consisting of drawings and manufacturing information, as generally the latter will have a much higher level of cross-referencing. The first step is to identify those variables that will describe the application accurately enough for a realistic estimate of the effort required. The important variables to be estimated include the number of nodes, and the compactness of the linking between individual notes. 6.1.1. Number and type of nodes requiring effort In the case of electronic information that is already in the style required, the number of nodes is the number of files. However, where the information is in a non-electronic format or of an incorrect style the number of nodes can be estimated by adding together  The number of indexes, including menus, drawing lists, parts lists, etc.  The number of drawings, videos, sound files, etc.  The number of manuals, then multiplying this by the number of chapters and the number of chapter sections.

693

Table 4 Times based on historical data, to produce specific node types Task

Time (min)

Authoring video material Scan text documents a Convert WP documents Scan images and drawingsa CAD productiona Convert drawings New lists and menus Integration, per MHA Production of thumbnail images Production of guided tours Production of link databases

120 30 12 27 60 15 24 15 12 30 12

a

Per A4 equivalent.

 The number of instruction sheets multiplied by the number of pages, (the instruction sheets include set-up procedures, check list and other loose-leaf instruction sheets). From a hypermedia application, it is quite simple to obtain the number of nodes, the number of structural links, compactness, complexity and time taken to complete. Based on the historical data gathered while completing previous applications, the time required to produce the different types of nodes can be obtained, see Table 4. The actual values used in this table were based on the data gathered while producing the FIRM applications. 6.1.2. Compactness of structural links The compactness can be estimated as the number of links per node. Estimating the actual number of links can be time-consuming. Hence, a simple weighting system can be used to estimate the compactness, see Table 5, giving L¼1þ

4 X Xi

(11)

i¼1

The possible values for the link factor weighting ranges from 1 to 9, representing a low compactness to a very highly compact MHA. 6.2. Estimating the effort An approximation of the time to author a particular hypermedia application or individual MHA can

694

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

Table 5 Weighting for linking factor used to represent link compactness i

Linking type X

Non

Partial

Complete

1 2 3 4

Structural linking to figures, tables, indexes, references to external documents, etc. Items on a drawing (embedded and external) referred to in the text linked Linking to a a diagram or list (one link per item) Linking to diagram or list (more than one link per item)

0 0 0 0

1 1 1 1

2 2 2 2

obtained by multiplying the number and types of hypermedia nodes by the appropriate heuristic times (Table 4) and then by the linking factor associated with the link compactness, (Table 5).

7. Cost of authoring Once the effort required has been calculated, then the cost of authoring the application can be estimated. The total cost of the authoring process, is given by cost ¼ Or ½Ce E þ Cext 

(12)

Where E is defined in Eq. (10) or determined by the heuristic approach, Or the overhead rate, Cext any external cost incurred (e.g. scanning bureau) and Ce is the cost of employment. Within the literature, there are estimates of effort and cost for design of multimedia applications [38], but not for industrial hypermedia. The obvious difference, between true multimedia and hypermedia Table 6 Comparison of the estimated effort for Exemplar application

1 2 3 3 4 5 6 7 8 9 10 11 12 13 14 15

MHAs

in the

FIRMapplication

or web based applications, is that with the latter the emphasis is on the linking aspects of the production. The method of allocating costs will vary between organisations. However, the basic procedure is to take the effort and multiply it by the cost of employment. From the process model, Fig. 5, in additional to effort, there are additional costs from: site services, materials and capital equipment required to undertake the process. The cost of the information audit should be included. The rationale is that the effort required to perform the audit is an integral part of the authoring methodology and has a direct effect on the efficiency of the authoring process. A poor audit will result in greater effort in the authoring process. The times given in Table 6 were determined from the FIRM. The effort to author the application, including the information collection, totalled over 530 h. The application authoring was undertaken by one person and all that work was undertaken in-house. It is assumed that the design phase and information

produced by the Engineering and Heuristic approaches

Heuristic Time

Linking Factor

Total hours

18.25 14.57 13.41 10.48 8.57 7.28 6.02 4.62 3.38 3.04 2.51 2.24 1.62 1.42 1.2 0.9

3 2 4 2 4 3 4 2 3 3 5 9 5 1 2 1

54.75 29.14 53.64 20.96 34.28 21.84 24.08 9.24 10.14 9.12 12.55 20.16 8.1 1.42 2.4 0.9

Engineering, total hours

Difference

54.50 28.42 56.12 23.70 34.79 20.23 27.96 9.48 7.45 8.15 10.63 17.32 7.21 2.78 2.77 2.13

0.25 0.72 2.48 2.74 0.51 1.61 3.88 0.24 2.69 0.97 1.92 2.84 0.89 1.36 0.37 1.23

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

audit had identified all the information required and the file structures required to store the information. Based on these pessimistic views, the company’s cost benefits analysis showed that the introduction of this large-scale industrial hypermedia application will pay for itself, in the normal course of events, in 2 years [23]. This estimate is based on normal preventative maintenance processes. It excludes the benefits that could occur if there was a single catastrophic failure that required in-depth fault finding. This savings in down time and other costs associated with this single event could pay for the complete application. While allocating cost to tangible benefits is relative straightforward, this is not the case with intangible benefits, especially those based on perception, where judgement and experience rather than hard facts are required. As allocating costs to benefits can be a difficult process, in practice the minimum payback period could shorter.

8. Comparison of methods Table 6 shows a comparison in effort for 15 separate applications, when computed by both the engineering and heuristic approaches. The results were plotted on a scatter diagram and a best-fit curve can be fitted using linear regression, Fig. 6. The analysis of the variance showed a coefficient of determination (R2 ) of 0.976, this shows that there is a strong positive correlation between the estimate

Fig. 6. Comparison of estimated times using the engineering and heuristic approach.

695

Table 7 Comparison between the actual and estimated effort, using the heuristic approach, for the HELPMATE application; time is in minutes Media

Scanned pages Electronic documents CAD drawings Images Total

Time for activity Actual

Heuristic

3500 450 550 525 5025

3000 300 600 450 4470

Difference (%) +16 6 +8 16 12

using heuristic and the detailed estimate using actual times. To further validate the heuristic model a comparison was undertaken using the HELPMATE application, Table 7. The estimated time was determined to be 12% less than the actual measured values, given the application this was considered acceptable. In practice the times may have to be varied due to external variables. In the case of HELPMATE, there were language problems (the application was in Danish, the information in Japanese and the developers based in Southampton), and the document quality was poor.

9. Conclusions Costing the development of hypermedia applications depends very much on the accounting method used. However, what is generally common to different applications is the effort required in producing them. We have presented an engineering model for calculating the effort required to author a hypermedia application, as authoring requires the largest effort when producting a hypermedia application [30]. By its very nature the engineering method of cost modelling is very detailed, and hence we have developed a complementary heuristic based model that will allow the production an acceptable estimation of the effort and hence costs involved. In modelling the effort spent during the authoring process, we are able to give an accurate cost for developing any hypermedia or similar application. These costs can be weighed against the tangible and intangible benefits to decide whether or not to proceed with a specific project. Our experience of working

696

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

with large industrial organisations within the Europe, is that many use closed hypermedia systems (embedded links) on hand-crafted sites, and as these sites grow they suffer from link integrity problems [39]. This often cancels the perceived benefits of using open hypermedia to aid navigation and retrieval of appropriate documents. So, in reality there are other factors to consider, and it must be recognised that while the majority of problems to be overcome, when introducing an industrial open hypermedia system, are technical, some of the most difficult relate to the inertia of the management culture within the organisation. Hence, this paper has examined the benefits of introducing industrial strength hypermedia. Based on the authoring methodology presented in [25], a model that allows an estimation of the effort or time to produce the hypermedia application is presented. The actual figures and paths through the model will depend largely on the decision made by the team responsible for the authoring process. At present, the general interest in the use of open hypermedia systems to industrial tasks is cautious, as this technology is perceived as ‘new’, although it has been commercially available for a number of years. However, this attitude is changing and the perceived risk associated with using hypermedia technology is reducing, especially with the increase use of the World Wide Web, online documentation, and e-commerce. Therefore, as the need for managing and navigating the information in an industrial environment increases, the demand for industrial hypermedia will be increased. Industrial hypermedia is not a panacea to all of industry’s information management problems. It is, however, an effective weapon in the armoury for reducing information overload or information deprivation. While there is still an enormous amount of research to be undertaken in the field of industrial hypermedia, this research has endeavoured to resolve some of the fundamental issues facing the acceptance of hypermedia in industry.

Acknowledgements The contribution to this research by the staff and management at Pirelli Cables, Eastleigh, The Ford Motor Company, Southampton and, OSS, Odense, Denmark is acknowledged for allowing their personnel

to take part in the trials. The authors acknowledge the Engineering and Physical Science Research Council for funding FIRM under grant number GR/L/10482 and the European Commission for funding HELPMATE under ESPRIT IV, project 25282. The authors also acknowledge the contribution made to this paper by Dr. I Heath, their colleagues at IT Innovation, and Mr Q Gee.

References [1] L. Hsu, P. Liu, T. Dawidowsky, A mutlimedia authoring-in-thelarge enviroment to support complex product docuaments, Multimedai Tools and Applications 8 (1) (1999) 11–64. [2] K. Malcolm, S. Poltrock, D. Schuler, Industrial strength hypermedia: requirements for a large engineering enterprise, in: Proceedings of the Hypertext ’91, Seattle, 1991, pp. 13–24. [3] R. Greenough, D. Fakun, Innovative information system to support team-based maintenance, in: Proceedings of the Third International Conference on Managing Innovation in Manufacture, University of Nottingham, 1998, pp. 323–328. [4] R. Crowder, W. Hall, I. Heath, R. Bernard, D. Gaskell, A hypermedia maintenance information system, IEE Computing and Control Engineering Journal 7 (3) (1996) 121–128. [5] R. Crowder, W. Hall, I. Heath, G. Wills, Factory information provision using hypermedia, International Journal of Computer Applications in Technology 11 (1998) 442–453. [6] R. Crowder, W. Hall, I. Heath, G. Wills, Industrial strength hypermedia: design, implementation and application, International Journal of Computer Integrated Manufacturing 13 (3) (2000) 173–186. [7] D. Fakun, R. Greenough, User-interface design heuristics for developing usable industrial hypermedia applications, Human Factors and Ergonomics in Manufacturing 12 (2) (2002) 127– 149. [8] R.M. Newman, Designing hypermedia documentation for safety critical applications, in: Proceedings of the International Conference on Coding and Computing, IEEE, 2000, pp. 247– 252. [9] H. Ip, K. Law, S. Chan, An open framework for a multimedia medical document system, Journal of Microcomputer applications 18 (1995) 215–232. [10] A. Dunlop, M. Papiani, A. Hey, Providing access to a multimedia archive using the World Wide Web and an objectrelational database management system, IEE Computing and Control 7 (5) (1996) 221–226. [11] P. De Bra, Pros and cons of adaptive hypermedia in web-based education, Cyberpsychology and Behavior 3 (1) (2000) 71–77. [12] R. Crowder, G. Wills, W. Hall, Evaluation of a hypermedia maintenance support application, Computers in Industry 51 (2003) 327–344. [13] L. Baresi, S. Morasca, P. Paolin, Estimating the design effort of web applications, in: Proceedings of the Ninth Software

R. Crowder et al. / Computers in Industry 56 (2005) 681–698

[14]

[15] [16] [17]

[18]

[19]

[20]

[21] [22] [23]

[24]

[25]

[26]

[27] [28]

[29]

[30] [31]

[32]

Metrics Synposium (METRICS 2003), Sydney, Australia, 2003, pp. 62–72. M. Ruhe, R. Jeffery, I. Wieczorek, Cost estimation for web applications, in: Proceedings. of the 25th International Conference on Software Engineering, 2003, pp. 285–294. T. Rollo, Sizing e-commerce, in: Proceedings of the Australian Conference on Software Measurement (ACOSM 2000), 2000. COSMIC-FFP measurement manual, available at http:// www.cosmicon.com, Version 2, 1999. H. Davis, W. Hall, I. Heath, G.H.R. Wilkins, Towards an integrated environment with open hypermedia systems, in: Proceedings of the ACM Conference on Hypertext, EHCT 1992, Milan, Italy, 1992, pp. 181–190. A. Dirberger, J. Bolter, On the design of hyper spaces, Communications of the ACM, Special Issue on Hypermedia Design 38 (8) (1995) 98. C. Marshall, F. Shipman, J. Coombs III, VIKI: spatial hypertext supporting emergent structure, in: Proceedings of the ECHT 1994, ACM, Edinburgh, UK, 1994, pp. 18–23. E. Mendes, I. Watson, C. Triggs, N. Mosley, S. Counsell, A comparison of development effort estimation techniques for web hypermedia applications, in: Proceedings of the Eighth IEEE Symposium on Software Metrics (METRICS 2002), 2002, pp. 131–140. E. Mendes, N. Mosley, S. Counsell, Web metrics — estimating design and authoring effort, IEEE Multimedia (2001) 50–57. D. Lowe, W. Hall, Hypermedia Engineering: The Web and Beyond, Wiley, 1998. G. Wills, Design and evaluation of industrial hypermedia, Ph.D. Thesis, Department of Electrical Engineering, University of Southampton, 2000. E. Auriol, R. Crowder, R. McKendrick, R. Rowe, T. Knudsen, Integrating case-based resoning and hypermedia documentation: an application for the diagnosis of a welding robot at odense steel shipyard, Engineering Applications of Artificial Intelligence 12 (1999) 691–703. I. Heath, W. Hall, R. Crowder, G. Wills, J. Ballantyne, A new authoring methodology for large-scale hypermedia applications, Multimedia Tools and Applications 12 (2-3) (2000) 129– 144. R. Crowder, W. Hall, I. Heath, G. Wills, Integration of manufacturing information using open hypermedia, Computers in Industry 38 (1) (1999) 31–42. J. Kolder, Case Based Reasoning, Morgan Kaufmann, 1966. G. Wills, Y. Sim, R. Crowder, W. Hall, Open hypermedia for product support, International Journal of Systems Science 33 (6) (2002) 421–432. K. Laudon, J. Laudon, Management Information Systems: New Approach to Organisation and Technology, Prentice Hall International, 1998. A. Ginige, D. Lowe, J. Robertson, Hypermedia authoring, IEEE Multimedia 2 (3) (1995) 24–35. G. Hogbin, D. Thomas, Investing in Information Technology, Managing the Decision-Making Process, IBM McGraw-Hill Series, 1994. R. Crowder, Y. Sim, G. Wills, R. Greenough, A review of the benefits of using hypermedia manuals, in: Proceedings of the

[33]

[34] [35] [36]

[37]

[38]

[39]

697

Twelfth ACM conferences on Hypertext and Hypermedia, Arhus, Denmark, 2001, pp. 245–246. I. Heath, W. Hall, R. Crowder, G. Wills, The application of a hypermedia research system in industry, in: Proceedings of the Tenth ACM conference on Hypertext and Hypermedia, Technical Briefing, Darmstadt, Germany, 1999, pp. 203–204. C. Dury, Management and Cost Accounting, 4th ed., Thomson, 1996. J. Arnold, T. Hope, Accounting for Management Decision, 2nd ed., Prentice Hall, 1990. R. Botafogo, E. Rivlin, B. Shneiderman, Structural analysis of hypertexts: Identifying hierarchies and useful metrics, ACM TOIS 10 (2) (1992) 143–179. M. Mendes, Metrics for improving the quality of hypermedia authoring, Ph.D. Thesis, Department of Electronics and Computer Science, University of Southampton, 1999. S. Lechtenberg, G. Joubert, Effort estimation for multimedia information systems development, in: J.-Y. Roger, B. StanfordSmith, P. Kidd (Eds.), Technologies for the Information Society: Developments and Opportunities, IOS Press, 1998, pp. 615–622. H. Davis, Data integrity problems in an open hypermedia link, Ph.D. Thesis, University of Southampton, 1995.

Richard Crowder is currently Senior Lecturer in Robotics and Control in the School of Electronics and Computer Science, University of Southampton. He joined the Intelligence Agents Multimedia Group to develop his interests in the industrial applications of knowledge technologies. He currently is undertaking research into the use of such technologies within the Engineering Design Environment. He obtained his PhD in Electrical Engineering from the University of Leicester in 1977. Prior to joining the University of Southampton, he worked in the machine tool industry, specialising in high-performance systems for the aerospace industry. He has addition research interests in biologically inspired robotics, with particular reference to manipulation. Richard has authored approximately 90 refereed journal and conference papers. He has supervised a number of successful PhD candidates. He is a Chartered Engineer and a Member of the Institution of Electrical Engineers. Gary Wills is currently a lecturer in the Intelligence Agents Multimedia Group within the School of Electronics and Computer Science, University of Southampton. He holds a PhD for research into the Design and Evaluation of Industrial Hypermedia from the University of Southampton. His current research interests relate to personal knowledge environments for industrial, educational and medical domains. He has published over forty academic papers in this area. He also is a Chartered Engineer and a Member of the Institution of Electrical Engineers.

698

R. Crowder et al. / Computers in Industry 56 (2005) 681–698 Wendy Hall is Professor of Computer Science at the University of Southampton, UK and is currently Head of the School of Electronics and Computer Science (ECS). She was the founding Head of the Intelligence Agents Multimedia (IAM) Group in ECS. She is the co-author of the book Hypermedia and the Web: an Engineering Approach (Wiley, 1999) and has published over

300 papers in areas such as hypermedia, multimedia, digital libraries, multi-agent systems and knowledge technologies. She is currently Immediate Past President of the British Computer Society, a member of the Prime Minister’s Council for Science and Technology, a member of IW3C2, a Member of the Executive Committee of UKCRC and a Non-executive Director of several companies and charitable trusts. She was awarded a CBE in 2000, and is a Fellow of the BCS, the IEE, the Royal Academy of Engineering and the City and Guilds of London Institute.