- Email: [email protected]
Designing reliability information flows
Reliability Engineering and System Safety 88 (2005) 147–155 www.elsevier.com/locate/ress
Designing reliability information flows Valia T. Petkova*, Lu Yuan, Roxana A. Ion, Peter C. Sander Faculty of Technology Management/Sub-department Product and Process Quality, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands Available online 25 August 2004
Abstract It is well-known [Reliab. Eng. Syst. Saf. 75 (2002) 295] that in modern development processes it is essential to have an information flow structure that facilitates fast feedback from product users (customers) to departments at the front end, in particular development and production. As information is only relevant if it is used when taking decisions, this paper presents a guideline for building field feedback information flows that facilitate the decision taking during the product creation and realisation process. The guideline takes into consideration that the type of decisions depends on the span-of-control, therefore following Parsons [Structure and Process in Modern Societies (1990)] the span-of-control is subdivided into the following three levels: strategic, tactic, and executive. The guideline is illustrated with a case in which it is used for analysing the quality of existing field feedback flows. q 2004 Elsevier Ltd. All rights reserved. Keywords: Fast Field Feedback; Information flow; Product development
1. Introduction In the literature, there has been quite some attention to the vital role of information flows in situations where a high degree of innovation forces manufacturers to speed up product creation processes (PCPs1) to the limit [3,4]. The need for a strong emphasis on fast and effective information loops, within a PCP as well as between a PCP and the customer,2 emerges from four trends currently dominating (electronics) industry [5]: - Due to recent advances in (digital electronics and software) technology, increasingly complex products are put on the market. - Technological innovations put a pressure on the timeto-market. * Corresponding author. E-mail address: [email protected] (V.T. Petkova). 1 PCP—a process that systematically transforms new product ideas into a set of products. 2 Customer—a person who uses a product that he did not produce himself. 0951-8320/$ - see front matter q 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.ress.2004.07.004
- Due to the increasing globalisation, more and more PCPs are divided over partners in different parts of the world, e.g. design, engineering, suppliers, production, assembly. This requires efficient information and communication systems. - Customers expect very complex products to work according to their (customers’) requirements, regardless of the product specifications. Brombacher [6] presented an interesting idea, the maturity index of reliability (MIR), to judge quality and reliability oriented information flows. The authors have contributed to this research via a series of papers concentrating on the relation between information flows and product reliability aspects [7–9]. Although the MIRconcept is a relevant contribution to the literature, it has some important weaknesses [10]. This is explained in Section 3. The aim of this paper is to give a guideline for constructing information flows. A side effect of such a guideline is that it can be used as an evaluation tool for existing information flows. To be more precise, the focus of
148
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
this paper is on the construction of field feedback information flows concentrating on the quality and reliability (shortly denoted by quality) aspects of innovative,3 electronic consumer products. We concentrate on field feedback4 because for innovative products, it is not always clear how a customer will use them. Field feedback is the ultimate proof that the product does or does not fulfil customer desires. The function of an information flow is to bring the required information at the right time to the person who is expected to choose between different options. So, we explicitly couple information with decisions. This paper concentrates on product quality and therefore our definition of the quality of an information flow is restricted to product quality. We use a definition that is introduced by Brombacher [6]: The quality of a product quality oriented information flow is the extend to which the information flow is able to measure, understand and improve the quality and reliability of a product. As this paper focuses on field feedback, we concentrate on the quality (including reliability) of a product in the field. We concentrate on decisions that are relevant during the PCP. Because different people have different tasks and responsibilities, we do not discuss individual wants, instead, we concentrate on the following well-known subdivision of the span of control in three (management) levels [2]: - Strategic: the decisions concern the company policy, in particular the range of products, including the choice of the technology (market orientation). - Tactic: the decisions concern the choice of the organisation type and production systems, including the required control systems. - Executive: the decisions concern the method of working and the operating procedures. (In the literature sometimes the word ‘operational’ is used instead of ‘executive’).
improve) the product quality. These product creation and monitoring activities require two kinds of information: - Technical information. This information enables product designers and developers to come up with technical solutions that transform ideas in real products. It also supports engineers to technically improve the product in case the product quality/reliability is below standard. For example, in case of customer complains, a root cause5 analysis is based on technical information. From this description, it follows that technical information is in particular relevant on the executive level. - Statistical information. For monitoring purposes it is required to collect quantitative information about the frequency of product failures.6 This type of information not only proves, for example, whether a product family satisfies company requirements for product reliability, it is also essential for reliability prediction of new products. It follows that statistical information is in particular relevant on the tactic and strategic levels. Note that our definition of product failure is based on the opinion of customers. If a customer reports a product failure, the product might still satisfy the technical specs. In such a case a repair centre will usually report ‘no fault found’, because, driven by manufacturers, they define product quality as: ‘Quality is the degree to which a specific product conforms to a design or specification.’ [11]. In this paper we use the term product quality from the point of view of the customer: ‘. the quality of a product depends on how well it fits patterns of consumer preferences’ [12]. Statistical information is necessary for, among others, the following reasons: - To determine the absolute levels of defects. - To find out whether there are modules/components that fail relatively often. - To assess the lifetime distribution of the time to first failure; this is, for example, relevant for warranty purposes. - To determine whether the company is learning from the past, for example by checking whether there are any differences in product quality over different types of products, or over product generations.
Each level should get the information that is relevant for the decisions and actions that belong to the responsibility of that level. Information is only valuable if it is in time, meaning that information must be synchronised with the operated timetable. As soon as the strategic decisions about the range of products have been taken, products have to be created, and a product quality control system has to monitor (and
As we will give a guideline for the construction of information flows, we have to connect decisions with information need. In general, one can say that on higher levels in the organisation (strategic and tactic) the decisions are mainly based on population characteristics as safety issues, market share and warranty claims/costs (i.e.
3 Innovative product—a product with a new functionality or product design. 4 Field feedback—the information about the performance of products in interaction with customers.
5 Root cause—the most basic causal factor or factors that, if corrected or removed, will prevent recurrence of the failure. 6 Product failure—a customer decides to report that the product is not able to meet the customer’s implicit and/or explicit requirements.
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
statistical information). On the executive level, the decisions concern the quality and reliability of the individual products; these decisions are mainly based on root causes of field failures (i.e. technical information). This paper looks as follows. Section 2 presents a general discussion about the information requirements in fast PCPs. The MIR-concept is discussed in Section 3. It is explained that the MIR-concept has some serious shortcomings and suggests an approach that slows down improvement actions. Section 4 presents an operational guideline for building field feedback information flows. Section 5 presents a case study that illustrates the procedure. The conclusions are presented in Section 6.
2. Information requirements This section presents the most basis demands that ought to be fulfilled by an information system in order to be able to create products that satisfy the customer. As stated in the introduction, information and information flows should be judged on their value for decisionmaking. This means that relevant data should be collected in time, and categorised, analysed, and stored in such a way that potential users of the information are invited to use it. For example, designers need information about available components, modules, software, potential interaction/interference problems, performance of previous generations of product, etc.; operators need information that helps them to run the production machines in the best possible way; the production manager needs information about the yield; and the CEO wants to see whether the processes are under control, e.g. by looking at field failure data. In short, the relevance of the information is determined by the timeliness, and by the potential contribution to the decision that is at stake. From an information (flow) point of view the general approach of a problem is the following: 1. Start with a preliminary definition of the problem and an overview of the decisions that might be relevant. 2. Determine who should be involved in the problem definition, problem analysis, data collection and problem solving process. 3. Determine what information is necessary and when it has to be available. 4. Check whether the necessary information is available (and correct) or can be collected in time. 5. Collect and analyse the data with the relevant decisions in mind. 6. Translate the results of the analysis in decisions (and actions). This general approach leads immediately to prerequisites for the information flow structure:
149
- The information flow has to be designed with the product development roadmap in mind. As specified earlier, because of the pressure on the time-to-market; there is not much time available for testing and for product improvement activities. At the same time, the innovation speed brings about that it is difficult to predict the way customers will use the product. This makes it difficult for developers to estimate the field behaviour of their products. Unfortunately, collecting field feedback usually takes a lot of calendar time (more than one year from product launch is no exception), and therefore field feedback is usually too late to improve the product concerned or even the next generation [8,9]. - The available information should be complete enough for quality improvement actions. Unfortunately, companies collect field information mainly for logistic and financial purposes [8], and hardly for finding root causes of reliability problems. Pecht and Ramappan [13] make more or less the same statement: ’.the types of failure and the causes of failure for electronics have changed over the years.In many cases there are significant numbers of reported failures which are due to ‘unknown’, ‘not verified’, and ‘other’ causes, indicating the lack of comprehensive failure analysis or the inability to clearly dissent the cause. In other cases, failures are attributed to ‘design’, ‘testing’ and ‘vendor related’ factors without addressing failure causes’. - Information deployment. Finding the root causes of failures and collecting useful field information is necessary but not enough for quality and reliability improvement of products. If the information is not analysed and if the results of the analyses are not reported to the right people in the company, no adequate action can be taken. Furthermore, decisions are often taken on a common sense basis and there is no structure in which those decisions are taken. Moreover, the people that should receive field information are often not able to specify the type of information they need [14]. - Information is often hidden in a huge amount of data that is difficult to analyse. In order to keep the people who acquire the data motivated, they should get feedback about the value of the data [14]. This can be formulated in a more general sense: there should be a closed loop between the one(s) who (should) collect data, the one(s) who (should) analyse the data and the one(s) who (should) use the final results (analysed data). This closed loop principle should make sure that the people who collect information are aware of the information that is needed. This is strongly related with the next aspect: - Information should be checked on integrity (is the available information representative for the actual present situation?), and correctness (is the correctness of the information checkable?) [15] - Information should be understood; therefore, it should be selected in close collaboration with the people who need it.
150
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
- Finally yet importantly, there should be a fair balance between the relevance of the information and the costs [15]. Information flows are only informative if they satisfy all these requirements. In the next section we discuss the MIRconcept, as this concept is the only systematic tool we know for the evaluation of field feedback information flows.
3. The MIR-concept In the literature there are quite a few papers about the MIR-concept as a tool to analyse product quality related information flows (among others [1,5–8,16,17]). The MIRconcept has been used over thirty times when assessing the information flow structure in industry, in Europe as well as in South-East Asia. In this section it will be shown that the ideas behind the MIR-concept are valuable, but that it has serious weaknesses in its details. The MIR-concept aims at ‘classifying the (quality of-) information flows in an organisation with respect to their ability to measure, understand and improve the quality and reliability of a product in the field’ [6]. According to Brombacher [6] the four MIR levels are defined on an ordinal scale: 1. Quantification (how much): Quantitative information is available per product indicating the number of failures in field and production. 2. Identification (where): Quantitative information available on primary/secondary failure location: † Primary (organisation): Location of the cause of the failure within the Development Process; † Secondary (position): Location of the failure within the product (Part NR, Conditions, etc). 3. Cause (why): Detailed information is available for all dominant failures on root-cause level. This can be translated into risks for future products. 4. Improvement (what to do): Methods and tools are in place to anticipate on reliability risks for future products and eliminate these risks where needed. This set of four criteria has turned out to be a valuable tool for evaluating information flows. However, the translation of the MIR-concept in the four MIR levels on an ordinal scale has its weaknesses. We give four critical remarks about the MIR levels: - The MIR levels are presented as a hierarchical system on an ordinal scale, but the structure is basically a classification. For example, the MIR concept implies that MIR 3 can only be reached after MIR 1 and MIR 2. However, MIR 1 is about statistical information: how much, while MIR 3 is about root causes: technical information. Companies should start with root cause
analyses immediately after market release. Based on the root cause of the first customer complaint, engineers should try to find out whether the problem is only an incident, or a serious problem. In case of a serious problem, for example a safety problem, one should take drastic actions immediately. Of course, one should also start with collecting statistical information immediately after product launch, but root cause analyses should be performed, and the results used, long before the statistical information will be available. Valuable time will pass if companies strictly follow the MIR-procedure and first concentrate on MIR 1. In our view, one should first go for MIR 3. Doing this, one should of course avoid premature decisions that are only based on relatively unimportant incidents. We conclude that in our opinion the MIR levels should be seen as classes, and for this reason it is better to talk about MIR aspects than about MIR levels. We prefer this terminology despite the fact that MIR 4 is not independent from MIR 1 and MIR 3, as is visualised in Fig. 1. - The definition of the MIR levels lacks consistency. The MIR levels 1, 3 and 4 concern the number of failures, the root causes of the failures, and the prevention and elimination of the root causes. These aspects are relevant for all information flows. MIR level 2, however, concerns the location of the cause of the failure. It is not evident that for each failure it is possible to locate the cause of the failure before being sure about the root cause. In short: the MIR levels 1, 3 and 4 are basic levels, while level 2 is an auxiliary level that is only useful in special cases. The structure of the MIR concepts wins in clarity if the second MIR aspect is skipped. - In the case studies that have been published, for example [7], not only information flows get a MIR level, but also organisations. However, the MIR concept is about classifying information flows, and companies have many information flows. Some of these may generate useful statistical information and others may generate useful root cause oriented (technical) information. In our view it does not make sense to combine conclusions
Fig. 1. Structure of the MIR-concept.
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
151
about unrelated information flows in one overall company wide MIR level. - The MIR concept does not take into account the timeliness of the information. This is a serious omission, because in time-driven development processes not only the information content is important, but also the speed with which the information is gathered and deployed. The usability of information strongly depends on the moment at which the information is available.
ii. Are there any problems with the design? iii. Are the results of a design change in line with the expectations? When the problems have been defined, it is time to determine what information is required and when it has to be available.
Because of these criticisms, we change the perspective. Instead of focussing on a tool for the evaluation of existing information flows, we give priority to the construction of information flows. If we know how to construct information flows, it must be possible to evaluate them as well.
i. Safety: are there any safety problems that require a recall of already sold products? The decision is based on the risk, i.e. on an estimate of the likelihood of occurrence and of the consequences. Knowing the root cause, that is determined on the executive level, engineers should be able to estimate the risk, and then the decision about a recall can be taken. The required information has to be available as soon as possible after the detection of the safety hazard. That means that each individual safety problem has to be reported to the responsible manager without any delay. Conclusion: the executive level analyses the safety problem via the root cause, and the strategic level takes the decision. ii. Product quality image/costs: are there any serious product quality problems that indicate that it makes sense to stop selling the product immediately? As soon as serious customer complaints come in, the root cause has to be defined. Based on the root cause, and on engineering judgement about the frequency of the problem, supplemented with cost considerations, the decision has to be taken whether the sales should be stopped or not. Because of cost considerations, there is no time to wait until there is accurate statistical evidence about the percentage of failed products. There has to be a field feedback flow that generates the required information as soon as possible. In order to speed up the data gathering process, it might be advantages to provide the dealers/service centres with the tools and knowledge to determine the root course. Conclusion: the executive level has to determine the root cause and to estimate the frequency of the problem based on the root cause; the strategic level decides. iii. Learning potential: is the manufacturer learning from the past, for example, is the fraction of warranty claims decreasing from generations to generation? If it is not, the reason for this should be sorted out, because only then the right decisions can be taken.The decisions concern the fraction of warranty claims, so they should be based on statistical evidence. This takes a lot of
4. Construction of information flows As mentioned before, the value of information is its ability to facilitate decision-making. We structure the guideline using the classification of decisions in strategic, tactic, and executive level. We concentrate on decisions that are based, at least partly, on field feedback. As companies have their own way of distributing responsibilities, the following will not cover every company, but it should give a good impression about the aspects that should be taken in consideration. The guideline has the following structure: 1. Relevant problems that require decisions 1a. Examples of problems on strategic level: i. Safety: are there any safety problems that require a recall of already sold products? ii. Product quality image/costs: are there any serious product quality problems that indicate that it makes sense to stop selling the product immediately? iii. Learning potential: is the manufacturer learning from the past, for example, is the fraction of warranty claims decreasing from generations to generation? 1b. Examples of problems on tactic level: i. Product quality image/costs: if it has been decided that there is no reason for a recall, the question rises whether the product quality should be improved as fast as possible? ii. Product quality image/costs: the product quality of the present generation may be acceptable, but what aspects of the product quality should be seriously improved in the next generation? 1c. Examples of problems on executive level: i. Do customers use the product according to the expectations?
2. Information need (what and when) 2a. Information need on strategic level:
152
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
calendar time. Later root cause information is required when improvement action are defined. Conclusion: Statistical information has to be gathered and the right structure for this will usually be defined on the tactical level. The strategic level decides. 2b. Information need on tactic level: i. Product quality image/costs: it has been decided that there is no reason for a recall, but should the product quality be improved as fast as possible? Because of the time pressure, the decision has to be based on the root cause and there has to be a fast field feedback flow that generates the required root cause information. Conclusion: the executive level has to determine the root cause and the tactical level decides. ii. Product quality image/costs: the product quality of the present generation may be acceptable, but what aspects of the product quality should be seriously improved in the next generation? Of course, also in this situation root cause information is essential, but there might be enough time to estimate the fraction of identical problems as well. The available calendar time is laid down in the product roadmap. The field feedback flow has to fit in this roadmap. Conclusion: The executive level has to find the root cause and the tactical level has to define the right structure to be able to collect the statistical information in time; the tactical level decides. 2c. Information need on executive level: i. Do customers use the product according to the manufacturers expectations? In some cases, the root cause gives sufficient information. In others, specific information about customer use has to be collected. Conclusion: the tactical level should facilitate the data collection. ii. Are there any problems with the design? In addition to the root cause information, statistical data about field failures is useful to detect problems that do not manifest themselves to often. The statistical information concerns aspects like frequencies of failures, the distribution of the failures as a function of relevant failure mechanisms like calendar time, time in use, number of cycles, or customer use. The root causes have to be known in order to be able to generate redesign proposals. Conclusion: the executive level should be able to find the root cause. iii. Are the results of a design change in line with the expectations? Detailed information about the behaviour of the old as well as of the new design has to be available.
The combination of technical information and statistical information about the frequency distribution of product failures on root cause level is required. Conclusion: the executive level should be able to find the root cause, and statistical information has be collected. It is not surprising that decisions on the tactical and strategic level require information that is collected on the executive level. For example, on the executive level people collect root cause information. The task and responsibilities of the people who work on the executive level, like operators, designers, etc. are distributed on the tactical level. Depending on this distribution of tasks, the information flows should be structured. For example, if dealers are supposed to determine root causes, then there should be a field feedback flow that transports the root cause information from dealers to designers (among others). If, however, dealers and service centres are not paid for tracing root causes, then there has to be another centre, say an Analysis Centre, that does trace root causes. In this case there has to be a rule that makes clear which products have to be sent to this Analysis Centre, because it is usually unaffordable and not very useful to send all failed products to the Analysis Centre. An Analysis Centre might be extremely useful during the first period after the product launch. As soon as an Analysis Centre has been introduced, the field feedback flow becomes more complicated. For example, if we have a stand-alone product, then the root cause must be in the combination product-user-environment; but if the product is a module in a system, then the root cause can also be in the interaction between the different modules. If the Analysis Centre investigates the product outside of the system in which it operates, then the root cause might be untraceable. Furthermore, there has to be a field feedback flow from the dealers/service centres via the Analysis Centre to the people that have to take the relevant decisions. As the detailed information that is necessary for a root cause analysis is product specific, we will not go into this. The same holds true for the statistical information, although the common elements are less determined by the product characteristics than for root cause analyses. In order to demonstrate the use of the guideline, in the next section it is applied for analysing and improving the field feedback system of an existing company. For each of the management levels a relevant decision is taken, the required information is determined, and the available information and information flows are compared with the theoretical requirements.
5. Case study The objective of the case study is to evaluate the value of the guideline for assessing the quality of existing
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
information flows. For each of the three (management) levels one or more relevant decisions are considered; the required information is determined, and the available information and information flows are compared with the requirements. The case study was carried out in a Dutch company that produces air-refreshing systems, for office use as well as for home use. We will call the company Fresh Air. The airrefreshing systems consist of both hardware and software. The hardware is developed by Fresh Air, but it is produced and tested by a supplier. The software is developed and produced by Fresh Air. In addition, the product is installed at the customer by an account (there are many accounts). This means that the PCP is distributed over different and independent organisations: Fresh Air, the hardware supplier, and the accounts. The primary business process is depicted in Fig. 2. According to Fresh Air, the non-computer related hardware is expected to function about five years, the computer related hardware about three years (because of the fast innovation in the computer industry), and the software is updated more frequently. Recently, Fresh Air gets many complaints by its accounts about the frequent software updates. Therefore, the management of Fresh Air wonders whether the number of software updates should be structurally reduced. In this section the required information flows are analysed using the guideline and improvements are proposed. 5.1. Case method 5.1.1. Case selection For the case, a recent product of Fresh Air was chosen. In this way it was guaranteed that the knowledge about the relevant details was still present. The chosen product is the first product of Fresh Air that uses internet technologies. The functionality of the product is more complex than that of previous products, and therefore the customers might not know how to use the product. After the product was officially released, it had three software updates within three months and that caused many customer complaints. 5.1.2. Data collection Structured interviews within Fresh Air were used to collect data. For each of the management levels of Fresh Air, the most relevant decisions they had to take were
153
discussed. For each potential decision the discussion focussed on the following aspects: † What information was needed for making the decision? † How was this information generated? † Was the information generated in time? People with key functions along the product creation and realisation process were interviewed, that is product management, research and development (R&D), software development, supplier hardware and account. The results of the interviews were crosschecked. 5.1.3. Data analysis During the data analysis phase, the guideline was used to identify the existing and the required information flows at three levels: strategic, tactic and executive. The existing information flows were compared with the required information flows. 5.2. Case results For each of the three management flows, improvement directions were identified 5.2.1. Strategic level The problem at stake is the customer complaint about the frequency of software updates. The management of Fresh Air had to decide whether a plan should be developed for the improvement of the software quality of the present and future products. This decision should be based on statistical information (how many software updates are required after the installation?), on the identified root causes (what exactly was wrong with the original software?), and on the consequences of software updates for the customer. The information could be gathered via two available feedback flows between the customer and Fresh Air. One feedback flow concerns the field feedback about existing products: the frequency and the root causes. The other one gives field feedback about the consequences of a software update for the customer. The first feedback flow operates as follows. If a customer experiences a problem with a product, he informs the account. The account tries to resolve the problem. If the account cannot resolve the problem, then the account
Fig. 2. Primary business process.
154
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
contacts Fresh Air. Product Support of Fresh Air will try to resolve the product failure. If the problem remains unresolved, then RandD as well as Product Management are informed. In this way Product Management learned that within the first three months after the product was officially released, three software updates took place. The second field feedback flow is a direct line between on the one hand product management, and on the other hand, some key accounts and key customers. This field feedback flow concentrates on the collection of information that is relevant for the development of new products. Product management discusses with some key accounts and key customers the experiences with present and previous products, and the requirements for future products. This second field feedback flow made clear that during a software update, the air refreshing systems of the customers have to stop working for at least one day. According to the customers and the accounts, this is not acceptable and it damages the product quality image of Fresh Air. Based on these two feedback flows, the management took the strategic decision to start a process for the improvement of the software quality of future products. The required information for this decision was available: statistical information about the number of software updates, and the opinion of customers. 5.2.2. Tactic level Given the strategic decision to improve the software quality, the middle management had to work out an improvement programme. It decided to start a training programme. The details are given in the section about the executive level. In our opinion, the decision of the middle management is premature, because for an effective training programme, it is essential that the root cause of the software problems are known. For example, it does not make sense to start a general training programme if the root cause of the software problems is the result of wrong software specifications. So the question is: what causes the late and frequent software updates? The situation should be clarified by a direct information flow between the customers and the software developers. However, the only direct line between Fresh Air and its customers was the feedback flow between Product Management, some key accounts and some key customers. There is no direct contact between software developers and customers. As a result, the software developers were hardly informed about the way customers used the equipment, and the interviews made clear this was one of the reasons why software updates were necessary. The conclusion was that the middle management did not have enough information to define an effective and efficient improvement programme. It was advised to introduce a formal field feedback flow from the customers back to the software developers.
5.2.3. Executive level Given the decision at the tactic level to organise more training courses, the software developers, at the executive level, had to define the main topics of these courses. Because there was no field feedback flow from the customers to the software developers, there was no information available about the root cause of the bad performance of the software. Therefore, the software developers decided to focus the training programme on the technical design of the software. When we evaluated the situation, the training programme had run for about half a year without a serious positive effect. As the decision to start the programme was not founded on a serious analysis of the problem and as the required information was not available, we are afraid that the training programme will turn out to be disappointing. 5.3. Conclusion Based on the analysis above, it can be concluded that on the strategic level there was enough (statistical) information available to focus the attention on the right issues However, it is not yet clear whether the decision at the strategic level could not have been taken earlier. Due to lack of root cause information, the company could not come to the right conclusions on the tactic and executive level. In particular, there was no information available about the root causes of the software failures. In order to solve this problem, it was advised to introduce a field feedback flow between the customers and accounts back to the software developers. This feedback flow will concentrate on the timely feedback of root cause information.
6. Conclusions and recommendations The main conclusions of this research project are the following: - Field feedback is an essential tool for product improvement. - The maturity index for reliability has proven to be a very useful tool for the evaluation of quality oriented field feedback information flows, but it has some weaknesses. One of the major weaknesses is that it does not evaluate the timeliness of the required information. Another one is that it concentrates too much on statistical information, what tends to delay product quality improvement actions. - This paper presents a guideline for the construction of information flows. By means of a case study, it shows that the guideline can be used as a field feedback evaluation tool as well. - The case has shown that it is possible to use the proposed guideline to analyse and structure the information flows.
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
It is explained that successful corrective actions are possible by a co-ordinated action of all three management levels.
References [1] Molenaar PA, Huijben AJM, Bouwhuis D, Brombacher AC. Why do quality and reliability feedback loops not always work in practice: a case study. Reliab Eng Syst Saf 2002;75:295–302. [2] Parsons T. Structure and process in modern societies. Illinois: The Free Press of Glencoe; 1960. [3] Smith RG, Reinertsen DG. Developing products in half the time: new rules, new tools. New York: Van Nostrand Reinhold; 1998. [4] Ulrich KT, Eppinger SD. Product design and development. New York: McGraw-Hill Inc; 2000. [5] Lu Y, Loh HT, Ibrahim Y, Sander PC, Brombacher AC. Reliability in a time-driven product development process. Qual Reliab Eng Int 1999;15:427–30. [6] Brombacher AC, Steinz HC, Volman HPJ. Safety and reliability assessment on products and organizations. ISA EXPO 1998Technical paper.
155
[7] Sander PC, Brombacher AC. MIR: the use of reliability information flows as a maturity index for quality management. Qual Reliab Eng Int 1999;15:439–47. [8] Petkova VT, Sander PC, Brombacher AC. The role of the service centre in improvement processes. Qual Reliab Eng Int 1999;15:431–7. [9] Petkova VT, Sander PC, Brombacher AC. The use of quality metrics in service centres. Int J Prod Econ 2000;67:27–36. [10] Petkova VT. An analysis of field feedback in consumer electronics industry. Thesis, Eindhoven University of Technology; 2003. [11] Gilmore HL. Product conformance cost. Qual Prog 1974;June:16. [12] Kuehn AA, Day RL. Strategy of product quality. Harvard Business Rev 1962;,, November-December:101. [13] Pecht MG, Ramappan V. Are components still the major problem: a review of electronic system and device field failure returns. IEEE Trans Components, Hybrids, Manuf Technol 1992;15:1160–4. [14] Gu¨thenke G, Leiters M. Fast elimination of product faults in current series. Total Qual Manage 1999;10:569–75. [15] Salau¨n I, Flores K. Information quality: meeting the need of the customer. Int J Inform Manage 2001;21:21–37. [16] Berden TPJ, Brombacher AC, Sander PC. The building bricks of product quality: an overview of some basic concepts and principles. Int J Prod Econ 1999;67:3–15. [17] Brombacher AC. MIR: Covering non-technical aspects of IEC61508 reliability certification. Reliab Eng Syst Saf 1999;66:109–20.
Designing reliability information flows Valia T. Petkova*, Lu Yuan, Roxana A. Ion, Peter C. Sander Faculty of Technology Management/Sub-department Product and Process Quality, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands Available online 25 August 2004
Abstract It is well-known [Reliab. Eng. Syst. Saf. 75 (2002) 295] that in modern development processes it is essential to have an information flow structure that facilitates fast feedback from product users (customers) to departments at the front end, in particular development and production. As information is only relevant if it is used when taking decisions, this paper presents a guideline for building field feedback information flows that facilitate the decision taking during the product creation and realisation process. The guideline takes into consideration that the type of decisions depends on the span-of-control, therefore following Parsons [Structure and Process in Modern Societies (1990)] the span-of-control is subdivided into the following three levels: strategic, tactic, and executive. The guideline is illustrated with a case in which it is used for analysing the quality of existing field feedback flows. q 2004 Elsevier Ltd. All rights reserved. Keywords: Fast Field Feedback; Information flow; Product development
1. Introduction In the literature, there has been quite some attention to the vital role of information flows in situations where a high degree of innovation forces manufacturers to speed up product creation processes (PCPs1) to the limit [3,4]. The need for a strong emphasis on fast and effective information loops, within a PCP as well as between a PCP and the customer,2 emerges from four trends currently dominating (electronics) industry [5]: - Due to recent advances in (digital electronics and software) technology, increasingly complex products are put on the market. - Technological innovations put a pressure on the timeto-market. * Corresponding author. E-mail address: [email protected] (V.T. Petkova). 1 PCP—a process that systematically transforms new product ideas into a set of products. 2 Customer—a person who uses a product that he did not produce himself. 0951-8320/$ - see front matter q 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.ress.2004.07.004
- Due to the increasing globalisation, more and more PCPs are divided over partners in different parts of the world, e.g. design, engineering, suppliers, production, assembly. This requires efficient information and communication systems. - Customers expect very complex products to work according to their (customers’) requirements, regardless of the product specifications. Brombacher [6] presented an interesting idea, the maturity index of reliability (MIR), to judge quality and reliability oriented information flows. The authors have contributed to this research via a series of papers concentrating on the relation between information flows and product reliability aspects [7–9]. Although the MIRconcept is a relevant contribution to the literature, it has some important weaknesses [10]. This is explained in Section 3. The aim of this paper is to give a guideline for constructing information flows. A side effect of such a guideline is that it can be used as an evaluation tool for existing information flows. To be more precise, the focus of
148
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
this paper is on the construction of field feedback information flows concentrating on the quality and reliability (shortly denoted by quality) aspects of innovative,3 electronic consumer products. We concentrate on field feedback4 because for innovative products, it is not always clear how a customer will use them. Field feedback is the ultimate proof that the product does or does not fulfil customer desires. The function of an information flow is to bring the required information at the right time to the person who is expected to choose between different options. So, we explicitly couple information with decisions. This paper concentrates on product quality and therefore our definition of the quality of an information flow is restricted to product quality. We use a definition that is introduced by Brombacher [6]: The quality of a product quality oriented information flow is the extend to which the information flow is able to measure, understand and improve the quality and reliability of a product. As this paper focuses on field feedback, we concentrate on the quality (including reliability) of a product in the field. We concentrate on decisions that are relevant during the PCP. Because different people have different tasks and responsibilities, we do not discuss individual wants, instead, we concentrate on the following well-known subdivision of the span of control in three (management) levels [2]: - Strategic: the decisions concern the company policy, in particular the range of products, including the choice of the technology (market orientation). - Tactic: the decisions concern the choice of the organisation type and production systems, including the required control systems. - Executive: the decisions concern the method of working and the operating procedures. (In the literature sometimes the word ‘operational’ is used instead of ‘executive’).
improve) the product quality. These product creation and monitoring activities require two kinds of information: - Technical information. This information enables product designers and developers to come up with technical solutions that transform ideas in real products. It also supports engineers to technically improve the product in case the product quality/reliability is below standard. For example, in case of customer complains, a root cause5 analysis is based on technical information. From this description, it follows that technical information is in particular relevant on the executive level. - Statistical information. For monitoring purposes it is required to collect quantitative information about the frequency of product failures.6 This type of information not only proves, for example, whether a product family satisfies company requirements for product reliability, it is also essential for reliability prediction of new products. It follows that statistical information is in particular relevant on the tactic and strategic levels. Note that our definition of product failure is based on the opinion of customers. If a customer reports a product failure, the product might still satisfy the technical specs. In such a case a repair centre will usually report ‘no fault found’, because, driven by manufacturers, they define product quality as: ‘Quality is the degree to which a specific product conforms to a design or specification.’ [11]. In this paper we use the term product quality from the point of view of the customer: ‘. the quality of a product depends on how well it fits patterns of consumer preferences’ [12]. Statistical information is necessary for, among others, the following reasons: - To determine the absolute levels of defects. - To find out whether there are modules/components that fail relatively often. - To assess the lifetime distribution of the time to first failure; this is, for example, relevant for warranty purposes. - To determine whether the company is learning from the past, for example by checking whether there are any differences in product quality over different types of products, or over product generations.
Each level should get the information that is relevant for the decisions and actions that belong to the responsibility of that level. Information is only valuable if it is in time, meaning that information must be synchronised with the operated timetable. As soon as the strategic decisions about the range of products have been taken, products have to be created, and a product quality control system has to monitor (and
As we will give a guideline for the construction of information flows, we have to connect decisions with information need. In general, one can say that on higher levels in the organisation (strategic and tactic) the decisions are mainly based on population characteristics as safety issues, market share and warranty claims/costs (i.e.
3 Innovative product—a product with a new functionality or product design. 4 Field feedback—the information about the performance of products in interaction with customers.
5 Root cause—the most basic causal factor or factors that, if corrected or removed, will prevent recurrence of the failure. 6 Product failure—a customer decides to report that the product is not able to meet the customer’s implicit and/or explicit requirements.
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
statistical information). On the executive level, the decisions concern the quality and reliability of the individual products; these decisions are mainly based on root causes of field failures (i.e. technical information). This paper looks as follows. Section 2 presents a general discussion about the information requirements in fast PCPs. The MIR-concept is discussed in Section 3. It is explained that the MIR-concept has some serious shortcomings and suggests an approach that slows down improvement actions. Section 4 presents an operational guideline for building field feedback information flows. Section 5 presents a case study that illustrates the procedure. The conclusions are presented in Section 6.
2. Information requirements This section presents the most basis demands that ought to be fulfilled by an information system in order to be able to create products that satisfy the customer. As stated in the introduction, information and information flows should be judged on their value for decisionmaking. This means that relevant data should be collected in time, and categorised, analysed, and stored in such a way that potential users of the information are invited to use it. For example, designers need information about available components, modules, software, potential interaction/interference problems, performance of previous generations of product, etc.; operators need information that helps them to run the production machines in the best possible way; the production manager needs information about the yield; and the CEO wants to see whether the processes are under control, e.g. by looking at field failure data. In short, the relevance of the information is determined by the timeliness, and by the potential contribution to the decision that is at stake. From an information (flow) point of view the general approach of a problem is the following: 1. Start with a preliminary definition of the problem and an overview of the decisions that might be relevant. 2. Determine who should be involved in the problem definition, problem analysis, data collection and problem solving process. 3. Determine what information is necessary and when it has to be available. 4. Check whether the necessary information is available (and correct) or can be collected in time. 5. Collect and analyse the data with the relevant decisions in mind. 6. Translate the results of the analysis in decisions (and actions). This general approach leads immediately to prerequisites for the information flow structure:
149
- The information flow has to be designed with the product development roadmap in mind. As specified earlier, because of the pressure on the time-to-market; there is not much time available for testing and for product improvement activities. At the same time, the innovation speed brings about that it is difficult to predict the way customers will use the product. This makes it difficult for developers to estimate the field behaviour of their products. Unfortunately, collecting field feedback usually takes a lot of calendar time (more than one year from product launch is no exception), and therefore field feedback is usually too late to improve the product concerned or even the next generation [8,9]. - The available information should be complete enough for quality improvement actions. Unfortunately, companies collect field information mainly for logistic and financial purposes [8], and hardly for finding root causes of reliability problems. Pecht and Ramappan [13] make more or less the same statement: ’.the types of failure and the causes of failure for electronics have changed over the years.In many cases there are significant numbers of reported failures which are due to ‘unknown’, ‘not verified’, and ‘other’ causes, indicating the lack of comprehensive failure analysis or the inability to clearly dissent the cause. In other cases, failures are attributed to ‘design’, ‘testing’ and ‘vendor related’ factors without addressing failure causes’. - Information deployment. Finding the root causes of failures and collecting useful field information is necessary but not enough for quality and reliability improvement of products. If the information is not analysed and if the results of the analyses are not reported to the right people in the company, no adequate action can be taken. Furthermore, decisions are often taken on a common sense basis and there is no structure in which those decisions are taken. Moreover, the people that should receive field information are often not able to specify the type of information they need [14]. - Information is often hidden in a huge amount of data that is difficult to analyse. In order to keep the people who acquire the data motivated, they should get feedback about the value of the data [14]. This can be formulated in a more general sense: there should be a closed loop between the one(s) who (should) collect data, the one(s) who (should) analyse the data and the one(s) who (should) use the final results (analysed data). This closed loop principle should make sure that the people who collect information are aware of the information that is needed. This is strongly related with the next aspect: - Information should be checked on integrity (is the available information representative for the actual present situation?), and correctness (is the correctness of the information checkable?) [15] - Information should be understood; therefore, it should be selected in close collaboration with the people who need it.
150
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
- Finally yet importantly, there should be a fair balance between the relevance of the information and the costs [15]. Information flows are only informative if they satisfy all these requirements. In the next section we discuss the MIRconcept, as this concept is the only systematic tool we know for the evaluation of field feedback information flows.
3. The MIR-concept In the literature there are quite a few papers about the MIR-concept as a tool to analyse product quality related information flows (among others [1,5–8,16,17]). The MIRconcept has been used over thirty times when assessing the information flow structure in industry, in Europe as well as in South-East Asia. In this section it will be shown that the ideas behind the MIR-concept are valuable, but that it has serious weaknesses in its details. The MIR-concept aims at ‘classifying the (quality of-) information flows in an organisation with respect to their ability to measure, understand and improve the quality and reliability of a product in the field’ [6]. According to Brombacher [6] the four MIR levels are defined on an ordinal scale: 1. Quantification (how much): Quantitative information is available per product indicating the number of failures in field and production. 2. Identification (where): Quantitative information available on primary/secondary failure location: † Primary (organisation): Location of the cause of the failure within the Development Process; † Secondary (position): Location of the failure within the product (Part NR, Conditions, etc). 3. Cause (why): Detailed information is available for all dominant failures on root-cause level. This can be translated into risks for future products. 4. Improvement (what to do): Methods and tools are in place to anticipate on reliability risks for future products and eliminate these risks where needed. This set of four criteria has turned out to be a valuable tool for evaluating information flows. However, the translation of the MIR-concept in the four MIR levels on an ordinal scale has its weaknesses. We give four critical remarks about the MIR levels: - The MIR levels are presented as a hierarchical system on an ordinal scale, but the structure is basically a classification. For example, the MIR concept implies that MIR 3 can only be reached after MIR 1 and MIR 2. However, MIR 1 is about statistical information: how much, while MIR 3 is about root causes: technical information. Companies should start with root cause
analyses immediately after market release. Based on the root cause of the first customer complaint, engineers should try to find out whether the problem is only an incident, or a serious problem. In case of a serious problem, for example a safety problem, one should take drastic actions immediately. Of course, one should also start with collecting statistical information immediately after product launch, but root cause analyses should be performed, and the results used, long before the statistical information will be available. Valuable time will pass if companies strictly follow the MIR-procedure and first concentrate on MIR 1. In our view, one should first go for MIR 3. Doing this, one should of course avoid premature decisions that are only based on relatively unimportant incidents. We conclude that in our opinion the MIR levels should be seen as classes, and for this reason it is better to talk about MIR aspects than about MIR levels. We prefer this terminology despite the fact that MIR 4 is not independent from MIR 1 and MIR 3, as is visualised in Fig. 1. - The definition of the MIR levels lacks consistency. The MIR levels 1, 3 and 4 concern the number of failures, the root causes of the failures, and the prevention and elimination of the root causes. These aspects are relevant for all information flows. MIR level 2, however, concerns the location of the cause of the failure. It is not evident that for each failure it is possible to locate the cause of the failure before being sure about the root cause. In short: the MIR levels 1, 3 and 4 are basic levels, while level 2 is an auxiliary level that is only useful in special cases. The structure of the MIR concepts wins in clarity if the second MIR aspect is skipped. - In the case studies that have been published, for example [7], not only information flows get a MIR level, but also organisations. However, the MIR concept is about classifying information flows, and companies have many information flows. Some of these may generate useful statistical information and others may generate useful root cause oriented (technical) information. In our view it does not make sense to combine conclusions
Fig. 1. Structure of the MIR-concept.
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
151
about unrelated information flows in one overall company wide MIR level. - The MIR concept does not take into account the timeliness of the information. This is a serious omission, because in time-driven development processes not only the information content is important, but also the speed with which the information is gathered and deployed. The usability of information strongly depends on the moment at which the information is available.
ii. Are there any problems with the design? iii. Are the results of a design change in line with the expectations? When the problems have been defined, it is time to determine what information is required and when it has to be available.
Because of these criticisms, we change the perspective. Instead of focussing on a tool for the evaluation of existing information flows, we give priority to the construction of information flows. If we know how to construct information flows, it must be possible to evaluate them as well.
i. Safety: are there any safety problems that require a recall of already sold products? The decision is based on the risk, i.e. on an estimate of the likelihood of occurrence and of the consequences. Knowing the root cause, that is determined on the executive level, engineers should be able to estimate the risk, and then the decision about a recall can be taken. The required information has to be available as soon as possible after the detection of the safety hazard. That means that each individual safety problem has to be reported to the responsible manager without any delay. Conclusion: the executive level analyses the safety problem via the root cause, and the strategic level takes the decision. ii. Product quality image/costs: are there any serious product quality problems that indicate that it makes sense to stop selling the product immediately? As soon as serious customer complaints come in, the root cause has to be defined. Based on the root cause, and on engineering judgement about the frequency of the problem, supplemented with cost considerations, the decision has to be taken whether the sales should be stopped or not. Because of cost considerations, there is no time to wait until there is accurate statistical evidence about the percentage of failed products. There has to be a field feedback flow that generates the required information as soon as possible. In order to speed up the data gathering process, it might be advantages to provide the dealers/service centres with the tools and knowledge to determine the root course. Conclusion: the executive level has to determine the root cause and to estimate the frequency of the problem based on the root cause; the strategic level decides. iii. Learning potential: is the manufacturer learning from the past, for example, is the fraction of warranty claims decreasing from generations to generation? If it is not, the reason for this should be sorted out, because only then the right decisions can be taken.The decisions concern the fraction of warranty claims, so they should be based on statistical evidence. This takes a lot of
4. Construction of information flows As mentioned before, the value of information is its ability to facilitate decision-making. We structure the guideline using the classification of decisions in strategic, tactic, and executive level. We concentrate on decisions that are based, at least partly, on field feedback. As companies have their own way of distributing responsibilities, the following will not cover every company, but it should give a good impression about the aspects that should be taken in consideration. The guideline has the following structure: 1. Relevant problems that require decisions 1a. Examples of problems on strategic level: i. Safety: are there any safety problems that require a recall of already sold products? ii. Product quality image/costs: are there any serious product quality problems that indicate that it makes sense to stop selling the product immediately? iii. Learning potential: is the manufacturer learning from the past, for example, is the fraction of warranty claims decreasing from generations to generation? 1b. Examples of problems on tactic level: i. Product quality image/costs: if it has been decided that there is no reason for a recall, the question rises whether the product quality should be improved as fast as possible? ii. Product quality image/costs: the product quality of the present generation may be acceptable, but what aspects of the product quality should be seriously improved in the next generation? 1c. Examples of problems on executive level: i. Do customers use the product according to the expectations?
2. Information need (what and when) 2a. Information need on strategic level:
152
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
calendar time. Later root cause information is required when improvement action are defined. Conclusion: Statistical information has to be gathered and the right structure for this will usually be defined on the tactical level. The strategic level decides. 2b. Information need on tactic level: i. Product quality image/costs: it has been decided that there is no reason for a recall, but should the product quality be improved as fast as possible? Because of the time pressure, the decision has to be based on the root cause and there has to be a fast field feedback flow that generates the required root cause information. Conclusion: the executive level has to determine the root cause and the tactical level decides. ii. Product quality image/costs: the product quality of the present generation may be acceptable, but what aspects of the product quality should be seriously improved in the next generation? Of course, also in this situation root cause information is essential, but there might be enough time to estimate the fraction of identical problems as well. The available calendar time is laid down in the product roadmap. The field feedback flow has to fit in this roadmap. Conclusion: The executive level has to find the root cause and the tactical level has to define the right structure to be able to collect the statistical information in time; the tactical level decides. 2c. Information need on executive level: i. Do customers use the product according to the manufacturers expectations? In some cases, the root cause gives sufficient information. In others, specific information about customer use has to be collected. Conclusion: the tactical level should facilitate the data collection. ii. Are there any problems with the design? In addition to the root cause information, statistical data about field failures is useful to detect problems that do not manifest themselves to often. The statistical information concerns aspects like frequencies of failures, the distribution of the failures as a function of relevant failure mechanisms like calendar time, time in use, number of cycles, or customer use. The root causes have to be known in order to be able to generate redesign proposals. Conclusion: the executive level should be able to find the root cause. iii. Are the results of a design change in line with the expectations? Detailed information about the behaviour of the old as well as of the new design has to be available.
The combination of technical information and statistical information about the frequency distribution of product failures on root cause level is required. Conclusion: the executive level should be able to find the root cause, and statistical information has be collected. It is not surprising that decisions on the tactical and strategic level require information that is collected on the executive level. For example, on the executive level people collect root cause information. The task and responsibilities of the people who work on the executive level, like operators, designers, etc. are distributed on the tactical level. Depending on this distribution of tasks, the information flows should be structured. For example, if dealers are supposed to determine root causes, then there should be a field feedback flow that transports the root cause information from dealers to designers (among others). If, however, dealers and service centres are not paid for tracing root causes, then there has to be another centre, say an Analysis Centre, that does trace root causes. In this case there has to be a rule that makes clear which products have to be sent to this Analysis Centre, because it is usually unaffordable and not very useful to send all failed products to the Analysis Centre. An Analysis Centre might be extremely useful during the first period after the product launch. As soon as an Analysis Centre has been introduced, the field feedback flow becomes more complicated. For example, if we have a stand-alone product, then the root cause must be in the combination product-user-environment; but if the product is a module in a system, then the root cause can also be in the interaction between the different modules. If the Analysis Centre investigates the product outside of the system in which it operates, then the root cause might be untraceable. Furthermore, there has to be a field feedback flow from the dealers/service centres via the Analysis Centre to the people that have to take the relevant decisions. As the detailed information that is necessary for a root cause analysis is product specific, we will not go into this. The same holds true for the statistical information, although the common elements are less determined by the product characteristics than for root cause analyses. In order to demonstrate the use of the guideline, in the next section it is applied for analysing and improving the field feedback system of an existing company. For each of the management levels a relevant decision is taken, the required information is determined, and the available information and information flows are compared with the theoretical requirements.
5. Case study The objective of the case study is to evaluate the value of the guideline for assessing the quality of existing
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
information flows. For each of the three (management) levels one or more relevant decisions are considered; the required information is determined, and the available information and information flows are compared with the requirements. The case study was carried out in a Dutch company that produces air-refreshing systems, for office use as well as for home use. We will call the company Fresh Air. The airrefreshing systems consist of both hardware and software. The hardware is developed by Fresh Air, but it is produced and tested by a supplier. The software is developed and produced by Fresh Air. In addition, the product is installed at the customer by an account (there are many accounts). This means that the PCP is distributed over different and independent organisations: Fresh Air, the hardware supplier, and the accounts. The primary business process is depicted in Fig. 2. According to Fresh Air, the non-computer related hardware is expected to function about five years, the computer related hardware about three years (because of the fast innovation in the computer industry), and the software is updated more frequently. Recently, Fresh Air gets many complaints by its accounts about the frequent software updates. Therefore, the management of Fresh Air wonders whether the number of software updates should be structurally reduced. In this section the required information flows are analysed using the guideline and improvements are proposed. 5.1. Case method 5.1.1. Case selection For the case, a recent product of Fresh Air was chosen. In this way it was guaranteed that the knowledge about the relevant details was still present. The chosen product is the first product of Fresh Air that uses internet technologies. The functionality of the product is more complex than that of previous products, and therefore the customers might not know how to use the product. After the product was officially released, it had three software updates within three months and that caused many customer complaints. 5.1.2. Data collection Structured interviews within Fresh Air were used to collect data. For each of the management levels of Fresh Air, the most relevant decisions they had to take were
153
discussed. For each potential decision the discussion focussed on the following aspects: † What information was needed for making the decision? † How was this information generated? † Was the information generated in time? People with key functions along the product creation and realisation process were interviewed, that is product management, research and development (R&D), software development, supplier hardware and account. The results of the interviews were crosschecked. 5.1.3. Data analysis During the data analysis phase, the guideline was used to identify the existing and the required information flows at three levels: strategic, tactic and executive. The existing information flows were compared with the required information flows. 5.2. Case results For each of the three management flows, improvement directions were identified 5.2.1. Strategic level The problem at stake is the customer complaint about the frequency of software updates. The management of Fresh Air had to decide whether a plan should be developed for the improvement of the software quality of the present and future products. This decision should be based on statistical information (how many software updates are required after the installation?), on the identified root causes (what exactly was wrong with the original software?), and on the consequences of software updates for the customer. The information could be gathered via two available feedback flows between the customer and Fresh Air. One feedback flow concerns the field feedback about existing products: the frequency and the root causes. The other one gives field feedback about the consequences of a software update for the customer. The first feedback flow operates as follows. If a customer experiences a problem with a product, he informs the account. The account tries to resolve the problem. If the account cannot resolve the problem, then the account
Fig. 2. Primary business process.
154
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
contacts Fresh Air. Product Support of Fresh Air will try to resolve the product failure. If the problem remains unresolved, then RandD as well as Product Management are informed. In this way Product Management learned that within the first three months after the product was officially released, three software updates took place. The second field feedback flow is a direct line between on the one hand product management, and on the other hand, some key accounts and key customers. This field feedback flow concentrates on the collection of information that is relevant for the development of new products. Product management discusses with some key accounts and key customers the experiences with present and previous products, and the requirements for future products. This second field feedback flow made clear that during a software update, the air refreshing systems of the customers have to stop working for at least one day. According to the customers and the accounts, this is not acceptable and it damages the product quality image of Fresh Air. Based on these two feedback flows, the management took the strategic decision to start a process for the improvement of the software quality of future products. The required information for this decision was available: statistical information about the number of software updates, and the opinion of customers. 5.2.2. Tactic level Given the strategic decision to improve the software quality, the middle management had to work out an improvement programme. It decided to start a training programme. The details are given in the section about the executive level. In our opinion, the decision of the middle management is premature, because for an effective training programme, it is essential that the root cause of the software problems are known. For example, it does not make sense to start a general training programme if the root cause of the software problems is the result of wrong software specifications. So the question is: what causes the late and frequent software updates? The situation should be clarified by a direct information flow between the customers and the software developers. However, the only direct line between Fresh Air and its customers was the feedback flow between Product Management, some key accounts and some key customers. There is no direct contact between software developers and customers. As a result, the software developers were hardly informed about the way customers used the equipment, and the interviews made clear this was one of the reasons why software updates were necessary. The conclusion was that the middle management did not have enough information to define an effective and efficient improvement programme. It was advised to introduce a formal field feedback flow from the customers back to the software developers.
5.2.3. Executive level Given the decision at the tactic level to organise more training courses, the software developers, at the executive level, had to define the main topics of these courses. Because there was no field feedback flow from the customers to the software developers, there was no information available about the root cause of the bad performance of the software. Therefore, the software developers decided to focus the training programme on the technical design of the software. When we evaluated the situation, the training programme had run for about half a year without a serious positive effect. As the decision to start the programme was not founded on a serious analysis of the problem and as the required information was not available, we are afraid that the training programme will turn out to be disappointing. 5.3. Conclusion Based on the analysis above, it can be concluded that on the strategic level there was enough (statistical) information available to focus the attention on the right issues However, it is not yet clear whether the decision at the strategic level could not have been taken earlier. Due to lack of root cause information, the company could not come to the right conclusions on the tactic and executive level. In particular, there was no information available about the root causes of the software failures. In order to solve this problem, it was advised to introduce a field feedback flow between the customers and accounts back to the software developers. This feedback flow will concentrate on the timely feedback of root cause information.
6. Conclusions and recommendations The main conclusions of this research project are the following: - Field feedback is an essential tool for product improvement. - The maturity index for reliability has proven to be a very useful tool for the evaluation of quality oriented field feedback information flows, but it has some weaknesses. One of the major weaknesses is that it does not evaluate the timeliness of the required information. Another one is that it concentrates too much on statistical information, what tends to delay product quality improvement actions. - This paper presents a guideline for the construction of information flows. By means of a case study, it shows that the guideline can be used as a field feedback evaluation tool as well. - The case has shown that it is possible to use the proposed guideline to analyse and structure the information flows.
V.T. Petkova et al. / Reliability Engineering and System Safety 88 (2005) 147–155
It is explained that successful corrective actions are possible by a co-ordinated action of all three management levels.
References [1] Molenaar PA, Huijben AJM, Bouwhuis D, Brombacher AC. Why do quality and reliability feedback loops not always work in practice: a case study. Reliab Eng Syst Saf 2002;75:295–302. [2] Parsons T. Structure and process in modern societies. Illinois: The Free Press of Glencoe; 1960. [3] Smith RG, Reinertsen DG. Developing products in half the time: new rules, new tools. New York: Van Nostrand Reinhold; 1998. [4] Ulrich KT, Eppinger SD. Product design and development. New York: McGraw-Hill Inc; 2000. [5] Lu Y, Loh HT, Ibrahim Y, Sander PC, Brombacher AC. Reliability in a time-driven product development process. Qual Reliab Eng Int 1999;15:427–30. [6] Brombacher AC, Steinz HC, Volman HPJ. Safety and reliability assessment on products and organizations. ISA EXPO 1998Technical paper.
155
[7] Sander PC, Brombacher AC. MIR: the use of reliability information flows as a maturity index for quality management. Qual Reliab Eng Int 1999;15:439–47. [8] Petkova VT, Sander PC, Brombacher AC. The role of the service centre in improvement processes. Qual Reliab Eng Int 1999;15:431–7. [9] Petkova VT, Sander PC, Brombacher AC. The use of quality metrics in service centres. Int J Prod Econ 2000;67:27–36. [10] Petkova VT. An analysis of field feedback in consumer electronics industry. Thesis, Eindhoven University of Technology; 2003. [11] Gilmore HL. Product conformance cost. Qual Prog 1974;June:16. [12] Kuehn AA, Day RL. Strategy of product quality. Harvard Business Rev 1962;,, November-December:101. [13] Pecht MG, Ramappan V. Are components still the major problem: a review of electronic system and device field failure returns. IEEE Trans Components, Hybrids, Manuf Technol 1992;15:1160–4. [14] Gu¨thenke G, Leiters M. Fast elimination of product faults in current series. Total Qual Manage 1999;10:569–75. [15] Salau¨n I, Flores K. Information quality: meeting the need of the customer. Int J Inform Manage 2001;21:21–37. [16] Berden TPJ, Brombacher AC, Sander PC. The building bricks of product quality: an overview of some basic concepts and principles. Int J Prod Econ 1999;67:3–15. [17] Brombacher AC. MIR: Covering non-technical aspects of IEC61508 reliability certification. Reliab Eng Syst Saf 1999;66:109–20.