Classification and Mapping of PSS Evaluation Approaches

Management and on Control IFAC Conference Manufacturing Modelling, IFAC Conference on Manufacturing Modelling, IFAC Manufacturing June Conference 28-30, 2016. Troyes, France Modelling, Management and on Control Available online at www.sciencedirect.com Management and Control Management and Control June 28-30, 28-30, 2016. 2016. Troyes, Troyes, France France June June 28-30, 2016. Troyes, France

ScienceDirect

IFAC-PapersOnLine 49-12 (2016) 1555–1560 Classification and Mapping of PSS Evaluation Approaches Classification and Mapping of PSS Evaluation Approaches Classification and Classification and Mapping Mapping of of PSS PSS Evaluation Evaluation Approaches Approaches Dimitris Mourtzis*, Michael Doukas, Sophia Fotia *

Dimitris Dimitris Mourtzis Mourtzis***,, Michael Michael Doukas, Doukas, Sophia Sophia Fotia Fotia Dimitris Mourtzis , Michael Doukas,Department Sophia Fotia Lab for Manufacturing Systems and Automation, of Mechanical * * *Lab for Manufacturing Systems and Automation, Department Mechanical Engineering and Aeronautics, University of Patras, Patras 26500 Greece *Lab for Manufacturing Systems and Automation, Department of of Lab for Manufacturing Systems and Automation, Department of Mechanical Mechanical Engineering and Aeronautics, University of Patras, Patras 26500 Greece (Tel: +30 2610 997262; e-mail: [email protected] ) Engineering and University Engineering and Aeronautics, Aeronautics,e-mail: University of of Patras, Patras, Patras Patras 26500 26500 Greece Greece (Tel: (Tel: +30 +30 2610 2610 997262; 997262; e-mail: [email protected] [email protected] )) (Tel: +30 2610 997262; e-mail: [email protected] ) *

Abstract: The evaluation of a Product-Service System (PSS), particularly during its design phase but also Abstract: evaluation aa Product-Service (PSS), particularly design also throughout its lifecycle, is aof problem.System It exhibits dynamic nature during and is its multidimensional, since Abstract: The The evaluation ofchallenging Product-Service System (PSS), particularly during its design phase phase but but alsoa Abstract: The evaluation of are a Product-Service (PSS), particularly during its design phase and but alsoa throughout its lifecycle, is It exhibits dynamic nature multidimensional, since large network stakeholders usually problem. involvedSystem in design and development eco-systems their throughout its of lifecycle, is aa challenging challenging problem. It the exhibits dynamic nature and andofis is PSS multidimensional, since a throughout its lifecycle, is a challenging problem. It exhibits dynamic nature and is multidimensional, since large network of stakeholders are usually involved in the design and development of PSS eco-systems and their influence mustof be consideredareduring the evaluation ofdesign the PSS offering. Nevertheless, an effective PSSa large network stakeholders usually involved in the and development of PSS eco-systems and their large network stakeholders usually involved in PSS theofdesign and leading development of PSS eco-systems and their influence must be considered during the evaluation the PSS offering. an effective PSS evaluation canofdetermine the are market success of the offering, to Nevertheless, fewer design modifications and influence must be considered during the evaluation of the PSS offering. Nevertheless, an effective PSS influence must be considered during the evaluation of the PSS offering. Nevertheless, an effective PSS evaluation can determine the market success of the PSS offering, leading to fewer design modifications and shortened time-to-market. Still, existing literature lacks an exhaustive review and classification of PSS evaluation can determine the market success of the PSS offering, leading to fewer design modifications and evaluation can determineMoreover, the market success of the PSS offering, leadingof to performance fewerand design modifications and shortened time-to-market. Still, existing literature lacks an exhaustive review classification of PSS evaluation approaches. a lack of critical in-depth validation approaches in real shortened time-to-market. Still, existing literature lacks an exhaustive review and classification of PSS shortened time-to-market. Still, existing literature lacks an exhaustive review and classification of PSS evaluation approaches. Moreover, lack of critical in-depth performance in real industrial environments is observed.aaTowards bridging this gap, validation the presentof work identifies approaches the most important evaluation approaches. Moreover, lack of critical validation of performance approaches in evaluation approaches. Moreover, aTowards lack as of bridging critical in-depth in-depth validation ofwork performance approaches in real real industrial environments is observed. this gap, the present identifies the most important perspectives in the PSS ecosystem, such supplier selection and service level, and discusses the utilised industrial environments is observed. Towards bridging this gap, the present work identifies the most important industrial environments is observed. Towards bridging this gap, the present work identifies the most important perspectives in ecosystem, such as supplier and level, and discusses utilised evaluation used for each of them. onselection this review, classification PSSthe evaluation perspectivesapproaches in the the PSS PSS ecosystem, such as Based supplier selection anda service service level, of andexisting discusses the utilised perspectives the PSS suchPSS as supplier selection and service level, of andexisting discusses the used utilised evaluation used for of on review, PSS approaches isinprovided, byecosystem, mapping eco-system perspectives to the methods frequently for evaluation approaches approaches used for each each these of them. them. Based Based on this this review, aa classification classification of most existing PSS evaluation evaluation evaluation approaches used for each of them. Based on this review, a classification of existing PSS evaluation approaches is provided, by mapping these PSS eco-system perspectives to the methods most frequently used for their evaluation. The results can be used guidelines for PSS vendors totoselect the mostmost suitable methodused for the approaches is provided, by mapping theseasPSS eco-system perspectives the methods frequently for approaches is provided, by mapping these PSS eco-system perspectives to the methods most frequently used for their evaluation. The results can be used as guidelines for PSS vendors to select the most suitable method for the evaluation of their PSS offerings. their evaluation. The results can be used as guidelines for PSS vendors to select the most suitable method for the their evaluation. The results can be used as guidelines for PSS vendors to select the most suitable method for the evaluation of their PSS offerings. evaluation of their PSS © 2016, IFAC (International Federation of Automatic Manufacturing Control) Hostingsystems, by Elsevier Ltd. All rightsLife reserved. evaluation of their PSS offerings. offerings. Keywords: Performance evaluation, Classification, Systems design, cycles Keywords: Performance evaluation, Classification, Manufacturing systems, Systems design, Life cycles Keywords: Keywords: Performance Performance evaluation, evaluation, Classification, Classification, Manufacturing Manufacturing systems, systems, Systems Systems design, design, Life Life cycles cycles 1. INTRODUCTION Considering modern requirements, such as the need to 1. INTRODUCTION Considering modern requirements, such toas asvolatile the need need to respond rapidly, accurately, and efficiently market INTRODUCTION 1. Considering modern requirements, such the to INTRODUCTION As a consequence 1.of the manufacturing systems evolution Considering requirements, the need to respond rapidly, accurately, and efficiently efficiently toasvolatile volatile market demands,rapidly, themodern competitiveness and such sustainability of an respond accurately, and to market As a consequence of the manufacturing systems evolution from craftsmanship to customer-oriented manufacturing respond rapidly, accurately, and efficiently to volatile market As aa consequence of the manufacturing systems evolution demands, the only competitiveness anda continuous sustainability of an an enterprise can be sustained by monitoring As consequence the customer-oriented manufacturing systems evolution the competitiveness and sustainability of from manufacturing paradigm (Mourtzisof&to 2014), the servitization of demands, demands, the only competitiveness anda lifecycle sustainability of PSS an from craftsmanship craftsmanship toDoukas, customer-oriented manufacturing enterprise can be sustained by continuous monitoring of the performance throughout the of their from craftsmanship toDoukas, customer-oriented manufacturing enterprise can only be sustained by aa continuous monitoring paradigm & the of manufacturing and the (PSS) have enterprise can only be sustained by continuous monitoring paradigm (Mourtzis (Mourtzis & Product-Service Doukas, 2014), 2014), Systems the servitization servitization of offerings. of the the performance throughout the lifecycle lifecycle of their Similarly, concept evaluation is also of vital for PSS paradigm (Mourtzis & Doukas, the servitization of of throughout the their PSS manufacturing and the Systems (PSS) been globally observed the past2014), few years (Meier al., of the performance performance throughout the lifecycle of their PSS manufacturing and the Product-Service Product-Service Systems (PSS)ethave have offerings. Similarly, concept evaluation is also vital for PSS success. Measuring the performance of PSS is one of PSS the manufacturing and the Product-Service Systems (PSS) have offerings. Similarly, concept evaluation is also vital for been globally observed the past few years (Meier et al., 2010).globally PSS canobserved be described a hybrid solution comprising Similarly, concept evaluation is also vital for PSS been the as past few years (Meier et al., offerings. success. Measuring the for performance of PSS PSS is one one of of the the most important tasks a firm since it influences its been globally observed the past few years (Meier et al., success. Measuring the performance of is 2010). PSS can be described described as hybrid solution comprising comprising products andcan services for the as purpose of increasing value for success. Measuring the for performance of PSS is one of and the 2010). PSS be aa hybrid solution most important tasks a firm firm since it influences influences its competitiveness in the market, its since cost-effectiveness, 2010). PSS can be described as a hybrid solution comprising most important tasks for a it its products and services for foretthe the purpose ofPSS increasing value for for most important tasks for a firm since it influences its customersand (Shimomura al.,purpose 2015). of is an inherently products services increasing value competitiveness in the the market, its itsThis cost-effectiveness, and finally, its business performance. task aims at waste products and services for the purpose of increasing value for competitiveness in market, cost-effectiveness, and customers (Shimomura et et al., al., 2015).(Goedkoop PSS is is an anetinherently inherently dynamic, multi-dimensional concept al., 1999 competitiveness in the market, itsThis cost-effectiveness, and customers (Shimomura 2015). PSS finally, its business business performance. task aims aimsmanpower at waste waste elimination, better process control, efficient customers (Shimomura et al., 2015). PSS is an inherently finally, its performance. This task at dynamic, multi-dimensional concept (Goedkoop et al., 1999 and Lee et al., 2012), includes various actors (Xing et al., finally, its business performance. This task aims at waste dynamic, multi-dimensional concept (Goedkoop et al., 1999 elimination, better process control, control, efficient manpower utilization, better and employment of efficient flexible manpower systems dynamic, concept (Goedkoop et al.,et1999 elimination, process and Leeand etmulti-dimensional al., 2012), 2012), includes various actors (Chen (Xing 2013), presents uncertainties in its design elimination, better process control, efficient manpower and Lee et al., includes various actors (Xing et al., al., utilization, and employment of flexible systems (Chryssolouris, 2006). and Lee et al., 2012), includes various actors (Xing et al., utilization, and employment of flexible systems 2013), and presentsitsuncertainties uncertainties in its design (Chen (Chen et The al., utilization, and employment of flexible systems 2015), and rendering evaluation in a its challenging task.et 2013), presents design al., (Chryssolouris, 2006). 2013), and presents its design (Chen al., 2006). 2015), rendering itsuncertainties evaluation challenging task.toet The The evaluation becomes even more in due the (Chryssolouris, 2. LITERATURE (Chryssolouris, 2006). REVIEW ON PSS EVALUATION 2015), rendering its evaluation aachallenging challenging task. 2015), rendering its evaluation a challenging task. The evaluation becomes even more challenging due to the 2. LITERATURE REVIEW ON ON PSS PSS EVALUATION EVALUATION interrelated becomes structure of stakeholders who have due a long-term evaluation even more challenging to the 2. LITERATURE REVIEW evaluation becomes even morewith challenging due to the 2. the LITERATURE REVIEW ON has PSSbeen EVALUATION Over last decade, much research carried out on interrelated structure of stakeholders who have a long-term relationship and communication each other (Baines et interrelated structure of stakeholders who have a long-term interrelated structure of stakeholders who have a long-term Over the last decade, much research has been carried on PSS design methods, methodologies, and tools, whileout there Over the last decade, much research has been carried on relationship and communication with each other (Baines et al., 2007, and Shimomura et al., 2015). Academic community relationship and communication with each other (Baines et Over the last decade, much research has been carried out out on relationship and communication with each other (Baines et PSS design methods, methodologies, and tools, while there are numerous review papers which gather this research, design methods, methodologies, and tools, while there al., 2007, and Shimomura Shimomura et al., al., 2015). Academic Academic community has 2007, long been aware of the importance of evaluation in the PSS al., and et 2015). community PSS design methods, methodologies, and tools, while there al., and Shimomura et al., 2015). Academic community are numerous review papers which isgather gather this research, however, the PSS concept evaluation yet at this a preliminary numerous review papers which research, has long been aware of the the importance of evaluation evaluation in but the are PSS2007, development stages (Komoto & Tomiyama, 2008), has long been aware of importance of in the are numerous review papers which gather this research, has long been aware of the importance of evaluation in the however, the PSS concept evaluation is yet at a preliminary level (Baines et al., 2007, Vasantha et al., 2012, and Tran & the PSS concept evaluation is yet at aa preliminary PSS development stagesby(Komoto (Komoto &et Tomiyama, Tomiyama, 2008), but as initially indicated Baines & al. (2007),2008), later but by however, PSS development stages however, the PSS concept evaluation is yet at preliminary PSS development stages (Komoto & Tomiyama, 2008), but level (Baines et al., 2007, Vasantha et al., 2012, and & Park, 2015). Yet, the evaluation of PSS has not been in focus level (Baines et al., 2007, Vasantha et al., 2012, and Tran & as initiallyet indicated indicated by Baines et al. al. by (2007), later by level (Baines et al., 2007, Vasantha et al., 2012, and Tran Vasantha al. (2012),by andBaines more resent (Tran later & Park, as initially et (2007), by Tran & as initially indicated by Baines et al. (2007), later by Park, 2015). Yet, the evaluation of PSS has not been in focus of PSS researchers in early years, due to the fact that Park, 2015). Yet, the evaluation of PSS has not been in focus Vasantha etevaluation al. (2012), (2012), and more resent by (Tran (Tranfield & Park, Park, 2015), theet of and PSS more is stillresent an immature with Park, 2015). Yet, the evaluation of PSS has not been in focus Vasantha al. by & Vasantha et al. (2012), and more resent by (Tran & Park, of PSS PSS researchers researchers in early years, due due to the the fact fact This that emphasis was given in on early development methodologies. years, to that 2015), the evaluation evaluation of PSS PSS is still still an an immature field with with of few concrete results and approaches. Due to the substantial 2015), the of is immature field of PSS researchers in years,search due to the about fact This that 2015), the evaluation of PSS isand stillservices, an immature field with emphasis was given given on1,early development methodologies. fact is reflected in Fig. in which results the emphasis was on development methodologies. This few concrete results and approaches. Due to the substantial differences between products the concept of few concrete results and approaches. Due to the substantial emphasis was given on development methodologies. This few concrete resultsdiffers and approaches. Due to the the concept substantial fact is reflected reflected in Fig. Fig.and 1, in in which search results results about the general PSS literature, thewhich corresponding aboutabout the PSS fact is in 1, search the differences between products and services, of PSS evaluation from conventional evaluation differences between products and services, the concept of fact is reflected in Fig. 1, in which search results about the differences between products and services, the concept of general PSS literature, and the corresponding about the PSS evaluation methods, from Google Scholar and Scopus PSS literature, and corresponding about the PSS PSS evaluation differs conventional evaluation problems, since product andfrom service activities interact and general PSS evaluation differs from conventional evaluation general PSSmethods, literature, and the theGoogle corresponding about PSS PSS evaluation differs from conventional evaluation evaluation from Scholar and the Scopus citation databases, are presented. In the Scholar period between 1999evaluation methods, from Google and Scopus problems, since product and service activities interact and influence each other (Chen et al., 2015). Furthermore, the problems, since product and service activities interact and evaluation methods, from Google Scholar and Scopus problems, since product andetservice activities interact and citation databases, are presented. In the therelated period between between 19992016, only 18% are of presented. the literature to PSS 1999work databases, In period influence each other (Chen al.,the 2015). Furthermore, the citation relationships among the criteria of PSS concept are much influence each other (Chen et al., 2015). Furthermore, the citation databases, are presented. Inevaluation, therelated period while between 1999influence each other (Chen et al., 2015). Furthermore, the 2016, only 18% of the literature to PSS work mentions the importance of PSS only 5% only 18% of related to PSS work relationships amongthan the criteria criteria of pure the PSS PSS concept much more complicated those of products orare services relationships among the of the concept are much 2016, 2016, only 18% of the theofliterature literature related while to approaches. PSS work relationships among the criteria of the PSS concept are much mentions the importance PSS evaluation, only 5% provides comprehensive evaluation mentions the importance of PSS evaluation, while only 5% more complicated than those of pure pure products or solution services (Lee et al., 2012).than Thethose previous cause the PSS more complicated of products or services mentions the importance of PSS evaluation, while only 5% more complicated than those of pure products or services provides comprehensive PSS evaluation approaches. Interestingly, as depicted in Fig. 1, although the formal PSS provides comprehensive PSS evaluation approaches. (Lee et al., 2012). The previous cause the PSS solution evaluation process toThe be treated as cause a complex multi-criteria (Lee et 2012). previous the provides comprehensive PSS solution (Lee et al., al.,process 2012). previous cause the PSS PSS solution Interestingly, Interestingly, as depicted depicted in Fig. Fig. 1, evaluation althoughetthe the formal PSS topic was introduced in 1999 by Goedkoop al. approaches. (1999),PSS the as in 1, although formal evaluation toThe be treated treated asal., a 2015). complex multi-criteria decision-making problem (Chen etas evaluation process to be a complex multi-criteria Interestingly, as depicted in Fig. 1, although the formal PSS evaluation process to be treated as a complex multi-criteria topic was introduced in 1999 by Goedkoop et al. (1999), the topic was introduced in 1999 by Goedkoop et al. (1999), the decision-making problem (Chen et al., 2015). decision-making topic was introduced in 1999 by Goedkoop et al. (1999), the decision-making problem problem (Chen (Chen et et al., al., 2015). 2015).

Copyright © 2016 IFAC 1555 2405-8963 © 2016, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright 2016 responsibility IFAC 1555Control. Peer review© under of International Federation of Automatic Copyright © 2016 IFAC 1555 Copyright © 2016 IFAC 1555 10.1016/j.ifacol.2016.07.801

IFAC MIM 2016 June 28-30, 2016. Troyes, France 1556

Dimitris Mourtzis et al. / IFAC-PapersOnLine 49-12 (2016) 1555–1560

100 90 80 70 60 50 40 30 20 10 0

No. of Publications vs. Year

7 8 8 10 0 0 0 0 0 0 0 0 3 2

17

26

32

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

No. of Publications

first work on PSS evaluation appears 8 years later in the work of Shimomura & Sakao (2007). The investigation of the literature led to the identification of the following widelyconsidered PSS perspectives: customer, sustainability, early design, risk and uncertainty, knowledge, suppliers’ selection, service level, human resource allocation, cost, feasibility of new PSS idea, and lifecycle.

Year

Fig. 1. Comparison between literature on PSS (blue bar) and PSS evaluation (red bar). 2.1 Customer perspective of PSS evaluation For realizing value in a PSS, the design should be focused on customers and their requirements. Therefore, the evaluation of customer satisfaction, in order to support the PSS design, has attracted attention in the PSS research community. Many methods have been suggested for that purpose, mainly using decision making theory. Regarding customer satisfaction, existing studies aim at designing PSSs effectively and efficiently, considering customer satisfaction for design alternatives. Kimita et al. (2008a) introduce a method for estimating customer satisfaction using a non-linear value function, which enables designers to compare design solutions at the conceptual stage. Geng & Chu (2012) evaluate the customer satisfaction in the PSS concept by using Important-Performance Analysis (IPA) integrating the Kano’s model and by taking into consideration the mutual influence relationships among attributes by Decision Making Trial and Evaluation Laboratory (DEMATEL). Moreover, Pan & Nguyen (2015) used the Balanced Scorecard (BSC) and Multiple Criteria Decision Making (MCDM) approaches in order to identify appropriate performance criteria for the purpose of achieving customer satisfaction. The transition from purely product-oriented offerings to PSS is difficult since many customers hesitate to adopt this new concept. This issue is recognized by Lee et al. (2015), contributing to support PSS strategies by evaluating their likely acceptability by customers. This is carried out through combining the Analytic Network Process and the Niche Theory to quantify customer value on given PSS offerings. Regarding the evaluation of customer requirements, Song et al. (2013) suggest a framework for addressing the vagueness and the subjectivity of the evaluation of these requirements. The proposed model evaluates requirements under vagueness, considering the customer activity cycle, using Rough Group Analytic Hierarchy Process methods. In the same direction, Lee & Huang (2009) developed the fuzzy Kano’s questionnaire (FKQ) and the fuzzy Kano’s mode (FKM) for analysing customer requirements. After the questionnaires are completed, the designers can obtain data sets of customer expectation, quality, and satisfaction.

2.2 Sustainability perspective of PSS evaluation PSS is the par excellence business strategy that promises sustainability (Goedkoop et al., 1999 and Baines et al., 2007). For a sustainable growth of a PSS, the evaluation should be done in a balanced manner, which covers the basic sustainability pillars, namely environment, economy, and society. However, the majority of literature on PSS evaluation, usually focuses on addressing the aspect of “environmental” performance of PSS, while very few are devoted to the evaluation of the tree perspectives (Peruzzini et al. 2013). Works that integrate all three aspects into the evaluation procedure are very limited (Lee et al., 2012 and Kim et al., 2013). Sustainability problems are characterized by divergent values and norms among decision-makers, high uncertainty about causes, solutions, and risks (Chen et al., 2015). For this reason, most of the literature work, addresses the problem of sustainability evaluation as a fuzzy decisionmaking problem. Moreover, axiomatic design provides a logical and systematic method for creating and evaluating PSS design (Kimita et al., 2008a), which is a solution based on the hybrid uncertain Information Axiom. The term hybrid characterizes the environment that is governed at the same time by fuzziness and randomness. Taking advantages of the previous concepts, Chen et al. (2015) propose an evaluation method based on the Information Axiom, in order to evaluate PSS solutions within hybrid environments. Hu et al. (2012) propose a PSS evaluation model based on the Fuzzy Analytic Hierarchy Process, while Lee et al. (2012) suggest a dynamic and multidimensional approach to measure PSS sustainability, using the System Dynamics (SD) simulation method and “triple bottom line” (TBL) approach for the systematically linked socio-economic-environmental sustainability system. The index measurement approach proposed by Xing et al. (2013) comprises an approach that utilises appropriate indicators to measure performance, cost, and environmental impact. For measuring purposes, the Net Present Value (NPV) and Life Cycle Assessment (LCA) are applied. Abramovici et al. (2014) propose a framework of KPIs in order to evaluate sustainability aspects through the entire PSS lifecycle. Moreover, Chirumalla et al. (2013) recommend a performance measurement framework for PSS development based on BSC using multi-criteria and multihierarchical levels. Chou et al. (2015) establish a hierarchical structure of multiple criteria for assessment of sustainability, and use a scale ranging from one to five to perform the PSS evaluation, inspired by the SERVQUAL/SERVPERF model. 2.3 Entire PSS Lifecycle evaluation In literature, a commonly accepted PSS lifecycle is the one proposed by Rese et al. (2012) that comprise the planning, development, implementation, delivery and use, and closure. However, only few literature work is devoted to the evaluation of the entire PSS lifecycle. As previously shown, most of them target the assessment of PSS sustainability, such as the works of Abramovici et al. (2014), Xing, et al. (2013), and Chirumalla et al. (2013). It is observed that most PSS evaluation approaches that address the entire PSS lifecycle are based on KPIs. In this direction, Kim et al. (2013) propose an evaluation scheme in which all the phases of PSS lifecycle are taken into account, from customer and company perspectives, by using appropriate PSS lifecycle-

1556

IFAC MIM 2016 June 28-30, 2016. Troyes, France

Dimitris Mourtzis et al. / IFAC-PapersOnLine 49-12 (2016) 1555–1560

dependent performance criteria. A particular phase of PSS is the early design phase, which is utterly important for the subsequent success of the offering since it can determine the superiority of the final design solution. Few studies however, analyse and correlate the connection between PSS design process and solution (Shimomura et al., 2015). In order to evaluate the early phase of PSS design, Exner & Stark (2015) suggest an innovative approach named SHP4PSS that incorporates Virtual Reality tools in order to enable a realistic representation of different lifecycle phases. Moreover, the works of Kimita et al. focus on PSS assessment of early design phase, from both customer (Kimita et al., 2008a) and cost (Kimita et al., 2008b) perspectives. Finally, (Mourtzis, et al., 2015) gather the most appropriate KPIs for PSS, which are monitored in different lifecycle phases, and present a conceptual framework for effective PSS design models. 2.4 Cost evaluation in PSS. Mannweiler et al. (2010) put forward a methodology to assess the costs arising throughout the whole lifecycle of PSS. In this methodology, a two-dimensional classification is carried out. The investigated costs are first classified into disinvestment, utilization, and investment, and secondly into physical components, non-physical components, and lifecycle characteristics. Another contribution for cost assessment of PSS comes from Kimita et al. (2008b), who propose a methodology to evaluate the services from both customer importance and the economic cost perspectives. In the context of this work, 15 functions and 17 activities of service engineering are considered. Finally, the financial cost of each function is calculated concurrently with a customer importance analysis. 2.5 Risk and Uncertainty evaluation in Supply Chain Zeiler & Bertsche (2014) present a modelling method based on the conjoint system model and a Monte Carlo simulation procedure in order to simulate the reliability and maintenance for the risk management of PSS-related services in different stages of PSS lifecycle. Using fuzzy-based decision-making techniques (TOPSIS, DEMATEL, Delphi, and Analytical Hierarchy Process), suggest an approach for evaluating the uncertainty in networks of: (i) the supply chain (Durugbo & Wang, 2014) and (ii) the delivery phase (Wang & Durugbo, 2013)). In a similar context, Omer et al. (2014) uses multiple domain matrices and agent-based modelling in order to identify and evaluate a set of risk factors that concern the supply chain. Abramovici et al. (2013) propose a KPIs framework for the use phase of PSS, aiming at early risk monitoring. A two-step evaluation framework has been introduced (Sun et al., 2012) at a microscopic and a macroscopic level. The microscopic level is associated with the quality and cost of the part machining service networks, in a quantitative manner. The macroscopic level deals with the networks’ time, stability and robustness. The evaluation result attempts to benefit the network’s establishment, monitoring, and optimization. 2.6 Evaluation of other PSS aspects Concentrating on the sustainability and quality of PSS offerings of a machine tool manufacturer, a software tool is developed based on a KPIs monitoring framework (Mert &

1557

Aurich, 2015). The software evaluates the quality of PSS regarding the fulfilment of customer requirements. Sun et al. (2012) develop evaluation indices to measure the interrelationship between a PSS's provider and receiver. These indices regard time, quality, cost, stability, and reliability. For the evaluation of supplier selection, Geng & Liu (2015) propose a methodology that use the rough set mining approach for extracting the criteria weights according to the surveyed performance information, and after that, a set of criteria weights feeds the VIKOR approach that carries out the evaluation. Under the perspective of service evaluation, fisrt Shimomura and Sakao (2007) use QFD (Quality Deployment Function), DEMATEL, and AHP (Analytical Hierarchy Process) to demonstrate a method for measuring the effectiveness of service, which is conducted by a provider during the design process in order to generate the largest value for all the concerned agents. Li et al. (2014) present the universal Enterprise Manufacturing Service Maturity Model (EMSMM) in order to carry out the evaluation of a company’s current service development phase and level. Yoon et al. (2012) develop an evaluation framework in order to evaluate the feasibility of a new PSS development considering appropriate measurements from the provider and customer perspectives. From the provider side of PSS evaluation, authors focus on economic, technology, and political-legal feasibility, while for the customer’s side, expected value intension to adoption, and referred use of service. Shimomura et al. (2013) propose a method to guide the human resource allocation in order to design higher quality PSS, using a neural network to predict customer satisfaction realised by each human resource. Schenkl et al. (2014) develop an enhanced MDM-based knowledge map as an evaluation framework of knowledge within a company in order to specify the required level of knowledge that provider needs for a specific PSS. 3.

MAPPING PSS PERSPECTIVE AND METHODS

PSS has been established as a prevailing business model in most companies worldwide. However, academic literature presents gaps related to the evaluation approaches for PSS. Indicatively, there is a lack of critical and in-depth evaluation of PSS performance, while there are few methodologies that consider feedback from the entire lifecycle of a PSS. These gaps may be attributed to the lack of PSS-oriented software tools and the limited comprehensive quantitative methods that can help organizations understand the perceived value that a potential customer may hold and to evaluate the level of service that is required by the market. Finally, most evaluation methods consider only the customer satisfaction without balancing or reviewing it from a company perspective. Besides, most approaches emphasize on the environmental and social gains without considering the economic impact of the PSS. This section aims to provide a mapping of the existing PSS evaluation approaches according to “what” and “how” they evaluate different PSS perspectives. This mapping is presented in Fig. 2, where the centre of cycle contains all the literature work devoted to the PSS Evaluation. Replying to the “what” question, each reference from the centre is linked to the PSS

1557

IFAC MIM 2016 June 28-30, 2016. Troyes, France 1558

Dimitris Mourtzis et al. / IFAC-PapersOnLine 49-12 (2016) 1555–1560

perspectives (left hand side of Fig. 2) that it intends to evaluate. Moreover, the literature works are linked to the methods that have been used in order to build up the evaluation procedure. This right-hand side of the figure replies to the “how” question, used by the authors to evaluate the corresponding PSS perspectives. The acronyms of Fig. 3 are explained in Table 1. Additionally, Table 1 describes the usage of the main evaluation methods. This chart can be used as a quick reference for academics and practitioners who are interested in understanding: (i) which are the available evaluation methods for PSS, (ii) which PSS perspectives are addressed by existing evaluation methods, (iii) how often has each method been used, (iv) how often each PSS perspective has been evaluated, (v) and what methods and perspectives are preferred in international literature. Interpreting Fig. 3, the PSS perspectives that are mostly evaluated are denoted with heavy line. These perspectives are the Lifecycle, Sustainability, Risk and Uncertainty, and Customer Satisfaction. Similarly, the degree of the utilization of the several evaluation methods by authors, are coded with color and gradually heaviness of grey linked lines. As it is seen in Fig. 3, the mostly used methods are the MCDM, Questionnaires and KPIs. On the other hand, sparse lines deriving from a perspective and arriving at a method represent their limited use in literature. Tool Questionnaire IPA DEMATEL ANP Delphi Method KPIs Niche Theory Non-Linear Function Kano’s Model Balanced Scorecard (BSC) Rough Set Theory QFD SERVQUAL Neural Networks Information Axiom TBL framework SHP4PSS

4.

CONCLUSIONS

This work presented an extensive literature review on PSS evaluation approaches, identified and analysed significant perspectives of PSS eco-system, and classified the tools that are commonly used to evaluate them. The identified perspectives of PSS evaluation were customer, sustainability, early design, risk and uncertainty, knowledge, supplier selection, service level, human resource allocation, cost, feasibility of new PSS idea, and lifecycle. Several different methods have been used for the evaluation of these perspectives, such as questionnaires and KPIs, while it was found that Multi-Criteria Decision Making is the preferred evaluation method. The deriving methods classification and their mapping with the PSS perspectives comprises a guide for the selection of the most appropriate method for PSS evaluation, for both academic and industrial applications. Future work will focus on the development of a comprehensive PSS evaluation methodology, incorporating feedback from the entire PSS lifecycle, KPI measurements from the customer side, production/manufacturing, shop-floor experts, and other relevant stakeholders.

Table 1. Tools used in the evaluation of PSS perspectives

Description of the usage of tool for the PSS evaluation Usually the first step of most evaluation approaches, which aims at gathering critical customer related attributes Important-Performance Analysis (IPA) is a two-dimensional grid-based graphical representation of attribute performance and importance, for providing customer satisfaction evaluation (Martilla & James, 1977) DEcision-MAking Trial and Evaluation Laboratory (DEMATEL) is a tool for analysing the influence relationships among the PSS elements in a system and identifying causal and effect groups in the system through a causal diagram (Fontela & Gabus, 1976) Analytic Network Process (ANP) is a generalization of the Analytical Hierarchy Process (Saaty, 1980), which is one of the most widely used Multiple-Criteria decision Making methods to better capture human psychology in decision making Tests a MCDM framework verifying the answers of respondents by repeatedly collecting answers of experts via interview or questionnaire surveys (Warfield,1976) Qualitative or quantitative indices of cost or benefit character, the value of which can be compared against an internal target or an external “benchmark” to give an indication of performance Tool for analysing the preferences of customers for new PSS offerings given its ability to examine the competition between two offerings via customer survey (Pianka, 1981) A non-linear function is used to estimate satisfaction, for better capturing of the relationship between quality and satisfaction in the customer perception of a PSS The two-dimensional quality and evaluation model to characterize product or service attributes, which are classified into: must-be (M), one-dimensional (O), attractive (A), indifferent (I), and reverse (R) (Kano, 1984) BSC is a four-dimensional framework for effective management of operational organization’s activities. Its four dimensions are: financial, customer, internal process, and earning and growth (Kaplan & Norton, 1992) This Theory can enable stakeholders to express their true perception and evaluation without a priori information (Pawlak, 1998) Quality Function Deployment (QFD) transforms the quantitative customer demands into qualitative parameters. In the PSS, it use to translate the customer requirements into the priorities of products-services (Akao, 1990) Evaluates service quality (SERVQUAL) aspects using a 7-point linguistic scale. The global quality measure is the average of the dimensions in the terms of the customer’s perception and expectations towards the service (Parasuraman et al., 1991) Architecture to represent a number of PSS interconnected processing elements. These elements, called neurons, are organized commonly in three layers: input, hidden, and output layers Tool for selecting the best design solution among all feasible alternatives. Information Axiom approaches are classified into four categories: traditional deterministic, traditional random, fuzzy, and hybrid uncertain (Suh, 2001) Triple Bottom Line (TBL) framework. Provides a comprehensive view, which depicts dynamic interaction among the three dimensions of PSS sustainability (environmental, economical, and societal) (Elkington, 1999) Smart Hybrid Prototype (SHP) approach integrates physical prototypes, digital models, and software in Virtual Reality (VR) to enable an experiencing of PSS for an urban mobility use case (Exner & Stark, 2015) 1558

IFAC MIM 2016 June 28-30, 2016. Troyes, France

Dimitris Mourtzis et al. / IFAC-PapersOnLine 49-12 (2016) 1555–1560

AHP

Customer requirements Supplier Selection

(Song, et al., 2013) (Geng & Liu, 2015) (Lee, et al., 2009) (Kimita, et al., 2008 (a)) (Pan & Nguyen, 2015) Customer (Geng & Chu, 2012) Acceptance (Lee, et al., 2015) (Mourtzis, et al., 2015) (Abramovici, et al., 2014) Lifecycle (Exner & Stark, 2015) (Chou, et al., 2015) (Kim, et al., 2013) Sustainability (Xing, et al., 2013) (Lee, et al., 2012) (Chen, et al., 2015) Knowledge (Hu, et al., 2012) (Chirumalla, et al.,2013) (Peruzzini, et al., 2013) (Schenkl, et al., 2014) Service Level (Li, et al., 2014) (Shimomura & Sakao, 2007) Human (Shimomura, et al., 2013) Resources (Kimita, et al., 2008 (b)) Allocation (Mannweiler, et al., 2010) (Durugbo & Wang, 2014, 2015) Cost (Zeiler & Bertsche, 2014) (Abramovici, et al., 2013) (Omer, et al., 2014) Risk and Uncertainty (Yoon, et al., 2012)

Customer Satisfaction

Feasibility

1559

Balanced Scorecard Non-linear Function Neural Networks DEMATEL Rough Set Theory HP4PSS

MDM Niche Theory

QFD KPIs MCDM TBL

IPA Questionnaire

Kano’s Model Information Axiom SERVQUAL/SERVPERF

Monte Carlo simulation System Dynamic Simulation

Perspectives of PSS Evaluation

PSS Evaluation Literature

Methods for PSS Evaluation

Fig. 2. Mapping of the PSS evaluation approaches to the PSS perspectives. ACKNOWLEDGEMENTS This work has been partially supported by the H2020 EC funded project “Cloud Manufacturing and Social Softwarebased Context Sensitive Product-Service Engineering Environment for Globally Distributed Enterprise DIVERSITY” (GA No: 636692). REFERENCES Abramovici, M., Aidi, Y., Quezada, A., and Schindler, T. (2014). PSS Sustainability Assessment and Monitoring framework (PSS-SAM) - Case study of a multi-module PSS Solution. Procedia CIRP, IPSS, vol. 16, pp. 140 – 145. Abramovici, M., Jin, F., and Dang, H. B. (2013). An Indicator Framework for Monitoring IPS² in the Use Phase. ProductService Integration for Sustainable Solutions, pp. 311–322. Akao, Y. (1990). Quality Function Deployment, Productivity Press, Cambridge. Baines, T. et al. (2007). State-of-the-art in product-service systems, J Eng Manuf, vol. 221(10), pp. 1543-1552. Chen, D., Chu, X., and Yang, X. (2015). PSS solution evaluation considering sustainability under hybrid uncertain environments. Expert Systems with Applications, vol. 42(14), pp. 5822-5838. Chirumalla, K. et al. (2013). Performance measurement framework for product-Service systems development: A balanced scorecard approach. International Journal of Technology Intelligence and Planning, vol. 9(2), pp. 146164.

Chou, C., Chen, C., and Conley, C. (2015). An approach to assessing sustainable product-service systems. J Clean Prod, vol. 86, pp. 277-284. Chryssolouris, G. (2006). Manufacturing Systems: Theory and Practice. 2nd ed., Springer-Verlag, New York. Durugbo, C., and Wang, X. (2014). Network-Oriented Uncertainty Evaluation of IPSS Collaborative Readiness. Procedia CIRP, IPSS, vol. 16, pp. 229 – 234. Elkington, J. (1999). Cannibals with forks: the triple bottom line of 21st century business. Oxford, Capstone. Exner, K., and Stark, R. (2015). Validation of product-service systems in virtual reality. Procedia CIRP, vol. 30, pp. 96101. Fontela, E., and Gabus, A. (1976). The DEMATEL observer. Battelle Geneva Research Center, Switzerland, Geneva. Geng, X., and Chu, X.(2012). A new importance-performance analysis approach for customer satisfaction evaluation supporting PSS design. Expert Syst Appl, vol. 39,pp. 14921502. Geng, X., and Liu, Q. (2015). A hybrid service supplier selection approach based on variable precision rough set and VIKOR for developing product service system. Int J Comput Integr Manuf, vol. 28(10), pp. 1063-1076. Goedkoop, M., van Halen, C., te Riele, H., and Rommes, P. (1999). Product Services Systems, Ecological and Economic Basics. VROM, The Hague. Hu, H. et al. (2012). Development of sustainability evaluation model for implementing product service systems. IJ Environ Sci and Technol, vol. 9(2), pp. 343-354.

1559

IFAC MIM 2016 1560 June 28-30, 2016. Troyes, France

Dimitris Mourtzis et al. / IFAC-PapersOnLine 49-12 (2016) 1555–1560

Kano, N., et al. (1984). Attractive quality and must-be quality, Hinshitsu: The Journal of the Japanese Society for Quality Control, vol. 2, pp. 39–48. Kaplan, R., and Norton, D. (1992). The balanced scorecard: measures that drive performance, Harvard Business Review, pp. 71-79. Kim, K. et al. (2013). An Evaluation Scheme for ProductService System Models with a Lifecycle Consideration from Customer’s Perspective. Re-Engineering Manufacturing for Sustainability–20th CIRP Int. Conference on Life Cycle Engineering, pp. 69-74. Kimita, K., Shimomura, Y., and Arai, T. (2008(a)). Evaluation of customer satisfaction for PSS design. Expert Syst Appl, vol. 36, pp. 1268-1279. Kimita, K., Hara, T., Shimomura, Y., and Arai, T. (2008(b)). Cost Evaluation Method for Service Design Based on Activity Based Costing. Manufacturing Systems and Technologies for the New Frontier, pp. 477-480. Komoto, H., and Tomiyama, T. (2008). Integration of a service CAD and a life cycle simulator. CIRP Annals, 57, pp. 9–12. Lee, C., Geum, Y., Lee, H., and Park, Y. (2012). Dynamic and multidimensional measurement of product-service system (PSS) sustainability: a triple bottom line (TBL)-based system dynamics approach. J Clean Prod, vol. 32, pp. 173 182. Lee, S., Geum, Y., Lee, S., and Park, Y. (2015). Evaluating new concepts of PSS based on the customer value: Application of ANP and niche theory. Expert Syst Appl, vol. 42(9), pp. 4556-4566. Lee, Y. C., and Huang, S. Y. (2009). A new fuzzy concept approach for Kano’s model. Expert Syst Appl, vol. 36(3), pp. 4479–4484. Li, H. et al. (2014). A universal enterprise manufacturing services maturity model: A case study in a Chinese company. Int J Comput Integr, vol. 27(5), pp. 434-449. Löfstrand, M., Kyösti, P., Reed, S., and Backe, B. (2014). Evaluating availability of functional products through simulation. Simualtion Modelling Practice & Theory, 47, pp. 196-209. Mannweiler, C., Siener, M., and Aurich, J. (2010). Lifecycle Cost oriented Evaluation and Selection of Product-Service System Variants. 2nd CIRP IPSS, pp. 21-26. Martilla, J.A., and James J.C., 1977. Importance-Performance Analysis, Journal of Marketing, vol. 41(1), pp. 77-79. Meier, H., Roy, R., Seliger, G., (2010). Industrial ProductService –IPS2. CIRP Annals; vol. 59, pp. 607–627. Mert, G. and. Aurich, J.C, (2015). A Software Demonstrator for Measuring the Quality of PSS, Procedia CIRP, vol. 30, pp. 209-214 Mourtzis, D., Doukas, M., (2014). The evolution of manufacturing systems: From craftsmanship to the era of customisation. Design and Management of Lean Production Systems, pp. 1–29. Mourtzis, D., Fotia, S., and Doukas, M. (2015). Performance Indicators for the Evaluation of Product-Service Systems Design: A Review. Advances in Production Management Systems: Innovative Production Management Towards Sustainable Growth, vol. 460, pp. 592-601. Omer, M., Ganguly, A., Behncke, F., and Hollauer, A. (2014). A complexity driven approach for risk evaluation in useoriented product-service systems supply chains. In 9th SOSE Conference, vol. 20(8), pp. 2071-2078.

Pan, J.-N., and Nguyen, H. T. N. (2015). Achieving customer satisfaction through product-service systems. Eur J Oper Res, vol. 247(1), pp. 179-190. Parasuraman, A., Zeithaml, VA., Berry, LL. (1991). Refinement and Reassessment of the SERVQUAL Scale, Journal of Retailing vol. 67(4). Pawlak, Z., (1998). Rough Set Theory and its Applications to Data Analysis, Cybernetics and Systems: An International Journal, vol. 29 (7), pp. 661-688 Peruzzini, M., Germani, M., Marilungo, E. (2013). ProductService Sustainability Assessment in Virtual Manufacturing Enterprises, IFIP AICT, vol. 408, pp. 13–2. Pianka, E.R.,(1981). Competition and Niche Theory, Theoretical ecology principles and applications, vol. 8, pp.167-196. Rese, M., Meier, H., Gesing, J., and Boblau, M. (2012). An Ontology of Business Models for Industrial PSS. The Philosopher's Stone for Sustainability, pp. 191-196. Saaty, T. L. (1980). The Analytic Hierarchy Process. McGrawHill, New York. Schenkl, S., Schmidt, D. M., Schockenhoff, D., and Maurer, M., 2014. Knowledge Evaluation for PSS Providers. Procedia CIRP, vol. 16, pp. 86–91. Shimomura, Y., and Sakao, T., 2007. A Service Evaluation Method for Service/Product Engineering, ICED. Shimomura, Y. et al., 2013. A method for human resource evaluation to realise high-quality PSSs. CIRP Annals Manufacturing Technology, vol. 62(1), pp. 471–474. Shimomura, Y., Nemoto, Y., and Kimita, K., 2015. A method for analysing conceptual design process of product-service systems. CIRP Annals - Manufacturing Technology, vol. 64(1), pp. 145-148. Song, W., Ming, X., Han, Y., and Wu, Z., 2013. A rough set approach for evaluating vague customer requirement of industrial product-service system. Int J Prod Res, vol. 51(22), pp. 6681-6701. Suh, N.P, 2001. Axiomatic Design: Advances and Applications, Oxford University Press, New York. Sun, H. et al. (2012). Evaluation method of product-service performance. Int J Comput Integr, vol. 25(2), pp. 150-157. Tran, Τ., and Park, J. (2015). Development of a strategic prototyping framework for product service systems using co-creation approach. Procedia CIRP, IPSS, 30, pp. 1-6. Vasantha, G., Roy, R., Lelah, A., and Brissaud, D. (2012). A review of product–service systems design methodologies. Journal of Engineering Design, vol. 23(9), pp. 635-659. Wang, X., and Durugbo, C. (2013). Analysing network uncertainty for industrial product-service delivery: A hybrid fuzzy approach. Expert Syst Appl, 40(11), pp. 4621-4636. Warfield, J. N. (1976). Societal Systems-Planning, Policy, and Complexity. Wiley Law Publications, New York. Xing, K., Wang, H.-F., and Qian, W. (2013). A sustainabilityoriented multi-dimensional value assessment model for product-service development. Int J Prod Res, vol. 51(19), pp. 5908-5933. Yang, J.-H. (2009). A balanced performance measurement scorecard approach for Product Service Systems. BIFE, pp. 548-551. Yoon, B., Kim, S., and Rhee, J., 2012. An evaluation method for designing a new product-service system. Expert Syst Appl, vol. 39(3), pp. 3100-3108. Zeiler, P., and Bertsche, B. (2014). Simulation-based risk management of product-service systems. RAMS, pp. 1- 6.

1560