- Email: [email protected]
TELOS: a customer satisfaction evaluation software
Computers & Operations Research 27 (2000) 799}817
TELOS: a customer satisfaction evaluation softwareq Evangelos Grigoroudis!,*, Yannis Siskos!, Olivier Saurais" !Technical University of Crete, Decision Support Systems Laboratory, University Campus, 73100 Chania, Greece "Management of Strategic Resources S.A., Av. Lavaux 101, 1009 Pully, Switzerland
Abstract Telos is a consumer-based tool for measuring and analysing customer satisfaction. The software uses survey data on customer satisfaction judgements while the analysis of collected information is based on a preference disaggregation model. The implemented methodology follows the principles of multicriteria analysis using mainly ordinal regression techniques. The main advantage of the presented software is that TELOS fully considers the qualitative form of customers' judgements and preferences. A simple numerical example is used for illustrating TELOS's basic features like simplicity, friendliness, and e!ectiveness. Finally, an overview of customer service management technologies is brie#y presented and several extensions of TELOS's features and capabilities are discussed. Scope and purpose Customer satisfaction research is one of the fastest growing segments of the marketing research "eld. Its main objective is to measure strengths and weaknesses of a business organisation, while getting closer to the customer. Several methodological approaches, based on di!erent scienti"c areas, are used in order to de"ne, measure and analyse customer satisfaction. Many of these models do not consider the qualitative form of customers' judgements, although this information is the basic customer satisfaction input data. Usually, necessary input data cannot be easily collected, and, at the same time, the results are not fully reliable. Furthermore, in several cases measurements are not su$cient enough to analyse in detail customer satisfaction because models' results are mainly focused on a simple descriptive analysis. TELOS, a customer satisfaction measuring and analysing software which is discussed through this paper, overcomes these disadvantages, by implementing a multiple criteria preference disaggregation methodology, which provides to the user a complete and e!ective series of results. The main objective of the paper is to present TELOS's basic features and capabilities, as well as to highlight possible extensions and future research. ( 2000 Elsevier Science Ltd. All rights reserved.
q
This research was partly supported by the Greek Ministry of Development * General Secretariat of Research and Technology. * Corresponding author. Fax: 30-821-64824. E-mail address: [email protected] (E. Grigoroudis) 0305-0548/00/$ - see front matter ( 2000 Elsevier Science Ltd. All rights reserved. PII: S 0 3 0 5 - 0 5 4 8 ( 9 9 ) 0 0 1 1 9 - 7
800
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Keywords: Customer satisfaction software; Customer service technology; Preference disaggregation modelling; Ordinal regression analysis
1. Introduction Measuring customer satisfaction is a growing concern to business organisations throughout the world. The concept that you cannot manage what you cannot measure is one of the main principles of management science, since measurement is one of the "ve key functions of managers [1]. Indeed, the idea of measurement is fundamental to the scienti"c method, and it is arguably the bedrock of human knowledge.1 The most important bene"ts of a customer satisfaction program can be summed up in the following points [2]: f Measurement provides a sense of achievement and accomplishment. f Measurement provides a baseline standard of performance and a possible standard of excellence. f Measurement o!ers an immediate meaningful and objective feedback. f Measurement indicates what should be improved and the ways through which this improvement can be achieved. f Measurement motivates people to perform and achieve higher levels of productivity. Extensive research has de"ned several customer satisfaction measurement approaches. The most important techniques among others are: f con"rmation/discon"rmation of expectations method [3}7], f statistical tools and categorical data analysis like loglinear models, logit and probit analysis [8}10], f data analysis techniques like conjoint analysis [11], and f graphical display tools like di!erence histograms and probability plots [12]. TELOS software measures and analyses customer satisfaction using a new collective preference disaggregation model proposed by Siskos, Grigoroudis, Zopounidis and Saurais [13]. This model is based on the principles of multicriteria analysis using ordinal regression techniques (see Section 2), and its main advantages are: 1. The model respects the qualitative form of customers' satisfaction data. 2. Input data can be easily collected using a very simple and short questionnaire.
1 The 19th century physicist Lord Kelvin is reputed to have said that if you cannot measure something, you do not understand it.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
801
3. The results of the model are not only focused on descriptive analysis of customer satisfaction data but they are also able to assess an integrated benchmarking system. 4. The model does not require strong assumptions regarding customer satisfaction or consumer behaviour generally. Customer Service Software packages are based on di!erent approaches and suit di!erent ways to serving the customer. A great majority of these systems are referred to `helpdeska or `helplinea technologies. Call centre technologies and complaints handling systems can be perceived as special cases in this particular category. Advanced call centre technologies include Computer Telephones Integration, and Interactive Voice Response [14]: f Interactive voice response (IVR) delivers an audio response to keyed-in or spoken request. f Computer telephones integration (CTI) can link the answering agent to customer "les stored in database, forward information from one agent's screen to another, connect the caller to an automated response, connect the caller to a personalised message in audio format or send fax or e-mail. Helpline and helpdesk options have grown up enormously: there are over 150 options to choose from [15]. Web-integrated technologies o!ering #exibility and interactivity can also be used for customer service management. The most important and speedy developed feature of these technologies are the Web-based customer satisfaction surveys. Another Customer Service Software category refers to Data Mining techniques and Customer Data Warehouses [16]. These techniques support the analysis of large amount of data in the belief that there are patterns of behaviour that can be detected and highlighted. This involves the use of machine learning, statistics or database technology as components of large-scale data analysis to extract new knowledge. In several cases Data Mining is combined with visualisation techniques in order to comprehend data obtained by simulation or measurement procedures [17]. Finally, customer satisfaction-oriented software packages include questionnaire design and survey analysis tools. These systems are mainly focused on customer satisfaction data collection providing sophisticated data manipulation techniques but simple data analysis tools. The aim of this paper is to present TELOS, a customer satisfaction evaluation software, illustrating its basic features and capabilities. The software is based on the preference disaggregation model MUSA (MUlticriteria Satisfaction Analysis) and is able to provide complete and 1 e!ective results to the user, through1 the evaluation of concrete and understandable indices of customer satisfaction. This paper is organised into 4 sections. Section 2 presents brie#y the basic principles of the multicriteria preference disaggregation approach, as well as the implemented methodological frame. An overview of TELOS is provided in Section 3, through the presentation of a simple illustrative example. Section 4 summarises some concluding remarks, as well as several extensions of TELOS's features and capabilities. Finally, detailed presentations of particular MUSA model's procedures are given in Appendix A.
802
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
2. Preference disaggregation model 2.1. Philosophy TELOS software implements MUSA, a multicriteria preference disaggregation model [13] which main objective is the aggregation of individual judgements into a collective value function assuming that client's global satisfaction depends on a set of criteria or variables representing product's or service's characteristic dimensions. This assumption of a value or treelike structure is presented in Fig. 1 and it is also mentioned as `value treea or `value hierarchya [18}20]. It should be noted that the number of layers may not be uniform across a value hierarchy. Desirable properties and alternative approaches to structuring value hierarchies are presented in Section 2.3. The collective value function represents customer's preference structure, indicating the consequences of the aforementioned set of criteria, and it follows the assumptions of independence conditions [18,19]. According to the model, each customer is asked to express his/her judgements, namely his/her global satisfaction and his/her satisfaction with regard to the set of discrete criteria. The multicriteria model evaluates a set of marginal satisfaction functions in such a way that the global satisfaction criterion becomes as consistent as possible with customer's judgements. 2.2. Model presentation and notation The MUSA preference disaggregation model assesses global and partial satisfaction functions >H and XH, respectively, given customers' judgements > and X . It should be noted that the model i i follows the principles of ordinal regression analysis under constraints using linear programming
Fig. 1. Hierarchical structure of a customer satisfaction problem.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
803
Fig. 2. Satisfaction function and error variables for the jth customer.
techniques [21}23]. The ordinal regression analysis equation has as follows (Fig. 2): n >I H" + b XH!p`#p~, i i i/1 n + b "1, (1) i i/1 where >I H is the estimation of the global satisfaction function, >H, XH are the partial satisfaction i functions, p` and p~ are the overestimation and the underestimation error, respectively, and b is i the weight of the ith criterion. It should be noted that >H and XH are discrete monotonic functions normalised between 0 and i 100. As mentioned in the previous section, these value functions represent customer's global and partial preference structure, where >H is the value function of > and XH is the value function of X i i (for a detailed presentation of the variables used in the MUSA model see Appendix A). In order to reduce the number of the mathematical constraints the following transformation equations are used: z "yHm`-!yHm for m"1, 2,2, a!1, m w "b xHk`-!b xHk for k"1, 2,2, a !1 and i"1, 2,2, n, ik i i i i i
(2)
where yHm is the value of ym (the mth global satisfaction level), and xHk is the value of xk (the kth i i satisfaction level of the ith criterion), respectively (Fig. 3). According to the aforementioned de"nitions and assumptions, the basic estimation model can be written in a linear program formulation, as follows: M [min]F" + p`#p~ j j j/1
804
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 3. The transformation variables z and w in global and partial value functions. m ik
under the constraints n xji ~1 yj~1 + + w ! + z !p`#p~"0 for j"1, 2,2, M, ik m j j i/1 k/1 m/1 a~1 + z "100, (3) m m/1 n ai ~1 + + w "100, ik i/1 k/1 z *0, w *0 ∀m, i and k, m ik p`*0, p~*0 for j"1, 2,2, M, j j where M is the number of customers, n is the number of criteria, and yj, xj are the global and i partial satisfaction judgements of the jth customer, respectively. The preference disaggregation methodology also consists of a post optimality analysis stage in order to address the problem of model stability. Exploring the polyhedron of near optimal solutions, which is generated by the constraints of the above linear program, the "nal solution is obtained [13] (see Appendix A for a detailed presentation). Finally, the assessment of a performance norm could be very useful in customer satisfaction analysis. The average global and partial satisfaction indices are used for this purpose and can be assessed according to the following equations: a S" + pmyHm, m/1 ai s " + pkxHk, (4) i i i k/1 where S and s are the average global and partial satisfaction indices, and pm and pk are the i i frequencies of customers belonging to the ym and xk satisfaction level, respectively. i
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
805
2.3. Measurement procedure Implementation of the MUSA model should follow in general the customer satisfaction measurement procedure presented in Fig. 4. The most important phase is the assessment of the set of satisfaction criteria and the de"nition of the value hierarchy, which is based on the preliminary preference analysis phase. These satisfaction dimensions should assure a consistent family of criteria, with the following properties: (1) monotonicity, (2) exhaustiveness, and (3) non-redundancy [24,25]. Keeney and Rai!a proposed also that the set of criteria and the formulated value hierarchy should be operational, decomposable, and minimal [18]. Company knowledge is the "rst source of information about satisfaction criteria, but customer satisfaction should extend beyond the company and into the arena of customer, particularly when requirements and expectations are to be de"ned. Satisfaction research must be viewed from
Fig. 4. Customer satisfaction measurement procedure.
806
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817 Table 1 Universal satisfaction criteria [26] Criteria related to the product f f f f f f f
Value}price relationship Product quality Product bene"ts Product features Product design Product reliability and consistency Range of products or services
Criteria related to service f f f f
Guarantee or warranty Delivery Complaint handling Resolution of problems
Criteria related to purchase f f f f f
Courtesy Communication Ease or convenience of acquisition Company reputation Company competence
the customer's perspective, since there is no substitute for communicating directly with the customer. The set of typical satisfaction criteria listed in Table 1 is universally recognised and applies to many di!erent products and services [26]. These criteria are candidates for inclusion in almost all customer satisfaction surveys. However, they need to be further de"ned, clari"ed and interpreted for each application. Other major examples of de"ning a concrete set of satisfaction performance attributes include [1,26]: (1) the American Customer Satisfaction Index, (2) the Malcolm Baldrige National Quality Award, and (3) the Customer Satisfaction Index of the American Automobile Industry. There are two main approaches to developing a value hierarchy, which are based on whether or not sources of customer satisfaction or dissatisfaction are available. If this information is available, then a `bottom-upa approach may be appropriate. With this approach, customers with di!erent levels of satisfaction/dissatisfaction are examined to determine the ways in which they di!er. In situations where this information is not available, a `top-downa approach starting with customer's global satisfaction and successively subdividing objectives is more appropriate [20].
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
807
3. Using TELOS 3.1. Overview TELOS software implements the collective preference disaggregation methodology, brie#y presented in the previous section, assuming that the hierarchical structure of the customers' satisfaction problem can be de"ned as shown in Fig. 1. It is obvious that if we have a situation with n criteria, then n#1 satisfaction problems should be solved (1 globally and n for each one of the main criteria). It is very important to mention that the type of information that is handled by TELOS can be either quantitative (price, time, etc.) or qualitative (company's image, personnel's behaviour, etc.). Generally, in order to collect input data for the customer satisfaction problem a prede"ned qualitative satisfaction scale for the set of criteria/subcriteria should be used (see examples in Fig. 5). There is no restriction in the number and speci"cation of satisfaction levels in TELOS, which may be di!erent from one criterion/subcriterion to another. TELOS may be characterised as a consumer-based software because it requires input data collected by a survey through a certain type of questionnaire. A simple example questionnaire is shown in Fig. 5, in order to illustrate the format and design. The satisfaction criteria structure in this questionnaire follows the value hierarchy presented in Table 1. 3.2. Input}output and editing data Input and output data "les used by TELOS have a very simple form because they are basically text "les (ASCII "les). This means that they are fully compatible with almost all application
Fig. 5. A simple questionnaire example.
808
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
programs, and may be accessed through f spreadsheets, f database management systems (DBMS), and f text editors and word processing packages. Through this way, TELOS data may be read from an external text "le or they may be entered directly to the program. The result data "le is also saved in a text format so that the user may use it to perform any kind of complementary analysis with other software packages (Fig. 6).
Fig. 6. Input and output data with TELOS.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
809
The information required to create TELOS data "les refers basically to the de"nition of the variables of the MUSA model (see Appendix A) and it consists of: f f f f f f
title of the problem, number of customers, number of criteria, number of subcriteria per criterion, global, criteria, and subcriteria satisfaction scaling (number and titles of satisfaction levels), and main data table (customers' judgements).
Fig. 7 presents an example of a TELOS data "le, according to the questionnaire of Fig. 5. It should be noted that the main data table consists of ordinal data, and for this reason the appeared numbers represent coding for the de"ned satisfaction levels. 3.3. Results presentation TELOS provides basic descriptive analysis based on the calculated frequencies. Available results consist of global criteria and subcriteria satisfaction frequencies, and they give a general view of the customer satisfaction data. In the presented example (see Fig. 8), customers seem to be quite satis"ed from the business organisation (50% of them answered that globally they are very satis"ed), while they are very satis"ed according to the purchase process (60% have answered that they are very satis"ed form the speci"c dimension). TELOS also includes global and partial explanatory analysis in order to perform in depth customer satisfaction analysis. Global explanatory analysis consists of: Global satisfaction index: it shows, in a range of 0}100%, the level of global satisfaction of the customers (see Eq. (4)); it may be considered as the basic average performance indicator for the business organisation. Added value curve: this curve shows the real value (0}100) that customers give for each level of the global ordinal satisfaction scale (it corresponds to the global satisfaction function >H). `Fragilea customers: the number of customers received satisfaction value less than a particular level can be calculated, using the global added value; this curve represents the distribution function of the added value curve. In this way, if a particular level of the added value curve is believed to be critical, the percentage of `fragilea customers can be calculated. Partial explanatory analysis focuses on criteria/subcriteria analysis and, similarly, consists of: Criteria/subcriteria satisfaction indices: they show, in a range of 0}100%, the level of partial satisfaction of the customers according to the speci"c criterion/subcriterion, similarly to the global satisfaction index. Weights of criteria/subcriteria: they show the relative importance within a set of criteria or subcriteria. As Figs. 9 and 10 present, customers seem to have a moderate average global satisfaction index (72.7%). This situation is caused by the criterion of `producta, which is the most important satisfaction dimension (52.9%), while it has the lowest average satisfaction index (57%). The other criteria play a less important role to customers' satisfaction (18}29%) and they have higher satisfaction indices, but still not in a desirable level (77}78%).
810
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 7. Input and edit TELOS data "les.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
811
Fig. 8. Descriptive analysis.
Also, it should be noted that the shape of the added value curve shows how demanding, customers are. In this way, a `standarda customers' type can be seen in Fig. 9. Finally, combining weights and satisfaction indices results, TELOS can provide a series of `Performance/Importancea diagrams (see Figs. 11 and 12). These diagrams are also mentioned as action, decision, and strategic or perceptual maps. Each of these maps is divided into quadrants according to performance (high/low), and importance (high/low), that may be used to classify actions [27]: 1. Status quo (low performance and low importance): Generally, no action is required. 2. Leverage opportunity (high performance/high importance): These areas can be used as advantage against competition.
812
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 9. Global explanatory analysis.
3. Transfer resources (high performance/low importance): Company's resources may be better used elsewhere. 4. Action opportunity (low performance/high importance): These are the criteria/subcriteria that need attention. Fig. 11 shows that improvement e!orts should be focused on the criterion of `producta. Although it is not a critical satisfaction dimension, it is located very close to the `action opportunitya quadrant and it can cause a possible emergency situation. Focusing each time on a particular criterion, similar analysis may be performed and useful conclusions may be extracted. For example, guarantee can be considered as a competitive advantage of the service criterion for the business organisation (Fig. 12).
4. Concluding remarks TELOS may be characterised as a consumer-based tool for measuring and analysing customer satisfaction. The software package is based on a collective preference disaggregation methodology, as described in Section 2. Thus, the main advantage is that TELOS fully respects the qualitative form of customers' judgements and preferences. Other important features of the proposed software include simplicity, friendliness, and e!ectiveness. As described in the previous section, a customer satisfaction problem may be easily constructed, solved, and analysed using TELOS. Furthermore, obtained results are su$cient to give a clear understanding, and analyse in depth customer satisfaction.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
813
Fig. 10. Partial explanatory analysis.
Several extensions of TELOS may be proposed in order to develop an integrated Customer Satisfaction Decision Support System including: 1. Incorporation of other statistical methods (as those mentioned in Section 1) in order to develop an integrated model base subsystem. The system could provide an alternatively and/or complementary implementation of these methods. For example, MUSA method requires completely and correctly answered questionnaires as input data. In case of missing data, data mining techniques could be used in order to "ll in the empty cells in the data table. 2. Addition of an expert system in order to fully explain provided results and recommend the best decision to be taken. Additionally, the expert system may guide users in the value hierarchy development process.
814
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 11. Global `performance/importancea diagram.
Fig. 12. Criteria `performance/importancea diagrams.
3. Development of a database management system which could assist in the establishment of a permanent customer satisfaction barometer. For example, a history database could record the evolution of customer satisfaction for a particular time period. In this way, the e!ectiveness of business organisation's strategies could be evaluated through customer satisfaction measurement. 4. Addition of network support in order to perform comparative analysis for a number of di!erent departments/stores within a company. In this way an interior benchmarking system may be
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
815
established. This system can relate customer satisfaction and company's performance and it may motivate departments and/or employees to perform and achieve higher levels of productivity. Finally, it should be emphasised that TELOS is more than a decision aid software because it serves for the development of a truly customer-focused management and culture.
Appendix A. Special topics on the musa model A.1. Variables dexnition The MUSA model uses the following variables: > a ym n X i a i xk i >H yHm XH i xHk i
client's global satisfaction number of global satisfaction levels the mth global satisfaction level (m"1, 2,2, a) number of criteria client's satisfaction according to the ith criterion (i"1, 2,2, n) number of satisfaction levels for the ith criterion the kth satisfaction level of the ith criterion (k"1, 2,2, a ) i value function of > value of the ym satisfaction level value function of X i value of the xk satisfaction level i
A.2. Satisfaction variables equations The main output variables of the MUSA model are global and partial satisfaction functions, and criteria weights. In order to calculate these variables in relation to the variables of LP(3), the following equations are used: m~1 yHm" + z for m"2, 3,2, a, t i/1 +ai ~1w b " i/1 it for i"1, 2,2, n, i 100 +k~1w xHk"100 i/1 it for i"1, 2,2, n and k"2, 3,2, a. i +ai ~1w i/1 it It should be noted that the normalisation constraints of >H and XH have the following form: i yH1"0, yHa"100, xH1"0, xHai "100 for i"1, 2,2, n. i i
(5)
(6)
816
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
A.3. Post-optimality analysis The problem of model stability may be viewed as a post-optimality analysis problem in linear programming. Therefore, in order to face the problem of multiple optimal solutions or the existence of near optimal solutions, n linear programs (equal to the number of criteria) are formulated and solved. Each linear program maximises the weight of a criterion and has the following form: ai ~1 [max]F@" + w for i"1, 2,2, n, ik k/1 under the constraints F)FH#e
(7)
all the constraints of LP(3), where e is a small percentage of FH. The average of the solutions given by the n LPs(8) may be taken as the "nal solution. In case of non-stability this average solution is less representative.
References [1] Massnick F. The customer is CEO: how to measure what your customers want and make sure they get it. New York: Amacon, 1997. [2] Gerson RF. Measuring customer satisfaction. Menlo Park, CA: Crisp Publication, Inc., 1993. [3] Anderson EW, Fornell C, Lehmann DR. Perceived quality, customer satisfaction, market share and pro"tability. Working Paper 2.4, University of Michigan, 1992. [4] Anderson RE. Consumer dissatisfaction: the e!ects of discon"rmed expectancy on perceived product performance. Journal of Marketing Research 1973;10:38}44. [5] Oliver RL. A cognitive model for the antecedents and consequences of satisfaction decisions. Journal of Marketing Research 1980;17:460}9. [6] Oliver RL, Beardon WO. Discon"rmation processes and consumer evaluations in product usage. Journal of Business Research 1985;13:235}46. [7] Yi Y. A critical review of customer satisfaction. In: Zeithmal VA, editor. Review of marketing. Chicago: American Marketing Association, 1989. [8] Agresti A. Analysis of ordinal categorical data. New York: Wiley, 1984. [9] Agresti A. Categorical data analysis. New York: Wiley, 1990. [10] Hanushek EA, Jackson JE. Statistical methods for social scientists. New York: Academic Press, 1977. [11] Green PE, Wind J. Multi-attribute decisions in marketing: a measurement approach. Hinsdale, IL: The Dryden Press, 1973. [12] Dendy LR, Gnanadesikan R, Kettenring JR, Suzansky JW. An analysis of questionnaire data on work design and job satisfaction: a case study in the use of simple graphical displays. In: Geisser S, Hodges JS, Press SJ, Zellner A, editors. Bayesian and likelihood methods in statistics and econometrics. Amsterdam: Elsevier, 1990. [13] Siskos Y, Grigoroudis E, Zopounidis C, Saurais O. Measuring customer satisfaction using a collective preference disaggregation model. Journal of Global Optimization 1998;12:175}95. [14] Cogan J. Technology and call centres. Customer Service Management 1997;16:48}51. [15] Kendall H. Your 10-point guide to choosing helpdesk technology. Customer Service Management 1997;16:44. [16] Inmon WH, Hackathorn RD. Using the data warehouse. New York: Wiley, 1994.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
817
[17] Walker GR, Rea PA, Whalley S, Hinds M, Kings NJ. Visualisation of telecommunication network data. BT Technology Journal 1993;11(4):54}63. [18] Keeney RL, Rai!a H. Decisions with multiple objectives: preferences and value tradeo!s. New York: Wiley, 1976. [19] Keeney RL. Value-focused thinking: a path to creative decisionmaking. Cambridge, MA: Harvard University Press, 1996. [20] Kirkwood GW. Strategic decision making. Belmont: Duxbury Press, 1997. [21] Jacquet-Lagre`ze E, Siskos J. Assessing a set of additive utility functions for multicriteria decision-making: the UTA method. European Journal of Operational Research 1982;10(2):151}64. [22] Siskos J. Analyse de regression et programmation lineH aire. Revue de Statistique AppliqueH e 1985;23(2):41}55. [23] Siskos J, Yannacopoulos D. UTASTAR: an ordinal regression method for building additive value functions. Investigaiao Operacional 1985;5(1):39}53. [24] Roy B. MeH thodologie multicrite`re d'aide a` la deH cision. Paris: Economica, 1985. [25] Roy B, Bouyssou D. Aide multicrite`re a` la deH cision: meH thodes et cas. Paris: Economica, 1993. [26] Dutka A. AMA handbook of customer satisfaction: a complete guide to research, planning and implementation. IL: NTC Business Books, 1995. [27] Customers Satisfaction Council. Customer satisfaction assessment guide. Schaumburg, IL, Motorola University Press, 1995.
Evangelos Grigoroudis received his B.A. and M.Sc. in Production Engineering and Management from the Department of Production Engineering and Management of the Technical University of Crete, where he is currently a Ph.D. candidate. His research interests focus on the areas of multiple criteria decision making and customer satisfaction measurement. Yannis Siskos is Professor of operations research and Director of the Decision Support Systems Laboratory at the Department of Production Engineering and Management of the Technical University of Crete. He received his Doctorat D'Etat in management science from the University of Paris-IX Dauphine, a DEA and a Doctorat 3e Cycle in computer science and operations research from the University Pierre et Marie Curie. His research interests are in the area of multiple criteria decision making and the design and development of decision support systems for large-scale managerial tasks. He is the author of over 60 articles in refereed journals including the European Journal of Operational Research, Decision Support Systems, the Journal of the Operational Research Society, Mathematical and Computer Modelling, the Journal of Global Optimization. RAIRO Recherchc OpeH rationnelle. Olivier Saurais received his degree from ESSEC (France). He currently holds the position of director of Management of Strategic Resources S.A., Lausanne (Switzerland). He has developed and conducted numerous customer satisfaction surveys. His interests fall into the area of customer satisfaction, marketing-management and operations research.
TELOS: a customer satisfaction evaluation softwareq Evangelos Grigoroudis!,*, Yannis Siskos!, Olivier Saurais" !Technical University of Crete, Decision Support Systems Laboratory, University Campus, 73100 Chania, Greece "Management of Strategic Resources S.A., Av. Lavaux 101, 1009 Pully, Switzerland
Abstract Telos is a consumer-based tool for measuring and analysing customer satisfaction. The software uses survey data on customer satisfaction judgements while the analysis of collected information is based on a preference disaggregation model. The implemented methodology follows the principles of multicriteria analysis using mainly ordinal regression techniques. The main advantage of the presented software is that TELOS fully considers the qualitative form of customers' judgements and preferences. A simple numerical example is used for illustrating TELOS's basic features like simplicity, friendliness, and e!ectiveness. Finally, an overview of customer service management technologies is brie#y presented and several extensions of TELOS's features and capabilities are discussed. Scope and purpose Customer satisfaction research is one of the fastest growing segments of the marketing research "eld. Its main objective is to measure strengths and weaknesses of a business organisation, while getting closer to the customer. Several methodological approaches, based on di!erent scienti"c areas, are used in order to de"ne, measure and analyse customer satisfaction. Many of these models do not consider the qualitative form of customers' judgements, although this information is the basic customer satisfaction input data. Usually, necessary input data cannot be easily collected, and, at the same time, the results are not fully reliable. Furthermore, in several cases measurements are not su$cient enough to analyse in detail customer satisfaction because models' results are mainly focused on a simple descriptive analysis. TELOS, a customer satisfaction measuring and analysing software which is discussed through this paper, overcomes these disadvantages, by implementing a multiple criteria preference disaggregation methodology, which provides to the user a complete and e!ective series of results. The main objective of the paper is to present TELOS's basic features and capabilities, as well as to highlight possible extensions and future research. ( 2000 Elsevier Science Ltd. All rights reserved.
q
This research was partly supported by the Greek Ministry of Development * General Secretariat of Research and Technology. * Corresponding author. Fax: 30-821-64824. E-mail address: [email protected] (E. Grigoroudis) 0305-0548/00/$ - see front matter ( 2000 Elsevier Science Ltd. All rights reserved. PII: S 0 3 0 5 - 0 5 4 8 ( 9 9 ) 0 0 1 1 9 - 7
800
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Keywords: Customer satisfaction software; Customer service technology; Preference disaggregation modelling; Ordinal regression analysis
1. Introduction Measuring customer satisfaction is a growing concern to business organisations throughout the world. The concept that you cannot manage what you cannot measure is one of the main principles of management science, since measurement is one of the "ve key functions of managers [1]. Indeed, the idea of measurement is fundamental to the scienti"c method, and it is arguably the bedrock of human knowledge.1 The most important bene"ts of a customer satisfaction program can be summed up in the following points [2]: f Measurement provides a sense of achievement and accomplishment. f Measurement provides a baseline standard of performance and a possible standard of excellence. f Measurement o!ers an immediate meaningful and objective feedback. f Measurement indicates what should be improved and the ways through which this improvement can be achieved. f Measurement motivates people to perform and achieve higher levels of productivity. Extensive research has de"ned several customer satisfaction measurement approaches. The most important techniques among others are: f con"rmation/discon"rmation of expectations method [3}7], f statistical tools and categorical data analysis like loglinear models, logit and probit analysis [8}10], f data analysis techniques like conjoint analysis [11], and f graphical display tools like di!erence histograms and probability plots [12]. TELOS software measures and analyses customer satisfaction using a new collective preference disaggregation model proposed by Siskos, Grigoroudis, Zopounidis and Saurais [13]. This model is based on the principles of multicriteria analysis using ordinal regression techniques (see Section 2), and its main advantages are: 1. The model respects the qualitative form of customers' satisfaction data. 2. Input data can be easily collected using a very simple and short questionnaire.
1 The 19th century physicist Lord Kelvin is reputed to have said that if you cannot measure something, you do not understand it.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
801
3. The results of the model are not only focused on descriptive analysis of customer satisfaction data but they are also able to assess an integrated benchmarking system. 4. The model does not require strong assumptions regarding customer satisfaction or consumer behaviour generally. Customer Service Software packages are based on di!erent approaches and suit di!erent ways to serving the customer. A great majority of these systems are referred to `helpdeska or `helplinea technologies. Call centre technologies and complaints handling systems can be perceived as special cases in this particular category. Advanced call centre technologies include Computer Telephones Integration, and Interactive Voice Response [14]: f Interactive voice response (IVR) delivers an audio response to keyed-in or spoken request. f Computer telephones integration (CTI) can link the answering agent to customer "les stored in database, forward information from one agent's screen to another, connect the caller to an automated response, connect the caller to a personalised message in audio format or send fax or e-mail. Helpline and helpdesk options have grown up enormously: there are over 150 options to choose from [15]. Web-integrated technologies o!ering #exibility and interactivity can also be used for customer service management. The most important and speedy developed feature of these technologies are the Web-based customer satisfaction surveys. Another Customer Service Software category refers to Data Mining techniques and Customer Data Warehouses [16]. These techniques support the analysis of large amount of data in the belief that there are patterns of behaviour that can be detected and highlighted. This involves the use of machine learning, statistics or database technology as components of large-scale data analysis to extract new knowledge. In several cases Data Mining is combined with visualisation techniques in order to comprehend data obtained by simulation or measurement procedures [17]. Finally, customer satisfaction-oriented software packages include questionnaire design and survey analysis tools. These systems are mainly focused on customer satisfaction data collection providing sophisticated data manipulation techniques but simple data analysis tools. The aim of this paper is to present TELOS, a customer satisfaction evaluation software, illustrating its basic features and capabilities. The software is based on the preference disaggregation model MUSA (MUlticriteria Satisfaction Analysis) and is able to provide complete and 1 e!ective results to the user, through1 the evaluation of concrete and understandable indices of customer satisfaction. This paper is organised into 4 sections. Section 2 presents brie#y the basic principles of the multicriteria preference disaggregation approach, as well as the implemented methodological frame. An overview of TELOS is provided in Section 3, through the presentation of a simple illustrative example. Section 4 summarises some concluding remarks, as well as several extensions of TELOS's features and capabilities. Finally, detailed presentations of particular MUSA model's procedures are given in Appendix A.
802
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
2. Preference disaggregation model 2.1. Philosophy TELOS software implements MUSA, a multicriteria preference disaggregation model [13] which main objective is the aggregation of individual judgements into a collective value function assuming that client's global satisfaction depends on a set of criteria or variables representing product's or service's characteristic dimensions. This assumption of a value or treelike structure is presented in Fig. 1 and it is also mentioned as `value treea or `value hierarchya [18}20]. It should be noted that the number of layers may not be uniform across a value hierarchy. Desirable properties and alternative approaches to structuring value hierarchies are presented in Section 2.3. The collective value function represents customer's preference structure, indicating the consequences of the aforementioned set of criteria, and it follows the assumptions of independence conditions [18,19]. According to the model, each customer is asked to express his/her judgements, namely his/her global satisfaction and his/her satisfaction with regard to the set of discrete criteria. The multicriteria model evaluates a set of marginal satisfaction functions in such a way that the global satisfaction criterion becomes as consistent as possible with customer's judgements. 2.2. Model presentation and notation The MUSA preference disaggregation model assesses global and partial satisfaction functions >H and XH, respectively, given customers' judgements > and X . It should be noted that the model i i follows the principles of ordinal regression analysis under constraints using linear programming
Fig. 1. Hierarchical structure of a customer satisfaction problem.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
803
Fig. 2. Satisfaction function and error variables for the jth customer.
techniques [21}23]. The ordinal regression analysis equation has as follows (Fig. 2): n >I H" + b XH!p`#p~, i i i/1 n + b "1, (1) i i/1 where >I H is the estimation of the global satisfaction function, >H, XH are the partial satisfaction i functions, p` and p~ are the overestimation and the underestimation error, respectively, and b is i the weight of the ith criterion. It should be noted that >H and XH are discrete monotonic functions normalised between 0 and i 100. As mentioned in the previous section, these value functions represent customer's global and partial preference structure, where >H is the value function of > and XH is the value function of X i i (for a detailed presentation of the variables used in the MUSA model see Appendix A). In order to reduce the number of the mathematical constraints the following transformation equations are used: z "yHm`-!yHm for m"1, 2,2, a!1, m w "b xHk`-!b xHk for k"1, 2,2, a !1 and i"1, 2,2, n, ik i i i i i
(2)
where yHm is the value of ym (the mth global satisfaction level), and xHk is the value of xk (the kth i i satisfaction level of the ith criterion), respectively (Fig. 3). According to the aforementioned de"nitions and assumptions, the basic estimation model can be written in a linear program formulation, as follows: M [min]F" + p`#p~ j j j/1
804
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 3. The transformation variables z and w in global and partial value functions. m ik
under the constraints n xji ~1 yj~1 + + w ! + z !p`#p~"0 for j"1, 2,2, M, ik m j j i/1 k/1 m/1 a~1 + z "100, (3) m m/1 n ai ~1 + + w "100, ik i/1 k/1 z *0, w *0 ∀m, i and k, m ik p`*0, p~*0 for j"1, 2,2, M, j j where M is the number of customers, n is the number of criteria, and yj, xj are the global and i partial satisfaction judgements of the jth customer, respectively. The preference disaggregation methodology also consists of a post optimality analysis stage in order to address the problem of model stability. Exploring the polyhedron of near optimal solutions, which is generated by the constraints of the above linear program, the "nal solution is obtained [13] (see Appendix A for a detailed presentation). Finally, the assessment of a performance norm could be very useful in customer satisfaction analysis. The average global and partial satisfaction indices are used for this purpose and can be assessed according to the following equations: a S" + pmyHm, m/1 ai s " + pkxHk, (4) i i i k/1 where S and s are the average global and partial satisfaction indices, and pm and pk are the i i frequencies of customers belonging to the ym and xk satisfaction level, respectively. i
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
805
2.3. Measurement procedure Implementation of the MUSA model should follow in general the customer satisfaction measurement procedure presented in Fig. 4. The most important phase is the assessment of the set of satisfaction criteria and the de"nition of the value hierarchy, which is based on the preliminary preference analysis phase. These satisfaction dimensions should assure a consistent family of criteria, with the following properties: (1) monotonicity, (2) exhaustiveness, and (3) non-redundancy [24,25]. Keeney and Rai!a proposed also that the set of criteria and the formulated value hierarchy should be operational, decomposable, and minimal [18]. Company knowledge is the "rst source of information about satisfaction criteria, but customer satisfaction should extend beyond the company and into the arena of customer, particularly when requirements and expectations are to be de"ned. Satisfaction research must be viewed from
Fig. 4. Customer satisfaction measurement procedure.
806
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817 Table 1 Universal satisfaction criteria [26] Criteria related to the product f f f f f f f
Value}price relationship Product quality Product bene"ts Product features Product design Product reliability and consistency Range of products or services
Criteria related to service f f f f
Guarantee or warranty Delivery Complaint handling Resolution of problems
Criteria related to purchase f f f f f
Courtesy Communication Ease or convenience of acquisition Company reputation Company competence
the customer's perspective, since there is no substitute for communicating directly with the customer. The set of typical satisfaction criteria listed in Table 1 is universally recognised and applies to many di!erent products and services [26]. These criteria are candidates for inclusion in almost all customer satisfaction surveys. However, they need to be further de"ned, clari"ed and interpreted for each application. Other major examples of de"ning a concrete set of satisfaction performance attributes include [1,26]: (1) the American Customer Satisfaction Index, (2) the Malcolm Baldrige National Quality Award, and (3) the Customer Satisfaction Index of the American Automobile Industry. There are two main approaches to developing a value hierarchy, which are based on whether or not sources of customer satisfaction or dissatisfaction are available. If this information is available, then a `bottom-upa approach may be appropriate. With this approach, customers with di!erent levels of satisfaction/dissatisfaction are examined to determine the ways in which they di!er. In situations where this information is not available, a `top-downa approach starting with customer's global satisfaction and successively subdividing objectives is more appropriate [20].
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
807
3. Using TELOS 3.1. Overview TELOS software implements the collective preference disaggregation methodology, brie#y presented in the previous section, assuming that the hierarchical structure of the customers' satisfaction problem can be de"ned as shown in Fig. 1. It is obvious that if we have a situation with n criteria, then n#1 satisfaction problems should be solved (1 globally and n for each one of the main criteria). It is very important to mention that the type of information that is handled by TELOS can be either quantitative (price, time, etc.) or qualitative (company's image, personnel's behaviour, etc.). Generally, in order to collect input data for the customer satisfaction problem a prede"ned qualitative satisfaction scale for the set of criteria/subcriteria should be used (see examples in Fig. 5). There is no restriction in the number and speci"cation of satisfaction levels in TELOS, which may be di!erent from one criterion/subcriterion to another. TELOS may be characterised as a consumer-based software because it requires input data collected by a survey through a certain type of questionnaire. A simple example questionnaire is shown in Fig. 5, in order to illustrate the format and design. The satisfaction criteria structure in this questionnaire follows the value hierarchy presented in Table 1. 3.2. Input}output and editing data Input and output data "les used by TELOS have a very simple form because they are basically text "les (ASCII "les). This means that they are fully compatible with almost all application
Fig. 5. A simple questionnaire example.
808
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
programs, and may be accessed through f spreadsheets, f database management systems (DBMS), and f text editors and word processing packages. Through this way, TELOS data may be read from an external text "le or they may be entered directly to the program. The result data "le is also saved in a text format so that the user may use it to perform any kind of complementary analysis with other software packages (Fig. 6).
Fig. 6. Input and output data with TELOS.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
809
The information required to create TELOS data "les refers basically to the de"nition of the variables of the MUSA model (see Appendix A) and it consists of: f f f f f f
title of the problem, number of customers, number of criteria, number of subcriteria per criterion, global, criteria, and subcriteria satisfaction scaling (number and titles of satisfaction levels), and main data table (customers' judgements).
Fig. 7 presents an example of a TELOS data "le, according to the questionnaire of Fig. 5. It should be noted that the main data table consists of ordinal data, and for this reason the appeared numbers represent coding for the de"ned satisfaction levels. 3.3. Results presentation TELOS provides basic descriptive analysis based on the calculated frequencies. Available results consist of global criteria and subcriteria satisfaction frequencies, and they give a general view of the customer satisfaction data. In the presented example (see Fig. 8), customers seem to be quite satis"ed from the business organisation (50% of them answered that globally they are very satis"ed), while they are very satis"ed according to the purchase process (60% have answered that they are very satis"ed form the speci"c dimension). TELOS also includes global and partial explanatory analysis in order to perform in depth customer satisfaction analysis. Global explanatory analysis consists of: Global satisfaction index: it shows, in a range of 0}100%, the level of global satisfaction of the customers (see Eq. (4)); it may be considered as the basic average performance indicator for the business organisation. Added value curve: this curve shows the real value (0}100) that customers give for each level of the global ordinal satisfaction scale (it corresponds to the global satisfaction function >H). `Fragilea customers: the number of customers received satisfaction value less than a particular level can be calculated, using the global added value; this curve represents the distribution function of the added value curve. In this way, if a particular level of the added value curve is believed to be critical, the percentage of `fragilea customers can be calculated. Partial explanatory analysis focuses on criteria/subcriteria analysis and, similarly, consists of: Criteria/subcriteria satisfaction indices: they show, in a range of 0}100%, the level of partial satisfaction of the customers according to the speci"c criterion/subcriterion, similarly to the global satisfaction index. Weights of criteria/subcriteria: they show the relative importance within a set of criteria or subcriteria. As Figs. 9 and 10 present, customers seem to have a moderate average global satisfaction index (72.7%). This situation is caused by the criterion of `producta, which is the most important satisfaction dimension (52.9%), while it has the lowest average satisfaction index (57%). The other criteria play a less important role to customers' satisfaction (18}29%) and they have higher satisfaction indices, but still not in a desirable level (77}78%).
810
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 7. Input and edit TELOS data "les.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
811
Fig. 8. Descriptive analysis.
Also, it should be noted that the shape of the added value curve shows how demanding, customers are. In this way, a `standarda customers' type can be seen in Fig. 9. Finally, combining weights and satisfaction indices results, TELOS can provide a series of `Performance/Importancea diagrams (see Figs. 11 and 12). These diagrams are also mentioned as action, decision, and strategic or perceptual maps. Each of these maps is divided into quadrants according to performance (high/low), and importance (high/low), that may be used to classify actions [27]: 1. Status quo (low performance and low importance): Generally, no action is required. 2. Leverage opportunity (high performance/high importance): These areas can be used as advantage against competition.
812
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 9. Global explanatory analysis.
3. Transfer resources (high performance/low importance): Company's resources may be better used elsewhere. 4. Action opportunity (low performance/high importance): These are the criteria/subcriteria that need attention. Fig. 11 shows that improvement e!orts should be focused on the criterion of `producta. Although it is not a critical satisfaction dimension, it is located very close to the `action opportunitya quadrant and it can cause a possible emergency situation. Focusing each time on a particular criterion, similar analysis may be performed and useful conclusions may be extracted. For example, guarantee can be considered as a competitive advantage of the service criterion for the business organisation (Fig. 12).
4. Concluding remarks TELOS may be characterised as a consumer-based tool for measuring and analysing customer satisfaction. The software package is based on a collective preference disaggregation methodology, as described in Section 2. Thus, the main advantage is that TELOS fully respects the qualitative form of customers' judgements and preferences. Other important features of the proposed software include simplicity, friendliness, and e!ectiveness. As described in the previous section, a customer satisfaction problem may be easily constructed, solved, and analysed using TELOS. Furthermore, obtained results are su$cient to give a clear understanding, and analyse in depth customer satisfaction.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
813
Fig. 10. Partial explanatory analysis.
Several extensions of TELOS may be proposed in order to develop an integrated Customer Satisfaction Decision Support System including: 1. Incorporation of other statistical methods (as those mentioned in Section 1) in order to develop an integrated model base subsystem. The system could provide an alternatively and/or complementary implementation of these methods. For example, MUSA method requires completely and correctly answered questionnaires as input data. In case of missing data, data mining techniques could be used in order to "ll in the empty cells in the data table. 2. Addition of an expert system in order to fully explain provided results and recommend the best decision to be taken. Additionally, the expert system may guide users in the value hierarchy development process.
814
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
Fig. 11. Global `performance/importancea diagram.
Fig. 12. Criteria `performance/importancea diagrams.
3. Development of a database management system which could assist in the establishment of a permanent customer satisfaction barometer. For example, a history database could record the evolution of customer satisfaction for a particular time period. In this way, the e!ectiveness of business organisation's strategies could be evaluated through customer satisfaction measurement. 4. Addition of network support in order to perform comparative analysis for a number of di!erent departments/stores within a company. In this way an interior benchmarking system may be
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
815
established. This system can relate customer satisfaction and company's performance and it may motivate departments and/or employees to perform and achieve higher levels of productivity. Finally, it should be emphasised that TELOS is more than a decision aid software because it serves for the development of a truly customer-focused management and culture.
Appendix A. Special topics on the musa model A.1. Variables dexnition The MUSA model uses the following variables: > a ym n X i a i xk i >H yHm XH i xHk i
client's global satisfaction number of global satisfaction levels the mth global satisfaction level (m"1, 2,2, a) number of criteria client's satisfaction according to the ith criterion (i"1, 2,2, n) number of satisfaction levels for the ith criterion the kth satisfaction level of the ith criterion (k"1, 2,2, a ) i value function of > value of the ym satisfaction level value function of X i value of the xk satisfaction level i
A.2. Satisfaction variables equations The main output variables of the MUSA model are global and partial satisfaction functions, and criteria weights. In order to calculate these variables in relation to the variables of LP(3), the following equations are used: m~1 yHm" + z for m"2, 3,2, a, t i/1 +ai ~1w b " i/1 it for i"1, 2,2, n, i 100 +k~1w xHk"100 i/1 it for i"1, 2,2, n and k"2, 3,2, a. i +ai ~1w i/1 it It should be noted that the normalisation constraints of >H and XH have the following form: i yH1"0, yHa"100, xH1"0, xHai "100 for i"1, 2,2, n. i i
(5)
(6)
816
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
A.3. Post-optimality analysis The problem of model stability may be viewed as a post-optimality analysis problem in linear programming. Therefore, in order to face the problem of multiple optimal solutions or the existence of near optimal solutions, n linear programs (equal to the number of criteria) are formulated and solved. Each linear program maximises the weight of a criterion and has the following form: ai ~1 [max]F@" + w for i"1, 2,2, n, ik k/1 under the constraints F)FH#e
(7)
all the constraints of LP(3), where e is a small percentage of FH. The average of the solutions given by the n LPs(8) may be taken as the "nal solution. In case of non-stability this average solution is less representative.
References [1] Massnick F. The customer is CEO: how to measure what your customers want and make sure they get it. New York: Amacon, 1997. [2] Gerson RF. Measuring customer satisfaction. Menlo Park, CA: Crisp Publication, Inc., 1993. [3] Anderson EW, Fornell C, Lehmann DR. Perceived quality, customer satisfaction, market share and pro"tability. Working Paper 2.4, University of Michigan, 1992. [4] Anderson RE. Consumer dissatisfaction: the e!ects of discon"rmed expectancy on perceived product performance. Journal of Marketing Research 1973;10:38}44. [5] Oliver RL. A cognitive model for the antecedents and consequences of satisfaction decisions. Journal of Marketing Research 1980;17:460}9. [6] Oliver RL, Beardon WO. Discon"rmation processes and consumer evaluations in product usage. Journal of Business Research 1985;13:235}46. [7] Yi Y. A critical review of customer satisfaction. In: Zeithmal VA, editor. Review of marketing. Chicago: American Marketing Association, 1989. [8] Agresti A. Analysis of ordinal categorical data. New York: Wiley, 1984. [9] Agresti A. Categorical data analysis. New York: Wiley, 1990. [10] Hanushek EA, Jackson JE. Statistical methods for social scientists. New York: Academic Press, 1977. [11] Green PE, Wind J. Multi-attribute decisions in marketing: a measurement approach. Hinsdale, IL: The Dryden Press, 1973. [12] Dendy LR, Gnanadesikan R, Kettenring JR, Suzansky JW. An analysis of questionnaire data on work design and job satisfaction: a case study in the use of simple graphical displays. In: Geisser S, Hodges JS, Press SJ, Zellner A, editors. Bayesian and likelihood methods in statistics and econometrics. Amsterdam: Elsevier, 1990. [13] Siskos Y, Grigoroudis E, Zopounidis C, Saurais O. Measuring customer satisfaction using a collective preference disaggregation model. Journal of Global Optimization 1998;12:175}95. [14] Cogan J. Technology and call centres. Customer Service Management 1997;16:48}51. [15] Kendall H. Your 10-point guide to choosing helpdesk technology. Customer Service Management 1997;16:44. [16] Inmon WH, Hackathorn RD. Using the data warehouse. New York: Wiley, 1994.
E. Grigoroudis et al. / Computers & Operations Research 27 (2000) 799}817
817
[17] Walker GR, Rea PA, Whalley S, Hinds M, Kings NJ. Visualisation of telecommunication network data. BT Technology Journal 1993;11(4):54}63. [18] Keeney RL, Rai!a H. Decisions with multiple objectives: preferences and value tradeo!s. New York: Wiley, 1976. [19] Keeney RL. Value-focused thinking: a path to creative decisionmaking. Cambridge, MA: Harvard University Press, 1996. [20] Kirkwood GW. Strategic decision making. Belmont: Duxbury Press, 1997. [21] Jacquet-Lagre`ze E, Siskos J. Assessing a set of additive utility functions for multicriteria decision-making: the UTA method. European Journal of Operational Research 1982;10(2):151}64. [22] Siskos J. Analyse de regression et programmation lineH aire. Revue de Statistique AppliqueH e 1985;23(2):41}55. [23] Siskos J, Yannacopoulos D. UTASTAR: an ordinal regression method for building additive value functions. Investigaiao Operacional 1985;5(1):39}53. [24] Roy B. MeH thodologie multicrite`re d'aide a` la deH cision. Paris: Economica, 1985. [25] Roy B, Bouyssou D. Aide multicrite`re a` la deH cision: meH thodes et cas. Paris: Economica, 1993. [26] Dutka A. AMA handbook of customer satisfaction: a complete guide to research, planning and implementation. IL: NTC Business Books, 1995. [27] Customers Satisfaction Council. Customer satisfaction assessment guide. Schaumburg, IL, Motorola University Press, 1995.
Evangelos Grigoroudis received his B.A. and M.Sc. in Production Engineering and Management from the Department of Production Engineering and Management of the Technical University of Crete, where he is currently a Ph.D. candidate. His research interests focus on the areas of multiple criteria decision making and customer satisfaction measurement. Yannis Siskos is Professor of operations research and Director of the Decision Support Systems Laboratory at the Department of Production Engineering and Management of the Technical University of Crete. He received his Doctorat D'Etat in management science from the University of Paris-IX Dauphine, a DEA and a Doctorat 3e Cycle in computer science and operations research from the University Pierre et Marie Curie. His research interests are in the area of multiple criteria decision making and the design and development of decision support systems for large-scale managerial tasks. He is the author of over 60 articles in refereed journals including the European Journal of Operational Research, Decision Support Systems, the Journal of the Operational Research Society, Mathematical and Computer Modelling, the Journal of Global Optimization. RAIRO Recherchc OpeH rationnelle. Olivier Saurais received his degree from ESSEC (France). He currently holds the position of director of Management of Strategic Resources S.A., Lausanne (Switzerland). He has developed and conducted numerous customer satisfaction surveys. His interests fall into the area of customer satisfaction, marketing-management and operations research.