Expert Systems with Applications 38 (2011) 7151–7157
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Applying Bayesian Belief Network approach to customer churn analysis: A case study on the telecom industry of Turkey Pınar Kisioglu, Y. Ilker Topcu ⇑ Industrial Engineering Department, Istanbul Technical University, Istanbul, Turkey
a r t i c l e
i n f o
Keywords: Churn analysis Bayesian Belief Networks Telecom
a b s t r a c t In telecommunication industry, for many organizations, it is really important to take place in the market. As competition increases between companies, customer churn becomes a great issue to deal with by the telecommunication providers. For an effective churn management, companies try to retain their existing customers, instead of acquiring new ones. Previous researches focus on predicting the customers with a propensity to churn in telecommunication industry. In this study, a model is constructed by Bayesian Belief Network to identify the behaviors of customers with a propensity to churn. The data used are collected from one of the telecommunication providers in Turkey. First, as only discrete variables are used in Bayesian Belief Networks, CHAID (Chi-squared Automatic Interaction Detector) algorithm is applied to discretize continuous variables. Then, a causal map as a base of Bayesian Belief Network is brought out via the results of correlation analysis, multicollinearity test and experts’ opinions. According to the results of Bayesian Belief Network, average minutes of calls, average billing amount, the frequency of calls to people from different providers and tariff type are the most important variables that explain customer churn. At the end of the study, three different scenarios that examine the characteristics of the churners are analyzed and promotions are suggested to reduce the churn rate. Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction There is a continuous change in telecommunication industry all over the world. In 1996, US government removes the obstacles in local and long distance calling, cable TV, broadcasting, and wireless services. In Europe, markets are formed by the deregulations in England, Sweden, and Finland. Then, 15 European countries ended the restrictions in telecommunication industry. Chile, Malaysia, and Peru discontinued the telephone monopolies. All of these applications open new markets in those countries and put an end to limited communication (http://www.sequent.com). As the new markets are developed, competition between companies increases sharply. Since the competition gets hard and telecommunication becomes a selling product, companies encounter to minimize costs, add value to their services, and guarantee differentiation. Now, the customers can choose their service providers, so companies pay attention to customer care in order to keep their situation in the market (http://www.sequent.com). Under the hard conditions of competition, companies try to focus on the behaviors of customers. According to needs of customers, telecommunication companies decide their service offers, give ⇑ Corresponding author. E-mail addresses:
[email protected] (P. Kisioglu),
[email protected] (Y.I. Topcu). 0957-4174/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2010.12.045
a shape to their communication network and in addition change their organizational structure (http://www.iec.org/online/tutorials/acrobat/bus_int.pdf). If a customer ends up his subscription from the existing provider and subscribe to another one, the customer is called a ‘churner’. Churn is a major problem for companies with many customers, e.g., credit card providers or insurance companies. In telecommunication industry, the sharp increase of competition makes customer churn a great concern for the providers (Richeldi & Perrucci, 2002). For wireless telecommunication industry, the monthly predicted churn rate is about 2.2%. This means that a company loses about 27% of its subscribers every year (Wei & Chiu, 2002). On the other hand, the cost of churn is a huge problem for the companies. In European and US markets, the cost of churn is about $4 billion per year, and throughout the world, it costs $10 billion per year (SAS International, 2001). Furthermore, acquiring new customers is much more expensive than retaining the existing ones. For this reason, paying attention on current subscribers is more efficient than acquiring new customers (Richeldi & Perrucci, 2002). The scope of competition changes into reducing the churn rate of the subscribers and keeping them from other competitors (Kim & Yoon, 2004). With an effective churn management, a company can decide whether its customers have a churn propensity or not (SAS
7152
P. Kisioglu, Y.I. Topcu / Expert Systems with Applications 38 (2011) 7151–7157
International, 2001). The aim of churn management is to minimize the lost due to churn while maximizing the profits by retaining valuable customers (http://www.sciencedirect.com/science?_ob= ArticleURL&_udi=B6V03-4). Previous studies about churn prediction in the telecommunication industry mainly applied data mining techniques such as neural networks, decision trees, and cluster analysis to predict churn rate. However, to our knowledge, there has been no research using Bayesian Belief Network to identify the factors that have effects on customers to churn. Hung, Yen, and Wang (2006) studying customer churn in Taiwan market, showed that using neural network gives better results than decision trees. Another research about Taiwan telecommunication industry is studied by Wei and Chiu (2002). They designed a churn prediction model to estimate churn rate of customers using subscriber contractual information and call pattern changes. Ferreira, Vellasco, Pacheco, and Barbosa (2004) studying the loss of valuable customers in Brazilian mobile telecommunications industry, analyzed the potential savings and profits as a result of the research. A similar article to Ferreira et al.’s research, studied by Mozer, Wolniewicz, Grimes, Johnson, and Kaushansky (2000), identified possible churners and calculated savings. Nath and Behara (2002) studied customer churn using Naive– Bayes algorithm with a database of fifty thousand customers in American mobile telecommunication industry. Karahoca and Kara (2006) studying the segmentation of Turkish customers according to their profitability, compared a new technique with other clustering techniques. Jahanzeb and Jabeen (2007) showed the opinions of customers about churn management strategies by a survey and compared two telecommunication companies in Pakistan. This paper is organized as follows. In Section 2, there is a brief explanation about Bayesian Belief Network. Section 3 exhibits the model constructed to identify the factors that cause customers to churn. Section 4 presents three different scenarios and comment. Finally, in Section 5, the results of the model and further suggestions for future studies are discussed.
2. Bayesian Belief Network Bayesian Belief Network (BBN) is a graphical model that represents the casual relationships between the variables with their conditional probabilities (Heckerman, 1995). Due to its cause and effect diagram, it is used in many real-world problems. It is applied to large engineering project risk management by Lee, Park, and Shin (2009) and is used for performance prediction of box-office success on Korean movies by Chang and Lee (2009). A model is constructed to be applied in Turkish transportation system (Ülengin et al., 2007), and another is used to analyze the complex structure of inflation of Turkey (Sßahin, Ülengin, & Ülengin, 2006). Since BBN is one of the most powerful methods for reasoning under uncertainty, it has different application areas such as diagnosis of schizophrenia (Ouali, Cherif, & Krebs, 2006), innovation performance in R&D collaborations (Kim & Park, 2008), and customer lifecycle slope estimation for customer relationship management (Baesens et al., 2004). For telecommunication industry, most of the studies have proposed customer churn prediction by using data mining techniques as mentioned in previous section. However, these methodologies have some disadvantages. Heuristic-based approach and analytical methods are inconvenient for complicated problems, and also it is hard to collect pure data for statistical methods. Moreover, a mathematical model is required for simulation, correlated variables are unsuitable for decision trees, and a clear confidential data set is needed for neural networks (Lee et al., 2009).
In this study, a Bayesian Belief Network (BBN) is used because of the following advantages. First of all, having missing data in the data set does not make any problem for BBNs. Samples with incomplete data can be fixed by adding or integrating the probabilities over all possible values of the variable. Second, causal relationships are identified by applying a BBN. Thus, BBN makes it easier to understand the problem domain and estimate the results. Another advantage of BBN is that building a model does not consume time and need much effort. The structure of the network is developed, and then, it is easy to add new variables to the model. Finally, BBN is a combination of both a data set and users’ prior knowledge; so, the model, constructed through BBN, is sufficiently dependable (Kim & Park, 2008). 3. Proposed model In this paper, we specify the relationship between variables and use BBN as a prediction model of customer churn in telecommunication sector. Fig. 1 shows the flow chart of our research. First, raw data are collected from the database of one of the Turkish telecommunication providers. The variables are prepared, and a new data set is built for the study. Since continuous variables cannot be used in BBN networks, each continuous variable is discretized using CHAID algorithm. To show the interactions of variables, we check the independence of each variable by correlation analysis and multicollinearity test. A casual map is constructed as a base of BBN network according to the results of the tests and experts’ opinions. Finally, sensitivity analysis is applied, and three different scenarios are developed. In each scenario, the possible savings are calculated with suggested promotions. We use the software Netica (www.norsys.com) to construct BBN network and make the sensitivity analysis. Also, SPSS 13.0 is used for decision trees, correlation analysis, and multicollinearity test. 3.1. Data and variables All data, used in this study, are collected from a telecommunication company in Turkey with their permission. The data set contains data of 2000 subscribers, including 534 churners, from January 2008 to July 2008. Initially, there are 23 different variables in data set. After data preparation, the number of variables is 9. In the initial data set, the variable billing amount has six-month data such as the other two variables, minutes of usage and frequency of usage. The average of six-month data of billing amount is calculated and is inserted to the data set as a new variable instead of six variables in total. The same procedure is also applied for the other variables, minutes of usage and frequency of usage. In addition, a new variable is added to the data set: trend in billing amount. It is obtained from billing amounts of the customers and shows whether their billing amounts are in upward, downward, or constant trend. The detailed explanation of variables is as follows: Place of residence: the place of residence, where the subscribers live, is important for churn analysis. Subscribers in the data set live in rural and urban areas. Subscribers living in rural areas are more loyal than subscribers living in urban areas due to the fact of limited number of telecom providers in rural areas. Age: analysis shows that customers under the age of 35 have a high propensity to churn in Turkey. Tenure: subscribers with less than 1865 days tenure have a high propensity to churn. Tariff type: there are three different types of tariff: one for commercial companies, institutions, etc. and two for individual subscribers. We consider individual subscribers rather than others
7153
P. Kisioglu, Y.I. Topcu / Expert Systems with Applications 38 (2011) 7151–7157
Fig. 1. Steps of the research.
because individual subscribers have a higher churn rate. Individual subscribers can use first or second type of tariff. First type of tariff is suitable for customers with a high number of outgoing calls. Second type of tariff has lower constant fee than first type of tariff. However, unit cost per minute is much more expensive in second type of tariff when compared with first type of tariff. Average billing amount: in the data set, there are monthly billing amount of subscribers. Average billing amount over 6 months is used as a unique variable. Thus, average billing amount is more effective in explaining churn than six separate variables. Trend in billing amount: customers are grouped according to their monthly billing amount such as customers in upward, downward, and constant trends. To do this, the difference between the two consecutive months is calculated and the positive differences are counted. If the number of positive differences is less than three, then the customers are in downward trend. The customers with three positive differences are in constant trend. Otherwise, they are in upward trend. Average minutes of usage: the average number of minutes of outgoing calls made by the customer during the six months. Customers with low minutes of usage have a high propensity to churn. Average frequency of usage: the average number of outgoing calls made by the customer during the six months. Churn: churn is the dependent variable of this study. It refers that a subscriber becomes a churner at the end of the six months. 3.2. Discretization of continuous variables Since discrete variables are used in BBN, we apply decision tree (CHAID algorithm) to discretize continuous variables. Independent variables tenure, average billing amount, average minutes of usage, and average frequency of usage are split into 3 levels and the variable age is split into 2 levels. Fig. 2 shows the result of the splitting rule of age.
CHURN Node 0 Category % 1.0 0.0
1.0 0.0
n
26.7 534 73.3 1466
Total
100 .0 2000
AGE Adj. P-value=0.000, Chi-square=27. 801, df=1
<= 34 Node 1 Category %
> 34
n
Node 2 Category %
n
1.0 0.0
37.4 62.6
143 239
1.0 0.0
24.2 391 75.8 1227
Total
19.1
382
Total
80.9 1618
Fig. 2. Splitting rule of age.
3.3. Correlation analysis and multicollinearity test To develop a casual map, we perform correlation analysis and collinearity test. Table 1 shows the results of the correlation analysis. Many variables are correlated with each other. Tenure, average minutes of usage, and trend in billing amount are heavily correlated with the dependent variable churn. The interactions between variables are validated by multicollinearity test. The results are presented in Table 2. If variance inflation factor is greater than 1.4, multicollinearity exists. Tenure, age, average billing amount, average minutes of usage, and average frequency of usage are correlated with other variables.
7154
P. Kisioglu, Y.I. Topcu / Expert Systems with Applications 38 (2011) 7151–7157
Table 1 Correlation between variables. Place of res. Place of res. Age Tenure Tariff Avg. billing Trend Avg. minutes Avg. frequency Churn
1 0.015 0.019 0.184 0.333 0.054 0.34 0.154 0.085
Age 0.015 1 0.376 0.028 0.045 0.047 0.021 0.017 0.118
Tenure 0.019 0.376 1 0.045 -0.033 0.042 0.011 0.033 0.276
Tariff
Avg. billing
0.184 0.028 0.045 1 0.508 0.021 0.398 0.188 0.062
Trend
0.333 0.045 0.033 0.508 1 0.061 0.765 0.391 0.156
Avg. minutes
0.054 0.047 0.042 0.021 0.061 1 0.128 0.074 0.312
Avg. frequency
0.34 0.021 0.011 -0.398 0.765 0.128 1 0.305 0.304
0.154 0.017 0.033 0.188 0.391 0.074 0.305 1 0.11
Churn 0.085 0.118 0.276 0.062 0.156 0.312 0.304 0.11 1
Table 2 Multicollinearity test. Model
Constant Place of res. Age Tenure Tariff Trend Avg. billing Avg. minutes Avg. frequency
Unstandardized coefficients
Standardized coefficients
B
Beta
Std. error 0.486 0.038 0.002 5.00E 05 0.03 0.15 0.003 0.001 0.001
0.052 0.022 0.001 0 0.019 0.011 0.001 0 0.001
t
Sig-
9.398 1.69 2.579 12.485 1.526 13.868 4.864 9.339 0.921
0.034 0.061 0.297 0.034 0.276 0.192 0.318 0.025
0 0.091 0.01 0 0.127 0 0 0 0.357
95% Confidence interval for B
Collinearity statistics
Lower bound
Tolerance
VTF
0.933 0.702 0.689 0.763 0.978 0.25 0.336 0.526
1.071 1.424 1.452 1.31 1.022 3.996 2.978 1.902
0.384 0.081 0 0 0.008 0.171 0.002 0.001 0.002
Upper bound 0.587 0.006 0.003 0 0.068 0.129 0.004 0.001 0.004
Dependent variable: churn.
3.4. Construction of Bayesian Belief Network We constructed BBN based on experts’ knowledge and prior analysis. In Fig. 3, a casual map is shown that some of the variables have no effect on churn directly but have effect indirectly through other variables. 4. Scenario analysis After the BBN is executed, we perform sensitivity analysis based on BBN to examine how much it reduces the uncertainty of churn if we know the information about other nodes. We develop three different scenarios according to the sensitivity analysis. 4.1. Scenario 1 The conditional probability of churn has the highest value when average minutes of usage is low, tenure is short, and a trend in billing amount is downward. Average minutes of usage and tenure are grouped into smaller parts to analyze in detail. Subscribers with a downward trend in billing amount have average billing amount PLACE OF RESIDENCE
AVERAGE FREQUENCY OF USAGE
TARIFF
AVERAGE MINUTES OF USAGE
CHURN
AGE
TENURE
between 9 and 90 TL. As seen in Fig. 4, by dividing the variables (average minutes of usage, tenure, and billing amount), new BBN is obtained as follows. The following chart (Fig. 5) is obtained utilizing the BBN. According to the figure, the percentage of churners with average billing amount 9–15 TL and 15–30 TL decrease, while their average minutes of usage increase. If customers have 10 min of usage for free, the number of churners decreases. In the following tables (see Table 3 and 4), expected savings are calculated. In the first column, the intervals of average minutes of usage are shown, and in the second column, average billing amount of the customers are shown according to the intervals. The number of customers is extracted from the data set. The forth column shows churn percentage of customers before promotion as shown in the previous chart (see Fig. 5). After giving 10 min free for these customers, their churn rate will decrease because their average minutes of usage will shift to the next interval. The results are shown in the fifth column. There will be a number of customers who are retained after promotion. For example, there are 63 customers who have average minutes of usage in the interval of 0– 10 min, and 82% of them are churners before promotions. If they have 10 min for free, they will behave like the customers in the interval of 10–20. Then, there will be 13 (= 63 (82%–61%)) customers that are retained. In the last column, the number of customers retained in the sixth column is multiplied with their average billing amount to calculate the expected saving. It is same for other customers with different average minutes of usage, and the total expected saving will be 199.97 TL according to this scenario. The calculations are made in the same way for the customers with an average billing amount 15–30 TL and after promotion the total saving 58.38 TL as shown in Table 4. 4.2. Scenario 2
AVERAGE BILLING AMOUNT
TREND IN BILLING AMOUNT Fig. 3. Causal map.
In this scenario, the effects of average frequency of usage, average minutes of usage, and average billing amount on customer churn are discussed in detail. Average frequency of usage is divided
7155
P. Kisioglu, Y.I. Topcu / Expert Systems with Applications 38 (2011) 7151–7157
Fig. 4. BBN of scenario 1.
Fig. 6. Churn percentage of customers with first and second type of tariff. Fig. 5. The percentage of churners with average minutes of usage and average billing amount.
4.3. Scenario 3 into two parts: customers who made outgoing calls and customers who made no outgoing calls. Similarly, average billing amount is divided into smaller parts. Customers who have an average billing amount in the 0–15 TL interval have lower churn rate if they make at least one outgoing calls. A promotion can be offered for this segment of customers such as giving 5 min free for outgoing calls. This promotion will help the company to reduce the churn rate. Table 5 explains the expected savings. The expected saving will be 174.9 TL. The same promotion can be applied to customers with an average billing amount between 15 and 30 TL (see Table 6). The expected saving will be 78.24 TL.
Tariff type has no direct effect on the dependent variable churn but has effects on average billing amount and average minutes of usage. Since tariff types for individual subscribers have higher propensity to churn than others, only individual subscribers are used in this scenario. There are two types of tariff for individual subscribers; we call them first and second type of tariff. First type of tariff is suitable for customers with a high number of outgoing calls. Second type of tariff has lower constant fee than first type of tariff. However, unit cost per minute is much more expensive in second type of tariff when compared with first type of tariff. Average billing amount is divided into smaller parts to have a de-
Table 3 Expected savings of customers with an average billing amount 9–15 TL. Avg. minutes of usage
Avg. billing amount
Number of customers
Churn percentage before promotion (%)
Churn percentage after promotion (%)
Number of customers retained after promotion
Expected saving
0–10 10–20 20–30 30–40
11.58 12.69 10.96 13.5
63 9 2 6
82 61 30 20
61 30 20
13 3 1
150.94 38.07 10.96
Total saving
199.97
7156
P. Kisioglu, Y.I. Topcu / Expert Systems with Applications 38 (2011) 7151–7157
Table 4 Expected savings of customers with an average billing amount 15–30 TL. Avg. minutes of usage
Avg. billing amount
Number of customers
Churn percentage before promotion (%)
Churn percentage after promotion (%)
Number of customers retained after promotion
Expected saving
0–10 10–20 20–30 30–40
17.83 20.61 19.94 18.68
9 5 8 9
87 77 76 60
77 76 60
1 1 1
17.83 20.61 19.94
Total saving
58.38
Table 5 Expected savings of customers with an average billing amount less than 15 TL. Outgoing calls
Avg. billing amount
Number of customers
Chum percentage before promotion (%)
Churn percentage after promotion (%)
Number of customers retained after promotion
Expected saving
No Yes
11.66 12.45
187 294
51 43
43
15
174.9
Table 6 Expected savings of customers with an average billing amount between 15 and 30 TL. Outgoing calls
Avg. billing amount
Number of customers
Churn percentage before promotion (%)
Churn percentage after promotion (%)
Number of customers retained after promotion
Expected saving
No Yes
19.56 20.47
132 529
27 24
24
4
78.24
Table 7 Expected savings for customers with average billing amount less than 30 TL. Billing amount
Tariff type
Avg. billing amount
Number of customers
Churn percentage before promotion (%)
Churn percentage after promotion (%)
Number of customers retained after promotion
Expected saving
0–15
1st 2nd
14.4 11.97
38 450
62 45
45
7
101
15–30
1st 2nd
23.23 20.94
376 587
27 25
25
5
116
Total saving
217
Table 8 Expected savings for customers with average billing amount greater than 30 TL. Billing amount
Tariff type
30–45
1st 2nd
45–60
Avg. billing amount
Number of customers
Churn percentage before promotion (%)
Churn percentage after promotion (%)
Number of customers retained after promotion
Expected saving
36.73 35.05
239 100
13 16
13
3
105
1st 2nd
51.61 49.22
73 15
7 16
7
2
98
60–90
1st 2nd
71.95 69.43
62 12
8 11
8
1
69
90–120
1st 2nd
104.72 103.75
10 2
5 22
5
1
104
120–400
1st 2nd
171.24 149.96
15 3
1 4
1
1
150
Total saving
527
tailed study. Fig. 6 shows the percentage of churn and average billing amount for the customers with first and second type of tariffs. Customers with first type of tariff have higher churn percentage than second type of tariff, while their average billing amount is less than 30 TL. To reduce the amount of churn, a promotion can be offered to customers such as changing their first type tariff to second type of tariff. Table 7 shows the expected savings will be 217 TL when customer is retained by this promotion. On the other hand, Fig. 6 shows that customers with second type of tariff have higher churn percentage than first type of tariff,
while their average billing amount is greater than 30 TL. For these customers, a promotion of changing their tariff type can be offered. Table 8 shows the expected savings will be 527 TL when customer is retained by this promotion. 5. Conclusions and further suggestions In this paper, we apply BBN to find out the most important factors that have effects on customer churn in telecommunication industry. Following results can be obtained from this research.
P. Kisioglu, Y.I. Topcu / Expert Systems with Applications 38 (2011) 7151–7157
First, explanatory variables that have effects on customer churn have high correlation with each other. Secondly, CHAID algorithm is used to discretize continuous variables, so we take advantage of using continuous variables in our model in an efficient way. Another thing is that BBN enables us to show casual relations between variables. Finally, due to the casual relations, three different scenarios are analyzed. In the first scenario, we consider customers with the highest propensity to churn. These customers have low average minutes of usage and a downward trend in billing amount, and also their tenure is short. A special promotion helps to retain them. In second scenario, we deal with average frequency of usage which is one of the most correlated variables with the dependent variable churn. BBN shows that subscribers who have no outgoing calls to different operators have higher churn rate than subscribers who have outgoing calls to different operators. A promotion is suggested to subscribers with no outgoing calls to different operators in order to avoid them from churn. In third scenario, tariff type is used to understand whether this variable has an indirect effect on churn or not. If subscribers have the right type of tariff, the number of churners due to ones that have wrong choice of tariff type will decrease. In each scenario, the expected savings from customers that will not churn after these promotions are calculated. Prediction of the number of customers who are likely to leave their carriers’ service to another and the factors that cause subscribers to churn are the main focus of many previous research. Data mining techniques such as decision trees and neural networks are usually used in these studies. However, these methods do not consider if there is any relationship between variables. For instance, it is obvious that monthly minutes of usage and monthly billing amount are interdependent variables. BBN mainly deals with the causal relationships between the factors and is the most efficient way to show such relations. In this study, a data set with 2000 customer data is used, and this research can be more meaningful when high quality of data is supplied with a large data set. There can be more churners in a large data set, so the number of churners retained and the expected savings will be much more than we have in this study. In addition, new variables such as additional service usage, the number of outgoing calls in different periods in a day, the number of incoming calls can be added to have a complex analysis. A survey can be made to evaluate customer satisfaction and understand if the promotions help company to save its customers. The inclusion of these materials as mentioned above can extend the future studies.
7157
References Baesens, B., Verstaeten, G., Poel, D., Petersen, M., Kenhove, P., & Vanthienen, J. (2004). Bayesian network classifier for identifying the slope of the customer lifecycle of long-life customers. European Journal of Operational Research, 156, 508–523. Chang, W., & Lee, K. J. (2009). Bayesian belief network for box-office performance. A case study on Korean movies. Expert Systems with Applications, 36, 280–291. Ferreira, J. B., Vellasco, M., Pacheco, M. A., & Barbosa, C. H. (2004). Data mining techniques on the evaluation of wireless churn. In ESANN’2004. Proceedings of the European Symposium on Artificial Neural Networks, Bruges, Belgium, 28–30 April. Heckerman, D. (1995). A tutorial on learning Bayesian networks, Technical Report, MSR-TR-95-06, Microsoft Research, Redmond, WA.
(2009). Retrieved 12.02.09. (2008). Retrieved 15.05.08. (2009). Retrieved 09.01.09. Hung, S., Yen, D., & Wang, H. (2006). Applying data mining to telecom churn management. Expert Systems with Applications, 31, 515–524. Jahanzeb, S., & Jabeen, S. (2007). Churn management in the telecom industry of Pakistan: A comparative study of Ufone and Telenor. Database Marketing and Customer Strategy Management, 14(2), 120–129. Karahoca, A., & Kara, A. (2006). Comparing clustering techniques for telecom churn management. In The 5th WSEAS International Conference on Telecommunications and Informatics, Istanbul, Turkey, May 27–29. Kim, H., & Park, Y. (2008). The impact of R&D collaboration on innovative performance in Korea: A Bayesian network approach. Scientometrics, 75(3), 535–554. Kim, H., & Yoon, C. (2004). Determinants of subscriber churn and customer loyalty in the Korean mobile telephony market. Telecommunications Policy, 28, 751–765. Lee, E., Park, Y., & Shin, J. G. (2009). Large engineering project risk management using Bayesian belief network. Expert Systems with Applications, 36, 5880–5887. Mozer, M. C., Wolniewicz, R., Grimes, D. B., Johnson, E., & Kaushansky, H. (2000). Predicting subscriber dissatisfaction and improving retention in the wireless telecommunications industry. IEEE Transactions on Neural Networks, 11(3), 690–696. Nath, S. V., & Behara, R. S. (2002). Customer churn analysis in the wireless industry: A data mining approach. Retrieved 28.08.2008. Ouali, A., Cherif, A. R., & Krebs, M. (2006). Data mining based Bayesian networks for best classification. Computational Statistics and Data Analysis, 51, 1278–1292. Richeldi, M., & Perrucci, A. (2002). Churn analysis case study, Telecom Italia Lab Report, Torino, Italy. S ß ahin, S ß . Ö., Ülengin, F., & Ülengin, B. (2006). A Bayesian causal map for inflation analysis: The case of Turkey. European Journal of Operational Research, 175, 1268–1284. SAS International (2001). Predicting churn. SAS Institute, Cary, NC. _ Aktasß, E., & Kabak, Ö. (2007). An integrated Ülengin, F., Önsel, Sß., Topçu, Y. I., transportation decision support system for transportation policy decisions: The case of Turkey. Transportation Research Part A, 41, 80–97. Wei, C., & Chiu, I. (2002). Turning telecommunications call details to churn prediction: A data mining approach. Expert Systems with Applications, 23, 103–112.