Review of bankruptcy prediction using machine learning and deep learning techniques

Availableonline online at www.sciencedirect.com Available Available onlineatatwww.sciencedirect.com www.sciencedirect.com

ScienceDirect Procedia Computer Science 162 (2019) 895–899 Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science 00 (2019) 000–000

www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia

7th 7th International International Conference Conference on on Information Information Technology Technology and and Quantitative Quantitative Management Management

(ITQM (ITQM 2019) 2019)

Review and Review of of bankruptcy bankruptcy prediction prediction using using machine machine learning learning and deep learning learning techniques techniques deep a,b,c b,c a,b,c,e, Yi Minglong Lei Leidd,, Yong Yi Qu Qua,b,c,, Pei Pei Quan Quanb,c,, Minglong Yong Shi Shia,b,c,e,* * a School of Economics and Management, University of Chinese Academy of Sciences, Beijing, 100190, China a b School of Economics and Management, University of Chinese Academy of Sciences, Beijing, 100190, China Research Center on Fictitious Economy & Data Science, Chinese Academy of Sciences, Beijing 100190, China b

Research Center Fictitious & DataManagement, Science, Chinese Academy of Sciences, Beijing 100190, China Laboratory of Bigon Data MiningEconomy and Knowledge Chinese Academy of Sciences, Beijing 100190, China Key Laboratory of BigofData Mining and Knowledge Management, Chinese AcademyBeijing, of Sciences, Beijing 100190, China d Faculty Information Technology, Beijing University of Technology, 100124, China d Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China e College of Information Science and Technology, University of Nebraska at Omaha, NE 68182, USA e College of Information Science and Technology, University of Nebraska at Omaha, NE 68182, USA

c Key c

Abstract Abstract Bankruptcy prediction has long been a significant issue in finance and management science, which attracts the attention of Bankruptcy prediction has longWith beenthe a significant issue in finance and management which attracts the attention of researchers and practitioners. great development of modern information science, technology, it has evolved into using researchers and practitioners. With the great development of modern information technology, it has evolved into using machine learning or deep learning algorithms to do the prediction, from the initial analysis of financial statements. In this machine or deep do the prediction, analysis of financial statements. In this paper, welearning will review the learning machine algorithms learning or to deep learning modelsfrom usedthe in initial bankruptcy prediction, including the classical paper, we will review the machine learning or deep learning models used in bankruptcy prediction, including the classical machine learning models such as Multivariant Discriminant Analysis (MDA), Logistic Regression (LR), Ensemble method, machineNetworks learning (NN) models as Multivariant Discriminant (MDA), Logisticmethods Regression Ensemble method, Neural andsuch Support Vector Machines (SVM),Analysis and major deep learning such(LR), as Deep Belief Network Neural Networks (NN) andNeural Support Vector (CNN). Machines and major deep learning such asand Deep Belief Network (DBN) and Convolutional Network In(SVM), each model, the specific process methods of experiment characteristics will (DBN) and Convolutional Neural Network (CNN). In each model, the specific process of experiment and characteristics be summarized through analyzing some typical articles. Finally, possible innovative changes of bankruptcy prediction andwill its be summarized through analyzing some typical articles. Finally, possible innovative changes of bankruptcy prediction and its future trends will be discussed. future trends will be discussed. © 2019 2020 The The Authors. Authors. Published Published by by Elsevier Elsevier B.V. B.V. © This is an open accessPublished article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) © 2019 The Authors. by Elsevier B.V. Selection and/or peer-review under responsibility of the organizers of ITQM 2019 Peer-review under responsibility of the scientific committee of the 7thofInternational Selection and/or peer-review under responsibility of the organizers ITQM 2019Conference on Information Technology and Quantitative Management (ITQM 2019) Keywords: Bankruptcy prediction, Machine learning, Deep learning Keywords: Bankruptcy prediction, Machine learning, Deep learning

1. Introduction 1. Introduction Bankruptcy prediction, also named as corporate bankruptcy prediction or corporate failure prediction, has long Bankruptcy prediction, also named as corporate bankruptcy prediction or corporate failure prediction, has long been a significant topic in the field of accounting and finance [1], since the health of a firm is highly important been a significant topic in the field of accounting and finance [1], since the health of a firm is highly important to its creditors, investors, shareholders, partners, even its buyers and suppliers. Researchers and practitioners to its creditors, investors, shareholders, partners, even its buyers and suppliers. Researchers and practitioners have been dedicated to developing methods and techniques to predict the bankruptcy of firms more quickly and have been dedicated to developing methods and techniques to predict the bankruptcy of firms more quickly and

* Corresponding author. Yong Shi. * Corresponding author. Yong Shi. E-mail address: [email protected]. E-mail address: [email protected]. 1877-0509 © 2020 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 7th International Conference on Information Technology and Quantitative Management (ITQM 2019) 10.1016/j.procs.2019.12.065

896

Yi Qu et al. / Procedia Computer Science 162 (2019) 895–899 Author name / Procedia Computer Science 00 (2019) 000–000

more accurately [2]. This research topic can be traced back to almost 50 years ago, when discriminant analysis [1] and logistic regression [3] were two well-known statistical machine learning techniques used in bankruptcy prediction. Since 1990’s, machine learning models have been extensively applied as tools to predict bankruptcy of firms, such as decision tree, neural networks and Support Vector Machiness [4]. Recently, deep learning has emerged and gradually developed into a powerful technique for a wide range of applications. It has achieved great success in auto-driving, computer vision, voice recognition, natural language processing, as well as classification problems in business and management like bankruptcy prediction and credit scoring. In this paper, we will mainly review the machine learning and deep learning techniques used in bankruptcy prediction, to summarize its specific process, characteristics, advantages and weaknesses by analyzing typical articles. Similar to the credit scoring, bankruptcy prediction is also typically a classification problem, which means it can be dealt with by classifying algorithms. In general, the task of bankruptcy prediction is to predict whether the firm will go bankrupt or not, which is a binary classification problem. To accurately conduct the prediction, we have to use algorithms to train the datasets, such as the financial data from the firm’s financial statements. Speaking of the financial statement data, Beaver (1966) probably was the first one to study the prediction of bankruptcy using these data [5]. However, it’s about calculating the financial ratios and compare them with predetermined cut-off thresholds [6], which is rather simple. This training process is where machine learning and deep learning techniques are applied. Through the training process of dataset, we can obtain a classifier with good classification accuracy, which can be used to do the bankruptcy prediction. This is the basic principle of bankruptcy prediction using machine learning or deep learning techniques. The rest of this paper will be arranged as follows: Section 2 will review the classical machine learning models used in bankruptcy prediction. Section 3 will summarize some typical bankruptcy prediction cases using deep learning algorithms. Section 4 mainly discusses the possible innovative changes of bankruptcy prediction and its future trends. 2. Machine learning techniques In this section, we will list and review several representative machine learning techniques used in bankruptcy prediction, such as Multivariant Discriminant Analysis (MDA), Logistic Regression (LR), Ensemble method and the well-known Neural Networks (NN), Support Vector Machines (SVM). Some classical papers will be reviewed to illustrate the specific process of experiment and its characteristics. 2.1. Multivariant Discriminant Analysis (MDA) Multivariant Discriminant Analysis (MDA) is based on applying the Bayes classification procedures and the strict assumptions that both positive and negative classes have Gaussian distributions with equal covariance matrices. The covariance matrix and the class means are estimated from the training set [6]. Altman (1968) used multivariant discriminant analysis to do the classification of solvent and insolvent firms based on their financial statement data [1]. Altman used five key financial ratios as the inputs, including Working Capital / Total Assets, Earnings Before Interest and Taxes (EBIT) / Total Assets, which have been widely used in subsequent research. 2.2. Logistic Regression (LR) Logistic Regression (LR) is essentially a linear model, using a sigmoid function f(x) = 1/(1 + 𝑒𝑒 −𝑥𝑥 ) to conduct the classification and the output is between 0 and 1, which enables the LR to have the capacity of probabilistic interpretation [6]. Ohlson (1980) introduced LR into the study of default prediction, using a novel set of financial ratios as inputs [3]. Unlike the model of Altman (1968), the output of LR is the probability of



Yi Qu et al. / Procedia Computer Science 162 (2019) 895–899 Author name / Procedia Computer Science 00 (2019) 000–000

897

default while MDA generates a score that is adopted to classify an observation between the good and bad classes [2]. 2.3. Ensemble method Ensemble method, also called multi-classifier method, is to incorporate several computational algorithms to achieve better performance. Nanni and Lumini (2009) conducted a series of experiments on financial datasets including Australian credit data, German credit data and Japanese credit data, finally found that ensemble method led a better classification performance than stand-alone models in bankruptcy prediction and credit scoring [7]. Boosting and Bagging are two major means of ensemble method. Boosting is a technique that firstly obtain a base classifier from the initial dataset, then adjust the distribution of training dataset based on the performance of the base classifier, and train the next base classifier with the adjusted sample distribution. It assigns a weight to each set of training, which can be used to design a set of bootstrap samples from the original data [8]. Kim and Upneja (2014) applied Adaboost, which is a typical method of Boosting, to predict the financial distress of restaurants and got successful forecast [9]. Unlike Boosting, Bagging relies on bootstrap, which generates random subsets of data by sampling from given dataset. It’s a technique created by several independent classifiers and runs a subroutine of its learners and then combines them through a model averaging technique, in order to reduce the overfitting of the model [10]. One representative approach of Bagging is Random Forest (RF), which is based on another traditional machine learning model, Decision Tree (DT). Kruppa et. al. (2013) presented a general framework to estimate credit risk by individual default probabilities, using the RF, which performed better than LR [11]. 2.4. Neural Networks (NN) Neural Networks (NN) is one of the most popular method in machine learning and probably is the inspiration of other computational methods [2]. It makes an analogy with human neural processing, containing several layers in which the input variables determine the first layer and the last layer produces the outputs variable. The output variables are mostly the tag or label of each sample. There have been numerous researches on the application of NN to predict bankruptcy. Such as, Zhao et. al. (2014) built an automatic credit scoring system with high accuracy (87%) and efficiency using Multi-Layer Perceptron Neural Network (MLPNN), with the experiment on German credit data [12]. In addition, several single NN can also be combined to have an ensemble model, which might perform better than a single classifier. Tsai and Wu (2008) investigated, compared the performance of single NN classifier and ensemble NN classifier on bankruptcy prediction and credit scoring. However, the multiple (ensemble) NN classifier didn’t outperform a single best NN classifier in many cases [13], which indicates that the multiple classifiers may not perform better in binary classification problems. 2.5. Support Vector Machines (SVM) Support Vector Machines (SVM) was firstly introduced by Cortes and Vapnik (1995) [13]. The most significant part of SVM is its kernel function, which is to transform the original data into high-dimensional data, in order to make the groups separable. This principle makes SVM a useful and powerful tool in classification. When the original sample data are transformed into high-dimensional data by the kernel function, what we need to do is to find a hyperplane with the widest distance to classify. What’s more, the selection of kernel greatly affects the performance of classification. As the linear kernel doesn’t provide great predictability in non-separable datasets, the RBF kernel and other non-linear kernel are difficult to analyse or even to discuss, but its prediction could be improved in non-separable cases [2]. A representative case in numerous studies is that Shin, Lee and Kim (2005) used SVM in corporate bankruptcy prediction and achieved better accuracy than back-propagation

898

Yi Qu et al. / Procedia Computer Science 162 (2019) 895–899 Author name / Procedia Computer Science 00 (2019) 000–000

NN, with the training set size is smaller [15]. Unlike the NN, SVM captures the geometric characteristics of feature space without deriving weights of networks from the training data, which makes it capable of extracting the optimal solution with the small training set size. 3. Deep learning techniques Deep learning, which has emerged almost 10 years ago, now is fairly hot and popular in both academic research and practical applications. It has been widely applied in the field of image recognition [16], voice recognition [17] and natural language processing [18]. However, there have been few researches on the application of deep learning in finance and management science. One typical application is to use Recurrent Neural Network (RNN) to do the prediction of stock price fluctuations [19, 20] since the RNN is suitable for time series analysis. Another major model in deep learning is Convolutional Neural Network (CNN), which have ever been put into predicting bankruptcy. Hosaka (2019) used financial statement data of Japanese listed companies, and transformed the numerical financial ratio data into grayscale image so that it will be adapted to the characteristic of CNN and can be directly analysed by CNN. Hosaka proposed a framework of CNN to deal with the bankruptcy prediction problems and this model outperforms representative conventional models including most of the traditional machine learning techniques [21]. Mai et. al. (2019) introduced deep learning into the prediction of bankruptcy, using layers of neural networks to extract features from textual data from over 10000 U.S. public companies. It’s been found that if some textual data (e.g. news, public report of companies) are conjunction with classical numerical data (e.g. financial ratio data), then the deep learning will yield superior performance in forecasting bankruptcy using textual disclosures, which will improve the accuracy of prediction furtherly [22]. These interesting findings provide new perspectives and inspirations for the research in the field of bankruptcy prediction. 4. Discussion and future trend After the review and summary above, now we are trying to discuss possible innovative change and future trend in bankruptcy prediction using machine learning and deep leaning techniques. One particular trend is the diversification of data sources. Former bankruptcy prediction papers would usually use numerical data, such as financial statement data, accounting data. Now the textual data, like news or public report even some comments from experts, are used to do the prediction. It generates a new concept called Multiple-Source Heterogeneous Data, which requires more advanced classification technique like deep learning technique, CNN. Another possible innovative trend is to take interpretability into consideration. Like Zhu et. al. (2013) proposed a comprehensive model to balance accuracy, complexity and interpretability in the consumer credit classification [23]. However, it’s hard to find out which of the inputs has a stronger impact on bankruptcy using most of the conventional methods. This means most of the intelligent techniques are not suitable for investigating the cause of bankruptcy. Therefore, it’s necessary to think about how to promote the interpretability of prediction models. Acknowledgements This work has been supported by Major Research Project of National Natural Science Foundation of China (#91546201). And we’d like to express our sincere gratitude for the contribution of all the editors and reviewers.



Yi Qu et al. / Procedia Computer Science 162 (2019) 895–899 Author name / Procedia Computer Science 00 (2019) 000–000

899

References [1] Altman E I. Financial ratios, discriminant analysis and the prediction of corporate bankruptcy[J]. The journal of finance, 1968, 23(4): 589-609. [2] Barboza F, Kimura H, Altman E. Machine learning models and bankruptcy prediction[J]. Expert Systems with Applications, 2017, 83: 405-417. [3] Ohlson J A. Financial ratios and the probabilistic prediction of bankruptcy[J]. Journal of accounting research, 1980: 109-131. [4] Lin W Y, Hu Y H, Tsai C F. Machine learning in financial crisis prediction: a survey[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2011, 42(4): 421-436. [5] Beaver W H. Financial ratios as predictors of failure[J]. Journal of accounting research, 1966: 71-111. [6] Atiya A F. Bankruptcy prediction for credit risk using neural networks: A survey and new results[J]. IEEE Transactions on neural networks, 2001, 12(4): 929-935. [7] Nanni L, Lumini A. An experimental comparison of ensemble of classifiers for bankruptcy prediction and credit scoring[J]. Expert systems with applications, 2009, 36(2): 3028-3033. [8] Begley J, Ming J, Watts S. Bankruptcy classification errors in the 1980s: An empirical analysis of Altman's and Ohlson's models[J]. Review of accounting Studies, 1996, 1(4): 267-284. [9] Kim S Y, Upneja A. Predicting restaurant financial distress using decision tree and AdaBoosted decision tree models[J]. Economic Modelling, 2014, 36: 354-362. [10] Breiman L. Bagging predictors[J]. Machine learning, 1996, 24(2): 123-140. [11] Kruppa J, Schwarz A, Arminger G, et al. Consumer credit risk: Individual probability estimates using machine learning[J]. Expert Systems with Applications, 2013, 40(13): 5125-5131. [12] Zhao Z, Xu S, Kang B H, et al. Investigation and improvement of multi-layer perceptron neural networks for credit scoring[J]. Expert Systems with Applications, 2015, 42(7): 3508-3516. [13] Tsai C F, Wu J W. Using neural network ensembles for bankruptcy prediction and credit scoring[J]. Expert systems with applications, 2008, 34(4): 2639-2649. [14] Cortes C, Vapnik V. Support-vector networks[J]. Machine learning, 1995, 20(3): 273-297. [15] Shin K S, Lee T S, Kim H. An application of Support Vector Machiness in bankruptcy prediction model[J]. Expert systems with applications, 2005, 28(1): 127-135. [16] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105. [17] Graves A, Mohamed A, Hinton G. Speech recognition with deep recurrent neural networks[C]//2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 2013: 6645-6649. [18] Yao K, Zweig G, Hwang M Y, et al. Recurrent neural networks for language understanding[C]//Interspeech. 2013: 2524-2528. [19] Yoshihara A, Fujikawa K, Seki K, et al. Predicting stock market trends by recurrent deep neural networks[C]//Pacific rim international conference on artificial intelligence. Springer, Cham, 2014: 759-769. [20] Chong E, Han C, Park F C. Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies[J]. Expert Systems with Applications, 2017, 83: 187-205. [21] Hosaka T. Bankruptcy prediction using imaged financial ratios and convolutional neural networks[J]. Expert Systems with Applications, 2019, 117: 287-299. [22] Mai F, Tian S, Lee C, et al. Deep learning models for bankruptcy prediction using textual disclosures[J]. European Journal of Operational Research, 2019, 274(2): 743-758. [23] Zhu X, Li J, Wu D, et al. Balancing accuracy, complexity and interpretability in consumer credit decision making: A C-TOPSIS classification approach[J]. Knowledge-Based Systems, 2013, 52: 258-267.