A credit ranking model for a parafinancial company based on the ELECTRE-III method and a multiobjective evolutionary algorithm

Accepted Manuscript Title: A credit ranking model for a parafinancial company based on the ELECTRE-III method and a multiobjective evolutionary algorithm Authors: Diego Alonso Gastelum Chavira, Juan Carlos Leyva Lopez, Jesus Jaime Solano Noriega, Omar Ahumada Valenzuela, Pavel Anselmo Alvarez Carrillo PII: DOI: Reference:

S1568-4946(17)30364-2 http://dx.doi.org/doi:10.1016/j.asoc.2017.06.021 ASOC 4290

To appear in:

Applied Soft Computing

Received date: Revised date: Accepted date:

24-2-2017 11-5-2017 12-6-2017

Please cite this article as: Diego Alonso Gastelum Chavira, Juan Carlos Leyva Lopez, Jesus Jaime Solano Noriega, Omar Ahumada Valenzuela, Pavel Anselmo Alvarez Carrillo, A credit ranking model for a parafinancial company based on the ELECTREIII method and a multiobjective evolutionary algorithm, Applied Soft Computing Journalhttp://dx.doi.org/10.1016/j.asoc.2017.06.021 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

A credit ranking model for a parafinancial company based on the ELECTREIII method and a multiobjective evolutionary algorithm Diego Alonso Gastelum Chavira*, Juan Carlos Leyva Lopez**, Jesus Jaime Solano Noriega**, Omar Ahumada Valenzuela* and Pavel Anselmo Alvarez Carrillo**

[email protected], [email protected] , [email protected], [email protected], [email protected]

*Catedratico CONACYT – Universidad de Occidente **Universidad de Occidente

Graphical abstract

Highlights

  

The ordering is based on objective and subjective criteria selected by the DM´s. The results indicate that there are significant differences among the loan applicants The ranked list of applicants provides a relative position instead of a fixed rule

Abstract Credit rating is an assessment performed by lenders or financial institutions to determine a person’s creditworthiness based on the proposed terms of the loan. Frequently, these institutions use rating models to obtain estimates for the probabilities of default for their clients (companies,

organizations, government, and individuals) and to assess the risk of credit portfolios. Numerous statistical and data mining methods are used to develop such models. In this paper, the potential of a multicriteria decision-aiding approach is studied. As a first step, the proposed methodology models the problem as a multicriteria evaluation process with multiple and in some cases, conflicting dimensions, which are integrated to derive sound recommendation for DMs. The second step of the methodology involves building a multicriteria outranking model based on ELECTRE III method. An evolutionary algorithm is used to exploit the outranking model. The methodology is applied to a small-scale financial institution operating in the agricultural sector. We compare loan applications based on their attributes and the credit profile of the customer or credit applicant. Our methodology offers the flexibility of combining heterogeneous information together with the preferences of decision makers (DMs), generating both relative and fixed rules for selecting the best loan applications among new and existing customers, which is an improvement over traditional methods. The results reveal that outranking models are well suited to credit rating, providing good ranking results and suitable understanding on the relative importance of the evaluation criteria. Keywords: Credit ranking model; Multicriteria Decision Analysis; Evolutionary Multiobjective Optimization; Rural Credit Application 1. Introduction Even when customers are the main target of companies, not all customers are profitable (for example, some customers are riskier than others), and lenders have to analyze whether to grant credit to an applicant, considering at the same time the risk of default and the profitability of customers. This is true for the majority of financial institutions but in particular for those serving farmers and others engaged in risky activities; it is even more important in places where credit is not yet mature. As an example, we present the case of rural financing in Mexico, which is characterized by inadequate outreach and limited access to finance, where financial services in rural areas are provided mostly by numerous small, often weak institutions, and parafinancial companies are becoming important sources of rural finance using different means [1]. In Mexico, a parafinancial is a reputable company that acts as a financial intermediary, which due to its management capacity, bargaining power and market positioning facilitates access, distribution and credit recovery, resources and / or services to producers and business sectors that individually face obstacles to these services. In the parafinancial model, the financing goes from the government banking institutions to agribusinesses that lend to farmers and farmer associations. Much of the lending is in kind through inputs the companies provide [2]. In this project, we tackle the needs of one of these non-banking financial institutions or parafinancial companies. Furthermore, given the nature and scale of farming in most operations in Mexico, the amount of financial information that farmers generate is limited; this leaves the DMs from financial institutions without the level of information found in other sectors of the economy, and this is required by more traditional models. The issue is how to make better decisions based on available data and supporting those decisions with the DM’s knowledge of the agricultural sector, and of the customers (using the DM´s preferences). We address some of these issues in this paper. There are usually two ways to model the DM´s preferences in Multicriteria Decision Analysis (MCDA): utility/value functions and outranking relations. Utility/value functions are monotone functions which measure the overall aggregated performance of the alternatives along all the evaluation criteria. On the other hand, an outranking relation is a binary relation between pairs of alternatives a, b  A , where A is a set of alternatives. This binary relation is based on the strength

of the arguments supporting the statement that "a is at least as good as b" (concordance) and the strength of the arguments against this statement (discordance) [9]. In this paper, we aim to solve the credit ranking problem as an MCDA problem of ranking a medium-sized set of credit applicants from a financial institution using the outranking method ELECTRE-III to model the preferences of the DM and a Multiobjective Evolutionary Algorithm (MOEA). The document is organized as follows: Section 2 introduces the credit ranking problem, and section 3 presents recent developments in multicriteria credit ranking. In section 4, we describe the methodology to develop a parafinancial Company’s model using ELECTRE-III method. A case study regarding loan applications received by the company, using a multiobjective evolutionary algorithm and a comparison with the distillation procedure of ELECTRE-III is presented in section 5. Finally, section 6 presents conclusions and future work. 2. Credit ranking problem Credit ranking is an assessment performed by lenders or financial institutions to determine a person’s creditworthiness based on the proposed terms of the loan. A related problem is credit scoring, which is a method of predicting potential risk corresponding to a credit portfolio. Models based on this method can be used by financial institutions to evaluate portfolios in terms of risk [3]. The benefits include reducing the cost of credit analysis, enabling faster decisions, insuring credit collections, and diminishing possible risks. Even a slight improvement in credit ranking accuracy is capable of reducing large credit risks, translating into significant future savings [4]. Many models attempt to distinguish good customers from bad customers based on credit applications. There are a number of classification methods, where logistic regression is considered to be the traditional statistical approach, and it has been the method of preference in this industry [5]. A common problem with logistic regression and discriminant analysis is that the training sample must contain a relatively high number of companies in a default situation, so that the number of healthy and distressed companies is as balanced as possible. Another issue is the classification itself; when considering the relative solvency of the companies rather than two states (eg. Good or Bad), we gain more information on the relation among the alternatives [6]. Another issue is the type and amount of information usually required from traditional classification methods; in practice, the building of credit risk models for private companies is necessarily limited by data availability [7]. To address some of these issues, financial institutions are allowed to use both objective and subjective models to improve their credit ranking models. In the first case, the rating is mechanical and considers mainly financial criteria (hard information) of the enterprise under evaluation; in the second case, the organization of enterprises into risk classes is based on the subjective analysis of non-financial criteria (soft information). Consequently, owing to small- and medium-size enterprises (SMEs) lack of sufficient or reliable track records, a rating that mainly relies on soft information seems the most useful approach to evaluate creditworthiness [8]. Furthermore, using non-financial variables together with financial ones as predictors of a company failure significantly improves the accuracy of the prediction model. However, this result is even more important for SMEs [7]. An alternative approach that can handle these heterogeneous sources are multicriteria decisionaiding (MCDA) techniques; these techniques enable the DM to introduce his or her preferences during the model building process and thus gain insight into the characteristics of the model; ultimately, the model is able to be calibrated so that it meets the requirements of the risk

management department of a bank or credit institution, which are well suited to handling the different issues presented by financial institutions in credit ranking problems [9]. Consider a set of medium-sized credit applicants evaluated by multiple criteria, some of which are in conflict with each other and are modeled according to an ELECTRE III framework [10]. In addition, consider a valued outranking relation that integrates the preferences of a decision maker over the criteria that evaluate those credit applicants. The problem at hand is as follows: Based on information contained in the valued outranking relation and its exploitation with a MOEA (RP2NSGA-II in [11]), to obtain a partial preorder of the set of loan applications, where the number of inconsistencies between the relative position and the preferences of the decision maker are minimized. Thus, we obtain a ranking of credit applicants whereby those with better credit profiles are in the upper levels of the ranking. In this paper, using the RP2-NSGA-II, we rank 105 credit applicants, representing a medium-sized set of alternatives. When a medium-sized set of alternatives A  a1 , a2 ,...,am  is evaluated based on a set of criteria G  g1 , g 2 ,...,g n  in order to obtain a multicriteria ranking, it is common practice to put them in

groups or classes of such alternatives. Each class contains a number of alternatives, which are indifferent among themselves and not indifferent to the members of other classes. When there is a relation of preferences among classes, we refer to a partial preorder where the incomparability among alternatives and groups of alternatives is allowed (see Figure 1). Such a problem is similar to a company that operates as a non-banking financial institution providing credit to farmers and other agricultural companies. Its customer base is composed almost entirely of SMEs, with limited financial information. Given the risky nature of the agricultural sector, the use of non-financial information in the model, the DM´s expert judgment regarding customers and the overall economic and market conditions are important criteria in deciding among the competing projects for a limited amount of funding. This application is the main contribution of this paper. A new model that provides better recommendations based on objective criteria and a structured decision process; the model addresses the particular needs of small-scale financial institutions in the agricultural sector; these institutions play a large part in providing credit and training in the rural sector, which is often overlooked by banking institutions. These companies require simple yet interactive methods to determine the best customers to grant a credit line. We argue that these needs can be met by the MCDA approach. 3. Literature review in MCDA In MCDA literature, there are many ways to rank credit applicants, based on the preferences of a DM (MACBETH [12]; AHP[13], TOPSIS [14]; ELECTRE III; TODIM[15], etc.). Under the most common setting, working with the MCDA paradigm in a credit ranking model, a credit rating model aggregates all pertinent information about a credit applicant, including financial and business data, as well as information about the external economic/business environment. A low performance even on a single credit evaluation criterion may (automatically) lead to the rejection of a credit applicant. Furthermore, outranking models enable the analyst to obtain important insight into the characteristics of the alternatives and the role of the decision attributes [16]. From the related literature, we can observe that there are different multicriteria models with MCDA techniques to address the credit ranking problem in financial institutions using utility/value function and outranking relations methods:

3.1. Utility/value function In [17] a contribution that addresses medium- and long-term loans to firms, enabling the multicriteria assignment of each application to a category and incorporating qualitative expert judgments using MACBETH is presented. The model of this contribution enables the multicriteria assignment of each application to a category that may range from rejection to acceptance with different spreads. In that sense, [18] presented a real case study related to the credit granting problem in a Greek commercial bank. For this application, the authors used UTADIS, which is implemented in a Multicriteria Decision Support System named FINCLAS. In the same way of the above contribution, [19] applied UTADIS method for credit risk modeling in financial institutions from the UK. The authors, compared their results with other methods such as discriminant analysis and logistic regression. They found that the classification accuracy obtained with UTADIS was higher than the others with an improvement of 70% at the validation stage. Likewise, [20] proposed a method composed by AHP and GRA to evaluate and select among the companies demanding commercial credit in a bank. The proposed method consists of two phases: The first one uses AHP to determine the weights of the criteria. Then, in the second phase, GRA is used to rank alternatives to select the best company. Similarly, [21] used AHP and TOPSIS to identify good and bad customers, applied to a bank in Iran. Using the bank expert’s opinions and the AHP method, they identified the main indicators in the field, for real and legal customers. Then using the TOPSIS method, they ranked credit applicants. Finally, TOPSIS´s results were compared against the logistic regression method. 3.2 Outranking relations One contribution uses ELECTRE-TRI to create a judgmental credit risk model for innovative SMEs, assigning innovative SMEs to risky categories [8]; In [22], the authors presented an evaluation of credit risk for strategic partners of a financial company using an intuitionistic fuzzy ELECTRE-III approach. In this contribution, the evaluation criteria are related to organization, economic, financial and management factors. Another approach involves pairwise comparisons based on the multicriteria decision aid (MCDA) paradigm with the help of their preference relations. This approach employs concepts from the outranking relations framework. The preliminary results of its application to a real-world data set involving a credit risk assessment problem and a comparison with an arsenal of well-known classification techniques are encouraging [9]. One MCDA application used an outranking multicriteria decision-aiding approach and an evolutionary algorithm to fit a credit rating model on the basis of the ELECTRE method, with an application to a large sample of Greek firms [23]; A deterministic approach for creditworthiness evaluation based on MURAME was proposed in [24], which consisted in the evaluation of the creditworthiness of firms with the proposed method, by an important northeastern Italian bank. According with the authors, the solution required two phases: a rating assignment phase, and a rating quantification phase. In the first phase, the debtors’ rating classes were determined, whereas in the second one, estimations of the probabilities of

default and of transition were obtained to calculate quantities and information regarding the (cardinal) rating and the (ordinal) ranking of the debtors’ credit quality. An approach with an application similar to the previous contribution was designed to address a credit worthiness evaluation problem faced by an important north-eastern Italian bank that needs to score and/or rank firms (which act as alternatives) applying for a loan [25]. They did not classify debtors into different homogeneous risk groups, as would be typical in classification techniques. They focused on the determination of ordinal ranks for them. The results obtained in this contribution show consistency between the scores/ranks produced by their methodology and the scoring/ranking of the firms provided by the bank. In the same sense, [26] used TODIM (a method that combines multiattribute value functions and outranking elements) to create a decision support system for the identification of multiple criteria, and for calculation their weights, for the evaluation of SME’s credit risk; including an application to real credit loan applications. As it can be observed, the use of MCDA methods in credit ranking models is relatively new but shows a great deal of potential in the few works that have been published. We further improve this area of research by providing new methods to handle large sets of alternatives by novel evolutionary algorithms. 4. ELECTRE-III method ELECTRE is a family of methods used for choosing (ELECTRE-I and IS), sorting (ELECTRE TRI) and ranking (ELECTRE II, III and IV), multicriteria problems. It was developed by Roy [27] to integrate the imprecise and uncertain nature of decision makers. ELECTRE continues to be a popular research field within MCDA. It has been successfully applied on diverse areas including the creditworthiness evaluation problems, bankruptcy and financial distress predictions, credit ratings, risk classification of loan applications, credit card assessments and prediction of share repurchase announcements [20]. According with [28], there are 599 articles where ELECTRE family has been used; where ELECTRE-III appears in 212 times with 22 applications related to business management and 7 applications with financial management; specifically, in areas of portfolio selection, forecasting of performance of stocks and financial distress prediction. Recently, [22] presented an evaluation of credit risk ELECTRE-III approach, as we noted above. With ELECTRE-III, a set of alternatives A  { a1 , a2 ,..,am } , a set of criteria G  { g1 , g 2 ,..,g n } and inter-criteria values must be defined. This is part of the preference modeling, which is briefly described as follows: The set of alternatives corresponds to the available elements to be ranked. On the other hand, the set of criteria is related with the elements used to evaluate the set of alternatives. The inter-criteria values, meanwhile, are the relative importance (weights) w j , the preference, indifference and veto thresholds of each criterion g j ; j  1,..,n . By using the previous sets and inter-criteria parameters from the ELECTRE-III method, we obtain a valued outranking relation S A , which integrates the preferences of a DM (see Roy [10]) for more information related to ELECTRE-III). 5. The exploitation model

At this point, we have a value outranking relation S A to be exploited to obtain at least a partial preorder of alternatives or partial order of classes of alternatives O*PK ( A ) . For this, an exploitation model is defined. A brief introduction is presented below (see [29] for more information). Let A  { a1 , a2 ,..,am } be a set of decision alternatives and S A be a valued outranking relation defined on A  A . Each pair of alternatives ( ai , a j )  S A represents the credibility degree  ( ai , a j ) of the predicate “ ai is at least as good as a j ”. Let  be a cut value between 0 and 1 (0    1) , which is associated with S A . For each cut level

 , we might induce a crisp outranking relation S A over S A , where if  ( ai , a j )   , we say that “ ai is at least as good as a j with credibility level  ” is true ai Sa j ; otherwise, it is false aiSa j , where  the negation logical operator. A relation S A can be deduced from the following preference relations:  is



Indifference ( I A ).

ai I Aa j

 ai S Aa j  a j S Aai



Preference ( PA ).

ai PA a j

 ai S A a j  a j S A ai



Preference ( PA ).

ai PA a j

 ai S A a j  a j S A ai



Incomparability ( R A ).

ai R A a j

 ai S A a j  a j S A ai

Considering the previous preference relations, a nested family of crisp outranking relations can be constructed and defined from S A as follows: S A  ( a ,b )  AxA : S( a ,b )  ,   0 ,1 ; where 0 is a minimum value for  . These crisp outranking relations represent the cut levels (-cuts) in S A , where the cut level  is the minimum value of S A for which aS A b is true. Figure 2 shows an example of a family of nested crisp outranking relations S A . Every single S A is generated through different cut levels  over a valued outranking relation S A . See Fodor [30] for a detailed description of this method. By definition, indifferent alternatives are grouped together; in doing so, the classes of alternatives are formed. On the other hand, alternatives that are not indifferent to each other are separated (assigned to different classes). Let Pk ( A )  { C1 ,C2 ,...,Ck } be a partition of set A in k classes of alternatives. When comparing a pair of classes Cr ,Cq  Pk( A ) , the most frequent relation by comparing alternatives of Cr with





alternatives of Cq is taken as a preference relation O among them, where O  I A , PA , PA , RA . Once the relations between classes are determined, a partial order of classes O*PK ( A ) is obtained. Given the relation mentioned above, it is common to find that some alternatives of a particular class are not indifferent to each other or that some alternatives belonging to a class are indifferent to other alternatives from other classes. In order to find a solution for a multicriteria ranking problem

without these kinds of irregularities is a difficult task, more so when A is a medium-sized set (15 – 200 alternatives) or greater. As it was mentioned before, a multicriteria ranking problem of a set of alternatives could be addressed as a multiobjective problem. In this section, we present the objectives and the model to be solved by RP2-NSGA-II. 5.1 Maximum Cut Level objective Each potential solution is associated to a -cut representing the level of credibility for the crisp outranking relation S A . Then, it is desirable to have potential solutions with a credibility level  close to 1. This indicates a high credibility level of the obtained ranking through the decoded procedure included in RP2-NSGA-II. The name of this objective is Maximum Cut Level Objective. 5.2 MinCut objective This objective is sustained based on the idea that the alternatives in a class should be indifferent among themselves. Using this property; the MinCut objective works by maximizing the indifference of the alternatives inside the classes. Here, it penalizes the pairs of alternatives that lie within a class but are not indifferent. This objective function is minimized in the respective multiobjective optimization problem and it is defined as follow: MinCutPk ( A ) 



* ( ai I Aa j )

(1)

Cr Pk ( A ) ai ,a j Cr

5.3 Minimum pair-wise preference disagreement objective The quality of a crisp outranking relation O*PK ( A ) should be judged according to the number of discrepancies and concordances with S A and the crisp outranking relation S A . Thus, it is necessary to have a function that counts the number of pair-wise preference disagreements. This is a function nV that quantifies the number of preference between the alternatives in the crisp outranking relation

S A that are inconsistent in the sense of O*PK ( A ) . This objective is defined as follow: nV P ( A )  k





min (* ( Cr O1Cq ),* ( aiO2a j ))

(2)

Cr ,Cq Pk ( A ) ai Cr ,a j Cq O1  O2

where O1  O2 denotes that preference relation O1 of Cr over Cq in O*PK ( A ) is different to the preference relation O2 of ai over a j in S A . Based on the previous objectives, the multicriteria ranking problem can be modeled as the following multiobjective optimization problem, which is addressed with RP2-NSGA-II: Min( Min Cut( ~ p )), Min( nV ( ~ p )), Max( ( ~ p )) Subjet to : ~ p    [ 0,1],   0

(3)

where  is the set of crisp antisymmetric outranking relations of classes of alternatives of A ; ~p is a crisp antisymmetric outranking relation of classes for a set of alternatives in A ; and 0 is a minimum level of credibility. For more details, see [29]. 6. The RP2-NSGA-II algorithm RP2-NSGA-II is a Multiobjective Evolutionary Algorithm based on NSGA-II [31]. RP2-NSGA-II tries to find a partial preorder of alternatives or partial order of classes of alternatives O*PK ( A ) from

a valued outranking relation S A in which the number of inconsistencies among O*PK ( A ) and the preferences of the DM S A are minimized. This partial preorder constitutes a ranking recommendation for a decision maker. The following subsections present brief details of the fundamental aspects of RP2-NSGA-II. In the jargon of evolutionary algorithms, the potential solution to a problem is called individual. In RP2-NSGA-II, each individual ~p for the multicriteria ranking problem is associated to a cut level, which represents the level of credibility of a crisp outranking relation S A . An individual ~p is composed of m( m  1 ) / 2 genes, where m is A ; that is, p1 , p 2 ,.., p m( m1 ) / 2 , where pi is the position of gen i in ~p . Each gene can take only one of four possible values v  { 0,1,2,3 } , where 0  ai I Aa j , 1  ai P  a j , 2  ai PA a j and 3  ai RAa j ; ai ,a j  A . Figure 3a A

shows the representation of a potential solution. There is a time where an individual ~p needs to be decoded; i.e., to obtain its meaning in the problem to be solved. In this case, in the decoding process of an individual, a partial order of classes of alternatives is obtained. Figure 3b shows the decoding process of the individual shown in Figure 3a. The selection, crossover and mutation operators as well as the mechanism to initialize, evaluate and discard a population were omitted; see [29] for more details. 7. Methodology This problem was addressed in two phases. The first phase consists of modeling the parafinancial company’s decision maker preferences with the aid of the ELECTRE-III method, which combines the logic of outranking models to construct a valued outranking relation that integrates the DM’s preferences. The second phase includes exploiting the valued outranking relation using a multiobjective evolutionary algorithm. The result of this exploitation is to obtain an ordering and present a recommendation in the form of a ranking for the problem at hand. If the DM does not agree with the result of any stage, he or she can return to any stage and make changes. The general outline of the MCDA process of this case study is based on [32], which is shown in Figure 4. 7.1 Phase 1: Preference modeling In the first phase, as mentioned, we used ELECTRE-III to model the decision maker preferences of parafinancial company obtaining a valued outranking relation.

There are many motivations for selecting ELECTRE-III as the method to model the DM’s preferences: 1.

First, because ELECTRE III enables the inclusion of qualitative and quantitative criteria, hence can be useful in modeling subjective reasoning, which is a powerful combination in settings like the one described for financial institutions in the agricultural sector and for other real-world problems.

2.

Second, ELECTRE-III is different from other techniques because it works with scales and criteria, therefore it is not necessary to normalize the scales in the integration of different criteria, thus allowing the use of financial and agricultural criteria, the DM’s preferences and other related criteria.

3.

Third, ELECTRE-III has been designed to handle the imprecise nature of the DM’s preferences using indifference and preference thresholds.

4.

Fourth, the ELECTRE-III method is non-compensatory, which implies that for any given alternative, a low score in one or more criteria is not compensated by better results in another criterion, this is relevant when considering credit ranking models since there could be some issues like credit delinquency that should not be compensated by any other criteria.

5.

Fifth, ELECTRE-III accepts incomparability among alternatives, which occurs when in any pair of alternatives a and b, there is not sufficient or clear evidence to favor any type of preference or indifference.

7.2 Phase 2: Exploitation Once the valued outranking relation S A is obtained, the set of alternatives based on the information contained in S A must be ranked. The resultant ranking of alternatives is the output of the second phase: the exploitation. This phase could be performed by traditional methods, but it is known that when the set of alternatives is a medium-sized set or greater, the number of inconsistencies among the obtained ranking and S A increases when classic methods are used, e.g., Distillation or Net Flow Rule (see [32] and [33]). For that reason, in the exploitation phase, we used a Multiobjective Evolutionary Algorithm called RP2-NSGA-II (please see [29]) embedded in SADGAGE ([11]), obtaining a partial preorder of the set of alternatives (applications) O*PK ( A ) . Thus, we obtain a ranking of credit applicants, and those with better credit profiles are in upper levels of the ranking. The previous multicriteria ranking problem can be treated as a multiobjective problem because each class must, as much as possible, contain only indifferent alternatives to each other, and at the same time, these alternatives must not be indifferent to the alternatives of other classes. Typically, when a multiobjective problem is solved, there is no single solution; instead, a set of solutions is obtained. Each solution of that set represents an optimal ranking of alternatives. Nevertheless, the set of solutions, known as the Pareto Front, could contain a lot of optimal solutions. Thus, we selected RP2-NSGA-II because it uses a strategy to find a Region of Interest (ROI) in the Pareto Front. This

ROI can be seen as a restricted Pareto Front with optimal solutions, which correspond to the DM preferences. 7.3. Execution parameters for the RP2-NSGA-II In addition to the valued outranking relation that integrates the DM’s preferences; other parameters must be determined to adequately operate the RP2-NSGA-II. These parameters are: size of the population, number of generations and crossover probability. The mutation operator does not need to be customized in this procedure, because it is automatically generated. In addition to the parameters for the multiobjective evolutionary algorithm, it is necessary to select an initial range for the credibility level: min and max . Further details regarding these settings are provided in the next section. 8. Case study As mentioned previously, the company is a non-banking financial institution that provides services, credit and agricultural supplies (such as seeds and fertilizers) to farmers in the northern region of the state of Sinaloa in northwest of Mexico. This type of institution serves as an intermediary between other financial institutions (usually banks) and their customers, and it has the responsibility of approving their loans, providing resources (partially in cash as well as in supplies) and collecting the loans. During each agricultural cycle, the parafinancial company faces the problem of selecting which farmers will be granted lines of credit for planting their crops. Given the risk involved in agricultural activities, there are only a few approved technology packages for growers (these include a set of standard activities for growing and harvesting the crops and a budget for those activities). Farmers must select their preferred crop from a fixed list: corn, beans, sorghum, chickpea and wheat. The amount of resources demanded by applicants usually exceeds the amount of the parafinancial company funds. There may be new customers entering or old ones leaving, so every season, the company must review all applications it receives to determine whether to grant or not to an applicant a line of credit for the new agricultural cycle. The current practice of the parafinancial company is to use experience and common sense to analyze credit applicants and select those with better credit profiles (trial-and-error method). In other words, the company does not use any objective method that enables risk mitigation of granting credit to customers with poor credit profiles and/or failing to grant credit to customers with a good credit profile. The latter case involves potential customers who are graded poorly or not accepted because of insufficient funds, whereas the loan is granted to customers with potentially lower credit profiles, hence riskier customers. In addition, given the scale and traditional practices of growers, very few have the required cash flow or documentation (nor the information) required by traditional credit ranking methods, thus restricting the decision model to the amount of information available in their application and the history of the grower with the company. The objective of the parafinancial company is to adopt and use a methodology that enables it to focus on customers with better credit profiles by ranking them from better to worse, preferably granting credit to better customers (greater chance of repaying the loan) until the existing funds are completely allocated. 8.1 Evaluation criteria

The evaluation criteria in this study are of qualitative and quantitative nature as shown in Table 1. g1. Quantity of hectares: Refers to the number of hectares available to farmers during the agricultural cycle, which may change because the farmers can rent from others in any given cycle. For the company, there is a range of desirable hectares for each account (20-30 Hectares) with a maximization objective (or direction). For this specific criterion, we used the trapezoid function for each number of hectares presented by the growers. The function has four parameters: Min, Lower, Upper and Max, which are defined in Figure 5:

where: Min Lower Upper Max

is the smallest possible number of hectares to accept. is the lower limit of the desirable number of hectares. is the upper limit to the desirable number of hectares. is the largest possible number of hectares to accept.

For a given number of hectares x, the trapezoid function is defined as: Hectares ( x, Min, Lower ,Uppper , Max )

0 ( x  Min ) /( Lower  Min )  Hectares   1  ( Max  x ) /( Max  Upper )   0

, x  Min ,Min  x  Lower ,Lower  x  Upper Upper ,  x  Max , x  Max

g2. Type of crop: Another criterion is the type of crop that the grower selects to plant in his field. There is a limited assortment of crops that can receive loans from the company, and these are beans, corn, wheat, chickpeas and sorghum. Because the parafinancial company specified a relative preference among the potential crops for the 2013-2014 season, we developed an ordinal scale for the crops using numbers 1-5, where corn has the highest value (5) and beans have the lowest value (1), still these could change given the market conditions for each season. The direction of this criterion is to maximize. 5. Corn 4. Chickpeas 3. Sorghum 2. Wheat 1. Beans g3. Years with the account: It is the time (in years), the grower has been a customer of the company. An account that has been with the company for a larger number of years is preferred over one with fewer years. However,

after an account has remained active for 7 years, it is considered to be loyal to the company, and there is an indifference among all the accounts that are at least 7 years old. This behavior is shown in Figure 6:

where: Min Loyalty

is the minimum amount of years of an account at the company. is the minimum amount of years to be considered a loyal customer.

The function of this criterion is defined according to the number of years x of the account: Years( x , Min , Loyalty )

 ( x  Min ) /( Loyalty  Min ) Years   1

Min ,  x  Loyalty ,x  Loyalty

g4. Type of collateral: Each grower who applies for a loan at the parafinancial company must sign a collateral agreement to ensure that the company can claim the debt if the loan is defaulted. The direction of this criterion is to maximize with an ordinal scale of 1-3 as follows: 1. Land Lease 2. Mortgage 3. Movable Assets The first collateral (Land lease) lets the company rent the land for a number of years if the customer defaults on his loan until the loan is repaid. The second is a mortgage agreement, in which the company obtains the deed of the land if the loan is defaulted. The third is for movable assets, such as trucks, cars and other equipment, which can be possessed by the company in order to repay the loan. g5. Location of the field in kilometers: This quantity is the distance from the field location where the crop will be planted to the closest the company distribution center to reduce the operational costs of supervising and harvesting the crops. There are three distribution centers in the region. This criterion has a minimization direction and is defined as follows: d( x )  min( c1( x ),c2 ( x ),c3 ( x )) where: x is the location of the grower´s field. cn is the distance, in kilometers, between the grower´s field x and the nearest distribution center n of the company.

g6. Time in the community: This quantity is the time in years that a grower has been a resident in his or her community. The parafinancial company values customers that have been living in their communities for some years

because this method ensures the borrowers will remain in the same location for the duration of the loan. The growers with the same residence for 7 years or more are considered to be permanent residents; thus, any customer that complies with this criterion has an identical value of preference to the company. The direction of the criterion is to maximize and it is defined as in Figure 7:

where: Min Permanent

is the minimum number of years that is possible to accept. is the minimum number of years to be considered a permanent resident.

For a given number of years of residence in community x, the preference functions is as follows: Time Re sidence( x, Min, Permanent ) ( x  Min ) /( Permanent  Min ) Time Re sidence   1

,Min  x  Permanent) , x  Permanent

g7. Credit delinquency: This category is provided to the accounts of applicants by the parafinancial company depending on their payment records, if loans were paid in timely manner as well as fully. This criterion is defined on an ordinal scale that is associated with discrete values from 1 to 5. The orientation of the criterion is to minimize, and the possible values for the scale are: 1. None 2. Low 3. Medium 4. High 5. Very High g8. Technical assessment of the field: This assessment indicates soil quality for the selected crop in the field where it will be planted. An agronomist with the parafinancial company analyzes the specific soil and assigns a value according to the vocation for the crop, which is proposed by the grower. The direction of the criterion is to maximize according to the following scale: 1. Regular 2. Good 3. Very good Each potential customer was assessed according to the criteria in Table 1. The results of that evaluation are partially presented in the performance table shown in Table 2. The complete evaluation is available at http://mcdss.udo.mx/datasets/parafinancial/performance.zip. 8.2 Thresholds and weights The ELECTRE family of multicriteria methods uses the concept of thresholds as one of its main components. To select these parameters in our project, we used the opinion and experience of an executive (DM) from the company, which helped us to select the thresholds that better reflected his

or her preferences. The DM aided us in selecting the indifference, preference and veto thresholds shown in Table 3. Similar to other methods, ELECTRE-III has weight parameters, but the difference with other methods is how these weights are used: We used them as “relevance coefficients” or “relative importance values” and not “substitution rates”; thus, avoiding all compensatory issues. In this project, we assisted the DM so that he could select the weights for each criterion according to the Personal Construction Theory (PCT), which was proposed by Roger [34]. Table 4 shows the selected weights for our problem. Using the performance table, thresholds and the relative importance of each criteria, a valued outranking relation was obtained with ELECTRE-III method. This valued outranking relation aggregates the DM´s preferences over the criteria that evaluate the 105 applicants. This last step finishes the DM´s preferences modeling process. The obtained valued outranking relation matrix is available at http://mcdss.udo.mx/datasets/parafinancial/valuedOR.zip. Then, in order to generate the ranking of credit applicants, the valued outranking relation was input to RP2-NSGA-II algorithm for its exploitation. The parameters to execute RP2-NSGA-II were set as follows: population size = 120; number of generations = 10,000; crossover probability = 90; minimum credibility level min  0.66 and max  0.75 . We also used the distillation procedure of ELECTRE-III method to exploit the valued outranking relation, and in that way, obtain a ranking of loan applications (see [35], [36] for an explanation about the distillation method). To compare the proposed method with the distillation method, we used the same input data, i.e. alternatives, criteria, performance of the alternatives, weight, thresholds and a cut level 0.66 which was obtained with RP2-NSGA-II. The aim of this additional activity was to compare results of both methods. 8.3 Results After running RP2-NSGA-II, there were several non-dominated solutions in the known Pareto Front. The concept of non-domination indicates that there are two solutions, a and b ; we can say that they are non-dominated if a has at least a better value in one of the objective functions with respect to b . Otherwise, solution b is dominated by a if the latter is better in all values of the considered objective functions. From that set of non-dominated solutions, we selected the ranking that showed the fewest number of global inconsistencies in the sum of the objective functions. In our case, the selected ordering had 730 inconsistencies in the classes and 879 preference disagreements among the alternatives in the crisp outranking relations, which are inconsistent in terms of the optimal crisp outranking relation. With a cut level of 0.66, this level of disagreement is 7.97% of all possible inconsistencies. The selected ranking from set of non-dominated solutions is shown in Figure 8. Based on this ranking, we also present all potential of the parafinancial company customers and their class in Table 5. The results indicate that there are significant differences among loan applicants; for example, the first class in the results (C7) exhibits better performance than the rest of classes as well as loan applications within those classes. However, alternative a58 should be in class 7 because in the

valued outranking relation for a cut level of 0.66, alternative a58 is preferred above all other alternatives. It is considered as an inconsistency. We observe that classes 1 and 2 integrate 59 and 33 credit applicants, respectively; which can be explained by the distribution of values of each criterion. The low granularity of data in criteria with the highest weights (Type of crop, Type of collateral, Credit delinquency, and Technical assessment of the field) also contributed to this result. In some cases, the selected threshold values and defined weights for each criterion contributed as well. The applicants (alternatives) in class 7 have the best profile for this problem (selecting applicant’s loans), but in some cases, such as applicant 101 in class 7 has a profile that is almost identical with applicant 62 in terms of performance criteria, which signals that both can be in either class 2 or 7. This behavior occurs because the objectives of the RP2-NSGA-II algorithm attempt to find solutions with alternatives that have the fewest inconsistencies. With the ranking obtained for loan applications and their organization into classes, the DM’s can start approving loans and assigning resources based on this ranking, since the ordering is based on both the objective and subjective criteria selected by the DM´s. Concerning the obtained ranking with distillation procedure of ELECTRE-III, it had 2,514 (22.80%) inconsistencies respect to the valued outranking relation. A partial ranking obtained with distillation procedure is show in Figure 9. From the ranking, alternatives a58 and a101 are at the top, which is in concordance with the valued outranking relation for a cut level of 0.66. However, alternative a62 should be at the same level as alternatives a58 and 101. The latter is considered as two inconsistencies. A complete version of the ranking obtained with distillation procedure is available at http://mcdss.udo.mx/datasets/parafinancial/distillation.zip. 9. Conclusions and future work The present case study used the ELECTRE-III method to model DM preferences in the form of valued outranking relations to generate applicant credit rankings. The use of evolutionary techniques, such as the one implemented in the RP2-NSGA-II, enabled us to generate an ordering that can be used to support credit-granting decisions for a parafinancial company customers (agricultural growers). This project also demonstrates the effectiveness of our approach for this type of problems. The case study was based on the criteria selected by the company to assess agricultural growers who apply for a loan and their projects. Because these criteria were not clearly defined, there were some inconsistencies, which we improved upon after the initial results; a refinement process was also conducted in close collaboration with company’s executives. We expect to further refine the selection criteria and improve the objectives by providing feedback regarding the effects of DM´s stated preferences on the results. We also expect to train the users and administrators on the model, given that the market and policy changes occurring every season significantly affect the relative riskiness of crops. The advantage of the proposed method over traditional ones is its flexibility in handling heterogeneous data and adopting DM preferences, while incorporating issues such as those related to the market (crop preferences), customer history (farmers), and the handling of multiple objectives. Also, the proposed method attempts to find solutions without inconsistencies, as opposed to deterministic methods, such as the distillation method, which lacks those characteristics.

When comparing, the rankings obtained with RP2-NSGA-II and the distillation method, we can perceive that the ranking of the latter method is more complex to interpret by a decision maker. Moreover, that ranking has more inconsistencies than the ranking obtained by RP2-NSGA-II. However, further testing is required to determine in which instances RP2-NSGA-II performs better than the distillation method and vice versa. The ranked list of credit applicants provides a relative position instead of a fixed rule as is the case of traditional methods such as logistic regression or discriminant analysis. Some may argue that it is good to have a strict rule for very risky applicants, but ELECTRE III also provides for such a rule with a veto threshold, in case one or more of the critical criteria is above or below the veto threshold; this offers the best of both methods (one such rule could be credit delinquency). Future work is expected to include a review of last season´s results to adjust the criteria weights and thresholds, therefore being able to better detect those customers and activities that are more profitable or have less risk. Another goal is to work on the objectives to evaluate other alternatives from the efficient frontier, where DMs can evaluate results based on selected points that are nondominated. We also expect to include other criteria that are relevant to the company – such as profitability of customers and a combination of risk and profits – to find a well-balanced portfolio. Also, we need to compare RP2-NSGA-II with other multicriteria methods such as PROMETHEE II [13] or Fuzzy TODIM and classical methods like logistic regression discriminant analysis. However, this first application targeted the most important issue, which was to reduce their risk exposure in their lending operations.

References [1] B. Mundial, Agriculture and Rural Development Public Expenditure Review, (2009). [2] IDFC, Perspectives on Development Finance, (2015). [3] A. Khashman, Credit risk evaluation using neural networks: Emotional versus conventional models, Applied Soft Computing, 11 (2011) 5477-5484. [4] C.-F. Tsai, M.-L. Chen, Credit rating by hybrid machine learning techniques, Applied soft computing, 10 (2010) 374-380. [5] Z. Zhang, G. Gao, Y. Shi, Credit risk evaluation using multi-criteria optimization classifier with kernel, fuzzification and penalty factors, European Journal of Operational Research, 237 (2014) 335-348. [6] F. García, V. Giménez, F. Guijarro, Credit risk management: A multicriteria approach to assess creditworthiness, Mathematical and computer modelling, 57 (2013) 2009-2015. [7] E.I. Altman, G. Sabato, N. Wilson, The value of non-financial information in small and medium-sized enterprise risk management, The Journal of Credit Risk, 6 (2010) 95. [8] S. Angilella, S. Mazzù, The financing of innovative SMEs: A multicriteria credit rating model, European Journal of Operational Research, 244 (2015) 540-554. [9] M. Doumpos, C. Zopounidis, A multicriteria classification approach based on pairwise comparisons, European Journal of Operational Research, 158 (2004) 378-389. [10] B. Roy, Decision-aid and decision-making, European Journal of Operational Research, 45 (1990) 324-331. [11] J.C. Leyva López, P.A. Álvarez Carrillo, D.A. Gastélum Chavira, J.J. Solano Noriega, A webbased group decision support system for multicriteria ranking problems, Operational Research, (2016) 1-36. [12] C.A.B. e Costa, J.-C. Vansnick, A theoretical framework for measuring attractiveness by a categorical based evaluation technique (MACBETH), in: Multicriteria Analysis, Springer, 1997, pp. 15-24. [13] T. L. Saaty. How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48 (1990) 9–26. [14] K. Yoon, C. Hwang, TOPSIS (technique for order preference by similarity to ideal solution)–a multiple attribute decision making, w: Multiple attribute decision making–methods and applications, a state-of-the-at survey, in, Berlin: Springer Verlag, 1981. [15] LFAM Gomes, MMPP Lima. TODIM: basic and application to multicriteria ranking of projects with environmental impacts. Fund Computing Decis Sci,16 (1991) 113–27. [17] C.B. e Costa, L. Barroso, J.O. Soares, Qualitative modelling of credit scoring: a case study in banking, Journal of European Research Studies, 1 (2002) 2. [18] C. Zopounidis, M. Doumpos. A preference disaggregation decision support system for financial classification problems. European Journal of Operational Research 130 (2001) 402-413.

[19] G. Baourakis, M. Conisescu, G. van Dijk, P.M. Pardalos, C. Zopounidis. A multicriteria approach for rating the credit risk of financial institutions. Comput Manag Sci. 6 (2009) 347–356. [20] (Z. K. ERTUğ, N. GIRGINER. Evaluation of banks' commercial credit applications using the Analytic Hierarchy Process and Grey Relational Analysis: A comparison between public and private banks. S. Afr. j. econ. manag. sci., Pretoria, 18 (2015) 308-324. [21] E. A. Bahabadi, M. Mohammadi. Customer Ranking Based on Credit Risk Using MCDM Methods. Management and Administrative Sciences Review, 5 (2016) 173-185. [22] (F. Shen, D. Lan, Z. Li. An Intuitionistic Fuzzy ELECTRE-III Method for Credit Risk Assessment. In: Xu J., Hajiyev A., Nickel S., Gen M. (eds) Proceedings of the Tenth International Conference on Management Science and Engineering Management. Advances in Intelligent Systems and Computing, Springer. 2017, 502. [23] M. Doumpos, C. Zopounidis, A multicriteria outranking modeling approach for credit rating, Decision Sciences, 42 (2011) 721-742. [24] M. Corazza, S. Funari, F. Siviero. A MURAME-Based Technology for bank decision support in credit worthiness assesment. Banks and Banks Systems, 9 (2014) 8-18. [25] M. Corazza, S. Funari, R. Gusso, An evolutionary approach to preference disaggregation in a MURAME-based creditworthiness problem, Applied Soft Computing, 29 (2015) 110-121. [26] (T. S. H. Gonçalves, F. A. F. Ferreira, M. S. Jalali, I. Meidutė-Kavaliauskienė. An idiosyncratic decision support system for credit risk analysis of small and medium-sized enterprises, Technological and Economic Development of Economy, DOI: 10.3846/20294913.2015.1074125 (2015). [27] B. Roy. Classement Et Choux en Présence de Points de Vue Multiples (La Méthode ELECTRE). Revue Francaise d' Informatque et de Recherche Operationnelle. 8, (1968) 57-75. [28] K. Govindan, M.B. Jepsen, ELECTRE: A comprehensive literature review on methodologies and applications, European Journal of Operational Research, 250 (2016) 1-29. [29] J.C. Leyva López, J.J. Solano Noriega, J.L. García Alcaraz, D.A. Gastélum Chavira, Exploitation of a Medium-Sized Fuzzy Outranking Relation Based on Multi-objective Evolutionary Algorithms to Derive a Ranking, International Journal of Computational Intelligence Systems, 9 (2016) 745-764. [30] J. Fodor, M. Roubens, Fuzzy Preference Modelling and Multicriteria Decision Support. 1994, in, Kluwer Academic Publishers, Dordrecht. [31] K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE transactions on evolutionary computation, 6 (2002) 182-197. [32] J.C.L. López, M. Araoz Medina, A multi-objective extension of the net flow rule for exploiting a valued outranking relation, International Journal of Multicriteria Decision Making, 3 (2013) 3654. [33] J.C. Leyva-Lopez, M.A. Aguilera-Contreras, A multiobjective evolutionary algorithm for deriving final ranking from a fuzzy outranking relation, in: International Conference on Evolutionary Multi-Criterion Optimization, Springer, 2005, pp. 235-249.

[34] M.G. Rogers, M. Bruen, L.-Y. Maystre, Electre and decision support: methods and applications in engineering and infrastructure investment, Springer Science & Business Media, 2013. [35] B. Roy. ELECTRE III: algorithme de classement basé sur une représentation floue des préférences en présence des critères multiples, Cahiers du CERO, 20 (1978) 3-24. [36] A. Bufardi, R. Gheorghe, P. Xirouchakis. Fuzzy Outranking Method: Recent Developments. In Cengiz Kahraman Ed. Fuzzy Multi-Criteria Decision Making: Theory and Applications with Recent Developments. USA: Springer Science + Business Media, LLC, 2008.

Figure 1. Different orders of a set of alternatives. Source: Own elaboration.

Figure 2. Nested family of crisp outranking relations. Source: Own elaboration.

Figure 3. Representation and decoding process of an individual. Source: Own elaboration

Figure 4. General flow of an MCDA process Source: Adapted from [19].

Figure 5. Preferences function for the amount of hectares. Source: Own elaboration

Figure 6. Preferences function for the customer loyalty. Source: Own elaboration

Figure 7. Preferences function for the customer loyalty. Source: Own elaboration

Figure 8. Class ranking obtained using RP2-NSGA-II Source: Own Elaboration.

Figure 9. Partial ranking obtained using distillation procedure of ELECTRE-III method Source: Own Elaboration.

Table 1. Criteria in the Case Study Label Criterion g1 Quantity of Hectares g2 Type of Crop g3 Years with the account g4 Type of Collateral g5 Location of the Field in Kilometers g6 Time in the Community g7 Credit Delinquency g8 Technical assessment of the field Source: Interviews with the company´s DMs.

Table 2. Performance of the alternatives on each criterion. g2 g3 g4 g1 Alternatives g1 a1 a2 a3 : a103 a104 a105

0.45 0.47 0.47 : 0.11 0.11 0.63

1 1 1 : 5 5 2

0.714 1.000 1.000 : 1.000 1.000 0.429

3 1 1 : 1 3 3

16 13 7 : 7 14 5

Preference orientation Maximize Maximize Maximize Maximize Minimize Maximize Minimize Maximize

g1

g1

g1

1.000 1.000 1.000 : 1.000 1.000 1.000

3 4 2 : 2 2 4

2 2 2 : 3 2 2

Source: Own elaboration. Table 3. Values of the Indifference ( q ), Preference ( p ) and Veto ( v ) thresholds Label Criterion g1 Quantity of Hectares g2 Type of Crop g3 Years with the account g4 Type of Collateral g5 Location of the Field in Kilometers g6 Time in the Community g7 Credit Delinquency g8 Technical assessment of the field Source: Constructed from DM preferences.

q

p

v

5 1 1 1 5 1 1 1

10 1 2 1 10 2 1 1

0 0 0 0 0 0 2 0

Table 4. Weights of the criteria g1 g2 g3 g4 g5 g6 g7 g8 RtG g1 g2 G g3 E g4 G g5 L g6 L g7 G g8 G Total

L L L L L E E

E G G L L G G

L E L L L E E

G G G G G G G

G G G G L G G

L E L L L L E

L E L L L L E -

2 4 2 4 0 1 4 4 21

RtG+1 Final Weight 3 5 3 5 1 2 5 5 29

0.103 0.172 0.103 0.172 0.034 0.069 0.172 0.172 1.000

Notes: 1. RtG  RtG  1 considering criterion 5. 2. For each cell, ij ,{ G , E , L } indicates that criterion g i is (Greater, Equal, Less) important than criterion g j . 3. The final weight for each criterion g i is obtained by dividing RtGi  1 over the total points. Source: Constructed from DM preferences.

Table 5. RP2-NSGA-II Results ordered based on the position of the class in the ranking. Alternative

Class

a68 a100 a101 a6 a16 a30 a31 a39 a41 a58 a59 a60 a61 a62 a64 a65 a70 a71 a74 a75 a77 a79 a80 a83 a87 a88 a89

7 7 7 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2

Class position in the ranking 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2

Source: Own Elaboration.

Alternative

Class

a90 a91 a92 a93 a94 a96 a98 a103 a104 a53 a57 a56 a8 a9 a105 a1 a2 a3 a4 a5 a7 a11 a12 a14 a15 a17 a18

2 2 2 2 2 2 2 2 2 5 5 6 3 3 8 1 1 1 1 1 1 1 1 1 1 1 1

Class position in the ranking 2 2 2 2 2 2 2 2 2 3 3 4 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6

Alternative

Class

a19 a20 a21 a22 a23 a24 a25 a26 a27 a28 a29 a32 a33 a34 a35 a36 a37 a38 a40 a43 a44 a45 a46 a47 a48 a49 a51

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Class position in the ranking 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6

Alternative a52 a54 a55 a63 a66 a67 a69 a72 a73 a76 a78 a81 a82 a84 a85 a86 a95 a97 a99 a102 a10 a13 a42 a50

Class position Class in the ranking 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 1 6 4 7 4 7 4 7 4 7