- Email: [email protected]
Decision support for proposal grouping: A hybrid approach using knowledge rule and genetic algorithm
Available online at www.sciencedirect.com
Expert Systems with Applications Expert Systems with Applications 36 (2009) 1004–1013 www.elsevier.com/locate/eswa
Decision support for proposal grouping: A hybrid approach using knowledge rule and genetic algorithm Zhi-Ping Fan a, Yuan Chen a,b,*, Jian Ma b, Yan Zhu b a
Department of Management Science and Engineering, School of Business Administration, Northeastern University, Shenyang 110004, China b Department of Information Systems, City University of Hong Kong, Kowloon, Hong Kong
Abstract Proposal grouping is a special procedure in the sponsorship process for research projects. In practice, it is conducted to simplify the following procedure of reviewer assignment. As the proposals grow, this procedure becomes complex. Practical managers spend an increasing amount of time struggling for identifying valid proposals, classifying proposals and partitioning proposals into groups as well as maintaining some control over the quality and composition of the resulting groups. This paper proposes an approach for proposal grouping, in which knowledge rules are designed to deal with proposal identification and proposal classification, and the genetic algorithm is developed to search for the expected groupings. In addition, a corresponding system is designed and developed to support the proposed approach. Compared to the previous manual grouping, the proposed approach significantly reduces the time required for grouping, ensures more diverse group composition, and increases overall grouping quality. Ó 2007 Elsevier Ltd. All rights reserved. Keywords: Project management; Knowledge rule; Genetic algorithm; Decision support system
1. Introduction A large portion of current academic research is sponsored through various agencies and funds with specific interests in different areas of research. Typically, the sponsorship process begins with a call for proposals (CFP), which is distributed to the relevant communities, such as universities and research institutions. Proposals are then submitted to the body (e.g., funding agencies) that issued the CFP. These proposals are assigned to experts for the peer review. Experts normally review the proposals according to the instructions on the rules and criteria of the funding agency. The review results are collected, and ranked based on the aggregation methods (Cook, Golany, & Kress, 2005). * Corresponding author. Address: Department of Management Science and Engineering, School of Business Administration, Northeastern University, Shenyang 110004, China. Tel.: +86 852 3442 6205. E-mail address: [email protected] (Y. Chen).
0957-4174/$ - see front matter Ó 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2007.11.011
The sponsorship process in China is special for its additional procedure: Proposal grouping. Before peer review, proposals in similar research disciplines are firstly partitioned into groups, and then proposal groups are assigned to reviewers. The goal of proposal grouping is to reduce the assigning times. Undoubtedly, by proposal grouping, the times of assigning proposal groups are less than that of assigning proposals. In most cases, the managers (decision makers) need to take some considerations about the group composition. For example, the proposals in one group should belong to different affiliations. For years, the procedure of proposal grouping has been conducted mainly by hand. In recent years, with the significant increase of proposals, such manual grouping is time consuming and tedious. Moreover, it is very difficult for the managers to regulate the group structure in the manual way. In year 2007, NSFC received more than 70,000 proposals. It can imagine that grouping so many proposals by hand is a so challenging work. In such circumstances, an effective grouping approach is highly desired.
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
In the past four decades, topics related to the research and development project selection have been widely discussed. A variety of decision models and methods have been developed to support the project selection (Henriksen & Traynor, 1999; Hochbaum & Levin, 2006; Tian, Ma, & Liu, 2002). They refer to many aspects of the selection process from different perspectives. Unfortunately, the problem of proposal grouping, to our knowledge, has not been stressed until now in this research area. Therefore, we seek to some parallel problems in other research areas. According to our literature review, the most similar problem could be found in the area of student group formulation. This problem can be described as partitioning a given number of students into groups that are diverse in their attributes. To solve this problem, many heuristic procedures are proposed (Beheshtian-Adekani & Mahmood, 1986; Weitz & Jelassi, 1992; Weitz & Lakshminarayanan, 1998). A bibliography of applications of heuristic procedures is provided in Weitz and Lakshminarayanan (1998). At the same time, a variety of linear and integer models are also developed for this problem (Baker & Benn, 2001; Reeves & Hickman, 1992; Saber & Ghosh, 2001). Most of these models are solved utilizing traditional optimal technologies. The above methods for the student grouping problem illuminate our research greatly. However, both the heuristic procedures and the mathematical models are failed to solve our problem directly for the following three reasons. Firstly, the heuristic procedure may provide a poor solution, or in fact, no solution at all. For example, Weitz and Lakshminarayanan apply the heuristic to a decision support system and they find that the grouping procedure may be blocked or only get the partial solutions (Weitz & Jelassi, 1992). Since our approach will be potentially applied to seven departments and more than hundreds of research disciplines, it requires high effectiveness for the grouping approach. Furthermore, given the integer nature of the mathematical models, there is no guarantee of arriving at a solution within a reasonable time limit especially for those large size problems (Dhar & Ranganathan, 1990). To the problem of proposal grouping, a distinct character is its huge number of proposals. So, the traditional optimal technologies can hardly guarantee the expected solutions. In addition, most of these models require rigorous assumptions and only can deal with wellstructured problems. The procedure of proposal grouping usually involves not only well-structured problems, but also semi-structured or ill-structured problems. For example, identifying valid proposals is a key step in this procedure, but it cannot be solved well if we only depend on the models. So, this paper attempts to propose an approach for proposal grouping that can make up the drawbacks of the above methods. This approach has the following two features. Firstly, knowledge rules are designed for the semi-structured or ill-structured problems in the grouping procedure like proposal identification and proposal classi-
1005
fication. On the other hand, genetic algorithm (GA) is developed to search for the expected groupings in the rational time. The knowledge rules and genetic algorithm complement each other and provide powerful support to the grouping procedure. It must be mentioned that the original motivation of our study is to support the real-world managers conduct the proposal grouping efficiently and effectively. So, a system is developed to support the proposed approach. The approach and the system are proposed to fit the grouping procedure of Natural Science Foundation of China (NSFC). However, it can be easily modified to cover other similar situations. The paper is organized as follows. Section 2 presents the proposal grouping problem that triggered our research interest. Section 3 proposes an approach for proposal grouping. Section 4 designs a decision support system to support the approach proposed in Section 3. Section 5 validates the proposed approach by a real life case from NSFC, and some conclusions are drawn in Section 6. 2. Problem presentation 2.1. Background information of NSFC In China, there are more than 30 government funding agencies. They are very similar in the project selection. NSFC (http://www.nsfc.gov.cn) is the largest and most reputable one. It provides financial support for the basic research projects that have great potential of scientific breakthrough or social impacts. In NSFC, there are four bureaus, one general office, three associated units and seven scientific departments. The bureaus, general office and associated units are mainly responsible for policy making, administration and other related affairs, while the scientific departments are responsible for the selection and management of the projects. In each department, there are several division managers who implement the proposal grouping and reviewer assignment. NSFC is supported by the Chinese central government, and its annual budget has been continuously increasing. For example, the budget for General Program has been increased from not more than 800 million (Chinese Yuan) in 2001 to over 2685 million in 2006. In the meanwhile, the number of proposals has been dramatically increasing from 23,636 in 2001 to 58,811 in 2006 (see Fig. 1). However, the average funded rate in recent six years is only about 18%. So, fair and rational selection process has been pursued by NSFC. 2.2. Proposal grouping in NSFC In NSFC, the procedure of proposal grouping involves several steps. Firstly, division managers must check whether the proposals accord with the application requirements. Then, they need to classify the proposals into different research disciplines according to the keywords provided in the proposal. After that, proposals with similar research
1006
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
Fig. 1. Statistical data about proposals and funded projects from year 2001 to 2006 [source: http://www.nsfc.gov.cn].
group
group
group
group Proposal Proposal Proposal
group
group
Poor group composition
group
group
Proposal
Expected group composition
Fig. 2. The examples of poor group composition and expected group composition.
disciplines would be partitioned into groups. For many years, this procedure has been conducted mainly by hand. In most cases, division managers have to struggle for accomplishing all the grouping steps as soon as possible. Nevertheless, they always spend more than one week on this procedure. In addition, as described above, they can hardly control the group composition. Given such situation, some considerations should be taken into account for the desired grouping approach. First of all, the efficiency of the whole grouping procedure should be improved. Since the procedure includes multiple steps, every step would influence the efficiency of the grouping. For example, manual classification of proposals takes managers much time. So, the proposed approach is supposed to involve every grouping step. Also, the group size in each group should be similar. The group size will influence the reviewer’s workload since the proposal group will be packed and assigned to reviewers in the following procedure. In most cases, evaluating proposals is a volunteer and unpaid work for reviewers. Therefore, equivalent workload is expected by reviewers. Obviously, it could be achieved by guaranteeing the similar group size between groups. Further, the group composition should be diverse. After proposal grouping, proposals in one group will be assigned to the same reviewers. The judgments of reviewers will be
influenced by the structure of the group. In the past, reviewers usually received the proposal group in which some proposals belong to the same affiliation. Reviewers always feel confused to evaluate such proposals. So, similar to student grouping problem, the expected group is that proposals in each group should be different as possible considering their attributes. Fig. 2 displays the examples of poor group composition and expected group composition. Other attributes related with proposal’s applicant are gender, professional title, applicant experience, etc. However, which attribute to be considered is usually determined according to the department’s management policy. In addition, the approach should be flexible. The proposed approach is expected to be applied in multiple departments of NSFC or other funding agencies, which may have different situations and requirements. Also, some conflicts would inevitably happen in the grouping procedure. So, leaving the space for manual cooperation will make the approach flexible and extensible. 3. The proposed approach Based on the manual grouping procedure described in Section 2, this section will present the proposed grouping approach. It is composed of five steps, which are shown in Fig. 3. An explanation of each step follows:
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
Step 1 Identifying valid proposals
Step 2 Classifying proposals
Knowledge rules
Knowledge rules
Step 5 Adjusting grouping results Manual
1007
Step 3 Determining attribute weight AHP
Step 4 Partitioning proposals into groups Genetic algorithm
Fig. 3. The proposed grouping procedure.
Step 1. Identifying valid proposals Firstly, valid proposals should be identified. In NSFC, there are many kinds of programs like General Program, Distinguished Young Scholar Program, etc. Each program has some distinct requirements to applicants. For example, in the Distinguished Young Scholar Program, the age of applicants must be not more than 35 years old. The proposals that satisfy the requirements can be turned into the following step of proposal grouping. In practice, the division manager has to check hundreds of proposals per week to see if they meet the basic requirements of NSFC. In this paper, knowledge rule is used to help the manager identify the proposal validation. It is a good tool to deal with such unstructured or semi-structured problem (Turban & Aronson, 1998). Using knowledge rules, the proposals for each kind of program can be easily categorized to invalid or valid. As for the invalid proposals, the division manager can turn down the proposals by filling in the NSFC Proposal Evaluation Form and send it to the department manager for approval. The valid proposals will be prepared for the next step after the confirmation of the division manager. Sample rules for identifying valid proposals are shown in Table 1. Step 2. Classifying proposals After the above step of identifying proposal validation, the next step is to classify proposals. Recall that proposals in similar research disciplines could be partitioned into groups. In NSFC, each proposal is required to provide three keywords to indicate the research domain it belongs to. NSFC maintains a dictionary of keywords that forms a tree structure. The closer a keyword node is to the root, the larger research domain it represents. For example, keyword ‘A010103’ stands for ‘Geometry’ and ‘A01010302’ for ‘Algebraic Geometry’. Similar to identification of proposal validation, managers also spend much time on proposal classification. Aside from the low efficiency, due to the limitation of human
Table 1 Sample rules for identification of valid proposals 1. Rules for general program IF Number of On-going Projects + Number of Submitted Proposals > 2 THEN Status = Invalid IF Unregistered (PI) or Unregistered (affiliated organization) THEN Status = Invalid IF Position of PI = Full-Time Graduate Student OR Position of PI = Part-Time Graduate Student OR Position of PI = Retired THEN Status = Invalid IF PI In Bad Reputation List Record THEN Status = Invalid 2. Rules for Distinguished Young Scholar Program IF Proposal Category = Distinguished Young Scholar AND Age > 35 THEN Status = Invalid IF Proposal Category = Distinguished Young Scholar Program AND Numbers of On-going Distinguished Young Scholar Program > 0 THEN Status = Invalid IF NOT (Highest Degree = PhD) AND NOT (Position = Senior) AND (No. of Ref. Letters < 2) THEN Status = Invalid
capability, the division manager can hardly identify the multiple disciplines of each proposal correctly. Consequently, knowledge rules are also designed to help the managers classify proposals. Sample rules are shown in Table 2. Step 3. Determining attribute weight Then, the attribute weight should be determined in step 3. As introduced in Section 2, the grouping with diverse group composition is expected. It is defined as the sum of differences between every pair of proposals in this group (Weitz & Lakshminarayanan, 1998). The difference between two proposals could be obtained by adding the weighted contributions of all attributes by which the two
1008
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
Table 2 Sample rules for classification of proposals For q = 1 to Q For i = 1 to m If pi R {categorized proposals} Then For j = 1 to Qi If Rij ¼ Rq Then pi will be added to Sq End loop End if Next If j = Qi Then pi will be added to set {categorized proposals} End if End if Next Next Q number of research disciplines Rq the qth discipline defined by NSFC for q = 1, 2, . . . , Q m number of proposals pi the ith proposal for i = 1, 2, . . . , M Qi number of research disciplines of proposal pi for i = 1, 2, . . . , M Rij the jth research discipline of proposal pi for i = 1, 2, . . . , M and j = 1, 2, . . . , Qi Sq Set of proposals for the qth research discipline
proposals differ. Mathematically, the difference of two proposals could be formulated as X wk jaki akj j; ð1Þ d ij ¼ k
where dij is the difference between proposal i and j; k is the index of the attribute, and k P 1; aki is a binary indicator, which takes the value of 1 if proposal i exhibits the attribute k, otherwise it take the value of 0 and wk is the weighting factor for attribute k. From Eq. (1), we can deduce that the difference of two proposals is determined by their attribute value and the attribute weight. As described in Section 2.2, in practice, many attributes tied up with each proposal are supposed to be considered in proposal grouping. However, what attributes should be chosen is a decision making task related to the decision maker’s (DM) objective. In different department, the situation of applicants is different. For example, the proportion of female applicants to male applicants in the department of Earth Sciences is less than that in the department of Management Sciences. So, if the rational proportion of female applicants needs to be stressed, gender (female) will be chosen to measure the difference of proposals. In most cases, at least one attribute will be taken into consideration. On the other hand, weights reflect the relative importance of each attribute, which would influence the grouping result (Weitz & Jelassi, 1992). However, it is an uneasy task for managers to assign numerical scores to the attribute. While they implicitly give weight of each attribute, they may find the explicit formulation of weight difficult. Part of this has to do with the well-known problem of attempting to express qualitative judgments quantitatively (Bucha-
nan & Schortliffe, 1984). There are many approaches to be used for determining the attribute weight (Chu & Kalaba, 1979; Ma & Fan, 1999; Saaty, 1980; Zadeh, 1987). AHP (Saaty, 1980) is a suitable approach to undertaking this task for two reasons: It is user friendly because users may directly input judgment data without the requirement of mathematical knowledge. Relevant inconsistencies in managerial judgments or perceptions are dealt with appropriately (Saaty & Hu, 1998). So, AHP can be implemented to determine the importance of an attribute to another through pair-wise comparison by the managers. In the AHP, the scale of absolute values of 1–9 is used for making the pairwise comparison judgments. The scales are listed and explained in Table 3. Thus, making the pairwise comparison judgments among attributes can determine the weights of different attribute. Step 4. Partitioning proposals into groups In step 4, the valid proposals in similar research discipline will be partitioned into groups through two substeps. In substep 4.1, a mathematical model is built according to the description of expected groups in Section 2.2. Then, in the following substep 4.2, the model is solved by GA considering the character of the model. Step 4.1. Building the model Firstly, decision variables and parameters to be used in the model are defined as follows: xig is the decision variable, if proposal i is allocated to group j then xig = 1; otherwise xig = 0; dij is the difference between proposal i and j as same as defined above; M is the total number of proposals and G is the total number of groups. Then, to get the expected groups described in Section 2.2, a model is built as follows: max
M 1 X G X M X i¼1
min Subject to
g¼1
d ij xig xjg ;
G1 X G X M M X X xig xit ; g¼1 t>g i¼1 i¼1 G X
ð2Þ
j>i
xig ¼ 1 for i ¼ 1; 2; . . . ; M;
ð3Þ ð4Þ
g¼1
xig 2 f0; 1g for i ¼ 1; 2; . . . ; M and g ¼ 1; 2; . . . ; G:
ð5Þ Table 3 Comparison scale (Saaty & Vargas, 1991) Absolute value
Definition
1 3 5 7
Equal importance Moderate importance of one over another Strong or essential importance of one over another Very strong or demonstrated importance of one over another Extreme importance of one over another Reciprocals for inverse comparison
9 Reciprocals
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
In the above model, there are two objectives. Objective function (2) is to maximize diversity of the group composition. Based on the difference between two proposals, diversity in one group could be represented as the sum of difference of each pair of proposals in this group. Objective function (3) is to minimize the difference of group size. That is, the solution with similar group size is preferred. Aside from the objectives, the constraint function (4) is to guarantee that each proposal is allocated to one group. Notice that the two objectives subject to constraint (4) give rises to a challenging nonlinear zero–one program containing M constraints and MG variables. If an application of the model is the grouping of 500 proposals to 20 groups, it would give rise to a formulation with 500 constraints and 10,000 variables. Consequently, the multiobjective, combinatorial, and large size character of the model results in a challenge for the solution. Step 4.2. Solving the model There are many techniques to deal with multiple objectives (Hwang & Yoon, 1981; Steuer, 1992; Steuer & Choo, 1983; Vassilew & Narula, 1993; Yu, 1985), in which the linear weighting method has been widely used since it is very simple (Hwang & Yoon, 1981; Yu, 1985). Multiple objectives are combined to one scalar objective by the sum of weighted objectives. Considering the two objectives in our model are different in their scales, we normalize them using the objective attainment (Chen, Wang, & Lin, 2008). The attainments of the two objectives are defined as follows: uðZ 1 Þ ¼ 1
Z max Z1 1 max Z 1 Z min 1
uðZ 2 Þ ¼ 1
Z 2 Z min 2 ; min Z max Z 2 2
and ð6Þ
where Z1 and Z2 denote the two objectives in the model; u(Z1) and u(Z2) denote the correspondingly attainment of the objective Z1 and Z2. The value of u(Z1) and u(Z2) is between 0 and 1. Notice that the maximum problem of Z1 has been represented as an equal minimum problem. That is, if the objective Z1 achieves its maximum value, the u(Z1) will get the minimum value of 0. So, the single objective combining the two objectives is min uðZ 1 Þ þ uðZ 2 Þ;
ð7Þ
where the weight of each objective is set as the same. Then, GA is applied to solve the single objective model since it has been successfully implemented to find good solutions to a wide variety of combinatorial problems in rational time (Jones & Beltramo, 1991; Mu¨hlenbein, Gorges-Schleuter, & Kra¨mer, 1988; Peachter, Luchian, & Petruic, 1994). Invented by Holland and his associates in the 1960s (Holland, 1975), it searches the solution space in parallel, working with whole sets of feasible solutions, the so-called population, rather than with individual solutions. By iterating between consecutive generations, it tries
1009
to generate better solutions by combining or modifying good solutions, in order to gradually progress towards the optimal solution. The GA solution process developed to solve our model mainly includes representation, selection, genetic operation, and stopping criteria. Each stage of the process is explained as following: (a) Representation. In the existing literatures of GA, there are different schemes to represent a chromosome: an adjacency representation, an ordinal representation, a path representation, a position listing representation, an adjacency listing representation, and group-number encoding (Michalewicz, 1996). To the problem under consideration, group-number encoding is adopted since it was proposed particularly for grouping problem (Jones & Beltramo, 1991). For our problem, each grouping is represented as (y1, y2, . . . , yM), where the ith integer yi 2 {1, 2, . . . , G} indicates the group number is assigned to proposal i. The structure of the chromosome is shown in Fig. 4. For example, y1 = 6 denotes that proposal 1 is allocated to group 6. (b) Selection. The selection stage consists of determining from which chromosome the next population would be generated. For the next generation, GA is inclined to keep the chromosomes that have better fitness. For our problem, the chromosome fitness is measured by the objective function (7). Further, the stochastic universal sampling method as well as the tournament selection technique has been applied to select the best fitness chromosome. (c) Genetic operation. Genetic operation involves crossover and mutation. The goal of crossover is to exchange information between two parent chromosomes in order to produce two new offspring for the next population. The aim of mutation is to increase genetic diversity into the population by introducing random variations into the members of the population. The group-number encoding creates a possibility of applying standard operators. A mutation would replace a single gene yi by a random number from {1, 2, . . . , G}. Crossovers (single point) would always produce a legitimate offspring. However, as pointed out in Jones and Beltramo (1991), an offspring (after mutation or crossover) may contain less than G groups; moreover, an offspring of two parents, both representing the same grouping, may represent a totally different grouping, due to different numbering of groups. Special repair algorithms (rejection method, renumbering the parents) are used to eliminate these problems.
chromosome
1
2
...
M
y1
y2
...
yM
Fig. 4. Representation of the chromosome.
1010
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
(d) Stopping criteria. There are several stopping criteria that are commonly cited in the literatures (Gen & Cheng, 2000). For example, several consecutive generations reach the same solution, the GA search stops. As another example, the search stops when the improvement in solution quality becomes negligible. Allowing for computational efficiency, this paper adopts a maximum number of runs as the stopping criterion as used in Azadivar and Tompkins (1999). Step 5. Adjusting grouping results The essential purpose of the proposed approach is to help the manager to conduct the grouping task quickly and rationally. However, some unpredictable thing could inevitably happen in practice. Also, some managerial policies will be changed in the future. In such situations, it needs manager to manually adjust the grouping results. For example, applicant may ask that his proposal may not be evaluated by some reviewers. Thus, the manager needs to adjust the grouping results manually by changing such proposal to another group. 4. System design and development To support the grouping approach proposed in Section 3, the corresponding system is designed and developed. The underlying technology includes Microsoft Windows 2000 Server, Microsoft SQL Server, and Microsoft Internet Information Server. The system is built using Microsoft Active Server Page. The major components of the system
include the database, the knowledge base, the model base, and user interfaces. (a) Database. There are two major categories of data stored in the database: Human resource data and Proposal data. Human resource data consists of those for Internal Manager and Applicant. Internal Manager consists of Top Manager, Department Manager, and Division Manager. Applicant consists of Individual Applicant and his/her affiliated Organization. Each Individual Applicant should have an affiliated Organization. Individual Applicants submit proposals through their affiliated Organizations to the funding agency. Proposal data is gleaned from the application sheet, in which fields comprising the applicant’s information are applicant ID, name, age, gender, professional title, education, etc. (b) Knowledge base. According to the proposed grouping approach in Section 3, knowledge rules for identifying valid proposals and knowledge rules for classifying proposals are designed. These two categories of knowledge rules are stored in the knowledge base. (c) Model base. The model base includes the GA and Weight adjustor. GA is developed to search for the satisfactory groupings. Weight adjustor is used to adjust the weights provided by decision makers (managers), when the judgment matrices are inconsistent or the grouping focuses change. (d) User interfaces. The user interface consists of a menudriven dialogue through which the users interact with the system. There are two categories of potential users: internal managers and some external experts.
Fig. 5. The page of experts’ judgments on the attribute weight.
Sometimes, some external experts will be invited to determine the attribute weight together with the internal managers. So, the user interfaces must be a webbased interface with sufficient security mechanisms embedded so that external experts can participate in different locations. As an example, the page for determining attribute weight in the prototype system is shown in Fig. 5.
Number of proposals
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
1011
20 18 16 14 12 10 8 6 4 2 0 0
1
2
3
4
5
6
7
8
9
10
Group
5. Application and evaluation
Number of proposals
20 18 16 14 12 10 8 6 4 2 0
0
1
2
3
4
5
6
7
8
9
10
Group Fig. 7. The group size in each group by the proposed approach.
On the contrary, in our approach, multiple attributes could be easily considered. Also, the maximum diversity in group components under multiple attributes could be achieved in compromising to the minimum differences of group size. For instance, we choose three attributes tied up with proposal like gender (female), professor, experience and assign them the same weight. Specifically, we set 1 – female and 0 – male; 1 – professor and 0 – otherwise; 1 – experienced applicant and 0 – otherwise. Then, we trace the group composition in the manual grouping (see Fig. 8). Under the same attributes, we also test our approach and present the grouping result in Fig. 9. Contrasting Figs. 8 and 9, we can observe that under multiple attributes, the proposed approach is significantly better than the manual approach considering the diverse group composition. In addition, the system promotes the efficiency in proposal grouping. By manual grouping, the manager needs
Number
In order to assist the project selection in NSFC, we developed the Internet-based Science Information System (ISIS, https://isis.nsfc.gov.cn). It has been used for NSFC’s electronic submission and on-line evaluation of proposals, and dissemination of project outputs since May 2000. However, proposal grouping has not been probed and incorporated into ISIS. So, the designed system to support the proposed grouping approach will be complemented to the ISIS. To validate the proposed approach, the system is tested using the simulation data of research domains in different department of NSFC. There are two major criteria for the success of the system. One is that the groupings created through the use of the system should be as good or better than those obtained via the traditional manual grouping. Here the ‘‘good or better” could be measured from two perspectives: whether the group size is similar between groups and whether the group components are diverse in each group. Another criterion is that the use of the system should result in significant time savings. Under such criteria, this paper presents the comparison of the manual grouping and proposed system grouping using the simulation data of the department of M1 in 2006. The department of M1 accepted 4240 proposals in 2006. There are 168 proposals in research domain #K1. These proposals were partitioned into 10 groups. As introduced above, the division manager needs to consider many factors simultaneously when conducting the grouping. Sometimes, he has to neglect the difference of group size between groups to pursue the diversity group composition. It gave rise to the possible significant difference of group size between groups (see Fig. 6). When these proposal groups are assigned to reviewers, the resulting workload of reviewers will not be equal. In practice, such situation is not expected by both reviewers and managers. In contrast, since the minimum difference of group size is one objective in our model, it will be achieved as possible in compromising to the other objective of maximum group diversity. Fig. 7 shows the group size in each group using the proposed approach. From Figs. 6 and 7, we can see that the proposed approach can well guarantee the similar group size between groups. On the other hand, pursuing diverse group composition by hand is more difficult for the division manager. In most cases, the manager only could focus on one or two attributes under the limitations of time and human judgments.
Fig. 6. The group size in each group by manual grouping.
20 18 16 14 12 10 8 6 4 2 0
Female Professor Experienced
1
2
3
4
5
6
7
8
9
10
Group Fig. 8. The resulting group composition by manual grouping under three attributes.
Number
1012
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013 20 18 16 14 12 10 8 6 4 2 0
Acknowledgements Female Professor
This work was partly supported by the National Science Fund for Distinguished Young Scholars of China (Project No. 70525002), National Science Fund for Excellent Innovation Research Group of China (Project No. 70721001), the Competitive Earmarked Research Grant of Hong Kong SAR (CERG, Project No. CityU 118705, CityU 1237/03E) and Strategic Research Grant of City University of Hong Kong (Project No. 7002049).
Experienced
1
2
3
4
5
6
7
8
9
10
Group Fig. 9. The resulting group composition by the proposed approach under three attributes.
to spend at least one week. Using the system, the grouping can be finished within hours. It must be mentioned that the number of proposals are different in departments. For example, there are 25,199 proposals in the department of Life Sciences in 2006, nearly six times more than that in Management Sciences. So, time savings will be different in departments. Obviously, the more the proposals are, the more significant the time savings are. 6. Conclusions This paper proposed an approach for the practical problem of proposal grouping. In this approach, two technologies, knowledge rule and genetic algorithm, are utilized to solve the different problems happened in the grouping procedure. A system to support the proposed approach is designed and developed. It is expected to be incorporated into the ISIS in the future. The proposed approach could be extended to other government funding agencies that have to deal with the same problem of proposal grouping in NSFC. The proposed approach has three major benefits. Firstly, many time-consuming chores which must be performed manually by managers have been reduced considerably. It will results in significant time savings. Secondly, an improved diversity of group composition under multiple attributes as well as similar group size between groups is being achieved. In addition, an obvious point to note is that, compared with the manual way, the resulting grouping is no longer mainly determined by the managers who must perform the grouping, and those who perform the grouping are unlikely to have much influence over future policy in this respect. In summary, the proposed approach provides a powerful support to the real-world managers to conduct the proposal grouping effectively and efficiently. In the future, the number of proposals will keep on increasing, efforts concerning the best way to organize the whole sponsorship process for research projects is certain to continue. Based on the proposal grouping, our future study will focus on the other procedures in the process.
References Azadivar, F., & Tompkins, G. (1999). Simulation optimization with qualitative variables and structural model changes: A genetic algorithm approach. European Journal of Operational Research, 113, 169–182. Baker, B. M., & Benn, C. (2001). Assigning pupils to tutor groups in a comprehensive school. Journal of the Operational Research Society, 52, 623–629. Beheshtian-Adekani, M., & Mahmood, M. A. (1986). Development and validation of a tool for assigning students to groups for class projects. Decision Sciences, 17, 92–113. Buchanan, B. G., & Schortliffe, E. H. (1984). Rule-based expert system. Reading, MA: Addison Wesley. Chen, Y. W., Wang, C. H., & Lin, S. J. (2008). A multi-objective geographic information system for route selection of nuclear waste transport. Omega, 36, 363–372. Chu, A. T. W., Kalaba, R. E., et al. (1979). A comparison of two methods for determining the weights of belonging to fuzzy sets. Journal of Optimization Theory and Application, 27, 531–538. Cook, W. D., Golany, B., Kress, M., et al. (2005). Optimal allocation of proposals to reviewers to facilitate effective ranking. Management Science, 51(4), 655–661. Dhar, V., & Ranganathan, N. (1990). Integer programming vs. expert systems: An experimental comparison. Communications of the ACM, 33(3), 323–336. Gen, M., & Cheng, R. (2000). Genetic algorithms and engineering optimization. New York: John Wiley & Sons, Inc. Henriksen, A. D., & Traynor, A. J. (1999). A practical R&D projectselection scoring tool. IEEE Transactions on Engineering Management, 46(2), 158–170. Hochbaum, D. S., & Levin, A. (2006). Methodologies and algorithms for group-ranking decision. Management Science, 52(9), 1394–1408. Holland, J. H. (1975). Adaptation in natural and artificial systems. Michigan University Press. Hwang, C. L., & Yoon, K. (1981). Lecture notes in economics and mathematical systems: Multiple attribute decision making, methods and application, a state of art survey. New York: Springer. Jones, D. R., & Beltramo, M. A. (1991). Solving partitioning problems with genetic algorithms. In: Proceedings of the fourth international conference on genetic algorithms (pp. 442–449). San Mateo. Ma, J., Fan, Z. P., et al. (1999). A subjective and objective integrated approach to determine attribute weights. European Journal of Operational Research, 112, 397–404. Michalewicz, Z. (1996). Genetic algorithms + data structures = evolution programs. New York, Berlin, Heidelberg: Springer-Verlag. Mu¨hlenbein, H., Gorges-Schleuter, M., & Kra¨mer, O. (1988). Evolution algorithm in combinatorial optimization. Parallel Computing, 7, 65–85. Peachter, B., Luchian, H., & Petruic, M. (1994). Two solutions to the general timetable problem using evolutionary methods. In: Proceedings of the first IEEE international conference on evolutionary computation (pp. 300–305). Piscataway. Reeves, G. R., & Hickman, E. P. (1992). Assigning MBA students to field study project teams: A multicriteria approach. Interfaces, 22(5), 52–58.
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013 Saaty, T. L. (1980). The analytic hierarchy process. New York: McgrawHill. Saaty, T. L., & Hu, G. (1998). Ranking by eigenvector versus other methods in the analytic hierarchy process. Applied Mathematics Letters, 11(4), 121–125. Saaty, T. L., & Vargas, L. G. (1991). Prediction, projection and forecasting. Dordrecht: Kluwer Academic Publishers. Saber, H. M., & Ghosh, J. B. (2001). Assigning students to academic majors. Omega, 29, 513–523. Steuer, R. E. (1992). Manual for the ADBASE multiple objective linear programming package. Athens: University of Georgia. Steuer, R. E., & Choo, E. U. (1983). An interactive weighted Tchebycheff procedure for multiple objective programming. Mathematical Programming, 26, 326–344. Tian, Q. J., Ma, J., & Liu, O. (2002). A hybrid knowledge and model system for R&D project selection. Expert Systems with Applications, 23(3), 265–271.
1013
Turban, E., & Aronson, J. E. (1998). Decision support systems and intelligent systems (5th ed.). Upper Saddle River, New York: PrenticeHall. Vassilew, V., & Narula, S. C. (1993). A reference direction algorithm for solving multiple objective integer linear programming problems. Journal of Operational Research Society, 44(12), 1201–1209. Weitz, R. R., & Jelassi, M. T. (1992). Assigning students to groups: A multi-criteria decision support system approach. Decision Sciences, 23, 746–757. Weitz, R. R., & Lakshminarayanan, S. (1998). An empirical comparison of heuristic methods for creating maximally diverse groups. Journal of the Operational Research Society, 52, 623–629. Yu, P. L. (1985). Multiple criteria decision making: Concepts techniques, and extensions. New York: Plenum. Zadeh, L. (1987). A theory of approximate reasoning. In R. R. Yager, S. Ovchinikov, R. M. Tong, & H. T. Nguyen (Eds.), Fuzzy sets and applications. New York: John Wiley & Sons.
Expert Systems with Applications Expert Systems with Applications 36 (2009) 1004–1013 www.elsevier.com/locate/eswa
Decision support for proposal grouping: A hybrid approach using knowledge rule and genetic algorithm Zhi-Ping Fan a, Yuan Chen a,b,*, Jian Ma b, Yan Zhu b a
Department of Management Science and Engineering, School of Business Administration, Northeastern University, Shenyang 110004, China b Department of Information Systems, City University of Hong Kong, Kowloon, Hong Kong
Abstract Proposal grouping is a special procedure in the sponsorship process for research projects. In practice, it is conducted to simplify the following procedure of reviewer assignment. As the proposals grow, this procedure becomes complex. Practical managers spend an increasing amount of time struggling for identifying valid proposals, classifying proposals and partitioning proposals into groups as well as maintaining some control over the quality and composition of the resulting groups. This paper proposes an approach for proposal grouping, in which knowledge rules are designed to deal with proposal identification and proposal classification, and the genetic algorithm is developed to search for the expected groupings. In addition, a corresponding system is designed and developed to support the proposed approach. Compared to the previous manual grouping, the proposed approach significantly reduces the time required for grouping, ensures more diverse group composition, and increases overall grouping quality. Ó 2007 Elsevier Ltd. All rights reserved. Keywords: Project management; Knowledge rule; Genetic algorithm; Decision support system
1. Introduction A large portion of current academic research is sponsored through various agencies and funds with specific interests in different areas of research. Typically, the sponsorship process begins with a call for proposals (CFP), which is distributed to the relevant communities, such as universities and research institutions. Proposals are then submitted to the body (e.g., funding agencies) that issued the CFP. These proposals are assigned to experts for the peer review. Experts normally review the proposals according to the instructions on the rules and criteria of the funding agency. The review results are collected, and ranked based on the aggregation methods (Cook, Golany, & Kress, 2005). * Corresponding author. Address: Department of Management Science and Engineering, School of Business Administration, Northeastern University, Shenyang 110004, China. Tel.: +86 852 3442 6205. E-mail address: [email protected] (Y. Chen).
0957-4174/$ - see front matter Ó 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2007.11.011
The sponsorship process in China is special for its additional procedure: Proposal grouping. Before peer review, proposals in similar research disciplines are firstly partitioned into groups, and then proposal groups are assigned to reviewers. The goal of proposal grouping is to reduce the assigning times. Undoubtedly, by proposal grouping, the times of assigning proposal groups are less than that of assigning proposals. In most cases, the managers (decision makers) need to take some considerations about the group composition. For example, the proposals in one group should belong to different affiliations. For years, the procedure of proposal grouping has been conducted mainly by hand. In recent years, with the significant increase of proposals, such manual grouping is time consuming and tedious. Moreover, it is very difficult for the managers to regulate the group structure in the manual way. In year 2007, NSFC received more than 70,000 proposals. It can imagine that grouping so many proposals by hand is a so challenging work. In such circumstances, an effective grouping approach is highly desired.
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
In the past four decades, topics related to the research and development project selection have been widely discussed. A variety of decision models and methods have been developed to support the project selection (Henriksen & Traynor, 1999; Hochbaum & Levin, 2006; Tian, Ma, & Liu, 2002). They refer to many aspects of the selection process from different perspectives. Unfortunately, the problem of proposal grouping, to our knowledge, has not been stressed until now in this research area. Therefore, we seek to some parallel problems in other research areas. According to our literature review, the most similar problem could be found in the area of student group formulation. This problem can be described as partitioning a given number of students into groups that are diverse in their attributes. To solve this problem, many heuristic procedures are proposed (Beheshtian-Adekani & Mahmood, 1986; Weitz & Jelassi, 1992; Weitz & Lakshminarayanan, 1998). A bibliography of applications of heuristic procedures is provided in Weitz and Lakshminarayanan (1998). At the same time, a variety of linear and integer models are also developed for this problem (Baker & Benn, 2001; Reeves & Hickman, 1992; Saber & Ghosh, 2001). Most of these models are solved utilizing traditional optimal technologies. The above methods for the student grouping problem illuminate our research greatly. However, both the heuristic procedures and the mathematical models are failed to solve our problem directly for the following three reasons. Firstly, the heuristic procedure may provide a poor solution, or in fact, no solution at all. For example, Weitz and Lakshminarayanan apply the heuristic to a decision support system and they find that the grouping procedure may be blocked or only get the partial solutions (Weitz & Jelassi, 1992). Since our approach will be potentially applied to seven departments and more than hundreds of research disciplines, it requires high effectiveness for the grouping approach. Furthermore, given the integer nature of the mathematical models, there is no guarantee of arriving at a solution within a reasonable time limit especially for those large size problems (Dhar & Ranganathan, 1990). To the problem of proposal grouping, a distinct character is its huge number of proposals. So, the traditional optimal technologies can hardly guarantee the expected solutions. In addition, most of these models require rigorous assumptions and only can deal with wellstructured problems. The procedure of proposal grouping usually involves not only well-structured problems, but also semi-structured or ill-structured problems. For example, identifying valid proposals is a key step in this procedure, but it cannot be solved well if we only depend on the models. So, this paper attempts to propose an approach for proposal grouping that can make up the drawbacks of the above methods. This approach has the following two features. Firstly, knowledge rules are designed for the semi-structured or ill-structured problems in the grouping procedure like proposal identification and proposal classi-
1005
fication. On the other hand, genetic algorithm (GA) is developed to search for the expected groupings in the rational time. The knowledge rules and genetic algorithm complement each other and provide powerful support to the grouping procedure. It must be mentioned that the original motivation of our study is to support the real-world managers conduct the proposal grouping efficiently and effectively. So, a system is developed to support the proposed approach. The approach and the system are proposed to fit the grouping procedure of Natural Science Foundation of China (NSFC). However, it can be easily modified to cover other similar situations. The paper is organized as follows. Section 2 presents the proposal grouping problem that triggered our research interest. Section 3 proposes an approach for proposal grouping. Section 4 designs a decision support system to support the approach proposed in Section 3. Section 5 validates the proposed approach by a real life case from NSFC, and some conclusions are drawn in Section 6. 2. Problem presentation 2.1. Background information of NSFC In China, there are more than 30 government funding agencies. They are very similar in the project selection. NSFC (http://www.nsfc.gov.cn) is the largest and most reputable one. It provides financial support for the basic research projects that have great potential of scientific breakthrough or social impacts. In NSFC, there are four bureaus, one general office, three associated units and seven scientific departments. The bureaus, general office and associated units are mainly responsible for policy making, administration and other related affairs, while the scientific departments are responsible for the selection and management of the projects. In each department, there are several division managers who implement the proposal grouping and reviewer assignment. NSFC is supported by the Chinese central government, and its annual budget has been continuously increasing. For example, the budget for General Program has been increased from not more than 800 million (Chinese Yuan) in 2001 to over 2685 million in 2006. In the meanwhile, the number of proposals has been dramatically increasing from 23,636 in 2001 to 58,811 in 2006 (see Fig. 1). However, the average funded rate in recent six years is only about 18%. So, fair and rational selection process has been pursued by NSFC. 2.2. Proposal grouping in NSFC In NSFC, the procedure of proposal grouping involves several steps. Firstly, division managers must check whether the proposals accord with the application requirements. Then, they need to classify the proposals into different research disciplines according to the keywords provided in the proposal. After that, proposals with similar research
1006
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
Fig. 1. Statistical data about proposals and funded projects from year 2001 to 2006 [source: http://www.nsfc.gov.cn].
group
group
group
group Proposal Proposal Proposal
group
group
Poor group composition
group
group
Proposal
Expected group composition
Fig. 2. The examples of poor group composition and expected group composition.
disciplines would be partitioned into groups. For many years, this procedure has been conducted mainly by hand. In most cases, division managers have to struggle for accomplishing all the grouping steps as soon as possible. Nevertheless, they always spend more than one week on this procedure. In addition, as described above, they can hardly control the group composition. Given such situation, some considerations should be taken into account for the desired grouping approach. First of all, the efficiency of the whole grouping procedure should be improved. Since the procedure includes multiple steps, every step would influence the efficiency of the grouping. For example, manual classification of proposals takes managers much time. So, the proposed approach is supposed to involve every grouping step. Also, the group size in each group should be similar. The group size will influence the reviewer’s workload since the proposal group will be packed and assigned to reviewers in the following procedure. In most cases, evaluating proposals is a volunteer and unpaid work for reviewers. Therefore, equivalent workload is expected by reviewers. Obviously, it could be achieved by guaranteeing the similar group size between groups. Further, the group composition should be diverse. After proposal grouping, proposals in one group will be assigned to the same reviewers. The judgments of reviewers will be
influenced by the structure of the group. In the past, reviewers usually received the proposal group in which some proposals belong to the same affiliation. Reviewers always feel confused to evaluate such proposals. So, similar to student grouping problem, the expected group is that proposals in each group should be different as possible considering their attributes. Fig. 2 displays the examples of poor group composition and expected group composition. Other attributes related with proposal’s applicant are gender, professional title, applicant experience, etc. However, which attribute to be considered is usually determined according to the department’s management policy. In addition, the approach should be flexible. The proposed approach is expected to be applied in multiple departments of NSFC or other funding agencies, which may have different situations and requirements. Also, some conflicts would inevitably happen in the grouping procedure. So, leaving the space for manual cooperation will make the approach flexible and extensible. 3. The proposed approach Based on the manual grouping procedure described in Section 2, this section will present the proposed grouping approach. It is composed of five steps, which are shown in Fig. 3. An explanation of each step follows:
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
Step 1 Identifying valid proposals
Step 2 Classifying proposals
Knowledge rules
Knowledge rules
Step 5 Adjusting grouping results Manual
1007
Step 3 Determining attribute weight AHP
Step 4 Partitioning proposals into groups Genetic algorithm
Fig. 3. The proposed grouping procedure.
Step 1. Identifying valid proposals Firstly, valid proposals should be identified. In NSFC, there are many kinds of programs like General Program, Distinguished Young Scholar Program, etc. Each program has some distinct requirements to applicants. For example, in the Distinguished Young Scholar Program, the age of applicants must be not more than 35 years old. The proposals that satisfy the requirements can be turned into the following step of proposal grouping. In practice, the division manager has to check hundreds of proposals per week to see if they meet the basic requirements of NSFC. In this paper, knowledge rule is used to help the manager identify the proposal validation. It is a good tool to deal with such unstructured or semi-structured problem (Turban & Aronson, 1998). Using knowledge rules, the proposals for each kind of program can be easily categorized to invalid or valid. As for the invalid proposals, the division manager can turn down the proposals by filling in the NSFC Proposal Evaluation Form and send it to the department manager for approval. The valid proposals will be prepared for the next step after the confirmation of the division manager. Sample rules for identifying valid proposals are shown in Table 1. Step 2. Classifying proposals After the above step of identifying proposal validation, the next step is to classify proposals. Recall that proposals in similar research disciplines could be partitioned into groups. In NSFC, each proposal is required to provide three keywords to indicate the research domain it belongs to. NSFC maintains a dictionary of keywords that forms a tree structure. The closer a keyword node is to the root, the larger research domain it represents. For example, keyword ‘A010103’ stands for ‘Geometry’ and ‘A01010302’ for ‘Algebraic Geometry’. Similar to identification of proposal validation, managers also spend much time on proposal classification. Aside from the low efficiency, due to the limitation of human
Table 1 Sample rules for identification of valid proposals 1. Rules for general program IF Number of On-going Projects + Number of Submitted Proposals > 2 THEN Status = Invalid IF Unregistered (PI) or Unregistered (affiliated organization) THEN Status = Invalid IF Position of PI = Full-Time Graduate Student OR Position of PI = Part-Time Graduate Student OR Position of PI = Retired THEN Status = Invalid IF PI In Bad Reputation List Record THEN Status = Invalid 2. Rules for Distinguished Young Scholar Program IF Proposal Category = Distinguished Young Scholar AND Age > 35 THEN Status = Invalid IF Proposal Category = Distinguished Young Scholar Program AND Numbers of On-going Distinguished Young Scholar Program > 0 THEN Status = Invalid IF NOT (Highest Degree = PhD) AND NOT (Position = Senior) AND (No. of Ref. Letters < 2) THEN Status = Invalid
capability, the division manager can hardly identify the multiple disciplines of each proposal correctly. Consequently, knowledge rules are also designed to help the managers classify proposals. Sample rules are shown in Table 2. Step 3. Determining attribute weight Then, the attribute weight should be determined in step 3. As introduced in Section 2, the grouping with diverse group composition is expected. It is defined as the sum of differences between every pair of proposals in this group (Weitz & Lakshminarayanan, 1998). The difference between two proposals could be obtained by adding the weighted contributions of all attributes by which the two
1008
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
Table 2 Sample rules for classification of proposals For q = 1 to Q For i = 1 to m If pi R {categorized proposals} Then For j = 1 to Qi If Rij ¼ Rq Then pi will be added to Sq End loop End if Next If j = Qi Then pi will be added to set {categorized proposals} End if End if Next Next Q number of research disciplines Rq the qth discipline defined by NSFC for q = 1, 2, . . . , Q m number of proposals pi the ith proposal for i = 1, 2, . . . , M Qi number of research disciplines of proposal pi for i = 1, 2, . . . , M Rij the jth research discipline of proposal pi for i = 1, 2, . . . , M and j = 1, 2, . . . , Qi Sq Set of proposals for the qth research discipline
proposals differ. Mathematically, the difference of two proposals could be formulated as X wk jaki akj j; ð1Þ d ij ¼ k
where dij is the difference between proposal i and j; k is the index of the attribute, and k P 1; aki is a binary indicator, which takes the value of 1 if proposal i exhibits the attribute k, otherwise it take the value of 0 and wk is the weighting factor for attribute k. From Eq. (1), we can deduce that the difference of two proposals is determined by their attribute value and the attribute weight. As described in Section 2.2, in practice, many attributes tied up with each proposal are supposed to be considered in proposal grouping. However, what attributes should be chosen is a decision making task related to the decision maker’s (DM) objective. In different department, the situation of applicants is different. For example, the proportion of female applicants to male applicants in the department of Earth Sciences is less than that in the department of Management Sciences. So, if the rational proportion of female applicants needs to be stressed, gender (female) will be chosen to measure the difference of proposals. In most cases, at least one attribute will be taken into consideration. On the other hand, weights reflect the relative importance of each attribute, which would influence the grouping result (Weitz & Jelassi, 1992). However, it is an uneasy task for managers to assign numerical scores to the attribute. While they implicitly give weight of each attribute, they may find the explicit formulation of weight difficult. Part of this has to do with the well-known problem of attempting to express qualitative judgments quantitatively (Bucha-
nan & Schortliffe, 1984). There are many approaches to be used for determining the attribute weight (Chu & Kalaba, 1979; Ma & Fan, 1999; Saaty, 1980; Zadeh, 1987). AHP (Saaty, 1980) is a suitable approach to undertaking this task for two reasons: It is user friendly because users may directly input judgment data without the requirement of mathematical knowledge. Relevant inconsistencies in managerial judgments or perceptions are dealt with appropriately (Saaty & Hu, 1998). So, AHP can be implemented to determine the importance of an attribute to another through pair-wise comparison by the managers. In the AHP, the scale of absolute values of 1–9 is used for making the pairwise comparison judgments. The scales are listed and explained in Table 3. Thus, making the pairwise comparison judgments among attributes can determine the weights of different attribute. Step 4. Partitioning proposals into groups In step 4, the valid proposals in similar research discipline will be partitioned into groups through two substeps. In substep 4.1, a mathematical model is built according to the description of expected groups in Section 2.2. Then, in the following substep 4.2, the model is solved by GA considering the character of the model. Step 4.1. Building the model Firstly, decision variables and parameters to be used in the model are defined as follows: xig is the decision variable, if proposal i is allocated to group j then xig = 1; otherwise xig = 0; dij is the difference between proposal i and j as same as defined above; M is the total number of proposals and G is the total number of groups. Then, to get the expected groups described in Section 2.2, a model is built as follows: max
M 1 X G X M X i¼1
min Subject to
g¼1
d ij xig xjg ;
G1 X G X M M X X xig xit ; g¼1 t>g i¼1 i¼1 G X
ð2Þ
j>i
xig ¼ 1 for i ¼ 1; 2; . . . ; M;
ð3Þ ð4Þ
g¼1
xig 2 f0; 1g for i ¼ 1; 2; . . . ; M and g ¼ 1; 2; . . . ; G:
ð5Þ Table 3 Comparison scale (Saaty & Vargas, 1991) Absolute value
Definition
1 3 5 7
Equal importance Moderate importance of one over another Strong or essential importance of one over another Very strong or demonstrated importance of one over another Extreme importance of one over another Reciprocals for inverse comparison
9 Reciprocals
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
In the above model, there are two objectives. Objective function (2) is to maximize diversity of the group composition. Based on the difference between two proposals, diversity in one group could be represented as the sum of difference of each pair of proposals in this group. Objective function (3) is to minimize the difference of group size. That is, the solution with similar group size is preferred. Aside from the objectives, the constraint function (4) is to guarantee that each proposal is allocated to one group. Notice that the two objectives subject to constraint (4) give rises to a challenging nonlinear zero–one program containing M constraints and MG variables. If an application of the model is the grouping of 500 proposals to 20 groups, it would give rise to a formulation with 500 constraints and 10,000 variables. Consequently, the multiobjective, combinatorial, and large size character of the model results in a challenge for the solution. Step 4.2. Solving the model There are many techniques to deal with multiple objectives (Hwang & Yoon, 1981; Steuer, 1992; Steuer & Choo, 1983; Vassilew & Narula, 1993; Yu, 1985), in which the linear weighting method has been widely used since it is very simple (Hwang & Yoon, 1981; Yu, 1985). Multiple objectives are combined to one scalar objective by the sum of weighted objectives. Considering the two objectives in our model are different in their scales, we normalize them using the objective attainment (Chen, Wang, & Lin, 2008). The attainments of the two objectives are defined as follows: uðZ 1 Þ ¼ 1
Z max Z1 1 max Z 1 Z min 1
uðZ 2 Þ ¼ 1
Z 2 Z min 2 ; min Z max Z 2 2
and ð6Þ
where Z1 and Z2 denote the two objectives in the model; u(Z1) and u(Z2) denote the correspondingly attainment of the objective Z1 and Z2. The value of u(Z1) and u(Z2) is between 0 and 1. Notice that the maximum problem of Z1 has been represented as an equal minimum problem. That is, if the objective Z1 achieves its maximum value, the u(Z1) will get the minimum value of 0. So, the single objective combining the two objectives is min uðZ 1 Þ þ uðZ 2 Þ;
ð7Þ
where the weight of each objective is set as the same. Then, GA is applied to solve the single objective model since it has been successfully implemented to find good solutions to a wide variety of combinatorial problems in rational time (Jones & Beltramo, 1991; Mu¨hlenbein, Gorges-Schleuter, & Kra¨mer, 1988; Peachter, Luchian, & Petruic, 1994). Invented by Holland and his associates in the 1960s (Holland, 1975), it searches the solution space in parallel, working with whole sets of feasible solutions, the so-called population, rather than with individual solutions. By iterating between consecutive generations, it tries
1009
to generate better solutions by combining or modifying good solutions, in order to gradually progress towards the optimal solution. The GA solution process developed to solve our model mainly includes representation, selection, genetic operation, and stopping criteria. Each stage of the process is explained as following: (a) Representation. In the existing literatures of GA, there are different schemes to represent a chromosome: an adjacency representation, an ordinal representation, a path representation, a position listing representation, an adjacency listing representation, and group-number encoding (Michalewicz, 1996). To the problem under consideration, group-number encoding is adopted since it was proposed particularly for grouping problem (Jones & Beltramo, 1991). For our problem, each grouping is represented as (y1, y2, . . . , yM), where the ith integer yi 2 {1, 2, . . . , G} indicates the group number is assigned to proposal i. The structure of the chromosome is shown in Fig. 4. For example, y1 = 6 denotes that proposal 1 is allocated to group 6. (b) Selection. The selection stage consists of determining from which chromosome the next population would be generated. For the next generation, GA is inclined to keep the chromosomes that have better fitness. For our problem, the chromosome fitness is measured by the objective function (7). Further, the stochastic universal sampling method as well as the tournament selection technique has been applied to select the best fitness chromosome. (c) Genetic operation. Genetic operation involves crossover and mutation. The goal of crossover is to exchange information between two parent chromosomes in order to produce two new offspring for the next population. The aim of mutation is to increase genetic diversity into the population by introducing random variations into the members of the population. The group-number encoding creates a possibility of applying standard operators. A mutation would replace a single gene yi by a random number from {1, 2, . . . , G}. Crossovers (single point) would always produce a legitimate offspring. However, as pointed out in Jones and Beltramo (1991), an offspring (after mutation or crossover) may contain less than G groups; moreover, an offspring of two parents, both representing the same grouping, may represent a totally different grouping, due to different numbering of groups. Special repair algorithms (rejection method, renumbering the parents) are used to eliminate these problems.
chromosome
1
2
...
M
y1
y2
...
yM
Fig. 4. Representation of the chromosome.
1010
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
(d) Stopping criteria. There are several stopping criteria that are commonly cited in the literatures (Gen & Cheng, 2000). For example, several consecutive generations reach the same solution, the GA search stops. As another example, the search stops when the improvement in solution quality becomes negligible. Allowing for computational efficiency, this paper adopts a maximum number of runs as the stopping criterion as used in Azadivar and Tompkins (1999). Step 5. Adjusting grouping results The essential purpose of the proposed approach is to help the manager to conduct the grouping task quickly and rationally. However, some unpredictable thing could inevitably happen in practice. Also, some managerial policies will be changed in the future. In such situations, it needs manager to manually adjust the grouping results. For example, applicant may ask that his proposal may not be evaluated by some reviewers. Thus, the manager needs to adjust the grouping results manually by changing such proposal to another group. 4. System design and development To support the grouping approach proposed in Section 3, the corresponding system is designed and developed. The underlying technology includes Microsoft Windows 2000 Server, Microsoft SQL Server, and Microsoft Internet Information Server. The system is built using Microsoft Active Server Page. The major components of the system
include the database, the knowledge base, the model base, and user interfaces. (a) Database. There are two major categories of data stored in the database: Human resource data and Proposal data. Human resource data consists of those for Internal Manager and Applicant. Internal Manager consists of Top Manager, Department Manager, and Division Manager. Applicant consists of Individual Applicant and his/her affiliated Organization. Each Individual Applicant should have an affiliated Organization. Individual Applicants submit proposals through their affiliated Organizations to the funding agency. Proposal data is gleaned from the application sheet, in which fields comprising the applicant’s information are applicant ID, name, age, gender, professional title, education, etc. (b) Knowledge base. According to the proposed grouping approach in Section 3, knowledge rules for identifying valid proposals and knowledge rules for classifying proposals are designed. These two categories of knowledge rules are stored in the knowledge base. (c) Model base. The model base includes the GA and Weight adjustor. GA is developed to search for the satisfactory groupings. Weight adjustor is used to adjust the weights provided by decision makers (managers), when the judgment matrices are inconsistent or the grouping focuses change. (d) User interfaces. The user interface consists of a menudriven dialogue through which the users interact with the system. There are two categories of potential users: internal managers and some external experts.
Fig. 5. The page of experts’ judgments on the attribute weight.
Sometimes, some external experts will be invited to determine the attribute weight together with the internal managers. So, the user interfaces must be a webbased interface with sufficient security mechanisms embedded so that external experts can participate in different locations. As an example, the page for determining attribute weight in the prototype system is shown in Fig. 5.
Number of proposals
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013
1011
20 18 16 14 12 10 8 6 4 2 0 0
1
2
3
4
5
6
7
8
9
10
Group
5. Application and evaluation
Number of proposals
20 18 16 14 12 10 8 6 4 2 0
0
1
2
3
4
5
6
7
8
9
10
Group Fig. 7. The group size in each group by the proposed approach.
On the contrary, in our approach, multiple attributes could be easily considered. Also, the maximum diversity in group components under multiple attributes could be achieved in compromising to the minimum differences of group size. For instance, we choose three attributes tied up with proposal like gender (female), professor, experience and assign them the same weight. Specifically, we set 1 – female and 0 – male; 1 – professor and 0 – otherwise; 1 – experienced applicant and 0 – otherwise. Then, we trace the group composition in the manual grouping (see Fig. 8). Under the same attributes, we also test our approach and present the grouping result in Fig. 9. Contrasting Figs. 8 and 9, we can observe that under multiple attributes, the proposed approach is significantly better than the manual approach considering the diverse group composition. In addition, the system promotes the efficiency in proposal grouping. By manual grouping, the manager needs
Number
In order to assist the project selection in NSFC, we developed the Internet-based Science Information System (ISIS, https://isis.nsfc.gov.cn). It has been used for NSFC’s electronic submission and on-line evaluation of proposals, and dissemination of project outputs since May 2000. However, proposal grouping has not been probed and incorporated into ISIS. So, the designed system to support the proposed grouping approach will be complemented to the ISIS. To validate the proposed approach, the system is tested using the simulation data of research domains in different department of NSFC. There are two major criteria for the success of the system. One is that the groupings created through the use of the system should be as good or better than those obtained via the traditional manual grouping. Here the ‘‘good or better” could be measured from two perspectives: whether the group size is similar between groups and whether the group components are diverse in each group. Another criterion is that the use of the system should result in significant time savings. Under such criteria, this paper presents the comparison of the manual grouping and proposed system grouping using the simulation data of the department of M1 in 2006. The department of M1 accepted 4240 proposals in 2006. There are 168 proposals in research domain #K1. These proposals were partitioned into 10 groups. As introduced above, the division manager needs to consider many factors simultaneously when conducting the grouping. Sometimes, he has to neglect the difference of group size between groups to pursue the diversity group composition. It gave rise to the possible significant difference of group size between groups (see Fig. 6). When these proposal groups are assigned to reviewers, the resulting workload of reviewers will not be equal. In practice, such situation is not expected by both reviewers and managers. In contrast, since the minimum difference of group size is one objective in our model, it will be achieved as possible in compromising to the other objective of maximum group diversity. Fig. 7 shows the group size in each group using the proposed approach. From Figs. 6 and 7, we can see that the proposed approach can well guarantee the similar group size between groups. On the other hand, pursuing diverse group composition by hand is more difficult for the division manager. In most cases, the manager only could focus on one or two attributes under the limitations of time and human judgments.
Fig. 6. The group size in each group by manual grouping.
20 18 16 14 12 10 8 6 4 2 0
Female Professor Experienced
1
2
3
4
5
6
7
8
9
10
Group Fig. 8. The resulting group composition by manual grouping under three attributes.
Number
1012
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013 20 18 16 14 12 10 8 6 4 2 0
Acknowledgements Female Professor
This work was partly supported by the National Science Fund for Distinguished Young Scholars of China (Project No. 70525002), National Science Fund for Excellent Innovation Research Group of China (Project No. 70721001), the Competitive Earmarked Research Grant of Hong Kong SAR (CERG, Project No. CityU 118705, CityU 1237/03E) and Strategic Research Grant of City University of Hong Kong (Project No. 7002049).
Experienced
1
2
3
4
5
6
7
8
9
10
Group Fig. 9. The resulting group composition by the proposed approach under three attributes.
to spend at least one week. Using the system, the grouping can be finished within hours. It must be mentioned that the number of proposals are different in departments. For example, there are 25,199 proposals in the department of Life Sciences in 2006, nearly six times more than that in Management Sciences. So, time savings will be different in departments. Obviously, the more the proposals are, the more significant the time savings are. 6. Conclusions This paper proposed an approach for the practical problem of proposal grouping. In this approach, two technologies, knowledge rule and genetic algorithm, are utilized to solve the different problems happened in the grouping procedure. A system to support the proposed approach is designed and developed. It is expected to be incorporated into the ISIS in the future. The proposed approach could be extended to other government funding agencies that have to deal with the same problem of proposal grouping in NSFC. The proposed approach has three major benefits. Firstly, many time-consuming chores which must be performed manually by managers have been reduced considerably. It will results in significant time savings. Secondly, an improved diversity of group composition under multiple attributes as well as similar group size between groups is being achieved. In addition, an obvious point to note is that, compared with the manual way, the resulting grouping is no longer mainly determined by the managers who must perform the grouping, and those who perform the grouping are unlikely to have much influence over future policy in this respect. In summary, the proposed approach provides a powerful support to the real-world managers to conduct the proposal grouping effectively and efficiently. In the future, the number of proposals will keep on increasing, efforts concerning the best way to organize the whole sponsorship process for research projects is certain to continue. Based on the proposal grouping, our future study will focus on the other procedures in the process.
References Azadivar, F., & Tompkins, G. (1999). Simulation optimization with qualitative variables and structural model changes: A genetic algorithm approach. European Journal of Operational Research, 113, 169–182. Baker, B. M., & Benn, C. (2001). Assigning pupils to tutor groups in a comprehensive school. Journal of the Operational Research Society, 52, 623–629. Beheshtian-Adekani, M., & Mahmood, M. A. (1986). Development and validation of a tool for assigning students to groups for class projects. Decision Sciences, 17, 92–113. Buchanan, B. G., & Schortliffe, E. H. (1984). Rule-based expert system. Reading, MA: Addison Wesley. Chen, Y. W., Wang, C. H., & Lin, S. J. (2008). A multi-objective geographic information system for route selection of nuclear waste transport. Omega, 36, 363–372. Chu, A. T. W., Kalaba, R. E., et al. (1979). A comparison of two methods for determining the weights of belonging to fuzzy sets. Journal of Optimization Theory and Application, 27, 531–538. Cook, W. D., Golany, B., Kress, M., et al. (2005). Optimal allocation of proposals to reviewers to facilitate effective ranking. Management Science, 51(4), 655–661. Dhar, V., & Ranganathan, N. (1990). Integer programming vs. expert systems: An experimental comparison. Communications of the ACM, 33(3), 323–336. Gen, M., & Cheng, R. (2000). Genetic algorithms and engineering optimization. New York: John Wiley & Sons, Inc. Henriksen, A. D., & Traynor, A. J. (1999). A practical R&D projectselection scoring tool. IEEE Transactions on Engineering Management, 46(2), 158–170. Hochbaum, D. S., & Levin, A. (2006). Methodologies and algorithms for group-ranking decision. Management Science, 52(9), 1394–1408. Holland, J. H. (1975). Adaptation in natural and artificial systems. Michigan University Press. Hwang, C. L., & Yoon, K. (1981). Lecture notes in economics and mathematical systems: Multiple attribute decision making, methods and application, a state of art survey. New York: Springer. Jones, D. R., & Beltramo, M. A. (1991). Solving partitioning problems with genetic algorithms. In: Proceedings of the fourth international conference on genetic algorithms (pp. 442–449). San Mateo. Ma, J., Fan, Z. P., et al. (1999). A subjective and objective integrated approach to determine attribute weights. European Journal of Operational Research, 112, 397–404. Michalewicz, Z. (1996). Genetic algorithms + data structures = evolution programs. New York, Berlin, Heidelberg: Springer-Verlag. Mu¨hlenbein, H., Gorges-Schleuter, M., & Kra¨mer, O. (1988). Evolution algorithm in combinatorial optimization. Parallel Computing, 7, 65–85. Peachter, B., Luchian, H., & Petruic, M. (1994). Two solutions to the general timetable problem using evolutionary methods. In: Proceedings of the first IEEE international conference on evolutionary computation (pp. 300–305). Piscataway. Reeves, G. R., & Hickman, E. P. (1992). Assigning MBA students to field study project teams: A multicriteria approach. Interfaces, 22(5), 52–58.
Z.-P. Fan et al. / Expert Systems with Applications 36 (2009) 1004–1013 Saaty, T. L. (1980). The analytic hierarchy process. New York: McgrawHill. Saaty, T. L., & Hu, G. (1998). Ranking by eigenvector versus other methods in the analytic hierarchy process. Applied Mathematics Letters, 11(4), 121–125. Saaty, T. L., & Vargas, L. G. (1991). Prediction, projection and forecasting. Dordrecht: Kluwer Academic Publishers. Saber, H. M., & Ghosh, J. B. (2001). Assigning students to academic majors. Omega, 29, 513–523. Steuer, R. E. (1992). Manual for the ADBASE multiple objective linear programming package. Athens: University of Georgia. Steuer, R. E., & Choo, E. U. (1983). An interactive weighted Tchebycheff procedure for multiple objective programming. Mathematical Programming, 26, 326–344. Tian, Q. J., Ma, J., & Liu, O. (2002). A hybrid knowledge and model system for R&D project selection. Expert Systems with Applications, 23(3), 265–271.
1013
Turban, E., & Aronson, J. E. (1998). Decision support systems and intelligent systems (5th ed.). Upper Saddle River, New York: PrenticeHall. Vassilew, V., & Narula, S. C. (1993). A reference direction algorithm for solving multiple objective integer linear programming problems. Journal of Operational Research Society, 44(12), 1201–1209. Weitz, R. R., & Jelassi, M. T. (1992). Assigning students to groups: A multi-criteria decision support system approach. Decision Sciences, 23, 746–757. Weitz, R. R., & Lakshminarayanan, S. (1998). An empirical comparison of heuristic methods for creating maximally diverse groups. Journal of the Operational Research Society, 52, 623–629. Yu, P. L. (1985). Multiple criteria decision making: Concepts techniques, and extensions. New York: Plenum. Zadeh, L. (1987). A theory of approximate reasoning. In R. R. Yager, S. Ovchinikov, R. M. Tong, & H. T. Nguyen (Eds.), Fuzzy sets and applications. New York: John Wiley & Sons.