Service-oriented network performance evaluation framework based on LA-FAHP

The Journal of China Universities of Posts and Telecommunications June 2015, 22(3): 74–83 www.sciencedirect.com/science/journal/10058885

http://jcupt.xsw.bupt.cn

Service-oriented network performance evaluation framework based on LA-FAHP Zhao Xing (

), Lu Zhaoming, Wang Luhan, Wen Xiangming, Lei Tao

School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

Abstract Since the different characteristics of various network services determine that their requirements for network are also disparate, the performance of one network varies according to the services running on it. However, most of previous network performance evaluation (NPE) researches conduct evaluations based on the network parameters, but without considering from the perspective of specific service running on the network. In view of this issue, a novel service-oriented NPE framework is proposed. First, the characteristics discrepancy among different types of services are investigated. Next, in order to conduct comprehensive evaluation of multiple services, an enhanced low-complexity adaptive (LA)-fuzzy analytical hierarchy process (FAHP) is introduced; meanwhile by applying the experts-construct-directly (ECD) algorithm proposed later, the consistency check required in previous studies can be omitted, thereby significantly reducing the computation complexity and assessment workload for experts. Then, in accordance with the features of each service, corrections are made to their respective membership functions, thus making the proposed LA-FAHP adaptive to various service evaluation scenarios. The subsequent comparison with other NPE methods well proves the effectiveness and high sensitivity of proposed framework, and the analysis verifies the low computation complexity of the proposed algorithms as well. Keywords

network performance evaluation, low-complexity adaptive- fuzzy analytical hierarchy process (LA- FAHP), service-oriented

1 Introduction The explosive emergence of wireless applications has brought enormous various wireless traffic demands [1]. The addition of diverse types of services traffic not only influence the performance of wireless network, but also cause great challenges for network management and flow distribution. Different services require disparate quality guarantee levels, therefore how to distribute the service traffic properly according to the situation of current network so as to optimize the network performance on the basis of existing network architecture has become an urgent problem. NPE could obtain accurate understanding of network’s current performance by collecting and analyzing network parameters, thus it becomes an important means to address these problems. In addition, Received date: 20-01-2015 Corresponding author: Zhao Xing, E-mail: [email protected] DOI: 10.1016/S1005-8885(15)60655-0

most of today’s terminal devices have multiple network interfaces and can access to multiple networks. Conducting NPE can also help network users choose the optimal network to access to depending on the service running on this terminal. In order to evaluate network performance objectively, it’s necessary to select metrics scientifically. The IP network performance indexes determined by Internet Engineering Task Force (IETF) include: Connectivity, throughput, bandwidth, delay, delay variation and packet loss rate (PLR), etc. [2]. But network performance relates to network topology, traffic volume and business types closely, performance evaluation should not only consider the parameters above, but also consider the network requirements of specific service. Different services have different characteristics, for instance, video service and online gaming are very sensitive to time delay. While another traditional service, best effort (BE) service including Web-surfing, Email, FTP and file-sharing, in

Issue 3

Zhao Xing, et al. / Service-oriented network performance evaluation framework based on…

which BE means that the service makes no guarantees regarding the speed with which data will be transmitted to the recipient or that the data will even be delivered entirely, is delay tolerant conversely. Thereby they have different requirements for network, so the performance of one network carrying these services are also discrepant. A network is very likely to perform badly when running real-time video service but maybe counted as good conversely when opening a Web page or sending an Email. That is to say, whether the performance of a network is good or not should be evaluated from the perspective of the specific service running on it. However, previous researches [3–4] mostly only consider the network parameters and lack the association of network evaluation with specific service type, which is not practical in real network situation. As for evaluation models and methods, at present, the most frequently adopted evaluation models and methods in network evaluation field include: Linear weighted model, planar assessment model, analytic hierarchy process (AHP) [5], fuzzy analysis model, data envelopment analysis (DEA), grey correlation evaluation model and Delphi model. In comparison, AHP has characteristics of systematism, flexibility, practicability, etc., which is especially suitable for decision-making and evaluation of multi-objective, multi-level, multi-factor and complex system and is an ideal method to evaluate the overall efficiency and gain of communication network [6]. But traditional methods of AHP can be of no use when uncertainty in data of problems is observed. However, people’s judgments on complex problems are actually full of ambiguities. To deal with this fuzziness, FAHP was proposed which combines fuzzy theory [7] with AHP to take the fuzziness of human judgments into account. Van Laarhoven et al. proposed the first method of implementing FAHP in 1993 [8] in which triangular fuzzy numbers (TFNs) were compared according to their membership functions. For a long time, there is no clear definition about FAHP, many researchers have proposed varieties of improved algorithms to apply into different fields [9–13]. Among them, Chang [9] introduces TFN to construct comparison matrix and significantly reduces the computation complexity, so that it’s widely accepted. However, the consistency check for the comparison matrix existing in previous researches increases the computation complexity [13]. And their membership functions are usually same for all the attributes [4], which cannot

75

reflects the different influence degree of one network index on different services thereby cannot be directly applied in this paper. Therefore, according to the motivations analyzed above, a novel service-oriented NPE framework is proposed in this paper. And an enhanced LA-FAHP method is further introduced in the process of comprehensive evaluation. To summarize, the contributions of this paper are: 1) A novel service-oriented NPE framework is proposed, which evaluates the network performance from the perspective of service innovatively. 2) A novel LA-FAHP method is proposed, in which the consistency of fuzzy comparison matrix is satisfied at the beginning of destruction by introducing an ECD algorithm. Therefore, it gets rid of the consistency check existing in previous FAHP methods, which significantly reduces the computation complexity and assessment workload. 3) Considering the different characteristics of various services, this paper adjusts the comparison matrix of network parameters set accordingly, which makes the NPE result of single service more accurate and targeted. 4) The membership functions of network parameters adopted in this paper when calculating their scores are modified adaptively according to the feature of each service, which further distinguishes the influence degree of one same network parameter on different services. Therefore, by modifying the membership functions accordingly, the proposed LA-FAHP can be applied in the evaluation of any other services, thereby making it ‘adaptive’ in multiple service application scenarios. The rest of this paper is organized as follows. Sect. 2 gives the overview of proposed service-oriented NPE framework from confirming attributes to obtaining final results. Sect. 3 illustrates the specific procedure of service-oriented evaluation method adopted in this paper. Comparison with other methods and computation complexity analysis are given in Sect. 4. The conclusion of this paper and future work are then presented in the final section.

2 Overview of service-oriented NPE framework If all the services in one network are video services, then the network performance state in the condition of pure video services can be derived through a normal NPE process. And if all the services in the network are BE service, the network performance score in the condition of pure BE service can be derived too. In the same manner

76

The Journal of China Universities of Posts and Telecommunications

the NPE score of other single service scenarios can be obtained either. Since a real network is mixed with various types of services, the performance state of one network supposes to be the synthesis of that in single service scenarios. Therefore, through aggregating the NPE results of each service by a certain weight, the comprehensive

Fig. 1

2015

performance of one network can be derived thereby achieving the goal of service-oriented NPE. According to this line of reasoning, the architecture of service-oriented NPE framework is illustrated in Fig. 1, which consists of three parts.

The service-oriented NPE process based on FAHP

1) Preprocessing The first part as the basic of succeeding researches is to analyze the problem before solving it, and the primary objective is to confirm appropriate evaluation attributes and establish an analytical hierarchical model. With the overall network performance as goal, this paper treats each type of service as main-attribute; network parameters such as delay, delay jitter (DJ) as sub-attributes. Each main-attribute has corresponding set of sub-attributes. The analytical hierarchical model will be established after confirming attributes, concept of which will be further illustrated in Sect. 3. 2) Weight acquirement As the core part, the objective of this part is to derive the weight of each main-attribute and sub-attribute. In view of the ambiguities in deciding which attribute is more significant, LA-FAHP is proposed in Sect. 3. 3) Aggregative calculation After obtaining the weight of each attribute, the overall NPE result can be calculated through aggregative calculation, which can be divided into following steps. a) Deriving original data of network parameters: Based on the grading rules set in advance, the score of each sub-attribute can be achieved. It’s worth noting that modifying grading rules in accordance with each service is done in this paper to further distinguish the influence degree of one same network parameter on different services. b) Deriving score of each sub-attribute: Original data of network parameters such as delay can be derived from network simulation or actual measurement. In order to

facilitate comparison with other NPE methods, this paper takes data provided by Ref. [3] as standard samples. c) Obtaining NPE result of each service: Through combining the weight and score obtained earlier of each sub-attribute, the respective NPE result of each service can be derived. d) Getting overall NPE result: By combining the weight and score of each service in the same way, the overall NPE result can be derived thereby completing the integrated NPE process.

3 Service-oriented NPE based on LA-FAHP Among the various evaluation methods, FAHP introduces fuzzy consistent matrix to address the fuzziness of complexity judgment, which is especially suitable for multi-attribute decision-making with fuzziness suchlike NPE problem in this paper. But previous FAHP methods applied in evaluation usually need to conduct consistency check after constructing the fuzzy comparison matrices [13]. And if the consistency radio (CR) [14] is not of acceptable value, the matrices must be modified by the assessors until CR is under the threshold, which not only increases the computation complexity but also brings additional work to the assessors. Therefore, this paper proposes a novel low-computation adaptive FAHP — LA-FAHP method improved on the basis of Chang’s classic FAHP method [9], which has a relatively low computation complexity through omitting the consistency check. Furthermore in order to be adaptive in the evaluation of

Issue 3

Zhao Xing, et al. / Service-oriented network performance evaluation framework based on…

multiple services with different characteristics, the proposed LA-FAHP modifies the membership functions of each attribute according to the different features of each service in the process of aggregation; which makes LA-FAHP further distinguish the influence degree of one same network parameter on different services. Hence the proposed LA-FAHP method is called a low-computation adaptive FAHP. The detailed procedures of comprehensive service-oriented NPE based on the proposed LA-FAHP are illustrated below. 3.1

Establishing the hierarchy model of evaluation

Same as traditional AHP, the first step of LA-FAHP is to classify the designed factors according to the problem, find out the relationship between each other and establish the hierarchy structure, which generally can be divided into three layers. 1) Top layer: Also known as the goal layer, usually includes only one element which is the intended target or ideal result of the problem analyzed. 2) Middle layer: Also known as the criterion layer, includes middle stages. 3) Bottom layer: Also known as solution layer, includes the various kinds of measures, plans and indicators for the realization of the target. The final goal in this paper is to derive the overall network performance score, so the overall network performance as the evaluation object should be in the top layer. The comprehensive NPE result is determined by the evaluation result of each single service, therefore various services are in the second layer as main-attributes for the goal. And each of them corresponds to a set of network parameters needed during evaluation, such as delay, DJ, which are included in the bottom layer as sub-attributes for the object. And according to the data released by Cisco [1], the proportions of video service and BE service in global consumer internet traffic are 60.04% and 39.87% respectively in 2013. The sum of their proportions is 99.91%, which means the current network is mainly consisted of these two kinds of services and the performance of one network is mainly determined by these two services either. Therefore in this paper they will be mainly taken into consideration in the process of evaluation. Delay, DJ, and PLR are chosen as necessary

77

network metrics during evaluation. So in the three-layer hierarchy evaluation model established in Fig. 2, only video service and BE service are illustrated.

Fig. 2

The hierarchy model of service-oriented NPE framework

However, it’s worth noting that, the proposed evaluation framework is not limited to be applied in these two scenarios. Real network traffic is full of kinds of services. When the traffic consists of more than the two types of services mentioned above, the proposed framework is also working. Actually, the evaluation results of other services can be easily get in the same way as video service and BE service. 3.2

Comprehensive evaluation

The hierarchy model helps find out the relationships between overall network performance, services and network parameters. The following major challenge is to determine the relative importance degree of mainattributes and sub-attributes, which is usually in the form of pairwise comparison matrix determined by experienced experts as shown below. In which n is the number of attributes need to be compared and aij shows the relative importance of criterion i (ci ) in comparison with criterion

j (c j ) .

 a11 a Α =  21  ⋮   an1

a12 ⋯ a1n  a22 ⋯ a2 n  (1) ⋮ ⋮   an 2 ⋯ ann  n× n When establishing the value of aij , assessors can

provide a precise numerical value, a linguistic term, or a fuzzy number. They are encouraged to give fuzzy scales when they are not sure about the exact numerical values [13]. For the purpose of delivering the ambiguity in human decision-making, this paper adopts TFN to construct fuzzy pairwise comparison matrices. The definitions of TFN as well as fuzzy matrixes are given as below:

78

The Journal of China Universities of Posts and Telecommunications

Definition 1 If a fuzzy number denoted by M = (l , m, u ) has the membership function as below, then M is called TFN which can be represented by the triplet (l , m, u ) .

 x−l  m − l ; x ∈ [l , m]  f M ( x) =  x − u (2)  m − u ; x ∈ [m, u ]  otherwise  0; where l , m, u ∈ R; l≤m≤u. Let M 1 = (l1 , m1 , u1 ) , M 2 = (l2 , m2 , u2 ) be two TFNs and λ be a constant (λ > 0) , then: M 1 + M 2 = (l1 + l2 , m1 + m2 , u1 + u2 )   M 1 − M 2 = (l1 − u2 , m1 − m2 , u1 − l2 )  M 1 ± λ = (l1 ± λ , m1 ± λ , u1 ± λ )   λ − M 1 = (λ − u1 , λ − m1 , λ − l1 )  Definition 2 If a matrix criterion that

(3)

A = (aij ) n× n meets the

A

is

claimed as a fuzzy matrix. Definition 3 If a fuzzy matrix A = (aij ) n× n meets the criterion that rij + rji = 1 (i, j = 1, 2,… , n) , then

A is

called fuzzy complementary matrix. Definition 4 If a fuzzy complementary matrix A = (aij ) n× n meets ∀i, j , k = 1, 2,… , n; rij = rik − rjk + 0.5 , then A is called fuzzy consistent matrix. In order to make the fuzzy comparison matrix meet the consistency demand, an ECD algorithm is proposed according to the definitions of fuzzy consistent matrix given above, by which a fuzzy consistent comparison matrix can be directly constructed by the experts. Let s be the number of assessment experts, A = (aij ) n× n be the matrix that needs to be determined by s experts, the scale of grading value adopted is 0.1~0.9 scaling as shown in Table 1. The detailed steps of proposed ECD algorithm are below. Table 1

Scale for fuzzy consistent comparison matrix Scale values 0.5 0.6 0.7 0.8 0.9

(0, 0, 0) , let s experts choose one element ai1 j1 which they are most confirmed about and give their grading on it. Then value of ai1 j1 is given by ai1 j1 = (li1 j1 , mi1 j1 , ui1 j1 ),

li1 j1 ≤mi1 j1 ≤ui1 j1 , in which:   li1 j1 + ui1 j1  mi1 j1 =  (5) 2   ui1 j1 = max(Vi1 j1k )  1 ≤i1 , j1≤n, 1≤k≤s  Where Vi1 j1k stands for the relative importance of

li1 j1 = min(Vi1 j1k )

criteria ci1 and c j1 given by expert k. If Vi1 j1k > 0.5 , then it means that criterion i1 (ci1 ) is more important than criterion j1 (c j1 ) considered by expert k.

(4)

0≤rij ≤1 (i, j = 1, 2,… , n) , then

2015

Meanings Equally important Slightly important Important Strongly important Extremely important

Algorithm 1 Step 1 For i ≠ j , let aij = (0, 0, 0) ; for i = j , let

aij = (0.5, 0.5, 0.5) . Among all the elements with value of

Then (i1 , j1 ) are regarded as an independent collection

P1 = (i1 , j1 ) , and the value of a j1i1 = (l j1i1 , m j1i1 , u j1i1 ) can be get by: a j1i1 = 1 − ai1 j1

(6)

Step 2 Among the rest elements with value of (0, 0, 0) , let s experts choose another one element ai2 j2 which they are most confirmed about and give their grading on it. The value of ai2 j2 and a j2i2 can be obtained by the same way as ai1 j1 and a j1i1 . If (i2 , j2 ) doesn’t have repetitive number with the first collection (i1 , j1 ) , then it’ll be treated as another new independent collection; else (i2 , j2 ) and (i1 , j1 ) are merged into one new collection. The TFN value of elements whose subscripts are made up of any two numbers in the independent collection (s) can be calculated by Eq. (7), and the original value should be replaced by them accordingly. Then let t = 3 , t ∈ Z . aij = aik − a jk + 0.5 (7) Step 3 Assuming that the element chosen among the rest elements with value of (0, 0, 0) is ait jt . The value of

ait jt and a jt it can be get by the same manner. If (it , jt ) doesn’t have repetitive number with the former collection (s), then it’ll be treated as another new independent collection; if (it , jt ) has repetitive numbers with all the former (t − 1) collections, then merge all the collections including number it and jt into one new collection. At last calculate the value of elements whose subscripts are made up of any two numbers in the independent collection(s) and replace the original values with them.

Issue 3

Zhao Xing, et al. / Service-oriented network performance evaluation framework based on…

Let t = t + 1 . If t < n , then repeat step 3; if t = n , then terminate the computation process. Through using the proposed ECD algorithm, the comparison matrix of services with respect to the goal is constructed by 20 experienced experts firstly in Table 2. In real network the weight of each service should be determined according to its proportion in the current network. That can reflect overall performance status of current network more accurately. But real network is complex and fast-changing, the types as well as proportions of the services existing in one network are also constantly changing. Acquiring the proportion of each type of service needs very high real-time performance, which will cause tremendous computing workload to the NPE system. So in order to make the NPE framework more general to apply to any network as well as reduce the computation complexity, this paper derives fixed weight from the comparison matrix for each service. The value of comparison matrix as shown in Table 2 is assigned based on the proportion of each service in the global consumer internet traffic of 2013 [1]. The more proportion one type of service takes, the more important it is to the overall performance of one network. Thus it can be seen in Table 2, the value of second element in line one is (0.4,0.6, 0.8) , which means video service is more important than BE service. In real practice, Table 2 will be composed of more elements since more services will be considered. Evaluation of main-attributes with respect to the goal

Video Service BE service

Video Service (0.5,0.5,0.5) (0.2,0.4,0.6)

BE service (0.4,0.6,0.8) (0.5,0.5,0.5)

Then the fuzzy comparison matrices of sub-attributes with respect to main-attributes are given in Table 3, 4. Where R11 , R12 , R13 respectively represents delay, DJ and PLR of video service and R21 , R22 , R23 represents those of BE service. Since video service is very sensitive to delay while BE service is delay tolerant but sensitive to PLR; so delay is more important than PLR for video service but less important for BE service, which can also be reflected from the data in Table 3 and Table 4: a11,13 = (0.3, 0.5, 0.8) while a11,13 = (0.3, 0.45, 0.6) . In addition, when evaluating other types of services, it only needs to choose the most important network parameters for this service and construct corresponding fuzzy comparison matrix as done before.

Evaluation of sub-attributes with respect to video service

Table 3

R11 R12 R13

R11

R12

R13

(0.5,0.5,0.5) (0.1,0.25,0.4) (0.2,0.5,0.7)

(0.6,0.75,0.9) (0.5,0.5,0.5) (0.6,0.7,0.8)

(0.3,0.5,0.8) (0.2,0.3,0.4) (0.5,0.5,0.5)

Evaluation of sub-attributes with respect to BE service

Table 4 R21 R22 R23

R21

R22

R23

(0.5,0.5,0.5) (0.2,0.45,0.7) (0.4,0.55,0.7)

(0.3,0.55,0.8) (0.5,0.5,0.5) (0.5,0.6,0.7)

(0.3,0.45,0.6) (0.3,0.4,0.5) (0.5,0.5,0.5)

In the following, the steps of Chang’s extent analysis method (EAM) [9] are given to calculate the weight vector of attributes in Tables 2–4: Algorithm 2 Let X = {x1 , x2 ,..., xn } be an object set, and

G = {g1 , g 2 ,..., g n } be a goal set, each object is taken and extent analysis for each goal gi is performed respectively. Therefore, m extent analysis values for each object can be obtained with M 1 gi , M 2 gi ,..., M m gi (i = 1, 2,..., n) . Step 1 The value of fuzzy synthetic extent with respect to the ith object is defined as: −1

 n m j  Si = ∑ M ⊙  ∑∑ M gi  j =1  i =1 j =1  Where m m m  m  M gji =  ∑ l j , ∑ m j , ∑ u j  ∑ j =1 j =1  j =1 j =1  m

j gi

(8)

n n  n  =  ∑ li , ∑ mi , ∑ ui  i =1 j =1 i =1  i =1 i =1  −1 1 1   1  n m , n  j   n , n  ∑∑ M gi  =   i =1 j =1  ∑ ui ∑ mi ∑ li   i =1 i =1  i =1  Step 2 For TFN M 1 and M 2 , the possibility degree of M 2 = (l2 , m2 , u2 )≥M 1 = (l1 , m1 , u1 ) is: ∨ ( M 2≥M 1 ) = sup[min( f M1 ( x), f M 2 ( x))] = n

Table 2

79

m

∑∑ M

j gi

x≥y

l1 − u2  hgt( M 1 ∩ M 2 ) = ( m − u  2 2 ) − ( m1 − l1 ) 1; if and only if when M ≥M  2 1

(9)

The possibility degree for a convex fuzzy number to be greater than k convex fuzzy numbers M i (i = 1, 2,..., k ) can be defined by: ∨ ( M ≥M 1 , M 2 ,..., M k ) = ∨[( M ≥M 1 )and( M ≥M 2 )and ⋯ (10) and( M ≥M k )] = min ∨ ( M ≥M i ); i = 1, 2,..., k Assume that d ′( A i ) = min ∨ ( Si ≥Sk ), k = 1, 2,..., n; k ≠ i , the weight vector is given by:

80

The Journal of China Universities of Posts and Telecommunications

( 11 ) W ′ = (d ′( A1 ), d ′( A2 ),..., d ′( An ))T Where Ai (i = 1, 2,..., n) are n elements. The normalized weight vector is: W = (d ( A1 ), d ( A2 ),..., d ( An ))T

(12)

Through substituting the data of Tables 2–4 into above Eqs. (8)–(12), the normalized weight of each attribute can be calculated as shown in Table 5. The computational process is no longer illustrated due to the space limitations. Table 5 Goal Overall network performance

The normalized weight of each attribute

Main-attributes

Weight

Video service

0.620 3

BE service

0.379 7

Sub-attributes Delay DJ PLR Delay DJ PLR

Weight 0.453 5 0.171 0 0.375 5 0.322 4 0.279 1 0.398 4

After calculating the weight of each sub-attribute, the NPE score of each service is given by: n

Q = ∑ (Wi Ci )

(13)

i =1

Where Wi i (i = 1, 2,..., n)

is the weight value of

evaluation index and Ci is the evaluation score of index. Here n = 3 because three sub-attributes are adopted in this paper. For calculating Ci , this paper adopted the method of fuzzy mathematics to construct membership function for each index. The membership function of sub-attributes for video service’s condition is given by: 1; x≤ai  b −x λi ( x) =  i ; ai ≤x≤bi (14)  bi − ai 0; x≥b i 

1; x≤ai  2 2  x − 2ai x − bi ; ai ≤x≤bi λi ( x) =  − (ai − bi ) 2  0; x≥b i  1; x≤ai  2  ( x − bi ) ; ai ≤x≤bi λi ( x) =  2  (ai − bi ) 0; x≥b i 

2015

(15)

(16)

In order to elaborate the modifications more intuitively, Fig. 3 illustrates the functions in Eqs. (14)–(16). The first figure of Fig. 3 shows the two services’ membership functions of delay and DJ. The second one shows that of PLR. Where the horizontal axes stand for the values of delay (or DJ and PLR) and the vertical axes stand for the corresponding evaluation scores of this attribute. It can be seen from the first figure that, when x is fixed the score of BE service is higher than that of video service; when x becomes larger, the evaluation score of BE service will decrease in a more gentle way, which is coincident with previous analysis. And because BE service more sensitive to PLR, thus it can be seen in the second figure that, when x becomes larger the score of BE service will decrease more sharply in consequence.

where x is the sample data of certain sub-attribute, ai , bi are the lower boundary and upper boundary of membership function. Considering the features that BE service is not so sensitive to delay and DJ as video service, so the performance degradation of BE service won’t be as quick as video service with the increase of delay and DJ. For this reason this paper introduces quadratic function to modify the membership function of delay and DJ for BE service’s condition as Eq. (15). And since BE service is more sensitive to PLR, so when PLR increases, the performance of BE service will degrade faster than video service. Therefore this paper also modifies the membership of PLR as Eq. (16).

Fig. 3

The comparisons of membership functions

Through substituting the weight and score of each service for Wi , Ci in Eq. (13) respectively, the overall network performance score can be derived eventually. So far the service-oriented NPE is well completed.

4 Results and discussion In order to facilitate comparison with other NPE methods, this paper takes original data from Ref. [3] as the sample data of sub-attributes. According to the boundary values given in Table 6 [3], the NPE scores are calculated by applying Eqs. (13)–(16) as shown in Table 7. The range of scores is (0~1), in which ‘1’ means the network is in a

Issue 3

Zhao Xing, et al. / Service-oriented network performance evaluation framework based on…

81

perfect state and ‘0’ means the network is intolerably bad. Table 6

The boundary value of sub-attributes

Time delay/ (m⋅ s −1 )

DJ/ (m⋅ s −1 )

PLR/%

ai

0

0

0

bi

300

100

10

Table 7 Delay (m⋅ s −1 )

DJ/ (m⋅ s −1 )

65.506 65.483 65.625 65.668 65.646 65.764 66.084 66.614 72.721 74.368 76.668 77.252 77.784 77.882

1.649 1.679 1.865 2.790 2.003 3.879 5.085 6.591 14.862 15.066 11.714 12.498 11.760 12.032

Evaluation scores NPE score Video BE Overall service service score 0 0.898 0.985 0.937 0 0.898 0.985 0.937 0 0.898 0.985 0.937 0 0.896 0.984 0.936 0 0.897 0.985 0.936 0 0.894 0.984 0.934 0 0.891 0.984 0.933 0 0.888 0.983 0.931 3.731 0.725 0.733 0.728 41.199 0.486 0.575 0.526 60.606 0.489 0.577 0.528 57.471 0.486 0.576 0.526 85.603 0.487 0.576 0.527 120.301 0.486 0.576 0.526 PLR/%

Fig. 5

Fig. 6

4.1

Result analysis

To validate the effectiveness of the proposed framework, this paper compares the NPE result with other three methods [3,15–16], as presented in Figs. 4–6. Horizontal axes of Figs. 4–6 show the time period ranges and horizontal axes show the evaluation values ranging from 0 to 1. SOF stands for the comprehensive NPE results of proposed service-oriented NPE framework based on FAHP. SOF-V refers to the NPE values of single video service, SOF-B refers to BE service. EVSSM indicates NPE method based on entropy of vague sets and similarity measure [3]. CRITIC is for criteria importance through inter-criteria correlation [15]. IEM is for improved entropy method [16].

Fig. 4

The comparison of proposed method with EVSSM

The comparison of proposed method with CRITIC

The comparison of proposed method with IEM

It can be seen from Figs. 4–6 that the trends of this paper’s curves well coincide with others, which means the evaluation results obtained by proposed framework are reasonable. Thereby it proves the effectiveness of proposed framework. After aggregating evaluation values of each service, the comprehensive NPE score is quite accurate compared to others. And since video service has significant temporal continuity, it requires the network to keep in a good state during a period of time. On the contrary, BE service has high burstiness and low temporal continuity, which has relatively lower requirements for network performance. The figures show that SOF-B is higher than all the others in time interval 1~8 and SOF-V is lower than SOF, CRITIC and IEM all the time. It means the score of video service is lower than that of BE service in the same network condition, which is totally consistent with afore mentioned theoretical expectations. By analyzing the NPE result of single service, to some extent, network administrators can understand what the network performance will be when adding this type of service into existing traffic, which is conducive to distribute service traffic more properly and select access network based on specific service. Through analyzing the

82

The Journal of China Universities of Posts and Telecommunications

overall NPE result, network administrators can have a more accurate cognition of the current network state, which is the foundation of afterwards network managements such as routing and network selection. Meanwhile, from Table 7, it can be observed in that in the time intervals 8 ~ 10, the network becomes worse sharply. The decrease rates of evaluation scores in period 8 ~ 9 are 0.184 1, 0.254 2, 0.212 4, 0.149 5, 0.177 7, 0.134 9 and in period 9~10 are 0.328 8, 0.215 0, 0.285 3, 0.485 5, 0.280 3, 0.167 0 respectively. In period 8~9, the decrease rates of SOF, SOF-V, SOF-B are apparently greater than others. In period 9~10, except EVSSM, the decrease rate of proposed framework is still larger than the other two methods and is up to 70.8% higher than the IEM, which strongly indicates that the proposed framework as well has good sensitivity to the variety of network state. 4.2

Computation complexity analysis

1) ECD algorithm Consistency checking refers to checking the consistency of constructed fuzzy matrix and usually compare the consistent radio (CR) of this matrix with a threshold µ . If

2015

gets large, the times of judgments saved will be very considerable, which reduces the complexity of constructing comparison matrix significantly and reduces the workload of experts substantially. At the same time, since experts pick their most confirmed element every time, it can be said that the proposed ECD also improves the credibility of the fuzzy consistent comparison matrix constructed. 2) EAM algorithm The time complexity of Chang’s EAM adopted in this paper is k (k + 6) [9], where k stands for the number of attributes. Comparing to the method of logarithmic least squares method (LLSM), the time complexity that EAM reduced is k (k 2 + k − 4) [9]. Then let m stands for the number of services and n stands for the number of network parameters. Driving the weight vector of sub-attributes with respect to one service needs to conduct EAM once. m services correspond to m times. Then the time complexity of sub-attributes part is m[n(n + 6)] , and the time complexity reduced comparing to LLSM is mn(n 2 + n − 4) . Calculating the weight vector of main-attributes needs another EAM, whose time complexity is m(m + 6) and

CR≤µ , then this matrix is considered as consistent, else

complexity reduced relatively is m(m 2 + m − 4) . As a

the matrix must be modified by the experts again and again until the decision condition is met. Because human’s judgments are always not absolute objective, the matrixes constructed by the traditional methods [13] often need to be modified to meet the consistency check, which causes much additional computations as well as workload for experts. By using the ECD algorithm mentioned in Sect. 3, the comparison matrix constructed will meets the fuzzy consistency demand naturally, so there’s no need to conduct consistency check afterwards, which will significantly reduce the computation complexity as well as workload of experts compared with previous FAHP methods. And what calls for special attention is that, in the traditional construction of comparison matrix, experts need to give n(n + 1) / 2 judgments; while by using proposed

result, the total time complexity of executing EAM is mn(n + 6) + m(m + 6) , where m = 2, n = 3 in this paper.

ECD algorithm, a totally consistent fuzzy matrix can be destructed only in need of the value of Vi1 j1 ,Vi2 j2 ,… ,Vit jt

(t < n) , which includes only (n − 1) times of assessments. Namely, the proposed ECD (n − 1)(n − 2) / 2 (calculated from

algorithm reduces n(n + 1) / 2 − (n − 1) )

times of assessments for the experts. When the value of n

And the total time complexity reduced comparing to LLSM is mn(n 2 + n − 4) + m(m 2 + m − 4) , whose value is quite considerable when m and n get larger. Evidently, the EAM adopted in this paper is better than former researches (LLSM) either. In addition, since the services considered in practical application are mainly video service, BE service (including Web-surfing, Email, FTP and file-sharing) and online gaming, types of which are very limited ( m≤6 ), thereby the value of mn(n + 6) + m(m + 6) is within acceptable limits. To summarize, implementing the proposed serviceoriented NPE framework won’t cause high time complexity although adding the NPE process of single service. What’s more, it even reduces the computation complexity of constructing fuzzy consistent comparison matrices by using proposed ECD algorithm, thereby making the proposed framework feasible for practical application.

Issue 3

Zhao Xing, et al. / Service-oriented network performance evaluation framework based on…

5 Conclusions and future work

83

“The Design of Radio Access Network Architecture in 5G communication system”.

This paper elaborated the necessity of considering specific service type in the NPE process and subsequently proposed a service-oriented NPE methodology based on LA-FAHP. In view of the two types of services: video service and BE service, this methodology firstly choose three indexes as evaluation attributes and adopted LA-FAHP to obtain the weight of each attribute, then gets the evaluation score of each service and eventually obtain the comprehensive NPE score by aggregating all the services. Through result analysis, it can be seen that the proposed framework is effective and quite sensitive to the variety of network state. At the same time it keeps low computation complexity though considering multiple services. And it’s worth noting that the computation complexity can be further reduced by applying the proposed ECD algorithm. However, the index set chosen as evaluation attributes for video service and BE service is the same, how to choose more targeted attributes for each service needs to be working on in future. The membership functions of indexes can be improved either. Meanwhile, more improvements can be done based on the FAHP algorithm adopted in this paper to reduce the inevitable subjectivity in constructing comparison matrix. And subjective experiments can be implemented in future work to more powerfully verify the effectiveness as well as improve the accuracy of proposed framework. Acknowledgements This work was supported by the Hi-Tech Research and Development Program of China (2014AA01A701), the Ministry of Education-CMCC research fund (MCM 20120132), and Beijing Municipal Science and technology Commission research fund project

References 1. Cisco visual networking index: Global mobile data traffic forecast update, 2013−2018. White Paper. San Jose, CA, USA: Cisco. 2014 2. McGregor T, Braun H W, Brown J. The NLAMR network analysis infrastructure. IEEE Communications Magazine, 2000, 38(5): 122−128 3. Qian Y, Guo Y, Song J, et al. Network performance comprehensive evaluation based on entropy of vague set and similarity measure. Proceedings of the 14th International Conference on Communication Technology (ICCT’12), Nov 9−11, 2012, Chengdu, China. Piscataway, NJ, USA: IEEE, 2012: 555−559 4. Zhou C, Li X, Li T T, et al. Research on comprehensive evaluation of network performance based on FAHP. Application Research of Computers, 2009, 26(10): 50−57 (in Chinese) 5. Saaty T L. How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 1990, 48(1): 9−26 6. Wang H T, Song L H, Chen H, et al. Study on network measurement technologies and performance evaluation methods. Proceedings of the 2013 Cross Strait Quad-Regional Radio Science and Wireless Technology Conference (CSQRWC’13), Jul 21−25, 2013, Chengdu, China. Piscataway, NJ, USA: IEEE, 2013: 377−380 7. Zadeh L A. Fuzzy sets. Information and Control, 1965, 8(3): 338-353 8. Van Laarhoven P J M, Pedrycz W. A fuzzy extension of Saaty’s priority theory. Fuzzy Sets and Systems, 1983, 11(1/2/3): 199−227 9. Chang D Y. Applications of the extent analysis method on fuzzy AHP. European Journal of Operational Research, 1996, 95(3): 649−655 10. Kahraman C, Cebeci U, Ruan D. Multi-attribute comparison of catering service companies using fuzzy AHP: The case of Turkey. International Journal of Production Economics, 2004, 87(2): 171−184 11. Alem S M, Jolai F, Nazari-Shirkouhi S. An integrated fuzzy DEA-fuzzy AHP approach: A new model for ranking decision-making units. International Journal of Operational Research, 2013, 17(1): 38−58 12. Kong F, Liu H Y. Applying fuzzy analytic hierarchy process to evaluate success factors of E-commerce. International Journal of Information and Systems Sciences, 2005, 1(3/4): 406−412 13. Chan H K, Wang X J, White G R T, et al. An extended fuzzy-AHP approach for the evaluation of green product designs. IEEE Transactions on Engineering Management, 2013, 60(2): 327−339 14. Saaty T L. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 1977, 15(3): 234−281 15. Xu F S. New method for similarity measures between Vague sets. Computer Engineering and Applications, 2011, 47(22): 33−34 (in Chinese). 16. Luo Y Q, Xia J B, Chen T P. Comparison of objective weight determination methods in network performance evaluation. Journal of Computer Applications, 2009, 29(10): 2524−2526 (in Chinese)

(Editor: Lu Junqiang)