Tomogravity space based traffic matrix estimation in data center networks

Accepted Manuscript Tomogravity space based traffic matrix estimation in data center networks Guiyan Liu, Songtao Guo, Quanjun Zhao, Yuanyuan Yang

PII: DOI: Reference:

S0167-739X(16)30846-9 https://doi.org/10.1016/j.future.2018.03.011 FUTURE 4023

To appear in:

Future Generation Computer Systems

Received date : 24 December 2016 Revised date : 29 January 2018 Accepted date : 4 March 2018 Please cite this article as: G. Liu, S. Guo, Q. Zhao, Y. Yang, Tomogravity space based traffic matrix estimation in data center networks, Future Generation Computer Systems (2018), https://doi.org/10.1016/j.future.2018.03.011 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Tomogravity Space Based Traffic Matrix Estimation in Data Center Networks Guiyan Liua , Songtao Guoa,∗, Quanjun Zhaoa , Yuanyuan Yanga,b a

Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, College of Electronic and Information Engineering, Southwest University, Chongqing, 400715, China b Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY 11794, USA

Abstract Traffic matrix (TM) is an important input requirement to better system management in data center networks (DCNs). Directly estimating TM is cost and difficult since the flow behaviors in DCNs are irregular and the TM across the Top of Rack (ToR) switches is huge. Although indirect TM estimation tomography based methods can be applied in DCNs after decomposing tree-like structure, these approaches require a good prior TM obtained by gravity model to improve estimation accuracy. In addition, data collection from the Simple Network Management Protocol (SNMP) employed in DCNs can result in unavoidable data missing and data errors. Therefore, it is necessary to estimate a good prior TM and study the effect of link data missing or data errors on estimation accuracy. In this paper, we utilize the tomogravity space to achieve TM estimation in decomposed treelike DCNs without requiring a good prior TM because the gravity model can be replaced by gravity space. We propose two iterative algorithms to estimate TM between tomogravity space and gravity space, and use similar-Mahalanobis distance as a metric to control estimation errors. One iterative algorithm utilizes a prior TM calculated based on coarse-grained traffic characteristics, whereas the other considers moderate link data missing and no prior TM based on traffic characteristics. To further separately discuss the effect of link data errors, we obtain desirable link measurement from packet trace and routing matrix. Numerical results demonstrate that our iterative algorithms outperform the existing algorithms ∗

Corresponding author. Email address: [email protected] (Songtao Guo)

Preprint submitted to Elsevier

January 29, 2018

in terms of controlling data errors based on decomposed structure and produce robust results when adding different noise level on the link data. Keywords: Data center networks; traffic matrix estimation; coarse-grained traffic characteristics; iterative algorithms; tomogravity space. 1. Introduction With the increasing jobs or applications also considered as virtual network requests from cloud-based services [1–3], network traffics in data center networks (DCNs) have increased exponentially, which aggravates the burden of network management [4]. This is because the network traffic growth dramatically in DCNs will result in frequent congestion and packet loss in DCNs. To better support rapidly changing business requirements and engineering tasks, network operators need to know the packets forwarding between different switches. Traffic matrix (TM), which completely records the traffic volumes of origin-destination (OD) node pairs, is a critical input for many engineering tasks, such as anomaly detection [5], load balancing [6] and network planning and provisioning [1, 7–10]. As another application example, some security issues, e.g., security-driven scheduling [2], intrusion detection [3, 11–13], and policy-based virtual machine migration [14–16], are also studied by analyzing the changes of flow traffic. Therefore, it is of important significance to figure out the characteristics of traffic flows within DCNs and find out an efficient method to estimate traffic matrix. However, directly leaning traffic characteristics and estimating TM are difficult especially in large scale DCNs, because detailed traffic flow information needs to be collected. Only a few works [17, 18] have exploited some basic traffic characteristics within DCNs. Benson et al in [17] focused on network traffic behaviors of different types of DCNs and applications deployed in cloud data centers while in [18] they considered end-to-end traffic patterns of DCNs. Both of them collect packet traces by attaching a packet sniffer to a SPAN port on the switches at a random location. Another work in [19] reduces overhead traffic management in DCNs by detecting elephant flows, and collects packet traces by deploying additional instrument on servers in small DCNs. Although local accurate information can be well obtained from part of switches installed packet sniffers [17–19], global information of OD pairs cannot be well estimated from observed measured information unless one knows the relationship of all nodes in the network or installs packet sniffers on every switch in entire network [20, 21]. To learn the relationship among nodes, additional network resources will be uti2

lized because large amount of communications between them are required [22]. Moreover, installing packet sniffers on each switch requires lots of costs, thus it is expensive and impractical to install instrument to whole network especially in large-scale DCNs [23–25]. To obtain traffic matrix information without additional instrument, one can obtain the aggregate traffic information and collect link load from Simple Network Management Protocol (SNMP) named network tomography. It is a classical methodology of inferring fine-grained network characteristics from link measurements in traditional Internet Protocol (IP) networks [21, 26, 27]. In earlier works, researchers modeled OD flow as a simple model, such as Poisson distribution [28], Gauss distribution [29]. But estimation performance strongly depends on assumption of these models. Subsequently, tomographic model based tomography method was proposed in [21], which is a well-known approach in IP networks conjugating gravity model to estimate a prior TM. It can estimate TM quickly and dynamically track flow behaviors, but performance also depends on the selection of an appropriate prior TM. Zhou et al. [30] proposed a precise gravity model to obtain good prior TM to improve estimation accuracy. A more sophisticated approach based on Kalman filters [27] was proposed, which can well track TM evolution with time but appropriate parameters based on TM samples are necessary. Although the SNMP is uniquely supported in switches of DCNs, it is difficult ot directly apply the existing tomography based methods to estimate TM in DCNs [20]. The underlying reasons include that (i) the aggregate traffic information in DCNs cannot be flexibly used due to irregular flow behaviors [18]; (ii) the servers in DCNs play different roles from that in IP networks [20]; (iii) the traffic matrix estimation has higher time and space complexity since there exist a large number of redundant routes in DCNs [22]. To apply tomography based methods into DCNs, the authors in [23, 24] divided DCNs into several clusters, which deals with the large quantity of possible routes between OD pairs in DCNs. They borrowed classical tomographic model and kalman filters to estimate TM in DCNs. However, performance depends on a good prior TM and effective parameters are required in kalman filters since SNMP agent cannot provide effective historical data. They then proposed CREATE in [25] by computing service placement among switches and reducing the lowly utilized links in DCNs, and further extended the work in [31] and presented an ATME scheme for both private and public cloud DCNs by provisioning resource information. In addition, Nie et al. [32] adopted a deep belief network (DBN) and a logistic regression model to solve network tomography model. Nevertheless, in large-scale DCNs, it is tremendous computational complexity via deep learning to 3

train link count and most of traffic scheduling among switches will produce large communication overhead and lead to network congestion [19]. When congestion happens, clearly, data collection from SNMP could be inaccurate and incomplete, such as link data missing. As a result, it is necessary to consider the situation where there exist moderate link data missing and link data errors in data collection when estimating TM in DCNs from SNMP. However, it is difficult to indicate the estimation errors caused by inferring TM or link measurement collected from SNMP. Based on the above considerations, we obtain desirable link load from packet trace and routing matrix to reduce estimation errors caused by SNMP itself. In addition, to reduce communication overhead and computation complexity to estimate TM, we decompose whole DCNs into several sub-networks or clusters, according to tree-like data center network structure. Thereby, we transform the TM estimation of entire DCNs into the estimation of several small TMs, i.e., TM estimation is performed across the clusters (cluster to cluster pair) and within the clusters (ToR to ToR switch pair). Subsequently, we propose two TM estimation iterative algorithms, ICGA and SAWP, by defining probability vectors of tomography space and gravity space and utilize similar-Mahalanobis distance to control estimation errors between two spaces. According to coarse-grained traffic characteristics in DCNs based on decomposition, in our ICGA iterative algorithm, we can easily obtain total incoming and outgoing traffics of a switch, which are taken as a prior to estimate TM. Then considering that a good prior TM is unknown and moderate link data missing is allowed, we redefine probability vectors of tomographic space and propose second simplified iterative algorithm called SAWP. The main advantage of tomogravity space based TM estimation is that it can indeed not require good prior information by replacing gravity model with gravity space. In summary, our main contributions are described as follows: • Considering data missing and data errors when collecting link data from SNMP, we obtain desirable link load from packet trace and routing matrix, and separately analyze the effect of link data errors on estimation errors by adding different white noise level. • Since most of traffic scheduling among switches will produce large communication overhead and result in network congestion in large-scale DCNs, we decompose DCNs into several clusters to easily handle TM estimation. After decomposition, flow behaviors in DCNs are more stable and coarsegrained traffic characteristics can be well obtained using link data. 4

• We present two iterative algorithms, ICGA and SAWP, to estimate traffic matrix. The former requires a prior TM based on coarse-grained traffic characteristics, and the latter allows moderate link data missing without a good prior TM. In both algorithms, we utilize similar-Mahalanobis distance to control estimation errors. • Simulation results demonstrate that both algorithms are effective to control the estimation errors in DCNs no matter whether there is an explicit flow structure in a cluster and produce robust results with different white noise level on the link measurement. The remainder of this paper is organized as follows. We describe the related work in section 2. In section 3, we introduce the typical Cisco network and obtain the coarse characteristics as a prior TM. Then we present traffic matrix estimation problem and two iterative algorithms in section 4. Section 5 provides and analyzes simulation results. Finally, Section 6 concludes this work. 2. Related Work Various engineering tasks [4–10] and security issues [2, 3, 11–16] require traffic matrix (TM) to reflect the current state of network. Accurate flow information is helpful for scheduling network traffic and detecting the attacks. For example, network provisioning [1, 7–10] and policy-based virtual machine migration [14– 16] require virtual network requests as an important input in a real-time, intrusion detection [3, 11, 12] needs to deal with large volume of real-time network traffic, and distributed denial of service (DDoS) attacks [13] can be detected quickly and accurately by using an optimized TM at the early stage. Therefore, estimating TM is a prerequisite work, and can provide better support for these engineering tasks. In this section, we mainly discuss the related work on TM estimation to network tomography in Internet Protocol (IP) networks and data center networks (DCNs). Network tomography, originated by Vardi [28], refers to the methodology of inferring fine-grained network characteristics from aggregate measurements. It is a well-known technique that allows traffic matrices of IP networks to be inferred from link level measurements[21, 26, 29, 30, 33–38]. For example, the authors in [29] modeled OD flows as Gauss distribution and estimated TM by Expectation Maximization (EM) algorithm. In addition, tomogravity based tomography method [21] adopts the gravity model to get a prior TM. Fang et al. [33] proposed an iterative tomogravity algorithm which requires no complete knowledge 5

on individual edge links, but link data missing or errors caused by collected from SNMP were not considered. To improve TM estimation accuracy, Zhou et al. [30] proposed a precise gravity model by the Moore-Penrose inverse and minimum least-square solution. Moreover, Sparsity Regularized Matrix Factorization (SRMF) [26] leverages the spatio-temporal structure of traffic flows, and infers missing data in TM by utilizing the compressive sensing method. As an extensive, Nie et al. utilized compressive sensing method to reconstruct all the OD flows [34] and they further proposed a novel TM prediction and estimation methods based on deep learning in large-scale IP networks [39]. In addition, the authors in [35] presented an improved Fanout estimator called Tomofanout, to obtain a more accurate TM estimation performance. Furthermore, the real-time TM estimation methods were proposed in [36–38]. Apart from the methods which take the IP OD flows as study targets, most recent works [20, 23–25, 31] provided the TM estimation tomography method in DCNs. Kandula’s study [20] learned the nature of a single MapReduce data center, and pointed out that directly using tomography based methods to infer traffic volumes is poor in DCNs with their evaluations. However, after decomposing tree-like DCNs into several clusters, the authors in [23, 24] validated that the traffic flow across clusters are relatively smooth and TM can be obtained through borrowing classical tomography based methods. In addition, Hu et al. [25] proposed a CREATE method to estimate TM by leveraging the uneven traffic distribution. They first computed the service correlations between different top-of-rack (ToR) switches and then eliminated lowly utilized links in DCNs to reduce redundant routes. And they further extended the work in [31] and developed an ATME scheme by provisioning resource information for public and private cloud DCNs. But the accuracy of TM depends on a good prior TM and the low-utilization threshold. If the value of threshold sets non-zero link counts to zero, it may lead to estimation errors. Moreover, by exploring the connotative properties of traffic flows, Nie et al. [32] obtained a predictor of the TM via deep learning theory and modeled a TM by considering the time-varying and spatio-temporal properties. They adopted a deep belief network (DBN) and a logistic regression model to solve network tomography model. But in fact, it is computational to train large amounts of data and compute the gradient of the negative log probability function because the gradient is composed of a sum of all states of the model. In addition, there are some other works [40, 41] similar to ours, which estimate TM in road transportation. Abadi et al. [40] focused on traffic flow prediction for all the links in a transportation network over a short time horizon while Asif et al. [41] considered the problem of missing data in large and diverse road networks. 6

In [40], they proposed a methodology that predicts the traffic flow by two steps including traffic flow data completion and short-term traffic flow prediction. In [41], furthermore, they proposed various matrix and tensor based methods to estimate missing values by values by extracting common traffic patterns in large road networks. However, the proposed methods can hardly be directly applied to the data center networks because network traffic in DCNs has more fluctuations and flow behaviors are more irregular and unstable [17–19]. Compared with previous works [23–25, 32], we mainly consider that tomography based methods applied in DCNs to estimate TM require a precise gravity model to obtain a good prior TM and some moderate link data missing or data error is allowed when collecting link measurement from SNMP. In this paper, instead of estimating a good prior TM, we estimate TM in decomposed DCNs between tomogravity space and gravity space, and link load is obtained from packet trace and routing matrix. It is regarded as desirable link data, and used to separately discuss the effect of link data errors on estimation errors by adding different noise level to desirable link data. 3. Coarse-Grained Traffic Characteristics In this section, we describe the tree-like DCNs decomposition and coarsegrained characteristics based on decomposed Cisco network. 3.1. Network Structure There are varieties of data center network structures, ranging from recursive topologies such as fat-tree [42], Dcell [43], VL2 [44], to flexible topologies such as c-Through [45], Helios [46]. A typical data center network adopted in [47] is shown in Fig.1, which consists of three levels of tree-like structure, including core switches connecting Internet, aggregation switches and edge switches connecting lager number of servers from top to down, respectively. Considering the irregular and unstable behavior of flows, in this paper, similar to [23] and [24], we would like to make use of a decomposed Cisco network depicted in Fig.2, where tree-like DCNs are decomposed into several clusters, to obtain the coarse-grained traffic characteristics according to aggregate information by gravity model. The feasibility of the decomposition is based on the locally tree-like structure, which satisfies the conditional independence assumptions, namely, assuming that there is only one path between an OD pair and the routing matrix will be fixed in a short time.

7

Figure 1: An example of Cisco structure

Figure 2: The decomposed Cisco network structure of clusters with ToR to ToR switch pair and cluster to cluster pair

. As shown in Fig.2, T oR1 ∼ T oR4 and Agg1 ∼ Agg2 form a cluster as C1 , and the traffic flows of C1 are independent to that in cluster C2 composed of T oR5 ∼ T oR8 and Agg3 ∼ Agg4 . This is because clusters and core switches can be considered as OD pairs satisfying independent condition. And if we know the traffic flows go in (or out of) Agg1 ∼ Agg2 within the cluster, then the traffic flows that go in (or out of) T oR1 ∼ T oR4 is independent to that in T oR5 ∼ T oR8 . Therefore, it is reasonable to turn TM estimation in whole network into TM estimation across the clusters (cluster to cluster pair) and within the clusters (ToR to ToR switch pair). 3.2. Characteristics Based on Decomposed Network According to the aforementioned conditional independence assumption of network decomposition, estimating TM in tree-like DCNs is transformed into inferring TM across the clusters (cluster to cluster pair) and in the clusters (ToR to 8

ToR switch pair) based on decomposed network, which is much easier than TM estimation within whole network in terms of tomography method. Hence, we can first obtain the coarse-grained characteristics based on decomposed network by gravity model. The characteristics include how severs communicate with each other, how traffic flows vary among servers or Top of Rack (ToR) switches and how many traffic flows from T oRi in cluster Ci go into (or originate from) another clusters. In this following, we give the definitions of “out” and “in” flows and formulate the characteristic of clusters [23]. Lemma 1. The “out” flows are the flows that come from servers, denoted by Yout (T oR) and “in” flows are defined as the flows that are transferred to the servers, represented by Yin (T oR). Obviously, the total traffic on ToR switches can be given by Y (T oR) = Yin (T oR) + Yout (T oR). Then the traffic characteristics of clusters can be formulated as ∑ (Yin (T oRk ) + Yout (T oRk )) − Xintra (Ci ) = T oRk ∈T oRCi

∑

Aggj ∈AggCi

Xin (Ci ) =

∑

Aggj ∈AggCi

Xout (Ci ) =

∑

(1)

(Y (Aggj ))

Aggj ∈AggCi

(Y (Aggj )) − (Y (Aggj )) −

∑

(Yout (T oRk ))

(2)

(Yin (T oRk ))

(3)

T oRk ∈T oRCi

∑

T oRk ∈T oRCi

where Xintra (Ci ) denotes the total traffic within cluster Ci , Xin (Ci ) indicates the total traffic entering cluster Ci from other clusters, Xout (Ci ) represents the total traffic going out of Ci , and Y (AggCi ) is the traffic load on the aggregation switches in cluster Ci . AggCi denotes the set of the aggregation switches in cluster Ci . It is clear that from (1), (2), (3), we can know the traffic flows across clusters. In particular, the formulaes (1)-(3) specify the coarse-grained traffic characteristics of clusters. It is noted that we could not obtain the traffic flows directly when ToR switches as origination and destination nodes in different clusters. 3.3. A Prior TM We can use gravity model to obtain a prior traffic matrix based on coarsegrained traffic characteristics. The gravity model is based on the assumption of a 9

simple proportionality relationship [21]: (4)

Xij ∝ Yiout · Yjin

where Xij denotes the traffic from the ith end to the jth end, Yiout denotes the total traffic exiting at the ith end, and Yjin denotes the total traffic entering at the jth end. The model states that the TM element Xij is proportional to the product of the total traffic entering network at location i and the total traffic leaving network at location j. By traffic formulaes from Lemma 1 and gravity model, we can calculate the traffic flows X(Cij ) between the clusters Ci and Cj by X(Cij ) = Xout (Ci ) · ∑

Xin (Cj ) Ck ∈C Xin (Ck )

(5)

Moreover, the traffic exchanged by ToR switch pairs between T oRi and T oRj can be calculated by X(T oRij ) = (1 − ∑ ·

(1 −

∑

T oRj

Xout (Ci ) ) · Xout (T oRi ) T oRj ∈T oRC Xout (T oRj ) i

Xin (Ci ) ) Xin (T oRj ) ∈T oR Ci

Xintra (Ck )

· Xin (T oRj )

(6)

Therefore, by gravity model, we can obtain a prior TM of cluster to cluster pair and ToR to ToR switch pair, respectively. As for weights, since there is only one hop on the route, the number of core switches (aggregation switches) is equal to the the number of the routes in cluster to cluster pair (ToR to ToR switch pair). That is to say, the assignment to the weights that go through the corresponding core switches or aggregation switches depends on routing strategy. 4. Traffic Matrix Estimation Algorithm In this section, we first briefly describe the problem of traffic matrix (TM) estimation from link data. According to coarse-grained traffic characteristics in DCNs based on decomposition, we then present two iterative algorithms, ICGA and SAWP, to estimate TM between tomogravity space and gravity space with similar-Mahalanobis distance.

10

4.1. Problem Formulation Let p be the number of total traffic links in data center networks and q be the the number of all available paths between ToR switch pairs. In DCNs, the problem of TM estimation based on tomographic model can be expressed as [28] Y = AX

(7)

where Y = (y1 , y2 , ..., yp )T denotes the vector of traffic loads on links, which can be readily obtained by available SNMP, and X = (x1 , x2 , ..., xq )T indicates the vector of traffic flow demands on the routes in DCNs. The element yj in Y and xi in X respectively denote the load measurement for link j, and the corresponding traffic volume of ith ToR switch pair. The relationship between Y and X is associated with routing matrix A. In (7), A is a zero-one matrix, A = (aij )p×q , where aij equals to 1 when flow xj between ToR switch pairs passes through link yi ; 0 otherwise. The problem we need to solve is to compute X from known Y and A, that is, to find a set of ToR switch flows to reproduce the traffic loads as closely as possible. However, when the number of ToR switch pairs grows in the tens in DCNs, the number of links increases in the hundreds. Take the Fig.1 as an example, when incorporating 10 new ToR switches, the possible routes will increase more than 1000. The number of OD paris is often much larger than that of links, namely dim(Y) << dim(X). Hence, it is impractical to solve the ill-posed traffic matrix estimation problem from link data directly. 4.2. Iterative Algorithm with Coarse-Grained Characteristics In the tree-like DCNs, when a source ToR switch goes to the destination ToR switch in another cluster, it will traverse through core switch and then reach to the destination ToR switch. In this case the core switch does not generate traffic and just forward the traffic of the source ToR switch [48]. Consequently, we can regularize the tomographic model (7) between gravity space and tomogravity space. Besides, known the link level of information in DCNs and traffic characteristics based on decomposed network, we can alleviate the ill-posed TM problem by dividing a large TM estimation of whole network into several small TMs. Therefore, an iterative algorithm with coarse-grained traffic characteristics, called ICGA, is presented by taking advantage of decomposed network and traffic characteristics. In the following, we define the tomogravity space and gravity space as in [33]. Let O and D be the set of origination ends and destination ends respectively, and V is the total number of OD pairs, so that V = |O| × |D|, where |·| denotes 11

the size of a set. We let RV indicate the V dimensional vector space. Thus the one-to-one mapping between RV and |O| × |D| matrix can be expressed as w = (w1 , w2 , ..., wV )T ∼ (wod )|O|×|D| . To take the characteristics obtained in section 3.2, we let the probability vectors of tomographic space be Γ = {f ∈ RV | Af =

Y , f ≥ 0, 1T f = 1} N

(8)

and that of gravity space be ς = {g ∈ RV | g ∼ (god )|O|×|D| = pqT , g ≥ 0, 1T g = 1}

(9)

where 1 is the vector of 1’s, p ∈ R|O| , q ∈ R|D| and wT is the transpose of w. N is the total traffic loads between ToR switch pairs going in or out of the network, which equals to the summation of the columns or rows of the traffic matrix. In addition, both g and f have a matrix form and a vector form as traffic matrix does. Our ICGA iterative algorithm based on coarse-grained characteristics provides estimation of the traffic matrix, which can be described in Algorithm 1. Algorithm 1 Iterative Algorithm Based on Coarse-grained Characteristics (ICGA) Input: x ˆ(0), N , θ Output: x ˆ x ˆ(0) 1: Initialize: g(0) = N , k = 0; 2: repeat 3: Compute M (f , g) = Eg ∥f − g∥22 ; 4: Compute f (k + 1) = arg min{M (f , g(k)) : f ∈ Γ} ; 5: Compute g(k + 1) = arg min{M (f (k), g) : g ∈ ς} ; 6: k = k + 1; 7: until ∥f (k) − g(k) ∥2 < θ 8: Obtain x ˆ = N ∗ f (k); In the Algorithm 1, x ˆ(0) is a prior TM obtained by (5) and (6), which is used as the initialization for ICGA. M (f , g) = Eg ∥f − g∥22 is defined to sense the data errors of traffic matrix by leveraging the distance of f and g, called similarMahalanobis distance, and ∥·∥ is the L2 form. Ideally, there is an interesting tradeoff between the probability and the distance. i.e., when the value of the possibility is bigger, the difference between tomogravity space and gravity space 12

is much closer. Given an error limit θ, The algorithm terminates when the L2 form of f (k) and g(k) is infinitely close, i.e. ∥f (k) − g(k) ∥2 < θ, and k is the number of iterations. Once algorithm terminates iteration, it will output the x ˆ shown in line 8. Specifically, the solution to line 4 is described in section 4.3. As mentioned in [33], line 5 is explicit with ∑ (k) ∑ (k) (k+1) fod′ fo′ d (10) god = d′

(k+1)

where god

o′

(k)

and fod are the matrix form of g(k + 1) and f (k).

4.3. Simplified Iterative Algorithm In this subsection, we present a simplified iterative algorithm without considering a good prior TM, called SAWP, with the aim to regularize the tomographic model with gravity space instead of gravity model. In addition, due to the possible failure in data collection, e.g, incomplete SNMP, data missing is unavoidable. Therefore, we try to investigate the efficiency of TM estimation when partial link data missing is allowed. General tomographic model with the partial observed link data (7) can be described as follows : ˆ = Aˆ ∗ X Y

(11)

ˆ is a sub-vector of Y and Aˆ = (ˆ where Y aij ) is composed of the rows of routing matrix A in accordance with the observed links. Let Vˆ , which is a subset of V without corresponding OD pairs of which the traffic flows of link are zero, ˆ the be the total number of OD-pairs of certain. Therefore, for the observed Y, probability vectors of tomographic space and gravity space are respectively as follows [33] ˆ = {f ∈ RVˆ | Af ˆ = Y, ˆ f ≥ 0, 1T f = 1} Γ (12) ˆ

ςˆ = {g ∈ RV | g ∼ (god )|O|×|D| = pqT , g ≥ 0, 1T g = 1}

(13)

Note that when link data is missing, the set O of source nodes allows to be different from the set D of destination nodes, i,e,. Vˆ = |O| × |D|, where O ̸= D. Moreover, considering that when yi = 0 implies that fij = 0 (Here, fij is expressed in a matrix form) for all j with aij = 1, the problem of minimizing M (f , g(i)) under the linear constraint in (12) in Algorithm 1 can be reduced to Vˆ . Therefore, SAWP algorithm can be improved in computational complexity. Furthermore, considering no a good prior TM, g(0) in line 1 of Algorithm 1 can 13

be treated as g ˆ(0) = 1/Vˆ , and N in line 8 is not the total traffic exchanged within cluster to cluster pair or ToR to ToR switch pair any more. N is denoted ˆ ˆ = T 1T Y , showing the percentage of generalization coefficient, where as N 1 A∗f (fin) ˆ is the vector of non-zero f (fin) is the final f (k) satisfying the termination and Y elements in Y. Next, our problem turns to minimize M (f , g) = Eg ∥f − g∥22 in line 4 defined as problem 1 under the linear constraints in (12) with a set of Vˆ . Then problem 1 can be formulated as min M =

n ∑ i=1

gi (fi − gi )2

ˆ 1T f = 1, f ≥ 0 s.t. Hf = Y,

(14)

where H is a m × n matrix as routing matrix does, and m ≤ p, n = q. Next, ∑n we will prove the problem 1 is2 convex. First, the objective function M = i=1 gi · M0 , where M0 = (fi − gi ) , is the convex combination with respect to M0 since 1T f = 1 from (12). Since M0 is convex with respect to fi , the objective function M is also convex with respect to fi . Then, since the constraint condition in (14) is an affine function, the feasible region of solution is a convex set. Thus, the problem 1 is convex problem. After that, we use Lagrangian Multipliers to solve it and it can be expressed as follows: L(f , u, v) =

n ∑

2

gi (fi − gi ) +

i=1 n ∑

+ v(

i=1

m ∑ j=1

uj

n ∑ i=1

(Hji · fi − yj ) (15)

fi − 1)

where v is a parameter and u = (u1 , u2 , ..., um )T is m-dimensional vector of Lagrangian multipliers. Moreover, according to KKT conditions, we can obtain the first order derivative of (15) as follows: m

∑ dL = 2gi (fi − gi ) + uj Hji + v = 0 dfi j=1 14

(16)

n

∑ dL = (Hji · fi − yj ) = 0 duj i=1

(17)

n

dL ∑ = fi − 1 = 0 dv i=1

(18)

Thus, the optimal solution to problem 1 is

fi = arg minL(f , u, v) = gi −

m ∑ 1 ∗( uj Hji + v) 2gi j=1

(19)

Next, Lagrangian Multipliers are updated by the gradient method and they can be obtained by (17) and (18) as n ∑ k k uk+1 = [u + δ ( (Hji · fi − yj )]+ j 1 j

(20)

i=1

v

k+1

k

= [v +

n ∑

δ2k (

k=1

fk − 1)]+

(21)

where δ1k and δ2k are the step sizes taken in the direction of the negative gradient for the individual uj (j = 1, 2, ..., m) and v at k-th iteration, and [·]+ denotes the projection to [0, +∞). The line 4 in Algorithm 1 is described in (15)-(21) with full observed links, and the simplified iterative algorithm without prior information (SAWP) is described as Algorithm 2. In Algorithm 2, the inner iteration of line 4-line 20 shows the optimal solution to f (k), where g ˆi (k) in line 7 and fi (k) in line 16 denote the i-th vector of g ˆ(k) and f (k), and line 8 and line 16 represent the transform between matrix form and vector form. Besides, updating Lagrangian Multipliers is done by gradient descent method for individual uj and v cycling through j = 1, 2..., m. The inner iteration will stop and output fi (k) until the difference between the twice solution becomes lower than a predefined tolerance ϵ, and iner max is the total number of iterations required for reaching convergence. In line 21, g ˆ(k + 1) is calculated according to (10), where f (k) is an input. Then once the L2 form of difference between f (k) and g ˆ(k) is less than θ, the algorithm will output final f (k) to obtain TM. Compared with the ICGA, SAWP is easier to implement and does not need the complete traffic load information on aggregation loads to obtain traffic characteristics. In addition, since iterative steps line 4 and line 5 described in Algorithm 1 15

are both monotone in M (f , g), a local optimal matrix can always be guaranteed between tomogravity space and gravity space in our iterative algorithm. Algorithm 2 Simplified Iterative Algorithm without Prior Information (SAWP) ˆ H, Vˆ , m, n, θ, u, v, iner max, ϵ ˆ A, Input: Y, ˆ Output: X 1: Initialize g ˆ(0) = 1/Vˆ , k = 0; 2: repeat 3: ii ← 0; 4: repeat 5: ii ← ii + 1; 6: for i = 1 to n do 7: Compute g ˆi (k) ← g ˆ(k); 8: Compute g(i) ← g ˆi (k); 9: Get W (i) ← 0; 10: for j = 1 to m do 11: Compute W (i) = W (i) + u(j)H(j, i); 12: end for ∑ 13: Compute f (i) = gi − 2g1 i ∗ ( m j=1 uj Hji + v); 14: end for 15: Update Lagrangian multipliers u and v using (20) and (21), respectively; 16: Compute fi (k) ← f (i) ; 17: until (u(ii) − u(ii − 1) > ϵ and v(ii) − v(ii − 1) > ϵ) and ii < iner max 18: for i = 1 to n do 19: Compute f (k) ← fi (k); 20: end for 21: Compute g ˆ(k + 1) = GetgF unc(f (k)); 22: k = k + 1; 23: until ∥f (k) − g ˆ(k) ∥2 < θ T ˆ ˆ 24: Compute N = T1ˆ Y ; 1 A∗f (k) ˆ ∗ f (k); ˆ N 25: Compute X= 4.4. Complexity Analysis We now give a brief complexity analysis for ICGA and SAWP. Both algorithms aim to compute f (k) and g(k) (denoted as g ˆ(k) in SAWP) from tomogravity space and graphy space, respectively. The main difference between them is to 16

define different tomographic space. ICGA considers to take a prior TM based on coarse-grained traffic characteristics as an input whereas SAWP does not employ these characteristics. Although definition of tomographic space is different, in fact, the solution to f (k) in both algorithms is based on (15)-(21). In ICGA, the number of observed links is denoted as p and the number of OD pairs is q while in SAWP, the number of observed links and OD pairs are m and n, respectively. In our experiment, ICGA is based on full observed link and SAWP allows moderate link data missing, so the number of links in ICGA is greater than that of SAWP, i.e., p > m. In addition, the number of OD pairs will not change, i.e., q = n, because in the network communication, it always equals to square of the number of switches in DCNs in our paper. Next we analyze time complexity in detail. In both algorithms, TM is estimated between tomogravity space and gravity space, thus the local complexity comes from computing every element in f (k) and g(k) (or g ˆ(k)). As shown in SAWP, it is not difficult to observe that the time complexity for computing f (k) from line 4 to line 20 is T (SAW P ) = iner max ∗ mn + n, where m is the number of observed links and n is the number of OD pairs. And the time complexity of computing g(k) is T (SAW P ) = n2 , because g(k + 1) is calculated by (10) shown in GetgFunc f (k) where f (k) is an input of g(k + 1) and it has the same dimension as g(k + 1) does. Therefore, the time complexity of SAWP is T (SAW P ) = Iter max ∗ (iner max ∗ mn + n + n2 ), where Iter max indicates Iter max external-loop iterations to achieve convergence of estimating TM. In practice, the number of iterations of algorithm convergence also depends on diminishing step sizes. Similarly, in ICGA, the time complexity is T (ICGA) = Iter max ∗ (iner max ∗ pq + q + q 2 ). Since we have p > m and q = n, obviously, T (ICGA) > T (SAW P ) in case of moderate link data missing. Hence, SAWP has lower computational complexity compared with ICGA. 5. Performance Evaluation In this section, we first describe the simulation environment including network topology and traffic generation, and then give compared algorithms and performance metrics. Finally, we analyze the simulation results. 5.1. Simulation Environment In the simulation, we first use NS-2 simulator [49] to generate traffics of a medium scale network topology with 8 core switches, 16 aggregation switches and 32 ToR switches connecting 20 servers. Then we implement all algorithms 17

by Matlab (R2012a) on Intel i5-6500 CPU @3.20GHz, with 8GB of memory and the Windows 7 64-bit OS. We generate traffic flows based on the study of the traffic characteristics of DCNs [17, 18, 20]. We create an exponential On/Off distribution traffic generator. As for data packets, the size is set to be 1400 bytes and sending rate in On period is set to be 500kb. In addition, TCP flows are used to simulate real DCNs, because most of flows are TCP. And routing strategy is ECMP [50] because it is widely used in DCNs, so the weights of path between two ends are set to be equal. In the simulation, we randomly select 3-12 servers connecting each ToR switch to generate flows for other servers, but servers selected each time to generate traffic should satisfy that there are traffics going out of cluster and exchanging in the cluster. We conduct experiment during 300 seconds including 100s burst time and 200s idle time. At the same time, we record total packets of flows on each route in the corresponding time as true traffic matrix, and the total number of packets that go out of switch and enter switch to compute the rough traffic characteristics. Both of them are recorded in trace file. Routing matrix is obtained based on shortest routes of each flow. Thus it can be computed based on the format of trace file by finding three consecutive enqueue event with the same source address and destination address. This is because tree-like DCNs are symmetrical in nature and each route between two servers in different cluster has six hops, which means that there are three hops from source up to a core switch, and three hops from the core switch down to the destination. Once the core switch is determined, the path is unique [42]. Clearly the path selection depends on the first three hops for two servers in different cluster. After the packet trace and routing matrix are determined, to separate errors from link measurement and estimation, we can obtain desirable link data by equation (7) under the assumption that the core switch does not generate traffic and just forwards traffic when traffic going from source node to destination node [48]. Subsequently, we use this desirable link data to infer traffic characteristics in DCNs, and take these characteristics to configure other parameters to estimate TM. 5.2. Algorithms and Performance Metrics We compare our proposal with the existing algorithms: (1) TMBCT [23] : it obtains the hypothesis sets based on rough characteristics and refines them by least square program under constraints of observation. (2) TMBLS [23] : it is based on kalman filter algorithm, revealing the spatial-temporal relations of traffic matrix. (3) CREATE [25] : it estimates TM in DCNs by computing service correlations between ToR switches and eliminating lowly utilized links. (4) DBN [32] 18

: deep belief network (DBN) and logistic regression model are utilized to obtain a predictor of TM and solve network tomographic model. We mainly compare our algorithms with TMBCT and TMBLS, because they are all based on decomposed tree-like DCNs. To evaluate the effectiveness of SAWP, we then make comparison with CREATE, because both of them have moderate link data missing when estimating TM. Moreover, we make a comparative analysis with DBN to verify the accuracy of our algorithms. In addition, we choose some appropriate performance metrics to evaluate the algorithms. Since the traffic flows have little utilization on links in DCNs, and link utilization is also rather low in all layers except the core layer[17], unlike other engineering tasks such as load balancing and failure analysis, we measure the performance of algorithm mainly from the CDF of relative error (RE), relative total error, mean relative errors (MRE), root mean squared relative error (RMSRE) and bias and standard deviation. The RE is expressed as X − X ˆ i i REi = (22) Xi and MRE is denoted as in [51]

ˆ i 1 ∑ Xi − X M RE = · N τ X >τ Xi

(23)

i

and relative total error is formulated as [33] ∑ ˆ i Xi − Xi ∑ REtotal = i Xi

(24)

and RMSRE is denoted as [21]

v u u 1 RM SRE(τ ) = t Nτ

Nx ∑

i=1,Xi >τ

(

ˆi Xi − X )2 Xi

(25)

and sample bias and standard deviation are defined as [39] T 1∑ ˆ Biasi = (Xi (t) − Xi (t)) T t=1

19

(26)

v u u Stdi = t

T

1 ∑ ˆ ((Xi (t) − Xi (t)) − Biasi (i))2 T − 1 t=1

(27)

ˆ i denotes the corresponding estiwhere Xi denotes the true traffic volumes, and X ˆ i (t) are the corresponding true and estimated mated value. Similarly, Xi (t) and X traffic volume at discrete time t. Since the minor flows such as ACK flows have little link utilization in DCNs, we use τ to pick up the relative large traffic flows. N τ is the number of elements in traffic matrix larger than the threshold τ and Nx is the number of elements in the ground truth X. The CDF of RE indicates the percentage of accurate results while MRE focuses on the global deviation of the estimation from the real one. In addition, relative total error and RMSRE are metrics to evaluate the overall estimation errors. In our experiment, we choose a threshold that includes over 80% percent of traffics. Besides, both bias and variance are considered together to validate an available estimator [27]. 5.3. Simulation Results and Analysis We evaluate these algorithms based on the same traffic generation obtained by NS2. In the simulation, the initial value of u and v are in rage of 0.1-0.5, step size is 0.001 and other parameters of TM inference problem are configured based on Lemma 1. Simulation performance metrics are all described in (22)-(27). We first study these algorithms with respect to the CDF of RE from Fig.3 to Fig.6 with different time slices, and compare algorithms with respect to MRE in Fig.7. Then, we study relative total error with different time sequences in Fig.8 and Fig.9, and RMSRE in Fig.10 to evaluate the performance of overall estimation errors. Moreover, to evaluate the impact on robust and accuracy of proposed algorithms, Fig.11 plots bias and standard deviation in errors of different algorithms and Fig.12 depicts the effect of link data errors on estimation errors by adding different noise level on desirable link data. 1) The CDF of RE for three algorithms: We can observe from Figs.3 and 4 that TM estimation results of TMBCT, TMBLS and ICGA are quite different in cluster to cluster pair and ToR to ToR switch pair. ICGA and TMBCT perform better than TMBLS in cluster to cluster pair while TMBLS is better in ToR to ToR switch pair. This is because TMBCT and ICGA as a prior TM on every time slice, and TMBLS update traffic information instead. In addition, ICGA and TMBCT in Fig.3 produce estimation RE below 7% for nearly 80% traffic matrix while result drops approximately 70% for TMBLS. Fig.4 shows that more than 90% of estimation RE is below 8%, which is expected. These results mean that gravity model is more applicable in cluster to cluster TM estimation. 20

1

0.8

CDF

0.6 TMBLS

0.4

TMBCT ICGA

0.2

0 0

0.5

1

1.5

2

Relative L2 norm Error

Figure 3: The relative traffic estimation error of three algorithms in cluster to cluster pair

1

0.8

CDF

0.6 TMBLS

0.4

TMBCT ICGA

0.2

0 0

0.5

1

1.5

2

Relative L2 norm Error

Figure 4: The relative traffic estimation error of three algorithms in ToR to ToR switch pair

21

1

0.8

CDF

0.6

0.4

ICGA SAWP

0.2

0 0

0.5

1

1.5

2

Relative L2 norm Error

Figure 5: The relative error of two iterative algorithms in cluster to cluster pair 1

0.8

CDF

0.6

0.4

ICGA SAWP

0.2

0 0

0.5

1

1.5

2

Relative L2 norm Error

Figure 6: The relative error of two iterative algorithms in ToR to ToR switch pair

2) The CDF of RE for proposed algorithms: Figs.5 and 6 summarize the results of estimating TM by using ICGA and SAWP in cluster to cluster pair and ToR to ToR switch pair, respectively, with moderate link data missing. In the experiment, we randomly choose the 1% observed link data missing in different time periods. It shows that SAWP has nearly the same performance compared with the ICGA in cluster to cluster pair, because they both estimate TM between tomogravity space and gravity space with similar-Mahalanobis distance in decomposed DCNs. The only difference is that ICGA utilizes a prior TM calculated based on rough traffic characteristics while the SAWP makes no use of the extra information of ToR to ToR switch pair or cluster to cluster pair. In fact, the initial input for ICGA can be regarded as a result of single iteration of SAWP because prior TM is obtained from link data which is directly considered in tomographic space. 3) Mean relative error (MRE): From Fig.7, we can easily see that all of them 22

50 TMBLS 40

TMBCT

MRE

ICGA 30

SAWP

20

10

0 0

50

100

150

200

250

Number of Discrete Time

Figure 7: The mean relative error of four algorithms with different time periods 0.6 TMBLS TMBCT

Error Rate

0.5

ICGA SAWP

0.4

0.3

0.2 0

25

50

75

100

125

150

175

200

225

250

Number of Discrete Time

Figure 8: The relative total error in cluster to cluster pair with different time periods

can estimate TM in most of time. However, compared with TMBCT, ICGA and SAWP, TMBLS performs worse and its average accuracy is lower. Although TMBLS can well update new observations and capture dynamic characteristics of flows, it cannot well adapt rapid changes between flows and its small part of estimation results can lead to large deviation since traffics in DCN are fluctuating. In contrast, TMBCT and ICGA have good performance on MRE, because when traffic appears large deviation, it can be adjusted in time by similar-Mahalanobis distance. Moreover, the performance of SAWP is more stable due to a tradeoff between tomogravity space and gravity space. 4) Relative total error: it indicates that from Figs.8 and 9, ICGA and SAWP are better than TMBLS and TMBCT as the time decreases because similarMahalanobis distance are utilized to sense and control data errors. On the other hand, it shows that flows in DCNs are fluctuating. Although TMBLS is sensi23

0.6 TMBLS TMBCT ICGA SAWP

Error Rate

0.5

0.4

0.3

0.2 0

25

50

75

100

125

150

175

200

225

250

Number of Discrete Time

Figure 9: The relative total error in ToR to ToR switch pair with different time periods 0.8 CREATE

0.7

SAWP−cluster to cluster

RMSRE

0.6 SAWP−ToR to ToR 0.5 0.4 0.3 0.2 0

100

τ (Mb)

200

300

Figure 10: The RMSRE under different τ

tive to capture spatial-temporal characteristics of TM, recovering from frequent change of flows is slower compared to ICGA and TMBCT. In addition, compared to TMBLS, TMBCT has smaller relative total error because a prior TM is utilized in each time slice of TM estimation. 5) RMSRE: We compare the RMSRE of CREATE and SAWP under different τ in Fig.10 to evaluate overall estimation errors since both RMSRE and SAWP have moderate link missing. In CREATE, link utilization threshold is set to be 0.001. Fig.10 shows that SAWP and CREATE have lower RMSRE with the increasing of τ , because similar-Mahalanobis distance can well control estimation errors, and lower link utilization can be eliminated to reduce the complexity of TM estimation. But the performance of CREATE significantly depends on the link utilization threshold which can set non-zero link counts to zero. Besides, TM estimation with SAWP is more accurate in ToR to ToR switch pair than in cluster 24

4

x 107 DBN

3

TMBCT 2

TMBLS ICGA

Bias

1 0 −1 −2 −−3 −4 0

0.5

1

1.5

2

Standard Deviation in Error

2.5

3 x 109

Figure 11: Estimation bias versus standard deviation

to cluster pair. This is because ToR to ToR switch pair has more explicit structure than that in cluster to cluster pair. 6) Estimation bias and standard deviation: Since the accuracy of an estimator is a function of both its bias and variance, Fig.11 plots the bias versus the standard deviation for four algorithms to directly assess their impact on accuracy. It shows that ICGA and TMBCT have lower variance than TMBLS and DBN, and they generally exhibit higher bias. In contrast, TMBLS and DBN have higher variance with lower bias in general. Thus, ICGA achieves different tradeoff between bias and error variance. It can also be observed that the variances of ICGA, TMBCT and DBN are lower than that of TMBLS, which implies that they tend to estimate short term behavior of flows while TMBLS is prone to estimate long timescale OD pair. In addition, TMBLS is not sensitive enough with incisive jitters, thus it performs poorer in tracking sudden traffic changes. 7) Accuracy and robustness: To verify the effect of link data errors on estimation errors, we evaluate the performance of ICGA and SAWP by adding different noise level on link data in Fig.12. We induce the independent Gaussian white noise ε = N (0, σ) on the link data measurements, i.e., Y = Y · (1 + ε), where σ is the standard deviation of the white noise and · denotes an element-wise product. With the increase of the noise level, MRE of both algorithms is increasing both in cluster to cluster pair and ToR to ToR switch pair. However, it has less effect on the ToR switch pairs in the cluster as the noise level increases. This is because after decomposing the whole network into several clusters, the traffics in the cluster will be more stable. In addition, we can observe that ICGA performs better than SAWP, because it has a better prior TM. 25

0.5 SAWP−cluster to cluster 0.45

ICGA−cluster to cluster SAWP−ToR to ToR

MRE

ICGA−ToR to ToR 0.4

0.35

0.3 0

2

4

6

8

10

Noise level (%)

Figure 12: The MRE for ICGA and SAWP under different noise level in cluster to cluster pair and ToR and ToR switch pair respectively

6. Conclusions Since tomography based method applied in decomposed DCNs require a precise gravity model and data collection from SNMP results in unavoidable data missing and data errors, in this paper, we use the tomogravity based TM estimation method without requiring a good prior TM by replacing gravity model with gravity space, and take desirable link data obtained from packet trace and routing matrix as link load measurement. We propose two iterative algorithms, ICGA and SAWP, to estimate TM in DCNs by defining gravity space and tomographic space. We make first step to divide network structure to decompose TM estimation of whole network into several small TMs, which performs well in dealing with TM estimation in large DCNs. After decomposition, we can obtain a prior TM based on coarse-grained traffic characteristics, which will be taken as an initial input for ICGA. Then considering the situation where the link data missing or data errors in data collection, we redefine tomographic space and propose SAWP. In addition, we utilize similar-Mahalanobis distance to control the sensitivity to estimation errors. Compared with other existing algorithms, experimental results show that flow behaviors in the decomposed DCNs are relatively stable and both algorithms can effectively control data errors and are robust to noise measurements. 7. Acknowledgement This work was supported by the National Natural Science Foundation of China (No. 61772432, 61772433), Natural Science Key Foundation of Chongqing (cstc2015jcyjBX0094), Natural Science Foundation of Chongqing (CSTC2016JCYJA0449), 26

China Postdoctoral Science Foundation (2016M592619), Chongqing Postdoctoral Science Foundation (XM2016002), and the Fundamental Research Funds for the Central Universities (XDJK2015C010, XDJK2015D023, XDJK2016A011, XDJK2016D047, XDJK 201710635069). [1] G. Sun, V. Anand, D. Liao, C. Lu, X. Zhang, N. H. Bao, Power-efficient provisioning for online virtual network requests in cloud-based data centers, IEEE Systems Journal 9 (2) (2015) 427–441. [2] Z. Li, J. Ge, C. Li, H. Yang, H. Hu, B. Luo, V. Chang, Energy cost minimization with job security guarantee in internet data center, Future Generation Computer Systems 73 (C) (2017) 63–78. [3] V. Chang, Y. H. Kuo, M. Ramachandran, Cloud computing adoption framework: A security framework for business clouds, Future Generation Computer Systems 57 (2016) 24–41. [4] T. Benson, A. Anand, A. Akella, M. Zhang, Microte: Fine grained traffic engineering for data centers, in: Proceedings of the Seventh COnference on emerging Networking EXperiments and Technologies, ACM, 2011, pp. 8:1–8:12. [5] Q. Zhang, T. Chu, Structure regularized traffic monitoring model for traffic matrix estimation and anomaly detection, in: Control Conference, 2015, pp. 4980–4985. [6] M. H. Raza, A. Nafarieh, W. Robertson, Application of network tomography in load balancing, Procedia Computer Science 52 (1) (2015) 1120–1125. [7] G. Sun, V. Chang, G. Yang, D. Liao, The cost-efficient deployment of replica servers in virtual content distribution networks for data fusion, Information Sciences 000 (2017) (2017) 1–21, doi:10.1016/j.ins.2017.08.021. [8] G. Sun, D. Liao, S. Bu, H. Yu, Z. Sun, V. Chang, The efficient framework and algorithm for provisioning evolving vdc in federated data centers, Future Generation Computer Systems 2017 (73) (2016) 79–89. [9] F. Kong, X. Liu, L. Rao, Optimal energy source selection and capacity planning for green datacenters, Acm Sigmetrics Performance Evaluation Review 42 (42) (2016) 1–10. 27

[10] G. Sun, D. Liao, D. Zhao, Z. Sun, V. Chang, Towards provisioning hybrid virtual networks in federated cloud data centers, Future Generation Computer Systems 2017 (99) (2017) 1–15, doi:10.1016/j.future.2017.09.065. [11] A. S. Sohal, R. Sandhu, S. K. Sood, V. Chang, A. S. Sohal, R. Sandhu, S. K. Sood, V. Chang, A cybersecurity framework to identify malicious edge device in fog computing and cloud-of-things environments, Computers & Security 2017 (99) (2017) 1–15, doi:10.1016/j.cose.2017.08.016. [12] S. Anwar, J. M. Zain, M. F. Zolkipli, Z. Inayat, S. Khan, B. Anthony, V. Chang, From intrusion detection to an intrusion response system: Fundamentals, requirements, and future directions, Algorithms 10 (10) (2017) 1–24, doi:10.3390/a10020039. [13] S. M. Lee, D. S. Kim, J. H. Lee, J. S. Park, Detection of ddos attacks using optimized traffic matrix., Computers & Mathematics with Applications 63 (2) (2012) 501–510. [14] G. Sun, D. Liao, V. Anand, D. Zhao, H. Yu, A new technique for efficient live migration of multiple virtual machines, Future Generation Computer Systems 55 (2016) (2016) 74–86. [15] G. Sun, D. Liao, D. Zhao, Z. Xu, H. Yu, Live migration for multiple correlated virtual machines in cloud-based data centers, IEEE Transactions on Services Computing 2017 (99) (2017) 1–14, doi:10.1109/TSC.2015.2477825. [16] L. Cui, R. Cziva, F. P. Tso, D. P. Pezaros, Synergistic policy and virtual machine consolidation in cloud data centers, in: IEEE INFOCOM 2016 the IEEE International Conference on Computer Communications, 2016, pp. 1–9. [17] T. Benson, A. Akella, D. A. Maltz, Network traffic characteristics of data centers in the wild, in: Proceedings of the 10th ACM SIGCOMM conference on Internet measurement, ACM, 2010, pp. 267–280. [18] T. Benson, A. Anand, A. Akella, M. Zhang, Understanding data center traffic characteristics, ACM SIGCOMM Computer Communication Review 40 (1) (2010) 92–99.

28

[19] A. R. Curtis, W. Kim, P. Yalagandula, Mahout: Low-overhead datacenter traffic management using end-host-based elephant detection, Proceedings IEEE INFOCOM 2 (3) (2011) 1629–1637. [20] S. Kandula, S. Sengupta, A. Greenberg, P. Patel, R. Chaiken, The nature of data center traffic: measurements & analysis, in: Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference, ACM, 2009, pp. 202–208. [21] Y. Zhang, M. Roughan, N. Duffield, A. Greenberg, Fast accurate computation of large-scale ip traffic matrices from link loads, Acm Sigmetrics Performance Evaluation Review 31 (1) (2003) 206–217. [22] Z. Hu, J. Luo, Cracking network monitoring in dcns with sdn, in: 2015 IEEE Conference on Computer Communications (INFOCOM), IEEE, 2015, pp. 199–207. [23] Y. Qiao, Z. Hu, J. Luo, Efficient traffic matrix estimation for data center networks, in: IFIP Networking Conference, IEEE, 2013, pp. 1–9. [24] Z. Hu, Y. Qiao, J. Luo, Coarse-grained traffic matrix estimation for data center networks, Computer Communications 56 (2) (2015) 25–34. [25] Z. Hu, Y. Qiao, J. Luo, P. Sun, Y. Wen, Create: Correlation enhanced traffic matrix estimation in data center networks, in: 2014 IFIP Networking Conference, IEEE, 2014, pp. 1–9. [26] Y. Zhang, M. Roughan, W. Willinger, L. Qiu, Spatio-temporal compressive sensing and internet traffic matrices, in: 2009 ACM SIGCOMM Conference on Data Communication, 2009, pp. 267–278. [27] A. Soule, A. Lakhina, N. Taft, K. Papagiannaki, K. Salamatian, A. Nucci, M. Crovella, C. Diot, Traffic matrices: balancing measurements, inference and modeling, Acm Sigmetrics Performance Evaluation Review 33 (1) (2005) 362–373. [28] Y. Vardi, Network tomography: Estimating source-destination traffic intensities from link data, Journal of the American Statistical Association 91 (433) (1996) 365–377.

29

[29] Y. Tsang, M. Coates, R. Nowak, Passive network tomography using em algorithms, in: 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001. Proceedings (ICASSP), Vol. 3, IEEE, 2001, pp. 1469–1472. [30] H. Zhou, L. Tan, F. Ge, S. Chan, Traffic matrix estimation: Advancedtomogravity method based on a precise gravity model, International Journal of Communication Systems 28 (10) (2015) 1709–1728. [31] Z. Hu, Y. Qiao, J. Luo, Atme: Accurate traffic matrix estimation in both public and private datacenter networks, IEEE Transactions on Cloud Computing 2015 (99) (2015) 1–14, doi:10.1109/TCC.2015.2481383. [32] L. Nie, D. Jiang, L. Guo, S. Yu, H. Song, Traffic matrix prediction and estimation based on deep learning for data center networks, in: Globecom Workshops (GC Wkshps), IEEE, 2016, pp. 1–6. [33] J. Fang, Y. Vardi, C.-H. Zhang, An iterative tomogravity algorithm for the estimation of network traffic, Lecture Notes-Monograph Series 54 (2007) 12–23. [34] L. Nie, A novel network tomography approach for traffic matrix estimation problem in large-scale ip backbone networks, in: International Conference on Computer Science and Mechanical Automation, 2015, pp. 97–101. [35] L. Tan, H. Zhou, Tomofanout: a novel approach for large-scale ip traffic matrix estimation with excellent accuracy, Annals of Telecommunications 70 (3) (2015) 149–158. [36] L. Caggiani, M. DellOrco, M. Marinelli, M. Ottomanelli, A metaheuristic dynamic traffic assignment model for o-d matrix estimation using aggregate data, Procedia - Social and Behavioral Sciences 54 (2290) (2012) 685–695. [37] T. Djukic, G. Flotterod, H. Van Lint, S. Hoogendoorn, Efficient real time od matrix estimation based on principal component analysis, in: IEEE Conference on Intelligent Transportation Systems, 2012, pp. 115–121. [38] T. T. Tchrakian, B. Basu, M. O’Mahony, Real-time traffic flow forecasting using spectral analysis, IEEE Transactions on Intelligent Transportation Systems 13 (2) (2012) 519–526. 30

[39] L. Nie, D. Jiang, L. Guo, S. Yu, Traffic matrix prediction and estimation based on deep learning in large-scale ip backbone networks, Journal of Network and Computer Applications 76 (2016) 16–22. [40] A. Abadi, T. Rajabioun, P. A. Ioannou, Traffic flow prediction for road transportation networks with limited traffic data, IEEE Transactions on Intelligent Transportation Systems 16 (2) (2015) 653–662. [41] M. T. Asif, N. Mitrovic, J. Dauwels, P. Jaillet, Matrix and tensor based methods for missing data estimation in large traffic networks, IEEE Transactions on Intelligent Transportation Systems 17 (7) (2016) 1816–1825. [42] M. Al-Fares, A. Loukissas, A. Vahdat, A scalable, commodity data center network architecture, ACM SIGCOMM Computer Communication Review 38 (4) (2008) 63–74. [43] C. Guo, H. Wu, K. Tan, L. Shi, Y. Zhang, S. Lu, Dcell: a scalable and fault-tolerant network structure for data centers, ACM SIGCOMM Computer Communication Review 38 (4) (2008) 75–86. [44] A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, S. Sengupta, Vl2: a scalable and flexible data center network, in: 2009 ACM SIGCOMM Conference on Data Communication, 2009, pp. 51–62. [45] G. Wang, D. G. Andersen, M. Kaminsky, K. Papagiannaki, T. Ng, M. Kozuch, M. Ryan, c-through: Part-time optics in data centers, ACM SIGCOMM Computer Communication Review 41 (4) (2011) 327–338. [46] N. Farrington, G. Porter, S. Radhakrishnan, H. H. Bazzaz, V. Subramanya, Y. Fainman, G. Papen, A. Vahdat, Helios: a hybrid electrical/optical switch architecture for modular data centers, ACM SIGCOMM Computer Communication Review 41 (4) (2011) 339–350. [47] C. D. C. Infrastructure, 2.5 design guide, http://goo.gl/kBpzgh (2007). [48] Y. Zhang, M. Roughan, C. Lund, D. Donoho, An information-theoretic approach to traffic matrix estimation, in: Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, ACM, 2003, pp. 301–312. 31

[49] The network simulator - ns-2, http://www.isi.edu/nsnam/ns/ (2016). [50] C. Hopps, Analysis of an Equal-Cost Multi-Path Algorithm, RFC Editor, 2000. [51] A. Gunnar, M. Johansson, T. Telkamp, Traffic matrix estimation on a large ip backbone: a comparison on real data, in: Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, ACM, 2004, pp. 149–160.

32

Author Biography    Guiyan  Liu  received  the  B.S.  degree  in  telecommunications  engineering  from  Southwest  University, Chongqing, China, in 2014. She is currently working toward the PhD's degree in signal  and  information  processing,  Southwest  University.  Her  research  interests  include  stream  scheduling in data center networks and software defined networking.    Songtao  Guo  received  the  BS,  MS,  and  PhD  degrees  in  computer  software  and  theory  from  Chongqing  University,  Chongqing,  China,  in  1999,  2003,  and  2008,  respectively.  He  was  a  professor  from  2011  to  2012  at  Chongqing  University.  He  is  currently  a  full  professor  at  Southwest  University,  China.  He  was  a  senior  research  associate  at  the  City  University  of  Hong  Kong  from  2010  to  2011,  and  a  visiting  scholar  at  Stony  Brook  University,  New  York,  from  May  2011  to  May  2012.  His  research  interests  include  wireless  sensor  networks,  wireless  ad  hoc  networks  and  parallel  and  distributed  computing.  He  has  published  more  than  30  scientific  papers in leading refereed journals and conferences. He has received many research grants as a  principal  investigator  from  the  National  Science  Foundation  of  China  and  Chongqing  and  the  Postdoctoral Science Foundation of China.    Quanjuan Zhao received B.S. degree in electronic and information engineering in Henan Normal  University  in  2005  and  master's degree in communication and information system  from  Chongqing University of Posts and Telecommunications in 2008. He is currently working towards  to  PhD  in  Intelligence  Information  Processing  from  Southwest  University.  His  interests  include  streaming  transmission  in  data  center  networks  and  controller  assignment  in  software  defined  networking.    Yuanyuan  Yang  received  the  BEng  and  MS  degrees  in  computer  science  and  engineering  from  Tsinghua  University,  Beijing,  China,  and  the  MSE  and  PhD  degrees  in  computer  science  from  Johns Hopkins University, Baltimore, Maryland. She is a professor of computer engineering and  computer  science  at  Stony  Brook  University,  New  York.  Her  research  interests  include  wireless  networks,  data  center  networks,  optical  networks  and  high‐speed  networks.  She  has  published  over  300  papers  in  major  journals  and  refereed  conference  proceedings  and  holds  seven  US  patents in these areas. She has served as an Associate Editor‐in‐Chief and an Associated Editor for  IEEE  Transactions  on  Computers  and  an  Associate  Editor  for  IEEE  Transactions  on  Parallel  and  Distributed  Systems.  She  has  also  served  as  a  general  chair,  program  chair,  or  vice  chair  for  several major conferences and a program committee member for numerous conferences. She is  an IEEE Fellow.                 

  Guiyyan    Liu   

  Songgtao Guo                      Quanjun Zhao   

  Yuan nyuan Yang 

Highlights    (i)

We  decompose  DCNs  into  several  clusters  to  easily  handle  TM  estimation  so  that  coarse‐grained traffic characteristics can be well obtained using SNMP link data;  (ii) We  present  two  iterative  algorithms  of  TM  estimation  by  defining  probability  vectors  of  tomographic space and gravity space;    (iii) We utilize similar‐Mahalanobis distance to control sensitivity to estimation errors between  two spaces;  (iv) Simulation results demonstrate that our proposed two algorithms are effective to control  the estimation errors in data center networks.