Adaptive bearer split control for 5G multi-RAT scenarios with dual connectivity

Computer Networks 161 (2019) 183–196

Contents lists available at ScienceDirect

Computer Networks journal homepage: www.elsevier.com/locate/comnet

Adaptive bearer split control for 5G multi-RAT scenarios with dual connectivity Roberto P. Antonioli∗, Emanuel B. Rodrigues, Diego A. Sousa, Igor M. Guerreiro, Carlos F.M. e Silva, Francisco R.P. Cavalcanti Wireless Telecommunications Research Group (GTEL), Federal University of Ceará Av. Mister Hull, CP 6005, Fortaleza, 60455-760, Ceará, Brazil

a r t i c l e

i n f o

Article history: Received 30 April 2018 Revised 3 January 2019 Accepted 3 July 2019 Available online 3 July 2019 Keywords: 5G Mobile networks Long term evolution Dual connectivity Multiple radio access technologies Bearer split control

a b s t r a c t The Dual Connectivity (DC) technology has gained a lot of momentum as an underlying fundamental concept for the deployment of multi-Radio Access Technology (RAT) 5G mobile networks based on the tight interworking between the Long Term Evolution (LTE) and 5th Generation (5G) standards. In the so-called bearer split configuration of the DC technology, a flow control mechanism executed at a Master Node (MN) controls the bearer split ratios such that the correct amount of data is transmitted from the MN and Secondary Nodes (SNs) to enhance the Quality of Service (QoS) provision. In this context, the present work proposes a flow control algorithm that maximizes the user satisfaction by dynamically orchestrating the bearer split ratios based on the adaptation of utility functions. The adaptation is performed according to a predefined network operator goal, such as maintaining a certain QoS level at a given network node. Furthermore, the proposed solution tracks the channel and QoS metrics variations such that the best split ratios are employed. Simulations conducted in 5G multi-RAT scenarios show that the adaptability of the proposed algorithm effectively maximizes the user satisfaction as well as enhances the users and system throughput when compared to state-of-the-art algorithms from the literature. © 2019 Elsevier B.V. All rights reserved.

1. Introduction The explosion in mobile traffic volume, the enriched service scope and the ever increasing number of connected devices in mobile networks led industry and academia to develop the conceptual vision and engineering requirements of the next generation of mobile networks. The 5G of mobile networks are expected to support a wide range of applications/services with diverse requirements, which pose new challenges on the current network technologies and in terms of designing efficient and flexible resource management techniques to meet the diversified demands. The applications expected for the 5G era have been categorized in three different use cases by the International Telecommunication Union (ITU) [1]: • Enhanced Mobile Broadband (eMBB): which includes services that will require seamless multi-connectivity across different RATs operating over a wide range of frequency bands, and require very high throughputs and large bandwidths, such as 4K and 3D video, virtual and augmented re-

∗

Corresponding author. E-mail addresses: [email protected] (R.P. Antonioli), [email protected] (E.B. Rodrigues), [email protected] (D.A. Sousa), [email protected] (I.M. Guerreiro), [email protected] (C.F.M. e Silva), [email protected] (F.R.P. Cavalcanti). https://doi.org/10.1016/j.comnet.2019.07.005 1389-1286/© 2019 Elsevier B.V. All rights reserved.

ality, etc. Therefore, this category is focused on meeting people’s demand for an ever increasing digital lifestyle; • Ultra-Reliable and Low-Latency Communications (URLLC): that comprises vehicular communication (e.g., to support autonomous cars) and remote control (e.g., remote robotics, medical surgery or tactile Internet), which are applications related to the digital industry and demand very low latency, and very high reliability and availability; • Massive Machine-Type Communications (mMTC): that involves applications for a further developed digital society with a large number of connected devices transmitting small amounts of data such as in smart cities and smart homes/buildings, which are characterized by requiring low bandwidth, high connection density, enhanced coverage, and low energy consumption. Since the 5G networks must be able to support these diversified requirements imposed by these envisioned 5G era services/applications, it is expected that the 5G air interface, named New Radio (NR), will be deployed in a cooperative manner with existing standards (such as LTE and Wi-Fi) so that multiconnectivity technologies are incorporated [2]. Therefore, in the 5G heterogeneous scenarios, a tight interworking is expected between different RATs, allowing User Equipments (UEs) and devices to

184

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

benefit from the DC technology. This technology was standardized under the umbrella work of small cell enhancements in the 3rd Generation Partnership Project (3GPP) LTE Release 12 [3], where UEs with DC capabilities could connect to two LTE Evolved Node Bs (eNBs) operating in different carriers. More recently, in the 3GPP Release 14 [4], the DC technology is discussed already as a 5G operational requirement, where the multi-connectivity might be established with LTE eNBs and NR gNode Bs (gNBs).1 Considering LTE-NR multi-RAT scenarios, the tight interworking could be performed by the provision of a larger coverage region, supplied by the legacy LTE eNBs, to the base stations using the NR RAT [5]. The LTE connection is required to maintain data delivery continuity since the LTE eNBs provide a wide and more reliable coverage region. On the other hand, the NR connections offer improved capacity in hotspot areas and offload some traffic from the LTE eNBs [6]. In this scenario, which is the focus of the present work, the LTE eNB is referred to as MN, while NR gNB is denominated as SN. According to the agreements established in [3] and further discussed in [4], a promising DC configuration to be employed in future 5G networks is the so-called bearer split configuration. In the network architecture supporting this configuration, there is a backhaul link connecting the MN to the SNs, which in practical scenarios is non-ideal, being characterized by a certain latency and limited capacity [7]. Since in this configuration only the LTE MN has a link to the core network, the term Non-Stand-Alone (NSA) 5G is also being used indicating that the NR SN only works using the backhaul link to the LTE MN. Furthermore, in the bearer split configuration, as all the traffic from split bearers come from the core network only to the MN, the processing and routing of split bearers traffic is performed by the MN by means of a flow control algorithm located at its Packet Data Convergence Protocol (PDCP) layer. This mechanism is responsible for balancing the amount of data that should be forwarded to the SN via the backhaul link and the amount of data transmitted by the MN to the UE. The flow control algorithm for bearer split traffic should be efficient and clever in such a way to neither overload the transmission buffer of either MN or SN, nor underutilize the MN and SN available radio resources. Furthermore, the algorithm should enhance the QoS/Quality of Experience (QoE) provision such that the challenging unprecedented extremely high throughput, low latency and ultra-reliability requirements are met. To accomplish these objectives, the flow control algorithm might consider diverse system and network parameters such as buffer status, channel quality and backhaul characteristics. Additionally, the users’ QoS metrics should also be taken into account, such as throughput, packet delay and jitter. 1.1. Background on similar technologies There are two main existing technologies which have similar concepts to the DC technology, which are known as Coordinated Multi-Point (CoMP) [8–10] and Carrier Aggregation (CA) [11–13]. This section briefly describes these two technologies and highlights the main differences between them and the DC technology, which is the focus of this work. In the 3GPP LTE-Advanced standard, the CoMP technology was considered as an enabling technology to actively deal with the intercell interference and to increase the cell edge user throughput [8,9]. The CoMP can operate basically on three different modes [10]: (I) Joint Processing (JP) or Joint Transmission (JT), where two different Base Stations (BSs) cooperate to transmit the

1 The term gNB was introduced in [4] for base stations supporting the NR RAT and connectivity to the Next Generation Core (NGC).

same signal to a given UE on the same time-frequency resources; (II) Coordinated Scheduling (CS) or Coordinated Beamforming (CB), where the signal for a given UE is transmitted from a given BS while the interference from other BSs is coordinated; (III) Dynamic Point Selection (DPS), where a UE receives signal from only one BS, but the UE can dynamically change its association to another BS within a set of possible BSs in a cluster. The JP/JT mode is the most similar to the DC technology. However, in the DC technology, the UE is connected simultaneously to two different BS operating on different frequencies (possible from different RATs) and simultaneously receives different data from the involved BSs. Thus, the DC technology is substantially different from the JP/JT mode from the CoMP technology. For a comparison study between CoMP and DC, the reader is referred to Ramamoorthi and Kumar [14]. Also in the 3GPP LTE-Advanced standard, the CA technology was introduced as a means to enhance the user throughput by aggregated bandwidth from two different carriers [11–13]. In fact, the CA technology has more similarities with the DC technology than the CoMP technology. The intra-site CA can be understood as a particular case of the DC technology [15], where the UE is connected to multiple component carriers from a single BS, thus there is no latency in the communication between the entities responsible for managing the different carriers. There is also another option known as the inter-site CA [16], which consists of a deployment where small cells are realized with remote radio heads and a centralized base band processing unit at the macro cell communicates with the small cells via virtually zero latency high-speed fiber-based fronthaul connections [15]. In this latter case of the CA technology, all the Radio Resource Management (RRM) functionalities are performed at the macro cell, which highlights the importance of having a zero latency connection. Thus, one of the main differences between CA and DC is that in the latter there is a realistic backhaul link connecting the involved BSs (possibly from different RATs), which in practical scenarios is non-ideal, being characterized by a certain latency and limited capacity [7]. Another difference is that in the deployment based on the DC technology, the RRM functionalities are performed independently by each BS. This means that, by considering the bearer split configuration of the DC technology, this work is considering a more realistic and practical scenario, which requires a bearer split algorithm to control the data traffic sent to the SN. For the interested reader, performance comparisons between the CA and DC technologies can be found in [15,16]. 1.2. Related works and contributions Due to the short time since the disclosure of the 3GPP Release 14, where the DC technology is discussed in multi-RAT 5G scenarios, as far as we are concerned, no other work in the literature has proposed and analyzed a flow control mechanism for bearer split in a heterogeneous scenario composed of LTE eNBs and NR gNBs. There are, however, a limited number of studies in the literature for the scenario with LTE eNBs operating on different carriers. In [17], a mechanism is discussed where there is a fixed percentage, x%, of data that the MN sends to the SN via the backhaul link, and another fixed percentage (100-x)% is sent by the MN to the UE. However, a fixed mechanism would not be able to correctly handle the split bearers’ traffic in all possible scenarios. A dynamic mechanism based on the SN radio capacity and backhaul latency is proposed in [18]. In [15], a request-and-forward algorithm is proposed, where the SN sends data requests to the MN based on the users throughput, and its target buffering time and radio capability. The main objective of the algorithm proposed in [15] is to keep a certain amount of data buffered at the SN to be transmitted when users in DC are scheduled there. In [19], a scheme of flow control and traffic scheduling designed as a Mixed Integer Linear

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

Programming (MILP) is proposed aiming at maximizing the network throughput; the authors in [19] have discussed just an optimal solution for their optimization problem, no low-complexity algorithm has been proposed therein. Other works consider the integration between LTE and Wi-Fi, such as in [20], where the authors proposed a flow control mechanism with fairness considerations, and in [21], where the authors proposed an algorithm that computes the estimated transmission time on each link and directs the packets to the link with lowest estimated transmission time. 5G heterogeneous scenarios are considered in [22], where the authors presented a load balancing technique through user association, but the DC technology was not taken into account therein. Therefore, as mentioned before, none of the works found in the literature consider multi-connectivity multi-RAT 5G scenarios. Furthermore, these works focused on limiting the buffering time at the SN or maximizing the network throughput and not on meeting the users challenging and stringent QoS requirements such that the user satisfaction in the system is maximized. In this context, the main contributions of this work are: 1. Formulation of a utility-based optimization problem with the objective of maximizing the total utility in systems powered by the DC technology. 2. Derivation of a suboptimum solution which is a centralized joint flow control and resource allocation solution to be employed in cloud-based wireless networks. 3. Reinterpretation of the centralized suboptimum solution for the development of a flow control algorithm for practical multi-connectivity scenarios based on the bearer split configuration. 4. Proposal of a utility-based flow control algorithm, which is executed at the MN and targets to maximize the users’ utility derived from the network (users’ satisfaction) by dynamically adapting utility functions. From this adaptation, the MN finds the most adequate bearer split ratio that maximizes the users’ satisfaction by considering the users’ QoS metrics and channel conditions. To perform this calculation, the MN expects the SN to periodically report bearer-specific QoS metrics and channel conditions of UEs via the backhaul link. By using the information available at the MN and sent by the SN, the proposed solution is able to track the channel and QoS metrics variations such that the best split ratio is employed for each bearer. 5. Performance evaluation conducted in a system-level simulator considering LTE-NR multi-RAT scenarios, where the simulation parameters are all aligned with the 3GPP recommendations. Analyses are conducted considering perfect and imperfect backhaul links. 1.3. Paper organization

185

employing the LTE technology are equipped with a Uniform Linear Array (ULA) composed of θ antenna elements, while the NR BSs are equipped with a Uniform Rectangular Array (URA) comprised of φ antenna elements. The UEs are equipped with a single antenna. We assume that the BSs are using digital beamforming and that the number of Radio Frequency (RF) chains is equal to the number of antenna elements so that we can have as many beams active per time slot as the number of antenna elements. However, only one beam is active on each allocable resource per time instant. All BSs are using Discrete Fourier Transform (DFT) precoding to generate the beams radiation patterns [24]. The minimum allocable resource considered herein is a Resource Block (RB), which is a time-frequency chunk comprised of a given number of Orthogonal Frequency Division Multiplexing (OFDM) symbols in the time domain and composed of a number of adjacent subcarries in the frequency domain. The duration on the time domain of a RB is equal to the Transmission Time Interval (TTI), which has a different duration depending on the RAT. Each BS b disposes of a set of RBs Kb = {1, 2, . . . , Kb } to be assigned to the group of UEs Jb = {1, 2, . . . , Jb } connected to it. The total  sets of RBs and UEs in the system are given by K = b∈B Kb and  J = b∈B Jb , respectively, where |J | = J and |K| = K. The channel transfer function between the user j on RB k and the transmit antenna m of BS b at TTI n is represented by hj,k,b,m [n], which is considered to be the transfer function of the mid subcarrier that composes the RB k. The hj,k,b,m [n] values are calculated considering the main propagation characteristics of wireless channels (path loss, shadowing and small-scale fading). It is assumed that the BS b disposes of a total available power PT,b to be distributed among all allocated RBs during the transmission time, such that the power allocated by the BS b to the RB k is given by pk,b . The Signal to Interference-plus-Noise Ratio (SINR) of user j on RB k of the BS b at TTI n is given by γ j,k,b [n]. The BS employs a link adaptation mechanism that allows different transmission rates depending on the γ j,k,b value, which is used by the BS b to select from a set of 15 Modulation and Coding Schemes (MCSs), the one that allows the UE to transmit the highest data rate given that a predefined target BLock Error Rate (BLER) is met. Therefore, the rate allocated by the BS b to the user j on RB k is r j,k,b = f (γ j,k,b ), where f(·) is a link adaptation function. The total rate allocated to the UE j by the BS b at TTI n is given by

R j,b [n] =



r j,k,b [n]

(1)

k∈K j,b

where K j,b ⊂ Kb is the subset of RBs allocated to user j by the BS b at TTI n. The maximum possible achievable rate for a user j connected to the BS b at TTI n is

Rˆ j,b [n] =



r j,k,b [n].

(2)

k∈Kb

The remainder of this work is organized as follows. Section 2 presents the network modeling and Section 3 introduces the problem formulation along with the derived suboptimum solution. The performance evaluation and comparison of the proposed and the baseline solutions are conducted via extensive system-level simulations in Section 4. Finally, in Section 5, the main conclusions are drawn.

Notice that Rˆ j,b [n] represents the case where all RBs k ∈ Kb of the BS b would be assigned to a single user j, i.e., the maximum rate that user j can achieve, which is a measurement of the quality of the channel link between user j and BS b. 3. Problem formulation and suboptimum solution 3.1. General formulation

2. System modeling Let us consider the downlink of a 5G multi-RAT scenario following the characteristics of the dense urban scenario presented in [23]. The group of all BSs is organized in a set, represented by B = {1, 2, . . . , B}, where each one of them can employ only one RAT, i.e., either the BS employs the NR RAT or the LTE RAT. The BSs

As aforementioned, the flow control algorithms for the bearer split configuration will play an important role in future 5G mobile networks deployed based on this configuration. These algorithms are responsible for orchestrating the traffic division of split bearers such that neither the transmit buffers of both MN and SN run empty nor are overloaded. The effectiveness of such algorithms are

186

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

of utmost importance so that the benefits of enhanced QoS provided by the DC technology are perceived by the mobile users. In this context, we formulate a utility-based optimization problem with the objective of maximizing the total user utility derived from the network. In other words, our main objective is to maximize the total user satisfaction, where the concept of user satisfaction relates to meeting the users’ QoS demands. Using the definitions of the previous section, the proposed general optimization problem is formulated as follows

max

ρ j,k,b , pk,b



(3a)

b∈B j∈Jb

subject to 

   

M V U xj

ρ j,k,b ∈ {0, 1}, ∀ j ∈ J , ∀k ∈ Kb and ∀b ∈ B,

(3b)

ρ j,k,b ≤ 1, ∀k ∈ Kb and ∀b ∈ B,

(3c)

∀b ∈ B

(3d)

j∈J



pk,b ≤ PT,b ,

k∈Kb

pk,b ≥ 0,

∀k ∈ K,

the MNs and SNs perform their RRA independently. This suboptimal solution is obtained following the same utility-based mathematical assumptions presented in [25], where the authors derived simplified problems with a closed form solution by dividing the user set J in different service classes. Also, in order to derive simplified problems, the authors in [25] used a problem-splitting approach, where firstly a Dynamic Resource Assignment (DRA) is performed with fixed power allocation; then a stage of Equal Power Allocation (EPA) with fixed resource assignment is performed. Similar approaches based on the utility theory have also been used in [26–28]. We consider a realistic scenario composed of throughput-based, queue-based and delay-based services, which can be exemplified by web-browsing, video and Voice over IP (VoIP) services, respectively. Consequently, the optimization problem in (3) can be reformulated as the maximization of the total utility with respect to the users’ QoS, namely throughput (x j = T j ), average queue size (x j = Q j ) and Head Of Line (HOL) packet delay (x j = dhol ) for j throughput-based, queue-based and delay-based services, respectively. Therefore, let us assume that the set Jb of the users connected to BS b is separated in three subsets: Jthrb , Jqueueb and Jdelayb for throughput-based, queue-based and delay-based users, respectively. Then, the objective function of (3) is rewritten as

(3e)

where: ρ j,k,b is an assignment variable that assumes the value 1 if the RB k of the BS b is assigned to the user j and 0 otherwise; U(xj ) is a user utility function based on a generic variable xj that represents a QoS metric associated to user j; V(·) is a service utility function that depends on the user utility functions and differentiates the services in the system; and M(·) is an innovative RAT utility function that depends on the service and user utility functions and differentiates the multiple RATs in the system. Constraints (3b) and (3c) state that the RBs of each BS are discrete and that the same RB of a given BS cannot be shared by two or more users in the same TTI. Constraints (3d) and (3e) require that the total sum of the powers over all RBs must not surpass the total transmit power of the BS, and that these powers must be non-negative. Given that the UE-BS connections have been established by any known mechanism (e.g., according to the Reference Signal Received Power (RSRP) or Reference Signal Received Quality (RSRQ) measurements from each UE [3]), i.e., the Jb , ∀ b ∈ B are initialized, the optimal solution for this optimization problem represents a joint flow control and Radio Resource Allocation (RRA) algorithm performed in a cloud-based Radio Access Network (C-RAN) considering that there is no latency in the backhaul links. This means that the cloud (or any type of centralized unit) would determine the amount of traffic that goes to the transmit buffers of all MNs and SNs as well as the RBs and power allocations that should be performed to maximize the user satisfaction in the system. However, this solution would only work in the ideal case where the cloud has instantaneous and updated access to the users’ QoS metrics and channel quality indicators, which makes the centralized approach intractable for future scenarios. Furthermore, it is well known in the literature that optimization problems with the structure of (3) are NP-hard. Nevertheless, note that deciding the amount of assigned RBs from each BS to each UE is similar to determining the amount of data that each UE receives from each BS since a specific amount of data is transmitted on each RB. In other words, adding up the amount of data transmitted on each RB for each UE would give the amount of data received from each BS by each UE. Therefore, using this insight, a suboptimal solution to the formulated problem can be interpreted as a flow control algorithm to be deployed in a MN operating on the bearer split configuration along with one or several SNs. Then, after the execution of the flow control algorithm,

max ρ j,k,b





 

 j∈Jthr,b

b∈B



+





M V Uthr T j [n]

 



 





M V Uqueue Q j [n + 1]

j∈Jqueue,b



+

M V

 

Udelay dhol j [n ]

,

(4)

j∈Jdelay,b

while the constraints of problem (3) remain the same. The authors in [25,28] presented mathematical models for calculating the HOL packet delay, the queue size and throughput. These models are described below. The throughput of user j at TTI n is calculated using an exponential smoothing filtering, as indicated below:

T j [n] = (1 − fthru ) · T j [n − 1] + fthru · R j [n],

(5)

where fthru is a filtering constant. The average queue size over a time window of user j is calculated also using an exponential smoothing filtering, as indicated below:

Q j [n] = (1 − fqueue ) · Q j [n − 1] + fqueue · Q j [n],

(6)

where Qj [n] is the instantaneous queue size of user j and fqueue is a filtering constant. The queue size of user j at TTI n + 1 is expressed as [25,26,28]

Q j [n + 1] = Q j [n] − R j [n] · ttti + α j [n],

(7)

where α [n] is the amount of bits that arrive during TTI n. At the beginning of TTI n, given Rj [n], the predicted average queue size at  the beginning of TTI n + 1 is obtained by Eα j [n] Q j [n + 1] , which is the expectation with respect to α j [n] [26]. According to (6) and (7), we have





Eα j [n] Q j [n + 1] = (1 − fqueue ) · Q j [n]



+ fqueue · Q j [n] − R j [n] · ttti + E





  α j [n] ,

(8)

where E α j [n] = ω j · ttti , and ωj is the source data rate of the service consumed by user j. Finally, we consider a recursive model for calculating an approximate value of the HOL packet delay [25,28]. The recursive equation is hol dhol j [n + 1] = d j [n] + ttti −

1 · L



R j [n] · ttti Sp

,

(9)

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

where ttti is the duration of the TTI in seconds, L is the packet arrival rate, Sp is the packet size. Let us now analyze the objective function (4) for each set of users. Starting from the throughput-based users, after replacing (5) into (4), one can compute the derivative of M{V[Uthr (Tj [n])]} with respect to the transmission rate Rj by simply applying the chain rule from the theory of derivatives. Using the one orderorder Taylor approximation in the equation obtained after the derivative process and analyzing the constant terms that do not affect the value of the objective function [25,28], we arrive at a simplified objective function with respect to the throughput-based users given by

max ρ j,k,b

 



 





M V Uthr T j [n − 1]

b∈B j∈Jthr,b 





·



V Uthr T j [n − 1] ·







Uthr T j [n − 1] · R j [n].

(10)

The { } symbol represents the first derivative of the function with respect to its input, which in this case is the T j [n − 1]. Following a similar procedure for the queue-based users, we can (4), and then compute the derivative of  replace (8) into M V Uqueue Q j [n + 1] with respect to the transmission rate Rj by simply applying the chain rule from the theory of derivatives. Using the one order-order Taylor approximation in the equation obtained after the derivative process and analyzing the constant terms that do not affect the value of the objective function [25,28], we arrive at a simplified objective function with respect to the queue-based users given by

max ρ j,k,b

 



 





·

M V Uqueue Q j [n]

b∈B j∈Jthr,b 







V Uqueue Q j [n] ·

    Uqueue Q j [n]  · R j [n].

(11)

The absolute value operator is used to cancel the negative sign because the utility function (as detailed in Section 3.3) is assumed to be decreasing, which yields negative marginal utilities. Finally, for the delay-based users, we (9) into  can  replace   (4), and then compute the derivative of M V Udelay dhol n j [ ]

with

respect to the transmission rate Rj by simply applying the chain rule from the theory of derivatives. Using the Lagrange theorem of the mean in the equation obtained after the derivative process and analyzing the constant terms that do not affect the value of the objective function [25,28], we arrive at a simplified objective function with respect to the delay-based users given by

max ρ j,k,b

 



 





M V Udelay dhol j [n ]

b∈B j∈Jthr,b 





·



V Udelay dhol j [n ] ·

   hol  Udelay d j [n]  · R j [n].

(12)

The absolute value operator is used to cancel the negative sign because the utility function (as detailed in Section 3.3) is assumed to be decreasing, which yields negative marginal utilities. Note that the addition of the Eqs. (10)–(12) now composes the new and simplified objective function of problem (3), while the constraints remain the same. Such simplified optimization problem, which is linear in terms of Rj,b [n], has a closed form solution that selects the user j connected to BS b to transmit on RB k in TTI n according to [25,28,29]:





 jk,b [n] = arg max wbj · wsj · w j · r j,k,b [n] , j

(13)

where: wbj is the utility-based RAT weight associated to user j connected to the BS b; wsj is the utility-based service weight associ-

187

ated to user j and service s; and wj is the utility-based user weight associated to user j. Considering a centralized processing unit with access to the whole pool of RBs from all BSs, Eq. (13) performs a joint flow control and RRA algorithm by computing one weight per UE per RB per BS. Then, the UE with the highest weight on each RB from each BS is assigned to that RB for the data transmission. This RB assignment comprises the RRA functionality. Notice that for UEs in DC, weights from Eq. (13) are computed for each RB from their MN and SN. From the rate assigned to UEs in DC on both connections, which can be computed by multiplying the amount of assigned RBs times the achievable rate on each RB, we also have the amount of data that should be sent to each BS, which represents the flow control algorithm. From Eqs. (10), (11) and (12), the utility-based weights wbj , wsj and wj are mathematically given by

wbj =

wsj =

⎧     ] , if j ∈ Jthr,b ⎨M V Uthr Tj [n − 1 

M V Uqueue Q j [n] , ⎩M V U dhol n , delay j [ ]

if j ∈ Jqueue,b if j ∈ Jdelay,b .

⎧    ] , if j ∈ Jthr,b ⎨V Uthr Tj [n − 1 

V Uqueue Q j [n] ,  hol  delay d j [n] ,

⎩V  U

if j ∈ Jqueue,b if j ∈ Jdelay,b .

⎧    ⎨U  thr Tj [n − 1], if j ∈ Jthr,b , if j ∈ Jqueue,b w j = Uqueue Q j [n]  ⎩U  hol d j [n] , if j ∈ Jdelay,b . delay

(14)

(15)

(16)

Notice that the suboptimal solution in (13) is a lower complexity joint flow control and RRA mechanism to be performed in a centralized unit. In this case, all the values rj,k,b and the bearerspecific QoS metrics at each time instant would need to be available in the centralized unit to perform the joint algorithm, which introduces a tremendous signaling overhead in the system. 3.2. Reinterpretation of suboptimum solution Let us now give another interpretation to the solution presented in (13). In a practical scenario deployed based on the bearer split configuration, one or several SNs have a connection via internode interface with the MN. As agreed in [3,4] and assumed in [15,19], the internode interface is used by the MN and SNs for exchanging information in order to perform the flow control algorithm and other resource management related functionalities. Relying on this realistic assumption, we can use (13) as a flow control mechanism to be employed in the MN. In this context, the SNs would send to the MN the QoS metrics of split bearers and the rj,k,b values of the UEs using those bearers. This specific information is already calculated for the execution of the RRA algorithms such that the SNs would only need to report them to the MN. In fact, in order to reduce the signaling, the SNs need to send only the Rˆ j,b [n] values (given in Eq. (2)), which would be only one single value (instead of Kb values) per UE. Other possible metric to be sent instead of Rˆ j,b is any kind of channel quality indicator, such as the RSRQ or Received Signal Strength Indicator (RSSI). Upon receiving the split bearers information from the SNs, the MN performs the flow control algorithm, which is located at the MN PDCP layer [4]. Notice that the same type of split bearers information reported by the SNs needs to be gathered by the MN PDCP from the MN lower layers, since the QoS metrics and channel quality indicators are not usually available at the PDCP layer. Let us give more details about the reinterpretation from Eq. (13) such that we can derive the utility-based weights for the proposed bearer split algorithm. Eq. (13) computes the weight per UE per RB per BS to be used during the resource allocation considering a centralized unit. Then, the UE with the highest weight on each RB from

188

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

each BS is assigned to that RB for the data transmission. Also, by adding up the achievable rates on each RB from each BS assigned to a given UE j, we are able to obtain the total rate assigned to UE j on that BS. Based on this observation, it is possible to replace the achievable rates on each RB from each BS to a single value of total rate achieved on each BS, which is given by Rˆ j,b . Using such replacement, instead of computing one weight per UE per RB per BS, we only compute one weight per UE per BS, such that we directly obtain the total rate assigned to a given UE j on BS b. Since each UE has at most two simultaneous connections, we only need to compute one weight for the MN and one weight for the SN for each UE. Furthermore, the expression from Eq. (13) is now used for determining the bearer split ratios, and not anymore for computing weights for the resource allocation. Thus, we can also drop the arg max j {·} operator because we only need the total rate that should be given to each UE on the MN and on the SN. The flow control algorithm is then performed using only the expression wbj · wsj · w j times Rˆ j,b taken from Eq. (13). The MN calculates two weights for each split bearer transporting data to user j, given by

wMN = wbj · wsj · w j · Rˆ j,b j

(17)

and b s ˆ wSN j = w j · w j · w j · R j,b ,

(18)

where wMN and wSN are the resultant weights for the MN and SN, j j respectively. Notice that the wMN and wSN values are different for j j each user j in DC because of: (1) the Rˆ j,b value is different since the user j experiences different channel conditions comparing the MN and SN links, and (2) the RAT weight wbj is different for the MN and SN since the RATs have distinct weights due to their different transmission capacity (more details about the RAT weight are given in the next paragraphs and in Section 3.3). After that, the MN calculates the ratio of the split bearer transmitted via its radio interface and the ratio that goes to the SN, respectively, as follows: MN

ratio j

=

wMN j wMN j

(19)

+ wSN j

3.3. Utility functions

and SN

ratio j =

wSN j wMN j

+

wSN j

weight is employed to differentiate the weight each RAT receives during the calculation of the bearer split ratios, which depends on a certain criterion. In the present work, we adapt the RAT weight based on the user satisfaction at the NR SNs, which is a unique signaling also reported by the SNs to the MN. Since the NR RAT disposes of more available bandwidth and uses a shorter TTI than the LTE RAT, the NR RAT has clearly more transmission capacity then the LTE RAT. Therefore, the RAT weight is an adaptive function that gives higher weights for the NR RAT as long as the user satisfaction percentage on the SN is above 90% since the SNs are employing the NR RAT. When the satisfaction at the NR SN drops below 90%, the RAT weight is dynamically adapted to give higher weights for the LTE MN. This criterion exploits the higher transmission capacity of the NR RAT up to a point where the satisfaction of users connected to the NR SN drops below 90%, then the weights start balancing the split ratios so that the satisfaction is equal or higher than 90%. In other words, we exploit the capacity of the NR RAT up to the limit where the QoS level of the users connected to it is still satisfactory. Notice that by offloading more traffic to the NR RAT, the satisfaction of the users connected to the LTE MN is also maximized. As a summary of what has been discussed in this section, Fig. 1 illustrates how the proposed flow control algorithm works along with the signaling exchange between the MN and SN. Notice that we depicted only one user in Fig. 1, but the readers should understand that all users in DC report the channel quality estimation to the MN and SN. It is important to mention that the proposed flow control algorithm might be executed given a certain periodicity. It means that the computations and signaling exchange shown in Fig. 1 are not necessarily executed every TTI. Notice that by performing the proposed algorithm only at a given periodicity, the signaling over the backhaul link between the MN and SNs diminishes. Between two executions of the algorithm, the split ratio for all bearers would be constant and equal to the last calculated values. However, in case the user satisfaction at the SN is rapidly dropping, the SN could aperiodically inform the MN so that the current split ratios can be recalculated based on new metrics and weights.

MN

= 1 − ratio j .

(20)

Using this criterion for calculating bearer-specific split ratios, the proposed solution tracks the channel variations and captures the QoS levels that the users are experiencing in order to determine an optimized split ratio that maximizes the user satisfaction. Furthermore, the split ratios are dynamically optimized by adapting the utility-based RAT (wbj ) and service (wsj ) weights, which targets to balance the split ratios depending on the service priority and on the RATs transmission capacities. More details about these weights are given on the next paragraphs. The proposed flow control algorithm tracks the channel variations by means of using the term Rˆ j,b during the split calculations. The user QoS metrics are captured by the utility-based user weight (wj ) that measures the utility (satisfaction) that the user is experiencing according to its current QoS [25]. The utility-based service weight (wsj ) is employed in scenarios with multiple services so that specific weights are given for different services depending on the desired objective to be achieved. The service weight can be designed, for example, as an adaptive function that is dynamically adapted to meet some QoS requirement of a most prioritized service [25]. Finally, the innovative utility-based RAT weight (wbj ) has a similar behavior to the service weight for multi-RAT scenarios. The RAT

The user utility function U( · ) employed in the proposed solution is a logistic function for throughput-based, delay-based and queue-based users, i.e., the utility function is unified across all service classes and is given by Antonioli et al. [25]

U (x j ) =

1 1+e

μ(x j −xreq ) /σ j

.

(21)

This function was chosen because its parametrization allows us to maximize the total user utility in the system. The input variable req xj is the QoS metric of each user j and the x j value is the QoS requirement of a given service. Since the QoS represents different metrics depending on the user service, both are normalized by the req x j value so that the function and parameters are valid and independent for all services. For queue-based and delay-based services, the sigmoid is set to be decreasing (μ = 1) since the higher the queue size or delay, the less satisfied is the user. On the other hand, for throughput-based services, the user utility function is increasing (μ = −1) as the higher the throughput, the higher the user satisfaction. The parameter σ is set to 0.1088 and has the same value for all users [25]. Then, the user weight wj is given by 

U (· ), the first derivative of U(·) with respect to xj (see Eq. (16)), which is a bell shaped function with the same shape for all types of service, as shown in Fig. 2a. The service utility function V(·) is a scaled version of the hyperbolic tangent, which is chosen because its derivative provides

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

189

Fig. 1. Message diagram illustrating how the proposed flow control algorithm works and the signaling involved.

Fig. 2. User weight (wbj ), service weight (wsj ) and RAT weight (wj ) employed in the proposed flow control algorithm.

the service differentiation, where each service has a specific priority related to the magnitude of the function in the corresponding region [25]. The function formula is written as

 





V z j = ln 1 + e(z j −1)/λ .

(22)

The zj value is given by the U(xj ) value of the corresponding user, as shown in Eq. (15). The λ value is adapted in a multi-service scenario to protect the satisfaction level of a given prioritized service or higher priority mobile user plan. Then, the service weight wsj is 

given by V (· ), the first derivative of V(·) with respect to zj , which is a sigmoid function shown in Fig. 2b. The full adaptation criteria and details are demonstrated in [25].

Finally, the RAT utility function M(·) is used for multi-RAT scenarios so that the different RATs have different weights depending on their transmission capacity or traffic load, for example. As aforementioned, the RAT weight has a similar behavior to the service weight. In fact, the RAT utility function is given by

 





M y j = ln 1 + e(y j −1)/ρ .

(23)

The yj value is given by the V(zj ) value, as shown in Eq. (14). 

The RAT weight wbj is given by M (· ), the first derivative of M(·) with respect to yj , which is also a sigmoid function as shown in Fig. 2c. The ρ value is adapted based on a look-up table comprised of 41 non-linear spaced values of ρ , so that the function

190

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

age of satisfied users drops below 90%. The blue arrow in Fig. 3a indicates the NR SN weight reduction, which happens when the ρ value goes toward 0.1088. Notice from the inner most curve shape that when the NR SN has very low weights, if the user is experiencing a QoS level below the requirement (normalized user QoS metric less than 1), it is still reasonable to send data to be transmitted by the NR SN due to its higher transmission capacity. On the other hand, the LTE MN weight, shown in Fig. 3b, starts with the inner most curve shape (corresponding to a ρ = −0.1088) since the LTE MN has lower transmission capacity. Then, in the case the percentage of satisfied users connected to the NR SN drops below 90%, the curve shape is adapted to give higher weights to the LTE MN. The LTE MN weight increase is represented in Fig. 3b by the blue arrow, which happens when the ρ value goes toward 0.1088. Analyzing the case when the LTE MN weight is the lowest, inner most curve in Fig. 3b, one can see that the proposed solution only sends some data to the LTE MN if the user is experiencing QoS levels above the requirement, which is done because of the lower LTE MN transmission capacity. 3.4. Pseudo code of proposed solution In summary, the proposed solution, which is executed at the PDCP layer at the MN, performs the Algorithm 1 considering the bearers of UEs in DC. Algorithm 1 Pseudo code of the proposed flow control algorithm. 1: Fig. 3. Combined utility weight for NR SN and LTE MN for different values of ρ . The combined weight is obtained by multiplying only the utility weights, i.e., wbj · wsj · w j .

2: 3: 4:

is equally spaced considering y j = 0.5 and that the ρ value varies from −0.1088 to 0.1088. The same values are used for the λ parameter of the service weight [25]. As explained before, the NR RAT has a higher transmission capacity, which is the RAT of the SNs. The RAT weight adaptation is performed for each pair MN-SN, so that if a MN has several SNs, the adaptation is independent for each pair. Since our objective is to send as many data as possible to the SN up to the point where the percentage of satisfied users at the SN is below 90%, the start point is with ρ = −0.1088. Notice in Fig. 2c that for ρ = −0.1088, the SN weight is the maximum possible. Then, the SN reports to the MN, along with the users’ QoS metric and channel qualities, the user satisfaction of all connected users, not only the users in DC. If this value is below 90%, the flow control algorithm at the MN adapts the ρ value to reduce the SN weight, i.e., the ρ value goes toward 0.1088, which gives more weight to the MN. Two points are worth noting here. Firstly, by checking the satisfaction of all users connected to the SN, i.e., the users in DC and in single connectivity, the proposed flow control solution does not degrade the QoS level of any user connected to the SN. Secondly, the performed adaptation also benefits the user satisfaction at the MN since by offloading some of the traffic to the SNs, the QoS provision at the MN is enhanced. Fig. 3 depicts the combination of the RAT, service and user utility-based weights (only the multiplication wbj · wsj · w j , without the Rˆ j,b term) employed by the proposed solution considering the adaptability of the ρ parameter of the RAT weight. The combined weights are illustrated as a function of the normalized user QoS metric. The NR SN weight, shown in Fig. 3a, starts with the outer most curve shape (corresponding to a ρ = −0.1088), which gives higher weights to the NR SN and, consequently, sends more data from split bearers to the NR SN. Then, the curve shape is adapted dynamically to reduce the split ratio of the NR SN if its percent-

5: 6: 7: 8: 9: 10:

11:

Collect users’ QoS metrics (throughput, HOL delay or queue size) and Rˆ j,b Eq. (2) from the SN and MN Collect percentage of satisfied users connected to the SN if SN user satisfaction >90% then Adapt ρ to increase SN weight else  SN user satisfaction <90% Adapt ρ to increase MN weight end if Calculate wMN and wSN according to Eqs. (17) and (18), respecj j tively Compute the ratioMN and ratioSN according to Eqs. (19) and j j (20), respectively Transmit amount of data via backhaul to the SN according to ratioSN j Send amount of data to MN lower layers according to ratioMN j

3.5. Practical considerations and 3GPP compliance This section is devoted to provide a practical view of our proposal to the readers. To this end, we describe how our bearer split algorithm can be mapped into 3GPP network parameters and how they can be obtained. First of all, it is worth mentioning that the DC technology with LTE BSs operating on different carrier frequencies was discussed and standardized in the 3GPP Release 12 [3]. More recently, in the 3GPP Release 14 [4], the DC technology was discussed and standardized where the multi-connectivity might be established with LTE and NR BSs. For the 3GPP Release 15, the standardization of the DC technology between two NR BSs operating on different carrier frequencies is expected to be finalized. The proposed bearer split algorithm is designed to be executed at the PDCP layer of the MN, as discussed in [5] and standardized in [3,4]. In the specific study case of this work, the MN operates using the LTE RAT. However, no modification in our proposal in needed for deployment in scenarios with DC between LTE-only BSs or NR-only BSs. As agreed in [3,4], the bearer split algorithm is going to be executed at the PDCP layer of the MN regardless of

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

191

throughput as well as the total system throughput would be bottlenecked by the transmission capacity of the LTE MNs. The simulations considered ideal and non-ideal backhaul connections, where the non-ideal backhaul connections were modeled by assuming latencies ranging from 10 to 80 ms [7,16]. The same MAC scheduler is employed for all LTE and NR base stations, so that the gains and losses obtained arise strictly from the performance of the flow control algorithms. The cross-carrier Proportional Fair (PF) [15] was used as the MAC scheduler, which is a modified version of the traditional PF that attempts to guarantee fairness in scenarios with DC by the modifying the scheduling metric. Each BS b selects the user j to transmit on RB k at TTI n according to



 jk,b [n]

Fig. 4. Network topology adopted for the simulations. A LTE MN is positioned at the center of a three-sector hexagonal grid and one NR SN is randomly deployed per sector.

its technology. Thus, our proposed bearer split algorithm can be deployed in any DC scenario that follows the 3GPP standard. In terms of metrics, the proposed algorithm requires the QoS requirement of the data bearer carrying the user data. Such metric is transmitted upon the bearer establishment, which is already standardized and used in the Medium Access Control (MAC) schedulers [30]. Another metric needed by the proposed algorithm is the current value of either one of the following QoS metrics: throughput, delay or buffer size. The specific metric needed by our algorithm depends on the bearer type, as explained in Section 3.1. Such QoS metrics are also already standardized [30] and used in the MAC schedulers. Finally, in terms of signaling exchange between MN and SN, the proposed solution only requires the percentage of satisfied users at the SN. Given that the SN already has access to the QoS metrics and QoS requirements, the SN only needs to check the percentage of users who have their QoS requirements met and send this new single scalar value to the MN. Note that this specific signaling can be transmitted via control plane between the MN and SN. Besides that, since this is a very specific parameter, this signaling information can also be used for detecting whether our algorithm is being executed in the system. A full description of all the signaling involved in our proposed algorithm is given in Fig. 1. 4. Performance evaluation 4.1. Simulation assumptions The simulation environment adopted in the present work is aligned with the bearer split configuration presented in the 3GPP specifications in [3,4]. The network topology is comprised of a hexagonal grid of three-sector macro LTE eNBs, which are the MNs in our scenario. Then, one micro NR gNB, which is the SN, is randomly deployed on each sector of the LTE eNBs [23]. This deployment is illustrated in Fig. 4. The UEs dropping criterion follows a hotspot UE distribution per sector of the LTE MN, where 75% are deployed within the NR SN coverage area and the other 25% are uniformly deployed within the LTE MN sector. These specific percentages for the UEs dropping criterion were chosen because the NR SNs has higher transmission capacity than the LTE MNs. Consequently, similar to other studies from the literature [15,23], the majority of the UEs are positioned in the NR coverage area so that they can be in DC. If most of the UEs were outside of the NR coverage area, they would be in single connectivity and their overall



r j,k,b [n] = arg max  , j∈Jb b∈B T j,b [n − 1]

(24)

 where b∈B T j,b [n − 1] is the user throughput considering all previous and current connections to all BSs b ∈ B up to TTI n − 1, i.e, this is the total user throughput considering the DC. The performance of our proposed solution is compared with three bearer split algorithms from the literature, which were proposed in [15,17,21]. The algorithm from [17] is a fixed algorithm that sends a fixed percentage, x%, of data is sent by the MN to the SN via the backhaul link, and another fixed percentage (100-x)% is sent by the MN to the UE. We consider a configuration where the algorithm from [17] sends 30% of data to the MN and 70% of data to the SN, which was the best configuration for the fixed algorithm that we found during the simulations. The bearer split algorithms from [15,21] (described in Section 1.1) are more intelligent solutions that dynamically adapt the split ratios. In the figures presenting the results and in the results discussion, we refer to each comparison algorithm by their respective citation number. Each UE is configured to have one bearer and the traffic type of the bearer is a CBR flow. The traffic server generates packets with constant size of 10 Kbits every NR TTI, i.e., every 0.25 ms. This configuration emulates a high intensity traffic, similar to the ones expected in the eMBB use case, and follows the widely used CBR traffic model. The UEs are considered satisfied if their total  throughput b∈B T j,b is higher than a throughput requirement of 20M bps. Since CBR flows are throughput-based services, the proposed solution employed its throughput-based branch where the throughput considered for calculating the utility-based user weight is the total user throughput. Notice that this information is already available at the MAC layer because the cross-carrier PF uses it for scheduling the users. Thus, the flow control algorithm executed at the MN PDCP layer just needs to receive this information from the MN MAC layer. Finally, since the UEs are static, in order to capture the system performance in different coverage situations, several independent snapshots considering different user distributions are considered during the simulations. The UE speed shown in Table 2 is used for computing the fast fading variation. All the results are evaluated as a function of the number of UEs per MN sector. Furthermore, all the results are presented with the 95% bootstrap confidence interval of the mean of the samples. A summary of the simulation parameters are presented in Tables 1 and 2. 4.2. Simulation results The first analyses are performed considering ideal backhaul connections between the MNs and SNs, i.e., the backhaul latency is considered to be 0 ms. The first performance metric we investigate is the percentage of satisfied users in the system, which is the metric that the proposed algorithm attempts to maximize. Fig. 5 presents the percentage of

192

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196 Table 1 Simulation parameters for LTE and NR. a

Parameter

LTE

NR

Ref.’s

Layout Scenario Inter-site distance BS height Carrier frequency System bandwidth Subcarrier spacing Num. of RBs TTI duration Num. of subcarriers per RB Num. of OFDM symbols per RB Noise figure BS Tx power Tx antenna type Tx antenna element pattern

Macro layer: 1 hexagonal site with 3 sectors 3GPP 3D Urban Macro 500 m 25 m 3.5 GHz 20 MHz 15 kHz 100 1 ms 12 14 9 dB 49 dBm ULA with 8 elements 3GPP 3D

Micro layer: 1 randomly dropped NR BS per LTE sector 3GPP 3D Urban Micro – 10 m 28 GHz 100 MHz 60 kHz 125 0.25 ms 12 14 9 dB 35 dBm URA 4 × 4 3GPP 3D

[23] [31] [31] [24,31] [24,31] [24,31] [32,33] [33,34] [32,33] [32,33] [32,33] [24] [24,24] [24] [31]

a

Whenever two references appear, the first refers to LTE and the second to NR. Also, only one reference refers to both RATs.

Table 2 Common simulation parameters for both RATs. Parameter

Value

Ref.’s

Link adaptation UE distribution UE height UE speed UE bearer type UE QoS requirement Confidence interval MAC scheduler

Link level curvesa Uniform in the macro areab 1.5 m 3 km/h Constant Bit Rate (CBR) 20 Mbps 95% Cross-carrier PF

[35]

a b

[15]

−4

For a BLER of 10 . 75% in hotspot (within NR BS area).

Fig. 6. Total system throughput.

Fig. 5. Percentage of satisfied UEs.

satisfied UEs when the number of UEs increases. The best performance is achieved by the proposed solution for all system loads, while the worst performance is always presented by the algorithm from Wang et al. [15]. The main reasons for the worst performance of the algorithm from Wang et al. [15] is that such algorithm was designed and tuned for DC in LTE-only scenarios and it does not aim at maximizing the user satisfaction (its objective is just to keep some target buffer size at the SN). Notice that for light system loads, the proposed solution and the algorithms from [17,21] obtained similar performances. However, when the system load increased, the proposed solution achieved higher user satisfaction levels, which happens because of the RAT weight adaptability as well as the channel and QoS tracking performed by the proposed solution. As a consequence of these features, the proposed solution is able to compute the best bearer split ratio for each individual

bearer. Considering the satisfaction level of 90%, the proposed solution achieved a gain of 25% compared to the benchmark solution from [17] with respect to the number of satisfied users. Furthermore, one can see from Fig. 5 that the best performance from the benchmark solutions was obtained by the algorithms from [17,21]. Notice that the algorithm from [17], which is a fixed bearer split algorithm, only presents a good performance up to certain user load. After that, keeping a fixed split ratio is not the ideal solution, which can be seen by the performance loss presented by the algorithm from [17] for higher user loads. On the other hand, the algorithm from López-Pérez et al. [21], which simply directs the packets to the link with lowest transmission delay, also presents performance losses as the system load increases since it was not initially designed for applications in LTE-NR scenarios and does not target to maximize the user satisfaction. On the other hand, the proposed solution maintains the user satisfaction at satisfactory levels for higher system loads by adapting its utility functions depending on the current system circumstances. Fig. 6 depicts the total system throughput when the number of UEs in the system increases. One can see that the proposed flow control algorithm presented the best performance also in terms of total system throughput. Thus, besides guaranteeing high user satisfaction levels by maximizing the user satisfaction on both MN and SN, the proposed solution also maximizes the total system throughput. Since the bearer split ratio is adapted considering the QoS metrics and channel quality of the UEs, the transmit buffer of both MN and SN are not overloaded nor run empty, so that there is always data to sent by both MN and SN and the total system throughput is also maximized. Finally, one can observe again

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

Fig. 7. Mean throughput of UEs.

that the algorithms from [17,21] present second and third best performances, while the algorithm from [15] presents again the worst performance. Notice that the fixed algorithm from [17] presented always higher total system throughputs than the algorithm from [21], showing that sending the packets to the link with lowest transmission delay is not the best solution since the traffic of many users might end up being directed to that same link, thus decreasing the system utilization (i.e., the total system throughput). In Fig. 7, we present the mean throughput of the UEs. The best performance obtained in the user satisfaction percentage (Fig. 5) is a consequence of the results presented in Fig. 7. Notice that even for high system loads, the mean throughput achieved by the proposed solution and the algorithm from López-Pérez et al. [21] is always above the throughput requirement of 20M bps. Since the satisfaction percentage of the algorithm from López-Pérez et al. [21] is worse than the satisfaction percentage of the proposed solution (see Fig. 5), it means that even though the mean throughput achieved by the algorithm from López-Pérez et al. [21] was higher than the throughput requirement, a higher number of users were still experiencing throughput below the requirement, thus being deemed unsatisfied. A similar analysis is also valid for the fixed algorithm from [17]. On the other hand, the mean throughput achieved by the algorithm from Wang et al. [15] is below the throughput requirement for most user loads, which is the reason why this algorithm achieved such low satisfaction rates in Fig. 5. Since in Figs. 5–7 the best performances were always obtained by the proposed flow control algorithm and algorithms from [17,21], the final analyses are conducted only for these three solutions. Fig. 8 depicts the user satisfaction considering that the UEs in the system were separated into two groups: UEs in DC and UEs in Single Connectivity (SC). Notice again that the proposed solution presented the best performance of user satisfaction for the two groups of users, while the algorithms from [17,21] achieved lower values of percentage of satisfied users for both groups. Considering the satisfaction level of 90%, the proposed solution achieved gains of 60% with respect to the algorithm from [17] for UEs in SC and 35% with respect to the algorithm from [21] for UEs in DC. One can also observe in Fig. 8 the benefit provided by the DC technology. The user satisfaction for UEs in SC is always lower than the satisfaction of UEs in DC, which is explained by the fact that UEs in DC experience higher throughputs by means of simultaneously receiving data from multiple nodes. The QoS enhancements provided by the DC technology can be maximized given that proper bearer split ratios are computed, which is the main feature of the proposed solution. Consequently, the satisfaction of both groups of UEs is maximized.

193

Fig. 8. Percentage of user satisfaction separating the UEs in DC and in SC.

Fig. 9. 5%-tile and 90%-tile of UEs throughput.

It is worth highlighting that Fig. 8 also demonstrates that the flow control algorithm is not only important for the UEs in DC, but it is also relevant for the UEs in SC. By performing a proper bearer split ratio control, the network operator can guarantee that the transmit buffers of MNs and SNs are balanced such that all UEs can have their requirements met. For instance, if the flow control algorithm always sends more data to the SN even when the satisfaction of UEs connected to the SN is decreasing, the QoS experience of both DC and SC UEs connected to the SN will also decrease. Therefore, the flow control algorithm also needs to check the system behavior to control the bearer split ratios. This is performed in the proposed solution by checking the user satisfaction at the SN, so that the split ratio is higher for the SN only when its satisfaction level is above 90%. In order to end the analysis considering ideal backhaul connections between the MNs and SNs, Fig. 9 presents the 5%-tile and 90%-tile of UEs throughput. The 5%-tile represents the cell-edge performance, while the 90%-tile is a representation of the highest achieved throughputs. Notice that for both 5%-tile and 90%-tile, the proposed flow control algorithm was able to provide higher throughputs. Regarding the 5%-tile, we believe that it represents the throughput of UEs in SC with LTE MN, i.e., UEs that are in the MN cell-edge and not in the coverage region of the NR SNs so that their throughputs are the lowest among all users. Thus, even though the proposed solution provides higher 5%-tile throughputs, for high system loads, the achieved throughput is below the throughput requirement. This is one of the reasons why the satisfaction for UEs in SC is lower in Fig. 8. Analyzing the 90%-tile, one can observe that the proposed solution obtained gains as high as

194

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

metric and channel quality slightly changed. Even through the fixed algorithm from [17] also presents low performance losses as the backhaul delay increases, its performance even considering the smallest value of delay is worse than the performance of the proposed solution when considering the highest value of backhaul delay. On the other hand, the algorithm from López-Pérez et al. [21] presents very high values of performance loss when the backhaul delay increases. The reason behind this result is that the algorithm from López-Pérez et al. [21] sends the packets to the link with lowest transmission delay and since the delay for the SN link also includes the backhaul delay, many packets are sent to the MN when the backhaul delay is high. Since the MN has lower transmission capacity, the performance is severely degraded. 5. Conclusions

Fig. 10. 50%-tile and 90%-tile of UEs throughputs considering imperfection in the backhaul link.

25% with respect to the algorithm from [21], which reflects the fact that the proposed flow control algorithm tracks the QoS metrics and channel quality, so that higher bearer split ratios are directed to where the users are experiencing better QoS levels and channel quality. Now, we perform an analysis considering backhaul imperfections. Fig. 10 illustrates the 50%-tile and 90%-tile of UEs throughputs considering imperfections on the backhaul connection between the MNs and SNs. As mentioned before, the backhaul imperfection is modeled as a latency on the connection, which in our analysis are equal to 10ms, 40ms and 80ms. We can see from both Fig. 10a and b that the proposed flow control algorithm maintains its higher performance even under backhaul imperfections. In Fig. 10b, even with the highest latency on the backhaul connection, the proposed solution was able to keep the 90%-tile of the UEs throughputs above 30M bps in the considered scenario. Finally, one can observe that the proposed solution presents a slightly higher performance loss as the backhaul latency increases. This happens because the proposed solution computes the bearer split ratios given a certain QoS metric and channel quality at a given time instant, but when the data arrives at the SNs, the QoS

Considering 5G heterogeneous and multi-RAT scenarios composed of LTE and NR base stations deployed based on the bearer split configuration and powered by the DC technology, the present work has proposed a flow control algorithm to be employed in the MN to compute the bearer-specific split ratios. The proposed algorithm is derived from a reinterpretation of a suboptimal solution of a utility-based resource allocation problem and targets the maximization of the user satisfaction in the system. The main differences from previous works is that the proposed solution tracks the QoS experience and channel quality of UEs in DC to compute particular split ratios for each bearer. Besides that, the proposed flow control algorithm also captures the system behavior by balancing the split ratios based on the user satisfaction at nodes with higher transmission capacity. The performance evaluation was conducted on 5G multi-RAT scenarios composed of base stations employing LTE and NR RATs, where the system parameters are aligned with the 3GPP specifications. The proposed solution was compared with three bearer split algorithms from the literature, which were proposed in [15,17,21]. From the results, it was concluded that the proposed solution is able to achieve gains in terms of user satisfaction, UEs throughput and total system throughput. The baseline algorithms from [17,21] only present good performances for low system loads or under perfect backhaul conditions. As a trade-off of introducing low complexity and signaling in the system, the proposed solution was able to enhance the user satisfaction, achieving gains as high as 60% for some scenarios. The main reasons for the gains achieved by the proposed solution are the channel quality and QoS experience tracking as well as the adaptability of one utility weight to provide a better control of the bearer split ratios. Conflicts of interest No conflict of interest. Acknowledgment The authors also acknowledge the technical and financial support from Ericsson Research, Sweden and from the Ericsson Innovation Center, Brazil, under UFC.43 Technical Cooperation Contract Ericsson/UFC and by CNPq (Proc. 151004/2017-0). This study was financed in part by the Coordenaç ão de Aperfeiçaoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Roberto P. Antonioli would like to acknowledge FUNCAP and CAPES (Proc. 88881.188039/2018-01) for its scholarship support. References [1] ITU, Draft New Report ITU-R M.[IMT-2020.TECH PERF REQ] - Minimum Requirements Related to Technical Performance for IMT-2020 Radio Interface(s), 2017, (ITU-R SG05 Contribution 40).

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196 [2] Huawei, 5G Network Architecture - A High-Level Perspective, White Paper, 2016. [3] 3GPP, Study on Small Cell Enhancements for E-UTRA and E-UTRAN - Higher Layer Aspects, TR, 3rd Generation Partnership Project (3GPP), V.12.0.0, 2013. http://www.3gpp.org/dynareport/36842.htm. [4] 3GPP, Study on New Radio (NR) Acess Technology, TR, 3rd Generation Partnership Project (3GPP), V.14.1.0, 2017. http://www.3gpp.org/DynaReport/38912. htm. [5] I.D. Silva, G. Mildh, J. Rune, P. Wallentin, J. Vikberg, P. Schliwa-Bertling, R. Fan, Tight integration of new 5G air interface and LTE to fulfill 5G requirements, in: Proc. of the IEEE Vehic. Tech. Conf. (VTC), 2015, pp. 1–5, doi:10.1109/VTCSpring. 2015.7146134. [6] A. Zakrzewska, D. López-Pérez, S. Kucera, H. Claussen, Dual connectivity in LTE HetNets with split control- and user-plane, in: Proc. of the IEEE Global Telecommun. Conf., 2013, pp. 391–396, doi:10.1109/GLOCOMW.2013.6825019. [7] 3GPP, Scenarios and Requirements of LTE Small Cell Enhancements for E-UTRA and E-UTRAN, TR, 3rd Generation Partnership Project (3GPP), V.13.3.0, 2015. [8] 3GPP, Overview of 3GPP Release 12 V0.2.0, Technical Report, 3rd Generation Partnership Project (3GPP), 2015. http://www.3gpp.org/ftp/Information/WORK_ PLAN/Description_Releases/Rel-12_description_20150909.zip. [9] 3GPP, Coordinated Multi-Point Operation for LTE Physical Layer Aspects, TR, 3rd Generation Partnership Project (3GPP), 2013. http://www.3gpp.org/ftp/ Specs/html-info/36819.htm. [10] D.H.N. Nguyen, L.B. Le, T. Le-Ngoc, Optimal dynamic point selection for power minimization in multiuser downlink CoMP, IEEE Trans. Wirel. Commun. 16 (1) (2017) 619–633, doi:10.1109/TWC.2016.2626444. [11] 3GPP, Evolved Universal Terrestrial Radio Access (E-UTRA); Carrier Aggregation; Base Station (BS) Radio Transmission and Reception, TR, 3rd Generation Partnership Project (3GPP), 2013. http://www.3gpp.org/ftp/Specs/html-info/ 36808.htm. [12] K.I. Pedersen, P.H. Michaelsen, C. Rosa, S. Barbera, Mobility enhancements for LTE-advanced multilayer networks with inter-site carrier aggregation, IEEE Commun. Mag. 51 (5) (2013) 64–71, doi:10.1109/MCOM.2013.6515048. [13] S. Barbera, K. Pedersen, P.H. Michaelsen, C. Rosa, Mobility analysis for inter-site carrier aggregation in LTE heterogeneous networks, in: Proc. of the IEEE Vehic. Tech. Conf. (VTC), 2013, pp. 1–5, doi:10.1109/VTCFall.2013.6692288. [14] Y. Ramamoorthi, A. Kumar, Performance comparison of dual connectivity with CoMP in heterogeneous cellular networks, in: 9th International Conference on Communication Systems and Networks (COMSNETS), 2017, pp. 237–242, doi:10.1109/COMSNETS.2017.7945382. [15] H. Wang, C. Rosa, K.I. Pedersen, Dual connectivity for LTE-advanced heterogeneous networks, Wirel. Netw. 22 (4) (2016) 1315–1328, doi:10.1007/ s11276- 015- 1037- 6. [16] C. Rosa, K. Pedersen, H. Wang, P.H. Michaelsen, S. Barbera, E. Malkamaki, T. Henttonen, B. Sebire, Dual connectivity for LTE small cell evolution: functionality and performance aspects, IEEE Commun. Mag. 54 (6) (2016) 137–143, doi:10.1109/MCOM.2016.7498101. [17] 3GPP, 3GPP Contribution R2-132859: Throughput Evaluation and Comparison of with and without UP Bearer Split, Technical Report, 3rd Generation Partnership Project (3GPP), 2013. http://www.3gpp.org/DynaReport/ TDocExMtg- R2- 83- 30050.htm. [18] 3GPP, 3GPP Contribution R2-132833: Performance Evaluation of user Throughput Enhancement with Multi-Stream Aggregation Over Non-Ideal Backhaul, Technical Report, 3rd Generation Partnership Project (3GPP), 2013. http:// www.3gpp.org/DynaReport/TDocExMtg- R2- 83- 30050.htm. [19] M.S. Pan, T.M. Lin, C.Y. Chiu, C.Y. Wang, Downlink traffic scheduling for LTE-A small cell networks with dual connectivity enhancement, IEEE Commun. Lett. 20 (4) (2016) 796–799, doi:10.1109/LCOMM.2016.2522404. [20] B. Jin, S. Kim, D. Yun, H. Lee, W. Kim, Y. Yi, Aggregating LTE and Wi-Fi: toward intra-cell fairness and high TCP performance, IEEE Trans. Wirel. Commun. 16 (10) (2017) 6295–6308, doi:10.1109/TWC.2017.2721935. [21] D. López-Pérez, D. Laselva, E. Wallmeier, P. Purovesi, P. Lundén, E. Virtej, P. Lechowicz, E. Malkamaki, M. Ding, Long term evolution-wireless local area network aggregation flow control, IEEE Access 4 (2016) 9860–9869, doi:10.1109/ ACCESS.2016.2643690. [22] T.K. Vu, M. Bennis, S. Samarakoon, M. Debbah, M. Latva-aho, Joint load balancing and interference mitigation in 5G heterogeneous networks, IEEE Trans. Wirel. Commun. 16 (9) (2017) 6032–6046, doi:10.1109/TWC.2017.2718504. [23] 3GPP, Study on Scenarios and Requirements for Next Generation Access Technologies, TR, 3rd Generation Partnership Project (3GPP), V.14.3.0, 2017. http: //www.3gpp.org/DynaReport/38913.htm. [24] 3GPP, Study on Channel Model for Frequencies from 0.5 to 100 GHz, TR, 3rd Generation Partnership Project (3GPP), V.14.2.0, 2017. http://www.3gpp.org/ DynaReport/38901.htm. [25] R.P. Antonioli, E.B. Rodrigues, T.F. Maciel, D.A. Sousa, F.R. Cavalcanti, Adaptive resource allocation framework for user satisfaction maximization in multi-service wireless networks, Telecommun. Syst. (2017) 1–17, doi:10.1007/ s11235- 017- 0391- 3. [26] G. Song, Y.G. Li, Utility-based resource allocation and scheduling in OFDM-based wireless broadband networks, IEEE Commun. Mag. 43 (12) (2005) 127–134. [27] H. Lei, L. Zhang, X. Zhang, D. Yang, A packet scheduling algorithm using utility function for mixed services in the downlink of OFDMA systems, in: Proceedings of the IEEE Vehicular Technology Conference (VTC), 2007, pp. 1664–1668.

195

[28] E.B. Rodrigues, F.R.M. Lima, T.F. Maciel, F.R.P. Cavalcanti, Maximization of user satisfaction in OFDMA systems using utility-based resource allocation, Wireless Commun. and Mob. Comput. 16 (4) (2014) 376–392, doi:10.1002/wcm.2526. [29] L.M.C. Hoo, B. Halder, J. Tellado, J.M. Cioffi, Multiuser transmit optimisation for multicarrier broadcast channels: asymptotic FDMA capacity region and algorithms, IEEE Trans. Commun. 52 (6) (2004) 922–930, doi:10.1109/TCOMM. 2004.829570. [30] 3GPP, Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN); Overall Description; Stage 2, TS, 3rd Generation Partnership Project (3GPP), 2018. http://www.3gpp.org/ftp/ Specs/html-info/36300.htm. [31] 3GPP, Study on 3D Channel Model for LTE, TR, 3rd Generation Partnership Project (3GPP), V.12.3.0, 2016. http://www.3gpp.org/DynaReport/36873.htm. [32] 3GPP, Evolved Universal Terrestrial Radio Access (E-UTRA); Physical Channels and Modulation, TS, 3rd Generation Partnership Project (3GPP), 2010. http:// www.3gpp.org/ftp/Specs/html-info/36211.htm. [33] A.A. Zaidi, R. Baldemair, H. Tullberg, H. Bjorkegren, L. Sundstrom, J. Medbo, C. Kilinc, I.D. Silva, Waveform and numerology to support 5G services and requirements, IEEE Commun. Mag. 54 (11) (2016) 90–98, doi:10.1109/MCOM. 2016.1600336CM. [34] 3GPP, Evolved Universal Terrestrial Radio Access (E-UTRA); Base Station (BS) Radio Transmission and Reception, TS, 3rd Generation Partnership Project (3GPP), 2017. http://www.3gpp.org/ftp/Specs/html-info/36104.htm. [35] C. Mehlführer, M. Wrulich, J.C. Ikuno, D. Bosanska, M. Rupp, Simulating the long term evolution physical layer, in: Proc. of the European Signal Processing Conf., 2009, pp. 1471–1478. Glasgow, Scotland. Roberto P. Antonioli received his B.Sc. degree in teleinformatics engineering (magna cum laude) from the Federal University of Ceará (UFC), Fortaleza, Brazil, in 2016. In 2017, he received his M.Sc. degree in teleinformatics engineering also from UFC. He is currently a Ph.D. student and researcher at the Wireless Telecom Research Group (GTEL), UFC, where he works on projects in technical and scientific cooperation with Ericsson Research. In 2018, he was a visiting researcher for 5 months at Ericsson Research in Luleøa, Sweden, where he studied new scheduling architectures for 5G multi-connectivity networks. His research interests include 5G wireless communication networks with multiple radio access technologies and multi-connectivity, as well as scheduling algorithms for QoS/QoE provisioning. Emanuel Bezerra Rodrigues received his B.Sc. and M.Sc. degrees in electrical engineering from the Federal University of Ceará (UFC), Fortaleza, Brazil, in 2001 and 2004, respectively. He also received his Ph.D. degree with honors in Signal Theory and Communications from the Universitat Politécnica de Catalunya (UPC/BarcelonaTech), Barcelona, Spain, in 2011. In 2014, he joined UFC, where he is an Assistant Professor in the Department of Computer Science. He has been working in the Wireless Telecom Research Group (GTEL) since 2001 and has actively participated in several projects in cooperation between GTEL and Ericsson Research. During the last years, he has published several conference papers, journal/magazine articles, book chapters, and has been a reviewer of important international conferences and IEEE journals and magazines. His main research interests are Internet of Things, and radio resource management and QoS control for next-generation mobile and wireless networks. Diego Aguiar Sousa received the B.Sc. degree in computer engineering in University of Ceará (UFC), Sobral, Brazil, in 2011. He received M.Sc. and Ph.D. degree in Teleinformatics Engineering from the UFC, Fortaleza, Brazil, in 2013 and 2018, respectively. Since 2013, he has been a researcher at the Wireless Telecom Research Group (GTEL), UFC, participating of projects in a technical and scientific cooperation with Ericsson Research. Also, since 2013, he has been a Professor of the Federal Institute of Education Science and Technology of Ceará (IFCE), Paracuru, Brazil. His research interests include numerical optimization, 5G networks, coordinated scheduling, radio resource allocation for QoS/QoE provisioning. Igor M. Guerreiro received the B.S., M.Sc. and Ph.D. degrees in teleinformatics engineering from the Federal University of Ceará (UFC), Brazil, in 2007, 2010 and 2016, respectively. He currently holds a post-doc position at the UFC Department of Teleinformatics Engineering. Since 2007 he has been a researcher at Wireless Telecommunications Research Group (GTEL), Brazil, working in research projects within a technical cooperation with Ericsson Research, Sweden. In 2008, he was a guest researcher at Virginia Tech Advanced Research Institute (ARI), Arlington, Virginia, USA. Before starting the Ph.D. course at UFC, he visited Ericsson in 2010 in Luleøa, Sweden, for 5 months, and another time in San José, California, USA,

196

R.P. Antonioli, E.B. Rodrigues and D.A. Sousa et al. / Computer Networks 161 (2019) 183–196

for 3 months. As a Ph.D. student, he visited both Ericsson and the Royal Institute of Technology (KTH) in 2014-15 in Stockholm, Sweden, for a year-long period. Some topics of his research interests include techniques for MIMO transceiver design, strategies for distributed optimization for wireless communication systems, modeling and simulation of cellular communication, dynamic spectrum access methodologies and physical layer aspects for Internet of Things. Carlos F. M. e Silva received a five years diploma degree and M.Sc. degree in Electronics and Telecommunications Engineering from the University of Aveiro (UA), Portugal, in 2005 and 2010, respectively. In 2015 he also received a PhD degree in Teleinformatics Engineering from the Federal University of Ceará (UFC), Brazil. Since 2006, Carlos Silva has been a researcher in several European projects, such as WINNER II (system requirements for beyond 3rd generation wireless networks), FUTON (RRM for wireless and optical networks), and COGEU (cognitive radio systems for efficient use and sharing of TVWS in the European context). Currently, he has a postdoc position at the Wireless Telecom Research Group (GTEL), Brazil, where he manages GTEL’s team in the European-Brazilian project FUTEBOL and also works in cooperation projects with Ericsson Research. His main research interests include: spectrum usage optimization, TV White Spaces, Licensed Shared Access, Internet of Things, and Device-to-Device communications; testbed experimentation; and architectural aspects of the future 5G networks.

Francisco Rodrigo Porto Cavalcanti received the B.Sc. and M.Sc. degrees in electrical engineering from Federal University of Ceará (UFC), Fortaleza, Brazil, in 1994 and 1996, respectively, and the D.Sc. degree in Electrical Engineering from the State University of Campinas, São Paulo, Brazil, in 1999. Upon graduation, he joined the UFC, where he is currently an Associate Professor and holds the Wireless Communications Chair with the Department of Teleinformatics Engineering. In 20 0 0, he founded and, since then has directed the Wireless Telecom Research Group (GTEL), which is a research laboratory based on Fortaleza, which focuses on the advancement of wireless telecommunications technologies. At GTEL, he manages a program of research projects in wireless communications sponsored by the Ericsson Innovation Center in Brazil and Ericsson Research in Sweden. Prof. Cavalcanti has produced a varied body of work including two edited books, conference and journal papers, international patents and computer software dealing with subjects such as radio resource allocation, cross-layer algorithms, quality of service provisioning, radio transceiver architectures, signal processing and project management. Prof. Cavalcanti is a distinguished researcher of the Brazilian Scientific and Technological Development Council for his technology development and innovation record. He also holds a Leadership and Management professional certificate from the Massachusetts Institute of Technology, Cambridge.