SaaS enabled admission control for MCMC simulation in cloud computing infrastructures

Accepted Manuscript SaaS enabled admission control for MCMC simulation in cloud computing infrastructures J.L. V´azquez-Poletti, R. Moreno-Vozmediano, R. Han, W. Wang, I.M. Llorente PII: DOI: Reference:

S0010-4655(16)30186-2 http://dx.doi.org/10.1016/j.cpc.2016.07.004 COMPHY 5982

To appear in:

Computer Physics Communications

Received date: 3 February 2016 Revised date: 3 June 2016 Accepted date: 2 July 2016 Please cite this article as: J.L. V´azquez-Poletti, R. Moreno-Vozmediano, R. Han, W. Wang, I.M. Llorente, SaaS enabled admission control for MCMC simulation in cloud computing infrastructures, Computer Physics Communications (2016), http://dx.doi.org/10.1016/j.cpc.2016.07.004 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

*Manuscript (PDF version)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

SaaS enabled Admission Control for MCMC simulation in Cloud Computing Infrastructures J.L. V´azquez-Polettia,∗, R. Moreno-Vozmedianoa , R. Hanb , W. Wangc , I.M. Llorentea Facultad de Inform´ atica Universidad Complutense de Madrid Madrid, Spain b State Key Laboratory Computer Architecture, ICT, CAS Beijing, China c Department of Computing Imperial College London London, UK a

Abstract Markov Chain Monte Carlo (MCMC) methods are widely used in the field of simulation and modelling of materials, producing applications that require a great amount of computational resources. Cloud computing represents a seamless source for these resources in the form of HPC. However, resource over-consumption can be an important drawback, specially if the cloud provision process is not appropriately optimized. In the present contribution we propose a two-level solution that, on one hand, takes advantage of approximate computing for reducing the resource demand and on the other, uses admission control policies for guaranteeing an optimal provision to running applications. Keywords: Cloud Computing, SaaS, PaaS, Admission Control 1. Introduction Applications for simulation and modelling of materials are usually high computational demanding and require large amount of resources to solve ∗

Corresponding author. E-mail: [email protected].

Preprint submitted to Journal of Computer Physics Communications

June 2, 2016

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

complex problems, such as linear and non-linear differential equation solvers, molecular dynamics codes, Monte Carlo simulators, etc. [1, 2, 3]. Traditionally, scientists and engineers have used large supercomputers or HPC clusters to execute this kind of applications [4, 5, 6, 7]. In this context, cloud computing opens new avenues by reducing infrastructure costs and providing resources on-demand (IaaS or Infrastructure as a Service), by providing specialized platforms for developing, testing and deploying applications (PaaS or Platform as a Service), and offering specialized applications as a service (SaaS or Software as a Service) to the scientific community, such as equations solvers, simulators, etc. that run over cloud platforms and infrastructures [8, 9]. Some examples of HPC and cloud platforms used in computer materials science are the MMM@HPC Project [10] and the Scientific Cloud Computing (SCC) platform [11]. These cloud infrastructures can be private or public, depending on the property of the bare-metal machines. Private clouds (also named corporate or internal) can be built using virtualization solutions like OpenNebula [12] or OpenStack [13]. Public clouds are provided by third-party institutions in a pay-as-you-go basis. Some examples of these are Amazon EC2, Google Compute Engine and Microsoft Azure. Public cloud infrastructures have been on the focus of research for optimal resource provision strategies, specially how Service Level Agreements (SLAs) are to be accomplished when provisioning resources. Violating a SLA means that resource requirements were not met. These situations, with different levels of severity depending on the SLA, have already been studied. Moreover, some mechanisms to minimize their impact have been proposed but from the client side [14, 15, 16, 17]. From the provider side, admission control policies [18, 19, 20] represent a solution for smoothing or even avoiding these situations. As it will be shown later, some of these policies assume SLA violations by the sake of a bigger net benefit. Other policies follow a conservative strategy, reducing these violations thus reducing the revenue. However, material science applications can require huge amount of resources, and the number of users (scientific and engineers) using these applications can also be large, it is important to research on different mechanisms to avoid the exhaustion of the resources of the cloud infrastructure. With this idea in mind, in this paper we propose to combine two different methods to control the cloud resource consumption: first, from the point of view of the user, we propose the use of approximate computing techniques to reduce 2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

the number of resources consumed by the applications, and second, from the point of view of the application provider, we propose the use of admission control mechanisms to guarantee that running applications have enough resources to be completed. These methods for controlling cloud resource consumption come from two camps of previous works: one on producing approximate results with small correctness losses for cloud interactive services [21, 22, 23]; and other on algorithms for SLA-based service models in cloud computing [24]. Under both resource and time constraints in a cloud environment, this paper focuses on the first camps based on the motivation that end users can tolerate small correctness losses in approximate results. Even if we are focusing on public clouds, our research can also apply to private ones, as in both of them there are providers and consumers. The main difference is that the cost function is calculated in a different way and may vary among private infrastructures [25], being its study out of the scope of this contribution. The remainder of this contribution is as follows: first, some related work is provided; then, Section 3 explains the proposed architecture; experiments conducted in a simulation environment with a Markov Chain Monte Carlo (MCMC) application are shown and discussed in Section 4; and finally, the contribution ends with some conclusions and future work. 2. Related Work In the present Section, we will first review specific algorithms used in the simulation of materials. Then, related work on the generation of approximate results at SaaS level in order to reduce execution time will be provided. The Section will then end with a study at PaaS level, focusing in SLA definition and admission control algorithms. Monte Carlo (MC) simulations represent a class of algorithms widely applied in simulation of materials [26, 27, 28]. ABC sampling is a specific case for Monte Carlo simulation [29]. ABC sampling is the sampling process for determining the posterior probability of a parameter in a statistical model given the prior belief of the parameter and some observed measurements produced by the model. However, ABC sampling can be computationally expensive for high dimensional problem since ABC sampling relies simulation for each sample it generates. We have implemented the basic ABC Rejection method and applied it into an animal migration problem. We set the error 3

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

threshold for the ABC rejection algorithm and gradually decrease it from a large error. In order to generate 1,000 samples, 500% error threshold takes already 3 minutes on a modern laptop. As the threshold drops (e.g. below 100% error), the execution takes hours to finish. Within both time and resource constraints, producing approximate results is a feasible solution to reduce simulation time by sacrificing correctness (e.g. result accuracy) [30]. Motivated by the facts that small losses in the output quality cannot be evidently perceived and thus tolerable by users, efficiently and successfully applying such approximate simulation/processing mechanisms requires address two challenges: (1) approximate results of satisfactory quality can be produced even using small amount of resources; (2) the output quality of approximate results can be monotonically increased when using more resources, so as to guarantee the efficient usage of these resources. For such an issue, Synopsis-based Approximate Request Processing (SARP) is proposed to produce a list of approximate results, whose output qualities are proportional to their resource consumption [21]. Once a budget of computation resource is given, SARP tries to produce an approximate result that its has the maximum resource consumption than the resource budget. That is, this result has the best possible quality for the given budget. Moving to another level, it can be seen that SLAs and admission control algorithms are closely related concepts. In cluster computing there hashas been a definition of SLAs by means of Quality of Service (QoS) metrics like utilization, availability and packet loss rate [31]. Also, SLA-based policies to handle penalties and enhance the utility of the cluster can be found [32]. Media services are another area of application. There, admission control mechanisms have been introduced for on demand devices provision, where acceptance results in positive revenues [33]. Jobs with response time guarantees and deadline constraints have been also considered by other authors [34]. Cloud computing is not an exception for SLAs research. For instance, some work has been on a request based prediction system for resource consumption minimization [35]. Scheduling algorithms discard those requested services that drain resource capacity, preventing to reach an established deadline. A set of ”if(condition)-then(action)” rules are defined in the RESERVOIR project and serve as basic building blocks for autonomic computing, dealing with situations that are likely to happen [36]. 4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

In [37, 38], admission control is defined as policy-driven probabilistic, intending to use dynamic placement optimization mechanisms of satisfying site-level managed policies. The OPTIMIS project offers SLA mechanisms aiming to evaluate levels of risk and trust, negotiating the use of license protected software. Meanwhile, authors claim that actual mechanisms for cloud computing are limited to cost/performance tradeoffs [39]. Moreover, performance levels, the degree of trust expected from the service provider, or the level of risk according to cost thresholds are aspects considered by the policies used for service deployment in cloud infrastructures [40]. The SLA offered by Amazon EC2, the de facto standard public cloud infrastructure, promises an availability of the 99.95% a year. If the downtime exceeds 4.4 hours, a 10% of the monthly bill is returned1 . The authors of [41] propose a learning based Admission Control. The resulting system, applied to cloud services, becomes more dynamic and situation aware thanks to opportunistic algorithms. 3. Proposed architecture The main elements of the proposed architecture can be seen at Figure 1. We have broken up this Section in two depending on the cloud layer (SaaS and PaaS). 3.1. SaaS level In this section, the SaaS module processes requests from end users with both execution time (i.e. service latency) and QoS constraint. In the SARP framework, the module is designed to produce approximate results of different output qualities (i.e. correctnesses) when the service allocates different amounts of resources or the service load changes dynamically. At the SaaS level, the module has both offline and online stages. At the offline stage, the module first organizes the service input data using a hierarchical data structure and then organizes its information at different levels of granularity to produce a list of synopses with different lengths. Hence a synopsis with a larger length has a higher approximation level to the input data. 1

http://aws.amazon.com/ec2-sla/

5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Figure 1: Proposed architecture divided in the three main provision layers.

At the online stage, the module dynamically selects the synopsis according to the current resource allocation and workload status. Among all candidate synopses, this synopsis has the maximal length that can be processed within the required execution time and QoS constraint. Using the synopsis, the module processes each request and generates an approximate result for the Paas layer described in the following section. Depending on the quality level of the result, this request has a price and a penalty. 3.2. PaaS level In this Section, the request generated by the SaaS layer is explained, followed by the admission control algorithms and SLA model. 3.2.1. Service definition The admission control module uses information from both the initial service request (S) and that supplied by upper layer: Si (µi , δi , mi , Mi , ti , t0i )

(1)

where the mean (µ), standard deviation (δ), minimum (m) and maximum (M ) refer to the resources needed during the service lifetime (from ti to t0i ). Our provision model assumes that requests follow a cumulative distribution function Φ(x) (see example at Figure 2), and requested resource number will be always between m and M .

6

1 0.9

M = 60

0.8

Probability

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

0.7 0.6 0.5 0.4 0.3

m = 20

0.2 0.1 0

0

20

40

60

80

100

Resources

Figure 2: Cumulative distribution for a service with µ = 40, δ = 25, m = 20 and M = 60.

3.2.2. Admission control algorithms Every time a new service appears during a scheduling cycle, the module decides whether accept that service or not. The following admission control algorithms have been already proposed in a previous publication [24]. In particular, dynamic algorithms have been considered. These algorithms decide whether a new service is accepted or not based on the mean of available resources after each provisioning action (Rµ ) instead of the number of existing resources (Ra ). ΣTt=0 Ri (t) (2) T being Ri (t) the number of available resources in a given moment (t), and T corresponds to the current moment. Rµ (T ) =

Dmin: accept services only when Minimum is guaranteed A new service Si is accepted only when the provision of its minimum number of resources (mi ) is guaranteed along with the previously deployed services. The decision is taken by comparing mi and the result of subtracting the minimum number of resources of deployed services to the number of the mean existing resources (Rµ ): if mi ≥ (Rµ (T ) − mi ) then accept service; else reject service; end if 7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Dmax: accept services only when Maximum is guaranteed A new service Si is accepted only when the provision of its maximum number of resources (Mi ) is guaranteed along with the previously deployed services. The decision is taken by comparing Mi and the result of subtracting the minimum number of resources of deployed services to the number of the mean existing resources (Rµ ): if Mi ≤ (Rµ (T ) − Mi ) then accept service; else reject service; end if Davg: accept services only when mean is guaranteed A new service Si is accepted only when the provision of its average number of resources (µi ) is guaranteed along with the previously deployed services. The decision is taken by comparing µi and the result of subtracting the minimum number of resources of deployed services to the number of the mean existing resources (Rµ ): if µi ≤ (Rµ (T ) − µi ) then accept service; else reject service; end if 3.2.3. SLA model Like in [24], we use three SLA levels (SLA1 , SLA2 and SLA3 ) that are differentiated by the price for each provided resource (P ) and the penalty for each non provisioned resource (P 0 ): PSLA1 > PSLA2 > PSLA3 , 0 0 0 PSLA > PSLA > PSLA , 1 2 3

(3) (4)

In the present contribution, we are using the general information provided by the service, considered as a global request, like in the algorithms described in Section 3.2.2:

8

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Si ∈ SLA1 =⇒ Mi < Rµ , Si ∈ SLA2 =⇒ µi ≤ Rµ < Mi , Si ∈ SLA3 =⇒ mi ≤ Rµ

(5) (6) (7)

If an accepted service is later suspended because of lack of resources, penalty is calculated using the assigned SLA, the mean resources (µ) and the remaining scheduling cycles for that request. 4. Experiments A simulator has been produced for validating the present proposal. The simulation is performed as explained below. Simulation time is divided into cycles. At each cycle, the simulator checks for running services (therefore accepted) that may have ended. Then, resources are provisioned to running but not ended services. The amount of resources for each running service at that specific cycle is obtained using the service’s distribution (µi , δi ) along with the minimum (mi ) and maximum (Mi ) values. The simulator will terminate those running services that cannot be accomplished due to unavailability of resources in that cycle. The amount of unprovisioned resources are calculated using the mean (µi ) for each request (as explained at Section 3.2.3). Finally, new requests are accepted or rejected according to the chosen admission control algorithm (explained at Section 3.2.2). The applied PaaS SLA is also decided at this time. 4.1. Experiment data set For this particular experiment, we have used a parallel version of Markov Chain Monte Carlo focused on the Metropolis–Hastings algorithm applied to large–scale Bayesian inference problems [42]. The input data considered are a mixture of multidimensional Gaussian distributions, coming from different fields of science and engineering [43]. The different output qualities of these simulations are measured by the Gelman-Rubin factor [44], which improves with the number of iterations. This kind of simulation has been profiled, so that if the user request a given output quality (expressed as number of iterations), and impose an execution time limit, we determine the number of 9

4 3 Quality

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

2 1 0 Requests

Figure 3: Requests used in the experiments and their quality level.

resources (processors) needed to run the simulation in parallel. This profiling is shown in Table 1. Quality Iterations Execution Time (seconds)

Q4 Q3 Q2 Q1 20K 30K 40K 50K Processors 34,980 52,470 69,960 87,450 1 6,727 11,925 17,063 19,433 16 5,144 9,717 14,278 19,433 32 4,727 8,745 12,274 16,194 64

Table 1: Profiling used for generating the experiment input data. Q1 is the highest quality and Q4 the lowest.

For the experiments we have generated a set of 100 requests. Figure 3 shows them categorized by quality (Qn). The demanded resources distribution for each request is represented at Figure 4 by means of their mean (µi ), standard deviation (δi ), minimum (mi ) and maximum (Mi ). Finally, Figure 5 shows the arrival times. These series have been randomly generated: for each request we have generated a random quality value (between 1 and 4), a random distribution of required resources (with average values between 1 and 100 processors), and random arrival time. Using these random data as input, we have extrapolated the execution time of each request from the profiling data of Table 1. 4.2. Results We have considered five different scenarios with different provider sizes (R). For each scenario, the three admission control algorithms described in 10

140

Resources

120 100 80 60 40 20 0 0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95 100

Requests

Figure 4: Resource distribution for each request. The boxes represent the means (µi ) and standard deviations (δi ), and the whiskers represent the minimum (mi ) and maximum (Mi ).

Arrival time

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

20000 18000 16000 14000 12000 10000 8000 6000 4000 2000 0 Requests

Figure 5: Requests used in the experiments and their arrival time (ti ) expressed in seconds.

Section 3.2.2 (Dmin, Dmax, Avg) have been applied. R=50 Figures 6, 7 and 8 show the provisioned and not provisioned resources for each accepted request when applying each algorithm. Being this the scenario with the lowest number of resources (R), so is the number of accepted requests. Dmin accepted the most (20), followed by Davg (16) and closely by Dmax (15). R=100 Figures 9, 10 and 11 show the provisioned and not provisioned resources for each accepted request when applying each algorithm.

11

300000 Prov NotProv

Resources

250000 200000 150000 100000 50000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 6: Provisioned and not provisioned requests with the Dmin admission control algorithm when R = 50. 300000 Prov NotProv

Resources

250000 200000 150000 100000 50000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 7: Provisioned and not provisioned requests with the Dmax admission control algorithm when R = 50. 300000 Prov NotProv

250000 Resources

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

200000 150000 100000 50000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 8: Provisioned and not provisioned requests with the Davg admission control algorithm when R = 50.

12

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Dmin accepted 38 requests and Dmax, 30. In this scenario, Davg was almost in the middle accepting 33 requests. R=150 Figures 12, 13 and 14 show the provisioned and not provisioned resources for each accepted request when applying each algorithm. 56 requests were accepted by Dmin and 39 by Dmax. This time, Davg was nearer to Dmin, as 53 requests were accepted. R=200 Figures 15, 16 and 17 show the provisioned and not provisioned resources for each accepted request when applying each algorithm. Dmin accepted 71 requests and both Dmax and Davg accepted 51. R=1000 Figures 18, 19 and 20 show the provisioned and not provisioned resources for each accepted request when applying each algorithm. In the scenario with the highest number of resources (R), Dmin accepted all services but one, Dmax accepted 93 and Davg accepted 96. SLA accomplishments and violations Every accepted service had requested resources that were provisioned or not. Provisioned resources are considered SLA accomplishments and not provisioned, SLA violations. Table 2 shows them for each admission control algorithm and scenario. Dmin will accept the highest number of requests but incurring in the most SLA violations due to unavailable resources. Moreover, many requests from SLA2 and SLA3 are accepted by this algorithm just to be terminated almost instantly. Conservative policies such as Dmax provide the best ratio of accomplished/violated SLAs at the cost of accepting the lowest number of requests. Davg offers a balance, but not always is the best option unless being in the R = 100 scenario. In the highest number of resources scenario (R = 1, 000) all requests are within SLA1. All three algorithms offer good results, but Dmax is the one with the best ratio again. Studying the evolution of the algorithms with the increase of R, we can see that Dmin is constantly throwing the highest number of not provisioned 13

700000 Prov NotProv

Resources

600000 500000 400000 300000 200000 100000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 9: Provisioned and not provisioned requests with the Dmin admission control algorithm when R = 100. 700000 Prov NotProv

Resources

600000 500000 400000 300000 200000 100000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 10: Provisioned and not provisioned requests with the Dmax admission control algorithm when R = 100. 700000 Prov NotProv

600000 Resources

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

500000 400000 300000 200000 100000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 11: Provisioned and not provisioned requests with the Davg admission control algorithm when R = 100.

14

700000 Prov NotProv

Resources

600000 500000 400000 300000 200000 100000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 12: Provisioned and not provisioned requests with the Dmin admission control algorithm when R = 150. 700000 Prov NotProv

Resources

600000 500000 400000 300000 200000 100000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 13: Provisioned and not provisioned requests with the Dmax admission control algorithm when R = 150. 700000 Prov NotProv

600000 Resources

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

500000 400000 300000 200000 100000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 14: Provisioned and not provisioned requests with the Davg admission control algorithm when R = 150.

15

1.2x106

Prov NotProv

Resources

1x106 800000 600000 400000 200000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 15: Provisioned and not provisioned requests with the Dmin admission control algorithm when R = 200. 1.2x106

Prov NotProv

Resources

1x106 800000 600000 400000 200000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 16: Provisioned and not provisioned requests with the Dmax admission control algorithm when R = 200. 1.2x106

Prov NotProv

1x106 Resources

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

800000 600000 400000 200000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 17: Provisioned and not provisioned requests with the Davg admission control algorithm when R = 200.

16

1.2x106

Prov NotProv

Resources

1x106 800000 600000 400000 200000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 18: Provisioned and not provisioned requests with the Dmin admission control algorithm when R = 1, 000. 1.2x106

Prov NotProv

Resources

1x106 800000 600000 400000 200000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 19: Provisioned and not provisioned requests with the Dmax admission control algorithm when R = 1, 000. 1.2x106

Prov NotProv

1x106 Resources

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

800000 600000 400000 200000 0 0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Requests

Figure 20: Provisioned and not provisioned requests with the Davg admission control algorithm when R = 1, 000.

17

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

resources. Dmax is quite balanced (as explained before) but only with R = 1, 000 obtains the best ratio. Finally, Davg throws a positive ratio with R ≤ 100, but then this trend changes. SLA1 Prov NotProv

SLA2 Prov NotProv

R = 50 Dmin 152,7954 904,903 10 Dmax 1,527,358 544,996 0 Davg 1,528,209 626,141 10 R = 100 Dmin 2,936,114 3,330,204 8,688 Dmax 2,377,565 2,756,013 0 Davg 2,965,414 2,788,746 145,564 R = 150 Dmin 3,355,973 5,850,134 1,135 Dmax 3,928,602 3,084,127 0 Davg 3,215,964 6,549,080 159,187 R = 200 Dmin 4,575,519 10,354,045 613,411 Dmax 4,952,196 4,427,373 0 Davg 4,834,724 5,473,962 628,872 R = 1, 000 Dmin 23,107,384 13,781,128 0 Dmax 23,258,795 9,571,779 0 Davg 23,633,102 12,010,618 0

SLA3 Prov NotProv

211,898 0 101,200

0 0 0

127,536 0 0

133,920 0 0

2,614 0 0

896,039 0 0

652,964 345,145 0 0 925,024 0

2,449,670 0 0

3,417,718 0 1,327,240

593 0 0

2,027,796 0 0

0 0 0

0 0 0

0 0 0

Table 2: Provisioned and not provisioned resources per SLA from accepted requests for each admission control algorithm and scenario.

Being the Dmax that provisioning the highest number of requests, a quality per request breakdown for this algorithm is shown at Table 3. It can be seen that requests with the highest quality (Q1) are the most provisioned along the different scenarios. With an intermediate resource provider (R = 150 and R = 200) this behavior is better noticed as the number of Q1 requests is higher than the sum of the rest (Q2 + Q3 + Q4). At R = 1, 000 this difference is less significant.

18

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

R = 50

Prov NotProv NotAccepted R = 100 Prov NotProv NotAccepted R = 150 Prov NotProv NotAccepted R = 200 Prov NotProv NotAccepted R = 1, 000 Prov NotProv NotAccepted

Q1 Q2 Q3 Q4 3 2 3 2 2 1 0 1 22 20 22 21 7 5 3 1 4 1 2 4 16 16 19 19 11 2 6 2 6 4 4 2 10 16 15 20 16 6 5 2 7 5 4 4 3 12 16 18 22 13 17 16 6 7 4 6 0 2 4 2

Table 3: Requests accepted and provisioned at 100% (Prov), accepted but not provisioned at 100% (NotProv), and not accepted (NotAccepted), sorted by quality (Qn) with the Dmax policy.

Revenue study We have considered a price and a penalty for each resource from accepted requests. These values, which vary depending the applied SLA, are shown in Table 4. This way, SLA1 has the highest price but also the highest penalty. On the other hand, not providing SLA3 resources costs nothing. Price Penalty

SLA1 SLA2 SLA3 1 0.08 0.05 0.08 0.01 0

Table 4: Prices and penalties for each provisioned and not provisioned resource depending on the applied SLA. Values are expressed in generic monetary units.

Figure 21 shows the revenue at PaaS level for each admission control algorithm and scenario. With the least resources (R = 50), all algorithms obtain the same revenue. While raising the number of resources, Dmin is the most balanced until R = 200, which is when Davg takes this role. Dmax is a good solution above 19

2.5x107

Dmin Dmax Davg

2x107

7

Revenue

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

1.5x10

1x107

5x106

0 50

100

150 Provider resources (R)

200

1000

Figure 21: Revenue at PaaS level for each admission control algorithm and resource number (R).

R = 100. Moving to the SaaS level, more example prices and penalties have been considered for the experiments depending on the quality (Table 5). Unlike in the previous study, these figures apply to accepted requests entirely, not just resources. This way, a penalty is to be paid when an accepted request is not provided at 100%. Like SLA1 from the PaaS level, Q1 has the highest price and penalty. On the other hand, not providing an accepted Q4 request cost nothing (like SLA3). Price Penalty

Q1 Q2 Q3 Q4 1,000 800 500 300 800 400 100 0

Table 5: Prices and penalties for each accomplished and not accomplished accepted request depending on the quality (Qn). Values are expressed in generic monetary units.

Figure 22 shows the revenue at SaaS level for each admission control algorithm and scenario. In contrast to the previous study at PaaS level, accepting the most requests as possible (Dmin) is not a good option by means of SaaS revenue. In fact, the R = 140 and R = 200 scenarios throw negative results. Even with plenty resources (R = 1, 000), Dmin was the worst algorithm. On the other hand, a conservative strategy such as Dmax reveals to be the best in most of the cases. 20

40000 Dmin Dmax Davg

35000 30000 25000 Revenue

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

20000 15000 10000 5000 0 -5000 50

100

150

200

1000

Resources (R)

Figure 22: Revenue at SaaS level for each admission control algorithm and resource number (R).

5. Conclusions and future work A research area such as the simulation and modelling of materials requires of a great amount of computing resources for executing its applications. HPC in the form of supercomputers or clusters has been always the first choice of scientists and engineers. Cloud computing represents a breakthrough improvement in service provision, as reduces infrastructure costs while offers on-demand resources. However, physical resources are not infinite so there is the need of an optimal way to provide services, minimizing the impact of non provisioning promised services. Our contribution has been double: first we have proposed approximate computing techniques for reducing the number of resources consumed by applications; and second, we have proposed admission control mechanisms for guaranteeing enough resources for running applications. These contributions were initiated by previous works [21, 22, 23, 24] but then adapted and integrated in the present paper so a complete computing proposal is offered to the physics of materials community. For validation we have focused on a particular algorithm used in the advanced modelling and simulation of materials, and generated a considerable set of use cases based on real profiling. Validation allowed us not only to show the mechanics of our proposal but also to perform a two-level revenue study. One lesson learned is that users need to be aware of available resources, 21

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

in order to adjust request quality to the execution environment limitations. This way, the probabilities of accomplishing a request at 100% will increase. On the other hand, admission control algorithms are very important when dealing with limited resources, as they optimize the number of processed requests. Finally, we have demonstrated that this proposal suits the execution of a parallel version of Markov Chain Monte Carlo. It is our aim to work on other applications pertaining to the simulation and modelling of materials. Acknowledgements Research in this contribution was supported by the FedCloudNet project (MINECO TIN2015-65469-P). References [1] J. Bernholc, Computational materials science: the era of applied quantum mechanics, Physics Today (1999) 30–35. [2] D. Raabe, Computational Materials Science: The Simulation of Materials, Microstructures and Properties, Wiley-VCH Verlag GmbH & Co. KGaA, 2004. [3] J. G. Amar, The monte carlo method in science and engineering, Computing in Science & Engineering 8 (2006) 9–19. [4] P. Vashishta, R. K. Kalia, S. W. de Leeuw, D. L. Greenwell, A. Nakano, W. Jin, J. Yu, L. Bi, W. Li, Computer simulation of materials using parallel architectures, Computational Materials Science 2 (1994) 180 – 208. [5] M. T. Lusk, A. E. Mattsson, High-performance computing for materials design to advance energy science, MRS Bulletin 36 (2011) 169–174. [6] A. Schleife, E. W. Draeger, V. M. Anisimov, A. A. Correa, Y. Kanai, Quantum dynamics simulation of electrons in materials on highperformance computers, Computing in Science Engineering 16 (2014) 54–60.

22

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

[7] T. Belozerova, A. Demenev, V. Henner, P. Kharebov, E. Khenner, G. Sumanasekera, Use of supercomputer for modeling coherent processes in magnetic nano-structures, Computational Materials Science 102 (2015) 228 – 233. [8] J. Leukel, S. Kirn, T. Schlegel, Supply Chain as a Service: A Cloud Perspective on Supply Chain Systems, Systems Journal, IEEE 5 (2011) 16 –27. [9] G. Lin, G. Dasmalchi, J. Zhu, Cloud computing and it as a service: Opportunities and challenges, in: Web Services, 2008. ICWS ’08. IEEE International Conference on, p. 5. [10] W. Wenzel, et al., Multiscale material modelling on high performance computer architectures. MMM@HPC project fact sheet., 2010. [11] K. Jorissen, F. D. Vila, J. J. Rehr, A high performance scientific cloud computing environment for materials simulations, Computer Physics Communications 183 (2012) 1911–1919. [12] R. Moreno, R. S. Montero, I. M. Llorente, Elastic Management of Cluster-based Services in the Cloud, in: Proc. 1st Workshop on Automated Control for Datacenters and Clouds (ACDC09) on the 6th International Conference on Autonomic Computing and Communications, ACM Digital Library, 2009, pp. 19–24. [13] R. Kamboj, A. Arya, Openstack: Open source cloud computing iaas platform, International Journal of Advanced Research in Computer Science and Software Engineering 4 (2014) 1200–1202. [14] L. Ramakrishnan, C. Koelbel, Y.-S. Kee, R. Wolski, D. Nurmi, D. Gannon, G. Obertelli, A. YarKhan, A. Mandal, T. M. Huang, K. Thyagaraja, D. Zagorodnov, VGrADS: enabling e-Science workflows on grids and clouds with fault tolerance, in: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC ’09, ACM, New York, NY, USA, 2009, pp. 47:1–47:12. [15] C. Cachin, I. Keidar, A. Shraer, Trusting the Cloud, SIGACT News 40 (2009) 81–86.

23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

[16] V. C. Emeakaroha, M. A. Netto, R. N. Calheiros, I. Brandic, R. Buyya, C. A. D. Rose, Towards autonomic detection of {SLA} violations in cloud infrastructures, Future Generation Computer Systems 28 (2012) 1017 – 1029. Special section: Quality of Service in Grid and Cloud Computing. [17] A. Kertesz, G. Kecskemeti, I. Brandic, An interoperable and selfadaptive approach for sla-based service virtualization in heterogeneous cloud environments, Future Generation Computer Systems 32 (2014) 54 – 68. Special Section: The Management of Cloud Systems, Special Section: Cyber-Physical Society and Special Section: Special Issue on Exploiting Semantic Technologies with Particularization on Linked Data over Grid and Cloud Architectures. [18] J. B. Abdo, J. Demerjian, H. Chaouchi, T. Atechian, Enhanced revenue optimizing sla-based admission control for iaas cloud networks, in: Future Internet of Things and Cloud (FiCloud), 2015 3rd International Conference on, pp. 225–230. [19] M. Carvalho, D. Menasce, F. Brasileiro, Prediction-based admission control for iaas clouds with multiple service classes, in: 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom), pp. 82–90. [20] K. Konstanteli, T. Cucinotta, K. Psychas, T. A. Varvarigou, Elastic admission control for federated cloud services, IEEE Transactions on Cloud Computing 2 (2014) 348–361. [21] R. Han, J. Wang, F. Ge, J. L. V´azquez-Poletti, J. Zhan, SARP: producing approximate results with small correctness losses for cloud interactive services, in: Proceedings of the 12th ACM International Conference on Computing Frontiers, CF’15, Ischia, Italy, May 18-21, 2015, pp. 22:1– 22:8. [22] R. Han, L. Guo, M. M. Ghanem, Y. Guo, Lightweight resource scaling for cloud applications, in: 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), IEEE, pp. 644–651.

24

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

[23] R. Han, M. M. Ghanem, L. Guo, Y. Guo, M. Osmond, Enabling costaware and adaptive elasticity of multi-tier cloud applications, Future Generation Computer Systems 32 (2014) 82–98. [24] J.L.V´azquez-Poletti, R. Moreno-Vozmediano, I. Llorente, Handbook of Research on Architectural Trends in Service-Driven Computing, IGI Global, pp. 678–694. [25] G. D. Guerrero, R. M. Wallace, J. L. Vzquez-Poletti, J. M. Cecilia, J. M. Garca, D. Mozos, H. Prez-Snchez, A performance/cost model for a cuda drug discovery application on physical and public cloud infrastructures, Concurrency and Computation: Practice and Experience 26 (2014) 1787–1798. [26] S. Mordechai, Applications of Monte Carlo Method in Science and Engineering, InTech, 2011. [27] M. C. Simulations, http://ocw.mit.edu/courses/materials- scienceand-engineering/3-320-atomistic-computer-modeling-of-materials-sma5107-spring-2005/video-lectures/lecture-17-monte-carlo-simulations/, 2005. [28] B. K. Latha, G. S. Preeti, K. Murthy, V. Sastry, Director structures with dominant in-plane alignment in hybrid planar films of biaxial nematic liquid crystals: A monte carlo study, Computational Materials Science 118 (2016) 224 – 235. [29] M. A. Beaumont, Approximate bayesian computation in evolution and ecology, Annual review of ecology, evolution, and systematics 41 (2010) 379–406. [30] R. Han, L. Nie, M. M. Ghanem, Y. Guo, Elastic algorithms for guaranteeing quality monotonicity in big data mining, in: Big Data, 2013 IEEE International Conference on, IEEE, pp. 45–50. [31] K. Xiong, H. Perros, SLA-based resource allocation in cluster computing systems, in: Parallel and Distributed Processing, 2008. IPDPS 2008. IEEE International Symposium on, pp. 1 –12.

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

[32] C. S. Yeo, R. Buyya, Service Level Agreement based Allocation of Cluster Resources: Handling Penalty to Enhance Utility, in: Cluster Computing, 2005. IEEE International, pp. 1 –10. [33] M. Bichler, T. Setzer, Admission control for media on demand services, Service Oriented Computing and Applications 1 (2007) 65–73. 10.1007/s11761-007-0005-0. [34] M. Islam, P. Balaji, P. Sadayappan, D. Panda, Towards provision of quality of service guarantees in job scheduling, Cluster Computing, IEEE International Conference on 0 (2004) 245–254. [35] G. Reig, J. Alonso, J. Guitart, Prediction of Job Resource Requirements for Deadline Schedulers to Manage High-Level SLAs on the Cloud, Network Computing and Applications, IEEE International Symposium on 0 (2010) 162–167. [36] B. Rochwerger, D. Breitgand, E. Levy, A. Galis, K. Nagin, I. M. Llorente, R. Montero, Y. Wolfsthal, E. Elmroth, J. Caceres, M. BenYehuda, W. Emmerich, F. Galan, The Reservoir model and architecture for open federated cloud computing, IBM Journal of Research and Development 53 (2009) 4:1 –4:11. [37] B. Sotomayor, R. Montero, I. Llorente, I. Foster, Virtual infrastructure management in private and hybrid clouds, Internet Computing, IEEE 13 (2009) 14 –22. [38] I. M. Llorente, R. S. Montero, B. Sotomayor, D. Breitgand, A. Maraschin, E. Levy, B. Rochwerger, Cloud Computing: Principles and Paradigms, Wiley, pp. 157–191. [39] M. Comuzzi, C. Kotsokalis, G. Spanoudakis, R. Yahyapour, Establishing and Monitoring SLAs in Complex Service Based Systems, in: Web Services, 2009. ICWS 2009. IEEE International Conference on, pp. 783 –790. [40] A. Ferrer, F. Hernandez, J. Tordsson, E. Elmroth, C. Zsigri, R. Sirvent, J. Guitart, R. Badia, K. Djemame, W. Ziegler, T. Dimitrakos, S. Nair, G. Kousiouris, K. Konstanteli, T. Varvarigou, B. Hudzia, A. Kipp, S. Wesner, M. Corrales, N. Forgo, T. Sharif, C. Sheridan, OPTIMIS: 26

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

a Holistic Approach to Cloud Service Provisioning, Future Generation Computer Systems in press (2010). [41] J. Dhok, N. Maheshwari, V. Varma, Learning based opportunistic admission control algorithm for MapReduce as a service, in: Proceedings of the 3rd India software engineering conference, ISEC ’10, ACM, New York, NY, USA, 2010, pp. 153–160. [42] E. L. Angelino, Accelerating Markov chain Monte Carlo via parallel predictive prefetching, Ph.D. thesis, Harvard University Cambridge, Massachusetts, 2014. [43] R. Nishihara, I. Murray, R. P. Adams, Parallel mcmc with generalized elliptical slice sampling, J. Mach. Learn. Res. 15 (2014) 2087–2112. [44] A. Gelman, D. Rubin, Inference from iterative simulation using multiple sequences, Statistical Science 7 (1992) 457–472.

27