Analysis of uncertainty indices used for building envelope calibration

Applied Energy 185 (2017) 82–94

Contents lists available at ScienceDirect

Applied Energy journal homepage: www.elsevier.com/locate/apenergy

Analysis of uncertainty indices used for building envelope calibration Germán Ramos Ruiz ⇑,1, Carlos Fernández Bandera 1 School of Architecture, University of Navarra, Spain

h i g h l i g h t s
Calibration methodology using Multi-Objective Genetic Algorithm (NSGA-II).
Multi-zone building calibration.
Calibration methodology that enables use of non-continuous time periods.
Analysis of the best objective functions to calibrate buildings.
The methodology captures the heat dynamic of the building.

a r t i c l e

i n f o

Article history: Received 28 May 2016 Received in revised form 15 October 2016 Accepted 19 October 2016

Keywords: Multi-zone calibration Energy simulation Uncertainty analysis Multi-objective optimisation Genetic algorithm (NSGA-II)

a b s t r a c t Nowadays there is a growing concern about climate change and the global warming effect produced by the concentration of greenhouse gases (GHG). At the Paris climate conference (COP21), 195 countries adopted a global climate agreement, limiting global warming to well below 2 °C. Buildings are large producers of GHG and therefore international standards such as ISO 50001 focus on improving their energy performance, including energy efficiency, use and consumption. To achieve this goal it is important to have a detailed knowledge of the thermal behaviour of buildings. The International Performance Measurement and Verification Protocol (IPMVP), proposes a calibrated simulation model (Option D) to gather this knowledge and to determine the savings associated with Energy Conservation Measures (ECMs). This paper improves the calibration methodology proposed by Ramos et al. (2016) [1], solving the limitations regarding the number of thermal zones and the use of free-floating time periods. Through a real case-study that guides the process, the paper explains how to achieve a calibrated Building Energy Simulation (BES) model using an optimisation process based on a meta-heuristic strategy (genetic algorithm - NSGA-II). Different uncertainty indices such as: CV(RMSE) and Goodness of Fit (GOF) are used as objective function to obtain the calibrated model. These indices, frequently used to measure the accuracy of models, are combined to provide a double possibility to find the best solution, as they are an objective function and a model accuracy measure. Ó 2016 Elsevier Ltd. All rights reserved.

1. Introduction Currently there is growing concern about energy. The population growth rate and the necessity of energy for well-being makes energy demand a question of great interest. Climate change and the global warming effect produced by the concentration of greenhouse gases (GHG) put the spotlight on research papers referring to improvement of energy production and reduction of energy demand. On the other hand renewable energy penetration requires control and optimisation of energy. ⇑ Corresponding author. 1

E-mail address: [email protected] (G. Ramos Ruiz). These authors contributed equally to this work.

http://dx.doi.org/10.1016/j.apenergy.2016.10.054 0306-2619/Ó 2016 Elsevier Ltd. All rights reserved.

Buildings have a key influence in global energy consumption [2] and therefore in the production of GHG. For that reason there are many studies on the energy optimisation of new and existing buildings. If these studies could be supported by a calibrated energy model, their energy saving quantification would improve greatly. ASHRAE [3] defines calibration as: ‘‘the process of comparing the output or results of a measurement or model with that of some standard, determining the deviation and relevant uncertainty and adjusting the measuring device or model accordingly”. As can be seen the definition emphasises the measurement of the uncertainty to determine the level of fitness of the model. On the other hand, the process of obtaining a calibrated Building Energy Simulation (BES) model is not an easy task, among many other reasons we can highlight some arguments of three of

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

the most important documents related to energy measurement: ASHRAE Guidelines 14-2002 [3], M&V Guidelines [4] and IPMVP [5], such as the skill experience of personnel with both the software and calibration process, the effort required for data acquisition, the accuracy of sensors used, the difficulties of simulating special parts of the building (atriums, double skin façades, complex HVAC systems, etc.) and the investment cost of different equipment. However, the use of calibrated BES models offers many benefits [6] such as the identification and quantification of the savings of different Energy Conservation Measures (ECM); the study of the building behaviour for on-going commissioning and retrocommissioning [7]; the analysis of the resulting improvement due to different HVAC strategies or building configurations because testing them in a simulation environment is not so intrusive for the building or its occupants [8]; the Fault Detection and Diagnosis if the simulation works together with the building automation system (FDD software) and the measurement of the energy stored in the building for demand response strategies. The calibration methodology is the search for the parameters that characterise the envelope of the building. The methodology consists of a careful survey of the building characteristics in order to create a model as similar as possible to the real construction. A short-term monitoring campaign is then carried out and temperature data are used to calibrate the model. The calibration models are a reliable portrayal of the building envelope and can then be used to assess the energy demand of the building quantitatively instead of qualitatively (non-calibrated BES models). This research is supported by our last work [1] which explains the methodology of calibrating buildings using genetic algorithms, but in this case it is focused on solving two main shortcomings: firstly the use of one thermal zone in the analysis and secondly the necessity of the objective function to have a continuous freeoscillation period of building operation. The second condition is not always possible because for a building to have a long freeoscillation period special circumstances are necessary. In addition, the period used should be as long as possible to capture the heat dynamic of the building in order to give the algorithm sufficient stimulus to find a proper solution for the envelope parameters. University buildings can deal easily with these limitations because of the holiday season, but for other buildings such as hospitals, shopping malls, hotels, residential homes and many others used in an intensive way these periods are rarely available. In this research, we are proposing several measures that greatly improve the methodology allowing these problems to be tackled. As described further on, we have implemented a new methodology that allows us to take advantage of all the intermittent freeoscillation periods. Furthermore, these periods can be different for each thermal zone because their sum will produce the same effect as that of one long period only. The script developed in our first work to reproduce the thermal history of the whole building now takes on a more important role because it enables us to maintain an individual thermal history for each thermal zone. Hence, for a whole building envelope calibration, we can count on the contribution of each thermal zone, although its free oscillation periods occur at different times. There is no limit to the number of thermal zones or to the number of small periods considered for each. The more information we have, the better quality calibration we get. The objective function will take advantage of the contribution from all the intermittent free-oscillation periods and will use them with the same effectiveness as if it were a single and continuous period for the whole building. This improvement in the methodology gives the process great flexibility and opens up the methodology for other buildings whose calibration process was difficult before due to their intensive use and their lack of long free-oscillation periods (hospitals, hotels, shopping malls, etc.). As an improvement, the new calibration methodology can be carried out in a

83

specific area of the building, thus being of assistance in multiproperty buildings. Furthermore, we make a detailed analysis of the different objective functions based on uncertainty analysis, to check which one is the best option to obtain a calibrated model. The paper is structured as follows. Section 2 briefly summarise the state-of-the-art of calibration strategies. Section 3 describes the calibration methodology highlighting the improvements regarding the use of several thermal zones and non-continuous free-floating time periods for calibration. In particular, Section 3.2.1 explains the special objective function used in this calibration approach, the fact that it is based on different uncertainty indices. Section 4 describes a detailed study of the different results obtained by the genetic algorithm (NSGA-II) guided by these uncertainty indices to assess which of them is the best choice to calibrate the BES model. Section 5 analyses the different models proposed by each optimisation of the genetic algorithm. And finally Sections 6 and 7 show the conclusions and the future work of this study. 2. A brief state-of-the-art There are different approaches to performing a calibration. Clarke et al. [9] first classified it in four groups based on: ‘‘(i) manual, iterative, pragmatic intervention, (ii) a suite of informative graphical comparative displays, (iii) the use of special tests and analysis procedures to isolate and compare individual energy flows, (iv) a technique for automatically adjusting user selected input parameters to reduce the discrepancy between measured and predicted data”. In general, as Coakley et al. [10] say, all the approaches can be defined as manual or automated. The tools and techniques of each one are different; the manual approach depends largely on the Modeller’s experience and expertise [11], and automated approach uses the mathematical and statistical properties of the model to achieve the calibrated BES model. An automated calibration technique is ultimately about how to optimise a BES model, but focusing the problem on a special objective: adjusting several parameters of the model to fit the simulated values with the measured ones. For these reason it is interesting to analyse both papers focusing in how calibrate BES models [8,12– 16] and papers related to optimising some aspects [17–24]. The techniques described in both groups are quite different and the use of genetic algorithms is limited to the ones related with optimisation problems. 3. Methodology In general this methodology was described more in depth in our last paper [1], but now we solve the two important limitations described before: the number of thermal zones analysed and the necessity of continuous periods of time to develop the calibration. One of the key aspects of this methodology is the use of data from temperature sensors to obtain a model with a high degree of accuracy. Saltelli et al. [25] describe these models as datadriven. Because the goal is to capture the heat dynamic of the real building, temperature sensors are adequate tools, and are cheaper than energy meters. The frequency selected is a ten-minute timestep due to the high accuracy that we are attempting to reach. This procedure allows us to dependably represent the thermal behaviour of the building. Other authors use simulated vs. measured temperatures as a calibration target but with a different approach [26,27]. Another key aspect of this methodology is that it focuses on calibrating the building envelope. The reason is that the energy demand of the building is closely related to the energy gained/lost

84

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

through its building envelope. As the Annex 49 guidebook [28] says: ‘‘The energy demand of a building can be defined as the amount of energy required to keep the indoor environment within the comfort ranges required by its users”, and the ‘‘minimum demand is only related to the subsystem ’building envelope’”. Once the BES model has its envelope calibrated it is possible to analyse different HVAC equipment and configurations to reduce the energy consumption needed to satisfy its demand. This amount of energy (consumption) will depend on the performance of HVAC systems and also on the amount of energy provided by the internal loads. A calibrated envelope captures the heat dynamic of the building, so it is the best ‘‘baseline” model to perform other detailed analysis. The framework of the calibration process is the same as it is shown in [1]. As explained in it there are three main sections in the process: (i) the modelling of BES, which depends on building characteristics and weather; (ii) the calibration process itself, which depends on a parametric analysis, a sensitive analysis and a genetic algorithm simulation guided by an appropriate objective function; (iii) and finally the analysis of the best results. In this last section the best models are checked to identify which has more energy-demand difference compared to the baseline model. In contrast, although the calibration methodology is the same, the development of the objective function is completely different solving the limitations related to the number of thermal zones and the time periods used. In order to compare the results of this improved methodology with the previous one, its description is guided by the same real case study described in [1], an office area of the ‘‘Edificio Amigos” building located at the University of Navarra (SPAIN) (Fig. 1) projected by Juan Miguel Otxotorena. In this case we have analysed 32 thermal zones, in seven non-continuous periods of time to test how the improved methodology responds with more than one thermal zone, and checks if the calibrated model reacts correctly in all the periods. For this reason the material specifications and characteristics of the different walls, roofs and partitions; the calculation of the model air-tightness (following ISO 13829 [29]); and the considerations about the importance of using a specific weather file [30,31] are the same than in the previous methodology [1]. Consequently, it has been assumed the results obtained in the parametric and sensitive analysis. In Appendix A, Table A.1 shows the results of this sensitive analysis. As mentioned in the earlier paper, the resulting search space is still large, over 2:08  1011 simulations. An attempt to perform all these simulations (‘‘brute force” technique) using a 4-core computer needing 20 s/simulation would require 32.992 years. For this reason, a meta-heuristic approach must be used, in our case genetic algorithms, to find good solutions in this huge search space. The next subsections explain in detail the improvements made in the methodology.

3.1. BES model Creating the baseline model is the first step in the calibration process. A manual calibration technique, like the one developed by Mustfaraj et al. [32], is based on the accuracy of the model; the higher the accuracy, the better results returned. The final adjustments in this manual process ‘‘tune” physical inputs to better match the model with reality [11]. One of the limitations of the previous methodology was the use of one thermal zone. This means that in buildings which have several thermal zones with different behaviours such methodology is not useful. The first innovation of this article is the use of as many thermal zones as needed by the calibration process, in this case we are using 32, but more can be implemented without additional effort. Consequently, better results are achieved because the thermal contribution of each zone is considered individually. The thermal oscillation of each zone fits better with the behaviour of the real building as can be seen in Section 5, therefore the BES model will have an increase in its accuracy. As a matter of fact, with this improved methodology, information from key oriented thermal zones (north, east, west an south) could be enough for a good calibration model. To achieve this goal a script developed in Python considers the individual contribution of each thermal zone to the global behaviour of the model. This script will be further explained in Section 3.2.2. Fig. 2 shows the thermal zoning considered in the model in different colours. Some of these thermal zones have specific HVAC equipment with its own thermostat to manage its energy demand (black numbers), and others are heated/cooled from a general system without specific control (red numbers). 3.2. Calibration strategy A BES model is considered to be calibrated if it has a correct behaviour in different periods of time, and this is the second innovation of this article, the new use of the free-floating periods. As we have said before, the purpose is to calibrate the envelope parameters to have access to the energy demand of the building, therefore, by taking into account free-floating periods, it is possible to reduce other variables such as internal loads, operation schedules and HVAC systems. minimising the complexity of the calibration process. However, generally the buildings do not have several continuous days of free-floating, which in fact was a limitation of the previous methodology. Usually, they have many short freefloating periods (nights, weekends, etc.). The second innovation solves this problem and allows us to use non-consecutive freefloating periods. As will be seen in Section 5, the sum of small periods has the same result as the use of a unique larger one. A new script developed in Python and described in Section 3.2.2 helps

Fig. 1. ‘‘Amigos” building elevation.

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

85

Fig. 2. Building thermal zones.

us to select these periods. As a result, the combination of these two innovations will increase enormously the building types where this calibration process will be able to carry out. In this case-study we use seven different free-floating periods, six short ones (weekends from Saturday at 19:00 h to Sunday at 24:00 h) and one long one (Christmas time), which is the same as that described in [1]. The selected periods are: December 2013 (weekends 7–8, 14–15, 21–22), Christmas time (from 24th December 19:00 h to 5th January 24:00 h), and January 2014 (weekends 11–12, 18–19, 25–26). During these periods the building was closed ensuring that no artificial energy from HVAC systems and internal loads was injected into the thermal zones, so that the external conditions (outside temperature and solar radiation) were the only energy inputs that affected the building thermally. Due to the fact that the interior temperature remained above 12 °C during all periods, the HVAC freeze protection was turned off. The sum of all periods (almost 20 full days), together with the great thermal amplitude (almost 8 °C due to the winter nights), makes these periods perfect for calibration purposes. Another important advantage of these improvements is that it will allow us to calibrate specific thermal zones that can be in different locations of the building because the script can isolate the information provided by each thermal zone and can consider different free-floating periods for each one. 3.2.1. Defining objective functions The calibration technique used in this paper is based on the optimisation of its uncertainty indices. There are many approaches to performing optimisation on ECMs, energy efficiency, etc. Nguyen et al. [33] make an in-depth analysis of the most important. As can be seen on its review, the software used to perform the simulation (EnergyPlus [34]), and the optimisation algorithm used (genetic algorithm) are the more common. About the uncertainty indices, actually there are different statistical indices commonly used for that purpose. They are based on CVðRMSEÞ and NMBE, differentiating between monthly data and hourly data. In Appendix A, Table A.2 reproduce the calibration criteria shown in [1] with the limit values established by the Measurement and Verification (M&V) of the Federal Energy Management Program (FEMP) [4]; the International Performance Measurement and Verification Protocol (IPMVP) [5]; and ASHRAE [3]. Due to the fact that the objective function used to guide the genetic algorithm is based on uncertainty indices, we assumed as calibrated the models with the best indices. Each uncertainty index gives us a different measurement and has a different meaning. In this paper we will analyse the frequent ones in order to find the best solution in shortest time. The indices used are CVðRMSEÞ, the coefficient of variation of the root mean square error; GOF,

goodness-of-fit; and a cost function, f i . With these indices we have organised seven different approaches (all options) to find the best solution, three using these indices individually, three using different pairs (GOF þ CVðRMSEÞ; f i þ CVðRMSEÞ; f i þ GOF), and one using the three indices together. Also for each model we have calculated the NMBE, normalised mean bias error; and R2 , coefficient of determination or R-squared, because they are frequently used to give more information about the model, but these indices have not been used as objective functions for the algorithm. The formulas of these indices are set down by the AHSRAE Research Project Report RP-1051 [11], by Technical Report 550060127 of the NREL [35], and by the International Performance Measurement and Verification Protocol (IPMVP). In particular the cost function f i is defined by Tahmasebi and Mahdavi [36,37] offsetting the values of CVðRMSEÞ and R2 . These indices are calculated based on building interior temperature (measured and simulated). Due to the fact that our calibration procedure has 32 thermal zones to calibrate, as Tahmasebi et al. explain [38], the measured and simulated values are obtained as an average over all thermal zones but in this case weighted to the surface of each one. The individual oscillation of each thermal zone varies the global weighted average of all zones, reflecting its contribution. Therefore if one of them has an out-of-range behaviour, it will be immediately detected because it will produce a deviation in the corresponding uncertainty index. We refer to the previous article [1] to see the description of each index. As will be pointed out below, when the number of possibilities is high, the set of good results is also high. So, after the simulations are complete, a post-process is required to check which solution fits best with the behaviour of the building.

3.2.2. Genetic algorithm optimisation The genetic algorithm is a common meta-heuristic optimisation algorithm belonging to the evolutionary algorithms based on Darwin’s theory of evolution. The population is governed by the survival principle and the best individuals, those that best match the objective function, survive. This algorithm includes mutation (random parameter changes) and crossover (switching parent’s simulations) as common operators. For multi-objective optimisation problems, the Non-dominated Sorting Genetic Algorithm II (NSGA-II) [39] is the most common [40]. The software that we use to perform the parametric analysis and genetic algorithm optimisation, jEPlus [41,42], has implemented this algorithm to tackle optimisation problems. To understand how the improved methodology deals with multi-zone models that take into account different time periods it is necessary to see the scheme of both methodologies. Fig. 3

86

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

Fig. 3. Calibration environment with genetic algorithm. On the left, the previous methodology [1], and on the right, the innovation we propose.

shows the two innovations carried out in the methodology. On the left, the calibration environment followed in the previous methodology; and on the right the present one. As can be seen, the difference is the outsourcing of the uncertainty calculus (second script). This change is the key to achieving better results than with the previous methodology. This second script calculates the uncertainty indices using the outputs of each simulation, when the simulation it is finished, outside EnergyPlus. The previous script was designed in Erl (EnergyPlus runtime language) which, as the manual says: ‘‘is an advanced feature of EnergyPlus and is not for beginners. (. . .) is not for all (or even most) users” [43]. The old script had to be developed and coded in the model file (.idf2) and its programming was complex. Then, thanks to the last versions of jEPlus it is possible to use Python scripts after EnergyPlus simulations which also permits use of its outputs as the objective of its optimisation process. This is particularly relevant as Python language is more user-friendly than Erl and, moreover, its programming possibilities are higher. Also, as it is an external script it helps to re-use the code in other calibration projects. In this case, the script uses the data of the selected thermal zones taking into account only the free-floating periods, and then it calculates its uncertainty indices (objective function) and feeds the genetic algorithm that decides which parameters change to optimise the solution. The first scrip, as described in [1], is responsible to solve the problem described by Machairas et al. [44] with simulations in short time periods. The run period of our simulations is two months, so it requires a warming period in order to match up the thermal inertia of the real building and BES model. This is the same script used in the previous methodology. Developed in Erl (EnergyPlus runtime language), its purpose is to transfer the measured temperature to the model and performs the simulations several days before (one and a half weeks) to ‘‘charge” the building with the energy needed to start the calibration period with the same thermal inertia. With this script we have obtained better results than with the convergence warm-up settings of EnergyPlus.

4. Optimisation results In this section we will analyse the optimisation results obtained by the genetic algorithm taking into account the seven objective functions. As described in Section 3.2.1, some of them are multiobjective (two or three targets) and others are single objective (one target). Initially we will evaluate the behaviour of each objective function globally and then we will compare its choices within the search space established by the sensitive analysis. 2

EnergyPlus input data file.

Fig. 4 shows the representation of all the simulations in each particular case. Each blue dot corresponds to one simulation guided by the optimisation process. The algorithm tries to find the best set of parameters to minimise the objective function. As can be seen, the form of the first three charts is a line of dots, because the uncertainty indices (CVðRMSEÞ; GOF; f i ) have been analysed individually. For convenience with the graphical representation the objective is duplicated. The following three are the representation of simulations taking into account each pair of objectives. The red dots represent the lower limit of all the simulations performed in each optimisation process, and are known as pareto optimal solutions, and the line joining them the pareto frontier. All are good solutions that need to be analysed individually, because as Coakley et al. [10] explain: ‘‘calibration will always remain an indeterminate problem which yields a non-unique solution”. The last chart is a 3D representation taking into account all the objectives, the black dots are the projection of blue dots in each Cartesian-plane. The optimisation process produces a simulation representation similar to a ‘‘comet”, with more dots in the ‘‘head” than in the ‘‘tail”. The search process of the algorithm chooses the parameters that minimise the objective functions. The concentration of dots means that the best solution is close to the ‘‘head” parameters. As previously mentioned, sensitive analysis defines the search space for optimisation. In each generation of the process the algorithm decides which parameters must change and which must not. Fig. 5 shows the selection made by each objective function. The figure is formed by 13 charts, one for each parameter (see Table A.1 in Appendix A). The horizontal axis of each chart shows all the possible values for each parameter, and vertical axis the number of times that the algorithm has selected it. Vertical bars in each chart represent the different objective functions analysed (from left to right the colours that identify each objective function are: darkblue - CVðRMSEÞ; blue - GOF; cyan - fi; lime - GOF & CVðRMSEÞ; green - fi & CVðRMSEÞ; sienna - fi & GOF and orange fi & GOF & CVðRMSEÞ). In general the solutions offered by the different objective functions are quite similar. Parameters like the U-factor and solar heat gain coefficient (SHGC) of glazings offer no doubt about the solution selected, unlike others such as the slab specific heat capacity (SHC) or Façade CE07 thickness where the solution is less clear.

5. Calibrated solution After performing the simulations it is necessary to analyse all the results to find both the best model and the objective function that generates it. Table 1 summarises this study. To have a clear

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

Fig. 4. Simulation results using different objective functions.

87

88

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

Fig. 5. Solution proposed by the different objective functions.

understanding of this table we have to define some concepts. Firstly, we are going to refer to all the cases performed by a specific objective function as the ‘‘set of simulations”. The number of evaluated cases is not always the same. This is because the NSGA-II algorithm works by sorting non-dominated solutions and keeps the best solution in each generation (elitism). Therefore, the total number of evaluated cases depends on the diversity produced by each objective function. Secondly, for each set of simulations, all uncertainty indices and all of its combinations have been calculated. This information allows us to identify another possible solutions in each set of simulations. This is because all the individual uncertainty indices of a calibrated BES model and its combinations must be ‘‘good” indices. Accordingly, all the calculated uncertainty indices of each set can help us to identify other possible calibrated models. Finally, because of the cases with the minimum uncertainty indices are very similar it is necessary to evaluate which one is the better choice. For this reason, each of them has been simulated again using a dynamic set-point temperature (through the first script) which is fed with the real building temperature. As we know that in the free-oscillation periods the building did not have any energy demand, the model which has less demand in these periods will be the better model. We call this measure in Tables 1 and 2 as the ‘‘demanded energy for calibration”. The results have been grouped in the sub-tables of Table 1 (one for each objective function considered). The first row of each subtable indicates the objective function selected. As we see in Section 3.2.1, seven objective functions are analysed and they are organised by number of objectives, from one objective to three. The column on the left of the table shows the order of the calculated uncertainty indices for each set of simulations. The first column of each set shows the minimum value for each calculated uncertainty index. In general, these values belong to different simulations of each set. When the calculated index matches the objective function, the row is in bold type. The second and third columns show the demanded energy for calibration in terms of energy (kW h) and as percentage reduction with respect the baseline model (%). The demanded energy for the baseline model is in the

upper-left-hand corner of the table. With this information it is possible to compare the results of each simulation (referring to each calculated uncertainty index) in order to establish which has achieved the lowest demanded energy for calibration. As the percentage has been calculated as a reduction with respect the baseline model, higher values means better models. For each set the best evaluated case has been shadowed in grey. It is interesting to see which uncertainty index causes a further reduction of energy. As can be seen, except when GOF is the objective function, the higher energy percentage reduction does not correspond with the objective function selected. Furthermore, sometimes one solution of each set has the minimum values of several calculated uncertainty indices; therefore it has the same values of energy reduction. The last row of each set indicates the number of simulations performed. As we have explained earlier, the number of cases evaluated for each objective function is different. This is due to the elitism of the NSGA-II algorithm mentioned above. The algorithm configuration used in this case-study was a population of 10 individuals and 200 generations, so the maximum number of simulations is 2.000. If the objective function produces more diversity (due to more objectives), the algorithm selects fewer best solutions in each generation and so needs to carry out more simulations. This value is useful to identify which objective function can obtain a better calibrated BES model in less time. It is also remarkable that there has not been any repetition among all the simulations performed by all the objective functions (12.202 simulations). As can be seen in Table 1, for all the simulations performed, the minimum demanded energy for calibration is found in the simulations with minimum values of the GOF uncertainty index (see shadowed values). This makes sense because the Goodness of Fit index is a combination between the CVðRMSEÞ and NMBE the first measures the value spread of the model (variability) and the second the percentage error between simulated and measured values. As a general rule with two objectives, if one of them is GOF the fitness of the model is higher. This single statistical index represents the overall Goodness of Fit of the model [11].

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

89

Table 1 Values of the different objective functions.

Regarding the statistical index R2 , Ashrae Fundamentals [45] says that it is ‘‘a widely used statistic to gage the goodness-of-fit of the model”, and recommend that the value never be less than 0.75. We have calculated this index for all the simulations per-

formed by each objective function, but not as an objective function. Our assumption was that in the set of simulations of each objective, probably the higher values of the statistical index R2 had also demanded less energy for calibration, but as can be seen in Table 1

90

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

Table 2 Best solutions of each objective function.

the higher values produce in general the worst energy reduction. Only taking into account this index with others the results are better. This fulfils the recommendation of IPMVP [5] that says ‘‘test should only be used as an initial check. Models should not be rejected or accepted solely on the basis of R2 ”. To summarise, Table 2 shows the best models obtained from each objective function. The first column indicates the objective function analysed; the second, the vector which represents the physical properties of each model (see Table A.1 in Appendix A); the next five columns, the uncertainty indices of each model; and the last two columns give information about the demanded energy for calibration. As in Table 1, the values of fi; CVðRMSEÞ; R2 and GOF are in percentages. The reason for use of four decimal figures is to observe the difference between indices with more detail, as the values of each solution are very close. The value of NMBE has been added to verify that all the simulations fulfil the requirements to be considered as calibrated solutions (see Table A.2 in Appendix A). As can be seen in Table 2 all of these seven simulations correspond to the ones grey shadowed in Table 1, so all of them refer to GOF index and its uncertainty GOF values match with those in Table 1. Comparing Table 2 with the one presented in [1], the demanded energy for calibration of the baseline model is higher because the period analysed has approximately 9 days more after adding six non-consecutive time periods (weekends). Proportionally our energy demand reduction is also higher since the model are composed of more thermal zones and have more baseline energy demand to reduce. The accuracy of the calibrated models obtained with this innovative methodology has been increased to over 52%, instead the 43% found in our earlier study [1]. In relation to the calibration criteria defined in Table A.2, all the solutions have a CVðRMSEÞ of around 4.3% and a NMBE of 0.09–0.44%, so they can all be considered as calibrated models. Now we will analyse in more detail the behaviour of one solution, the one guided by the f i objective function (see Table 2, the grey shadowed solution). Fig. 6 shows the seven free oscillation periods considered, the first three weekends of December, the Christmas period, and the last three weekends of January. The red curve represents the outdoor temperature, the green one the measured temperature, the dashed blue line the temperature curve that produces the baseline model, and the continuous blue line is the curve of the simulation analysed (GOF guided). The two curves at the bottom of the figure show the residuals (‘‘the gap”) between the measured temperature with respect to the baseline temperature (dashed brown line) and with the calibrated one (continuous brown line). The calibrated solution represents the behaviour of

the building better and, as can be seen, the ‘‘gap” between two curves rarely exceeds 1 °C. To compare these results with the ones obtained in [1], we have added the best solution obtained to Fig. 6. The black dashed thin line in the drawing shows the behaviour of that solution. As can be seen, the new solution (continuous blue line) generally fits better than the older one. One particular aspect to highlight in Fig. 6 is the unexpected behaviour of the last weekends. This behaviour is reduced as weekends pass (12 January ! 19 January ! 26 January) and it is related to what has happened before (Christmas period). The reason is the location of the temperature sensors. Thermostats are located in walls and after a long closed period their temperature is closer to the wall temperature than to the operative temperature of the room. As we use the information from the thermostats to reproduce the thermodynamic state of the real building in the BES model, the temperature we are using is influenced by this effect. In the first days after a closed period of 12 days, the walls are cooler than the interior environment, and for that reason the information from thermostats does not fit the operative temperature of the room. As soon as the building heats up, the temperature given by the thermostats is closer to the room operative temperature, as can be seen in Fig. 6. During this process the real building is being ‘‘energy charged”, and the time needed to finish the process depends on the building thermal inertia. This effect is proof of the high quality of the BES model achieved, because it has detected the problem. It is important to emphasise that if we are going to use the temperature of thermostats in a calibration process, it is necessary to analyse which aspects could affect its behaviour because some of them are not obvious (e.g. incident solar radiation, ventilation of HVAC systems, thermal inertia, etc.). In this case, it would have been desirable to place an additional sensor in each room to capture the operative temperature. Comparing the values of baseline and calibrated model, Fig. 7 shows a scatter plot that represents the dispersion between their temperatures with reference to the measured temperatures. The vertical axis shows the temperature of the calibrated (blue dots) and baseline (red dots) model. The horizontal axis represents the measured temperature. The relationship between them shows the dispersion of the sample. The grey line provides the perfect fit (measured vs. measured), so the closer the distance to it the better results. In general, blue dots fit better than red ones, which means that the calibrated model has improved the baseline. Finally, to see the importance that calibrated models have for all applied energy strategies, such as: ECMs, building commissioning

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

Fig. 6. Different temperatures during calibration periods.

Fig. 7. Temperature dispersion between calibrated and baseline model.

91

92

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

Fig. 8. HVAC energy demand profile of the different models: the baseline, the previous calibrated model [1] and the new one.

and FDD software the three models (the baseline model, the old calibrated one [1] and the new one) have been simulated using a constant setpoint temperature (23 °C) from Monday to Friday (7:00–19:00 h) and Saturday (7:00–16:00 h), the Pamplona SWEC3 weather and an ideal loads air system. Fig. 8 shows the comparison of its monthly HVAC energy demand profiles. The difference between its demands highlights the important role that energy models have in building design. For example, if the model is used to analyse different ECMs, inaccuracy in their energy consumption might affect the payback period of each measure; or if it used to calculate the maximum heating or cooling capacity of HVAC systems, its sizing calculation may not be enough at any time of year.

6. Conclusions Nowadays there is growing interest among architects and designers in the use of energy models that allow them to improve their design while keeping in mind one of the biggest environmental problems that affect the world: the reduction of building energy demand and therefore its greenhouse gas emissions. The accuracy of the models is a key element to carry out that mission because it will allow optimisation of decisions. Besides helping to solve this requirement, calibrated models have many other direct applications such as being a model-based control that could be used in the BAS (building automation system) for real-time control; taking into account the thermal energy storage in buildings for demand response applications; for on-going commissioning to detect building performance discrepancies of HVAC systems (fault detection diagnosis) and for Display Energy Certificates (DECs) that the building must achieve in use [40]. Also there are standards like ISO 50001, and protocols like EVO’s (Efficiency Valuation Organization) IPMVP [5], which demand the use of calibrated BES models to measure the energy efficiency of buildings. The aim of the methodology presented here is to provide a calibration strategy which serves to perform accurate BES models. Furthermore, the use of genetic algorithms intends to reduce the time needed to find calibrated BES models and make the process affordable (automated process). As stated above, the resulting models are predictive models of real building behaviour and have a good performance even in non-consecutive time periods (Fig. 6), which means that the model obtained has captured the heat dynamic of the building [46]. In addition, the analysis based on different uncertainty indices, which can guide the GA, is intended as advice about the most cost-effective way to reach the most accurate model. 3

Spanish Weather for Energy Calculations. Source: https://energyplus.net/weather.

In this article we maintain the advantages and characteristics of the methodology we proposed in [1], which we summarise as follows: the benefits of using free-floating periods; the improvement that the first script implied by enabling the model to acquire the thermal inertia needed to avoid the temperature dropping at the beginning of the free-floating period; the reduced investment that the use of temperature sensors implies; the dynamic heat achieved; etc. In the following conclusions we stress the specific advantages and improvements over our last proposed methodology:
Unrestricted number of thermal zones. The methodology enables the possibility of use of as many thermal zones as needed. The advantage of using a script outside EnergyPlus environment and its Python programming (easy to use and understand) allows us to use thermal zones without limitation.
It enables the possibility of using non-continuous free-floating periods. As a result, it is possible to use this strategy with the building in use considering only short periods in which HVAC systems are switched off, for example at night.
It provides the possibility of calibrating different zones of the building individually. Consequently, with the two previous advantages, it is possible to isolate different zones of the building and apply the methodology to calibrate them.
About uncertainty indices. After analysing the behaviour of the different uncertainty indices we conclude that all of them allow for good solutions; however our recommendation is to use only one index, GOF.4 This index brings together the NMBE and CVðRMSEÞ indices, so it allows us to assure good results that fulfil the requirements of FEMP, ASHRAE, and IPMVP for calibrated models. In addition, the use of one index facilitates its implementation (Python code) and reduces the number of simulations needed (due to elitism in NSGA-II).
Influence of temperature sensor location. As we have seen in Fig. 6, the situation of sensors can produce unexpected results in the calibration process. For this reason, it is important to consider the need for extra sensors in places where measurements incoherent with the behaviour of the building can be expected.

4 Although the best simulation has been found with the objective function f i , the solution corresponds to its calculated GOF index. Taking into account only the calculated index corresponding with the objective function, the best results have been obtained with GOF index.

93

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

7. Future works

References

As we stated at the beginning, gradually the interest in using calibrated BES models to improve the architectural design process is moving from the research community to the designers. The intention of the paper is to show a methodology to calibrate BES models quickly and accurately, however it still requires some skills such as Python programming that slow down its implementation. Currently, the Construction, Services and Structures Department of the University of Navarra is working on other options to improve the methodology to make it easy to use and currently it is producing good results.

[1] Ruiz GR, Bandera CF, Temes TG-A, Gutierrez AS-O. Genetic algorithm for building envelope calibration. Appl Energy 2016;168:691–705. [2] I.E. Agency, Promoting energy efficiency investments. Case studies in the residential sector. Tech rep. International Energy Agency (IEA), Agence Française de Dévelopment; 2008. [3] ASHRAE. Guideline 14-2002, measurement of energy and demand savings. Tech rep. American Society of Heating, Ventilating, and Air Conditioning Engineers, Atlanta, Georgia; 2002. [4] LiaWebster JB. M&V guidelines: measurement and verification for federal energy projects. Version 3.0. Tech rep. U.S. Department of Energy Federal Energy Management Program; 2008. [5] IPMVP committee and others. International performance measurement and verification protocol: concepts and options for determining energy and water savings, vol. I. Tech rep. Efficiency Valuation Organization; 2012. . [6] Raftery P, Keane M, O’Donnell J. Calibrating whole building energy models: an evidence-based methodology. Energy Build 2011;43(9):2356–64. [7] Claridge DE. Using simulation models for building commissioning. In: International conference on enhanced building operation, energy systems laboratory. Texas A&M University; 2004. [8] Yang Z, Becerik-Gerber B. A model calibration framework for simultaneous multi-level building energy simulation. Appl Energy 2015;149:415–31. [9] Clarke J, Strachan P, Pernot C. An approach to the calibration of building energy simulation models. Trans - Am Soc Heat Refrig Air Cond Eng 1993;99. p. 917–917. [10] Coakley D, Raftery P, Keane M. A review of methods to match building energy simulation models to measured data. Renew Sust Energy Rev 2014;37: 123–41. [11] Reddy TA, Maor I, Jian S, Panjapornporn C. Procedures for reconciling computer-calculated results with measured energy data. Tech rep. ASHRAE Research Project 1051-RP; 2006. [12] Heo Y, Choudhary R, Augenbroe G. Calibration of building energy models for retrofit analysis under uncertainty. Energy Build 2012;47:550–60. [13] Coakley D, Raftery P, Molloy P. Calibration of whole building energy simulation models: detailed case study of a naturally ventilated building using hourly measured data. In: First building simulation and optimization conference, Loughborough, UK. p. 10–1. [14] Manfren M, Aste N, Moshksar R. Calibration and uncertainty analysis for computer models–a meta-model based approach for integrated building energy simulation. Appl Energy 2013;103:627–41. [15] O’Neill Z, Eisenhower B. Leveraging the analysis of parametric uncertainty for building energy model calibration. Building simulation, vol. 6. Springer; 2013. p. 365–77.

Acknowledgements Finally we would like to recognise the assistance of the University of Navarra (Spain), and in particular its maintenance staff for providing us with both the building documentation and data from the sensors placed in it. In addition, we would like to thank Saviarquitectura Research Group and those responsible for the Master’s Degree in Environmental Management and Building Design (MDGAE) of the University of Navarra for the equipment loans. Appendix A. Reproduction of some tables of the previous work [1] Table A.1 shows the results of the sensitive analysis realised in [1]. The first column (A, B, etc.) related with the value number (value 1, value 2, etc.) shows the parameters selected by the genetic algorithm and will be useful to identify the solutions obtained in Section 5. Table A.2 shows the limit values established by the Measurement and Verification (M&V) of the Federal Energy Management Program (FEMP) [4]; the International Performance Measurement and Verification Protocol (IPMVP) [5]; and ASHRAE [3].

Table A.1 Sensitive parameters - search space for the genetic algorithm. Construction

A B C D E F G H I J K L M

Façade CE04c Façade CE06a Façade CE06b Façade CE07 Roof Top façade Slab Partition walls U-Factor Solar Heat Gain Coefficient

Type of parametrized value

Thickness (m) Brick density (kg/m3) Thickness (m) Thickness (m) Thickness (m) Insulation thickness (m) Gravel thickness (m) Thickness (m) Specific heat (J/kg K) Thickness (m) Density (kg/m3) W/m2 K Non-dimensional

Baseline model

Others values

Value 1

Value 2

Value 3

Value 4

Value 5

Value 6

Value 7

Value 8

Value 9

Value 10

0 1150 0 0 0 0.10 0.10 0 1000 0.35 1 1.4 0.6

0.05 1250 0.01 0.05 0.05 0.15 0.025 0.01 850 0.25 1000 0.8 0.4

0.06 1350 0.02 0.06 0.06 0.20 0.05 0.02 900 0.30 1100 0.9 0.5

0.07 1450 0.03 0.07 0.07 0.25 0.075 0.03 950 0.40 1200 1.0 0.7

0.08 1550 0.04 0.08 0.08 0.30 0.125 0.04 1050 0.45 1300 1.1 0.8

0.09 – 0.05 0.09 0.09 0.35 0.15 0.05 1100 0.50 1400 1.2 0.9

0.1 – 0.6 0.1 0.1 0.40 0.175 0.6 – – 1500 1.3 1.0

0.11 – 0.07 0.11 0.11 0.45 0.20 0.07 – – 1600 – –

– – – – – 0.50 – – – – 1700 – –

– – – – – 0.55 – – – – – – –

Table A.2 Calibration criteria of Federal Energy Management Program (FEMP), ASHRAE, and IPMVP. Data type Calibration criteria Monthly criteria % Hourly criteria %

Index

FEMP 3.0 Criteria

ASHRAE G14-2002

IPMVP

NMBE CVðRMSEÞ

5 15

5 15

20 –

NMBE CVðRMSEÞ

10 30

10 30

5 20

94

G. Ramos Ruiz, C. Fernández Bandera / Applied Energy 185 (2017) 82–94

[16] Robertson JJ, Polly BJ, Collis JM. Reduced-order modeling and simulated annealing optimization for efficient residential building utility bill calibration. Appl Energy 2015;148:169–77. [17] Peippo K, Lund P, Vartiainen E. Multivariate optimization of design trade-offs for solar low energy buildings. Energy Build 1999;29(2):189–205. [18] Catalina T, Virgone J, Blanco E. Development and validation of regression models to predict monthly heating demand for residential buildings. Energy Build 2008;40(10):1825–32. [19] Chow T, Zhang G, Lin Z, Song C. Global optimization of absorption chiller system by genetic algorithm and neural network. Energy Build 2002;34 (1):103–9. [20] Lu L, Cai W, Xie L, Li S, Soh YC. HVAC system optimization in building section. Energy Build 2005;37(1):11–22. [21] Chantrelle FP, Lahmidi H, Keilholz W, El Mankibi M, Michel P. Development of a multicriteria tool for optimizing the renovation of buildings. Appl Energy 2011;88(4):1386–94. [22] Caldas LG, Norford LK. A design optimization tool based on a genetic algorithm. Autom Constr 2002;11(2):173–84. [23] Turrin M, von Buelow P, Stouffs R. Design explorations of performance driven geometry in architectural design using parametric modeling and genetic algorithms. Adv Eng Inform 2011;25(4):656–75. [24] Magnier L, Haghighat F. Multiobjective optimization of building design using TRNSYS simulations, genetic algorithm, and artificial neural network. Build Environ 2010;45(3):739–46. [25] Saltelli A, Tarantola S, Campolongo F, Ratto M. Sensitivity analysis in practice: a guide to assessing scientific models. John Wiley & Sons; 2004. [26] Spitz C, Mora L, Wurtz E, Jay A. Practical application of uncertainty analysis and sensitivity analysis on an experimental house. Energy Build 2012;55:459–70. [27] Lü X, Lu T, Kibert CJ, Viljanen M. A novel dynamic modeling approach for predicting building energy performance. Appl Energy 2014;114:91–103. [28] Torio H, Schmidt D. A. ECBCS, low exergy systems for high performance buildings and communities (Annex 49 Final Report); 2011. [29] UNE-EN 13829. Thermal performance of buildings - determination of air permeability of buildings - fan pressurization method (ISO 9972:1996, modified); 2002. [30] Crawley DB. Which weather data should you use for energy simulations of commercial buildings? Trans - Am Soc Heat Refrig Air Cond Eng 1998;104:498–515. [31] Ruiz GR, Bandera CF. Importancia del clima en la simulación energética de edificios. In: Jornadas internacionales de investigación en construcción:

[32] [33]

[34]

[35]

[36]

[37]

[38] [39] [40] [41]

[42] [43] [44] [45] [46]

vivienda: pasado, presente y futuro: resúmenes y actas. Instituto Eduardo Torroja; 2013. Mustafaraj G, Marini D, Costa A, Keane M. Model calibration for building energy efficiency simulation. Appl Energy 2014;130:72–85. Nguyen A-T, Reiter S, Rigo P. A review on simulation-based optimization methods applied to building performance analysis. Appl Energy 2014;113:1043–58. Energy Efficiency & Renewable Energy. EnergyPlus energy simulation software. [accessed 15.11.15]. Robertson J, Polly B, Collis J. Evaluation of automated model calibration techniques for residential building energy simulation. Tech rep. NREL technical report 5500-60127; 2013. Tahmasebi F, Mahdavi A. Optimization based simulation model calibration using sensitivity analysis. In: Sikula O, Hirs J, editors. Simulace Budov a Techniky Prostredi. Tahmasebi F, Mahdavi A. Monitoring-based optimization-assisted calibration of the thermal performance model of an office building, Tirana, Albania, vol. 18; 2012. p. 2012–21. Tahmasebi F, Zach R, Schuß M, Mahdavi A. Simulation model calibration: an optimization-based approach. BauSIM 2012 1. Deb K, Pratap A, Agarwal S, Meyarivan T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evolut Comput 2002;6(2):182–97. Evins R. A review of computational optimisation methods applied to sustainable building design. Renew Sust Energy Rev 2013;22:230–45. Zhang Y, Korolija I. Performing complex parametric simulations with jeplus. In: SET2010 - 9th international conference on sustainable energy technologies; Shanghai, China. Zhang Y. Use jeplus as an efficient building design optimisation tool. In: CIBSE ASHRAE technical symposium, London. p. 18–9. U.S. Department of Energy. Energyplus application guide for ems (a.k.a. the book of energyplus runtime language); 2013. Machairas V, Tsangrassoulis A, Axarli K. Algorithms for optimization of building design: a review. Renew Sust Energy Rev 2014;31:101–12. ASHRAE. Handbook fundamentals. Tech rep. American Society of Heating, Ventilating, and Air Conditioning Engineers, Atlanta, Georgia; 2013. Cipriano J, Mor G, Chemisana D, Pérez D, Gamboa G, Cipriano X. Evaluation of a multi-stage guided search approach for the calibrationof building energy simulation models. Energy Build 2014;87:370–85.