The value of flexibility and sequential decision-making in maintenance strategies of infrastructure systems

Structural Safety 84 (2020) 101916

Contents lists available at ScienceDirect

Structural Safety journal homepage: www.elsevier.com/locate/strusafe

The value of flexibility and sequential decision-making in maintenance strategies of infrastructure systems

T

Santiago Zuluagaa, Mauricio Sánchez-Silvaa,

⁎

a

Department of Civil and Environmental Engineering, Universidad de los Andes, Bogotá, Colombia

ARTICLE INFO

ABSTRACT

Keywords: Infrastructure Flexibility Maintenance Sequential decisions Real options

The operation of infrastructure systems is a long-lasting managerial endeavour in which decision makers need to adapt to a changing environment and new events that result from an uncertain future. One of the essential aspects of infrastructure operation is maintenance, which guarantees acceptable operation standards for this kind of projects, which are normally subjected to several deterioration mechanisms. This paper discusses the concept of flexibility in maintenance planning for infrastructure systems and presents a model that allows to capture the value of flexible decisions in the context of sequential maintenance decisions. First, it presents an approach for addressing the sequential decision-making process of maintenance in infrastructure through an application of stochastic programming. Then, it describes and formulates the valuation of maintained systems under uncertainty through Real Options. Finally, the proposed methodology is studied through an illustrative example that shows the advantages of the proposed approach and highlights the role of flexibility in maintenance planning.

1. Introduction

In the context of infrastructure operation, decisions regarding maintenance (which are normally of large-scale and therefore result in significant capital requirements) are regularly analyzed through a lifecycle cost analysis (LCCA), which is mostly based in a discounted cash flow model that accounts for predicted future benefits and investments in order to give a measure of the value associated with the project (e.g. NPV). This methodology has become a commonplace in the decisionmaking for engineering projects; however, it has several limitations when evaluating and managing the uncertainty associated with future events and decisions [9]. Regarding the main issues associated with LCCA, existing literature highlights the estimation of future costs and investments, the selection of a representative discount rate, and the fact that LCCA does not incorporate the dynamics of decision-making derived from observing changes in the performance or the system state [10]. Life-cycle cost analysis also fails to consider the managerial flexibility that stakeholders have when operating a project, assuming that the investment is an all-or-nothing strategy; it fails to account for the fact that management can have an effect on the outcome of an investment over time when some of the uncertainties become known to the decision maker [11]. The study of flexibility in the operation of infrastructure systems is critical, as it can give valuable insight on how the value of a project is truly associated with possible future scenarios. Moreover, it is

Infrastructure systems are the foundation for most modern socioeconomic activities. An adequate operation of infrastructure projects should take into account the safety and availability of the system, while also seeking to minimize costs in order to achieve a feasible operation. One of the most significant aspects of infrastructure operation is maintenance, since these projects are designed to be long lasting but are also subjected to a wide range of deterioration mechanisms that make maintenance actions essential for a successful performance. Maintenance is defined by The British Standard BS3811 [1] as “the process of maintaining an item in an operational state by either preventing a transition to a failed state or by restoring it to an operational state following failure”. In the context of infrastructure systems, maintenance integrates both technical and administrative actions with the intention to retain or restore the state of an object or system that is subjected to one or more deterioration processes, to a level in which it can perform the functions required from it [2]. Maintenance has been studied extensively in the context of deteriorating systems, since it is very relevant on many engineering applications (e.g., civil, mechanical and electrical engineering, among others); there are several literature reviews available on the topic of maintenance in engineering [3–8].

⁎

Corresponding author. E-mail addresses: [email protected] (S. Zuluaga), [email protected] (M. Sánchez-Silva).

https://doi.org/10.1016/j.strusafe.2019.101916 Received 23 April 2019; Received in revised form 14 September 2019; Accepted 3 December 2019 0167-4730/ © 2019 Elsevier Ltd. All rights reserved.

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

important to study the nature of flexible decision-making, as it can lead to increases in economic value when benefits are better than predicted, and can also reduce the financial losses associated with underwhelming demand or income [12,13]. A useful alternative to analyze decision strategies is Real Option Analysis (ROA) [14], which takes into account the effect of flexibility in the financial evaluation of projects that face uncertainty over time, recognizing the presence of flexibility in present and future decisions. This paper has three main objectives: first, it presents an approach for addressing the sequential decision-making process of maintenance in infrastructure. In particular, it proposes a formulation of maintenance decisions as a stochastic optimization problem, and presents a solution based on simulation. The second objective of the paper is to describe and formulate the problem of assessing the value associated with the operation of maintained systems under uncertainty through ROA. However, the proposed valuation strategy is not implemented taking into account all the classical considerations of option analysis that are used in the context of financial assets, but it emerges from a more general form of option valuation that highlights the role of flexibility in the evaluation of engineering projects. Finally, this paper seeks to integrate the first two objectives, resulting in a novel formulation for evaluating maintenance strategies in infrastructure. The idea behind the proposed objectives is to integrate flexibility as a vital aspect for the analysis of infrastructure systems and the decisionmaking process for its maintenance. This paper is structured as follows. First, it presents a description of the problem of maintenance in infrastructure systems, reviewing the idea of traditional life-cycle cost analysis and discussing its limitations for addressing maintenance decisions in engineering practice. Then, it discusses the concept of flexibility from the perspective of policies and decision-making, and how it integrates with the estimation of the value of engineering projects. This section is followed by the mathematical formulation of a flexible strategy for the definition of maintenance policies, and afterwards, an option-based approach for the valuation of an infrastructure system. Finally, the proposed models are illustrated through an example.

Fig. 1. Two realizations of intervention sequences for a system with stochastic deterioration.

select a maintenance activity i, j that takes the system to a condition Vj (ti+) by making Vj (ti+) = k + i, j . The superscript “+” indicates the system state before and after the intervention, respectively. In most models, it is assumed that these interventions are instantaneous, although in practice they may take some time. The entire history of interventions in [0, tm] is a double sequence

= {(ti,

i, j ): ti

tm,

i, j

0}

(1)

In a dynamic maintenance process, once an intervention is executed the time to next intervention is also defined stochastically. Then, the larger the value of i, j , the larger the expected value of the time to the next intervention. Furthermore, if the deterioration process after an intervention is assumed independent of the state of the system (i.e. is not affected by the extent of maintenance activities), the mean time to next intervention would be [Yi + 1 |Vj (ti+)] = µj , where Yi + 1 is the time between the i-th and the i + 1-th interventions (with Y0 = 0 ). In other words, even when the time between interventions is uncertain, larger increases in capacity are expected to lead to longer periods of successful operation. As an example, Fig. 1 presents two different intervention sequences for a system with stochastic intervention times. Maintenance decisions have three main aspects to be considered: Firstly, the definition of a maintenance strategy (i.e., a way to evaluate possible maintenance alternatives) taking into account the implications on the future performance of the system. Secondly, the financial evaluation of the project: how to estimate the value derived from the operation of the system given the uncertainty in maintenance decisions; and lastly, the definition and estimation of the value of flexibility associated to the option of having a portfolio of possible interventions on a time-dependent system subjected to a stochastic degradation process.

2. The problem of maintenance in infrastructure 2.1. Problem description Consider a system with some time-dependent measure of capacity/ resistance given by the function V (t ) and an initial state V (0) = v0 . Let’s further consider that the system is subjected to an arbitrary deterioration mechanism, such that V (t ) = v0 D (t ) with t > 0 , where D (t ) is the total deterioration by time t. In addition, it is said that the system’s performance is acceptable as long as V (t ) k , where k is some predefined performance threshold (e.g., serviceability or failure limit states). In the context of large infrastructure, the limit state is frequently above total failure (or collapse, i.e., k > 0 ) as this state normally represents a serious hazard, which in turn leads to full replacement. In this paper, the selection of the limit state will not be discussed further; an extensive discussion can be found elsewhere [15,16]. In summary, the objective of operation is to keep an acceptable performance throughout a time window 0 t tm , where tm is called the time mission. In order to meet the project’s objective, the operator must carry out an intervention once the system reaches the threshold k . Note that it is assumed that the system is continually or quasi-continually inspected, which means that the operator knows the state of the system at any time. If the project’s performance reaches k , there is a set of possible 0} , where each interventions M = { i, j: i , j i, j represents a distinct level of restoration j at the i-th time of intervention. The case i,0 = 0 represents the case in which the operator decides to abandon the project, giving up the potential future benefits for the rest of the project’s lifetime. Then, at each intervention time ti tm , the operator has to

2.2. A review of the traditional analysis Traditional operation strategies are based on Life-Cycle Cost Analysis (LCCA). In LCCA, the target cost-benefit function used to assess the value of the operation of the project is evaluated with respect to the time mission tm [16]:

Z (tm, p) = B (tm, p)

C0 (p)

CL (tm, p)

CD (tm)

(2)

where p is a vector parameter that contains all relevant variables for the design and operation of the system, B (tm, p) is a function that describes the attainable benefits derived from the investment and operation of the project, C0 (p) is the initial implementation cost for the project (i.e., design, planning and construction costs), CL (tm, p) are all the costs 2

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

• The evaluation of the discounted costs involves estimating the un-

associated to the appropriate operation of the project (e.g. costs of maintenance), and CD (tm) is the cost of decommissioning (if existent) at the end of the life cycle (t = tm ). If p considers uncertainty in some parameters of the system, as is the case of the maintenance problem described above, then the cost-benefit function can be rewritten in terms of the expected value:

[Z (tm, p)] =

[B (tm, p)

C0 (p)

CL (tm, p)

CD (tm)]

(3)

•

Maintenance planning is concerned with estimating the time of interventions and the extent of the repairs [16]. If the operator wants to minimize the expected costs of maintenance through the lifetime of the system, then he must select a maintenance policy , i.e., a collection of intervention times and restoration sizes (see Eq. 1), that minimizes costs. Generally, this process is based on engineering judgement, and in many cases, it is prescriptive as it does not take into account the characteristics of the system or performance data [17]. Classical maintenance strategies that are widely used in engineering practice include periodic and age-dependent policies, in which the system is restored to its initial capacity. Even in the case of more effective strategies such as condition-based maintenance policies [18], in which the operator performs periodic inspections but not necessarily performs maintenance actions at each of them, the selection of the inspection rate is often prescriptive and does not capture the dependency between the sequential decisions made throughout the lifetime of the system. Because traditional LCCA does not allow evaluating decisions that are made based on future information, as is the case of sequential maintenance decisions, the selection of a policy is reduced to selecting a fixed intervention size that should be performed every time the 0 . There is exi, j system reaches the threshold k , i.e., i, j = tensive literature discussing the optimization of these parameters; see [19–23]. Let T = {t1, t2, …, tn |ti tm} be a set of intervention times that occur before the time mission of the project, tm . Then, this selection process can be expressed as an optimization problem in which the operator seeks to minimize the expected discounted costs related to maintenance actions: n

min

i, j M , T i = 1

tm 0

C(

i, j ) fi

(t ) e

t dt

In the following sections, these limitations will be addressed from the perspective of flexible policy design and option valuation. 3. Flexibility in management and decision-making 3.1. The concept of real options and managerial flexibility Flexibility is a multi-disciplinary concept that means different things depending on the context or the author; some literature reviews have identified over 50 concepts and definitions related to the term flexibility in the context of manufacturing processes [27,13]. However, the common ground upon which all disciplines come together is the fact that flexibility is essential in order to face uncertainty and change, and that it implies a capability or ease of modification, avoiding irreversible or rigid decisions [26]. In the context of management and decision-making, flexibility is a valuable way of dealing with market or context uncertainty. In competitive markets, uncertainty and volatility are two attributes that limit the amount of important information available to make future investment decisions [11,13]. Therefore, when faced with long-term financial investments, it is important to hedge the risk associated to unplanned scenarios by creating investment plans that include contingent decisions which could provide protection against unforeseen negative market conditions, and also provide the ability to harness unexpected growth opportunities should the market uncertainty unfold positively [12,28]. One of the growing bodies of literature that studies managerial flexibility in the context of capital investment is option theory. Particularly, Real Option Analysis (ROA) is an application of the financial option theory that allows to analyze real or physical assets [11]. In the financial context, an option is a contract that gives the owner or holder the right but not the legal obligation to conduct a transaction involving an underlying asset (e.g., the purchase or sale of the asset) at a fixed future date or within a period of time and at a predetermined price [14]. When extended to real assets, such as an infrastructure project, a real option is defined as the right, but not the obligation, to make a certain management decision, such as abandoning, expanding, staging or contracting an investment within the project [11,29]. Contrary to standard valuation methodologies such as the discounted cash flow analysis, where investments are assumed all-ornothing strategies, ROA highlights the importance and the implications of new information that becomes available as the project unfolds. In the context of infrastructure, where projects usually have long time missions and are subjected to important sources of uncertainty (e.g., demand, environmental conditions, technological advancements, among others), ignoring the dynamic nature of decision-making by using traditional valuation methodologies such as the discounted cash flow analysis may lead to incorrect investment decisions and to a

(4)

where C ( i, j ) is the cost of executing an intervention of size i, j, fi (t ) is the density of the time to the i-th intervention [16] and is the discount rate. Traditional LCCA has various limitations regarding both the selection of maintenance policies and their financial evaluation. Problems associated with traditional maintenance policies include:

• They have a prescriptive approach to define interventions since they

•

certainties that come with future investment decisions. This is not the same as estimating the uncertainty associated with the model and the parameters; it involves the attitude towards risk, new investment opportunities, and other stakeholder interests. Thus, analyzing maintenance decisions as static discounted cash flows fails to account for the unplanned variations in the decision-making environment [25], which result from the contingent nature of maintenance interventions. LCCA, through the discounted cash flow model, fails to recognize the financial value of incorporating flexibility. Maintenance decisions are made inside of an essentially uncertain context, in which critical information for investment decisions is either unknown, or known to change with little predictability [26]. Therefore, measuring the value of the operation of the project without accounting for flexibility can result in a misestimation of the future maintenance costs [9].

are not able to adapt to the dynamic nature of a system subjected to uncertainty. In particular, the selection of maintenance and inspection parameters for traditional maintenance policies is often rigid and does not update dynamically as the real performance data for the project unravels. This results in rigid maintenance policies that may misestimate the real performance of the system in cases when it does not adjust to the expected conditions predefined when designing the policy. They do not take into account the impact of interventions on the future behavior of the system, resulting in myopic decisions [24]. Given that there are various possible interventions when the system’s state needs to be restored, it is important to consider that selecting a specific intervention will have an effect on the selection of future interventions.

On the other hand, traditional valuation methodologies used in the context of maintenance, such as discounted cash flow model, have various limitations: 3

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

misestimation of the project’s value [30]. It is important to note that the study of flexibility in management decisions, such as the study of real options, is focused on the financial value of flexibility associated with having the possibility of changing the course of action of a project. This is different from the flexibility of the system or project itself, as the measure of an attribute and the attribute itself are not the same [26]. For instance, the robustness of a system is not the same as the financial value that its robustness represents for the operator. Likewise, while ROA presents a frame of work that allows to calculate the value of managerial flexibility in the operation of a project, it does not measure or address the flexibility of the system that is being operated.

4. Flexible maintenance strategy 4.1. Formulation of the strategy In this section, we present a look ahead strategy that describes the sequential decision-making process behind a maintenance policy. In this case, the set of intervention sizes is taken as a discrete set with m possible options at each one of n interventions: … i, m . For M = { i, j: i = 1, 2, …, n; j = 1, 2, …, m} with 0 i,1 i,2 simplicity, the available intervention sizes will remain constant throughout time, so the operator will have the same possible maintenance actions at every time of intervention ti ; however, the model allows integrating different availability of maintenance activities over time, which could reflect future technological advancements that give the operator new intervention possibilities at future times. In this strategy, the system starts operating at initial level of capacity v0 , which determines the time up to when the system reaches a state V (t1) = k ; i.e., the time to the first intervention. At time t1, the operator enumerates all the possible future sequences of intervention times and sizes k with k = 1, 2, …, K . Intervention times are estimated based on the expected values of the times between interventions which depend on the system state after the previous intervention; i.e., [Yi + 1 |Vj (ti+)] = µj . The set which contains all possible intervention sequences is derived from a tree-like structure, and the process of finding a sequence of interventions (i.e. a policy) that minimizes the cost of maintenance activities throughout the lifetime of the system is known as tree search [24]. Once the policy with the minimum cost = {(ti, i, j )} is identified, the first intervention 1, j M that takes the system to V (t1+) = k + 1, j is carried out according to the optimal sequence . The new system state V (t1+) leads to a new intervention time t2 which is defined stochastically given the previous decision and the process of defining and evaluating a tree of future possibilities is repeated; then, the system is intervened accordingly. This procedure is performed iteratively until the intervention time surpasses the time mission. Fig. 2 shows a graphical representation of the enumeration process for the first intervention of a system with four possible maintenance activities. In the upper part of the figure, we see the evolution of capacity with time for a system that has reached the limit state k at time t1. Each possible decision alternative (i.e. target capacity after intervention) is marked with circles. Note that greater increases in capacity lead to later intervention times (in expected value). If the process is repeated, a spanning tree of alternatives can be constructed, as shown in the lower part of the figure. The enumeration of alternatives must be done sequentially; this is, each branch of the tree that spans out from the initial decision at time t1 represents a specific sequence of maintenance decisions. The tree spans until the expected time of intervention is beyond the time mission tm . Note that every branch of the tree is a possible sequence of decisions k for which the discounted cost can be computed. The figure does not show the full tree of sequences up until that point, but only spans out twice in order to illustrate the process.

3.2. Sequential decision-making strategies and policies As seen in Section 2, the problem of selecting a maintenance policy can be expressed as an optimization problem where the operator seeks to minimize the expected costs related to the maintenance actions of the project, while monitoring the stochastic degradation process to which the system is subjected. In general, optimization problems that take into account decisions over time, which normally involve a sequential cycle between making decisions and observing new information, are known as sequential decision problems [24]. There are several approaches to solving this type of problems, depending on their deterministic or stochastic nature, and their size; these include linear programming, dynamic programming, stochastic programming and simulation optimization, among others [31]. In the context of dynamic programming, which is devoted to the study of sequential decision problems, a policy is defined as a rule or function that determines a decision given the information available for the state of the system at a given point in time [24]. There are many types of policies, which are classified depending on the problem at hand, their computational requirements, and their complexity. In this paper we will adopt the nomenclature presented in [16], which defines rules or functions for determining actions as strategies, and a policy as:

= {( i, i )}i

(5)

where is a double sequence that contains pairs of intervention times i at which the performance of the system is improved by the amount i . For simplicity, in this case we drop the subindex j that appears in Eq. 1, and define i as the intervention size at the i-th intervention. An approach that has been extensively used in literature for solving this kind of problems is to model the system states as a Markov chain [16]. Even when it allows to model the dependency between sequential decisions, this approach does not allow to alter the possible states of the system at future times or their correlation with the state of the system at a particular point in time. In this paper, we will focus on a more general approach that allows to incorporate dynamic decision spaces and correlations between alternatives and future states. Another approach to correcting the limitations of myopic policies is to make a decision now that looks into the future by generating possible future scenarios that stem from each available course of action and optimizing explicitly over some time horizon; this methodology is known as a look ahead strategy [24,32]. The simulation and optimization of future scenarios can be made explicitly by enumeration of all the possible action sequences and possible outcomes over a defined time horizon (also known as tree search). It becomes apparent that this methodology can only be used for problems with limited action spaces and short time windows, since it can be computationally intensive and the process intractable [24]. For more complex problems, there are other options such as sparse sampling, which limits the number of possible outcomes compared to the tree search within some error bounds, or roll-out heuristics that allow to approximate the value associated with future scenarios in order to optimize over present decisions. A detailed mathematical description of these strategies is beyond the scope of this paper and can be found in [24,33,34].

4.2. Optimization problem Let = {(ti, i, j )}i N be a maintenance policy with i, j M the intervention of size j at time ti . Let’s also consider that M is a discrete set with j = 1, 2, …, m . In addition, let ai, j {0, 1} be the decision variable, which takes the value of 1 if i, j M is executed and 0 otherwise. Finally, the counting process that defines the number of times the system reaches its limit state k throughout its lifetime (see [16] for further details on this type of process) is N = {N (t ), 0 t tm} . Then, as in Section 2, the objective is to minimize the discounted maintenance costs during the lifetime of the system, i.e., 4

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

distributions [35]. Thus, this problem cannot be evaluated sequentially using regular optimality equations (a common solution in dynamic programming), but needs to be solved by simulating a large number of realizations of the process and optimizing over the expected values of every feasible policy, i.e. performing a tree search [24]. 4.3. Implementation of the strategy For a reasonable number of interventions, as is the case in large infrastructure projects, simulation is a good alternative to address the tree search problem; any analytical solution would most likely be intractable due to the complexity of the problem and the stochastic nature of the process. In this case, the simulation of the decision process includes optimizing over a set of possible maintenance policies, and observing the times where the system reaches its limit state k . A flow diagram for the simulation is presented in Fig. 3. The first step of the process is to define the time horizon tm , the cost function C (·), the probability distribution of the times to between interventions Fj (·) , the discount rate , the number of realizations N, the initial state of the system v0 , and the limit state k . Also, it is required to define the parameters and variables for the decision process; these include the set of possible interventions and the sizes associated M , and the mean times between interventions for every level of intervention, µj . A realization consists on selecting the times and sizes of interventions required to keep the system operating until time tm ; one realization is constructed as follows. First, given the initial state of the system v0 , a random time to the first intervention Y1 is randomly selected from F1 (t ) (i.e., distribution of the time to first intervention conditioned on an initial system state v0 ), so that t1 = y1. This is followed by the enumeration of all the possible intervention sequences (called policies, ), under the assumption that future interventions will occur at mean times between interventions, i.e., µj ; This results in a tree structure, , as shown in Fig. 2. Then, the discounted total cost Ck associated each policy k is calculated. The details of this evaluation are described in Section 5. This allows selecting the policy with the minimum expected discounted cost, . Based on this analysis, the first intervention size 1, j is executed and the pair (t1, 1, j ) becomes the first term in the sequence p1. Then, the time to next intervention is generated randomly from F2 (t ) = P (Y2 t|V (t1+)) . After the tree search is carried out, a value of 2, j is obtained and the pair (t2, 2, j ) is added to the sequence p1. The process repeats until the system reaches the time mission, t tm , resulting in the sequence p1 = {(t1, 1, j ), (t2, 2, j ), …, (tn, n, j )} . This process is repeated for a large number of realizations, yielding N possible sequences, i.e., P = {p1 , p2 , …, pN } . The distribution of each intervention time ti N can be only computed through simulation, since the time to each intervention is the sum of the times between previous actions, and the distribution of each of these depends on the selected intervention sizes i, j for all previous decisions in each realization of the process. Thus, the distribution of the time to each intervention for the process cannot be modelled as the sum of identically distributed random variables; its distribution can only be approximated from the simulated realizations of the operation of the system (described above). In a similar manner, all other dependent variables (e.g. the maintenance cost throughout the lifetime of the system) and their distribution may only be obtained by simulation, and derived from the sequences of selected interventions over a large number of realizations, i.e., P . It is important to note that this process does not result in a fixed intervention sequence (i.e., a policy) that should be used when operating of the system; it is an evaluation strategy that allows to find the expected optimal maintenance decisions at the times of intervention given the relationship between maintenance decisions and the possible outcomes of the operation of the system. In addition, since this strategy

Fig. 2. Enumeration scheme for the intervention policies in a system with 4 possible interventions.

min

C i N

s. t.

ai , j

e

i, j

ti

j m

a i, j = 1

i

N

j m

ti ti 1 = Yi i N Yi + 1 ~Fj (t ) i N |ai, j = 1 a i, j

{0, 1}

i

N, j

m

= {ai, j : i N , j m}, is the discount interest rate, C (·) is where the cost function which depends on the size of the maintenance carried out at time ti, Yi + 1 is a random variable that describes the time between the i-th and the i + 1-th intervention of the system, and Fj (t ) is the distribution of Yi + 1 conditioned on the selected size of maintenance i, j ; i.e.,

Fj (t ) =

[Yi + 1

t | ai , j ]

(6)

Given that the times between interventions {Yi + 1: i N } depend on previous interventions, they do not have the same distribution; this leads to a process in which the decisions are made at different points in time in every path. Some authors have studied stochastic programs where there is endogenous uncertainty (i.e., the uncertainty of the problem depends on some or all decision variables), but the applications are limited to less complex programs and discrete probability 5

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

Fig. 3. Flow chart for the computation of the flexible maintenance strategy P over N realizations.

uses expected times between interventions µj to estimate future costs, any decision made at a specific moment in time will result in the same minimum cost solution; i.e., there exists a unique optimal solution M for each time ti (0, tm) that minimizes the expected future i, j intervention costs for the system.

future decision) to the present, time in which the whole value of the project has been considered [11]. In this application, maintenance decisions have been selected through the look ahead strategy presented in Section 4. Then, the decision would be to either maintain the system (under the policy found with the look ahead strategy) or to abandon it if the expected future benefits of operating the project are not favorable even when performing the best possible maintenance activities. This is a reasonable assumption under the perspective of a Public–Private Partnership (PPP) structure, where often the operator is given the option to abandon the operation of a system incurring in a previously defined penalty. However, the details of this problem are beyond the scope of this paper and will not be discussed further.

5. Derivation of an option-based valuation approach 5.1. Overall description of Real Options Analysis This section presents an application of Real Option Analysis (ROA), which is a dynamic valuation methodology that considers the value of flexibility in decision-making processes. In ROA, the expected value of the future benefits up to the moment where the next decision must be made (i.e., the next intervention) is computed at each decision time. Then, available courses of action are compared and the one that maximizes the present value of the project is selected. This process is carried out sequentially from the last decision (which does not depend on any

5.2. Financial valuation of sequential maintenance decisions Let the value of the project from the time of the i-th intervention up to the time horizon tm be denoted as Vi . As mentioned before, the first decision to be evaluated is the last intervention before the time mission, 6

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

normalizing the value of flexibility [36]:

Table 1 Available maintenance activities for the operator at each time of intervention ti . Activity (j)

i, j

20 40 70 100

m1 m2 m3 m4

FR = max

µj [Years]

[–]

(

tm tn

b (t ) (t

tn, )dt

C(

n, j ),

0

)

6. Example

(7)

6.1. Case description

where Vn is the value of the project discounted to the time of decision tn, b (t ) is the benefit function, (t , ) = exp( t ) is the discounting function with interest rate , and C ( n, j ) is the cost associated to the optimal n-th intervention n, j M found by implementing the look ahead strategy presented in Section 4. The max(·) function represents the choice made by the operator of continuing to operate the project, or to abandon it if the future benefits are not desirable compared to the intervention cost. Note that, if pertinent, a penalty can be implemented easily in this function to reflect the conditions of a specific contract structure. Consequently, the value Vi associated with the decision made at the time of the i-th intervention with 0 i < n , is computed as follows:

Vi = max

(

ti + 1 ti

b (t ) (t

ti, )dt + Vi + 1 (ti + 1

ti ,

f)

C(

i, j ),

Consider the case of a large structure that has been designed with an initial capacity measure v0 = 100 in order to operate through a time window of 100 years (0 t 100 ). This structure is subjected to external sources of degradation that make it reach its limit state k = 30 with a distribution that can be described by a Poisson process (i.e., the times between interventions are exponential). However, it is important to note that the mean time between these events depend on the maintenance actions performed by the operator as reported in Table 1. At each moment when the time-dependent capacity measure of the structure V (t ) goes below the predefined limit state k , the operator of the project has the option to select between four different maintenance activities with different associated restoration levels, and the option to abandon the operation of the project. The information regarding the set of available maintenance activities, possible restoration levels M and their associated mean time between interventions are presented in Table 1. In this example, it is assumed that each possible maintenance activity j has an intervention cost proportional to the intervention size i, j :

)

0

(8) where f is the risk-free interest rate. It is important to note that the future value of the project, which is captured in Vi + 1, has to be discounted with a risk-free interest rate f that is normally assumed smaller than the risk-adjusted rate . This happens because the market risk associated with the future benefits from the project has already been taken into account when computing Vi + 1, and so the risk associated with having to make a posterior decision is a private risk, i.e., it is not affected by the uncertainties from the market, but only by the operator’s decision criteria [11].

C(

i, j )

= Cf + Cv

i, j

v0

(11)

where C ( i, j ) is the cost of making an intervention of size i, j M , Cf is a fixed cost associated to the intervention of the system, Cv is the variable cost associated with the extent of the intervention, and is a constant that adjusts the increase of the variable cost relative to maintenance size. For this example, we will consider a cost function as the one presented in Eq. (11) with parameters Cf = 100, Cv = 25, and = 0.7 . For example, the cost of performing the intervention m3, which restores 70 units of capacity would be:

5.3. Value of adding flexibility Since each step in the sequential valuation process captures an increasing portion of the project’s value, the value V0 associated with the last decision, made at t = 0 , represents the net present value of the project given all the decisions and cash flows. On the other hand, traditional project valuations are based on the Net Present Value (NPV); this is, the assessment of all the discounted cash flows at time t = 0 . In the proposed strategy, the discounted cash flow analysis is carried out at each intervention time; in other words, each step in the sequential valuation process captures an increasing portion of the project’s value. This reflects the operator’s ability to decide at each of these intervention times, in contrast to the all-or-nothing valuation provided by the NPV. By analogy to traditional discounted cash flows, this (partial) calculated value for the project is known as the expected net present value (eNPV); the relationship between the reference (traditional) net present value (NPV) calculated using a traditional methodology and the eNPV calculated through ROA is [11]:

eNPV = NPV + Flexibility

(10)

where FR is the value of flexibility relative to a reference NPV calculated through a discounted cash flow methodology. It is important to note that the value of flexibility appears because of the possibility to change the course of action through the selection of an optimal maintenance activity (as presented in Section 4), or through the option to abandon the project at an intervention time when the future benefits are not financially favorable for the operator, as seen above.

10 20 35 50

as it is independent of any future decisions. The last intervention, made at a time tn tm , the value of the project in the interval [tn, tm] can be computed as:

Vn = max

Flexibility ,0 NPV

C (70) = 100 + 25

70 100

0.7

= 119.48

(12)

The initial cost for the project, which accounts for the design and construction of the structure, is C0 = 150 . On the other hand, the values for the instant benefit function b (t ) will be simulated using a Geometric Brownian Motion with a fixed volatility, in order to reflect the uncertainty in the benefits derived from the operation of the project. It is important to note that this benefit process may change according to a specific modelling scenario and that the one presented in this example is only used to show that the proposed approach may be implemented under several sources of uncertainty. This stochastic process can be expressed as [11]:

(9)

b (t ) = µ ( t) + b (t )

where the value of flexibility appears as the difference between the eNPV and the NPV, which is a global quantification of the value of flexibility. However, this value could be expressed by using other financial measures; a discussion on this matter is beyond the scope of this paper. Then, the relative value of flexibility FR can be measured by

t

(13)

where b (t ) is the instant benefit at time t , µ is a drift or growth parameter, is a volatility parameter for the process, and is a white noise random variable (i.e., it has a normal distribution with a mean of 0 and a variance of 1) for each time increment t . It is important to notice 7

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

Fig. 4. Sample paths for the stochastic benefit process b (t ) used in the example. Fig. 6. Bounds for the expected discounted future intervention costs through the lifetime of the system.

that, although the volatility parameter remains constant over time, the uncertainty associated to the value of b (t ) increases proportionally t . For this example, the drift to the advancement of time at a rate term µ in Eq. (13) will be taken as the risk-free interest rate f that reflects the natural increase of income over time. After discretizing each year into discrete instants, the value of the k-th income b (tk ) is calculated as:

b (tk ) = b (tk 1)[1 +

f

t+

t]

when implemented with optimal parameters, using a rigid strategy such as a standard renewal policy in which the system is restored to its initial capacity when reaching a predefined limit state is not the best way to carry out maintenance activities under uncertainty. Furthermore, the distribution of the optimal decisions made through the look ahead strategy shows that bigger restorations tend to be selected at earlier stages of the project operation. Since this decision strategy selects the optimal policy using the expected values of the time to the next intervention, the minimization of costs leads to selecting policies that end as close as the time mission as possible (i.e., the operator avoids overmaintaining). It should also be pointed out that these results are not a source of uncertainty for the times of intervention, since the optimality of these activities does not depend on previous decisions, but on all the possible sequences of future interventions using expected times between interventions. In order to put the optimization process in perspective, we computed the minimum and maximum values of the expected future costs for decisions made through the lifetime of the system, as presented in Fig. 6. This figure represents the estimated bounds for the future discounted intervention costs that result from taking a decision at time t (0, tm) . For example, the maximum expected discounted cost for future interventions at t = 10 , which would result from the most expensive maintenance policy using expected times for the time interval (10, tm) , is $204.69 units. On the other hand, the selected (optimal) intervention sequence that results in minimum costs yields an expected discounted cost for future interventions of $127.94 units. This are not the actual intervention costs for the future interval; they just represent the value of the objective function, i.e., the expected discounted future costs the operator uses to select the optimal interventions (presented in Fig. 5) under the look ahead strategy. It is important to note that the results presented in Figs. 5 and 6 are highly dependent on the cost function and its parameters, as well as the set of available maintenance activities. However, the discussion on what is the best approximation of the cost function or set of maintenance activities is beyond the scope of this paper, since this can change dramatically depending on the context in which the system is operating. As mentioned in Section 4, performing a Monte Carlo simulation of the operation process also allows estimating the distributions of the times to each intervention. After simulating N = 10 4 realizations of the system’s operation, the times to the first six interventions were computed. The computational time required to perform the simulations in MATLAB®was 153.49 s on an Intel® Core™i7-8565U Processor. Note that this time depends heavily on the decision space of the operator (i.e. maintenance alternatives), since the number of possible sequences at each intervention grows exponentially with the decision space. Since all the times between failures are exponential, the data was fitted to a Gamma distribution (which describes the sum of exponential random variables) by calculating the maximum likelihood estimates for the

(14)

where f = 0.03, and = 0.2; furthermore, three payments per year were assumed. In this case, a seed value b (0) = 10 and a risk-adjusted discount rate = 0.075 will be used for computing present values. In Fig. 4, five sample paths of the stochastic benefit process are shown. The purpose of this example is to compare the financial value of the project when using flexible maintenance strategies and the traditional approach. The simulation for the look ahead strategy was performed as described in Section 4, and the valuation of the project’s decisions was carried out using the options model presented in Section 5. On the other hand, the traditional approach was assumed to be a as good as new maintenance policy, in which the system was restored to its original capacity v0 at each moment of intervention; furthermore, the value of this policy was calculated using a standard discounted cash flow model in order to obtain a reference NPV. 6.2. Maintenance strategy and policy selection As mentioned in Section 4, there exists a unique optimal solution M at every possible time of intervention ti (0, tm) for the look ahead optimization problem, since the possible maintenance sequences are calculated using mean times between interventions, not realizations of these times. After performing the tree search at every time in the interval (0, tm) using the parameters defined for the example, the optimal maintenance sizes activities were computed for the system; the results are presented in Fig. 5. Firstly, it is important to note that the optimal decision is not unique throughout the lifetime of the system; this confirms the idea that, even i, j

Fig. 5. Optimal maintenance decisions through the lifetime of the system. 8

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

Fig. 8. Distribution of the net present value associated with the operation of the system using traditional and flexible approaches.

Fig. 7. Distribution of the times to the first six interventions for the system.

shape and scale parameters, a and b, for a distribution function of the form:

f (t | a , b ) = f (t ) =

1 t a 1exp b a (a )

t b

(15)

where f (t ) is the probability density function of t , a and b are the shape and scale parameters of the Gamma distribution, respectively, and (·) is the Gamma function. The results for the density f (t ) of each maintenance time is presented in Fig. 7; the estimated values of the parameters a and b are presented in Table 2, together with their 95% confidence intervals denoted as (a , a+) and (b , b+) , respectively. As seen in the Figure, the time to each intervention has considerable variations, making the selection of different optimal interventions feasible; however, this process allows to estimate the behavior of the system under a more realistic process of decision-making.

Fig. 9. Distribution of the relative value of flexibility associated with the operation of the system.

allows the operator to hedge the associated risks through the option of adjusting his policy as new information on the performance of the system becomes available. On the other hand, when the project has successful financial results for the traditional approach, flexibility allows the operator to take advantage of the favorable conditions and capture even more value in the operation of the project. As a result, the expected relative value of flexibility is [FR] = 5.07 , which is a considerable difference that may have important implications on the decisions made on the planning and design stages of the project. The results presented in this example show how the proposed model gives the operator the possibility of implementing flexibility in the decision-making process for maintenance activities and builds the framework for its analysis based on dynamic programming methods and Real Options theory. They also allow to make a more precise estimation of future scenarios and their consequences by showing possible financial impacts of choosing specific alternatives and highlight the role that flexibility plays in the context of maintenance planning. This is of vital importance in many infrastructure applications, where limited simulation of the decision-making process may lead to a misestimation of costs or may not reflect the actual possibilities that the operator has, resulting in poor decision support for practitioners.

6.3. Financial evaluation Based on the performed simulations, let us compare the present value of the project when using a traditional discounted cash flow model for the reference maintenance strategy (as good as new at each intervention), and the valuation given by the option analysis presented in Section 5. Fig. 8 shows the distributions for the net present value of the project using both approaches; note that these distributions overlap for some values of NPV. Furthermore, Fig. 9 shows the distribution for the value of flexibility relative to the NPV of the project FR (as defined in Eq. 9). The results show that incorporating flexibility when making maintenance decisions throughout the lifetime of the system, as opposed to defining a rigid strategy, is clearly convenient for the operator. While the expected NPV of the project under traditional conditions is of $20.68 units, the expected eNPV of the project (obtained using the look ahead strategy and option valuation) is of $433.67 units. It is important to note that the difference between the two values corresponds to the value of flexibility that exists in the managerial decisions of the project; the introduction of a more realistic decision and valuation methodology does not improve the financial conditions of the project itself. In this example, in which an important portion of the scenarios resulted in losses when taking the traditional approach, flexibility

7. Conclusions In engineering, the operation of maintained systems has been generally assumed as a process in which the decisions are made beforehand, and the uncertainty associated with the system is treated in a static manner. This traditional approach does not incorporate the value of flexibility associated to the process of decision-making in the face of uncertainty; therefore, the true value of operating the infrastructure is often misestimated. Maintenance planning, as one of the main decision processes behind the operation of infrastructure, should be assessed as a dynamic process in which new information helps to steer maintenance decisions in a way that adapts to the changing performance of the system, or even new maintenance possibilities. As has been done in

Table 2 Estimated parameters for the distribution of the times to each intervention with 95% confidence intervals. Intervention

a

t1 t2 t3 t4 t5 t6

1.12 3.08 6.66 13.48 26.47 55.42

a 1.10 2.99 6.40 12.75 24.40 48.99

a+

b

1.15 3.18 6.94 14.26 28.72 62.71

25.77 17.27 10.27 5.81 3.21 1.63

b 24.96 16.70 9.84 5.49 2.95 1.44

b+ 26.60 17.87 10.71 6.15 3.48 1.84

9

Structural Safety 84 (2020) 101916

S. Zuluaga and M. Sánchez-Silva

other areas such as finance, there is a need to address the evolving and uncertain nature of decisions when operating infrastructure systems over time, in order to maximize the value of the project for the involved agents and stakeholders. This paper presents a model that integrates two different aspects of managerial flexibility into the decision process related to maintenance activities. First, it studies the problem of how maintenance activities are selected taking into account the uncertainty in the reliability of the system and the future decisions associated to its maintenance; this is done through an application of a look ahead strategy, which emerges from the context of stochastic dynamic programming. The paper focuses on finding a maintenance strategy that adapts to the realization of the system’s degradation, and that minimizes the costs associated with maintenance through the lifetime of the project. Then, the financial value associated with maintenance decisions is studied through a model derived from option theory, as it allows to capture the value of flexibility in decision-making processes. These models are illustrated through an example that shows the dynamic nature of maintenance decisions in the context of infrastructure; it also shows how uncertainty can be incorporated into these decision processes and how it is related to the value of the operation of infrastructure. Finally, the example highlights the role and the importance of flexibility in decision-making, seen as the possibility of correcting the course of action through maintenance decisions that take into account information that becomes available as the system operates.

[9] Sánchez-Silva M. Flexibility of infrastructure management decisions: the case of a project expansion. Struct. Infrastruct. Eng. 2019;15(1):72–81. https://doi.org/10. 1080/15732479.2018.1486439. [10] Damodaran A. Investment valuation: tools and techniques for determining the value of any asset. Wiley; 2012. [11] Mun J. Real options analysis: Tools and techniques for valuing strategic investments and decisions vol. 137. John Wiley & Sons; 2002. [12] De Neufville R, Scholtes S. Flexibility in engineering design. MIT Press; 2011. [13] Sánchez-Silva M. Managing infrastructure systems through changeability. ASCE J. Infrastruct. Syst. 2019;25(1):04018040. [14] Black F, Scholes M. The pricing of options and corporate liabilities. J. Political Econ. 1973;81(3):637–54. https://doi.org/10.1086/260062. [15] Valdez-Flores C, Feldman RM. A survey of preventive maintenance models for stochastically deteriorating single-unit systems. Naval Res. Logist. (NRL) 1989;36(4):419–46. [16] Sánchez-Silva M, Klutke G-A. Reliability and life-cycle analysis of deteriorating systems, springer series in reliability engineering. Springer International Publishing Cham; 2016. https://doi.org/10.1007/978-3-319-20946-3. [17] Pandey M. Probabilistic models for condition assessment of oil and gas pipelines. NDT & E Int. 1998;31(5):349–58. https://doi.org/10.1016/S0963-8695(98) 00003-6. [18] Rao B. Handbook of condition monitoring. Elsevier; 1996. [19] Barlow RE, Proschan F. Mathematical theory of reliability. vol. 17. Siam; 1996. [20] Aven T, Jensen U. Stochastic models in reliability. Springer; 1999. [21] Nakagawa T. Maintenance theory of reliability. Springer Science & Business Media; 2006. [22] Junca M, Sanchez-Silva M. Optimal maintenance policy for a compound poisson shock model. IEEE Trans. Reliab. 2013;62(1):66–72. [23] Junca M, Sanchez-Silva M. Optimal maintenance policy for permanently monitored infrastructure subjected to extreme events. Probab. Eng. Mech. 2013;33:1–8. [24] Powell WB. Approximate dynamic programming: solving the curses of dimensionality. 2nd ed. John Wiley & Sons Inc; 2011. [25] Evans JS. Strategic flexibility for high technology manoeuvres: a conceptual framework. J. Manage. Stud. 1991;28(1):69–89. [26] Saleh JH, Mark G, Jordan NC. Flexibility: a multi-disciplinary literature review and a research agenda for designing flexible engineering systems. J. Eng. Des. 2009;20(3):307–23. [27] Sethi AK, Sethi SP. Flexibility in manufacturing: a survey. Int. J. Flexible Manuf. Syst. 1990;2(4):289–328. [28] Triantis AJ, Hodder JE. Valuing flexibility as a complex option. J. Finance 1990;45(2):549–65. [29] Martins J, Marques RC, Cruz CO. Real options in infrastructure: revisiting the literature. J. Infrastruct. Syst. 2013;21(1):04014026. [30] Smit H, Trigeorgis L. Strategic investment: real options and games. Princeton University Press; 2012. [31] Fu MC. Optimization for simulation: theory vs. practice. INFORMS J. Comput. 2002;14(3):192–215. [32] Camacho EF, Bordons Alba C. Model predictive control. London: Springer; 2007. [33] Kall P, Wallace S. Stochastic programming, Wiley-Interscience series in systems and optimization. Wiley; 1994. [34] Birge JR, Louveaux F. Introduction to stochastic programming. Springer Science & Business Media; 2011. [35] Hellemo L, Barton PI, Tomasgard A. Decision-dependent probabilities in stochastic programs with recourse. CMS 2018;15(3–4):369–95. [36] Cardin M-A, Ranjbar-Bourani M, de Neufville R. Improving the lifecycle performance of engineering projects with flexible strategies: example of on-shore lng production design. Syst. Eng. 2015;18(3):253–68. https://doi.org/10.1002/sys. 21301.

References [1] Misra KB. Handbook of Performability Engineering. London: Springer-Verlag; 2008. [2] Sánchez-Silva M, Frangopol DM, Padgett J, Soliman M. Maintenance and operation of infrastructure systems: review. J. Struct. Eng. 2016;142(9):F4016004. https:// doi.org/10.1061/(ASCE)ST.1943-541X.0001543. [3] Pierskalla WP, Voelker JA. A survey of maintenance models: the control and surveillance of deteriorating systems. Naval Res. Logist. Q. 1976;23(3):353–88. https://doi.org/10.1002/nav.3800230302. [4] Sherif YS, Smith ML. Optimal maintenance models for systems subject to failure-A Review. Naval Res. Logist. Q. 1981;28(1):47–74. https://doi.org/10.1002/nav. 3800280104. [5] Sherif YS. Reliability analysis: optimal inspection and maintenance schedules of failing systems. Microelectron. Reliab. 1982;22(1):59–115. https://doi.org/10. 1016/0026-2714(82)90051-8. [6] Valdez-Flores C, Feldman RM. A survey of preventive maintenance models for stochastically deteriorating single-unit systems. Naval Res. Logist. 1989;36(4):419–46. https://doi.org/10.1002/1520-6750(198908)36:4<419::AIDNAV3220360407>3.0.CO;2-5. [7] Cho DI, Parlar M. A survey of maintenance models for multi-unit systems. Eur. J. Oper. Res. 1991;51(1):1–23. https://doi.org/10.1016/0377-2217(91)90141-H. [8] Dekker R. Applications of maintenance optimization models: a review and analysis. Reliab. Eng. Syst. Saf. 1996;51(3):229–40. https://doi.org/10.1016/0951-8320(95) 00076-3.

10