Condition-based maintenance in the cyclic patrolling repairman problem

Journal Pre-proof Condition-based maintenance in the cyclic patrolling repairman problem Maik J.A. Havinga, Bram de Jonge

PII: DOI: Reference:

S0925-5273(19)30317-2 https://doi.org/10.1016/j.ijpe.2019.09.018 PROECO 7497

To appear in:

International Journal of Production Economics

Received date : 30 May 2019 Revised date : 27 September 2019 Accepted date : 27 September 2019 Please cite this article as: M.J.A. Havinga and B.d. Jonge, Condition-based maintenance in the cyclic patrolling repairman problem. International Journal of Production Economics (2019), doi: https://doi.org/10.1016/j.ijpe.2019.09.018. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier B.V.

*Title page including author details Click here to download Title page including author details: PatrollingRepairman_Revision2_TitlePage.pdf

Journal Pre-proof

Condition-based maintenance in the cyclic patrolling repairman problem Maik J.A. Havingaa , Bram de Jongea,∗ a Department

of Operations, Faculty of Economics and Business, University of Groningen, The Netherlands

Abstract

of

We consider the cyclic patrolling repairman problem combined with condition-based preventive maintenance. In the traditional cyclic patrolling repairman problem, one repairman inspects and repairs a set of machines in a fixed sequence. We introduce the possibility of performing preventive maintenance

pro

on the machines based on condition information. In order to determine optimal policies, we provide a Markov decision process formulation of the problem. Furthermore, this Markov decision process is used to analyze the performance of a control-limit policy, which is a commonly used heuristic for conditionbased maintenance problems. The control-limit policy significantly outperforms the traditional policy for the patrolling repairman problem with only corrective maintenance. The optimal policy uses a higher maintenance threshold for a machine if the repairman expects to be back at that machine quite fast, or

re-

if failure of another machine is imminent. The benefit of the optimal policy compared to the controllimit policy is largest for sufficiently detailed condition information, for relatively stable deterioration processes, and for medium corrective maintenance costs.

Keywords: Maintenance, Patrolling repairman, Condition-based maintenance, Preventive

Jo

urn al P

maintenance, Markov decision process, Control-limit policy

∗ Corresponding

author. E-mail address: [email protected].

Preprint submitted to Elsevier

September 26, 2019

*Manuscript Click here to view linked References

Journal Pre-proof

Condition-based maintenance in the cyclic patrolling repairman problem

Abstract We consider the cyclic patrolling repairman problem combined with condition-based preventive main-

of

tenance. In the traditional cyclic patrolling repairman problem, one repairman inspects and repairs a set of machines in a fixed sequence. We introduce the possibility of performing preventive maintenance on the machines based on condition information. In order to determine optimal policies, we provide a Markov decision process formulation of the problem. Furthermore, this Markov decision process is used

pro

to analyze the performance of a control-limit policy, which is a commonly used heuristic for conditionbased maintenance problems. The control-limit policy significantly outperforms the traditional policy for the patrolling repairman problem with only corrective maintenance. The optimal policy uses a higher maintenance threshold for a machine if the repairman expects to be back at that machine quite fast, or if failure of another machine is imminent. The benefit of the optimal policy compared to the control-

re-

limit policy is largest for sufficiently detailed condition information, for relatively stable deterioration processes, and for medium corrective maintenance costs. Keywords: Maintenance, Patrolling repairman, Condition-based maintenance, Preventive

1. Introduction

urn al P

maintenance, Markov decision process, Control-limit policy

The cyclic patrolling repairman problem originates from the 1950s and 1960s, a period from which the first scientific approaches to maintenance planning originate (Dekker, 1996). Mack (1957) was one of the first to study the cyclic patrolling repairman problem. The problem is characterized by a number of machines that are maintained by a single repairman. The repairman travels from machine to machine in a fixed sequence and repairs machines that are broken down. Most studies in this period mainly use renewal theory to analyze the performance of systems.

A comprehensive review on maintenance optimization is provided by De Jonge and Scarf (2020). Maintenance can broadly be divided into two types: corrective maintenance and preventive maintenance. Corrective maintenance (CM) has to be applied to a machine that has failed in order to make it operational again. Preventive maintenance (PM), on the other hand, can be used to improve the condition of an operational but deteriorated machine. Its aim is to postpone or to avoid failures. Generally, preventive maintenance is preferred over corrective maintenance, as in most cases it is less expensive to perform and requires less time than corrective maintenance. Most studies on the cyclic patrolling

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

repairman problem only consider corrective maintenance, thereby omitting the possibility of carrying out preventive maintenance.

Preventive maintenance can further be classified into time-based maintenance (TBM) and conditionbased maintenance (CBM). An overview of both methods is given by Ahmad and Kamaruddin (2012), and a comparison is provided by De Jonge et al. (2017). Under TBM, preventive maintenance is performed on a machine after a certain period of time, independent of the condition of that machine (e.g. Bouslah et al., 2018; De Jonge and Jakobsons, 2018). CBM uses the condition of a machine to decide whether maintenance should be performed (e.g. De Jonge et al., 2016; Olde Keizer et al., 2018). Since TBM does not take the state of the machine into account, substantial remaining lifetime of a machine can

Preprint submitted to Elsevier

September 27, 2019

Journal Pre-proof be wasted, and failure becomes more likely if a machine deteriorates faster than expected. However, TBM is easier to implement than CBM because maintenance actions can be scheduled beforehand and no investments are needed to monitor the deterioration state of a machine. Nonetheless, we see a trend from TBM towards CBM as monitoring becomes less expensive due to technological advancements (Choi et al., 2018). When a CBM policy is adopted, a machine needs to be monitored in order to learn its deterioration state. This can be done by (remote) continuous monitoring or by physical inspections by a repairman. With continuous monitoring, the state of a machine is known at all times. However, this method is very

of

expensive to implement and sometimes not possible at all. This is why in many applications, machines are inspected to learn their deterioration state.

In this paper, we consider CBM with inspections in the cyclic patrolling repairman problem. A single repairman is responsible for the maintenance activities of a given set of machines. Deterioration of each of

pro

the machines is modeled by a discrete-time Markov chain with a finite number of states. The repairman traverses the machines in a fixed cyclic order. On arrival at a machine he carries out an inspection to learn its deterioration state. Based on this state and the observed history of the other machines, the repairman decides whether maintenance will be carried out. We consider the problem from a cost perspective, and we aim to minimize the sum of the maintenance and downtime costs. We formulate a Markov decision process that can be used to determine optimal policies and to analyze a commonly used

re-

heuristic, i.e., the control-limit policy. Under such a control-limit policy, a machine will be maintained if, after inspection, the deterioration level of the machine exceeds a predetermined threshold. It turns out that both the control-limit policy and the optimal policy significantly outperform the pure corrective maintenance policy for almost all cases. Under the optimal policy, the decision whether to carry out maintenance turns out to not only depend on the observed deterioration state of the current machine,

urn al P

but also on the last observed deterioration states of the other machines. The threshold for carrying out maintenance is higher if the repairman expects to be back at the current machine relatively fast, or if failure of another machine is imminent. The benefit of the optimal policy compared to the controllimit policy is largest for sufficiently detailed condition information, for relatively stable deterioration processes, and for medium corrective maintenance costs. The remainder of this paper is organized as follows. In Section 2, we review literature that is related to the current study. In Section 3, we formally define the problem that we consider. In Section 4, we formulate a Markov decision process that can be used to determine optimal cost-minimizing policies and to evaluate control-limit policies. In Section 5, we numerically analyze various systems to gain insights into the problem. We end with conclusions and recommendations for future research in Section 6. 2. Literature review

We structure our discussion of related literature as follows. We begin with studies that assume that a machine has to be visited and inspected to observe its status, i.e., there is no remote monitoring. These

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

studies assume that machines are traversed in a certain fixed order, and that a machine is repaired if it is found in the failed state. Thereafter, we consider studies that assume that remote information of machines is available. This information is used to make dynamic decisions. Thirdly, we discuss other studies that are related to our study, although to a lesser extent. We end with the contribution of the current paper.

Studies that consider a set of machines that are visited in a certain fixed order and that are maintained when found broken down do not involve maintenance decisions. These studies are therefore typically devoted to the analysis of the system performance. Mack et al. (1957) were the first to study such a system, inspired by winding processes in the textile industry. These processes consist of transferring yarn

2

Journal Pre-proof from small packages, produced at the spinning stage of production, to larger cones which are needed for the weaving or knitting processes. During this process, thin, thick and weak points in the yarn are detected, cut out, and the broken ends joined together by an (automated) repairman so that winding can continue. The winding heads are inspected by a repairman in a fixed sequence and are repaired when an irregularity is detected during inspection. Since the repairman has to move between the winding heads and possibly take the time to repair them, irregularities can in general not be repaired immediately. This results in costs due to downtime. Whereas Mack et al. (1957) consider constant repair times, Mack (1957) extends this to stochastic repair times. Bunday and Mack (1973) consider a set of machines

of

with exponential lifetimes that are visited bi-directionally, i.e., first in one order, and then back in the reverse order. Bunday and El-Badri (1984) consider repairs that can be unsuccessful for machines that are visited in a cyclic order, and Bunday et al. (1985) do so for machines that are visited bi-directionally. Das and Wortman (1993) introduce the asymmetric patrolling repairman problem, including non-identical

pro

machines with varying repair and walking times. Furthermore, they also include potential unsuccessful repairs. Mittler and Kern (1997) also consider random repair and walking times and analyzed the waiting time distribution and the round-trip time distribution.

We continue with studies that adopt remote information of machines, i.e., failure of a machine is observed immediately without a visit to that machines. This is for instance realistic for (offshore) wind parks (Shafiee, 2015). Bertsimas and Van Ryzin (1991) introduce the dynamic traveling repairman

re-

problem. Required repairs arrive randomly over time at random locations in a Euclidean plane; these repairs require random service times. The aim is to determine a policy for service vehicles that minimizes the average waiting time. Jamil et al. (1994) consider the problem of locating the home base of a traveling repairman. Calls for service at nodes arrive according to independent Poisson processes. Jobs are placed in a queue and the repairman only travels back to the home base if the queue is empty after the completion

urn al P

of a job. Camci (2014) introduces the traveling maintainer problem. A route for carrying out preventive maintenance actions is planned, and this route is updated when a failure occurs. Camci (2015) assumes at most one maintenance action per machine during a given finite planning horizon. Again, a planned route will be changed when a failure occurs. L´ opez-Santana et al. (2016) and Rashidnejad et al. (2018) consider a setting with multiple repairmen and use a two-stage heuristic. First, preventive maintenance interventions are scheduled based on the ages of the respective machines. Thereafter, the routes of multiple repairmen are determined. Nguyen et al. (2019) consider a geographically distributed set of production sites, each consisting of multiple critical components. Preventive maintenance is scheduled based on the ages of the machines. A heuristic approach is used for the maintenance grouping and for the routing decisions. Failures are rectified by minimal repairs by local maintenance teams. We continue with some other studies that are related to our study. A limited number of repairmen who travel between a set of machines basically implies a limited repair capacity. Such capacity constraints can also be encountered in settings without geographically distributed machines, for instance in case of a repair shop with limited capacity, see e.g. De Smidt-Destombes et al. (2004). These type of problems are sometimes also referred to as machine interference problems, see e.g. Stecke and Aronson (1985) and

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Haque and Armstrong (2007). Another stream of related research is that on machines that are used for missions, with only a limited amount of time available for maintenance in between successive missions. Examples of such studies are Diallo et al. (2018) and Do et al. (2015). A final research area that is also related to the patrolling repairman problem, although less closely, is that of polling systems. Polling systems originate from queueing theory and occur when one server visits a set of queues in some order. Overview articles of polling systems are provided by Takagi (1988), Takagi (2000) and Boon et al. (2011). Polling systems are applied in e.g. computer networks (Takagi, 1991). The main contribution of our study is the availability of condition data about a set of geographically distributed machines. We assume that remote monitoring of the machines is not possible. This is 3

Journal Pre-proof justifiable as remote monitoring of conditions by using sensors is often either not possible or too expensive (Jardine et al., 2006). Therefore, inspections of the machines are required, and we do so in a cyclical order. Compared to earlier studies on the cyclic repairman problem, this introduces the possibility to perform preventive maintenance based on the conditions of the machines, which will in general lead to improved system performance and reduced costs. 3. Problem description We consider a discrete-time setting with n identical machines. These n machines work and deteriorate

of

independently and deteriorate according to a discrete-time Markov chain. Each machine has m+1 ordered deterioration states. State 1 is the best state (i.e. as-good-as-new), state m the most deteriorated but functioning state, and state m + 1 the failed state. The transition probability matrix of this Markov

pro

chain is denoted by P . The ordering of states implies that P has the increasing failure rate property, Pm+1 i.e., b(i, h) = j=h P [i, j] is nondecreasing in i for any h ∈ {1, . . . , m + 1} (Barlow and Proschan, 1965, p.154). In other words, if the current deterioration level is higher, it is more likely that any deterioration threshold h will be exceeded after a one-step transition. When in the failed state, the machine is down and can only become operational again by performing corrective maintenance. If the machine is not in the failed state, preventive maintenance can be carried out. Both maintenance types restore the machine

re-

to the as-good-as-new state.

There is one repairman available for inspecting and maintaining the machines. This repairman traverses the machines in a cyclic order, i.e., he travels from machine to machine in a fixed sequence and returns to the first machine after visiting the last machine. When the repairman arrives at a machine, he will inspect it to learn the deterioration state of that machine. Time is scaled such that traveling to

urn al P

a machine and inspecting it requires one time period. After an inspection, the repairman can choose to either perform maintenance or to move on to the next machine. If the machine is not in the failed state, the repairman can perform preventive maintenance. Performing preventive maintenance will take Tpm time periods. If the machine is in the failed state, corrective maintenance can be performed. This will take Tcm time periods. Because we consider a discrete time setting, both Tpm and Tcm are assumed to be positive integers. A machine will not be operational during maintenance. After finishing a maintenance action, the repairman will move on to the next machine. We consider the problem from a cost perspective and we aim to minimize the long-run cost rate. Performing preventive maintenance will cost cpm and performing corrective maintenance will cost ccm . Furthermore, a downtime cost cd is incurred per machine for every time period that it is in the failed state. Downtime costs also apply to downtime due to maintenance. 4. Markov decision process formulation

We formulate the problem that we consider as a Markov decision process (MDP). This framework is applicable because our problem is about sequential decision making with outcomes that are partially

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

uncertain. Because of the complex structure of the problem, this methodology is more appropriate than using renewal theory. It is difficult to identify suitable renewal points and to calculate the corresponding mean cost and mean duration per renewal cycle. Furthermore, also because of the complex structure of the problem, it is difficult to obtain structural properties of the optimal policy. Section 5 is therefore devoted to a numerical analysis. After formulating the MDP, we provide an aperiodicity transformation to make sure that the value iteration algorithm terminates if there are periodic optimal policies. Finally, we will point out how we evaluate control-limit policies.

4

Journal Pre-proof 4.1. MDP formulation Although we do not observe the deterioration level of each machine at each decision epoch, we model the problem as a regular MDP and not as a partially observable MDP. In our approach, we let a state be described by the last observed deterioration levels for all of the machines, potential maintenance activities at the last visit to each machine, and the type and remaining time of a possible ongoing maintenance action. A machine deteriorates at each time step and can fail in between visits to that machine. When moving to the next machine, the state of the MDP can be used to derive how long it has been since the repairman visited that machine. Combined with the previous deterioration state of that machine,

of

we can calculate the probabilities of finding that machine in a certain state. Furthermore, when we find that the machine is in the failed state, we can calculate its expected downtime since its last inspection. An MDP consists of a set of decision epochs, a set of states, state-dependent action sets, and stateand action-dependent reward or cost functions and transition probabilities. In the next sections, we

pro

specify these sets and functions. 4.1.1. Decision epochs

The decision epochs correspond to the start of the time periods and either coincide with the completion of an inspection or lie within the repair process of a machine. We denote the set of decision epochs by horizon, i.e. N = ∞. 4.1.2. States

re-

T = {1, 2, . . . , N }. Since our aim is to minimize the long-run cost rate, we consider an infinite time

The entire state of the system can be described by the deterioration states of the machines at their last visits, whether preventive or corrective maintenance has been performed at these last visits, and the

urn al P

number of time periods that are remaining until a potential current repair is finished. We note that this information is sufficient to calculate the time since the last visit to the machine that will be visited next. This observation will be used later on, especially when determining the state- and action-dependent costs.

For each machine i we let the variable Xi either take value PM or CM if, respectively, preventive or corrective maintenance has been performed at the last visit to machine i, or we set it equal to the deterioration level observed at the last visit to that machine if it has not been maintained. Although we leave a machine in the as-good-as-new state if maintenance is carried out, the values PM and CM are used so that we can determine how many repairs are performed between two consecutive visits to a machine, and based on that, how much time has past between these visits. We let X1 correspond to the machine that has just been inspected and for which an action must be chosen. The variable X2 corresponds to the next machine that will be visited, X3 to the machine thereafter, and so on. Thus, Xn corresponds to the previously visited machine. We let X = (X1 , X2 , . . . , Xn ). Furthermore, if a repair is performed, we let C ∈ {0, 1, . . . , max(Tpm , Tcm )} denote the number of

time periods that are left until the repairman inspects the next machine. E.g., if the repairman finds a

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

broken machine (i.e., if X1 = m + 1) and chooses to repair the machine, X1 will be set to CM at the next decision epoch to indicate that corrective maintenance is being performed, and C will be set to Tcm . At the subsequent decision epochs, C will be decreased by 1 each time until it equals 1. Then, repairs are finished and the repairman will move to the next machine. At the next decision epoch, C will be set to 0 and the repairman will have finished inspecting the next machine. If C equals 0, there was no ongoing maintenance in the previous time period. The counter C can only have a strictly positive value if X1 equals either PM or CM. We let the entire state be denoted by S = (X, C). The variable C can take max(Tpm , Tcm ) + 1 different values. Furthermore, the number of states per machine equals m + 3, so we have that the state

5

Journal Pre-proof

S = {S = (X, C) : X = (X1 , . . . , Xn ), X1 , . . . , Xn ∈ {1, . . . , m, m + 1, PM, CM}, C ∈ {0, 1, . . . , max(Tpm , Tcm )}}. 4.1.3. Actions The action space can be described by two possible actions. Firstly, the repairman can choose to do nothing (DN) and to move on to the next machine. Secondly, he can perform maintenance (MA). We do not distinguish in the actions between preventive and corrective maintenance, as the state of the machine

of

will specify the maintenance type. The repairman is not allowed to interrupt the repair process. This is a reasonable assumption because no new information about the state of the other machines is obtained action space AS as a function of the state S ∈ S equals  {MA}, if C > 0, AS = {DN, MA}, otherwise.

4.1.4. Costs

pro

during maintenance, implying that it cannot be optimal to interrupt a maintenance action. Thus, the

We let c(S, A) denote the cost that is incurred if action A is chosen in state S. We distinguish between

re-

two different costs. Firstly, we consider costs of performing maintenance. If a machine is operational, i.e., if X1 ∈ {1, . . . , m}, and action MA is chosen, preventive maintenance will be performed which has

a cost of cpm . If the machine is in the failed state, i.e., if X1 = m + 1, the repairman can perform corrective maintenance at a cost of ccm . Secondly, downtime costs are incurred for machines that are being maintained and for machines in

urn al P

the failed state. During maintenance, a machine will be down. If preventive maintenance is performed, this will take Tpm time periods and this will incur a total cost of cd · Tpm . For corrective maintenance, this cost will be cd · Tcm . We choose to incur these costs when we start a maintenance action. Furthermore, a machine can deteriorate and break down whilst the repairman is traveling along other machines and working on them. Because the deterioration level of a machine will only be observed upon inspection, the exact time at which a machine breaks down is not observed. However, because we are minimizing the long-run cost rate, it is sufficient to calculate the expected downtime of a machine since its last visit and, thereby, the expected downtime cost.

When we move to a new machine, and thereby from a state S to a state S 0 , the expected downtime of the machine we move to depends on its deterioration level X2 at the previous visit to this machine, the time since this last visit, which can be calculated based on S, and the deterioration level X10 that we observe by inspecting this machine (basically only whether it is in the failed state or not). Thus, the expected downtime cost of the machine we move to depends both on the current state S before moving to that machine and on the state S 0 after inspecting the machine. However, according to Puterman (1994, Equation 2.1.1), we may take the expectation over all states S 0 in order to calculate the expected

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

space S consists of (m + 3)n · (max(Tpm , Tcm ) + 1) states and equals

downtime cost for the machine we move to. This implies that we can state the expected downtime cost of the next machine only in terms of the current state S and action A. This is valid because our optimality criterion is the long-run cost rate. The expected downtime of the next machine depends on the time since the last visit to this machine and on the deterioration level at that time. Before moving to the next machine, we let t(S) denote the number of time periods since the last visit to this machine. This number of periods is always at least n since we need at least one time period per machine. The machine we move to has index i = 2 in the current state. For the other machines we add duration Tpm if preventive maintenance has been carried

6

Journal Pre-proof out at the last visit and we add Tcm if corrective maintenance has been carried out. Thus, ! ! n n X X t(S) = n + Tpm · 1X1 =PM + 1Xi =PM + Tcm · 1X1 =CM + 1Xi =CM . i=3

(1)

i=3

We let cf (S) denote the expected downtime cost for the next machine since its last visit. We let j ∈ {0, 1, . . . , t(S) − 1} denote the number of periods after its previous visit that this machine has failed. As such, j = 0 means that the machine was left in the failed state after its last inspection, and any j > 0 means that the machine degraded to the failed state j time periods after its last inspection.

of

We let c(j) denote the downtime cost that is incurred if the next machine broke down j time periods after its last inspection, which equals c(j) = cd · (t(S) − j). Furthermore, we let p(j | S) denote the

probability that the next machine has failed j periods after its previous inspection, given that the current

pro

state is S. It follows that if we move to the next machine, the expected downtime cost cf (S) of this machine since its last visit equals t(S)

cf (S) =

X j=0

c(j) · p(j | S).

(2)

We let pi (D) denote the probability that a machine is in the failed state i periods after its last inspection,

re-

given that its deterioration level was D at that inspection. We have pi (D) = (P i )[D, m + 1].

In order to translate the state Xi of machine i, which can also take values PM and CM, to the deterioration

i

urn al P

level of that machine, we define Xi∗ as the deterioration level corresponding to Xi :  1, if Xi ∈ {PM, CM}, Xi∗ = X , otherwise.

We now have that the probability p(j | S) that the next machine broke down exactly j periods since

its last inspection is equal to the probability pj (X2∗ ) that it was broken down j periods since its last inspection, minus the probability pj−1 (X2∗ ) that it was already broken down j − 1 periods since its last inspection. Hence, we have that  p (X ∗ ), if j = 0, 0 2 p(j | S) = p (X ∗ ) − p (X ∗ ), if j > 0. j j−1 2 2

Consequently, we have that the expected downtime cost of the next machine (2) can now be written as t(S)−1

cf (S) =

X j=0

c(j) · p(j | S)

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

= cd · t(S) · p0 (X2∗ ) + 

t(S)−1

X j=1

= cd · t(S) · p0 (X2∗ ) + t(S)−1

= cd ·

X

cd · (t(S) − j) · (pj (X2∗ ) − pj−1 (X2∗ )) 

t(S)−1

X j=1

(t(S) − j) · pj (X2∗ ) − (t(S) − j) · pj−1 (X2∗ )

pj (X2∗ ).

j=0

This cost will be incurred when the repairman chooses to move to the next machine. To summarize, the 7

Journal Pre-proof state and action depenedent immediate cost c(S, A) equals  c (S), if A =DN, f c(S, A) = 1 X1 ∈{1,...,m} · (cpm + cd · Tpm ) + 1X1 =m+1 · (ccm + cd · Tcm ) + 1C=1 · cf (S), if A =MA.

Thus, if action DN is chosen, only the expected downtime cost of the next machine is incurred. When performing maintenance we incur multiple costs. If the repairman starts repairing a machine, repair costs and costs for downtime during repairs are incurred. Since X1 will be set to PM or CM when the repairman starts a maintenance action, we know that he starts repairing when action MA is chosen

of

and X1 ∈ {1, . . . , m + 1}. When, on the other hand, action MA is chosen and X1 ∈ {P M, CM }, the

repairman continues an ongoing repair and no maintenance costs are incurred. Lastly, when a repair is finished, we incur the expected downtime cost of the next machine the repairman moves to. This occurs

pro

when C = 1. 4.1.5. Transition probabilities

We let S 0 = (X 0 , C 0 ) = ((X10 , X20 , . . . , Xn0 ), C 0 ) denote the state at the next decision epoch, given that the current state is S. We distinguish between transitions in case that the repairman moves to the next machine and transitions in case that the repairman stays at the same machine. A repairman stays at the same machine if the action MA is chosen and if C = 0 or C ≥ 2. If C = 0 and MA is chosen,

re-

the repair process will start and X1 will either be set to PM or CM. Furthermore, C will be set to the

corresponding repair time. If C ≥ 2, a repair has already been started and C will be decreased by 1 at

urn al P

the next decision epoch. This is summarized as follows    ((PM, X2 , . . . , Xn ), Tpm ) , if A = MA, C = 0 and X1 ∈ {1, . . . , m},   0 0 (X , C ) = ((CM, X2 , . . . , Xn ), Tcm ) , if A = MA, C = 0 and X1 = m + 1,    ((X , X , . . . , X ), C − 1) , if A = MA and C ≥ 2. 1

2

n

In all other cases, the repairman will move to the next machine. In those cases we always have that C 0 = 0. Because new information will only be obtained for the next machine, the deterioration levels of the other machines will just shift one position in the state description. Thus, we have (X 0 , C 0 ) = ((X10 , X3 , X4 , . . . , Xn−1 , Xn , X1 ), 0) .

Only the deterioration level X10 of the machine that we move to and inspect is stochastic. The corresponding probabilities depend on the time t(S) since the last visit to that machine as given by (1), and the deterioration level X2∗ in which we left the machine after that visit. We have that  (P t(S) )[X ∗ , X 0 ], if X 0 ∈ {1, . . . , m, m + 1}, 2 1 1 p(X10 | S) = 0, if X 0 ∈ {PM, CM}. 1

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

A complete expression for the transition probability p(S 0 | S, A) of moving from state S to state S 0 when action A is chosen is given by    1(X 0 ,C 0 )=((PM,X2 ,...,Xn ),Tpm ) ,     1(X 0 ,C 0 )=((CM,X ,...,X ),T ) , 2 n cm p(S 0 | S, A) =   1(X 0 ,C 0 )=((X1 ,X2 ,...,Xn ),C−1) ,     p(X 0 | X) · 1 0 0 1 (X ,C )=((X10 ,X3 ,...,Xn ,X1 ),0) ,

8

if A = MA, C = 0 and X1 ∈ {1, . . . , m}, if A = MA, C = 0 and X1 = m + 1, if A = MA and C ≥ 2, otherwise.

Journal Pre-proof 4.2. Value iteration and periodicity We use the value iteration algorithm to determine optimal policies for our Markov decision process, and we refer to e.g. Puterman (1994, Chapter 8) for a description of this algorithm. However, as described by Puterman (1994, Section 8.5.4), the stopping criterion of the value iteration algorithm may not be reached for any finite number of iterations if there are periodic optimal policies. In our setting, this can happen if there are many machines and if the repair times are relatively long. For such cases, the cycle time is very long, implying that it is very likely that each machine requires corrective maintenance at each visit. Another situation in which we can encounter periodic optimal policies is when evaluating

of

the control-limit policy for a very low preventive maintenance threshold and when machines deteriorate slowly. This will result in (almost) no failures and preventive maintenance for each machine at each visit. In order to ensure that the value iteration algorithm terminates for all problem instances, we apply τ ∈ (0, 1) and define c˜(S, A) = τ c(S, A),

for all A ∈ A and S ∈ S,

and

for all A ∈ A and S, S 0 ∈ S.

re-

p˜(S 0 | S, A) = (1 − τ )1S 0 =S + τ p(S 0 | S, A),

pro

the aperiodicity transformation described by Puterman (1994, Section 8.5.4). We choose an arbitrary

We now replace c(S, A) and p(S 0 | S, A) respectively by c˜(S, A) and p˜(S 0 | S, A) in our Markov decision

process. This transformed MDP has the same optimal policies as the original MDP, but guarantees convergence of the value iteration algorithm. We let g˜∗ be the approximated long-run cost rate of the translated MDP. The approximated long-run cost rate of the original MDP is given by g ∗ = τ1 g˜∗ .

urn al P

4.2.1. Control-limit policy

In addition to the optimal policy we will also consider a so-called control-limit policy (see e.g. Jiang, 2010). This policy is defined by a single preventive maintenance threshold M ∈ {1, . . . , m, m + 1}.

Maintenance is carried out when an inspection reveals that the deterioration level of a machine exceeds

this threshold level M . Otherwise, the repairman will move on to the next machine. An advantage of this policy is its easier implementation in practice. The performance of the control-limit policy for a given value of M can be evaluated by fixing the action for each state in the Markov decision process defined in Section 4.1. In other words, the action space Acl S will consist of a single action for each state:  {MA}, if C > 0 or X ∗ ≥ M , 1 Acl S = {DN}, otherwise.

The value iteration algorithm can be used to determine the cost rate gcl (M ) of the control-limit policy for

a given value of M . The optimal preventive maintenance threshold M ∗ can be determined by evaluating gcl (M ) for all M . Mathematically it should be chosen such that

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

M ∗ ∈ argminM ∈{1,...,m,m+1} gcl (M ). ∗ We let gcl denote the corresponding optimal cost rate of the control-limit policy: ∗ gcl = gcl (M ∗ ).

5. Numerical results In this section, we first analyze a base system with two machines. We determine the performance of the optimal policy and discuss its properties. Then, we discuss a practical application and we consider 9

Journal Pre-proof the effect of changing model parameters. For each of the considered parameters we consider a large number of values. In this way, we obtain a clear insight on the effect of changing these parameters on the performances of the optimal policy, the control-limit policy, and the pure corrective maintenance policy. We have considered other problem instances as well, and these reveal similar insights. We also report on the cost increase of using the control-limit policy instead of the optimal policy. We finish this section with a discussion of a system with four machines. Other studies that use discrete-time Markov chains to model deterioration often consider a small number of deterioration states and choose the transition probability matrices manually in their numerical

of

analysis (e.g. Icten et al., 2013; Kurt and Kharoufeh, 2010; Maillart, 2006). In our numerical analysis we use a discretized gamma process. The gamma process is applicable to model a wide variety of deterioration types, and it is therefore often a suitable choice (Van Noortwijk, 2009). We use a stationary gamma process with shape parameter a and scale parameter b. Failure will occur if deterioration level 1 is

pro

exceeded. In order to discretize the gamma process, we use the approach described by De Jonge (2019). In the discretization we use time steps with length ∆t = 1. The advantage of using an underlying continuoustime continuous-state deterioration process is that we can easily adjust for instance the volatility of the deterioration process and the granularity of the discretization. An example of another study that uses a discretized gamma process to model deterioration is Uit het Broek et al. (2019).

re-

5.1. Two machines

We start with a system with n = 2 machines that both have m = 10 deterioration states before failure. The deterioration process of both machines is modeled by a discretized stationary gamma process with parameter values a = 5 and b = 0.02, resulting in a deterioration process with a relatively

urn al P

low volatility. Preventive maintenance requires Tpm = 2 periods and incurs a direct cost cpm = 1; corrective maintenance requires Tcm = 4 periods and incurs a direct cost ccm = 10. The downtime cost equals cd = 1 per machine per period. Note that the total cost of preventive maintenance is cpm + Tpm · cd = 3, and that of corrective maintenance ccm + Tcm · cd = 14.

Table 1 shows the optimal policy for this system. This policy has the typical structure of the optimal

policy for a system with 2 machines. The first column indicates the deterioration level X1 of the machine that has just been inspected; the top row indicates whether preventive or corrective maintenance has been carried out at the last visit to the other machine, or, otherwise, what the deterioration level of this machine was at this visit. We omit states in which X1 is equal to either PM or CM, since the only allowed action in those states is to continue an ongoing maintenance action. Entries ‘MA’ denote that performing maintenance is optimal in the corresponding state, whereas ‘-’ denotes that maintenance should not be performed.

A first observation is that the states in the gray region of the table will never be reached. A machine will always be repaired if it has a deterioration level of 8, 9, or 10 at an inspection. Thus, we can never have left a machine in one of these deterioration states. Furthermore, if the current machine is broken and the other machine is left in a relatively good state (i.e. X2∗ ≤ 4), it will be optimal to repair the

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

inspected machine. This implies that, if we do not maintain a machine that is in the failed state, the subsequent inspection of the other machine cannot reveal a deterioration level below or equal to 4. If the deterioration level X1 of the inspected machine is at most 4, it is never optimal to carry out

maintenance. If an inspection reveals a medium deterioration level (5, 6, or 7), it depends on the state of the other machine at its last inspection whether performing maintenance is optimal. If the deterioration level of the other machine was quite low, it is not very likely that it needs maintenance at its next visit, meaning that we are back at the current machine quite fast and that maintenance can be postponed. If we have left the other machine in a quite high deterioration state, we also do not carry out maintenance in order to be back at that other machine before failure. For medium deterioration levels of the other 10

Journal Pre-proof machine it is sometimes optimal to maintain the current machine for medium deterioration level. We always carry out maintenance if an inspection reveals a deterioration level of 8, 9, or 10. A failed machine will only be maintained if the risk that the other machine fails in the meantime is not too high. The long-run cost rate of the optimal policy is g ∗ = 0.706. Table 1: Optimal policy for a system with 2 machines.

CM

1

2

3

4

5

6

7

8

9

10

11

MA MA MA MA

MA MA MA MA

MA MA MA MA

MA MA MA MA MA

MA MA MA MA MA MA

MA MA MA MA MA MA MA

MA MA MA MA MA -

MA MA MA MA -

MA MA MA -

MA MA MA -

MA MA MA -

MA MA MA MA MA MA MA MA

MA MA MA MA MA MA MA MA

of

1 2 3 4 5 6 7 8 9 10 11

PM

pro

X1 \X2

The optimal control-limit policy for the current system uses a preventive maintenance threshold

re-

of M ∗ = 7. Thus, preventive maintenance is sometimes carried out somewhat earlier compared to the optimal policy, and sometimes somewhat later. The long-run cost rate of the optimal control-limit policy ∗ equals gcl = 0.745, implying that the costs increase by approximately 5.50 % if we switch from the overall

optimal policy to the optimal control-limit policy.

urn al P

5.1.1. Practical example

The model that we consider can be applied to a wide variety of cases. If each time period in our model represents a day, the above-discussed instance is applicable to two machines that are located at different sites at a considerable distance from each other. Each day the repairman alternately travels to one of the two machines. If he decides to carry out preventive maintenance, the other machine will not be inspected for two additional days. Failure has more severe consequences, reflected in a double amount of time that is needed for corrective maintenance. The deterioration process results in a mean time to failure of approximately ten days, implying that both machines require a relatively high maintenance frequency.

Realistic cost values in the above example are a preventive maintenance cost of e 100, a corrective maintenance cost of e 1,000 (because the ratio between these costs is 1:10), and a downtime cost of e 100 per day (because the downtime cost per day equals the preventive maintenance cost). In this case, the optimal control-limit policy dictates to carry out preventive maintenance if an inspection reveals a deterioration state of at least M ∗ = 7. The corresponding cost is e 74.50 on average per day. This cost can be reduced to e 70.60 per day if the overall optimal policy is used. If the machines are used

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

continuously throughout the year, this implies a yearly cost saving of more than e 1,400. The average daily cost of a pure corrective maintenance policy is e 197.60, implying that a very large cost saving can be obtained by using condition information for maintenance decisions, either by using the simple control-limit policy or the more sophisticated optimal policy. From a practical point of view, it is also of interest to analyze the advantage of using a single repairman for both machines instead of a separate repairman for each machine. If each of the machines is serviced by a separate repairman who inspects his machine on a daily basis, it is optimal to carry out preventive maintenance if deterioration level 9 is reached or exceeded. Thus, the more frequent inspections lead to less frequent maintenance interventions. The corresponding cost per day is e 30.30 11

Journal Pre-proof per machine, and thus e 60.60 for the two machines together. The minimum daily cost of e 70.60 for using only a single repairman is thus only e 10 higher, and this easily outweighs the additional labor cost of an additional repairman. In other words, both downtime and maintenance costs only increase to a limited extent if a single repairman is shared. 5.1.2. Number of deterioration states We will continue to consider the effect of the number of deterioration states, where we note that a higher number of deterioration states results in a better approximation of the continuous-state gamma

of

deterioration process. Figure 1 shows the long-run cost rate of the policy with only corrective maintenance (CM), of the optimal control-limit policy (C-L), and of the overall optimal policy (OPT). The corrective maintenance policy does not use any condition information and its cost rate is therefore stable for a varying number of deterioration states. Already when a small number of deterioration states are used

pro

the control-limit policy and the optimal policy perform much better than the corrective maintenance policy.

Figure 2 shows the percentage increase in cost if we switch from the optimal policy to the controllimit policy. It turns out that a higher number of deterioration states, i.e. more detailed condition information, is needed for the optimal policy to clearly outperform the control-limit policy. This seems logical as Table 1 shows that the preventive maintenance thresholds of the optimal policy only slightly

re-

differs from the single threshold of the control-limit policy. Increasing m above 10 only results in minor additional benefits for both the control-limit policy and the optimal policy. Thus, in practice, a certain

1.5 0.0

0.5

1.0

Cost rate

2.0

urn al P

2.5

3.0

degree of accuracy in the condition information is required, but it does not need to be extremely precise.

CM C−L OPT

5

10

15

20

Number of deterioration states m

Figure 1: Cost rate for various policies as a function of the number of deterioration states m.

5.1.3. Gamma process parameters

The parameters a and b of the stationary gamma process influence the behavior of this process. The expected amount of additional deterioration per time period equals a · b, and its variance equals a · b2

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

(see De Jonge et al., 2017). Thus, if we decrease a and increase b while keeping a · b constant, the mean deterioration increment remains the same, but the volatility of the deterioration process increases. Figure 3 shows what happens if a changes while keeping a · b fixed at 0.1. Although the mean

deterioration increment is kept constant, the mean time to failure is not because of the overshoot behavior of the gamma process (see De Jonge et al., 2017). The mean time to failure is lower for less volatile deterioration processes, implying that the cost of the policy with only corrective maintenance (CM) is increasing in a. The control-limit policy (C-L) and the optimal policy (OPT) clearly benefit from a more stable deterioration process (higher a). This increases the value of condition information and makes sudden large deterioration jumps and failures less likely. 12

8 6 4

Cost increase (%)

2

10

15

Number of deterioration states m

of

0

5

20

pro

Figure 2: Percentage cost increase by switching from the optimal policy to the control-limit policy as a function of the shape parameter a of the gamma deterioration process.

The percentage increase in costs if we switch from the optimal policy to the control-limit policy is shown in Figure 4. As opposed to the control-limit policy, the optimal policy has the capability of fine tuning the preventive maintenance threshold based on the entire system state, but this turns out not to result in any benefits if the deterioration process is very volatile (low a). For smaller levels of volatility

1.5 0.5

1.0

Cost rate

2.0

urn al P

2.5

3.0

re-

the optimal policy clearly outperforms the control-limit policy, but both policies converge to each other if the deterioration process becomes more stable (high a).

0.0

CM C−L OPT

0.1

0.5

1.0

5.0

10.0

50.0 100.0

Shape parameter a (log−scaled)

Figure 3: Cost rate for various policies as a function of the shape parameter a of the gamma deterioration process.

5.1.4. Corrective maintenance cost

Figure 5 shows the effect of the corrective maintenance cost ccm on the performance of the various policies. The cost rates of all policies are obviously increasing in ccm , and the policy with only corrective

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

10

Journal Pre-proof

maintenance (CM) results in the fastest increase. Figure 6 shows the percentage increase in costs if we switch from the optimal policy to the control-limit policy. For low corrective maintenance costs both policies are reluctant to perform preventive maintenance, whereas for high corrective maintenance costs both policies carry out preventive maintenance very early. In both cases, the performance of the two policies is similar. Only for medium corrective maintenance costs the optimal policy clearly outperforms the control-limit policy. We note that the graph in Figure 6 is not smooth because of the discretized deterioration process. Sometimes a preventive maintenance threshold of a policy suddenly changes when ccm changes.

13

8 6 4

Cost increase (%)

2

0.5

1.0

5.0

of

0 0.1

10.0

50.0 100.0

Shape parameter a (log−scaled)

pro

3.0

Figure 4: Percentage cost increase by switching from the optimal policy to the control-limit policy as a function of the shape parameter a of the gamma deterioration process.

1

5

re-

1.5 0.0

0.5

1.0

Cost rate

2.0

2.5

CM C−L OPT

10

50

100

500

1000

urn al P

Corrective maintenance cost ccm (log−scaled)

6 4 0

2

Cost increase (%)

8

10

Figure 5: Cost rate for various policies as a function of the corrective maintenance cost ccm .

1

5

10

50

100

500

1000

Corrective maintenance cost ccm (log−scaled)

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

10

Journal Pre-proof

Figure 6: Percentage cost increase by switching from the optimal policy to the control-limit policy as a function of the corrective maintenance cost ccm .

5.1.5. Other parameters

When varying the other system parameters it turns out that increasing the preventive maintenance time Tpm results in an increase of the cost rate of both the control-limit policy and the optimal policy. The optimal policy clearly outperforms the control-limit policy if Tpm is low. If the time to perform corrective maintenance Tcm increases, the long-run cost rate of all policies increases. The optimal policy outperforms the control-limit policy for all reasonable values of Tcm . If the cost of preventive maintenance 14

Journal Pre-proof cpm increases, the long-run cost rates of both the control-limit policy and the optimal policy increase. Furthermore, the difference between these policies is larger if cpm is lower, as long as it is not extremely low. Lastly, when varying the downtime cost cd , both the total cost of preventive maintenance and of corrective maintenance are influenced. An increase of cd leads to an increase in the cost rates of all policies and a decrease in the benefit of the optimal policy as opposed to the control-limit policy. An overall observation is that the optimal policy performs significantly better than control-limit policy if the cost of preventive maintenance is relatively low, but not extremely low.

of

5.2. Four machines We continue with a system with n = 4 machines. We set all system parameters equal to those in Section 5.1, but we change the parameters of the gamma deterioration process to a = 2 and b = 0.025. This results in a higher mean time to failure of the machines.

pro

For this system with four machines the percentage cost increase of switching from the optimal policy to the control-limit policy is only 0.76 %. Also when we change the parameter values, the cost saving generally does not exceed 1.5 %. The only exception is the number of deterioration states m. Figure 7 shows the cost rate of the different policies as a function of m. The patterns are similar to those in Figure 1. Figure 8 shows the percentage cost increase when the control-limit policy is used instead of the optimal policy. As compared to the case with two machines, much more deterioration states, i.e., more

re-

detailed condition information, is needed for the optimal policy to clearly outperform the control-limit

1.5 0.5

1.0

Cost rate

2.0

urn al P

2.5

3.0

policy.

0.0

CM C−L OPT

0

5

10

15

20

Number of deterioration states m

Figure 7: Cost rate for various policies as a function of the number of deterioration states m (4 machines).

As a last result we will again change the behavior of the gamma deterioration process while keeping the expected deterioration increments fixed. Figure 9 shows the cost rate of the various policies as a function of the shape parameter a, and Figure 10 shows the cost increase of switching from the optimal policy to the control-limit policy. We note here that we have set the number of deterioration states to

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

m = 20 for these figures.

Again, it turns out that quite a stable deterioration process is required for the optimal policy to clearly outperform the control-limit policy. As opposed to the system with two machines, we do not observe a decrease in the benefit if the deterioration process becomes too stable. Apparently, the three other machines together result in enough variation for the optimal policy to outperform the control-limit policy. As we did for the system with two machines, we also varied the other parameters. The main insights are similar to those from the system with two machines. A general observation is that the additional benefit of the optimal policy as opposed to the control-limit policy becomes smaller for a higher number

15

8 6 4

Cost increase (%)

2

5

10

15

Number of deterioration states m

of

0 0

20

0.0

CM C−L OPT 0.1

0.5

re-

1.5 0.5

1.0

Cost rate

2.0

2.5

3.0

pro

Figure 8: Percentage cost increase by switching from the optimal policy to the control-limit policy as a function of the number of deterioration states m (4 machines).

1.0

5.0

10.0

50.0 100.0

urn al P

Shape parameter a (log−scaled)

6 4 0

2

Cost increase (%)

8

10

Figure 9: Cost rate for various policies as a function of the shape parameter a of the gamma deterioration process (4 machines).

0.1

0.5

1.0

5.0

10.0

50.0 100.0

Shape parameter a (log−scaled)

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

10

Journal Pre-proof

Figure 10: Percentage cost increase by switching from the optimal policy to the control-limit policy as a function of the shape parameter a of the gamma deterioration process (4 machines).

of machines. The optimal policy can use the last observed deterioration information of all other machines in order to determine whether to carry out maintenance, whereas the control-limit policy only uses the deterioration level of the current machine. If there are more machines, the combined information of all other machines becomes more stable, implying that the possibility to use this information has less value in this case.

16

Journal Pre-proof 6. Conclusion We have considered the cyclic patrolling repairman problem with condition-based preventive maintenance. In this problem, a set of machines are traversed by a single repairman in a fixed sequence. Upon arrival at a machine an inspection will be carried out to obtain the condition of this machine. Based on this condition, and the conditions of the other machines at their last visits and the time since the last visit to the current machine, the repairman determines whether maintenance will be carried out, or whether he will move on to the next machine. In order to determine optimal policies we have formulated the problem as a Markov decision process.

of

This formulation can also be used to analyze the performance of a control-limit policy, which is a commonly used heuristic. Under the control-limit policy, maintenance is carried out if, at an inspection, the deterioration level of a machine exceeds a certain threshold, independent of the available information

pro

about the other machines.

In a numerical study we have compared the performances of the optimal policy and the control-limit policy to that of a pure corrective maintenance policy. Already if only a few condition states are distinguished for each machine, both the optimal policy and the control-limit policy clearly outperform the pure corrective policy. More detailed condition information is required for the optimal policy to perform significantly better than the control-limit policy. Under the optimal policy, preventive maintenance will

re-

be carried out at a higher deterioration level if the repairman expects to be back at the current machine relatively fast, or if he believes that another machine is in urgent need for preventive maintenance. If the number of machines increases, the additional benefit of the optimal policy as opposed to the controllimit policy decreases. Furthermore, even more detailed condition information is needed to attain this potential benefit.

urn al P

There exist various opportunities for future research in this area. We have considered a set of identical machines, but real-life assets are also likely to consist of different types of machines (e.g. Faccio et al., 2014). In such a setting it may also be beneficial to inspect more critical machines or machines that deteriorate faster more frequently. Furthermore, we have assumed that all machines need inspections to obtain condition information, but part of the machines may also be equipped with remote monitoring that results in continuous condition information. It is interesting to study how this should be incorporated in a maintenance policy. Finally, we have assumed that failures can also only be observed by inspections (silent failures). One could also assume that failures are self-announcing and that the repairman could interrupt his fixed visiting sequence when a failure occurs. References

Ahmad, R., Kamaruddin, S., 2012. An overview of time-based and condition-based maintenance in industrial applications. Computers & Industrial Engineering 63 (1), 135–149. Barlow, R., Proschan, F., 1965. Mathematical Theory of Reliability. Wiley, New York, NY, USA.

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Bertsimas, D. J., Van Ryzin, G., 1991. A stochastic and dynamic vehicle routing problem in the euclidean plane. Operations Research 39 (4), 601–615. Boon, M. A. A., Van Der Mei, R. D., Winands, E. M. M., 2011. Applications of polling systems. Surveys on Operations Research and Management Science 16 (2), 67–82. Bouslah, B., Gharbi, A., Pellerin, R., 2018. Joint production, quality and maintenance control of a twomachine line subject to operation-dependent and quality-dependent failures. International Journal of Production Economics 195, 210–226.

17

Journal Pre-proof Bunday, B. D., El-Badri, W. K., 1984. A model for a textile winding process. European Journal of Operational Research 15 (1), 55–62. Bunday, B. D., El-Badri, W. K., Supanekar, S. D., 1985. The efficiency of bi-directionally patrolled machines when repairs are not always successful. European Journal of Operational Research 19 (3), 324–330. Bunday, B. D., Mack, C., 1973. Efficiency of bi-directionally traversed machines. Journal of the Royal Statistical Society C 22 (1), 74–81.

of

Camci, F., 2014. The travelling maintainer problem: integration of condition-based maintenance with the travelling salesman problem. Journal of the Operational Research Society 65 (9), 1423–1436. Camci, F., 2015. Maintenance scheduling of geographically distributed assets with prognostics informa-

pro

tion. European Journal of Operational Research 245 (2), 506–516.

Choi, T.-M., Wallace, S. W., Wang, C., 2018. Big data analytics in operations management. Production and Operations Management 27 (10), 1868–1883.

Das, T., Wortman, M. A., 1993. Analysis of asymmetric patrolling repairman systems. European Journal

re-

of Operational Research 64 (1), 45–60.

De Jonge, B., 2019. Discretizing continuous-time continuous-state deterioration processes, with an application to condition-based maintenance optimization. Reliability Engineering and System Safety 188, 1–5.

De Jonge, B., Jakobsons, E., 2018. Optimizing block-based maintenance under random machine usage.

urn al P

European Journal of Operational Research 265 (2), 703–709.

De Jonge, B., Klingenberg, W., Teunter, R. H., Tinga, T., 2016. Reducing costs by clustering maintenance activities for multiple critical units. Reliability Engineering and System Safety 145, 93–103. De Jonge, B., Scarf, P. A., 2020. A review on maintenance optimization. European Journal of Operational Research (forthcoming).

De Jonge, B., Teunter, R. H., Tinga, T., 2017. The influence of practical factors on the benefits of condition-based maintenance over time-based maintenance. Reliability Engineering and System Safety 158, 21–30.

De Smidt-Destombes, K. S., Van Der Heijden, M. C., Van Harten, A., 2004. On the availability of a k-out-of-N system given limited spares and repair capacity under a condition based maintenance strategy. Reliability Engineering and System Safety 83 (3), 287–300. Dekker, R., 1996. Applications of maintenance optimization models: a review and analysis. Reliability

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Engineering and System Safety 51 (3), 229–240. Diallo, C., Venkatadri, U., Khatab, A., Liu, Z., 2018. Optimal selective maintenance decisions for large serial k-out-of-n: G systems under imperfect maintenance. Reliability Engineering and System Safety 175, 234–245.

Do, P., Canh Vu, H., Barros, A., B´erenguer, C., 2015. Maintenance grouping for multi-component systems with availability constraints and limited maintenance teams. Reliability Engineering and System Safety 142, 56–67.

18

Journal Pre-proof Faccio, M., Persona, A., Sgarbossa, F., Zanin, G., 2014. Industrial maintenance policy development: A quantitative framework. International Journal of Production Economics 147 (A), 85–93. Haque, L., Armstrong, M. J., 2007. A survey of the machine interference problem. European Journal of Operational Research 179 (2), 469–482. Icten, Z. G., Shechter, S. M., Maillart, L. M., Nagarajan, M., 2013. Optimal management of a limited number of replacements under markovian deterioration. IIE Transactions 45 (2), 206–214. Jamil, M., Batta, R., Malon, D. M., 1994. The traveling repairperson home base location problem.

of

Transportation Science 28 (2), 150–161.

Jardine, A. K. S., Lin, D., Banjevic, D., 2006. A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing 20 (7), 1483–

pro

1510.

Jiang, R., 2010. Optimization of alarm threshold and sequential inspection scheme. Reliability Engineering and System Safety 95 (3), 208–215.

Kurt, M., Kharoufeh, J. P., 2010. Monotone optimal replacement policies for a markovian deteriorating

re-

system in a controllable environment. Operations Research Letters 38 (4), 273–279. L´opez-Santana, E., Akhavan-Tabatabaei, R., Dieulle, L., Labadie, N., Medaglia, A. L., 2016. On the combined maintenance and routing optimization problem. Reliability Engineering and System Safety 145, 199–214.

173–178.

urn al P

Mack, C., 1957. The efficiency of n machines uni-directionally patrolled by one operative when walking time is constant and repair times are variable. Journal of the Royal Statistical Society B 19 (1),

Mack, C., Murphy, T., Webb, N. L., 1957. The efficiency of n machines uni-directionally patrolled by one operative when walking time and repair times are constants. Journal of the Royal Statistical Society B 19 (1), 166–172.

Maillart, L. M., 2006. Maintenance policies for systems with condition monitoring and obvious failures. IIE Transactions 38 (6), 463–475.

Mittler, M., Kern, C., 1997. Discrete-time approximation of the machine interference problem with generally distributed failure, repair, and walking times. European Journal of Control 3 (4), 254–267. Nguyen, H. S. H., Do, P., Vu, H.-C., Iung, B., 2019. Dynamic maintenance grouping and routing for geographically dispersed production systems. Reliability Engineering and System Safety 185, 392–404. Olde Keizer, M. C. A., Teunter, R. H., Veldman, J., Zied Babai, M., 2018. Condition-based mainte-

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

nance for systems with economic dependence and load sharing. International Journal of Production Economics 195, 319–327.

Puterman, M., 1994. Markov Decision Processes. Wiley, New York, NY, USA. Rashidnejad, M., Ebrahimnejad, S., Safaei, N., 2018. A bi-objective model of preventive maintenance planning in distributed systems considering vehicle routing problem. Computers & Industrial Engineering 120, 360–381. Shafiee, M., 2015. Maintenance logistics organization for offshore wind energy: Current progress and future perspectives. Renewable Energy 77, 182–193. 19

Journal Pre-proof Stecke, K. E., Aronson, J. E., 1985. Review of operator/machine interference models. International Journal of Production Research 23 (1), 129–151. Takagi, H., 1988. Queueing analysis of polling models. ACM Computing Surveys (CSUR) 20 (1), 5–28. Takagi, H., 1991. Application of polling models to computer networks. Computer Networks and ISDN systems 22 (3), 193–211. Takagi, H., 2000. Performance Evaluation: Origins and Directions. Springer Berlin Heidelberg, Berlin,

of

Heidelberg, Ch. Analysis and Application of Polling Models, pp. 423–442. Uit het Broek, M. A. J., Teunter, R. H., De Jonge, B., Veldman, J., Van Foreest, N. D., 2019. Conditionbased production planning: Adjusting production rates to balance output and failure risk. Manufac-

pro

turing & Service Operations Management (forthcoming).

Van Noortwijk, J. M., 2009. A survey of the application of gamma processes in maintenance. Reliability

urn al P

re-

Engineering and System Safety 94 (1), 2–21.

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

20