Priority-based assignment and routing of a fleet of unmanned combat aerial vehicles

Computers & Operations Research 35 (2008) 1813 – 1828 www.elsevier.com/locate/cor

Priority-based assignment and routing of a fleet of unmanned combat aerial vehicles Vijay K. Shetty, Moises Sudit∗ , Rakesh Nagi∗ Department of Industrial and Systems Engineering and Center for Multisource Information Fusion, 438 Bell Hall, University at Buffalo (SUNY), Buffalo, NY 14260, USA Available online 1 November 2006

Abstract This paper considers the strategic routing of a fleet of unmanned combat aerial vehicles (UCAVs) to service a set of predetermined targets from a prior surveillance mission. Targets are characterized by their priority or importance level, and minimum and maximum service levels that, respectively, represent the lower bound of munitions for destruction and upper bound of munitions to limit collateral damage. Additional constraints to be respected are the payload capacities of the (possibly heterogeneous) UCAV fleet and the range based on fuel capacity and payload transported. The vital aspect of this paper is the integrated optimal utilization of available resources—weaponry and flight time—while allocating targets to UCAVs and sequencing them to maximize service to targets based on their criticality. The complexity of the problem is addressed through a decomposition scheme with two problems: a target assignment problem (modeled as a minimum cost network flow problem) and a vehicle routing problem, which in turn splits into multiple decision traveling salesman problems, one for each UAV. A Tabu search heuristic is developed to coordinate the two problems. Using test problems we establish the applicability of this approach to solve practical-sized problems. 䉷 2006 Elsevier Ltd. All rights reserved. Keywords: UCAV fleet routing; Priority-based target service; Tabu search heuristic

1. Introduction Reconnaissance unmanned aerial vehicles (UAVs) like the predator and global hawk have seen an appreciable change from UAVs to UCAVs (unmanned combat aerial vehicles), equipped to carry dumb bombs or semi-precision guided missiles. A UCAV can be defined as a returnable, controllable and responsive cruise missile, while others think of them more as a “fighter aircraft without a pilot” (definition from http://www.edwards.af.mil). UCAVs have a number of advantages over manned aerial vehicles: (1) Piloted aircraft have to use guided smart missiles for survivability while UCAVs can smartly guide small cheap unguided general purpose (dumb) munitions upon reaching a short range; this can ensure similar accuracy and is less expensive. (2) Manned aircraft have to stay in a formation for effective coordination unlike UCAVs, which fly in different directions to reduce predictability of the UCAV tour and create confusion among air-defense systems. (3) The UCAV is an affordable weapon system, which can conduct complex missions with multiple targets in a dynamic environment without loss of human life. ∗ Corresponding authors. Tel.: +1 716 645 2357x2103; fax: +1 716 645 3302.

E-mail addresses: [email protected] (M. Sudit), [email protected] (R. Nagi). 0305-0548/$ - see front matter 䉷 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.cor.2006.09.013

1814

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

When using UAVs for combat purposes, some additional issues need to be considered in their routing (as opposed to typical intelligence, surveillance and reconnaissance missions where UAVs have traditionally been used). UCAVs need to carry payloads of weaponry based on the targets they have to visit. Each target, based on its resistance and impermeability, may have to be serviced with a specific payload. Depending on the weapon payload, and the sequence in which the munitions are dropped, the fuel consumption and therefore the range of the UCAV are influenced. If heavier payloads are dropped earlier and lighter payloads are dropped later, the UCAV can achieve greater range. More specifically, the fuel consumed by a UCAV depends on various factors such as thrust to weight ratio, payload and fuel load, take off gross weight, and empty weight. Refs. [1,2] provide some detailed aerofoil design issues and thrust power as a function of the load factor and speed. While some of the design issues are too detailed for this paper, it suffices to indicate that the UCAV range will depend on the total weapon payload at takeoff and sequence of payloads dropped. This paper is motivated by and addresses major concerns in the mission of the US Air Force. According to the mission statement of Air Force Office of Scientific Research, “the benefits of a smart fleet of UAVs will maximize mission effectiveness and survivability, attain the superior level of functionality and interoperability, enhance lethal targeting capabilities, attain timeliness requirements, support moving target engagement, guarantee precision fire control and targeting, as well as guarantee the optimal overall performance of multi-agent UAVs.” Our work also relates to the new strike capabilities doctrine, and will support the following emerging US Air Force programs: new electronic order of battle and new engagement tactics; multi-mission vehicles; coordination–synchronization of multi-platforms; and multiple targeting and fire control. US Navy and Marines also have similar UAV deployment agendas, and this paper is equally of interest to them as well. This paper studies routing a fleet of UCAVs to maximize munition service for successful elimination or neutralization of a set of predetermined targets. These targets are assumed to have been determined through a prior surveillance mission, which specifies for each target, its location, priority or importance level, and minimum and maximum service levels. The service levels, respectively, represent the lower bound of munitions for destruction and upper bound of munitions to limit collateral damage. More specifically, the lower bound corresponds to the minimal number of “hits” (ammunition level) required to destroy the target, while the upper bound corresponds to the maximal delivery to the target to restrict the collateral damage to neighboring civilian population (e.g., schools and hospitals). In addition to the bounds on the amount of service, each target has a weight or importance value that indicates the criticality of that target. The UCAV fleet can be heterogeneous, i.e., possess different fuel (and associated range) and load capacities. The load capacity is the upper bound on the amount of weaponry (dumb bombs/sophisticated weaponry) a UCAV can carry. These are two interrelated attributes of each UCAV, which need to be considered when choosing the different UCAVs that will be part of a fleet for a specific mission. The routing plan desired by the commander should specify the allocation of targets to UCAVs and the trajectories they must follow such that UCAV range and load capabilities are respected. This problem is quite challenging for the human analyst because it combines target and service level assignment subject to UCAV capability constraints, as well as sequencing the target. The coordinated aspect of the model makes it harder for the human because it permits multiple UCAVs to service the same target to satisfy the requirements of the mission. (This is commonly referred to as split deliveries in the vehicle routing or distribution literature.) To overcome these challenges, the problem is modeled as a mixed integer linear program (MILP) which can be solved using a standard solver for small-dimensioned problems. However, for larger, more realistic dimensioned problems, finding the globally optimal solution is computationally prohibitive under today’s computational resources and stateof-knowledge. The problem is at least as hard as the well-known vehicle routing problem (VRP) which is NP-hard. Our problem has additional constraints accounting for resource capability and fuel capacity. The complexity of our routing problem is increased due to the consideration of delivery of “service” requirements and priorities. Therefore, we need to find an efficient procedure to solve this complex problem. The main contributions of this paper are modeling this important military problem using mathematical programming, and developing a heuristic procedure to solve it efficiently. The heuristic procedure is a decomposition scheme consisting of two problems: a target assignment problem (modeled as a minimum cost network flow problem), and a vehicle routing problem. The former assigns vehicles to targets and allocates them service under vehicle load capacity constraints to maximize priority-weighted service, while the latter solves a set of decision traveling salesman problems, one for each vehicle. Due to the decomposition it may turn out that the fuel capacity on a subset of vehicles is violated in the second subproblem. Hence, a Tabu search heuristic is developed to coordinate the two problems. Using test problems we establish the applicability of this approach to solve practical-sized problems.

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

1815

Fig. 1. An illustrative example of a UCAV mission.

To further illustrate the elements of the problem a simplified example is presented in Fig. 1. There are three heterogeneous UCAVs with different payload capacity and range attributes. We assume that the UCAVs with high weapon carrying capacity can travel less and vice versa. In the displayed solution, targets with smaller weight and lower limits (#2 and #3) are not hit at all, in particular because they were not convenient to include in an existing route. Targets with small weight and larger limits are partially hit (#10), while those with higher weight are hit with maximum available fire power (#8). The constraint in reaching either #2 or #3 or both is the fuel constraint for UCAV 1. The limit of collateral damage is reached for targets #1, 4, 5 and nearly for 8 because they have large weights (10). The remainder of this paper is organized as follows. Section 2 presents a brief literature review of coordination and control of UAVs and common routing problems. Problem description and mathematical formulation are presented in Section 3. Section 4 is devoted to the decomposition and Tabu search solution approach developed. Computational results of the solution method on a variety of numerical examples are presented in Section 5. Finally, the conclusions and recommendations for future work are presented in Section 6. 2. Literature review UAV routing problems can be generally classified among two kinds: (1) path planning for known origin and destination or discrete locations to be visited, and (2) search or surveillance types of routing. Refs. [3–8] are examples of path planning type of routing. Recently, there has been and increased interest in search-based routing problems [9–12]. The following sub-section covers literature on routing a fleet of UAVs. 2.1. Coordination and control of UAVs The optimal fleet coordination problem includes combined resource allocation and trajectory optimization aspects of the fleet. Typically, the problem is addressed in a decoupled fashion, where the first part decides the resource allocation, and given the allocation, the second part determines the possible trajectories. Then a centralized algorithm coordinates the solutions of the aforementioned parts to determine the best joint solution of allocation and routing. These problems

1816

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

are complicated because of multiple vehicles, targets and combined with appropriate resource allocation requirements. Papers [13,14] present an algorithm for optimal fleet coordination problem with team composition and goal assignment, resource allocation and trajectory optimization, where each UAV has capability to service certain waypoints. This can be viewed as a specialized version of the problem described in our work. The paper [15] describes a centralized algorithm to perform a coordinated guidance of multiple aircraft. Most of the work done in vehicle dynamics developed constraints for optimality using non-linear aircraft dynamics while [15] uses a linear approximation of aircraft dynamics to allow the formulation to be an MILP. The receding horizon approach of [16] for trajectory path planning is a relatively new approach. An MILP is developed for path planning towards the goal by logically incorporating constraints to avoid nofly zones and waypoints in the formulation. The receding horizon method solves larger problems, which are intractable by fixed time horizon formulations [16]. The receding horizon control (also known as model predictive control) designs an input trajectory, which optimizes the system model output over the planning horizon. Smaller problems are solved for small time horizons and feedback is incorporated into the system model while re-planning for the whole problem starting from the current state. A series of small trajectories are found towards the goal instead of one long strategy. This saves on computation time rather than solving the problem as a whole. Paper [17] presents a Java-based software for routing unmanned aerial vehicle; it also uses Tabu search as an underlying mechanism. However, the software is not designed to address the UCAV problem with target service levels and priorities. 2.2. Vehicle routing problems and Tabu search heuristics Most vehicle routing problems [18] are NP-hard problems for which no polynomially bounded algorithm has been found. Convergent algorithms can rarely solve large problems consisting of more than 50 customers and often permit relatively few side constraints. Side constraints could be in the form of fuel/range and load/volume capacities. Paper [18] describes dynamic routing of UAVs using Tabu search techniques. The problem is formulated as a VRP problem. There are many variants of the generic VRPs, which are also NP-hard. The optimal solution techniques available for small-dimensioned variants of the VRP, such as branch-and-bound, mathematical programming approaches or dynamic programming are currently not practical for UCAV routing problem. For large-dimensioned variants of the VRP, several heuristic approaches have been used in an attempt to overcome the problems associated with optimal approaches. On the other hand, greedy algorithms, which prove to be very useful in simpler problems, fail to achieve the desired results with respect to solution quality. Sweep algorithm and petal algorithm have solved VRP successfully to an extent [19]. The cluster first, route second algorithm is asymptotically optimal but its empirical performance is not competitive [19]. The survey of meta-heuristics for the VRP shows that the best known methods can find excellent and sometimes optimum solutions to instances with a few hundred customers. Fortunately, Tabu Search (TS) [20,21] provides excellent results for these types of problems. Procedures based on pure genetic algorithms or neural networks are clearly out performed, while those based on simulated annealing and on ant colony systems are not so competitive [22]. The Tabu search heuristic uses adaptive memory structures searching the solution space economically and effectively. Unlike a number of heuristic search methods, the Tabu search does not choose a neighborhood solution randomly. Instead it proceeds with the supposition that “there is no point in accepting a new (poor) solution unless it is to avoid a path already investigated.” This insures that new regions of a problem’s solution space are investigated and local minima are avoided. The Tabu search begins by marching to a local minimum. To avoid revisiting the moves in the early part of the search, the method records recent moves in one or more Tabu lists. As the algorithm proceeds the role of the memory changes. At initialization the goal is to make a coarse examination of the solution space, known as diversification, but as candidate solutions are identified the search is more focused to produce local optimal solutions in a process of intensification. When implementing Tabu search for a specific problem one would need to determine the right size, variability, and adaptability of the Tabu memory. When a move is declared Tabu, typically more than one solution is declared as Tabu. However, some of these solutions might be of excellent quality but have not yet been visited. To overcome this problem, aspiration criteria are introduced which allow overriding the Tabu state of a solution. Common aspiration criteria are used to allow solutions which are better than the current best known solution or better than a certain attractiveness threshold. The Tabu list length determines the length of time a solution stays on the Tabu list. Based on the length of the Tabu list, the behavior of the search can be significantly altered. If the list is shortened, intensification occurs and the local area will be searched more thoroughly as the search gravitates towards the local optimum. If the list is lengthened, diversifica-

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

1817

tion occurs and the search will be forced to leave its current area to explore new areas further away in the solution space [18,20,21]. 2.3. Traveling salesman heuristics As mentioned in Section 2.1, VRP approaches that perform resource allocation first and route generation later are faced with a traveling salesman problem (TSP) during the route generation phase. Our approach presents a decision TSP which requires us to determine a tour for the assigned targets within the (fuel) range constraint. If the minimal tour length exceeds the range then we know that the decision TSP is infeasible. Hence, we briefly summarize some well-known facts about the TSP and some prevalent heuristics for the same. Once again, a TSP solved on a standard solver such as CPLEX (www.ilog.com) is most often plagued with excessive computation time. Therefore, various heuristics for the TSP have been developed in the past [23]. The heuristic algorithms do not guarantee optimal tours but do find what one hopes are “near-optimal” tours. Some of these approaches involve proving performance guarantees [23] i.e., bounds on how far the solution is from the optimal, which can be constructed in the worst case. A number of constructive heuristic search methods for TSP have been developed: nearest neighbor heuristic, cheapest insertion heuristic, farthest insertion heuristic, savings cost heuristic, etc. The theoretically proven worst upper bound on the planar TSP tour is the tour obtained by doubling the minimumspanning tree. For example, Christofides’ algorithm proves that for any instance I of the TSP which obeys the triangle inequality, C(I ) 1.5 × optimal value. The algorithm computes in O(n3 ). 2.4. Summary In this research we present a heuristic algorithm to solve a new version of a vehicle routing problem based on target priorities with each target having upper and lower limits on weaponry payload requirements. This has not been addressed in prior literature. Fuel capacity of every UCAV restricts the amount of travel and is coupled with the allocation of targets to vehicles. In real-world scenarios some targets may not be destroyed with a single payload (with use of dumb bombs, and cheaper ammunition) unless precision weaponry is used. We allow more than one UCAV to hit a particular target to ensure that high priority targets receive the maximum possible damage with available resources. We do not explicitly avoid collisions between UAVs during the routing because of recent advances in collision avoidance technology. For example, the Goodrich Skywatch HP Traffic Advisory System [24] tested on the Proteus (UAV) can sense the air traffic and alter flight altitudes or turn in order to avoid collision. This reduces complexity of the formulation and helps focus on the utilization of resources to achieve better service at the targets. No-fly zone, which have to be avoided, are not considered in this research. However, pop-up targets and newly sensed targets can be incorporated within the proposed framework. 3. Problem description and formulation This study focuses on routing m UCAVs to n target locations. The targets are assumed to have already been detected by some sensors. Each target is allotted a weight based on the priority of the target. In a real time scenario, targets are prioritized based on the threat they pose, such as relocatable targets, short dwell time mobile targets, difficult and deeply buried targets, supplies, and infrastructure. Each of these targets requires a minimum weapon payload for destruction and an upper payload limit to bound collateral damage. Hence, minimum and maximum payload requirements for each target are included as constraints in the problem. Multiple UCAVs may hit a particular target to meet the payload requirements for target. It is desirable that the high priority targets are destroyed in totality for successful completion of the mission. The formulation attempts to maximize the weighted service delivered at various targets. The notation used are as follows: n V i dij

number of targets set of vertices (base or targets) index of vertices (for base i = 0) distance between vertexes i and j

1818

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

index of UCAVs number of UCAVs priority weight of target i lower level of service for target i upper level of service for target i distance limit of UCAV k payload capacity of UCAV k a large constant

k m wi Li Ui Dk Qk M

Decision variables:  xij k =  yik =

1

if UCAV k goes immediately to j from i,

0

otherwise;

1

if UCAV k visits target i (or base if i = 0),

0

otherwise;

zik = amount of “ service” delivered by UCAV k at target i;  t if target i is at position t in the sequence of targets of the tour, ai = 0 otherwise. Since the problem is a decision-making problem of routing UCAVs to targets with certain side constraints, the problem can be formulated as a mixed integer linear programming model as follows: max Z P =

(P )

n 

wi

i=1

m 

(1)

zik

k=1

subject to Li 

m 

zik Ui ,

i = 1, . . . , n,

(2)

k=1 n 

zik Qk ,

k = 1, . . . , m,

(3)

i=1 n 

n 

dij xij k Dk ,

k = 1, . . . , m,

(4)

i=0 j =0,j =i n 

xij k =

n 

xj ik = yik ,

i = 0, . . . , n; k = 1, . . . , m,

j =0,j =i

j =0,j =i

zik My ik ,

i = 1, . . . , n; k = 1, . . . , m,

m 

y0k = m,

(5) (6) (7)

k=1

ai − aj + nx ij k n − 1, xij k ∈ {0, 1},

i, j (j  = i) = 1, . . . , n; k = 1, . . . , m,

i, j (j  = i) = 0, . . . , n; k = 1, . . . , m,

(8) (9)

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

yik ∈ {0, 1}, zik 0, ai 0,

i = 0, . . . , n; k = 1, . . . , m,

i = 1, . . . , n; k = 1, . . . , m,

and

i = 1, . . . , n.

1819

(10) (11) (12)

The objective function (1) attempts to maximize the total weighted service to the targets from the UAV fleet. It is interesting to note that we are not explicitly interested in minimizing the distance traveled by the fleet, which is a typical routing objective. Nevertheless, a fuel constraint for each UAV (which will be introduced shortly) will hopefully orchestrate efficient routes for the assigned targets. Constraint (2) gives the upper and lower limits on the service required for any target. Constraint (3) represents that the weaponry allocated to a UCAV does not exceed its payload capacity. Constraint (4) represents that the distance traveled by a UCAV does not exceed its range (based on its fuel capacity). Constraint (5) maintains continuity of the route. Constraint (6) ensures that no service can be provided if a UCAV does not visit a target. Constraint (7) ensures that all UCAVs are assigned to the base. Constraint (8) ensures sub-tour elimination. Constraints (9) through (12) are binary and non-negativity constraints. The main contribution of the above formulation is the simultaneous consideration of a number of (conflicting) aspects of the routing of UCAVs. Some of these factors have been discussed in the literature, but discussions with subject matter experts from the armed forces have revealed that they have not been considered comprehensively and in an integrated formulation. In summary this formulation brings together the following: • • • • • •

priorities of targets, service level required to destroy each target, service level required to avoid collateral damage, heterogeneous fleet of UCAVs, fuel constraint on each UCAV, weaponry load constraint on each UCAV.

Without going through a formal proof, the formulation yields an NP-hard problem since we can find polynomial reductions of the m-TSP or VRP. So as expected, all our attempts to solve realistic-sized problems optimally yielded impractical running times. The following section will introduce a solution strategy that will compute feasible or implementable solutions efficiently. 4. Solution approach In US Air Force applications “a reasonable amount of time” can be viewed from two perspectives depending on whether it is the planning phase of the mission or the execution of the mission itself. In either case, the difficulty of the problem presented in this paper can have impractical running times for realistic-sized problems. As an example, even small problems with 10 UCAVs and 50 targets can take weeks to solve using state-of-the-art MILP solvers (even if using the most sophisticated settings in CPLEX, for example). Given that the optimality of such a problem is yet not practical in a reasonable amount of time, there is a need for efficient procedures, which produces feasible (or at least implementable) solutions is needed. This section describes an innovative procedure that combines the power of coordinating two decomposed problems: (i) the minimum cost network flow problem, and (ii) the traveling salesman problem. Both of these problems produce valid relaxations for the original problem, which in turn gives us upper bounds. Unfortunately, neither of the relaxations guarantees tight upper bounds. While the minimum cost network flow problem uses the same objective function Z P , the value of the solution will not improve on the linear programming relaxation bound. As we will see later, the TSP is solved only to check for feasibility of the routes. So we employ a meta-heuristic that ties together the solution of these two sub-problems. We chose Tabu search for its flexibility of either intensifying the search in a subspace of solutions or to diversifying to find potential new subspaces of interest. Even though Tabu search has been used in many optimization problems the notion of controlling a network flow subproblem and a TSP subproblem is a creative way to attain a “good” feasible (or implementable) solution for problem (P).

1820

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

Now we outline the specific decomposition of (P) for our proposed solution approach, which is achieved by relaxing constraint (6). Such decompositions have been prevalent for VRP as discussed in Section 2. The first part of the decomposition is for obtaining the service allocation through a minimum cost network flow subproblem. The second part is to find a feasible route for the UCAVs through a VRP subproblem. These two subproblems are solved iteratively until a good feasible solution is found. The first half of the problem, which can be termed as a target assignment problem (A) is as follows: max Z A =

(A)

n 

m 

wi

i=1

zik

k=1

subject to m 

Li 

zik Ui

∀i,

k=1 n 

zik Qk

∀k,

i=1

zik 0 − min

∀i∀k. n 

−wi

i=1

m 

zik

k=1

subject to m 

Li 

zik Ui

∀i,

k=1 n 

zik Qk

∀k,

i=1

zik 0

∀i∀k.

The above problem is a minimum cost flow problem. Cost per unit flow on the network is −wi . Arcs from UCAVs to targets have an upper limit of Qk and arcs from targets to the sink have an upper and lower limit on the target service requirements. This problem can be solved using a standard min-cost network flow algorithm [25]. The second half of the problem is a routing problem without an objective, but one that satisfies constraints (B): n n  

dij xij k Dk ,

k = 1, . . . , m,

i=0 j =0,j =i n 

xij k =

j =0,j =i m 

n 

xj ik = yik ,

i = 0, . . . , n; k = 1, . . . , m,

j =0,j =i

y0k = m,

k=1

ai − aj + nx ij k n − 1, xij k ∈ {0, 1}, yik ∈ {0, 1}, ai 0,

i, j (j  = i) = 1, . . . , n; k = 1, . . . , m,

i, j (j  = i) = 0, . . . , n; k = 1, . . . , m, i = 0, . . . , n; k = 1, . . . , m,

i = 1, . . . , n.

Thus, problem (B) is not an optimization but a feasibility problem, which itself decomposes into m independent decision-TSP problems, one for each UCAV. It is well known that an optimization TSP problem polynomially reduces

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

1821

into a decision-TSP problem, which indicates that (B) is NP-complete. Some readers might be more familiar with the decision-TSP as a problem of finding a Hamiltonian cycle of a graph within a specified length. Hence, we will employ a heuristic to obtain approximate solutions in an acceptable computational time. The network flow problem (A) determines the assignments to be made and the traveling salesman heuristic (TSH) determines a good route for problem (B). The fuel/range capacity constraint (4) has to be satisfied and simultaneously the best assignments have to be determined. Therefore, we employ the Tabu search heuristic. 4.1. Components of the Tabu search In this section, the components of the search procedure are described, followed by a detailed step-by-step outline. Moves: A basic variable corresponds to an arc in the network tree, called a basic arc, and a non-basic variable corresponds to an arc not in the network flow, called a nonbasic arc. A move is defined as elimination (Tabu) of the arc from the network that is a part of the basic feasible solution so that the non-basic arcs (variables zik ) could enter the solution space. The search is among the alternative solutions to the network flow problem (A), which may or may not be feasible for the constraints (B). The evaluation function is composed of the two independent measures: • Let Z be the change in the objective value. It is positive when the objective value increases. • Let d be the change in the distance or fuel consumed by all the UCAVs. It is positive when the new tour is economical compared to the previous computed tour. Based on the values of Z and d all the candidates are partitioned. In the basic feasible solution the zik is exchanged with the other k’s for that particular i. For each i the value of Z and d for each exchange k is computed. This iteration is performed until a predetermined integer l is reached. Moves that create infeasibility to the network or to the constraints (B) are not considered.  solution and Diversification: For a given solution to the network flow problem the value of dk = ni=0 Feasible n j =0,j =i dij xij k is computed for each k using TSH (see Section 4.3). If the constraints (B) are satisfied, a feasible solution is obtained. Otherwise, the factor (dk − Dk )/Dk is computed for each infeasible k. The k with the maximum ratio (dk −Dk )/Dk is selected and the corresponding i, which is farthest in the route, is put on the Tabu list. The network flow is solved again and the UCAVs are reassigned. The intention is to eliminate the arcs, which cause infeasibility to the complete problem (A + B) until feasibility is obtained. The Tabu list length is adaptive. If feasibility is not obtained in certain number of steps (n cycles in the Tabu list) the length is increased by a factor (say 1.1). Once a feasible solution is found, the arcs from the initial basic feasible solution (only to the network problem) are removed from the Tabu list and brought back into the solution space to look for better solutions. This diversifies the search. It continues for  predetermined steps. The best objective value and the least fuel consumed for the mission are stored in memory. Intensification: For every zik in the solution, each i is individually swapped with the other vehicles k for improvement in the current solution. This intensifies the search. 4.2. The Tabu search procedure The Tabu search procedure is outlined here in a stepwise fashion. It should be obvious, that theoretical guarantees to find a feasible solution to decision-TSP problems in (B) using a polynomial time heuristic are not possible. Nevertheless, empirical testing has shown that for practical instances of the problem, our TSP heuristic provides feasible solutions. The following procedure could be run until a feasible solution is found or until the decision-maker determines that the computation time has reached a particular limit. At that point additional UCAVs can be included, targets of less importance could be deleted or UCAV travel/load capacity can be increased. Step 1: Let Z ∗ = 0, select the values of Tabu list length (generally equal to the number of targets), diversification iterations  and intensification iterations l. Step 2: Find a feasible solution to flow and compute the value of Z according to (A). the network  Step 3: Calculate the value dk = ni=0 nj=0,j =i dij xij k for each k by TSH. Compute the value of Z for each vehicle k that violates its range.

1822

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

Step 4: The k with maximum value of ratio rk = (dk − Dk )/Dk is selected. The farthest node i is eliminated from the solution (arc zik exits the solution space). Repeat Step 2. If the network is infeasible the zik is removed from the Tabu list and the next rk is considered for Step 4. Feasibility to the network problem is maintained at all times. Step 5: When a feasible solution is obtained for (A) and (B) the zik from the initial solution for the network flow is made non-Tabu. Go to Step 2. Diversification is continued for  iterations (The diversification of the solution improves with higher iteration value). The incumbent solution is updated when Z > Z ∗ . Step 6: For each i an alternate k is assigned. Calculate the value of Z and d. The selection criterion is as follows: • Type 1: If the Z > 0 and d 0, the best-computed value of Z ∗ is incumbent solution. Correspondingly the Tabu list is updated. • Type 2: If the Z > 0 and d < 0, the best-computed value of Z ∗ is incumbent solution. Correspondingly the Tabu list is updated. • Type 3: If the Z 0 and d 0 the solution is accepted. But the incumbent is not updated. Type 1 is given highest priority and Type 3 the lowest priority of all the k’s interchanged for each i. If for a particular i all k’s are either Type 1 or Type 2 then the maximum is the selected Tabu move. For Type 3 the maximum is the selected Tabu move. All infeasible solutions to (A) and (B) are rejected. Step 7: The best value of Z at Step 5 is made the incumbent solution, Z ∗ = Z. Stop after l iterations at Step 5 have been completed. 4.3. TSP heuristic In the process of solving the part (B) of the problem we considered to use Christofides’ algorithm [23], which gives a performance guarantee of 150% from optimality for planar TSPs and a running time of O(n3 ). Unfortunately, the running time of restarting Christofides procedure every single time we have a change in the assignment is computationally unattractive. In our research we demonstrate an implementation of insertion or deletion of targets in a route that requires only O(n) in the running of the TSP for inserting every new node. Thus, it computes the tour in O(n2 ) from the start. Even though some anomalistic instances prevents us from proving a performance guarantee ratio, in practicality this procedure is found to be comparable to Chritofides’ algorithm. 4.3.1. Steps for traveling salesman heuristic (TSH) procedure 1. Consider a set of cities say Set A with n cities. A = {a1 , a2 , a3 , a4 , . . . , an }. 2. Select any three cities a1 , a2 and a3 from the set and connect the three cities to form an initial tour (I1 ). The arcs forming the tour are 1.2, 2.3, 1.3 or (1.2.3.1). 3. Select a city say ai ∈ A randomly to insert into the tour I1 . 4. Find the shortest distance of ai from the tour I1 . Let the shortest distance be 1.i. 5. Compute the minimum of the differences between the arc distances E1 = {(i.3) − (1.3)}, and E2 = {(i.2) − (1.2)}. (Note: The difference is between the potential inserting arc and the leaving arc.) 6. If E1 < E2 delete arc 1.3 and insert arc i.3. So the new tour I2 is (1.i.3.2.1). 7. Go to Step 3 and continue until the In−2 th tour that includes all cities is found. 4.3.2. Example explaining TSH Let us assume dab = 3; dbc = 6; dac = 4; dad = 10; dbd = 9; dcd = 7. Let cities a, b and c be randomly selected in Step 2 of the TSP heuristic to provide an initial tour I1 of length 13 units (Fig. 2). Assume the city d is randomly selected for insertion in tour I1 . Note that city d is closest to city c. We consider E1 = dbd − dbc = 9 − 6 = 3, E2 = dad − dac = 10 − 4 = 6. With E1 < E2 we determine that city d is to be inserted between b and c to provide the next tour I2 of length 13 + 7 + 9 − 6 = 23 units. This procedure is repeated for all other remaining cities (nodes).

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

1823

c

4 a 3

6

10

7

b 9 d Fig. 2. Example illustrating TSP heuristic.

Theoretically, the TSH does not guarantee 150% from optimality but it is considerably faster. To study its performance we briefly discuss experimental results in Section 5. 5. Computational experiments As mentioned earlier, our problem is NP-hard while decisions have to be made in real-time. This motivated us to develop a heuristic approach that is expected to perform efficiently both computationally as well as in the quality of the solutions. To establish this we created an empirical study for the performance of our approach. We utilized some small problems for which we could obtain an integer optimal solution using CPLEX 7.5 on 1.6 MHz Intel Pentium 4 with 512 MB RAM. For realistic problem sizes, where CPLEX could not provide an integer solution in a reasonable amount of time, we used the Linear Programming bound to measure the gap of the solution obtained by our heuristic. The CPLEX search parameters we used are down first branch, up branch first and rinsheur. The down first branch seemed to work best to get a feasible solution. The data that we utilized to perform the test were extracted from the technical description of numerous existing UCAVs (i.e., Boeing X-45A), as well as the distances for most of the examples where a correlation from the wellknown Solomon Data Sets used in vehicle routing problems. Finally to demonstrate the empirical performance of TSH, we used TSPLIB data set (http://www.iwr.uni-heidelberg.de/groups/comopt/software), which gives a baseline of comparison against other heuristic methods. 5.1. Test problems There are three separate examples considered in this section. We used realistic data from open sources, e.g., Boeing’s X-45A UCAV demonstrator (http://www.fas.org). UCAV type X45A

Payload 3000 lb s

Fuel capacity 30 min

Missiles 12

Distance (miles) 375

Storm Shadow is a stealth UCAV developed in response to a proposal call of American Institute of Aeronautics and Astronautics. It can carry 1000 lbs of semi-precision guided weaponry. So a combination of such UCAVs could be used for combat in a single mission. The UCAV could carry precision cruise missiles or dumb bombs. Prevalent UCAV payloads are described in Table 1. In our mathematical model, after decomposition, part (B) of the problem has to determine feasibility for fuel/range. Using the above-mentioned factors we can determine the amount of distance that needs to be traveled to meet the critical service requirements of each target. If the distance requirements do not satisfy the range  of the particular UCAV, the assignment is considered infeasible. We compute the distance traveled by each UCAV as ( ni=0 nj=0,j =i dij xij k ) and the range of UCAV k as Dk .

1824

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

Table 1 Payload data for munitions Missiles

Description

Payload (lbs)

Multipurpose bombs JDAM SDM SSB MK 36 BSU 49 MK 82

Joint direct attack munition Small diameter bombs Small smart bombs (guided) Destructor mine Ballute Air-to-ground armament

250 1000/500 500 250 500 500 500

see www.fas.org.

Table 2 Case 1 example: data and results Target

Upper limit (Ui )

2 5000 3 1000 4 250 5 250 6 1000 7 5000 8 500 9 250 10 500 11 1000 12 250 13 250 14 1000 15 250 16 500 17 500 18 500 19 250 20 250 Objective value Total fuel consumed in mission Computation time (s)

Lower limit (Li )

Weight (wi )

zik (heuristic)

zik (CPLEX)

5000 500 0 0 1000 2500 250 250 250 500 0 0 500 250 250 250 250 0 0

20 5 5 10 20 20 10 5 10 5 5 10 20 5 10 10 10 5 10

5000 500 0 0 1000 2750 250 250 250 500 0 0 500 250 250 250 250 0 0 205,000 708 25

5000 500 0 0 1000 2750 250 250 250 500 0 0 500 250 250 250 250 0 0 205,000 637 2700

Case 1: This case considers problems where the value of the solution to the LP relaxation of (P) is same as the objective function value for the optimal solution of (A). This does not necessarily mean the problem as a whole has integrality property. The problem is not solved until a route is found for each UCAV and confirms the possibility of the assignments determined by the network flow problem. CPLEX provides various branch strategies to solve mixed integer problems. The depth first strategy reaches the objective value promptly but performs poorly in the distance constraint (as shown in Table 2) while the heuristic outperforms CPLEX in the same computation time. If we increase the number of targets to 40 with 7 UCAVs, the heuristic takes 30 s to solve the problem with an objective value of 352,500 ( = 5; l = 2). Upper bound was determined to be 420,000. The upper bound is obtained from the LP relaxation of the problem (P). Fig. 3 presents the tours obtained for this example. In all these examples, the LP solution value for the was same as the IP objective function value. Most of CPLEX’s run time was spent trying to find a feasible integer solution and proving its optimality. Case 2: Unlike the previous case, the integer problem solution is different from the LP relaxation solution. This example had 15 targets and 4 UCAVs. The fuel capacities and service capabilities of the 4 UCAVs are presented in

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

1825

Fig. 3. Tours obtained from heuristic algorithm with 40 targets and 7 UCAVs.

Table 3 Case 2 example: data and results Vehicle k

1 2 3 4

Dk fuel capacity

1500 1500 700 1000

Qk service capability

3000 3000 8000 4000

Heuristic

CPLEX

Fuel consumed

Amount of service provided

Fuel consumed

Amount of service provided

1013 1398 485 424

2500 2000 7500 4000

439 1040 663 784

3000 2250 6250 4000

columns 2 and 3 of Table 3. The Tabu search heuristic solution provided an objective value of 266,250 (Fig. 4). The control values were  = 5; and l = 2. Columns 4 and 5 represent, respectively, the amount of fuel consumed and service provided by each UCAV. The MIP solution obtained using CPLEX had objective value of 263,750 after 17 h of computation and was reported to have an optimality gap of 2.37%. The LP relaxation solution to this problem had an objective value of 270,000. Columns 6 and 7 represent, respectively, the amount of fuel consumed and service provided by each UCAV in the CPLEX solution. Thus, this example demonstrates the effectiveness of the proposed heuristic. 5.2. Optimality Gap The Tabu search heuristic may not find an optimum solution but is able to find an initial feasible solution very fast whereas CPLEX takes a long time even to obtain a feasible solution. The examples shown in Table 4 are examples where there is a gap between Heuristic solution and the optimum solution (CPLEX) but there is an appreciable difference of computation time even for smaller problems. All the examples in Table 4 have the same payload capacity constraints for each UCAV. The only constraint that is changed is the range limit (or fuel) constraint. If the range/fuel constraint is too tight for smaller problems, CPLEX manages to find a solution while the heuristic fails to find a solution (illustrated in Example 1 of Table 4). As we increase the range/fuel constraint on the UCAVs the heuristic is able to find a solution. Examples 2 and 3 show a gap of 9.8% and 6.5%, respectively, from optimality. But the tradeoff is computation time to obtain solutions, which is critical while routing UCAVs. The control values  and l that control the amount of diversification and intensification on the best solution influence the optimality gap.

1826

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

Fig. 4. Case 2 example: tours of 4 UCAVs provided by the heuristic.

Table 4 Comparison of CPLEX and TSH (heuristic) Targets/UCAVs

1 (a)

10/4

Cplex soln. Range limits (Dk ) Heuristic

Objective

203,750

(a) (b) 3 (a)

Total fuel consumed

UCAV 1

UCAV 2

UCAV 3

UCAV 4

128 150

128 150

200 200

200 200

160

150

200

200

175 175 173 175 199 200

128 150 128 150 128 150

195 200 195 200 192 200

175 200 175 200 175 200

673

128 150 165 165

128 150 128 150

200 200 195 200

200 200 175 200

656

656

Fails to optimize

Range limits (Dk ) 2

Fuel capacity for each vehicle

Cplex soln. Range limits (Dk ) Heuristic Range limits (Dk ) Heuristic Range limits (Dk )

223,750

Cplex soln. Range limits (Dk ) Heuristic Range limits (Dk )

203,750

203,750 223,750

191,250

671 694

663

5.3. Discussion on the heuristic used for TSP The insertion based TSP heuristic used for computing the tour is O(n2 ). It was chosen over other well-known procedures for its speed. Since it influences the overall solution, it is important to determine its performance separately. The data set for this comparison is obtained from TSPLIB (http://www.iwr.uni-heidelberg.de/groups/comopt/software). This heuristic clearly does not guarantee an upper bound on the tour obtained but for most of the cases we found that the heuristic provides an upper bound of 1.5 from optimality. Cases where the heuristic fails to be 1.5 from the optimal are shown in the Table 5 (data sets 1 and 2). Randomization of the sequence of cities selected for forming a tour helps in getting better tours to the cities. Another justification of using this heuristic is that it helps in adding new targets (“pop-ups”) in the existing tour in O(n) when new targets are detected in a dynamic environment.

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

1827

Table 5 TSPLIB data No.

Data set from TSP LIB

Optimum known (a)

TSP heuristic O(n2 ) (b)

Ratio (b)/(a)

1 2 3 4 5 6 7 8 9

xqg237_tsp XQF131_tsp ch150_tsp_gz ch130 lin105 KroB150 berlin52 KroD100 tsp225

1019.00 564.00 6528.00 6110.00 14,379.00 26,130.00 7542.00 21,294.00 3919.00

1595.00 909.00 8412.81 7549.86 21,374.20 35,536.80 9551.40 27,566.00 5526.00

1.57 1.61 1.29 1.24 1.49 1.36 1.27 1.29 1.41

6. Conclusions and future work This research presents a new UCAV routing problem that considers several realistic concerns. UCAVs are assigned and routed to targets based on target priorities (measured as a weight). The lower limits on the targets make it easy to at least cause minimal damage on the targets and the upper limits help bound collateral damage. The routing is dealt tactfully in symphony with assignments to obtain near-optimal tours in significantly smaller computation time compared to a standard solver. The heuristic for TSP not only calculates quick tours when the assignments are finalized but is designed to insert “pop-up” targets without causing the initial tour to be completely changed. The heuristic algorithm works at two levels. At the higher level the objective function drives to make the best network flow assignment. Once the target assignment is accomplished the TSP heuristic tries to get feasible routes. The Tabu list contains the targets which cannot be serviced in the current iteration. Thus, the Tabu search procedure systematically explores the solution space with the objective of providing maximum possible service. For larger scale problems or problems that are intractable even for feasibility using CPLEX, the heuristic procedure provides good feasible solutions quickly. Furthermore, the iterative nature of the heuristic allows it to be stopped after a feasible solution has been obtained if the analyst prefers not to wait for a better solution at convergence. The major criteria of faster decision-making for real-time UCAV assignment has been the focus of this work, even if it is accompanied with potential loss of optimality for large scale problems. The problem considered and solution developed in this paper would need significant further work before it can be transitioned to military practice. First, a vast array of technological, policy, and operational challenges as outlined in [25] must be overcome before UCAVs can be successfully deployed. UCAVs have to be at least as effective as manned aircraft, possess demonstrated mission success rates, and be amenable to easy integration into the overall battle plan. UCAVs must be responsive to a continuously changing airspace picture. We believe that with suitable extensions the approach suggested in this paper is amenable to meet these challenges. Extensive testing needs to be performed (on classified data) before the approach can be transitioned to fielded use. Much work needs to be accomplished in cooperation of UCAVs facing pop-up targets. We believe that the framework of VRP with time windows can be employed for time-sensitive and opportunistic targets. The impact of weather and compromise to sensor systems is another dynamic aspect that must be studied thoroughly and integrated in a mission plan. We recommend developing a simulation-based approach and rigorous performance evaluation for validating this (or a variant) model and solution approach. Simulation can allow study of many more realistic battlefield considerations such as sensor networking, line of sight, and weather conditions that are too detailed to be considered in a single mathematical formulation. Design of experiment studies and analysis of variance (ANOVA) can provide the military commander with significant and interacting factors, and provide practical insights. Optimal values of the various parameters used in the heuristic procedure can also be determined. Finally, we now recognize that there might be versions of the commercial distribution problem where the customers have priorities (based on individual contracts and profits) and where the delivery quantities can be specified within bounds (to accommodate demand uncertainty at the customer site). Such problems might benefit from the decomposition scheme proposed in this work with Tabu search coordinating the subproblems.

1828

V.K. Shetty et al. / Computers & Operations Research 35 (2008) 1813 – 1828

Acknowledgments The authors would like to thank the two anonymous referees and guest editor for their constructive comments. These comments helped strengthen the paper significantly. References [1] Storm Shadow UCAV performance the American Institute of Aeronautics and Astronautics http://www.aerospaceweb.org/design/ucav/ performance.shtml. [2] Gano SE, Renaud JE. Optimized unmanned aerial vehicle with morphing for extended range and endurance. Ninth AIAA/ISSMO symposium and exhibit on multidisciplinary analysis and optimization, paper no. 2002-5668, 2002. [3] Donato G, Sinopoli B, Micheli M, Koo TJ. Vision based navigation for an Unmanned Aerial Vehicle. Proceedings of the IEEE international conference on robotics and automation; 2001. p. 1757–64. [4] Cunningham CT, Roberts RS. An adaptive path planning algorithm for cooperating unmanned air vehicles. Proceedings of the IEEE international conference on robotics and automation; 2001. p. 3981–86. [5] Enright JJ, Frazoli E. On multiple uav routing with stochastic targets: performance bounds and algorithms. Proceedings of the AIAA conference on guidance, navigation and control; 2005. [6] Alighanbari M, Bellingham JS, Tillerson M, How JP. Cooperative path planning for multiple UAVs in dynamic and uncertain environments. Proceedings of the 41st IEEE conference on decision and control; 2002. p. 2816–22. [7] Lian Z, Deshmukh A. Performance prediction of an unmanned airbone vehicle multi-agent system. European Journal of Operational Research 2006;172:680–95. [8] Chandler PR, Pachter M, Rasmussen S. UAV cooperative control. Proceedings of the American control conference, Arlington VA; 2001. p. 50–5. [9] Moore JT, Ryan JL, Bailey G, Carlton WB. Reactive tabu search in unmanned aerial reconnaissance simulations. Proceedings of the 1998 winter simulation conference; 1998. p. 873–9. [10] Flint M, Fernandez-Gaucherand E, Polycarpou M. Stochastic modeling of a cooperative autonomous UAV search problem. Military Operations Research 2003;8(4):13–32. [11] Maza I, Ollero A. Multiple UAV cooperative searching operation using polygon area decomposition and efficient coverage algorithms. Proceedings of the seventh international symposium on distributed autonomous robotic systems, France, 2004. [12] Minai AA, Jin Y, Polycarpou M. Cooperative real-time search and task allocation in UAV teams. Proceedings of the 42nd IEEE conference on decision and control; 2003. p. 712. [13] Richards A, Bellingham J, Tillerson M, How J. Co-ordination and control of multiple UAV’s. In: AIAA GNC conference; August 2002. [14] Richards A, Bellingham J, Tillerson M, How J. Multi-task allocation and path planning for cooperating UAVs. In: Cooperative control: models, applications and algorithms at the conference on coordination, control and optimization; November 2001. p. 1–19. [15] Richards A, How JP. Aircraft trajectory planning with collision avoidance using mixed integer linear programming. Proceedings of the American control conference, vol. 3; 2002. p. 1936–41. [16] Bellingham J, Richards A, How JP. Receding horizon control of autonomous aerial vehicles. Proceedings of the American controls conference, vol. 5; 2002. p. 3741–46. [17] Harder WR, Hill RR, Moore JT. A Java universal vehicle router for routing unmanned aerial vehicle. International Transactions in Operational Research 2004;11:259–75. [18] O’Rourke KP, Bailey TG, Hill RR, Carlton WB. Dynamic routing of unmanned aerial vehicles using reactive tabu search. 67th MORSS, US Military Academy, West Point, NY, 22–24 June 1999. [19] Laporte G, Gendreau M, Potvin J-Y, Semet F. Classical and modern heuristic for the vehicle routing problem. International Transaction in Operational Research 2000;7:285–300. [20] Glover F. Tabu search—part I. ORSA Journal of Computing 1989;1(3). [21] Glover F. Tabu search—part II. ORSA Journal of Computing 1990;2(1). [22] Toth P, Vigo D. The vehicle routing problem. SIAM monographs on discrete mathematics and applications, Philadelphia, PA; 2002. [23] Lawler EL, Lenstra JK, Rinnooy Kan AHG, Shmoys DB. The traveling salesman problem. New York: Wiley; 1985. [24] Proteus UAV collision–avoidance tests, Phase II, April, 2003. www.dfrc.nasa.gov. http://www.scaled.com/projects/proteus/more/mission/ Proteus%20Payload%20Users%20Guide.pdf. [25] Barry CL, Zimet E. UCAVs—technological, policy, and operational challenges. Defense horizons no. 3, 2001. http://www.ndu.edu/inss/DefHor/ DH3/DH3.htm.