- Email: [email protected]
Supporting Humans in Solving Multi-UAV Dynamic Vehicle Routing Problems
14th IFAC Symposium on 14th IFAC Symposium on Analysis and Evaluation of Human Machine Systems 14th IFACDesign Symposium on Available online at www.sciencedirect.com Analysis Design and Evaluation of Human Machine Systems 14th IFAC Symposium on Tallinn, Estonia, Sept. 16-19, 2019 Analysis Design and Evaluation of Human Machine Systems Tallinn, Estonia, Sept. 16-19, 2019 Analysis Design and Evaluation of Human Machine Systems Tallinn, Estonia, Sept. 16-19, 2019 Tallinn, Estonia, Sept. 16-19, 2019
ScienceDirect
IFAC PapersOnLine 52-19 (2019) 359–364
Supporting Humans in Solving Multi-UAV Supporting Humans in Solving Multi-UAV Supporting Humans in Solving Multi-UAV Dynamic Vehicle Routing Problems Supporting Humans in Solving Multi-UAV Dynamic Vehicle Routing Problems Dynamic Vehicle Routing Problems Dynamic Vehicle Routing Problems N. W. Klein Koerkamp C. Borst 11 Max Mulder
N. W. Klein Koerkamp C. Borst 111 Max Mulder M. Paassen N. W. Klein Koerkamp Borst 1 Max Mulder M. van vanC. Paassen M. M. N. W. Klein Koerkamp Borst Max Mulder M. M. vanC. Paassen M. M. van Paassen Control Control and and Simulation, Simulation, Faculty Faculty of of Aerospace Aerospace Engineering, Engineering, TU TU Delft, Delft, Control and Simulation, Faculty ofThe Aerospace Engineering, TU Delft, 2629 HS, Delft, Netherlands 2629 HS, Delft, The Netherlands Control and Simulation, Faculty of Aerospace Engineering, TU Delft, 2629 HS, Delft, The Netherlands 2629 HS, Delft, The Netherlands Abstract: Abstract: Real-time Real-time optimization optimization of of Vehicle Vehicle Routing Routing Problems Problems (VRP) (VRP) during during mission mission operaoperaAbstract: Real-time optimization of Vehicle Routing Problems (VRP) during mission operations raises concerns regarding obtaining a solution within a reasonable timeframe, especially tions raises concerns regarding obtaining a solution within a reasonable timeframe, especially Abstract: optimization of easily Vehicle Problems (VRP) during mission operations raises Real-time concerns regarding obtaining a solution within reasonable timeframe, especially in domains domains where operations cannot beRouting paused and athe the number of control parameters in where operations cannot easily be paused and number of control parameters tions raises concerns regarding obtaining a solution within a reasonable timeframe, especially in domains where operations cannot easily be paused and the number of control parameters is high. high. Humans, Humans, however, however, are are heuristic heuristic problem problem solvers solvers and and could could potentially potentially complement complement is in domains whereinhowever, operations cannot easily be paused andsolution the could number of control parameters is high. Humans, are heuristic problem solvers and potentially complement VRP algorithms providing quickly aa workable and safe from which the algorithms VRP algorithms in providing quickly workable and safe solution from which the algorithms is high. Humans, however, are heuristic problem solvers and could potentially complement VRP algorithms in providing quickly a study, workable and safe solution from which the algorithms can further find the optimum. In this a visual interface was developed and evaluated can further find the optimum. In this study, a visual interface was developed and evaluated VRP algorithms in providing quickly workable and safe solution whichthey the can further find the optimum. In thisa study, a visual interface wasfrom developed andalgorithms evaluated aiming to support humans in manually solving a dynamic VRP in which needed to aiming to support in In manually solving a dynamic VRP indeveloped which they to can further find thehumans optimum. this study, a visual interface wasUnmanned and needed evaluated aiming to support humans in delivery manually solving a dynamic VRP in which they needed to solve various simulated payload missions, featuring multiple Aerial Vehicles, solve various simulated payload missions, featuring multiple Unmanned Aerialneeded Vehicles, aiming to support humans in delivery manually solving a16) dynamic VRP in they to solve simulated payload delivery missions, multiple Aerial Vehicles, undervarious failure conditions. Experiment results (n = =featuring indicate thatUnmanned thewhich interface enabled the under failure conditions. Experiment results (n 16) indicate that the interface enabled the solve various simulated payload delivery missions, featuring multiple Unmanned Aerial Vehicles, under failure conditions. to Experiment results = 16) indicate that the interface enabled majority of participants participants quickly solve solve the (n perturbed scenarios, although not always always in the the majority of to quickly the perturbed scenarios, although not in under failure conditions. Experiment results (n = 16) indicate that the interface enabled the majority of participants to quickly solve theexperienced perturbed scenarios, although not always in the most efficient way. Interestingly, participants most difficulty in solving the seemingly most efficient way. Interestingly, participants most difficulty in solving the seemingly majority of participants toless quickly solve and theexperienced perturbed scenarios, although not always in the most efficient way.featuring Interestingly, participants experienced most difficulty in vehicles solving the seemingly easier scenarios, customers a relatively low number of compared to easierefficient scenarios, less customers andexperienced a relatively low difficulty number of compared to most way.featuring Interestingly, participants in vehicles solving the seemingly easier scenarios, featuring less customers and a relativelymost low number of vehicles compared to the more complex scenarios. the more complexfeaturing scenarios. easier scenarios, less customers and a relatively low number of vehicles compared to the more complex scenarios. Copyright © 2019. The Authors. Published by Elsevier Ltd. All rights reserved. the more complex scenarios. Keywords: Keywords: Human-machine Human-machine interface, interface, vehicle vehicle routing routing problem, problem, unmanned unmanned aerial aerial vehicles. vehicles. Keywords: Human-machine interface, vehicle routing problem, unmanned aerial vehicles. Keywords: Human-machine interface, vehicle routing problem, unmanned aerial vehicles. 1. INTRODUCTION INTRODUCTION aa concern concern when when aa need need for for re-optimization re-optimization arises arises due due to to 1. 1. INTRODUCTION aperturbations concern when a need for re-optimization arises to perturbations during mission operations. Third, for some during mission operations. Third, fordue some 1. INTRODUCTION a concern when a need for re-optimization arises due to scenarios there might not even be a solution. For example, perturbations during mission operations. Third, for some Vehicle routing routing problems problems (VRP) (VRP) are are at at the the core core of of many many scenarios there might not even be a solution. For example, Vehicle perturbations during mission operations. Third, for some for over-constrained VRP algorithms be there mightproblems, not even be a solution. For might example, Vehicle routing problems (VRP) are at the core ofjust-inmany scenarios logistics applications. With the focus on for over-constrained problems, VRP algorithms might be logisticsrouting applications. With the current current focus onof just-inscenarios there might not even be a solution. For example, Vehicle problems (VRP) are at the core many unable to find a solution at all, without first performing for over-constrained problems, VRP algorithms might be time logistics and data-driven analysis techniques, costlogistics applications. With the current focus on just-inunable to find a solution at all, without first performing time logistics and data-driven analysis techniques, costfor over-constrained problems, algorithms might be logistics applications. Withofthe current focus on just-inunable to find a solution at all,VRP without first performing some kind of constraint relaxation (Lau et al., 2003). efficient routing of a fleet vehicles plays an important time logistics and data-driven analysis techniques, costsome kind of constraint relaxation (Lau et al., 2003). efficient routing of data-driven a fleet of vehicles plays an important to find a solutionrelaxation at all, without time logistics and techniques, cost- unable some kind ofway constraint (Lau etfirst al., performing 2003). efficient routing of a fleet(Lin of vehicles plays an important role in in many industries etanalysis al., 2014). 2014). Determining A possible to overcome these challenges is role many industries (Lin et al., Determining some kind of constraint relaxation (Lau et al., 2003). efficient routing of a fleet of vehicles plays an important A possible way to overcome these challenges is by by havhavrole in many industries (Lin etinal., 2014). Determining efficient routes for each vehicle the fleet, such that the A possible way to overcome these challenges is by havhumans solving these problems as they are creative efficient routesindustries for each vehicle inal., the 2014). fleet, such that the ing role in many (Lin et Determining ing humans solving these problems as they are creative overall mission goal is achieved within distance, capacity A possible way to overcome these challenges is by havefficient routes for each vehicle in the fleet, such that the problem solvers who can adapt to novel situations. Preoverall mission goal is achieved within distance, capacity ing humans solving these problems as they are creative efficient routes for each vehiclethe in the fleet, such capacity that the problem solvers who these can adapt to novel situations. Preand time constraints, defines VRP (Laporte, 2007). ing humans solving problems as they are creative overall mission goal is achieved within distance, vious research on human control performance in solving and time constraints, defines the VRP (Laporte, 2007). problem solvers who can adapt to novel situations. Preoverall mission goal is defines achieved within distance, capacity research on human control performance in solving and time constraints, the VRP (Laporte, 2007). vious Although much attention is given in literature to obtaining problem solvers who can adapt toperformance novel situations. PreTraveling Salesman Problems (TSP) shows good human Although much attention is given inVRP literature to obtaining vious research on human control in solving and time constraints, defines the (Laporte, 2007). Traveling Salesman Problems (TSP) shows good human routes for formuch staticattention problems, in many many real-lifeto situations situations research on human control performance in solving Although is given in literature obtaining vious Traveling Salesman Problems (TSP) shows good human performance in creating optimal routes purely based on routes static problems, in real-life Although attention is given in literature obtaining in creating optimal routesshows purely based on a a in due many real-lifeto situations these problems problems become dynamic to changing changing customer performance Traveling Salesman Problems (TSP) good human routes formuch static problems, visual representation of the customer locations and vehicle these become dynamic due to customer performance in creating optimal routes purely based on a routes for static problems, in many real-life situations visual representation of the customer locations and vehicle locations, vehicle failures or service travel in creating optimal routes purely 2001; based on a these problems become dynamic due to changing customer routes representation (Anderson 2000; Vickers et Scott locations, vehicle failures or uncertain uncertain service and and travel performance visual of the customer locations and vehicle these problems become dynamic due to mission changing customer (Anderson et et al., al., 2000; Vickers et al., al., 2001; Scott locations, vehicle failures or uncertain service operations and travel routes times caused by perturbations during visual representation of the customer locations and vehicle routes (Anderson et al., 2000; Vickers et al., 2001; Scott times caused by perturbations during mission operations Dry 2006; T¨ u u u et al., 2009; Maclocations, vehicle failures or 2013). uncertain service operations and travel et et al., al., 2002; 2002; Dry et etetal., al., 2006; T¨ ut¨ t¨ unc¨ nc¨ u et et al., al., 2001; 2009; Scott Mac(Laporte, 1992; Pillac et al., al., routes (Anderson al.,By 2000; Vickers times caused byPillac perturbations during mission et al., 2002; Dry 2011). et al., 2006; T¨ ut¨ unc¨ uthese et al., 2009; visual Macgregor and Chu, leveraging human (Laporte, 1992; et 2013). times caused by perturbations during mission operations gregor and Chu, 2011). By leveraging these human visual et al., 2002; Dry et al., 2006; T¨ u t¨ u nc¨ u et al., 2009; Mac(Laporte, 1992; Pillac et al., 2013). pattern recognition qualities, good performance in solving gregor and Chu, 2011). By leveraging these human visual In solving dynamic VRPs, various challenges can be identi(Laporte, 1992; Pillac et al., 2013). recognition qualities, good performance in solving In solving dynamic VRPs, various challenges can be identi- pattern gregor and Chu, 2011). By leveraging these human visual real-time dynamic might be achieved. recognition qualities, good in solving VRPs, various challenges fied. First,dynamic the optimization optimization problem needs to tocan be be explicitly In solving identi- pattern real-time dynamic VRPs VRPs might be performance achieved. fied. First, the problem needs be explicitly pattern recognition qualities, good in solving In solving dynamic VRPs, various challenges can be identireal-time dynamic VRPs might be performance achieved. formulated. Although a wide range of VRP types has fied. First, the optimization problem needs to be explicitly This study comprises both the design and evaluation of formulated. Although a wide rangeneeds of VRP types has real-time dynamic VRPs might be achieved. fied. First, the optimization problem to be explicitly This study comprises both the design and evaluation of a a been studied studiedAlthough in literature, literature, constructing anVRP algorithm that formulated. a wide range ofan types that has This visual interface for solving a dynamic VRP under perturbeen in constructing algorithm study comprises both the design and evaluation of a formulated. Although a wide range of VRP types has visual interface for solving a dynamic VRP under perturbeen studied in literature, constructing an algorithm that takes into account all possible disturbances and stochastic This study comprises both ago thea design and evaluation of a visual interface VRP under perturtakes studied into account all possible disturbances and stochastic As it aims than been literature, constructing an algorithm that bations. bations. As such, such,for it solving aims to to agodynamic a step step further further than previous previous and stochastic properties of aainreal-life real-life application is challenging challenging (Psaraftis visual interface for solving dynamic VRPperformance under perturtakes into account all possible disturbances research by focusing on human control in properties of application is (Psaraftis bations. As such, it aims to go a step further than previous takes account all possible and stochastic bysuch, focusing ontohuman control performance in et al., al., into 2016). When not taking disturbances intoisaccount account all constraints constraints bations. As it aims go a step further than previous properties of aWhen real-life application challenging (Psaraftis research solving more complex TSP scenarios. A human-in-the-loop et 2016). not taking into all research by focusing on human control performance in properties of a real-life application is challenging (Psaraftis solving more complex TSP scenarios. A human-in-the-loop et al., 2016). When not taking into account all constraints inflicted by by these these disturbances disturbances and and properties, properties, aa theoretical theoretical research by focused focusing ona human control performance in experiment on multi-UAV (Unmanned Aerial inflicted solving more complex TSP scenarios. A human-in-the-loop et al., 2016). When not taking into account all constraints experiment focused on a multi-UAV (Unmanned Aerial inflicted by these disturbances and properties, a theoretical optimal solution will not transfer well to real-life opersolving more complex TSP scenarios. Avehicle human-in-the-loop experiment focused on a multi-UAV (Unmanned Aerial optimal solution will not transfer well to real-life operVehicle) payload delivery mission with failures. The inflicted by thesedue disturbances and properties, a VRP theoretical Vehicle) payload delivery with vehicle failures. The ations. Second, Second, tonot thetransfer complexity oftothe the opti- experiment focused on amission multi-UAV (Unmanned Aerial optimal solution willto well of real-life opertask of operator was perform real-time perturbation ations. due the complexity VRP operoptiVehicle) payload delivery mission with vehicle failures. The optimal solution will not transfer well to real-life of the the operator was to to perform real-time perturbation to current the complexity of the limitations, VRP opti- task mization problem and computational Vehicle) payload delivery mission with vehicle failures. The ations. Second, due management to ensure mission success. This application mization problem and current computational limitations, task of the operator was to perform real-time perturbation ations. Second, due to current the complexity ofathe VRP optimanagement to ensure mission success. Thisperturbation application mization problem and computational limitations, finding an optimal solution might take long time (in task of the operator was to perform real-time has been due the time pressure introfinding anproblem optimaland solution take a long time (in management to ensure success. application mization currentmight computational limitations, been chosen chosen due to to mission the inherent inherent timeThis pressure introfinding an of optimal solution might take days) a long time and (in has the hours or even (Toth management to ensure success. application has been chosen due to mission the inherent timeThis pressure introthe order order ofoptimal hours solution or sometimes sometimes even days) (Toth and duced with flight operations, where vehicles have limited finding an might take a long time (in duced with flight operations, where vehicles have limited Daniele, 2014). Although this might not be a problem for has been chosen due to the inherent time pressure introthe order of hours or sometimes even days) (Toth and payload and endurance due to battery capacity. Daniele, 2014). Although this might not be a problem for duced with flight operations, where vehicles have limited the order2014). ofschedules hours orwell sometimes even days) (Toth itand payload andflight endurance due towhere battery capacity. generating before mission execution, is duced with operations, vehicles have limited Daniele, Although this might not be a problem for generating schedules well this before mission is payload and endurance due to battery capacity. Daniele, 2014). Although might not beexecution, a problemit generating schedules well before mission execution, itfor is payload and endurance due to battery capacity. 1 E-mail: [email protected] generating schedules well before mission execution, it is 1
E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] 2405-8963 Copyright © 2019. The Authors. Published by Elsevier Ltd. All rights reserved. Copyright © 2019 IFAC 359 Copyright © under 2019 IFAC 359 Control. Peer review responsibility of International Federation of Automatic Copyright © 2019 IFAC 359 10.1016/j.ifacol.2019.12.088 Copyright © 2019 IFAC 359 1 1 1
2019 IFAC HMS 360 Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
2. THE VEHICLE ROUTING PROBLEM The generic family of VRPs can be defined as follows: “Given a set of transportation requests and a fleet of vehicles, determine a set of vehicle routes to perform all transportation requests with the given vehicle fleet at minimum cost; in particular, decide which vehicle handles which requests in which sequence so that all vehicle routes can be feasibly executed.” (Toth and Daniele, 2014) Various types of VRPs exist, each with its own specific set of constraints (Toth and Daniele, 2014). Under consideration for this research is the Distance-Constrained Capacitated Vehicle Routing Problem (DCVRP) with resource constraints at the depot, which, in addition to the generic VRP attributes, includes a distance constraint for each vehicle, fuel limitations, a capacity constraint, payload capacity, and a depot capacity limit (i.e., only a finite number of vehicles is allowed to depart and arrive at the depot simultaneously). More formally, the transportation requests in the DCVRP consist of the distribution of goods from a single depot, denoted as point 0, to customers, which are defined as a set of n other points, with N = {1, 2, ..., n}. Figure 1 depicts a graphical overview of the generic VRP. The customer demand, qi ≥ 0, is defined as the number of goods that needs to be delivered to the customer i ∈ N . The fleet of vehicles, defined as K = {1, 2, ..., |K|}, that is used to distribute the goods is assumed homogeneous. The homogeneity entails that the |K| vehicles in the fleet have identical distance constraints, capacity Q > 0 constraints, and associated cost. Each vehicle starts at the depot, delivers goods to a subset of customers S ⊂ N visiting customer locations only once, then returns to the depot. When traveling from customer i to customer j, the vehicle incurs the travel cost cij . Cost is assumed symmetric, where the cost of traveling from i to j is equal to the cost of traveling from j to i. A dynamic VRP is characterized by perturbations during execution of the original optimized plan, such that re-optimization is necessary (Ercan and Gencer, 2018). Sources of dynamism can include a change in customer demands, increases in travel time or distance due to required re-routes of vehicles and breakdowns of vehicles. In this article, the focus of dynamism is solely put on vehicle failures. In such situations, one or more vehicles can
Although many algorithms, both exact and (meta)heuristic, already exist to optimize and approximate solutions to dynamic VRPs, it is still a topic that is heavily researched (Ercan and Gencer, 2018; Braekers et al., 2016; Psaraftis et al., 2016). Not only does it draw attention because of its notorious difficulty as a combinatorial optimization problem, but also because of its practical relevance (Toth and Daniele, 2014). 3. INTERFACE DESIGN 3.1 Scope and Information Requirements In this paper the dynamic VRP under consideration corresponds to a DCVRP where the degree of dynamism is defined in terms of vehicle failure. Instead of developing an algorithm that deals with this situation, we focus on developing an interface that supports humans in solving it. The high-level constraints in the mission are: UAV payload capacity limits, the UAV flight time limit (due to limited battery capacity), and the depot capacity limit. In the current study, communication range (both related to ground station and UAV), airspace restrictions, vehicle separation requirements (with respect to both terrain and other vehicles), weather (such as wind), and UAV flight performance characteristics were left outside the scope. Given the mathematical formulation of the DCVRP and previous research on human control performance in traveling salesman problems, the following information would be critical to facilitate successful human control performance: 1) a map of geographical customer locations and their demands, 2) vehicle payload capacity, 3) vehicle routes and endurance (based on battery capacity), 4) temporal arrival schedule of the vehicles and 5) depot capacity limit. 3.2 Layout, Structure and Functionality In designing a visual interface for the dynamic DCVRP control problem considered in this study, inspiration was taken from research in Air Traffic Control that focused on aircraft spatio-temporal arrival management (De Wit et al., 2014) and en-route perturbation management (Klomp et al., 2015). Figure 2 provides a representation of the layout and structure of the designed direct manipulation interface for a simple scenario. The scenario consists of three UAVs delivering payload to six customers from a single depot location. The interface consists of three separate views, namely the map A , payload B , and timeline view C , where the map view presents information from a spatial perspective and the timeline view presents information from a temporal perspective. The red zone in the timeline view represents the depot capacity constraint.
breakdown, leaving a number of customers unserved. The remaining vehicles in the fleet should then be used to serve those customers and thus complete the overall mission, requiring each operative vehicle to have a surplus in payload. The consequence is that deviation from the original plan is required and a new optimal, but revised, plan must be found, all within the vehicle fuel limitations (affecting vehicle endurance) and depot capacity constraints.
Fig. 1. Graphical representation of the VRP. 360
2019 IFAC HMS Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
A
D1
1 D6 DEPOT
D2
D5
2 D3
D4
Payload Level nVehicles 3 3
2
2
1
1
0
C
B
DEPOT Arrivals
1
2
3
4 5 6 7 Arrival Time [min]
8
361
!
!
!
!
"
9
!
(a) t0 : Mission overview D1
D6
"
DEPOT
D2
D5
3
"
4
!
!
D3
2
2
1
1
D4
Payload Level nVehicles 3 3
DEPOT Arrivals
10
0
1
2
3
4 5 6 7 Arrival Time [min]
8
(b) t1 : First UAV launched
! " #
$
%
(c) t2 : Second UAV launched
! " #
$
%
(d) t3 : Third UAV launched
"
"
!
"
9
!
!
! " #
$
%
(e) t4 : UAV failure
! " #
$
%
(f) t5 : Select UAV
! " #
"
"
"
!
!
!
! " # $
%
(h) t7 : Add customer D1
! " #
$
%
(i) t8 : Confirmed flight plan
! " # $
#
!
"
"
! " # $
%
(k) t10 : Select flight plan leg
!
" # $ %
&
(�)t11 : Add customer D6
Fig. 2. Step-by-step overview of the interface for a simple scenario. 361
!
%
"
(j) t9 : Select UAV
&
%
#
$
(g) t6 : Select flight plan leg
!
! " # $
%
(m) t12 : Confirmed flight plan
2019 IFAC HMS 362 Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
At t0 (Figure 2a) the interface provides a mission overview by displaying customer 1 and depot 2 locations and the initial optimized flight plans 12 for each UAV. The flight plans of vehicles which have not yet left the depot are drawn using a dashed line to differentiate from the vehicles that already have. The first UAV is launched from the depot 4 and flies to its first customer 3 at time t1 (Figure 2b). The arrival time at the depot and corresponding service time is now also indicated 10 for this vehicle.
only are all customers served 16 , but this is also achieved using efficient routing of the vehicles 15 , while adhering to all applicable constraints.
The color of the UAV icon and the arrival time block correspond to the payload level of the vehicle, where bright yellow is used when all payload is available, dark yellow when payload capacity is reduced and amber is used when none is available. At t2 (Figure 2c), the second UAV is launched, of which its arrival time overlaps with the first UAV launched and hence exceeds the depot capacity. Therefore, both the arrival time block in the timeline view as well as the UAV icon in the map view are colored red to assist in activating the operator to identify the problem and take action. Also, the vehicle’s flight plan consists of waypoints 5 at the customer locations as well as any intermediate waypoints that the operator my include for path stretching purposes. The future maneuvers a UAV will make are depicted by the guidance reference 6 . The third and final UAV is launched at t3 (Figure 2d).
(1) Payload capacity (P): The payload capacity of a single UAV, serving as a metric for DCVRP problem size, consisted of four levels: 4, 5, 6 and 7 payload items. All vehicles in a scenario have the same payload capacity. (2) Perturbation severity (F): The perturbation severity dictates how many UAVs will fail in the scenario, consisting of two levels: single and double failure. All vehicle failures always occurred simultaneously after five seconds into the scenario.
At t4 (Figure 2e), one of the vehicles fails, disappearing from the interface and resulting in two customers not being served. One vehicle is selected at time t5 (Figure 2f), indicated by the green coloring of the UAV icon and the arrival time block. If a vehicle is selected, the payload window indicates the payload available 7 . An envelope 8 around the guidance reference indicates what locations can be reached given the vehicle’s battery status. At t6 (Figure 2g), a flight plan leg is selected and the corresponding flight time constraint is indicated 9 both in the map view and in the timeline view, where the vertical line indicates the maximum flight time. The red UAV icon and the arrival time overlap 10 indicate service time issues. The payload level of the vehicles in the fleet and the unvisited customers 11 yield information on the payload satisfaction. Customer D1 is included in the flight plan at t7 (Figure 2h), where the updated flight plan 12 is indicated with a dashed line and the arrival time is updated. At t8 (Figure 2i) the modified flight plan is confirmed and the UAV icon and arrival time block 11 changes color to amber in order to indicate no more payload is available after visiting all assigned customers. At t9 (Figure 2j), the other UAV has sufficient payload capacity 14 to cover the remaining customer and is selected. Once a flight plan leg is selected at time t10 (Figure 2k), in addition to the flight time constraint, the required delay to solve the depot arrival time overlap is visualized 13 . This combination gives an integrated overview of time management. At t11 (Figure 2�), the modified flight plan is shown. Also, it can be observed that all UAVs in the fleet have now used up their full payload capacity 14 . Finally, at t12 (Figure 2�) the flight plan is confirmed and the solution to the scenario is visible. Not 362
4. HUMAN-IN-THE-LOOP EXPERIMENT 4.1 Independent Variables The experiment design consisted of two within-participant independent variables, namely:
4.2 Scenarios Participants were asked to mitigate the effects caused by UAV failures during several multi-UAV payload delivery missions under the eight different experiment conditions, see Table 1. The vehicle failures resulted in unassigned customer locations and the task of the participant was to include the unassigned locations into the flight plans of the remaining vehicles, while satisfying all constraints (flight time, payload capacity and depot capacity). Table 2 lists the number of customer locations and the number of vehicles per condition, both of which are uniquely dictated by the payload capacity, number of vehicle failures, and payload margin. In Figure 3 two example scenarios are shown. All scenarios lasted six minutes. In every scenario, UAVs were deployed in batches from the depot every thirty seconds (equal to the depot service time), with the batch size equaling the depot capacity. Only lateral control was available, by means of flight plan waypoint modification. Any control actions taken by the participant could influence the solution space later in the scenario. Table 1. Experiment Conditions
1 Failure 2 Failures
Payload 4
Payload 5
Payload 6
Payload 7
F1P4 F2P4
F1P5 F2P5
F1P6 F2P6
F1P7 F2P7
Table 2. Customers and Vehicles per Condition
Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario
0: 1: 2: 3: 4: 5: 6: 7:
F1P4 F1P5 F1P6 F1P7 F2P4 F2P5 F2P6 F2P7
nCustomers
nVehicles
12 20 30 42 24 40 60 84
4 5 6 7 8 10 12 14
2019 IFAC HMS Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
363
Single Failure
Double Failure Infeasible Feasible
Number of Runs (-)
40 30 20 10 0
4
5 6 7 4 5 6 Payload Capacity (-)
7
Fig. 4. Total count of unsuccessful missions. Double Failure Run Time
300 200 100 0
4
5 6 7 4 5 6 Payload Capacity (-)
(a) Time to first solution
7
Single Failure Time to Last Solution (s)
Time to First Solution (s)
Single Failure 400
Double Failure Run Time
400 300 200 100 0
4
5 6 7 4 5 6 Payload Capacity (-)
7
(b) Time to last solution
Fig. 5. Box plot of the time to obtain the first and last solution per condition. 5. RESULTS 5.1 Mission success Fig. 3. Two example flight scenarios. The dotted lines indicate the UAVs that failed upon take-off. Table 3. Control Variables Variable Max Flight Time (s) Airspeed (m/s) Service Time (s) Scenario Duration (s) Failure Times (s) Payload Margin (-) Sector Size (m2 ) Depot Capacity (-)
Value 750 13 30 360 5 1 5000 x 5000 30% of nVehicles
Figure 4 shows a bar chart of the number of infeasible and feasible runs per condition. Infeasible runs were counted when participants were unable to meet any of the control goals, i.e., serving all customers, not overrunning the available flight time and not overrunning the depot capacity. The results show that in most cases the participants were able to solve the perturbed scenarios, even the more difficult ones. Surprisingly, the largest number of failed missions occurred in the scenarios featuring a single UAV failure and a relatively low payload capacity, such as condition F1P1 (scenario 0). 5.2 Solution time
All scenarios were created by using an off-line VRP optimization algorithm from Google Optimization Tools, which is a software suite for solving combinatorial optimization problems. Customer locations were generated using a random number generator, where a minimum distance criterion was implemented to prevent location clustering, and the Google toolset was used to calculate the initial optimized plan taking into account the scenario and fixed parameters listed in Table 3.
Figure 5 shows a box plot of the time to the first and last solution per condition. The solution times are defined as the amount of time between the start of the scenario and the moment the first and last feasible solution was achieved. All participants, having successful missions, could solve the scenarios within the run-time of 360 seconds and on average took about 150 seconds to solve the single failure scenarios and 200 seconds for the double failure scenarios. Observing the trends in the figure, it can be seen that for small problem sizes the time to first solution is relatively constant, whereas it increases for larger problem sizes. 5.3 Solution patterns
4.3 Dependent Measures The emphasis was put on measuring the number of successful missions, the time it took participants to solve the perturbed scenarios and comparing the human solution patterns with the solutions from a static DCVRP algorithm. 363
Figure 6 depicts the perturbed scenario, the optimized solution (calculated off-line with the Google VRP toolset) and two participant solutions for condition F2P7 (scenario 7). Due to the UAV flight time constraints, the conditions with few customers have a smaller solution space compared to conditions with many customers. Hence, participants
2019 IFAC HMS 364 Tallinn, Estonia, Sept. 16-19, 2019
(a) Perturbed scenario
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
(b) Optimized solution
(c) Participant 6 solution
(d) Participant 1 solution
Fig. 6. Selection of results for condition F2P7. generally either found one out of a small set of solutions, or were unable to solve the scenario. For this condition, participants with a good strategy were able to come up with solutions that are visually similar to the optimized solution, see Figure 6c, but rarely exactly the same because of the large solution space. Participants with a bad strategy, or participants who focused on satisficing over optimizing generally opted for solutions that visually look more chaotic, as depicted in Figure 6d. 6. CONCLUSION The goal of this study was to support human control performance in a dynamic multi-UAV DCVRP (featuring vehicle breakdowns) by means of a visual interface. Results show that the developed interface allowed human operators to effectively control perturbed DCVRPs across a range of problem sizes. Interestingly, the participants experienced most difficulty in solving scenarios featuring a single UAV breakdown and less customers. In contrast, computer algorithms generally experience most problems in solving larger problem sizes, as computational time can increase exponentially with the number of customers. For future research, the developed interface could facilitate human-automation cooperation, in which humans can quickly find an initial solution to a perturbed situation, after which an algorithm could further refine and optimize that solution. Further, and a more detailed, data analysis is needed to better compare human solutions with those from a dynamic DCVRP algorithm and explore the limits of human problem-solving activities in more realistic and constrained VRP (e.g., adding time windows to customers and vehicles with different battery capacities). REFERENCES Anderson, D., Anderson, E., Lesh, N., Marks, J., Mirtich, B., Ratajczak, D., and Ryall, K. (2000). Human-Guided Simple Search. In AAAI/IAAI, 209–216. Braekers, K., Ramaekers, K., and Van Nieuwenhuyse, I. (2016). The vehicle routing problem: State of the art classification and review. Computers & Industrial Engineering, 99, 300–313. De Wit, J., Tielrooij, M., Borst, C., Van Paassen, M.M., and Mulder, M. (2014). Supporting Runway Planning by Visualizing Capacity Balances of Arriving Aircraft Streams. In IEEE International Conference on Systems, Man, and Cybernetics, 3020–3025. San Diego, CA. Dry, M., Lee, M.D., Vickers, D., and Hughes, P. (2006). Human Performance on Visually Presented Traveling 364
Salesperson Problems with Varying Numbers of Nodes. The Journal of Problem Solving, 1(1), 20–32. Ercan, C. and Gencer, C. (2018). A Decision Support System for Dynamic Heterogeneous Unmanned Aerial System Fleets. Gazi University Journal of Science, 3(31), 863–877. Klomp, R.E., Borst, C., Van Paassen, M.M., and Mulder, M. (2015). Expertise Level, Control Strategies, and Robustness in Future Air Traffic Control Decision Aiding. IEEE Transactions on Human-Machine Systems, 46(2), 255–266. doi:10.1109/THMS.2015.2417535. Laporte, G. (1992). The vehicle routing problem: An overview of exact and approximate algorithms. European journal of operational research, 59(3), 345–358. Laporte, G. (2007). What You Should Know about the Vehicle Routing Problem. Naval Research Logistics (NRL), 54(8), 811–819. doi:10.1002/nav. Lau, H.C., Sim, M., and Teo, K.M. (2003). Vehicle routing problem with time windows and a limited number of vehicles. European journal of operational research, 148(3), 559–569. doi:10.1016/S0377-2217(02)00363-6. Lin, C., Choy, K.L., Ho, G.T.S., Chung, S.H., and Lam, H.Y. (2014). Survey of Green Vehicle Routing Problem: Past and future trends. Expert Systms with Appications, 41(1), 1118–1138. doi:10.1016/j.eswa.2013.07.107. Macgregor, J.N. and Chu, Y. (2011). Human Performance on the Traveling Salesman and Related Problems. The Journal of Problem Solving, 3(2), 1–29. Pillac, V., Gendreau, M., Gu´eret, C., and Medaglia, A.L. (2013). A review of dynamic vehicle routing problems. European Journal of Operational Research, 225(1), 1–11. Psaraftis, H.N., Wen, M., and Kontovas, C.A. (2016). Dynamic Vehicle Routing Problems: Three Decades and Counting. Networks, 67(1), 3–31. doi:10.1002/net. Scott, S.D., Lesh, N., and Klau, G.W. (2002). Investigating Human-Computer Optimization. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 155–162. Toth, P. and Daniele, V. (2014). Vehicle routing: problems, methods, and applications. Society for Industrial and Applied Mathematics. T¨ ut¨ unc¨ u, G.Y., Carreto, C.A.C., and Baker, B.M. (2009). A visual interactive approach to classical and mixed vehicle routing problems with backhauls. Omega, 37(1), 138–154. doi:10.1016/j.omega.2006.11.001. Vickers, D., Butavicius, M., Lee, M., and Medvedev, A. (2001). Human performance on visually presented Traveling Salesman problems. Psychological Research, 65(1), 34–45.
ScienceDirect
IFAC PapersOnLine 52-19 (2019) 359–364
Supporting Humans in Solving Multi-UAV Supporting Humans in Solving Multi-UAV Supporting Humans in Solving Multi-UAV Dynamic Vehicle Routing Problems Supporting Humans in Solving Multi-UAV Dynamic Vehicle Routing Problems Dynamic Vehicle Routing Problems Dynamic Vehicle Routing Problems N. W. Klein Koerkamp C. Borst 11 Max Mulder
N. W. Klein Koerkamp C. Borst 111 Max Mulder M. Paassen N. W. Klein Koerkamp Borst 1 Max Mulder M. van vanC. Paassen M. M. N. W. Klein Koerkamp Borst Max Mulder M. M. vanC. Paassen M. M. van Paassen Control Control and and Simulation, Simulation, Faculty Faculty of of Aerospace Aerospace Engineering, Engineering, TU TU Delft, Delft, Control and Simulation, Faculty ofThe Aerospace Engineering, TU Delft, 2629 HS, Delft, Netherlands 2629 HS, Delft, The Netherlands Control and Simulation, Faculty of Aerospace Engineering, TU Delft, 2629 HS, Delft, The Netherlands 2629 HS, Delft, The Netherlands Abstract: Abstract: Real-time Real-time optimization optimization of of Vehicle Vehicle Routing Routing Problems Problems (VRP) (VRP) during during mission mission operaoperaAbstract: Real-time optimization of Vehicle Routing Problems (VRP) during mission operations raises concerns regarding obtaining a solution within a reasonable timeframe, especially tions raises concerns regarding obtaining a solution within a reasonable timeframe, especially Abstract: optimization of easily Vehicle Problems (VRP) during mission operations raises Real-time concerns regarding obtaining a solution within reasonable timeframe, especially in domains domains where operations cannot beRouting paused and athe the number of control parameters in where operations cannot easily be paused and number of control parameters tions raises concerns regarding obtaining a solution within a reasonable timeframe, especially in domains where operations cannot easily be paused and the number of control parameters is high. high. Humans, Humans, however, however, are are heuristic heuristic problem problem solvers solvers and and could could potentially potentially complement complement is in domains whereinhowever, operations cannot easily be paused andsolution the could number of control parameters is high. Humans, are heuristic problem solvers and potentially complement VRP algorithms providing quickly aa workable and safe from which the algorithms VRP algorithms in providing quickly workable and safe solution from which the algorithms is high. Humans, however, are heuristic problem solvers and could potentially complement VRP algorithms in providing quickly a study, workable and safe solution from which the algorithms can further find the optimum. In this a visual interface was developed and evaluated can further find the optimum. In this study, a visual interface was developed and evaluated VRP algorithms in providing quickly workable and safe solution whichthey the can further find the optimum. In thisa study, a visual interface wasfrom developed andalgorithms evaluated aiming to support humans in manually solving a dynamic VRP in which needed to aiming to support in In manually solving a dynamic VRP indeveloped which they to can further find thehumans optimum. this study, a visual interface wasUnmanned and needed evaluated aiming to support humans in delivery manually solving a dynamic VRP in which they needed to solve various simulated payload missions, featuring multiple Aerial Vehicles, solve various simulated payload missions, featuring multiple Unmanned Aerialneeded Vehicles, aiming to support humans in delivery manually solving a16) dynamic VRP in they to solve simulated payload delivery missions, multiple Aerial Vehicles, undervarious failure conditions. Experiment results (n = =featuring indicate thatUnmanned thewhich interface enabled the under failure conditions. Experiment results (n 16) indicate that the interface enabled the solve various simulated payload delivery missions, featuring multiple Unmanned Aerial Vehicles, under failure conditions. to Experiment results = 16) indicate that the interface enabled majority of participants participants quickly solve solve the (n perturbed scenarios, although not always always in the the majority of to quickly the perturbed scenarios, although not in under failure conditions. Experiment results (n = 16) indicate that the interface enabled the majority of participants to quickly solve theexperienced perturbed scenarios, although not always in the most efficient way. Interestingly, participants most difficulty in solving the seemingly most efficient way. Interestingly, participants most difficulty in solving the seemingly majority of participants toless quickly solve and theexperienced perturbed scenarios, although not always in the most efficient way.featuring Interestingly, participants experienced most difficulty in vehicles solving the seemingly easier scenarios, customers a relatively low number of compared to easierefficient scenarios, less customers andexperienced a relatively low difficulty number of compared to most way.featuring Interestingly, participants in vehicles solving the seemingly easier scenarios, featuring less customers and a relativelymost low number of vehicles compared to the more complex scenarios. the more complexfeaturing scenarios. easier scenarios, less customers and a relatively low number of vehicles compared to the more complex scenarios. Copyright © 2019. The Authors. Published by Elsevier Ltd. All rights reserved. the more complex scenarios. Keywords: Keywords: Human-machine Human-machine interface, interface, vehicle vehicle routing routing problem, problem, unmanned unmanned aerial aerial vehicles. vehicles. Keywords: Human-machine interface, vehicle routing problem, unmanned aerial vehicles. Keywords: Human-machine interface, vehicle routing problem, unmanned aerial vehicles. 1. INTRODUCTION INTRODUCTION aa concern concern when when aa need need for for re-optimization re-optimization arises arises due due to to 1. 1. INTRODUCTION aperturbations concern when a need for re-optimization arises to perturbations during mission operations. Third, for some during mission operations. Third, fordue some 1. INTRODUCTION a concern when a need for re-optimization arises due to scenarios there might not even be a solution. For example, perturbations during mission operations. Third, for some Vehicle routing routing problems problems (VRP) (VRP) are are at at the the core core of of many many scenarios there might not even be a solution. For example, Vehicle perturbations during mission operations. Third, for some for over-constrained VRP algorithms be there mightproblems, not even be a solution. For might example, Vehicle routing problems (VRP) are at the core ofjust-inmany scenarios logistics applications. With the focus on for over-constrained problems, VRP algorithms might be logisticsrouting applications. With the current current focus onof just-inscenarios there might not even be a solution. For example, Vehicle problems (VRP) are at the core many unable to find a solution at all, without first performing for over-constrained problems, VRP algorithms might be time logistics and data-driven analysis techniques, costlogistics applications. With the current focus on just-inunable to find a solution at all, without first performing time logistics and data-driven analysis techniques, costfor over-constrained problems, algorithms might be logistics applications. Withofthe current focus on just-inunable to find a solution at all,VRP without first performing some kind of constraint relaxation (Lau et al., 2003). efficient routing of a fleet vehicles plays an important time logistics and data-driven analysis techniques, costsome kind of constraint relaxation (Lau et al., 2003). efficient routing of data-driven a fleet of vehicles plays an important to find a solutionrelaxation at all, without time logistics and techniques, cost- unable some kind ofway constraint (Lau etfirst al., performing 2003). efficient routing of a fleet(Lin of vehicles plays an important role in in many industries etanalysis al., 2014). 2014). Determining A possible to overcome these challenges is role many industries (Lin et al., Determining some kind of constraint relaxation (Lau et al., 2003). efficient routing of a fleet of vehicles plays an important A possible way to overcome these challenges is by by havhavrole in many industries (Lin etinal., 2014). Determining efficient routes for each vehicle the fleet, such that the A possible way to overcome these challenges is by havhumans solving these problems as they are creative efficient routesindustries for each vehicle inal., the 2014). fleet, such that the ing role in many (Lin et Determining ing humans solving these problems as they are creative overall mission goal is achieved within distance, capacity A possible way to overcome these challenges is by havefficient routes for each vehicle in the fleet, such that the problem solvers who can adapt to novel situations. Preoverall mission goal is achieved within distance, capacity ing humans solving these problems as they are creative efficient routes for each vehiclethe in the fleet, such capacity that the problem solvers who these can adapt to novel situations. Preand time constraints, defines VRP (Laporte, 2007). ing humans solving problems as they are creative overall mission goal is achieved within distance, vious research on human control performance in solving and time constraints, defines the VRP (Laporte, 2007). problem solvers who can adapt to novel situations. Preoverall mission goal is defines achieved within distance, capacity research on human control performance in solving and time constraints, the VRP (Laporte, 2007). vious Although much attention is given in literature to obtaining problem solvers who can adapt toperformance novel situations. PreTraveling Salesman Problems (TSP) shows good human Although much attention is given inVRP literature to obtaining vious research on human control in solving and time constraints, defines the (Laporte, 2007). Traveling Salesman Problems (TSP) shows good human routes for formuch staticattention problems, in many many real-lifeto situations situations research on human control performance in solving Although is given in literature obtaining vious Traveling Salesman Problems (TSP) shows good human performance in creating optimal routes purely based on routes static problems, in real-life Although attention is given in literature obtaining in creating optimal routesshows purely based on a a in due many real-lifeto situations these problems problems become dynamic to changing changing customer performance Traveling Salesman Problems (TSP) good human routes formuch static problems, visual representation of the customer locations and vehicle these become dynamic due to customer performance in creating optimal routes purely based on a routes for static problems, in many real-life situations visual representation of the customer locations and vehicle locations, vehicle failures or service travel in creating optimal routes purely 2001; based on a these problems become dynamic due to changing customer routes representation (Anderson 2000; Vickers et Scott locations, vehicle failures or uncertain uncertain service and and travel performance visual of the customer locations and vehicle these problems become dynamic due to mission changing customer (Anderson et et al., al., 2000; Vickers et al., al., 2001; Scott locations, vehicle failures or uncertain service operations and travel routes times caused by perturbations during visual representation of the customer locations and vehicle routes (Anderson et al., 2000; Vickers et al., 2001; Scott times caused by perturbations during mission operations Dry 2006; T¨ u u u et al., 2009; Maclocations, vehicle failures or 2013). uncertain service operations and travel et et al., al., 2002; 2002; Dry et etetal., al., 2006; T¨ ut¨ t¨ unc¨ nc¨ u et et al., al., 2001; 2009; Scott Mac(Laporte, 1992; Pillac et al., al., routes (Anderson al.,By 2000; Vickers times caused byPillac perturbations during mission et al., 2002; Dry 2011). et al., 2006; T¨ ut¨ unc¨ uthese et al., 2009; visual Macgregor and Chu, leveraging human (Laporte, 1992; et 2013). times caused by perturbations during mission operations gregor and Chu, 2011). By leveraging these human visual et al., 2002; Dry et al., 2006; T¨ u t¨ u nc¨ u et al., 2009; Mac(Laporte, 1992; Pillac et al., 2013). pattern recognition qualities, good performance in solving gregor and Chu, 2011). By leveraging these human visual In solving dynamic VRPs, various challenges can be identi(Laporte, 1992; Pillac et al., 2013). recognition qualities, good performance in solving In solving dynamic VRPs, various challenges can be identi- pattern gregor and Chu, 2011). By leveraging these human visual real-time dynamic might be achieved. recognition qualities, good in solving VRPs, various challenges fied. First,dynamic the optimization optimization problem needs to tocan be be explicitly In solving identi- pattern real-time dynamic VRPs VRPs might be performance achieved. fied. First, the problem needs be explicitly pattern recognition qualities, good in solving In solving dynamic VRPs, various challenges can be identireal-time dynamic VRPs might be performance achieved. formulated. Although a wide range of VRP types has fied. First, the optimization problem needs to be explicitly This study comprises both the design and evaluation of formulated. Although a wide rangeneeds of VRP types has real-time dynamic VRPs might be achieved. fied. First, the optimization problem to be explicitly This study comprises both the design and evaluation of a a been studied studiedAlthough in literature, literature, constructing anVRP algorithm that formulated. a wide range ofan types that has This visual interface for solving a dynamic VRP under perturbeen in constructing algorithm study comprises both the design and evaluation of a formulated. Although a wide range of VRP types has visual interface for solving a dynamic VRP under perturbeen studied in literature, constructing an algorithm that takes into account all possible disturbances and stochastic This study comprises both ago thea design and evaluation of a visual interface VRP under perturtakes studied into account all possible disturbances and stochastic As it aims than been literature, constructing an algorithm that bations. bations. As such, such,for it solving aims to to agodynamic a step step further further than previous previous and stochastic properties of aainreal-life real-life application is challenging challenging (Psaraftis visual interface for solving dynamic VRPperformance under perturtakes into account all possible disturbances research by focusing on human control in properties of application is (Psaraftis bations. As such, it aims to go a step further than previous takes account all possible and stochastic bysuch, focusing ontohuman control performance in et al., al., into 2016). When not taking disturbances intoisaccount account all constraints constraints bations. As it aims go a step further than previous properties of aWhen real-life application challenging (Psaraftis research solving more complex TSP scenarios. A human-in-the-loop et 2016). not taking into all research by focusing on human control performance in properties of a real-life application is challenging (Psaraftis solving more complex TSP scenarios. A human-in-the-loop et al., 2016). When not taking into account all constraints inflicted by by these these disturbances disturbances and and properties, properties, aa theoretical theoretical research by focused focusing ona human control performance in experiment on multi-UAV (Unmanned Aerial inflicted solving more complex TSP scenarios. A human-in-the-loop et al., 2016). When not taking into account all constraints experiment focused on a multi-UAV (Unmanned Aerial inflicted by these disturbances and properties, a theoretical optimal solution will not transfer well to real-life opersolving more complex TSP scenarios. Avehicle human-in-the-loop experiment focused on a multi-UAV (Unmanned Aerial optimal solution will not transfer well to real-life operVehicle) payload delivery mission with failures. The inflicted by thesedue disturbances and properties, a VRP theoretical Vehicle) payload delivery with vehicle failures. The ations. Second, Second, tonot thetransfer complexity oftothe the opti- experiment focused on amission multi-UAV (Unmanned Aerial optimal solution willto well of real-life opertask of operator was perform real-time perturbation ations. due the complexity VRP operoptiVehicle) payload delivery mission with vehicle failures. The optimal solution will not transfer well to real-life of the the operator was to to perform real-time perturbation to current the complexity of the limitations, VRP opti- task mization problem and computational Vehicle) payload delivery mission with vehicle failures. The ations. Second, due management to ensure mission success. This application mization problem and current computational limitations, task of the operator was to perform real-time perturbation ations. Second, due to current the complexity ofathe VRP optimanagement to ensure mission success. Thisperturbation application mization problem and computational limitations, finding an optimal solution might take long time (in task of the operator was to perform real-time has been due the time pressure introfinding anproblem optimaland solution take a long time (in management to ensure success. application mization currentmight computational limitations, been chosen chosen due to to mission the inherent inherent timeThis pressure introfinding an of optimal solution might take days) a long time and (in has the hours or even (Toth management to ensure success. application has been chosen due to mission the inherent timeThis pressure introthe order order ofoptimal hours solution or sometimes sometimes even days) (Toth and duced with flight operations, where vehicles have limited finding an might take a long time (in duced with flight operations, where vehicles have limited Daniele, 2014). Although this might not be a problem for has been chosen due to the inherent time pressure introthe order of hours or sometimes even days) (Toth and payload and endurance due to battery capacity. Daniele, 2014). Although this might not be a problem for duced with flight operations, where vehicles have limited the order2014). ofschedules hours orwell sometimes even days) (Toth itand payload andflight endurance due towhere battery capacity. generating before mission execution, is duced with operations, vehicles have limited Daniele, Although this might not be a problem for generating schedules well this before mission is payload and endurance due to battery capacity. Daniele, 2014). Although might not beexecution, a problemit generating schedules well before mission execution, itfor is payload and endurance due to battery capacity. 1 E-mail: [email protected] generating schedules well before mission execution, it is 1
E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] 2405-8963 Copyright © 2019. The Authors. Published by Elsevier Ltd. All rights reserved. Copyright © 2019 IFAC 359 Copyright © under 2019 IFAC 359 Control. Peer review responsibility of International Federation of Automatic Copyright © 2019 IFAC 359 10.1016/j.ifacol.2019.12.088 Copyright © 2019 IFAC 359 1 1 1
2019 IFAC HMS 360 Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
2. THE VEHICLE ROUTING PROBLEM The generic family of VRPs can be defined as follows: “Given a set of transportation requests and a fleet of vehicles, determine a set of vehicle routes to perform all transportation requests with the given vehicle fleet at minimum cost; in particular, decide which vehicle handles which requests in which sequence so that all vehicle routes can be feasibly executed.” (Toth and Daniele, 2014) Various types of VRPs exist, each with its own specific set of constraints (Toth and Daniele, 2014). Under consideration for this research is the Distance-Constrained Capacitated Vehicle Routing Problem (DCVRP) with resource constraints at the depot, which, in addition to the generic VRP attributes, includes a distance constraint for each vehicle, fuel limitations, a capacity constraint, payload capacity, and a depot capacity limit (i.e., only a finite number of vehicles is allowed to depart and arrive at the depot simultaneously). More formally, the transportation requests in the DCVRP consist of the distribution of goods from a single depot, denoted as point 0, to customers, which are defined as a set of n other points, with N = {1, 2, ..., n}. Figure 1 depicts a graphical overview of the generic VRP. The customer demand, qi ≥ 0, is defined as the number of goods that needs to be delivered to the customer i ∈ N . The fleet of vehicles, defined as K = {1, 2, ..., |K|}, that is used to distribute the goods is assumed homogeneous. The homogeneity entails that the |K| vehicles in the fleet have identical distance constraints, capacity Q > 0 constraints, and associated cost. Each vehicle starts at the depot, delivers goods to a subset of customers S ⊂ N visiting customer locations only once, then returns to the depot. When traveling from customer i to customer j, the vehicle incurs the travel cost cij . Cost is assumed symmetric, where the cost of traveling from i to j is equal to the cost of traveling from j to i. A dynamic VRP is characterized by perturbations during execution of the original optimized plan, such that re-optimization is necessary (Ercan and Gencer, 2018). Sources of dynamism can include a change in customer demands, increases in travel time or distance due to required re-routes of vehicles and breakdowns of vehicles. In this article, the focus of dynamism is solely put on vehicle failures. In such situations, one or more vehicles can
Although many algorithms, both exact and (meta)heuristic, already exist to optimize and approximate solutions to dynamic VRPs, it is still a topic that is heavily researched (Ercan and Gencer, 2018; Braekers et al., 2016; Psaraftis et al., 2016). Not only does it draw attention because of its notorious difficulty as a combinatorial optimization problem, but also because of its practical relevance (Toth and Daniele, 2014). 3. INTERFACE DESIGN 3.1 Scope and Information Requirements In this paper the dynamic VRP under consideration corresponds to a DCVRP where the degree of dynamism is defined in terms of vehicle failure. Instead of developing an algorithm that deals with this situation, we focus on developing an interface that supports humans in solving it. The high-level constraints in the mission are: UAV payload capacity limits, the UAV flight time limit (due to limited battery capacity), and the depot capacity limit. In the current study, communication range (both related to ground station and UAV), airspace restrictions, vehicle separation requirements (with respect to both terrain and other vehicles), weather (such as wind), and UAV flight performance characteristics were left outside the scope. Given the mathematical formulation of the DCVRP and previous research on human control performance in traveling salesman problems, the following information would be critical to facilitate successful human control performance: 1) a map of geographical customer locations and their demands, 2) vehicle payload capacity, 3) vehicle routes and endurance (based on battery capacity), 4) temporal arrival schedule of the vehicles and 5) depot capacity limit. 3.2 Layout, Structure and Functionality In designing a visual interface for the dynamic DCVRP control problem considered in this study, inspiration was taken from research in Air Traffic Control that focused on aircraft spatio-temporal arrival management (De Wit et al., 2014) and en-route perturbation management (Klomp et al., 2015). Figure 2 provides a representation of the layout and structure of the designed direct manipulation interface for a simple scenario. The scenario consists of three UAVs delivering payload to six customers from a single depot location. The interface consists of three separate views, namely the map A , payload B , and timeline view C , where the map view presents information from a spatial perspective and the timeline view presents information from a temporal perspective. The red zone in the timeline view represents the depot capacity constraint.
breakdown, leaving a number of customers unserved. The remaining vehicles in the fleet should then be used to serve those customers and thus complete the overall mission, requiring each operative vehicle to have a surplus in payload. The consequence is that deviation from the original plan is required and a new optimal, but revised, plan must be found, all within the vehicle fuel limitations (affecting vehicle endurance) and depot capacity constraints.
Fig. 1. Graphical representation of the VRP. 360
2019 IFAC HMS Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
A
D1
1 D6 DEPOT
D2
D5
2 D3
D4
Payload Level nVehicles 3 3
2
2
1
1
0
C
B
DEPOT Arrivals
1
2
3
4 5 6 7 Arrival Time [min]
8
361
!
!
!
!
"
9
!
(a) t0 : Mission overview D1
D6
"
DEPOT
D2
D5
3
"
4
!
!
D3
2
2
1
1
D4
Payload Level nVehicles 3 3
DEPOT Arrivals
10
0
1
2
3
4 5 6 7 Arrival Time [min]
8
(b) t1 : First UAV launched
! " #
$
%
(c) t2 : Second UAV launched
! " #
$
%
(d) t3 : Third UAV launched
"
"
!
"
9
!
!
! " #
$
%
(e) t4 : UAV failure
! " #
$
%
(f) t5 : Select UAV
! " #
"
"
"
!
!
!
! " # $
%
(h) t7 : Add customer D1
! " #
$
%
(i) t8 : Confirmed flight plan
! " # $
#
!
"
"
! " # $
%
(k) t10 : Select flight plan leg
!
" # $ %
&
(�)t11 : Add customer D6
Fig. 2. Step-by-step overview of the interface for a simple scenario. 361
!
%
"
(j) t9 : Select UAV
&
%
#
$
(g) t6 : Select flight plan leg
!
! " # $
%
(m) t12 : Confirmed flight plan
2019 IFAC HMS 362 Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
At t0 (Figure 2a) the interface provides a mission overview by displaying customer 1 and depot 2 locations and the initial optimized flight plans 12 for each UAV. The flight plans of vehicles which have not yet left the depot are drawn using a dashed line to differentiate from the vehicles that already have. The first UAV is launched from the depot 4 and flies to its first customer 3 at time t1 (Figure 2b). The arrival time at the depot and corresponding service time is now also indicated 10 for this vehicle.
only are all customers served 16 , but this is also achieved using efficient routing of the vehicles 15 , while adhering to all applicable constraints.
The color of the UAV icon and the arrival time block correspond to the payload level of the vehicle, where bright yellow is used when all payload is available, dark yellow when payload capacity is reduced and amber is used when none is available. At t2 (Figure 2c), the second UAV is launched, of which its arrival time overlaps with the first UAV launched and hence exceeds the depot capacity. Therefore, both the arrival time block in the timeline view as well as the UAV icon in the map view are colored red to assist in activating the operator to identify the problem and take action. Also, the vehicle’s flight plan consists of waypoints 5 at the customer locations as well as any intermediate waypoints that the operator my include for path stretching purposes. The future maneuvers a UAV will make are depicted by the guidance reference 6 . The third and final UAV is launched at t3 (Figure 2d).
(1) Payload capacity (P): The payload capacity of a single UAV, serving as a metric for DCVRP problem size, consisted of four levels: 4, 5, 6 and 7 payload items. All vehicles in a scenario have the same payload capacity. (2) Perturbation severity (F): The perturbation severity dictates how many UAVs will fail in the scenario, consisting of two levels: single and double failure. All vehicle failures always occurred simultaneously after five seconds into the scenario.
At t4 (Figure 2e), one of the vehicles fails, disappearing from the interface and resulting in two customers not being served. One vehicle is selected at time t5 (Figure 2f), indicated by the green coloring of the UAV icon and the arrival time block. If a vehicle is selected, the payload window indicates the payload available 7 . An envelope 8 around the guidance reference indicates what locations can be reached given the vehicle’s battery status. At t6 (Figure 2g), a flight plan leg is selected and the corresponding flight time constraint is indicated 9 both in the map view and in the timeline view, where the vertical line indicates the maximum flight time. The red UAV icon and the arrival time overlap 10 indicate service time issues. The payload level of the vehicles in the fleet and the unvisited customers 11 yield information on the payload satisfaction. Customer D1 is included in the flight plan at t7 (Figure 2h), where the updated flight plan 12 is indicated with a dashed line and the arrival time is updated. At t8 (Figure 2i) the modified flight plan is confirmed and the UAV icon and arrival time block 11 changes color to amber in order to indicate no more payload is available after visiting all assigned customers. At t9 (Figure 2j), the other UAV has sufficient payload capacity 14 to cover the remaining customer and is selected. Once a flight plan leg is selected at time t10 (Figure 2k), in addition to the flight time constraint, the required delay to solve the depot arrival time overlap is visualized 13 . This combination gives an integrated overview of time management. At t11 (Figure 2�), the modified flight plan is shown. Also, it can be observed that all UAVs in the fleet have now used up their full payload capacity 14 . Finally, at t12 (Figure 2�) the flight plan is confirmed and the solution to the scenario is visible. Not 362
4. HUMAN-IN-THE-LOOP EXPERIMENT 4.1 Independent Variables The experiment design consisted of two within-participant independent variables, namely:
4.2 Scenarios Participants were asked to mitigate the effects caused by UAV failures during several multi-UAV payload delivery missions under the eight different experiment conditions, see Table 1. The vehicle failures resulted in unassigned customer locations and the task of the participant was to include the unassigned locations into the flight plans of the remaining vehicles, while satisfying all constraints (flight time, payload capacity and depot capacity). Table 2 lists the number of customer locations and the number of vehicles per condition, both of which are uniquely dictated by the payload capacity, number of vehicle failures, and payload margin. In Figure 3 two example scenarios are shown. All scenarios lasted six minutes. In every scenario, UAVs were deployed in batches from the depot every thirty seconds (equal to the depot service time), with the batch size equaling the depot capacity. Only lateral control was available, by means of flight plan waypoint modification. Any control actions taken by the participant could influence the solution space later in the scenario. Table 1. Experiment Conditions
1 Failure 2 Failures
Payload 4
Payload 5
Payload 6
Payload 7
F1P4 F2P4
F1P5 F2P5
F1P6 F2P6
F1P7 F2P7
Table 2. Customers and Vehicles per Condition
Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario
0: 1: 2: 3: 4: 5: 6: 7:
F1P4 F1P5 F1P6 F1P7 F2P4 F2P5 F2P6 F2P7
nCustomers
nVehicles
12 20 30 42 24 40 60 84
4 5 6 7 8 10 12 14
2019 IFAC HMS Tallinn, Estonia, Sept. 16-19, 2019
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
363
Single Failure
Double Failure Infeasible Feasible
Number of Runs (-)
40 30 20 10 0
4
5 6 7 4 5 6 Payload Capacity (-)
7
Fig. 4. Total count of unsuccessful missions. Double Failure Run Time
300 200 100 0
4
5 6 7 4 5 6 Payload Capacity (-)
(a) Time to first solution
7
Single Failure Time to Last Solution (s)
Time to First Solution (s)
Single Failure 400
Double Failure Run Time
400 300 200 100 0
4
5 6 7 4 5 6 Payload Capacity (-)
7
(b) Time to last solution
Fig. 5. Box plot of the time to obtain the first and last solution per condition. 5. RESULTS 5.1 Mission success Fig. 3. Two example flight scenarios. The dotted lines indicate the UAVs that failed upon take-off. Table 3. Control Variables Variable Max Flight Time (s) Airspeed (m/s) Service Time (s) Scenario Duration (s) Failure Times (s) Payload Margin (-) Sector Size (m2 ) Depot Capacity (-)
Value 750 13 30 360 5 1 5000 x 5000 30% of nVehicles
Figure 4 shows a bar chart of the number of infeasible and feasible runs per condition. Infeasible runs were counted when participants were unable to meet any of the control goals, i.e., serving all customers, not overrunning the available flight time and not overrunning the depot capacity. The results show that in most cases the participants were able to solve the perturbed scenarios, even the more difficult ones. Surprisingly, the largest number of failed missions occurred in the scenarios featuring a single UAV failure and a relatively low payload capacity, such as condition F1P1 (scenario 0). 5.2 Solution time
All scenarios were created by using an off-line VRP optimization algorithm from Google Optimization Tools, which is a software suite for solving combinatorial optimization problems. Customer locations were generated using a random number generator, where a minimum distance criterion was implemented to prevent location clustering, and the Google toolset was used to calculate the initial optimized plan taking into account the scenario and fixed parameters listed in Table 3.
Figure 5 shows a box plot of the time to the first and last solution per condition. The solution times are defined as the amount of time between the start of the scenario and the moment the first and last feasible solution was achieved. All participants, having successful missions, could solve the scenarios within the run-time of 360 seconds and on average took about 150 seconds to solve the single failure scenarios and 200 seconds for the double failure scenarios. Observing the trends in the figure, it can be seen that for small problem sizes the time to first solution is relatively constant, whereas it increases for larger problem sizes. 5.3 Solution patterns
4.3 Dependent Measures The emphasis was put on measuring the number of successful missions, the time it took participants to solve the perturbed scenarios and comparing the human solution patterns with the solutions from a static DCVRP algorithm. 363
Figure 6 depicts the perturbed scenario, the optimized solution (calculated off-line with the Google VRP toolset) and two participant solutions for condition F2P7 (scenario 7). Due to the UAV flight time constraints, the conditions with few customers have a smaller solution space compared to conditions with many customers. Hence, participants
2019 IFAC HMS 364 Tallinn, Estonia, Sept. 16-19, 2019
(a) Perturbed scenario
N.W. Klein Koerkamp et al. / IFAC PapersOnLine 52-19 (2019) 359–364
(b) Optimized solution
(c) Participant 6 solution
(d) Participant 1 solution
Fig. 6. Selection of results for condition F2P7. generally either found one out of a small set of solutions, or were unable to solve the scenario. For this condition, participants with a good strategy were able to come up with solutions that are visually similar to the optimized solution, see Figure 6c, but rarely exactly the same because of the large solution space. Participants with a bad strategy, or participants who focused on satisficing over optimizing generally opted for solutions that visually look more chaotic, as depicted in Figure 6d. 6. CONCLUSION The goal of this study was to support human control performance in a dynamic multi-UAV DCVRP (featuring vehicle breakdowns) by means of a visual interface. Results show that the developed interface allowed human operators to effectively control perturbed DCVRPs across a range of problem sizes. Interestingly, the participants experienced most difficulty in solving scenarios featuring a single UAV breakdown and less customers. In contrast, computer algorithms generally experience most problems in solving larger problem sizes, as computational time can increase exponentially with the number of customers. For future research, the developed interface could facilitate human-automation cooperation, in which humans can quickly find an initial solution to a perturbed situation, after which an algorithm could further refine and optimize that solution. Further, and a more detailed, data analysis is needed to better compare human solutions with those from a dynamic DCVRP algorithm and explore the limits of human problem-solving activities in more realistic and constrained VRP (e.g., adding time windows to customers and vehicles with different battery capacities). REFERENCES Anderson, D., Anderson, E., Lesh, N., Marks, J., Mirtich, B., Ratajczak, D., and Ryall, K. (2000). Human-Guided Simple Search. In AAAI/IAAI, 209–216. Braekers, K., Ramaekers, K., and Van Nieuwenhuyse, I. (2016). The vehicle routing problem: State of the art classification and review. Computers & Industrial Engineering, 99, 300–313. De Wit, J., Tielrooij, M., Borst, C., Van Paassen, M.M., and Mulder, M. (2014). Supporting Runway Planning by Visualizing Capacity Balances of Arriving Aircraft Streams. In IEEE International Conference on Systems, Man, and Cybernetics, 3020–3025. San Diego, CA. Dry, M., Lee, M.D., Vickers, D., and Hughes, P. (2006). Human Performance on Visually Presented Traveling 364
Salesperson Problems with Varying Numbers of Nodes. The Journal of Problem Solving, 1(1), 20–32. Ercan, C. and Gencer, C. (2018). A Decision Support System for Dynamic Heterogeneous Unmanned Aerial System Fleets. Gazi University Journal of Science, 3(31), 863–877. Klomp, R.E., Borst, C., Van Paassen, M.M., and Mulder, M. (2015). Expertise Level, Control Strategies, and Robustness in Future Air Traffic Control Decision Aiding. IEEE Transactions on Human-Machine Systems, 46(2), 255–266. doi:10.1109/THMS.2015.2417535. Laporte, G. (1992). The vehicle routing problem: An overview of exact and approximate algorithms. European journal of operational research, 59(3), 345–358. Laporte, G. (2007). What You Should Know about the Vehicle Routing Problem. Naval Research Logistics (NRL), 54(8), 811–819. doi:10.1002/nav. Lau, H.C., Sim, M., and Teo, K.M. (2003). Vehicle routing problem with time windows and a limited number of vehicles. European journal of operational research, 148(3), 559–569. doi:10.1016/S0377-2217(02)00363-6. Lin, C., Choy, K.L., Ho, G.T.S., Chung, S.H., and Lam, H.Y. (2014). Survey of Green Vehicle Routing Problem: Past and future trends. Expert Systms with Appications, 41(1), 1118–1138. doi:10.1016/j.eswa.2013.07.107. Macgregor, J.N. and Chu, Y. (2011). Human Performance on the Traveling Salesman and Related Problems. The Journal of Problem Solving, 3(2), 1–29. Pillac, V., Gendreau, M., Gu´eret, C., and Medaglia, A.L. (2013). A review of dynamic vehicle routing problems. European Journal of Operational Research, 225(1), 1–11. Psaraftis, H.N., Wen, M., and Kontovas, C.A. (2016). Dynamic Vehicle Routing Problems: Three Decades and Counting. Networks, 67(1), 3–31. doi:10.1002/net. Scott, S.D., Lesh, N., and Klau, G.W. (2002). Investigating Human-Computer Optimization. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 155–162. Toth, P. and Daniele, V. (2014). Vehicle routing: problems, methods, and applications. Society for Industrial and Applied Mathematics. T¨ ut¨ unc¨ u, G.Y., Carreto, C.A.C., and Baker, B.M. (2009). A visual interactive approach to classical and mixed vehicle routing problems with backhauls. Omega, 37(1), 138–154. doi:10.1016/j.omega.2006.11.001. Vickers, D., Butavicius, M., Lee, M., and Medvedev, A. (2001). Human performance on visually presented Traveling Salesman problems. Psychological Research, 65(1), 34–45.